Thursday, 31 March 2011

ADMIRAL: the final push

Our most recent development review is at

Over the past few months of the ADMIRAL project, we've been transitioning from a substantially feature development oriented mode to stabilization and maintenance, in order to allow us to ramp up user engagement activities.  As part of this, the project management style has evolved to shorter, less elaborately planned sprints.  Summaries can be seen on the project plan page. These essentially consist of a combined review and planning session conducted at approximately 1-week intervals, driven by requirements recorded in the project issues list.

We intend to complete all items recorded as high and medium priority in the issues list, except where blocked by issues noted that we are unable to resolve with available resources.  This represents a kind of feature freeze in the ADMIRAL data store function, with enhancements focused on stabilization and manageability of the system.  With ADMIRAL user features stabilized, and a stable deployment of Databank, we will update all of the deployed systems, and encourage researchers to deposit real research data sets from ADMIRAL to Databank for preservation and publication.  Getting real research data deposited and published with DataCite DOIs represents the main project goal that we now want to see realized before the end.

Specifically, over the next three months, we aim to:
  • Test and integrate the remaining Databank features (see issue list items tagged "Databank")
  • Issue 9: displaying content tree of dataset prior to confirmation of submission
  • Issue 42: a web interface for user administration
  • Issue 45: basic Debian packaging for ADMIRAL (which we expect to allow us to deploy easily on more recent versions of Ubuntu)
  • if and when time permits, picking up and progressing some of the lower priority technical debt issues
while also dealing with any other critical issues that may arise.
    In parallel, we will engage with the various research groups to learn more about how and to what extent they are using ADMIRAL, and encourage them to start submitting datasets to the Databank service.

    Friday, 4 March 2011

    ADMIRAL Sprint 17

    We have recently completed our review of Sprint 17.

    This review was somewhat overdue, as we've been very busy with follow-ups to ADMIRAL deployment with two additional research groups (making a total of three deployments now).  Main acheivements over the past month have been:

    • Two new ADMIRAL deployments; Silk group storage upgraded to use departmental iSCSI facility
    • Resolved awkward technical Apache+LDAP issue
    • Started construction of stand-alone demonstration environment
    • Deployment and management improvements
    • Bug-fixing and usability improvements
    • Documentation of technical problem areas
    • Benefits case study write-up
    • ADMIRAL packaging adopted for 1st protoype of Wf4Ever project

    One of the recent lessons is that the general level of requirement for data storage has increased dramatically since our initial user surveys.  Where most groups were originally content with 200-400Gb storage, they are now asking for Terabytes (due to increased use of high definition video for observations).  So the ability to connect to a departmental iSCSI network storage facility has turned out to be a crucial development for us, especially for new research proposals that are required to include data management and sharing plans.

    Resolving the Apache+LDAP problems has been a most satisfying advance for us; the awkwardness of the Apache web server configuration had been a long-standing difficulty for us, and we will now be able to simplify the overall ADMIRAL configuration and monitoring.

    Looking forward, as we enter the final stages of this project, we intend to change our approach to sprint planning.  Instead of preparing a separate plan, we intend to be more reactive, responding to issues in the project issue list (, as these most closely reflect user feedback and other issues that need to be addressed.  We will still undertake periodic reviews to help us ensure that efforts are sensibly focused.  In addition to dealing with the issue list, two other developments are planned:
    • Web interface for user management
    • Investigation of Debian installation package for ADMIRAL deployment
    The rationale for choosing these is that they appear to be key features for facilitating continued management and new deployment of ADMIRAL systems within the department.