Sprint 6 and progress-to-date review

Sprint 6 has been generally fruitful:

• Silk Group file sharing is ready for use;
• Silk Group file sharing online user documentation has been prepared;
• Shuffl WebDAV storage module has been implemented;
• Shuffl is now able to run from and save workspace data to an ADMIRAL system;
• HTTP authentication works well with AJAX calls;
• JISC 6-month report has been submitted.

The full review is available at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_SprintReview_6.

This review follows a different pattern to the usual end-of-sprint review.  Having a data sharing platform ready for use by researchers was originally estimated to take about 1.5 months, but the work has actually taken about 3 months. The question driving this review is: "Why so long?".

In summary, we don't feel the execution has been greatly lacking (though other experts may disagree and we welcome discussion).  The mismatch between plan and execution seems to be due to oversights in the planning. Some lessons we offer are:

• We should not assume that using tried-and-tested software means that there is no development risk.
• We need to create a system that provides the user with a smooth, secure interface. Focusing on the user experience can easily push software components out of their design "comfort zone".
• Plan for an extended period of "scoping experiments" with any external technology, to be conducted alongside user requirements gathering.
• Plan to script as much as possible of a system build and configuration: this takes longer to begin with, but saves time in the long term as it facilitates repeating a build with minor variations.
• Plan to create automated tests for as much as possible. It may seem stange to be creating "unit tests" for accessing a file system, but they have been invaluable for our work in testing access control mechanisms.
• Plan to build separate systems for testing and live deployment. Automated tests should run with either.
• When working with a number of users, plan a separate system build and deployment phase for each.
• We estimate that ongoing user interactions require about half a day per week.
• Estimated project management overheads (for a small project on the scale of ADMIRAL): 25% for the project manager, and 10% for other team members.
• We estimate that about 20% of our development time is spent on creating technical documentation through wikis and blogs, but this is generally absorbed in the development effort estimates.

Projects aimed at changing user behaviour, such as ADMIRAL, are critically dependent on a solid technology foundation. Despite not being focused on technology development, we cannot dismiss the engineering effort that is needed to create a smooth, dependable experience for the users.

We estimate that access to in-depth expertise about the software we were using may have saved 25%-50% in some of the areas of time overrun.