Continuous Delivery

Agile Tokyo 2010

I've just got back from Tokyo, where I was honoured to be invited to give the keynote at Agile Tokyo 2010. Agile conferences have a short history in Japan - both Agile Tokyo and Agile Japan are only two years old - and there was a real buzz of excitement, with lots of smart people (280 delegates attended) and interesting conversations. So first of all I'd like to thank Yoshi Nagase and his wonderful team at Technologic Arts for hosting me, Yoko Yoshikawa for taking such good care of me during my time there, Gihyo for organizing the conference and for inviting me, and my fellow presenters for some great discussions.

[caption id="attachment_153" align="alignnone" width="3264" caption="Agile Tokyo 2010 by Yoko Yoshikawa"][/caption]

The most surprising discovery for me was that agile is considered new and somewhat subversive in Japan. Japanese IT companies have been doing waterfall on large projects for a long time now, and it is deeply entrenched. Indeed, everybody I spoke to confirmed that waterfall worked perfectly well for them as a software development methodology. This was staggering to me, since outside Japan large software projects run with waterfall routinely go over budget and end up with either cuts in scope or poor quality - several recent reports, such as this one, put IT project failure rates in the range of 60-70%. This appears not to be the case in Japan - it is perhaps the only country that could make waterfall work reliably and repeatably. As my host, Yoshi Nagase, commented, this is deeply ironic in a country that created Toyota and lean manufacturing.

However there is certainly an interest in agile now. Several large systems integrators have started running small agile pilot projects, and apparently the results have been successful. Why the sudden interest in agile if waterfall can deliver high quality software on time and on budget? The answer is easily extrapolated from this example of a project plan I saw at one of the companies I visited - the plan has been changed somewhat to protect the guilty, but not in any essential details - it is a pretty standard plan following the V-model.

For me, the most egregious problem with this plan is that the planning stage for phase 2 of the project comes before the testing stage for phase 1. So anything discovered during the testing and user acceptance stages cannot be fed in to phase 2 - it has to wait for phase 3. Effectively, that means that the feedback loop from discovering a problem in your requirements for phase 1 will have to wait until the end of phase 3 to be delivered to users - a cycle time of 1 year. This makes it extremely hard to respond to customer feedback. Even in the best case scenario, whereby you discover a requirement during the planning stage of a phase, you must wait 6 months for it to be delivered to users.

Japanese companies seem to be realizing that they simply cannot maintain a competitive advantage in the current economic climate with this kind of cycle time. They must get valuable software into the hands of users, and incorporate feedback from them, much faster. This of course is the value proposition of agile methods - the first principle of the agile manifesto, which also inspired the title of our book, is this: "our highest priority is to satisfy the customer through early and continuous delivery of valuable software".

The risk is that Japanese companies simply have very little experience doing agile delivery at all, let alone on large projects. Adopting agile in an effective way involves serious organizational change. Cross-functional teams including developers, testers, analysts, testers, operations staff, and project managers must be created. Developers must be trained in practices like test-driven development, continuous integration, refactoring, and incremental development. Testers will have to start collaborating more closely with analysts and developers. Managers will need to be trained in agile delivery methodologies - and they will need to make mistakes and gain experience. Teams will need new tools.

However if there is any country that can make such a large change in an organized and efficient manner, it is Japan. I'm really excited to see what agile methodologies do for Japanese IT, and also to see what the Japanese software delivery community does to agile - no doubt we will see many new ideas and evolutions. It's an exciting time to be in IT.