DevOps, a movement of people who care about developing and operating reliable, secure, high performance systems at scale, has always — intentionally — lacked a definition or manifesto. However (and this is fascinating in its own right) that doesn’t mean that we can’t measure the impact of DevOps, or how good people are at doing it. The proof of this, and also of the startling impact of the DevOps movement, is now available in the form of the 2014 State of DevOps report (which you can download for free).
The report, a collaboration between Nicole Forsgren Velasquez, Gene Kim, Puppet Labs, and yours truly, surveyed over 9,200 people worldwide, covering a wide range of industries and types of organization. Our goal for the report was ambitious. We set out to measure IT performance, business performance, the impact of particular practices (such as continuous integration, test automation, and version control), and also culture, and then to discover to what extent they influenced each other. How, you might ask, do you measure these things like culture and organizational performance? Following Douglas Hubbard’s definition of measurement as “A quantitatively expressed reduction of uncertainty based on one or more observations,” it turns out that you can measure anything if you put your mind to it. The report describes both our methodology and the way we measured these apparent intangibles.
Indeed we not only measured these things: we have sound, statistically significant data that shows that culture and DevOps practices impact both IT performance and organizational performance. In direct contradiction to a popular narrative of the last ten years, IT matters — indeed, the results show it is a competitive advantage — and DevOps culture and practices are instrumental in achieving both high IT performance and organizational performance. Readers of this blog will be especially interested to learn that:
- Trunk-based development, continuous integration, and automated testing measurably improve both IT performance and organizational performance.
- Having a high-trust culture has a strong impact on both IT performance and organizational performance.
- Using external change approval processes such as a change advisory board, as opposed to peer-based code review techniques, significantly impacts throughput while doing almost nothing to improve stability.
- Job satisfaction is the biggest predictor of organizational performance, and using DevOps practices are good predictors of job satisfaction.
I’m very excited by the report. We improved on last year’s method for measuring IT performance. We showed how you can measure culture and organizational performance. Most important, the analysis of our enormous data set demonstrates definitively that the strategies championed by the DevOps movement work, and that they provide a competitive advantage to your business.
Many thanks to my collaborators, the fabulous team at PuppetLabs, and to all of you who took the survey.
You can download the 2014 State of Devops Report for free.
Nhan Ngo, a QA engineer at Spotify, made four fabulous visualizations while reading Continuous Delivery. She has very kindly agreed to make them available under a Creative Commons license so feel free to share them, download them, and print them out (click to get a higher resolution version). Thank you Nhan!
By Gene Kim and Jez Humble
Last year, we both had the privilege of working with Puppet Labs to develop the 2012 DevOps Survey Of Practice. It was especially exciting for Gene, because we were able to benchmark the performance of over 4000 IT organizations, and to gain an understanding what behaviors result in their incredible performance. This continues research that he has been doing of high performing IT organizations that started for him in 1999.
In this blog post, Gene Kim and I will discuss the research hypotheses that we’re setting out to test in the 2013 DevOps Survey Of Practice, explain the mechanics of how these types of cross-population studies actually work (so you help this research effort or even start your own), then describe the key findings that came out of the 2012 study.
But first off, if you’re even remotely interested in DevOps, go take the 2013 Puppet Labs DevOps Survey here! The survey closes on January 15, 2014, so hurry! It only takes about ten minutes.
Continue reading The Science Behind the 2013 Puppet Labs DevOps Survey Of Practice
Thanks to all of you who came along to FlowCon! If you weren’t able to make it, you can watch the videos for free thanks to BMC and ThoughtWorks Studios. The slides are also available for downloading.
Let me first express my thanks to our producers: Geeta Schmidt and Niley Barros of Trifork and Rebecca Phillips of ThoughtWorks Studios. I also want to thank my fellow PC members Lane Halley, Elisabeth Hendrickson, Gene Kim and John Esser; our fabulous speakers; our generous sponsors; and everyone who came along.
Continue reading FlowCon 2013 Wrap-Up, With Some Hard Data on Gender Diversity in Tech Conferences.
I have been advised by people I trust that it’s not a good idea to talk about how you got serious female representation at your conference until after it’s over. However the shameful RubyConf “binders full of men” debacle and the Neanderthal level of discussion around it has wound me up enough to write this account somewhat prematurely. So here is how we achieved >40% female representation on our speaker roster at FlowCon.
Continue reading How To Create A More Diverse Tech Conference
One of the concepts that will feature in the new book I am working on is “risk management theatre”. This is the name I coined for the commonly-encountered control apparatus, imposed in a top-down way, which makes life painful for the innocent but can be circumvented by the guilty (the name comes by analogy with security theatre.) Risk management theatre is the outcome of optimizing processes for the case that somebody will do something stupid or bad, because (to quote Bjarte Bogsnes talking about management), “there might be someone who who cannot be trusted. The strategy seems to be preventative control on everybody instead of damage control on those few.”
Unfortunately risk management theatre is everywhere in large organizations, and reflects the continuing dominance of the Theory X management paradigm. The alternative to the top-down control approach is what I have called adaptive risk management, informed by human-centred management theories (for example the work of Ohno, Deming, Drucker, Denning and Dweck) and the study of how complex systems behave, particularly when they drift into failure. Adaptive risk management is based on systems thinking, transparency, experimentation, and fast feedback loops.
Continue reading Risk Management Theatre: On Show At An Organization Near You
At last year’s QCon San Francisco I got to curate a track on continuous delivery. One of the goals of the QCon conferences is “information Robin Hood” – finding ways to get out into public the secret sauce of high performing organizations. So I set out to find talks that would answer the questions I frequently get asked: can continuous integration, automated testing, and trunk-based development scale? How does continuous delivery affect the way we do product management? What’s the business case for continuous delivery? How do you grow a culture that enables it?
You’ll find the all these questions answered in the talks below, from the leaders who have been at the forefront of continuous delivery at Amazon, Facebook, Google and Etsy. They also discuss the tools they built and the and practices they use to enable continuous delivery. Finally, you get me talking about how you can adopt continuous delivery at your organization.
Thanks so much to Jesse Robbins, Frank Harris, Nell Thomas, John Penix and Chuck Rossi for these great talks, and to the folks behind QCon SF for an awesome conference.
Continue reading Videos from the Continuous Delivery track at QCon SF 2012
I spend quite a lot of time at conferences, and it consistently bothers me that they are so often focused on one particular function: development, testing, UX, systems administration. The point of continuous delivery is to accelerate the rate at which we can learn from each other – and from our customers. That requires everyone involved in the delivery process (including users, product owners and entrepreneurs) to collaborate throughout. So why isn’t there a conference which focuses on flow – the emergent property of great teams?
So I got together with a bunch of like-minded folks – Elisabeth Hendrickson, Gene Kim, John Esser and Lane Halley – and now there is a conference about creating flow: FlowCon. It’s on Friday November 1 in San Francisco, and it’s produced by ThoughtWorks and Trifork (creators of the GOTO conferences).
Continue reading Announcing FlowCon
I am not going to do a ton of book reviews on this blog (I have one more planned for next month). I’ll only bother posting reviews of books that I believe are both excellent and relevant to Continuous Delivery. This book easily satisfies both criteria. Full disclosure: Gene gave me a draft of this book for free for reviewing purposes.
You’ve probably heard of Gene Kim, Kevin Behr and George Spafford before. They are the three amigos responsible for The Visible Ops Handbook, which can be found in the book pile of every good IT operator. Their new book, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win, follows the format of Eliyahu Goldratt’s classic, The Goal.
Told from the perspective of newly-minted VP of IT Operations Bill Palmer, it describes the turnaround of failing auto parts company Parts Unlimited. This is to be achieved through the delivery of the eponymous Phoenix Project, a classic “too big to fail” software project designed to build a system which will revive the fortunes of the company.
Continue reading Book Review: The Phoenix Project
In his new book, Antifragile, Nassim Taleb discusses the behaviour of complex systems and distinguishes three kinds: those that are fragile, those that are robust or resilient, and those that are antifragile. These types of systems differ in how they respond to volatility: “The fragile wants tranquility, the antifragile grows from disorder, and the robust doesn’t care too much.” (p20) Taleb argues that we want to create systems that are antifragile – that are designed to take advantage of volatility. I think this concept is incredibly powerful when applied to systems and organizational architecture.
Continue reading On Antifragility in Systems and Organizational Architecture