2018 marks the fifth year of our research program investigating software delivery performance, how to measure it, what impacts it, and why it matters. You’d think in our fifth year we’d be struggling to find new ground to cover—but this year’s report is the most comprehensive yet, and I’m truly excited by our results. With over 1,800 responses from companies of all types and sizes, we’ve extended our model of software delivery performance to include availability; studied how cloud infrastructure and its implementation impacts performance; examined how leadership and learning affect culture and delivery performance; expanded our investigation of technical practices to include observability, continuous testing, and database practices—and those are just a few of the highlights! You can read the full report at https://cloudplatformonline.com/2018-state-of-devops.html
Let’s start with performance. As in previous years, we’ve captured measures of both delivery speed and stability, and found that teams aren’t trading off one against the other. Instead we find a high performing group that leads the pack in terms of both speed and stability, a low performing group which delivers much more slowly and has significantly lower stability, and a medium group that is in between on all measures. That result continues to hold true this year—and indeed, our high performing group is larger than in previous years, showing the industry continues to evolve. As with previous years, we find that high performance is possible for all kinds of organizations of all sizes. This year, we extended our model of performance to include availability, and found that high performers were over 3.5 times more likely to achieve good availability outcomes too. We call this combined measure of throughput, stability, and availability software delivery and operational performance (SDO performance).
The use of cloud infrastructure was a major focus of our investigation this year. We looked not just at the level of usage, but at how teams were using their cloud infrastructure. A personal bugbear of mine is organizations spending large quantities of time and money implementing cloud technologies but completely failing to leverage the capabilities they provide: organizations lifting and shifting existing systems, and then putting a ticketing system in front of the cloud API so that from the perspective of both users and development teams there is no visible improvement in their daily work.
NIST helpfully defines five essential characteristics of cloud: On demand self-service, broad network access, resource pooling, rapid elasticity, and measured service (see NIST SP 800-145, The NIST Definition of Cloud Computing). We found that only 22% of respondents had implementations that met these characteristics—but those that had met all five essential characteristics were an astonishing 23 times more likely to be in our group of elite performers (a subset of our high performers who are achieving the highest levels of both speed and stability) than in the low performing group. We also found that when people implemented cloud native architectures they were 1.8 times more likely to be in our elite performing group. With so many organizations moving to cloud infrastructure and deploying popular and powerful tools such as Kubernetes, our findings highlight the importance of going beyond targets such as “moving x% of services to the cloud” to leveraging the unique capabilities provided by cloud platforms to improve outcomes for software teams and end users.
As Kelsey Hightower, Developer Advocate for the Google Cloud Platform, points out, “The tools at our disposal continue to improve and not only bake in the best practices; they are helping redefine them. Organizations that demonstrate the ability to get out of their own way and leverage these tools are clearly moving the ball forward.”
As in previous years, we validated that continuous delivery improves both SDO performance and organizational culture while decreasing burnout and deployment pain. This year we also looked at the impact of shifting left on security, implementing monitoring and observability practices, integrating database changes into the deployment pipeline, and having testers work with developers to test continuously throughout the software development process. We find that all of these practices positively impact the ability of teams to effectively implement continuous delivery. Again, there’s a lot more detail in the report.
Finally, we’ve extended our investigation of leadership and culture. This year, we find that when leaders give their teams autonomy, teams trust leaders more and are more likely to speak up, sharing ideas to improve work and surfacing system failures or risks. Autonomy in this context means, amongst other things, that effective leaders establish and communicate goals but let the team decide how to work towards them, and that they allow teams to change rules that are obstacles to achieving their goals. When teams feel able to trust their leaders and speak up, this also drives a more trusting and mission-oriented culture. Leaders can also impact culture through creating a climate for learning—by investing in the development of their people—and by ensuring teams regularly hold, and learn from, retrospectives.
Believe it or not, I’ve only scratched the surface of the report in this article. You can find much more on the topics I summarize, as well as information on outsourcing, open source, quality outcomes, and more in the full report at https://cloudplatformonline.com/2018-state-of-devops.html, and at the webinar I’ll be co-presenting with Google Cloud’s VP Engineering, Melody Meckfessel, which you can register for at https://cloudonair.withgoogle.com/events/americas?expand=talk:devops
Nicole, Gene and I would like to thank particularly our premier sponsor Google Cloud, as well as our gold sponsors that made the report possible this year: Amazon Web Services, CA Technologies, CloudBees, Datical, Deloitte, Electric Cloud, GitLab, IT Revolution, Microsoft, PagerDuty, Pivotal, Redgate, Sumo Logic, Tricentis, and XebiaLabs. Finally, thanks so much to all of you who took the survey: I hope you find this year’s report rewarding, and that you get as much out of reading it as we got from creating it.
Posted Wednesday 29 August 2018 by Jez Humble. Filed under news.
Our new book, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations is out and available at Amazon and other retailers. Accelerate describes the results of our four year research program into the science of DevOps and Lean software delivery practices. Subscribe to our low-volume mailing list to get a substantial excerpt along with our report detailing how to calculate the ROI of implementing DevOps principles and practices in your organization.