Google released the 2024 Accelerate State of DevOps (DORA) report. It has some interesting and counterintuitive findings. This year, it rethinks throughput and stability aspects. Time to restore is now a throughput measure and rework rate is added to stability. AI-enabled software delivery impacts are mixed.
The report is based on surveying tens of thousands of technology professionals. It uses cluster analysis to form low-medium-high-elite groups, and the metrics within each group can change. Over the years, increased throughput correlated with increased stability. But this year, there's an anomaly in the medium performance group where improved stability doesn't lead to increased throughput.
The change failure rate metric has been an outlier. This year, they explored it as a proxy for rework rate. Time to recover from failure is now defined as a throughput measure. Regarding AI, 75.9% of respondents rely on it, with code writing being the top task. But there are conflicting findings - AI adoption leads to decreased valuable work time, throughput, and stability.
The report offers hypotheses. The author suggests that the industry may have identified the wrong constraint. Most enterprise-grade AI is coding assistants, but code generation may not be the bottleneck. The RedMonk team thinks AI is a seismic shift but industry metrics have focused on individuals rather than holistic systems. Maybe it's time to shift AI right or consider how the rest of the SDLC can adapt.
(Note: The original survey number was corrected from 39,000 to 3,000 for the 2023 report.)