What the organization needed to know
A large product organization wanted to know whether AI tooling actually moves delivery, not in a demo, but in shipped work. So This Dot ran a structured pilot across two engineering squads, Team 1 and Team 2, and measured it the way you measure anything you intend to defend.
Every claim traced back to source: weekly data pulled from GitHub and JIRA, cross-referenced against DX reports, with tickets, pull requests, and commits labeled so each datapoint tied to real work. The target was a 3× year-over-year jump in throughput.
Why adoption was hard to measure
“AI makes us faster” is easy to say and hard to prove. Most teams adopt the tools, feel busier, and still cannot tell leadership whether anything changed. This organization wanted the opposite: a number it could stand behind in a board review.
Adoption was also uneven going in. AI showed up in roughly half of one squad’s pull requests and almost none of the other’s. Usage that lives in a few enthusiasts’ workflows never moves a whole team’s output.
Throughput sits downstream of an entire pipeline: development, code review, QA, deploy. Speed up one stage in isolation and the bottleneck just slides to the next. Push development faster without QA capacity to match, and work piles up in testing. That is the J-curve that makes a real improvement look, briefly, like a regression.
How the pilot was measured and enabled
Measurement came first. The team fit an ordinary-least-squares regression to weekly throughput and reported the honest forecast, including how much the number would move as more weeks of data came in. No single hero metric carrying the story.
Tools alone were never the plan. Engineers built eight custom AI skills and one custom agent and committed them into the codebase: reusable workflows like ticket-context extraction and environment setup, not one-off prompts that vanish when a chat closes.
Tracking where time actually went across the lifecycle made the QA J-curve visible and expected rather than alarming, so leadership could plan for the backlog instead of reacting to it.
What changed
Team 1 reached a 78% likelihood of tripling its throughput by year-end, an OLS forecast, and was already running at 1.2× its January baseline. AI reached 80% of pull requests, up from 50%. Lead time fell from 11.9 to 6.7 days, code-review wait from 3.0 to 1.8. By the final report, 71% of all work, 984 of 1,384 items, had been touched by AI.
Team 2 told a sharper adoption story: every pull request now used AI, up from zero. Lead time dropped from 13.2 to 4.3 days, QA cycle time from 4.2 to 1.6, and tickets per sprint climbed from 10 to 14. A research-heavy stretch held headline throughput flat, with velocity expected to follow.
Why it matters
The point was never the tools. Adoption you can measure is adoption you can manage, and when engineers build and commit their own skills, the gains compound across the team instead of evaporating when the pilot ends.