Early signal from a six-designer pilot. The autonomous-agent demo isn't the right shape yet.
A quarter ago I greenlit a pilot on my team: six designers, full access to AI coding tools, mandate to ship at least one production change in their own surface area without an engineer holding the keyboard. The goal wasn’t to replace engineers. The goal was to find out whether the bottleneck between a designer’s intent and a working interface had moved enough that the designer should now be holding the pen on small changes themselves.
A quarter in, I have early signal. Some of it is what I expected. Some of it isn’t.
The first thing that surprised me is which tool worked. We started the pilot with Devin, the autonomous coding agent that’s been in the design press for the last year. The promise was attractive: describe what you want, the agent goes off and does it, you check in periodically and review pull requests. The actual experience for our designers was closer to spoon-feeding an intern. The agent would take initiative on the wrong things, miss the context that lived in adjacent files, and produce work that was syntactically right and behaviorally wrong. Designers were spending more time correcting the agent than they would have spent writing the code themselves.
Three weeks in, I made the call to swap. We moved the pilot to Claude Code paired with Claude Chat. The frame was different. The designer sits in their editor, types or speaks an intent, and gets back code they can read, edit, and run. The tool is a collaborator, not an apprentice. The designer is in the driver’s seat the whole time. The cycle time dropped by roughly an order of magnitude in our small sample. More importantly, the designers’ confidence in the work they were shipping went up. They could read what they’d produced.
That’s the first lesson. The autonomous-agent framing is an attractive demo. It’s not yet the right shape for designers who care about craft. The pair-programming framing, where the designer holds the intent and the model holds the syntax, is.
The second surprise is what kind of work this actually unlocked. I expected the wins to be small UI changes: a button alignment, a copy tweak, a missing empty state. Those wins happened, but the more interesting wins came from a different direction.
A designer on the mobile team has been mining our user community posts for pain-point patterns: pulling the text from hundreds of forum posts, prompting a model to cluster them by friction type, then taking the clusters back into our research database and matching them against open tickets. From there she’s been sketching prototypes against the most common friction points and pressure-testing them with real customers in days rather than weeks. The keyboard-shortcut insight in our latest mobile release came directly out of this loop, and it surfaced a fact about our clinicians I’d been wrong about for two years: they’re more tech-savvy than the team had assumed, and the product had been designed against the wrong baseline.
She didn’t ship a feature. She shipped a workflow. The workflow now runs every two weeks, and it’s reshaping what the team treats as research-able.
The third thing I’ll call out, more cautiously, is what this means for the design technologist role. I was hiring against a specific shape last quarter: a designer who could take a Figma file and turn it into shippable production components in our system. That’s still a real role and we still need it. But I think the more interesting role, the one that’s emerging from this pilot, is closer to a researcher with infrastructure: someone who can stand up workflows like the one above at scale, who knows what to build for the team versus what to buy, and who treats AI tools as a research instrument rather than a production accelerator. I don’t have a clean name for this role yet. I’d guess by Q3 we will.
A few cautions, because I’ve been doing this long enough to distrust early signal.
The pilot is six designers. The bias is toward the curious and the technically inclined; my next cohort will be a deliberate mix. The wins are real and they are localized. I have not seen a designer ship a complex new feature end-to-end in this loop, and I’m not sure I’d want them to. The work that’s flowing through is bounded enough that the model’s failure modes are catchable.
The autonomous-agent press will keep coming. The substantive work, in my small sample, is happening one keystroke at a time, with a designer in the chair.