A Creative Agent Harness Has to Make Taste Checkable
A coding harness can gate on tests. A harness for slides or design cannot, so it has to turn taste into reference sets, rubrics, constrained components and a human pick. Gamma's own account is the worked example, with the founder's claims marked as claims.
Most of what I read about agent harnesses assumes the work is code. Code has a convenient property: something outside the model can say whether it is right. The tests pass or they fail, and the loop keeps running until the gate opens.
A slide deck has no such gate. Nothing fails when a slide is ugly. So I wanted to know what a harness looks like when the output is judged by taste, and the most detailed account I have found is Jon Noronha, cofounder of Gamma, walking through their presentation agent in an interview with Greg Isenberg.
Caveats first. This is a founder describing his own product. Anthropic sponsors the episode, and Isenberg says his design agency has worked with Gamma. My note was built from the transcript only. Nobody outside the company audited anything below.
What Noronha says Gamma does
By his account, Gamma started as individual model calls wired into preset workflows and was rebuilt around an agent loop this year. His example run goes like this. A cloud sandbox starts with the prompt and the user's files. The agent asks who the audience is and what the deck is for. It can pull context from Notion, Slack and Linear. Vision models and embedding search pick a template and customize it. The user reviews an outline and a style. Text, layouts, images and charts are generated in parallel. Then a vision-based pass inspects the rendered slides for misaligned text, cropped heads, poor contrast and padding errors, and the fix-and-inspect cycle repeats before anything is delivered.
He estimates about 20 models and 200 calls for that one deck. That is an illustrative estimate, not a measured average, and I would not treat it as a target. Nothing in the interview shows that a large model count is an advantage in itself.
Four substitutes for "tests pass"
Reading this as an engineer, I see four things standing in for the gate that code gets for free. The grouping is mine, not his.
| Substitute | Gamma's version, as Noronha describes it | What it cannot tell you |
|---|---|---|
| Reference set | Templates chosen by vision models and embedding search, plus a system that rotates through more styles | Whether this user likes the chosen style |
| Rubric | Checks for brand-guideline adherence and for asking clarifying questions; visual QA for alignment, cropping, contrast, padding | Whether a compliant slide is a good slide |
| Constrained components | The agent customizes a template instead of drawing a slide from nothing | Where fixed stages end and free choice begins |
| Human pick | Questions up front, then outline and style review before generation | How much friction a given user will accept |
The reference set cuts both ways. Noronha says Gamma's recognizable default look became a limitation, which is why they built a system to cycle through more creative styles. A reference anchors the output and also wears a rut into it.
The rubric row is where I think the interview is most useful, because he admits its limit. Brand-guideline adherence is an objective proxy, and a model can satisfy it while producing an unattractive slide. More importantly, he says the available models have not been good enough at judging slide quality, and that Gamma is researching its own specialized model for that job. So the visual QA stage, as described, catches defects. A cropped head is a defect. Whether the slide is any good is a different question, and by his own telling a generic vision model does not answer it yet.
The constrained-components row is my inference. Starting from a template narrows the ways a slide can go wrong, which is what makes defect checks possible at all. Noronha does not frame it that way, and he never says where the boundary sits between stages Gamma fixes and choices the agent makes.
The human pick is tested, not assumed
The detail I keep coming back to is the fast-mode experiment. Gamma tried removing the questions and the outline review. According to Noronha, that variant lost on satisfaction and on paid conversion. They still shipped a fast-mode variant for new mobile users.
The checkpoint is a product decision that got tested, and the answer differed by user and device. I read it as evidence for testing where the checkpoint goes. I do not read it as evidence that more friction is always better.
There is a second feedback process behind this one. Inside a run, the agent inspects the artifact. Across runs, Gamma compares prompts and models with A/B tests and looks at editing time, sharing, exporting, paid conversion and retention. He gives no sample sizes, effect sizes or design details, so what I can take is the practice, not the results. The metrics also need reading. Long editing time could mean engagement or repair work. Conversion measures perceived value, not correctness.
What I take from it
My own research pipeline has the easy version of this problem. The yt-research skill refuses to finish until every timestamp anchor in the note verifies. That is a tests-pass gate, and it is cheap because an anchor either matches the transcript or does not. It says nothing about whether the note is worth reading. That part is taste, and no anchor scan settles it.
If I were building a harness for subjective output, I would start from one recurring job and write down four things: the artifact the user actually looks at, the defects a check can detect, the reference the output is anchored to, and the points where only the user's preference decides. I would inspect the rendered thing, not the agent's message saying it finished. I would add a specialized model only after seeing the same failure repeat, which matches Noronha's advice to get usage on existing models first.
Key Takeaways
- A creative harness cannot gate on correctness, so it gates on proxies: references, rubrics, constrained parts and a person choosing.
- Each proxy checks something narrower than quality. Brand compliance and defect detection are checkable. "Good" is not, and Gamma's cofounder says generic models still judge it poorly.
- Human checkpoints are a product variable. Gamma's reported test favored keeping them, with a mobile exception.
- The 20 models and 200 calls are one founder's illustration of one run. They are not a recipe.
Sources
Next I want to write down, for my own note pipeline, which checks are defect checks and which decisions still depend on a person reading the result.