How I Work with AI Agents

One morning in October I noticed that the Starship prompt on this site’s Themes page was missing its icons. I told the agent I was working with that the font was probably to blame, and within the hour the fix was live. It came with a cut-down copy of a Nerd Font holding the 75 icons the file uses, a browser test that checks every one of them, and two corrections found by a reviewing model from a different company. One was a CSS rule elsewhere on the site that undid the fix in a hidden mode. The other was the font’s licence notice, which only linked to the Apache 2.0 licence where the licence asks for its full text. I didn’t read the diff at any point. On this site that is the intended way of working, and on my other projects it isn’t.

How much process I put around AI agents depends on how many people work on a project, what the business needs from it, and how complex its code really is. On this site I am the only one a mistake affects, so an agent can take a bug report all the way to production while I look at the result on the live site. On the Web Gallery of Art and on my work projects, a change is written up as an OpenSpec change before any code exists, and on the bigger ones I read the pull request myself. The rest of this post applies the same rule to smaller questions: where a lesson gets written down, who is allowed to write it, and what I still check.

Planning changes with OpenSpec

OpenSpec is a small tool that keeps a project’s intended behaviour as Markdown files in the repository. Before an agent works on a change, the change gets a proposal saying why it’s needed and what it alters, the updates it makes to the specifications, and a list of tasks. The agent works through one task at a time, and the reviewer judges the pull request against those files. On the Web Gallery of Art the instruction to the reviewer is explicit: “A preference that no specification supports is not a finding.” Review is most of why I use it. An agent can make unrelated changes elsewhere in the codebase while it’s at it, and when someone has to inspect the result, every such change is something they didn’t ask to review and someone will later have to maintain. With the change written up first, anything outside it stands out.

The second reason shows up months later. Every proposal starts with its reason, and when the change merges, archiving it folds the outcome into the project’s specifications. The Web Gallery of Art has 45 archived changes since August, so the reason the feedback button no longer jumps up when the cookie notice appears on a desktop screen is still in the repository, written by whoever made the change.

What this site uses instead

This site doesn’t use OpenSpec. Its interface is designed in Claude Design, and the design project keeps a changelog in which every entry names the design file it touches. The repository’s instructions for agents record the last entry that has been implemented, so the next piece of work is whatever comes after it, and the agent may read the design but never change it. That covers the planning. What keeps a change from spreading is a short set of rules in my own agent instructions, which apply to every project: don’t silently expand the task, keep each commit to one coherent change, and stop and ask when the work would need a decision outside the task. On a site with many parts that only I work on, those rules are enough.

Where a lesson gets written down

A lesson from that work can end up in five places, and each has its own rule for who may write to it.

Which one a lesson goes into is a judgement call that depends on what the lesson is. The line I watch most is the one between the company’s decisions and a project’s. If a project follows the company’s stack and needs a feature the ADRs say nothing about, the decision goes into the ADRs. If the project departs from an existing ADR for business reasons, the departure stays in the project, unless it turns out to be a better default than the one the ADR states, and then it goes back to the ADR. The tiers in the last column are deliberate. A memory is a note for later sessions, so agents write there freely, while an ADR becomes standard practice for the whole company, which is why a board accepts it first. So far I haven’t had to delete a memory an agent wrote, and the board hasn’t rejected a proposal. That record is still short, so it doesn’t prove much yet.

A reviewer from another model family

Every pull request on this site goes to Codex, OpenAI’s coding agent, while the code itself comes from Claude. A comment of @codex review starts a review, and Codex either leaves findings on the diff or reacts with a thumbs up. The agent fixes each valid finding and asks again until the thumbs up arrives. On the Web Gallery of Art the same principle is in the agent instructions for material or high-risk changes: implementation and final review come from different model families. I have two reasons for it. The plain one is usage, since splitting the work across two services keeps me under each one’s limits. The other is that models from the same family learn from much of the same data, so I expect them to share blind spots and to miss the same problem twice, while a model from another family looks at the change differently. I haven’t measured that. The Starship fix is the kind of case it’s meant for: neither of Codex’s findings was a failing test, and one of them wasn’t code at all.

The loop has a cost. The pull request that made this site tell IndexNow, the protocol Bing and some other search engines use to hear about changed pages, which pages each deploy changed went through about nine review rounds and ten findings, each fixed in its own commit. Better instructions for the reviewer would have cut that down. The Web Gallery of Art gives Codex a section of review guidelines, such as judging only against the specifications and skipping generated output and anything the linters already enforce. This site gives it none.

What I still do

What I check follows the same rule as everything else. Where a pull request merges as soon as Codex approves it, as on this site, I check the release, and on the bigger projects without that I read the pull request too. Checking the release only shows what’s visible. I found the missing icons by looking at the live page, but I would never have spotted the incomplete licence notice that way, or the problem in IndexNow’s run queue that the same review loop caught in a GitHub Actions job nobody watches.

Some work doesn’t go to an agent at all: the direction of a design, the writing on this blog, conversations with clients, estimates, the decision to start a project, and setting a new one up. I keep the last because I enjoy that part. So far I haven’t had to take anything back from an agent.