AI is not a magic wand. How to work with a specification agent and not fall into a bug loop / Habr

The feature goes through a cycle of three commands:

1. /opsx:propose add-partial-refund: the agent creates a change folder with proposal, specification deltas, design and tasks. On this step, you review the plan until no line of code has been written.

2. /opsx:apply: the agent implements tasks from tasks.md and marks completed ones.

3. /opsx:archive: the change goes into the archive, and deltas are merged into the main specifications in specs/.

Requirements and plan live in the repository, read by both people and the agent, and you review the plan before code. After archiving specs/, it again describes the current system behavior, and the next task starts with an up-to-date picture. If the task is unclear, variants can be discussed with the agent through /opsx:explore.

Project context for the agent

The agent comes to the project each time as a novice on the first day. A person absorbs the context over six months during meetings and reviews, while the agent knows only what is written. Specifications describe the system behavior, but this is insufficient, so the live project must be fully described for the agent:

• system context: what the product is, who it's for, and where its boundaries are;

• business processes: how orders, payments, and returns work, who participates at each step;

• pipelines: CI/CD, background jobs, and integrations, what and in what order is executed;

• business logic and invariants: rules that cannot be violated under any refactoring or optimization;

• glossary: domain terms and their names in code.

Invariants are most often underestimated. You ask the agent to speed up refund processing, and it removes 'unnecessary' repeated reads from the database. Tests are green, but when parallel requests are made, the client receives more money than paid. The agent did exactly what was asked: that is, the reading protects the rule, no one told him.

A glossary is needed for the same reason. The analyst writes "cancel" in the task, but the code has Refund, Reversal, and Cancellation. Without a dictionary, the agent will choose randomly. When terms are described, the new task falls on the correct entities.

It's easiest to keep this in the repository next to the code, for example like this:

openspec/ config.yaml # stack, agreements, links to docs/context specs/ # how the system works now, by domains changes/ # changes: proposal, design, tasks, spec deltas docs/ context/ overview.md # what the system is, who it's for, boundaries business-processes.md # order placement, payment, refund step by step invariants.md # what cannot be violated under any optimization glossary.md # domain terms and their names in code pipelines.md # CI/CD, jobs, integrations decisions/ # ADR: why it was done this way, not otherwise AGENTS.md / CLAUDE.md # rules for the agent and links to everything above

And here's a fragment from invariants.md:

# Domain Invariants for "Orders" - The sum of all refunds for an order ≤ the paid amount. The check lives in the domain under order lock, not only in API. - The order status changes only through OrderStateMachine. Direct UPDATE status is prohibited. - Money is integers in kopecks. No floats. - A repeated request with the same Idempotency-Key returns the first result.

It's convenient to put a short summary (stack, agreements, where the context is located) in the context field of the openspec/config.yaml file. OpenSpec submits it to the agent when creating each artifact.

Context must be maintained. If the process has changed, update the file; otherwise, the agent will work with an outdated picture. Context updates should be included in the Definition of Done on par with tests.

Skills and wrappers

Specification and context tell the agent, what  to do. Skills describe, how  you do it: how migration is written, how PRs are reviewed. These are reusable instructions for the agent, and you have to write them tailored to your stack and style.

The first review filter can also be given to the agent if you provide it with a checklist against the specification. On the request "look, is everything okay," it usually responds that everything is okay. Example skill:

--- name: review-against-spec description: Review PR for compliance with changes in OpenSpec --- 1. Find the change in openspec/changes/: proposal, design, tasks, and spec deltas. 2. For each scenario from the deltas, check that the code implements it and that there is a test for it. 3. Check that none of the invariants from docs/context/invariants.md are weakened. 4. Find new dependencies and utility duplicates from /shared. 5. Check that tests fail if the business rule is broken. 6. Provide a list of discrepancies with the file and line. Don't fix anything yourself.

Such a skill removes the routine from the reviewer, and the person looks at the essence: whether the code solves the task and whether it breaks at the interfaces. The final word on the review remains with the person.

Skills and rules accompany just like code:

1. The agent made a mistake, or the bug reached production.

2. We examine the cause: was there a lack of context, rules, or checks in the skill?

3. We add the missing parts and run this change through the review as a regular PR.

4. The next task proceeds under the updated rules.

If you fix only the code, the agent will repeat the same error in the next feature.

People who review in the context of AI

Specifications and human skills do not replace. To distinguish 'almost correct' from 'correct', you need competencies and experience: domain knowledge and understanding of typical errors. A junior sees code that compiles and passes tests. An experienced developer sees in the same code a race condition or a broken invariant.

Monitor how employees use AI

You should look at one thing: hasn't the person become a go-between between the ticket and the agent. If an employee forwards the task to the agent and sends the result for review without reading it, there's no point in having them.

All the checking moves to the lead, and they become the bottleneck.

A mentor is needed

During implementation, a person who has already gone through this process or received training is needed: a lead or a mentor with SDD experience. His tasks:

• set up rules, context, and skills for the project;

• train the team to write specifications and review AI code;

• observe how people use the tool and make corrections;

• monitor metrics and promptly notice when quality has declined.

In the previous article, I compared this role to a Scrum Master: it's temporary, but without it, the methodology dissolves into 'we already have Cursor'.

Anti-pattern 'One person — four roles'

Some companies, in pursuit of AI, assign four roles to one employee: business analyst, systems analyst, developer, and tester. The logic is clear: 'AI writes the code, the person is left to assign tasks'. Often, this person is a business analyst who doesn't know the basics of the language and hasn't undergone training in SDD work.

Each role checks the work of the others. If you merge them into one person without training, checks disappear, and errors from all four roles accumulate in one head:

Role

What is checked

What is lost if the role is formal

Business Analyst

What the business needs and why

The specification describes the wrong problem

Systems Analyst

How a feature fits into the system, contracts, and integrations

The contract diverges from reality, breaking adjacent systems

Developer

How to implement and maintain

Code that no one understands and can't fix

Tester

Does it actually work

Tests confirm the code, not the requirements

Making a developer into a 'human orchestra' is probably better: they can at least check the code. However, if they lack soft skills and haven't studied analysis, meaning they can't conduct interviews with the client and describe requirements, the specification will be weak. A weak AI specification is implemented just as quickly as a good one.

If roles are still combined, it's only with training on missing skills and external validation: code review by another person and separate testing of critical scenarios. Without this, the time saved on staff ends up being spent fixing bugs in production.

Degradation loop

Without a process around AI, the team falls into a loop. A task is given to an agent with one prompt, the code looks correct, the review is done superficially, and a bug appears in production. It is fixed the same way, with a request "rewrite the module" without analyzing the cause. The new code is again "almost correct," and the bugs move to the interfaces.



You can exit the loop in one way: after each bug, analyze what was missing and add it.

Consequences for business:

• Costs increase: tokens, reviews, and fire-fighting consume the time of the most experienced people;

• Feature quality decreases, and with it, trust in AI within the team;

• Implementation goals are not achieved, and after a year, everything starts from scratch, but with a more complicated codebase.

Checklist for a manager and team lead

Manager

• Do one of your real tasks completely with AI before planning any cuts.

• Assign a responsible person for implementation: a lead with SDD experience or someone who has undergone training.

• Do not combine roles without training on missing skills and without external validation.

• Allocate time for context, rules, and skills preparation: they don't come along with the subscription. Measure results with numbers before and after: cycle time, bugs in production, percentage of rework.

Team lead

• Describe the project context: processes, pipelines, invariants, glossary.

• Introduce OpenSpec or another SDD tool and start any change from the specification.

• Write skills tailored to your stack: specification reviews, migrations, tests.

• Include updating the context and skills in the Definition of Done.

• Regularly monitor how the team uses AI: PR self-review, code understanding, behavior tests.

• After each bug in production, ask: what was missing from the agent and what was missing from the review?

Briefly AI accelerates development and analytics where there is a specification, described context, and people who know how to check the results. Without this, it accelerates the appearance of bugs, and the company pays more than before implementation.

AI is a tool, and you need to learn to use it yourself and teach the team.

And how is it with you: do you describe the project context for agents, and who is responsible for how the team uses AI?