Tasks that used to take a day to solve now take an hour. In the article, I’ll break down how the framework standard is structured, what failures it has been through, and how usage was optimized by a factor of two to three.
AI Systems Architect
From personal experience to enterprise adoption
Real-world test: a complete set of business requirements against a deadline
What I brought to this task
How I assembled the workspace canon and the project structure
From a working document to an approved package
Role, skill, and tool
Digitization: preserving the source’s meaning
- TAG41@@Source analysis and document preparation
DOCX assembly and result validation
Why the canon needed an owner
- TAG53@@How I used cross-agent review
- TAG57@@Where the canon broke down
- TAG61@@How instructions reached users
- TAG65@@How I organized support through agents
- TAG69@@Token usage analysis
- TAG73@@How I separated users’ memory from the canon owner’s memory
- TAG77@@Memory in users’ projects
- TAG81@@Memory of the canon owner’s agent
- TAG85@@The owner of reflection and institutional memory
- TAG89@@What the canon handover showed
- @@
What ultimately became the analyst’s workplace
Will AI replace humans?
What’s in the article:
- TAG108@@A case study on preparing business requirements
- TAG112@@Reducing token consumption
- TAG116@@Failures in the integration layer that led to changes in the rules TAG119
How to hand the system over to another owner so it doesn’t die
From personal practice to enterprise adoption
By that point, I had already been using agents in my own work: connecting different models, organizing cross-checking, and accumulating experience between sessions. I had a background implementing enterprise systems, so questions of training and support were nothing new to me.
Now I needed to assemble an analyst’s workplace that would enable a different pace of creating software documentation. Along with the tools, the way of working needed to be passed on: how to frame a task, check the result, get help, and save the solution found for others.
As implementation progressed, roles and skills were joined by a canon owner, release of updates, support via messengers, and several levels of memory. In the article, I’ll describe the problems that required these solutions and what had to be reconsidered along the way.
Field test: a complete set of business requirements under deadline pressure
The canon was field-tested before it became a finished product.The analyst needed to hand the client a set of business requirements for a new payment product: the regulatory sources had been updated, and the work had to proceed in parallel with other tasks. According to the analyst’s own estimate, completing that volume manually would have taken more than two weeks—and there was no time for that.
We quickly put together a minimally viable package: a business requirements template, roles, skills, and tools. For a week, I worked with an analyst copilot, observing where the agent stumbled and adjusting the canon to fit the task. Most of the time went not into generating text, but into establishing the conditions needed to trust it: which template to use to structure the document, where to obtain the data, how to identify conflicts and missing data, and what to do when the data was insufficient. Only then did the agent stop filling gaps with guesses.
On this task, the agent began making up data, and instructions alone were not enough: I lowered the temperature to 0.2 and top-p to 0.85. Other specialists working on other tasks were fine with a temperature of 0.6 at the same top-p. Generation parameters have to be tailored to the type of task, but the main safeguards still come from reliable sources, rules for handling missing data, and review of the results.
We started the work on one model, but the subscription allowance was not enough to cover the entire volume, so we continued with a model from another provider and returned to the first one after the allowance was updated. We devised the setup on the fly: the package had to be delivered on time, and the expensive combination was a solution adopted under pressure, not a recommendation for ongoing use.
Every document was reviewed by the analyst working with the agent: final acceptance remained his responsibility. I incorporated his comments into the instructions so that the same mistakes would not recur in subsequent documents.At the webinar, the analyst herself recalled the pace of the work:
“fourteen items… over eight working days.”
I don’t know whether that was fast or normal, but my colleague managed to fulfil her commitments to clients within a short deadline, which would have been impossible to do manually.
Later, by the time it was handed over to the client, the set had grown to 17 business requirements: each document had been proofread and published in Confluence.
The main outcome of this race was not just the set. The canon, which had met a real-world deadline, became a proven tool: subsequent specialists joined already established roles and rules, and their mistakes continued to improve the canon.
What I brought to this task
By that point, I already had experience in ERP development and implementation, integrating corporate systems, and managing development. I was accustomed to breaking down the work between departments: where the data came from, who made the decisions, to whom the result was handed over, and where rework arose.
I had applied classical development methodologies in practice: Agile, with decomposition into sprints and user stories; Waterfall, with phased gates; and task management in Jira. For me, these were not terms from a textbook—I knew how to break a large task into short iterations with clear acceptance criteria, how to conduct planning and retrospectives, and where the chosen approach breaks down and why.
Later, AI implementation projects and my own experience working with agents were added to this.
In my work, I used various applications and models, assigning agents to development, research, and checking one another’s results.Gradually, custom roles, skills, tools, and cross-session memory emerged. Thanks to this groundwork, I approached the task of automating analysts’ work with a set of solutions that I could apply and further develop.
At the same time, much of my personal practice depended on my involvement. When I set a task, I noticed when the agent had gone off track. I had to explain the mistake, decide when to bring in another model, and when to stop another round of checking. Some of the rules were documented, while others I added on the fly as needed.
Through personal experience, I realized that the quality of interaction with an agent begins with framing the task and providing feedback. For a specialist’s workspace, I wanted to go further—to ensure that a successful solution or corrected mistake did not remain confined to a single conversation, but instead became part of the agent’s experience, which it remembered and considered when generating conclusions.
This gave rise to the task of assembling a shared operational framework: defining agent roles, codifying approaches to completing tasks, integrating tools, and organizing the preservation of experience. The entry threshold for an analyst is not a prompt-engineering course, but a few setup steps; after that, the analyst focuses on the substance of the documentation, while the canon handles all the scaffolding and rules governing the agent’s work.
This is how I began building a corporate standard for an analyst’s workspace.
01 / STRUCTURE
How I assembled the workspace canon
The canon and project structure
The pilot’s working title was “Gold Standard”: a shared set of roles, skills, tools, and rules for agent operation across the entire department. This set was later simply called “the Canon.”It was supposed to work independently of the project and save the specialist from having to set up their workspace manually.
Setup consisted of a few simple steps and a couple of prompts to the agent: you give it a command and go drink coffee, and when you return, you find a fully set-up workstation: the environment is deployed, the dependencies and required libraries are installed, and the roles, skills, and tools are in place—you can start working on the task immediately. Along with the scaffolding, I introduced a unified standard for the structure of project directories and files so that specialists and agents could navigate the materials according to common rules.
Initially, I proposed a simplified three-layer project structure: source data, working drafts, and deliverable artifacts. It separated the materials for analysis from the work results and gave the agent clear storage locations.
When my analyst colleagues joined the work, their practices and feedback influenced my choice of structure. The agent rarely produces a suitable result on the first attempt: a document goes through several iterations, comments come in, the source data are refined, and the build is run again. The project structure had to support this process.
We settled on separate sections for each document type and working directories for individual documents within those sections. The working text in Markdown and its DOCX builds are stored side by side, in the md/ and docx/ subdirectories. I established an explicit contract: all of these materials are drafts by default. A successful file build does not, by itself, change its status.
For the approved release package, we set aside a separate directory. It contains document versions that have been reviewed and approved by the specialist.Thus, the current package can be exported and maintained as a whole, supporting a single source of truth without having to assemble the required versions from working directories each time.
From working document to accepted package
02-business-requirements/ └── БТ-ЦР-ДБОФЛ-002-Пополнение-СЦР-Клиента-ФЛ/ ├── md/ — working text └── docx/ — document buildsAll materials in the working directories are drafts.
Review and acceptance by a specialist → a separate directory for the accepted release package.
Role, skill, and tool
When designing the canon, I separated the professional role from the methods for performing specific tasks. I left the general instructions in the role: what the agent is responsible for, where its authority ends, how it handles uncertainty, and which skills it invokes for a given task.
I placed the detailed procedure for preparing the result in the skills. For example, creating a user interface description, preparing a Use Case, and writing an API description require different rules and checks. An agent acting as an analyst invokes the relevant method as the work progresses. This allows it to spend its context on the current task, loading detailed instructions when they are needed.
This separation helped account for differences between projects. The same document type may have different templates, section structures, and customer requirements.The overarching role is retained, while the skill is applied together with the instructions, template, and settings of the specific project.
I placed repeatable technical operations in tools—scripts that the agent runs according to the skill’s rules: parse the source file, validate XML, create a directory structure, or build a DOCX. A single tool could be used across multiple skills.
I codified this convention in the project-layout skill and in the tools for creating and checking the structure. The layout-builder role selects the appropriate layout, and the scaffold_layout.py script creates it according to the specified rules. The paths to sources, templates, working documents, and the release are stored in the project passport. Other roles and skills refer to it to understand where to read the materials and where to save the results.
The quality of the result depended on the consistency of the entire chain: the role defines responsibility and routing, the skill defines the methodology, the tool performs the operation, the template determines the form, and the artifact being produced is assembled based on approved sources and requirements.
TAG227@@Each element has its own role
TAG230@@What it defines
TAG243Role
TAG249Responsibility and limits of authority
TAG253Skill
TAG259Methodology for performing the task
TAG263Tool
TAG269Repeatable operation
TAG273Sources
TAG279@@Rationale for the content@@
Template@@
Document format@@
Result — an artifact, assembled based on the coordinated work of these elements.
Digitization: preserve the meaning of the source
Document digitization became the foundation of the analyst’s workspace. These are the agent’s eyes: full-fledged work begins with the ability to reliably read the source materials. Document generation relies on analysis of the source data, so it must be presented in machine-readable form while preserving the content and structure.
I paid particular attention to this process. It is not enough to extract a sequence of paragraphs from a PDF or DOCX. Tables, diagrams, and relationships between document elements must be conveyed correctly. A lost row, a misread cell merge, or an omitted process branch can affect the requirements that the agent will formulate at the next stage.
For tables, I made preserving their logical integrity a requirement. If one table continues across several pages, it must remain one table in the machine-readable representation. Rows, columns, cell merges, and the correspondence between data and headers must be preserved. A page boundary must not become a boundary of meaning.
This is not an abstract requirement—it grew out of practice. Colleagues were accustomed to working in DOCX, and the same format was needed to get documents approved by the client.When converting their templates into machine-readable form, I encountered a problem: they broke down at the most vulnerable point, where a complex table has a nonlinear hierarchy of subordination. Parent rows contain child rows, those rows have children of their own, and all of it is further divided into groups that must be mapped to data from XSD and XML—the regulator’s regulations. A flat representation of the table lost this hierarchy.
HTML proved to be the solution: representing the table using <table> preserved its human-readable rendering and conveyed the row hierarchy. This established a rule in the digitization skill: tables must use HTML only.
For diagrams, it is important to preserve participants, actions, relationships, conditions, and transition directions. Depending on the type of diagram, it can be represented using PlantUML, draw.io, or BPMN. Such a representation allows the agent to analyze the process and use it when preparing documentation. The accuracy of the transfer must nevertheless be checked against the original: a syntactically correct diagram does not confirm that all relationships have been conveyed correctly.
Therefore, I structured the digitization process as a separate skill with extraction and validation tools, rules for handling complex elements, and a mechanism for recording ambiguities. Its purpose is to prepare a reliable foundation on which the other roles and skills can operate.
Source analysis and artifact preparation
After digitization, substantive work with the sources begins. Here, I defined a common technical-analysis skill: it establishes the order of investigation, separates confirmed facts from assumptions, helps log questions, and resolves contradictions.For each substantive conclusion, I required the agent to maintain traceability to the source material so that the analyst could verify the basis on which the agent included it in the document.
The agent records gaps in the source data in a separate open-questions file—open-questions.md. Each question is assigned an identifier, a description of what needs to be clarified, and a current status. This file is kept in the project so that questions and answers received are not lost when moving between sessions.
If part of the document is prepared before an answer is received, the agent must leave an explicit note with a reference to the open question in the corresponding place. The specialist must be able to see which statements were formed with insufficient data, what requires clarification, and where the question description is located. Such a note does not allow the agent to make up the missing answer independently.
When continuing the work, the agent refers to the register, takes into account the answers already received, and keeps open questions until they are resolved. After clarification, the question status and the related sections of the document are updated. If the sources contradict each other, the agent records the discrepancy and shows its impact on the result.
On this common basis, I created skills for preparing specific artifacts.
- TAG331@@
For business requirements, the goals, scope of the change, and expected system behavior are important.
TAG335@@- TAG337@@
For a Use Case—the actors, preconditions, and main and alternative scenarios.
TAG341For an interface description—the screen elements, states, and responses to user actions.
TAG345For an API—parameters, data structures, responses, and error handling.
Each type of result requires its own methodology and set of checks.
At the same time, the documents describe a single system and must be consistent with one another. Therefore, I provided for checking the relationships between artifacts: whether entities are named consistently, whether interface behavior matches the scenario, and whether input and output data are aligned. Project templates and instructions defined the specific form of these descriptions.
DOCX assembly and result verification
Process description:
Prepare the role, skill, tools, and assigned template.
Assemble the working DOCX.
Check the content and formatting and address the comments.Submit the document to a specialist for acceptance.
Place the accepted document in the release package.Comments are sent back to the instruction, profile, or tool. The document is then assembled again.
To assemble documents in the approved format, I created a dedicated docx-layout skill. It linked the prepared Markdown with the template and profile of a specific project. The template specified the document’s structure and formatting, while the profile specified the parameters for applying them during assembly: styles, numbering, placement of tables, figures, and support elements.
I linked this skill to the validation and build scripts. For example, preflight.py checks the prerequisites and input files, while build_docx.py performs the build.
The agent receives through the skill a sequence of actions: what to prepare, which tool to run, what parameters to pass, and what to check in the result.
Special attention had to be given to understanding the template. It is not enough to choose a similar font and arrange the headings. The set of sections, table structure, merged cells, captions, table of contents, and metadata must be preserved. The document preparation methodology and the document assembly tool must understand these requirements in the same way.
I imposed two types of requirements on the result: correctness of content and correctness of presentation. After assembly, one needs to check that the tables and diagrams have been preserved, that headings and numbering work, and that the document conforms to the assigned template. Some checks are performed by scripts, while others require visual inspection and an expert assessment.
Comments on the result became the basis for correcting the instruction, profile, or tool itself, depending on the cause. The document was then assembled again: it was important to make sure that the fix was reproduced in the next generation.
Meanwhile, the document continues to live on after generation. It is read, approved, and handed over to development and testing. Therefore, the requirements for the result had to include the needs of the downstream participants. What helps an analyst assemble a document is not necessarily useful to a developer who needs to understand the requirements.
A separate rule was added for the case where the analyst’s Markdown was handed over to development: the file still contained revision comments and metadata for assembling the DOCX. The agent was unable to generate code based on it—the extra markup consumed the context, and figuring out what had gone wrong took up half the meeting.This gave rise to a task: add an artifact-cleanup skill so that a clean file with a fixed structure, containing PlantUML sources instead of images, would be passed to development. What is convenient for a human to read is not always convenient for a machine to process.
Separating roles, skills, tools, and checks made it possible to analyze errors in a targeted way. If the result was unsatisfactory, we had to understand exactly what had failed: the task definition, the instruction, the tool, or the check. Depending on the answer, we knew where to fix the root cause.
As the work progressed, the canon evolved: new skills appeared, instructions were refined, and tools were fixed. This raised the next question: how could improvements from users be incorporated while maintaining the consistency of the system as a whole? To this end, I designated a canon owner.
02 / DEVELOPMENT AND IMPLEMENTATION
Why the canon needed an owner
Setting up the agent harness is work at the intersection of prompt engineering and programming. It requires sound judgment: an understanding of how models behave, how tools are structured, and the consequences of changing instructions. Analysts are specialists in their own fields. Requiring them to have this competence as well would impose an additional burden.
Therefore, I gave the canon owner a separate repository with its own permissions, roles, skills, and tools. It is responsible for developing corporate standards for working with agents, accepts suggestions, reviews changes, and releases updates.
Approved fixes are incorporated into updates, whose verification I set up at the start of the work. Specialists no longer need to build new versions of the tools manually.And the owner needs to carry the change through to the workplace together with the updated instructions.
Improvement goes through the owner
How I used cross-agent review
While developing the canon, I used various applications and experimented with models from different providers, selecting those that handled specific tasks effectively in the fewest number of iterations. It is important here to distinguish the application in which the agent operates from the model connected to it.
The complete list of tools and modelsApplications and working environments:KiloCode (the Kilo plugin in VSCode), Cursor, Claude Desktop, Codex, Grok, and Hermes. I used Kilo and Cursor as IDEs, and ChatGPT, Grok, and Hermes as copilots.
Main models and model families:Claude, Qwen, DeepSeek, Grok; I also tested GLM and Kimi.
Other tests from my experience:models from Sokan and the Kilo bundle, in which Step 5 Preview performed well.
The combination in which a Western model checks the work of a Chinese model also performed well.This is an observation from my own practice, not a universal rule for choosing models.
I would submit the prepared solution to another agent for review, return the comments to the implementer, and then check the corrections.
In my experience, involving models from different providers produced a higher-quality result. Such review became a regular part of the work: another agent could notice a missed requirement, a contradiction in the instructions, or an error the implementer had not seen.
However, this approach has a downside. Agents can spend a long time discussing improvements, proposing new checks, and reworking an already functioning solution. Without a limit on review iterations, the review risks turning into an endless argument.
That is why it is important to set the boundaries of the review and the condition for its completion. Each subsequent change should address a specific shortcoming. If the agents continue to disagree, the decision remains mine.
Process diagram: preparing the solution
Process diagram: preparing the solution- TAG466
- TAG475@@A third agent checks it against the canonical standard. TAG478
The implementer prepares the solution
TAG470Another agent checks the result.
Specific comments are returned to the implementer.
TAG482The benefit of this interaction for me is the ability to look at the solution from different angles.The agreement of several models does not by itself prove that it works: the result still has to be checked against the requirements and tested on the task for which it was created.
Where the canon broke down
A standard that has not been tested against failures gives false confidence. Here are several cases after which the canon changed.
I broke the publishing process myself. A working Confluence publishing setup existed on the specialist’s machine, but I decided to introduce a single rule for everyone: retrieve the token not from a fixed location, but through a variable. I overengineered it—publishing stopped working, and the rollback did not help right away either: the outage lasted about three days. After that, I introduced a rule: before any standardization, run a regression check of typical agent scenarios, and deploy the change only after it passes.
A “configuration paradox” emerged in support. A specialist spent three hours “swearing” at the agent over a task that a developer can solve by hand in twenty minutes—and got the required result. On the surface, this looks like a loss, but this case formulated the canon’s main argument: quality is determined not only by the model, but also by the setup around it—the role, instruction, and process. Hence the principle: do not hand the specialist a ready-made agent; configure it together with the operator.
The agent “deceived” us during onboarding: when asked to confirm its role, it claimed someone else’s. The cause was noise: the role instructions carried traces of projects from the previous client. The instructions were cleaned up, and the canon established the principle: “the more precise the instruction, the fewer hallucinations.”
Another failure arose from my own rule: I insisted that the result be output in diff format.The tool had its own diff viewer, and the agent went into a loop: it stopped creating the file and displayed the entire document text on screen. The rule had to be removed from the instructions—a reminder that an instruction written without accounting for the tool’s behavior becomes a trap.
How the instructions reached users
Alongside developing the canon, I worked on its implementation: I analyzed specialists’ difficulties and updated the instructions based on those reviews. To this end, publishing in Confluence was configured, and an HTML navigator was created and distributed alongside the canon.
Personal tracks and sessions helped identify difficulties that I had not anticipated while developing the canon. Based on these reviews, I refined the instructions and training. At the same time, analysts did not need to learn how the agent harness worked on their own: analyzing its technical causes remained my responsibility.
The navigator became an onboarding tool and gave users a starting point: how to begin work, which role to choose, and how to perform the required operation. As questions arose and problems were solved, I enriched it with explanations. Once an answer was found, it became available to other specialists rather than remaining in a private correspondence.
TAG515@@Analyst navigator: from the task to the executor, the skills to connect, and the first message to the agent.
