Introduction

In the OSS project “Shortcut Expert” that I’m developing, we have adopted AI-driven development. I specify the behavior I want to achieve and have generally left the internal design and implementation to AI agents.

Initially, things went smoothly, but as I added features, the amount of rework needed to achieve the expected behavior increased. What previously took only one or two rounds of fixes has recently sometimes required five or six rounds, and I began to become concerned about increased token consumption and slower development.

When I examined the code to investigate the cause, I found problems with the design.

In Shortcut Expert, the logic for detecting keyboard and mouse operations had become complicated, and the division of responsibilities between C# and TypeScript had become unclear. Adding a single feature now requires changing many places.

I began to think that these design issues might also be contributing to the increase in rework, so I am currently refactoring.

From this experience, I felt that I need to carry out some architectural design and code review myself.

At the same time, I wondered whether the degradation of code maintainability could be detected at an earlier stage. So this time, I am focusing on software metrics and, after researching the definition of maintainability and studies on AI development, considering which indicators should be measured.

Target audience

• Those who leave implementation to AI agents
• Those who want to objectively evaluate code maintainability
• Those interested in software metrics

What is maintainability, to begin with?

In ISO/IEC 25010:2023, the international standard for software quality, maintainability is defined as the ability of software to be modified effectively and efficiently.

Maintainability consists of the following five aspects:

Aspect Meaning Modularity Changes to one part are unlikely to affect other parts Reusability Code can be used for other purposes Analysability Problems, change locations, and the scope of impact are easy to identify Modifiability The software can be changed efficiently without introducing defects Testability Correct behavior is easy to verify

In simple terms, it is whether you can continue adding future features and making fixes without breaking existing functionality and with little effort.

What maintainability problems arise in the era of AI implementation?

I examined maintainability issues in AI development, focusing on research published between 2025 and 2026.

1. Development speed may increase, but the code may become more complex

Speed at the Cost of Quality (MSR 2026) compares an OSS project that introduced Cursor with similar projects that did not.

As a result, development speed temporarily improved, while code complexity and the number of static analysis warnings continued to increase. The analysis also found that the deterioration of these quality indicators was associated with a subsequent decline in development speed.

In other words, development may be faster in the short term, but the maintenance burden may increase in the long term.

The findings were also quite similar to the problems I had experienced as an individual developer.

2. AI may not be able to maintain an appropriate design

Understanding the Code and Architectural Smells Generated by LLMs and Autonomous Agents (ASE 2026) found that AI-generated code also exhibited design problems such as code bloat and strong coupling.

Furthermore, CodeTaste (ICML 2026) reports that, while AI can implement specific refactoring methods with reasonable success when given detailed instructions, it is difficult for AI to determine an appropriate direction for improvement on its own.

Of course, choosing a design different from a human's is not necessarily wrong.

However, being able to implement code correctly and being able to autonomously select an appropriate design can be considered two different things.

3. AI-written code may be difficult to modify later

Is Agent Code Less Maintainable Than Human Code? (EMNLP 2026) conducts an experiment in which AI is asked to make additional changes to human-written and AI-written code, respectively.

As a result, there were cases in which the task success rate dropped by as much as 13.1% when modifying AI-written code.

Even more interestingly, conventional maintainability metrics could not adequately explain this difference.

Conventional metrics alone may not be sufficient to fully evaluate how easy the code is for AI to modify.

4. AI-implemented code is not necessarily sufficiently tested

Test Coverage Analysis of Agentic Pull Requests (ICSME 2026) analyzes 4,882 PRs generated by AI agents.

Among the PRs that modified the code being tested, 49.6% also modified the tests. It also reported cases in which existing tests did not sufficiently exercise the modified areas.

Not adding tests does not necessarily mean that testing is insufficient. However, it seems important not only to have tests, but also to ensure that the changed areas have been sufficiently verified.

5. However, AI does not necessarily worsen maintainability

Echoes of AI (Empirical Software Engineering, 2026) found no statistically significant differences in fixing time or code quality when code created with AI assistance and code created by humans were subsequently modified by another human.

This study focuses on AI-assisted development, and its conditions differ from autonomous development by AI agents.

At present, it cannot be said that code written by AI necessarily has lower maintainability.

Implications of the Research Findings

Summarizing the findings of this study, it seems that AI development requires attention to issues such as the following.

Problem Related maintainability characteristic Code complexity Analysability・Modifiability Accumulation of design issues Modularity・Modifiability Difficulty of making future changes Modifiability Insufficient testing Verification issues related to testability

These issues do not occur only with AI, but they also need to be considered in AI-led development.

In particular, in my personal development work, I believe that design complexity is related to the burden of adding features, soI would like to place emphasis on modularity and modifiability.

How should we measure maintainability?

What Are Software Metrics?

Software metrics are indicators that quantify properties such as the size, complexity, and dependencies of code.

Representative examples include cyclomatic complexity and cognitive complexity.

However, good metric values do not necessarily mean that maintainability is high. Conversely, poor values do not necessarily mean that refactoring is always required.

I regard metrics as a basis for decision-making when identifying code that may have maintainability problems and considering refactoring.

Representative Metrics for Measuring Maintainability

When mapped to the five ISO categories, the following metrics are candidates:

Maintainability Metric Issue to detect Modularity Coupling・cyclic dependencies・cost of propagating dependencies Structures in which the impact of changes tends to spread Reusability Code duplication rate Increase in duplicate implementations Analysability Cognitive complexity・LOC Complex processing or code bloat Modifiability Code Churn・number of files changed together Concentration of changes・expansion of the scope of changes Testability Coverage・Mutation Score Insufficient verification of changes・ability of tests to detect defects

However, these metrics do not necessarily directly measure each aspect. For example, the code duplication rate does not itself represent reusability, and neither coverage nor Mutation Score directly measures the ease of testing.

Ultimately, they are metrics for identifying issues related to maintainability.

Metrics to Prioritize in Solo Development

Based on the research I looked into this time and my own development experience, I plan to prioritize collecting the following metrics.

Priority Metric Purpose 1 Dependencies / Coupling Identify increasing design complexity 2 Change Frequency × Complexity Identify complex code that is changed frequently 3 Coverage of Changed Code / Mutation Score Verify whether changes have been adequately tested 4 Cognitive Complexity / LOC Identify extremely complex code

I am particularly interested in dependencies and coupling.

In Shortcut Expert, the problem was that not only individual processes, but also the division of responsibilities between C# and TypeScript had become unclear, and the number of change locations was increasing. Therefore, I would like to examine dependencies between modules as well as function-level complexity.

The combination of change frequency and complexity can also be used for “hotspot analysis” to find refactoring candidates.

For example, by identifying files that are changed frequently and also have high complexity, it is possible to narrow down the code that should be investigated first.

Reference: CodeScene — Hotspot Analysis

Conclusion

This time, I researched which metrics should be measured to evaluate the maintainability of AI-implemented code.

In AI development, it seems necessary to pay attention to code complexity, design problems, and the difficulty of subsequent changes, among other factors.

In particular, in my own solo development, I will emphasize modularity and ease of change, and monitor dependencies, change history, and other related factors.

Next time, I will actually implement the metrics selected this time, using tools such as SonarQube.