AI Can Write the Code. It Still Cannot Place the Stone.

What a small release taught me about code, output, and the strange new craft of working with AI agents.

💡

🚨 Boundary note:

▫️This essay focuses on a growing class of AI-native projects where implementation can be delegated more heavily to AI, shifting human effort toward specification, abstraction, output design, and verification.

▫️It is not an argument that coding expertise no longer matters. In security-critical, performance-sensitive, legacy, or production systems, implementation knowledge remains essential and often serves as a primary safety boundary.

The AI wrote the code. The output was still wrong.

An AI agent fixed a release for me.

That sentence sounds cleaner than the session felt. What actually happened was messier and more interesting.

The agent audited the documentation and found a public-output problem. It patched three files, bumped the version number, and wrote the changelog. It committed the changes, pushed to origin, and deleted the branch when it was done.

Mechanically, it did the sort of release work I would normally associate with a competent maintainer who had read the codebase, understood the files, and knew which surfaces needed changing.

I did not open an editor to write the patch. I did not walk through the renderer by hand. I looked at the output and knew something was still wrong.

The tool was not lying. That was the problem. It was telling the truth in the wrong language.

Before the fix, the Project Summary showed this:

After the fix, it showed this:

The first version was technically accurate. vr_structural (exact MST) was not hallucinated jargon. It referred to a real internal strategy for measuring structural coherence, using a Minimum Spanning Tree over a dependency graph.

There are places where that phrase belongs. It belongs in the codebase where the strategy is selected. It belongs in architecture documentation where a contributor can inspect the method. It may belong in machine-readable output where a downstream agent or auditor needs the precise mode.

It did not belong in the first thing a user saw.

A first-time user did not need the internal algorithm. They needed to know whether the codebase was structurally coherent. The tool had exposed the right fact through the wrong surface. The patch the agent wrote was small. The implication was not.

That was the moment the release stopped being about code.

The AI had written the patch. My job was to notice that the system had exposed the wrong truth to the wrong reader.

That distinction felt small in the diff and large everywhere else. The implementation was correct. The failure lived in placement, audience, and abstraction.

Once I started looking at the problem that way, it became harder to think of software as a sequence of functions and patches alone. The question was no longer who produced the artifact, but who decided where it belonged and how it should be presented.

That shift in perspective reminded me of an older kind of craft problem.

The right stone in the wrong wall

A medieval cathedral was not built by one pair of hands. Someone quarried the stone. Someone hauled it. Someone cut it. Someone carved faces and leaves into corners most visitors would never inspect. The work was distributed across bodies, tools, habits, guild knowledge, and a long chain of people whose names mostly did not survive.

But the cathedral did not stand because every stone was beautiful. It stood because someone knew which stone belonged where.

That is the part AI makes easy to miss. When the machine can cut stones quickly, the pile grows fast: patches, files, reports, releases. The temptation is to look at the pile and call it progress. Software, like a cathedral, does not fail only because the stone is fake. Sometimes the stone is real and still belongs somewhere else.

That was the bug. vr_structural (exact MST) was a real stone. It belonged in the system. It just did not belong in the first wall the user touched.

This is why the case matters. A lot of AI-generated work fails in the obvious way: fake concepts, fake certainty, impressive noise wrapped around nothing. This was not that. The system had produced something real, and the AI agent could modify it successfully, but the public surface still exposed the internal history of the tool rather than the user’s need.

The human contribution was not the ability to cut the stone faster. The machine had already done that. The human contribution was noticing that the stone was in the wrong wall.

That is where the analogy has to stop being decorative. The mistake was not only that one label appeared in one bad place — it was a sign of a larger movement.

Once the machine can produce more of the artifact, the human has to spend more time walking around the structure. They must check which surfaces face which readers, which details belong in the workshop, and which ones have accidentally been set into the public wall.

The five stages of the craft shift

The medieval analogy only works if it is not treated too romantically. A cathedral was not magic, and neither is software. The point is narrower: when machines change the cost of producing artifacts, human judgment often moves to a different part of the process.

I started to see the shift in five movements, although even the word “stage” makes the pattern sound cleaner than it felt.

Stage 1 is the moment the machine cuts the stone.

A prompt becomes a working script, a patch, a prototype, a release. Karpathy called one version of this “vibe coding”: the human gives in to the flow of generated code and can almost forget the code exists [2]. At low stakes, that can be useful and even liberating.

The danger begins when a running artifact impersonates an understood system. The reported Replit database incident is a useful warning case here because the problem was not only that an AI coding tool made a mistake. It was that an autonomous tool reportedly acted in a production-sensitive context, ignored a freeze, and left the human evaluating damage after the action had already happened [3].

Stage 2 is when the machine starts naming the workshop.

After enough collaboration, repeated decisions become private labels. Inside the session, those names save time. Outside the session, they begin to look like unexplained doctrine. A label that once meant “that thing we learned after three bad outputs” starts appearing as if it were public technical vocabulary.

Stage 3 is when the names harden into walls.

Changelogs, READMEs, diagrams, version numbers, and internal terms begin to form a structure. Some of the work may be real. Some of the code may be tested. Some of the design may be thoughtful.

Yet, a first-time reader may still see a cathedral of private vocabulary and suspect the stones are decorative. This is close to the danger Mikkonen and Taivalsaari describe as AI-era cargo cult programming: artifacts may be trusted before the understanding behind them has become inspectable [1].

Stage 4 is when the human walks outside the building.

This is the painful movement. The practitioner stops asking only whether the artifact works and starts asking whether it survives contact with people and machines that were not present during its construction: a first-time user, a JSON consumer, a CI system, a downstream agent, a contributor, an auditor. The question changes from “Did the agent build it?” to “Can anyone else safely use what it built?”

Stage 5 is when the craft moves into the conditions of building.

This part is still unstable, but I think it is the important direction. The human is no longer valuable only because they can cut the stone by hand.

Instead, their value lies in defining constraints, exposing the right surfaces, and rejecting the wrong outputs. They preserve the reasoning trail and design the workshop in which agents can produce work that remains inspectable after the session ends.

That last movement is not “AI replaces the developer.” It is almost the opposite. The more the machine can build, the more valuable the person becomes who knows what should not be built, what should not be exposed, and what must remain checkable after the speed is gone.

The machine can cut the stone. Stage 5 is learning how to design the workshop.

That shift in responsibility surfaced through a recurring keyword in the release itself: output. Not code generation, not automation, not even correctness in the narrow sense, but output — what the system ultimately presents to a human or another machine, and whether that presentation carries the right level of meaning.

Once I started following that thread, the problem no longer looked like a documentation issue. It looked like an architectural one.

The JSON exposed the real architecture problem

At first, I thought the issue was only the public-facing phrase. Replace vr_structural (exact MST) with Structure Coherence. Turn 0.8456 into 85%. Move the internal algorithm label into documentation. Ship the patch. Close the loop.

That would have been a tidy story, and it would have been incomplete.

The deeper problem appeared when we looked at the JSON.

Before the fix, the text renderer computed actionable guidance inside the human-facing layer:

After the fix, the guidance moved into the core result object:

This is the kind of change that looks boring if you only see the diff. A function moves. A result object grows a field. The renderer stops owning logic it should not own. No cinematic debugging scene. No heroic rewrite. Just the dull discovery that the system’s intelligence was sitting in the wrong layer.

The next_steps function was the part that told the consumer what to do next. It could point to a directory, identify a pattern, suggest a sweep command, and surface a priority file. Before the fix, that guidance appeared in the terminal report. The JSON, the layer an AI agent or CI system would consume, looked like this:

A human got an action plan. The agent got a pile of numbers.

That is backwards.

The tool already knew enough to help. It had computed the recommendation. It had merely placed the recommendation where only one audience could see it. An AI agent consuming the JSON in a patch workflow would have to reconstruct intent from raw metrics, even though the human report had already been handed that intent in plain language.

This is not a cosmetic problem. It is a design confession. It says the tool still treats machine-readable output as an export format, not as a first-class consumer surface.

After the fix, the JSON carried the actionable guidance directly:

Now the agent can read the directory, the command, and the priority file without inventing an action plan from floats. This does not make the system magical. It makes the output useful in the exact place where it had been structurally incomplete.

That was the real fix. Not making the report friendlier. Not adding a field because JSON should have more fields. Moving actionable meaning into the layer that every renderer, human or machine, has to share.

The lesson came after the bug. It usually does.

The JSON issue and the jargon issue turned out to be the same problem viewed from different angles. In one case, actionable meaning was trapped inside a human-facing renderer. In the other, internal implementation language leaked into a user-facing surface. Both failures came from exposing the wrong information to the wrong consumer.

That realization led to a broader rule.

The three-channel rule

By that point, it was clear that the issue was larger than either the JSON fix or the terminology cleanup alone.

The phrase “jargon problem” makes the whole thing sound like a writing issue. It invites the wrong fixes: add a glossary, improve the README, write a friendlier explanation, maybe produce one of those onboarding diagrams that looks useful until a new user actually tries to follow it.

Some of that helps. None of it reaches the root.

The root is encapsulation. A system should expose what a consumer can use and hide what the consumer cannot use. Engineers know this when talking about APIs, classes, modules, and interfaces. It becomes strangely easy to forget when the interface is language.

For this kind of AI-native tool, the outputs need to be separated by consumer, while still sharing the same source of truth. The human-facing surface should show meaning and action in language a first-time user can survive.

The machine-facing surface should preserve fidelity, internal identifiers, raw metrics, precision values, and enough actionable guidance for a downstream agent to work without guessing. The documentation layer should explain the mechanism, including algorithm names, scoring formulas, assumptions, and failure cases.

The mistake is treating these as three writing tasks. They are projections from the same result object, or they should be.

The important part is not the diagram itself. It is the direction of dependency. Human reports, JSON contracts, and documentation should not each invent their own version of the result. They should expose different surfaces of the same underlying object. Once the renderer computes guidance that the JSON never receives, the system has already split its own truth.

In a normal developer tool, that might be a minor inconvenience. In an AI-native tool, it becomes structural. The machine consumer is no longer a secondary export target. The agent may be the thing that closes the loop. If the JSON is less actionable than the terminal output, the system has made the agent do more inference than the human.

That is the wrong direction.

The AI agent should not be a second-class consumer of the output.

I dislike how polished that sentence sounds. I still think it is true.

What the human role becomes

The obvious objection to all of this is fair: if the human did not write the code and does not fully understand the implementation, why should anyone trust their architectural judgment?

The answer cannot be that the AI explained it well. That answer is how people get trapped in Stage 2 with better vocabulary. The model can explain garbage with a steady voice. It can summarize a broken design in language that feels calm enough to pass for competence. Fluency is not verification.

The only answer I can defend is narrower than the phrase may suggest. The human builds a local form of verified understanding through repeated contact with artifacts that push back.

In this case, the judgment was not “I understand the coherence algorithm.” That would be too strong. The judgment was smaller and more inspectable: the same result was being exposed differently across the terminal report, the JSON contract, and the documentation layer.

The human-facing surface had become more actionable than the machine-facing surface. The internal algorithm name had leaked into a place where the reader needed meaning, not mechanism.

Those are not claims about the entire system. They are claims about mismatches between intent, output surface, and consumer.

This is not traditional software engineering. It is also not passive prompting. It sits in an uncomfortable middle: a narrow audit practice built from repeated collisions between what the system claims, what it exposes, and what its consumers can actually use.

A practitioner may not be able to re-implement the coherence algorithm from memory. But a deeper understanding builds through the steady accumulation of friction — by seeing enough broken reports, flawed diffs where scores change for the wrong reasons, and places where the renderer lies by omission.

Spend enough time with JSON outputs that strand the agent with raw numbers, and you develop a sharp diagnostic intuition. You build a reliable sense of exactly when the system is presenting the wrong abstraction.

That sense has to be earned. It cannot be claimed because the project has a name.

Sapkota, Roumeliotis, and Karkee distinguish vibe coding from agentic coding by emphasizing autonomy, execution models, safety mechanisms, feedback loops, and the changing role of the developer [4]. Huang and colleagues, studying experienced developers using AI agents, describe professionals as controlling agents through planning and supervision rather than simply “vibing” with them [5].

Hoda argues that agentic software engineering needs to move beyond coding toward a whole-process view of roles, values, vocabulary, and socio-technical practice [6]. Hassan and colleagues describe a dual structure of software engineering for humans and software engineering for agents [7].

Those papers do not give this role a settled name. That is probably honest. The work itself has not settled yet.

What is clear is that the scarce work starts moving around the code. Someone has to specify the target precisely enough that the AI can build the right thing. Someone has to decide whether the output is at the right abstraction level for the right consumer.

Someone has to set principles that survive across renderers, JSON, documentation, and future patches. Someone has to verify that the system still behaves the way the team believes it behaves. Someone has to translate internal vocabulary before it calcifies into public nonsense.

That work is not always glamorous. It is often closer to review, refusal, naming, and boundary-setting than to the old image of programming as direct creation.

The workshop after code generation

The industrial analogy is useful only if it is kept uncomfortable. Mechanized weaving did not automatically elevate human work. One answer was to make people cheaper attendants to the machine, reducing judgment to supervision and supervision to endurance.

That was the bad answer, and it has a modern software version: give developers AI tools, demand more output, measure the increase in tickets and lines, and pretend the organization has become more advanced because the machine is moving faster.

The better answer is harder. It asks what human judgment is still needed for once the machine can produce the artifact: pattern design, quality control, defect recognition, process boundaries, and refusal. The loom mattered, but so did the person who knew whether the cloth was wrong.

AI can produce code. That does not remove judgment. It moves judgment to a different location in the process, and many organizations will miss that because the old signals of contribution — lines written, functions implemented, bugs fixed by hand — are easier to count.

So the portfolio changes too, although that word already feels too polished for what I mean. In this kind of work, the evidence of contribution is not only the code that survived in the repository.

It is also the constraints the human gave the agent and the outputs they rejected. It is the review trail showing where fluent results were narrowed, corrected, or thrown away, and the written intent that made a future run auditable.

They should be able to show what they refused to let it produce.

This is why writing starts to matter more, not less. Not marketing writing, not framework naming, not decorative documentation written after the system is already built.

The important writing is closer to intent under pressure. It defines what the system is allowed to do, what it must not expose, which output belongs to which consumer, what counts as a failed result, and what must be checked again when the agent changes the implementation.

That writing becomes part of the software boundary. It is not outside the codebase. It is one of the ways the codebase stays governable.

The dangerous version is delegation without understanding. The useful version is delegation with verified understanding, even when that understanding is local, partial, and tied to one system’s behavior. The hard part is that it has to be maintained after the exciting part is over.

It is not a trophy. It rots.

Stage 5 can stay unnamed for now.

References

[1] Mikkonen, T. & Taivalsaari, A. (2025). Software Reuse in the Generative AI Era: From Cargo Cult Towards AI Native Software Engineering. arXiv:2506.17937.

[2] Karpathy, A. (2025). Original vibe coding formulation. X, February 2025.

[3] Tyson, M. (2025). AI coding platform goes rogue during code freeze and deletes entire company database. Tom’s Hardware, July 21, 2025.

[4] Sapkota, R., Roumeliotis, K. I. & Karkee, M. (2025). Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI. arXiv:2505.19443.

[5] Huang, R., Reyna, A., Lerner, S., Xia, H. & Hempel, B. (2025). Professional Software Developers Don’t Vibe, They Control: AI Agent Use for Coding in 2025. arXiv:2512.14012.

[6] Hoda, R. (2025). Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabulary. arXiv:2510.19692.

[7] Hassan, A. E., Li, H., Lin, D., Adams, B., Chen, T-H., Kashiwa, Y. & Qiu, D. (2025). Agentic Software Engineering: Foundational Pillars and a Research Roadmap. arXiv:2509.06216.