In the last year, it became normal for us to ship code at a pace that would have felt unrealistic before. Features moved from idea to PR in hours. Boilerplates vanished. Engineers were less blocked. And yet, I found myself more anxious about what we were putting into production.
Because speed changed the shape of risk. Like coming to terms with driverless cars.
When a car drives itself, the hardest part isn’t making it move. It’s making it safe in the messy edge cases: bad weather, weird road markings, unpredictable humans, rare events. Autonomy doesn’t eliminate risk; it changes where risk lives.
In the agentic era, output is cheap. Judgment is expensive.
And while we move toward a world where AI writes 100% of the code, the question won’t be “can we ship?” It’ll be: can we trust what we shipped?
Agentic tooling is a real force multiplier. Let’s be honest about the upside.
– Boilerplate disappears
– Repetitive refactors become trivial.
– Engineers get unblocked faster.
– Prototypes become cheap enough to explore more ideas.
– Documentation, tests, and summaries can be drafted instantly.
Agents remove friction like driver assist removed the exhausting parts of driving.
But there’s a catch: some of the friction we removed was doing important safety work.
When code generation is nearly free, your biggest threats stop being syntax errors. They become systemic because AI scales both productivity and pattern replication.
AI code often looks good: tidy, idiomatic, confident. Functionally, it works.
But the danger is invisible cracks: weak invariants, shallow error handling, edge cases ignored. It passes superficial review and fails in production reality.
Everything looks “professional,” but nobody can explain why it’s correct. Increasing unexplainability is a symptom to watch out for.
2) Architectural drift accelerates
Agents are not aware of your architecture decisions, incident history, or why certain patterns are “banned here.” Without context of your company, guardrails, you get novelty-by-default: new paradigms, boundary erosion, inconsistent layering, service sprawl.
One insecure pattern doesn’t land once. It propagates. Insecure defaults, missing auth checks, unsafe dependency choices, rogue environment variables will now be repeated at machine speed.
Agents optimise for feature completion. They don’t naturally add observability, rollback paths, performance budgets, rate limiting, graceful degradation.
So we ship faster while reliability decays until on-call pays the bill.
Watch out for more shipped features, same dashboards, rising incident load.
As agent output gets more fluent, organisations start to mistake confidence for correctness. A green CI run and a tidy PR description become the new “looks fine to me,” and reviews quietly shift from deep reasoning to a quick skim.
In a world where AI writes most or all of the code, the biggest failure mode isn’t generation. It’s verification fatigue: humans staying nominally in the loop, but no longer truly engaged. Like a driver in self-drive mode drifting off, senior engineers can slip into lazy skims until production reminds us that plausibility isn’t proof.
The signal I’ve noticed is simple: reviews are getting shorter while production surprises are increasing.
Autonomy creates an ownership vacuum unless you name ownership explicitly.
If “the agent wrote it” becomes an excuse, trade offs go unowned, and long-term maintenance becomes nobody’s job. In an autonomous car, who is responsible for the accident? – AI? Owner of the car? The developer who wrote the software? You get the point.
If engineers stop practicing debugging, systems reasoning, and security intuition, we don’t just lose “hands.” We lose resilience. Those skills don’t come from speed or fluency they come from time spent in the uncomfortable parts of engineering: production failures, weird edge cases, fixing vulnerabilities, and learning why systems break.
And that creates a new question in an agent-first world: if agents take over the junior and mid-level work that used to build that muscle, how do we develop future senior engineers? Without deliberate pathways, we risk becoming a team that can ship quickly, but can’t diagnose, defend, or recover.
You see it when the system misbehaves: incidents drag on, and fewer people can explain the system end to end.
So part of my job now is not just raising throughput, it’s protecting the apprenticeship: creating intentional ways for engineers to build those instincts even when agents can do the first draft.
We are standing at a juncture where the definition of an engineer will soon change and shift far left [ sneak peak in my next article]…
I used to think my job was building a high-performing team of engineers.
Now I think my job is building a high-performing socio-technical system: humans and agents shipping together, safely.
My “team” includes code generation copilots, review assistants, and incident response helpers. And like any teammate, they need:
– direction
– context
– feedback
– boundaries
So yes: I’m spending leadership time “upskilling” agents, the same way I invest in growing people:
– teaching the tools what “good” looks like here
– tuning them toward risk, NFRs, and security signals
– feeding incident learnings back into prompts and checklists
– evaluating them like teammates: what do they catch, what do they miss, when do they hallucinate confidently?
Because my job more than ever is to protect what doesn’t automatically appear in a fast-moving codebase:
coherence, operability, security, reliability, and shared standards.
1) Treat verification as a first-class engineering product
Invest like it matters:
– better tests (not more tests)
– better review quality (not more reviews)
– better tooling to surface risk (not more dashboards)
2) Make NFRs impossible to “forget”
If “definition of done” doesn’t encode safety, safety won’t happen.
– Move NFRs from culture to system:
– CI checks for coverage in critical areas
– default observability hooks
– templates that force rollback paths
– paved roads (golden paths) that are easier than freelancing
If agents can produce fluent diffs, the PR needs to carry proof, not prose.
A good agent-assisted PR should include:
– What invariant is preserved or introduced?
– What are the failure modes and mitigations?
– How is it observable?
– How do we roll it back?
– What’s the test strategy beyond happy path?
3) Close the loop: incidents should train the system
Postmortems shouldn’t just change people’s memories. They should change:
– checklists
– templates
– agent prompts
– review rubrics
– automated guards
If the organisation learns but the tooling (agents) doesn’t, you’re rebuilding the same traps at higher speed.
6) Protect the apprenticeship on purpose
If agents remove the early learning reps, you need new mechanisms:
– structured incident shadowing
– rotating ownership of risky systems
– design reviews where juniors explain system behaviour
– pairing on “why” not “what”
You’re not slowing down. You’re ensuring you have humans who can take over when the car hits the roundabout in the rain.
A few practical signals I’m using as an EM:
– PR descriptions get slicker, but reviewers ask fewer questions
– change volume rises, but operational maturity stays flat
– incident count creeps up, or MTTR worsens
– fewer engineers can explain end-to-end system behaviour
– architectural decisions feel “random” or inconsistent sprint-to-sprint
– reliability work becomes “after we ship” forever
When those show up, the problem is rarely the agent. It’s the system around it.
This shift is happening in real time. I don’t have all the answers.
What I do have are the patterns I’m seeing, the risks I’m learning to name earlier, and a growing conviction that engineering leadership has to move to where the risk has moved.
Because the future isn’t humans or agents.
It’s humans and agents shipping together and the winners will be the teams that treat verification, operability, and accountability as the real product.
If you’re an EM experimenting with agentic workflows, I’d love to know:
– What guardrails have actually worked for you?
– Where have you seen verification fatigue show up?
– What are you doing to protect junior growth?
Whatever your specialist area, wherever you are in your journey, we’d love to hear from you.
