Confidence Is Not Competence

It went live on a Tuesday, and nobody noticed. That was the problem. The change was small, the kind of thing that used to cost an engineer an afternoon and now costs anyone an hour. AI wrote it. The person who pushed it has never been an engineer and never pretended to be, and they understood the problem they were solving better than anyone else in the building. They tested it the only way they had ever known how to. Does it do what I asked? It did. So it shipped.

I want to resist the easy version of this story, the one where someone was careless, because they weren’t. They did exactly what the tools invited them to do. AI had handed them two things that used to arrive together and now arrive apart: the power to write the code, and the confidence to ship it. What it could not hand them was the third thing, the one that takes years. The accumulated memory of watching code do something other than what it announced. The specific humility of having trusted a change that looked finished and wasn’t. Confidence got cheap. Competence costs exactly what it always did.

For three days there was nothing. Then the tickets, a trickle and then a flood, every one of them a version of the same sentence: customers seeing data that wasn’t theirs. Support walked in blind. They didn’t know a change had gone out, so they were apologizing for something they could neither see nor reverse, which is the cruelest place to put a support team, made accountable for a decision they were never allowed into.

Here is the line the whole thing turns on. Functional knowledge tells you what a system should do. Engineering judgment is knowing the quiet and specific ways it can do something else instead. Those are different skills, and they were never meant to sit inside one person by accident. AI closed the first gap almost entirely and widened the second without anyone choosing to. The person could now produce the behavior they wanted. They still had no instrument for the behavior they didn’t.

When a room finds out, the instinct is to locate the person and tighten the leash. The instinct is wrong, and if you lead the place, the failure is yours and not theirs. You gave a functional expert a tool that produces confident, working-looking output, a clean road to production, and nothing to feel the edge with, and then you were surprised when the gap surfaced where gaps always surface, in front of a customer.

Run the tape forward and the cost stops being a support queue. Your largest customer has watched the data slip once before. This is the second time. They don’t threaten and they don’t escalate, because people who are finished don’t bother to. They just decline to renew, and years of earned trust leave in a single email forwarded to procurement. We are not there yet, and that is the entire reason to write this down while it is still hypothetical. There is time to pull the reins in.

Pulling them in is not banning the tools. The energy behind all of this is real, it is not going away, and I would not want it to. It is two unglamorous commitments. Every change that touches a customer or their data has a named engineer who owns the outcome, not a reviewer who skims it and moves on. And the safe path has to be the easy path, because when reaching production depends on someone happening to know where the cliff is, eventually a night comes when nobody does.

Under both sits something I believe more, not less, the stronger these tools get. Every function earned its competence over years. Sales sells. Support carries the customer through the exact moment the product fails them. Engineers build the thing that holds under load at three in the morning. AI didn’t erase those specialties. It just made it briefly easy to forget they exist for a reason. The best teams I have run were never the ones where everyone could suddenly do everything. They were the ones where people did what they were great at and trusted the people beside them to do the same, and were free to, because no one was being asked to pretend the years didn’t count.