The AI did it

Someone on a call recently told me that a made-up number on their website was "just something the AI put there”. Like that was an explanation, not a confession.

Dan Davies puts a name on this phenomenon in his book The Unaccountability Machine: the "accountability sink" – a system built so that when it screws you, there's no one to blame.

You’ve experienced it: you try disputing a parking fine issued by an automated camera: the council’s call centre can't overturn it, their supervisor can't overturn it, the appeals office just rubber-stamps the same code the camera spat out. Everyone's telling the truth. Nobody's lying to your face. Nobody can do a damn thing. The system made the call and the system doesn't take your calls.

Generative AI isn't going to fix that. It's going to industrialise it. "The AI did it" is about to stop being a punchline and start being a liability shield. One even harder to fight (ever tried fighting a black box?).

Next time you're rolling out a new system in your organisation, stop and ask: could this turn into an accountability sink? Will there be a real human on the other end with the power to look at a bad outcome and fix it? Or are you just building yourself a complicated way to shrug?

Colin