Faster and louder

Yesterday, I mentioned that the optimisation of AI setups often leads to a simulacrum of productivity rather than actually being productive. A couple of hours later, I was reading Gary Marcus’ latest newsletter in which he made a related point.

Books, music, apps, lines of code, scientific papers – the quantity produced for them all is up sharply since generative AI became mainstream. The graphs are hockey sticks. The boosters will show you these numbers and call it a productivity revolution, and if you squint and don't ask any follow-up questions, it looks convincing.

But that’s just output. Book sales haven’t increased, nobody’s going wild about all the new incredible music out there, more apps aren’t being installed (quite the opposite)… More things exist but none of the significant metrics have gone up.

Productivity is output that matters. Volume that has no relationship to value isn’t productivity, it’s noise. We have much faster noise now.

Of course, this is something that’s difficult to measure. Which is why it usually gets ignored.

It’s a lot easier to count tokens spent, or words generated, or tasks “completed” than it is to figure out if anything actually moved forwards. And when things are hard to measure, we have a tendency to measure the wrong thing and convince ourselves it’s the same.[1]

Well before transformers showed up, managers were counting inputs when outputs were too difficult to track: hours in the office, emails sent, meetings attended, lines of code committed… AI just turbocharged it. You can now produce a year's worth of measurable-but-meaningless activity before your lunch break.

So, before you drop AI into another process: what does actual success look like? Not the completion of the task. The point of the task. And would you even be able to tell if AI was helping or just making the noise louder?

Colin

[1] I recommend The Tyranny of metrics by Jerry Z. Muller if you want to dig into this.