Outrage over Claude’s AI watermark is missing the most important point

It’s everywhere, and it’s getting harder to spot. We are – whether we like it or not, whether we know it or not – reading an increasing amount of words that nobody ever actually wrote.

LinkedIn, for instance, is so full of AI-generated text that it has been forced to add new updates to allow people to report it. But it is not just rushed executives relying on artificial intelligence to do their communicating for them; more and more scandals are appearing where someone has attempted to pass off synthetic words as their own.

And so, last week, Anthropic announced that it would do something (relatively small) about it. The company, which makes Claude, announced that it would add a text watermarking system so that readers can check whether the thing they are looking at was likely to have been produced by its large language model.

This kind of transparency is increasingly important. To be accused of using AI to generate text carries an increasing social and even financial cost. There have in the past few months, a growing number of books have come undone because their authors have been accused of using artificial intelligence in their creation. It’s important that we know where something has come from.

That is getting more difficult as the AI systems become more sophisticated. There exists a host of tools that aim to spot AI writing, but do so using rough heuristics that can lead to false positives. (Some of the more obvious signs of AI writing – like em dashes – are a mark of AI writing partly because they are found more often in literary work of the kind that has been used to train large language models, and carry an air of legitimacy.)

That’s why Anthropic was forced to add more transparency to its systems. (Or, more specifically, it’s why the European Union forced Anthropic to do so; all of this is to comply with the bloc’s AI Act.) The new update to Claude is a response to a real problem, and one that threatens to undermine our trust in the written word itself.

Anthropic was initially a little coy on exactly how it would work. But in the last few days it has revealed the principles of the system, if not its exact process. It works by favouring some words over others – not so much that it would be obvious, or change the quality of the output, Anthropic is keen to stress – so that automated checking systems can do statistical analysis and see whether those words appear more often than one would expect in normal, organic, human writing.

On Reddit, some users revolted. Their objections were various: it wasn’t fair because people might just be using AI a little bit, such as for editing or proofreading their work; that it would be easy to remove and so really it would only be unsophisticated people who were caught out; that it wouldn’t prove anything anyway, since it wouldn’t prove anything either way. But many of the comments really came down to one central objection: it wasn’t fair that people were going to get caught out.

But there was also another, perhaps more fundamental, criticism: it was going to make Claude’s text worse. That was put perhaps most emotively by tech commentator John Gruber, who claimed that the watermarking was a “perversion of writing”, in a blog post on Sunday.

“The exact words we choose when writing matter. I want any LLM I use to choose the very best, most precise words at every single decision point,” he wrote, while acknowledging that there were certain constraints on doing that. In the case of AI, that’s time and computation; it’s much the same in human writing, which depends on time and care, he wrote.

“In other words, these are necessary trade-offs. These factors are all in my interest: speed, cost, quality. Ideally I would like perfect writing, at instantaneous generation speed, at zero cost. None of those things are possible,” he wrote. “The idea that anything other than my needs should factor into the generation of text for me is patently offensive.”

This kind of outrage continued throughout his argument. "The best writing I see come out of these models is worse than anything I would choose to read for pleasure. And now Anthropic is saying they’re going to make it worse, on purpose, for purposes that do not benefit me in any way? Even if only slightly worse?”

“Get f***ed,” he wrote, but without the censorship.

There are, of course, lots of problems with this. The tone of indulged indignation, for one, is bizarre when we are arguing about an automated text generation system that most people use for free. But there’s a deeper problem, fittingly with the exact word that Gruber and others chose to use: writing.

Prompting LLMs isn’t writing, either. And one of the key things that will ensure we keep that word as the important thing it is will be to know when someone is trying to trick us

Claude is never writing; the people prompting it to automatically generate text aren’t writing, either. At least not as a verb and probably not as a noun. Writing – with its etymology in scratching and carving – is a human, embodied, creative act, the result of other essentially human things like thinking and choosing.

Large language models generate text; they do so through complex statistical calculations. That is many things – it can, in many ways, be important and helpful and good – but it is not writing. AI is going to take many things from us, and perhaps give us many others, but it can’t have that word.

Prompting LLMs isn’t writing, either, though the prompt itself may well be. And one of the key things that will ensure we keep that word as the important thing it is will be to know when someone is trying to trick us. That’s why Anthropic’s gestures towards transparency are so important; it keeps the meaning of the word, and of the words, clear.

This, surely, is the real worry beneath all this outrage. Those who claim that Anthropic’s new system is going to embarrass them are telling on themselves – they are, in some quiet and perhaps unknowing way, admitting they are ashamed, and that they don’t wish to be shamed. But passing off AI writing as your own is shameful and insulting.

The deep, undeniable, to me obvious fact is that people deserve to know where the things they read come from. This is true at the grandest, most artistic level: whatever else it is, literature at its best is a communion with other people’s ideas, only possible in community. But it is true at the most instrumental, least profound level, too: meaningful writing requires a writer, from whom the writing can be understood to have come.

This is the great power and threat of writing: all the language we use is a public commitment to having said something. That can be thrilling and it can be terrifying. All Anthropic has really done is make that clear.