The detection wars: AI is publishing’s ‘wicked problem’

AI is being used to cancel books and authors, turn readers against newsletters, and writers against each other. ALLi director Orna Ross discusses AI detection tools, due process, and why certifying writing as “human-made” matters far less than treating each other with humanity

This summer, the publishing platform Substack wired an AI-detection tool called Pangram into its platform without consulting Substack writers, or providing any meaningful opt-out. Substack readers can now quote Pangram’s assessment of the percentage likelihood that a post was “human-made”.

The result was some writers shouting “witch-hunt”, others cheering the platform for housekeeping that was long overdue, and neither showing much understanding of the other.

Pangram’s co-founder and chief technology officer Bradley Emi recently published a response of sorts: “LLMs don’t just mimic human text”. Silicon Valley veteran Lisa Kostova used her Substack to describe the article as using a well-worn trick in tech-land: “proof by intimidation,” confidently answering a question nobody had asked by dazzling use of jargon, charts and stats, while ignoring the moral and ethical concerns actually being raised.

Whether AI text can be distinguished from human text was never the community’s question. The questions were: What exactly does the score measure? Why was the community not consulted? Why is Substack taking no responsibility or setting any guardrails as they send their community of readers and writers off to police each other?

Neither company has deigned to meaningfully engage.

It’s been described by many as a civil war. That description is only slightly dramatic. The huge, raging divide on Substack is a microcosm of the argument running through every strand of publishing.

Since the start of the year, publishing has lurched from one AI scandal to the next.

A debut thriller won by Minotaur in a seven-figure, two-book deal after a reported 14-way auction was withdrawn by its agents in late July for suspected use of AI. The novel, whose title and author PL (Publishers Lunch) initially withheld for the author’s privacy, sold for an alleged $2.5 million or thereabouts. But rumors surfaced quickly from people who had read the manuscript that the book contained many of the hallmarks of AI, including negative parallelism, off-kilter metaphors, and a flat prose style. In late July, the author’s co-agents Marc Gerald and Ashley Coleman at Europa Content withdrew the book, after questioning the author about the validity of the rumors.

Famously, Hachette pulled Mia Ballard’s horror novel Shy Girl in the US and discontinued it in the UK after unproven allegations that the novel had been heavily AI-generated — even though it had been on sale in the US for months after going through Hachette’s own editorial process.

In the UK, there was “MattGPT”: did the right-wing polemicist Matt Goodwin use AI in the making of Suicide of a Nation, and were the alleged ChatGPT giveaways in the book’s references — dubious quotations, sourcing that didn’t check out — serious enough to damage its credibility? Goodwin publicly denied that AI wrote the text of his book.

And the world of literary criticism also had its first AI moment, when critic Alex Preston lost his New York Times reviewing berth after a reader spotted similarities between his review of Jean-Baptiste Andrea’s Watching Over Her and Christobel Kent’s earlier Guardian review of the same novel.

A wicked problem

In 1973, the planning theorists Horst Rittel and Melvin Webber coined the term “wicked problem” for the kind of challenge that has no definitive formulation, no clear stopping rule, and no solution that doesn’t generate fresh complications. The language was built for social planning but it fits AI in publishing uncannily well.

We all know we have a problem here but we cannot agree what it is, never mind how to solve it. Is it plagiarism? Disclosure? Copyright theft in training data? Confidentiality? Reader deception? Laziness? Weak governance? Unreliable detection? The answer, maddeningly, is yes. And that’s before we get to environmental and other AI issues external to publishing.

Behind every AI process and every AI use, there are human beings, and collapsing all this into “AI bad, humans good” gets us nowhere. AI itself has become the new fall guy for some very old publishing problems.

Whatever one feels about Matt Goodwin’s politics, that dispute is about accuracy, sourcing and accountability — factual reliability in a nonfiction book. Publishing has argued about standards, checking and responsibility for as long as it has existed.

The Preston case is about plagiarism and professional negligence. The author says a machine was the mechanism. Regardless, the ethical failure was human.

AI accelerated but did not create the ethical issue.

Shy Girl exposed the industry’s procedural weaknesses. Ballard denied personally using AI to write the novel, telling the Times that an acquaintance hired to edit an earlier self-published version incorporated AI tools. “My name is ruined for something I didn’t even personally do,” she wrote to the newspaper, and described her mental health as being at an all-time low.

That account has been met with disbelief in some quarters; it is also perfectly possible.

And again, so many deeper questions jump out. How did the book pass Hachette’s editors? But more importantly for the writing community: why do publishers and some writers feel they are measuring virtue, when what they are measuring is vocabulary?

A detector measures one thing: the statistical resemblance of a text’s patterns to machine-generated text. A score like 78.3 sounds like a lab result; something definite and observed. In actuality, it is a probability score from a pattern-matching tool.

And in Ballard’s case, a score solicited by commercially interested parties, run on a pirated file, delivered without any due process.

What we do need to care about

ALLi’s mission has always been ethics and excellence in self-publishing. AI changes the tools, the terrain and the tempo; it does not change that mission.

When it comes to AI, everyone passes the ethical parcel. Tech companies say the tools are neutral. Platforms say they only host. Publishers say they “are at the mercy of authors” and “trust in writers is plummeting”. Critics, readers and publishing consultants turn amateur detective, judge, and jury. Agents raise concerns that editors are uploading confidential manuscripts to AI for quick assessment. Penguin Random House has said publicly that it uses detection software as part of editorial evaluation but not whether authors’ permission has been sought.

Big publishing and big platforms cut deals with big tech and allow AI training on texts — while every finger that points, points at a writer.

The big question for our community is: why, in an industry that is quietly threading AI through every process and making content deals with AI tech companies, are writers the only people being interrogated, canceled, harangued, doxxed, review-bombed, and worse, for AI use?

We are already in a world where digital tools are increasingly powered by AI, across all publishing processes. We are trying to build certainty out of private workflows, inconsistent norms, unverifiable claims, but certainty is not on offer. If we want to write and publish without AI, we need to go completely analog — hand presses, print only, live events as marketing — but anyone with any sort of online presence is using tools that have AI built in.

Various “human-authored” schemes are being used to show books have been written without generative AI. Many see these programs as divisive, and even their advocates acknowledge such labels rest on trust.

Trust in publishing, with or without AI, is built the old way. Accuracy and sourcing in nonfiction. Attribution in criticism. Consent and confidentiality when handling manuscripts. Protection of authors’ work and copyright. Responsibility for output. Honesty with readers. These are the real issues, and ALLi’s position on them has not changed and will not.

Machines can put a score on our sentences; only we can answer for them.

Three things you can do

1. Decide your disclosure position before anyone demands it. A short note on your website about your AI policy turns a potential accusation into a conversation you set on your own terms. ALLi has a draft policy you can use, depending on whether you are AI-minimal, assisted or integrated.
2. Keep your provenance. This is good advice for any rights claims. Drafts, notebooks, version histories, and yes AI chats and skills are your best witnesses of authorship: what you write and how you write it.
3. Resist the game. The tech companies that gave us these tools are now putting the onus on us to use them in a certain way. The very terms of this debate are being dictated to us, and no one is affected as much as we are. Do not comply. Don’t run a fellow author’s work through a detector and announce the number, for any reason. And do not assume any score is the truth.

Handled, not solved

Wicked problems are not solved once and for all. Mandatory disclosure raises boundary questions; detection creates false positives; blanket bans drive use underground. Such problems are handled, imperfectly, through creating better norms, clearer responsibilities and more truthful conversation — and a presumption of good faith for authors, a presumption that everyone else in publishing already enjoys.

Wicked problems also demand human answers — and human answers begin with treating each other with humanity, which no badge, score or certificate can supply. The three broad approaches our members take — no or minimal AI, AI-assisted, and AI-integrated — are all legitimate ways to be an author now, and each deserves equal respect. Accountability is the ethical test, and the reader is paramount.

The solution is very old-fashioned, very analog, very flesh-and-blood: to add empathy to ethics and excellence. Empathy on this issue runs in many directions: toward authors navigating a fast-changing toolset in good faith; toward readers entitled to know what kind of work they are buying; toward one another, across the divide, in place of the doxxing and the witch-hunts.

Our answer must be what it always has been: good values, high standards, community and collaboration. Together, as ever.

Wicked!

Orna Ross is the founder of ALLi. She writes historical fiction, poetry and nonfiction guides. Browse her books on her website and follow her on Substack.