AI, Check My Work; or,
De l’art de vérifier plus tôt

The sentence I keep coming back to isn’t in my book. It was in an email I sent to a man in Vermont I’d never met, after explaining why the president of a Chamber of Commerce in the Sierra Nevada foothills was bothering a Chamber director most of a continent away.

I’d love to connect and, if you’re willing, have you check my work.

You already know what came back. What I hadn’t understood when I sent the email was how much of the whole project was sitting inside that request. I wanted somebody besides my AI book club to read my book, obviously. They tell me that’s how this writing thing is supposed to work. But I also wanted somebody who did this peculiar Chamber thing we do to tell me which parts belonged to the work and which belonged entirely to me and my particular little corner of California. AI couldn’t help me there.

By then I had spent quite a bit of time and some money creating the thing I was asking him to examine. For most of that time, I’d thought the purpose of the expenditure was to publish a book. And it was. I’m not going to pretend I accidentally became a publisher while conducting research. But something else had become possible along the way, and I’m still working out what to call it.

The calculus came first, though, and it deserves its own paragraph, because wanting to publish the book didn’t make the numbers look any better.

When the manuscript of Embracing the Local was more or less finished, I did some sober accounting of what it would take to publish it properly, and the sober accounting said no way. Professional editing, design, production, distribution, some gesture in the direction of marketing, and the strange secondary career of becoming a publisher in order to have published one book. Against all of that stood an audience I estimated at roughly twenty people if I was lucky, half of whom owed me a favor, all of them living along a twenty-mile stretch of rural highway near the end of nowhere.

Twenty-one, if somebody’s mother got enthusiastic.

I liked my estimate. It was a joke, mostly. It was also the only number I had any evidence for. These were people who knew the place, knew some of what had happened, and might reasonably want to read about it. Everybody else was hypothetical, and hypothetical readers are notoriously unreliable about buying books.

A person can accept that a project will lose money and still have a limit on how much money, time, and aggravation he is willing to lose. I never calculated the number I would have refused. I knew there was one, and the ordinary path to publishing appeared to sit on the other side of it.

There’s a version of this story that rails against commercial publishing for failing to recognize an opportunity lying around in the foothills. But a publisher who declines a book about the Twain Harte Area Chamber of Commerce isn’t necessarily failing to recognize my genius. She may be correctly observing that the project sits below the scale at which her business works, and declining to underwrite a loss that would land on her employees rather than on me.

Her position is difficult to condemn. Make a note of my magnanimity.

Somebody has to carry the risk, and the person carrying it reasonably expects to decide what gets made. That arrangement has almost certainly kept some wonderful books from ever seeing the light of day. It’s also almost certainly prevented some terrible ones, and I don’t get to establish which kind mine is by pointing out how much I wanted to publish it.

The more interesting question was what it would take for my project to be worth doing anyway, on terms I could afford, at a standard I wouldn’t be ashamed of.

I wrote in “AI! Burn Him” about the tools that changed that calculation, so I will spare you another tour of the publishing department in which every employee is me and four AI models. The writing was mostly there. What moved was the price of getting help with everything arranged around it, including things I hadn’t known enough to realize I would need help with. Editorial pushback. Research. Design decisions. File formats. Distribution systems. The coordination of all those things, which turns out to be its own form of expertise and not something that happens automatically because you have made a spreadsheet.

I could get further into unfamiliar work without first hiring somebody or spending several weekends becoming just competent enough to discover how much I still didn’t know. I could also get a confident wrong answer much faster than before, but we’ve covered that too.

For a while, I thought the interesting question was how small the publishing company could get before it stopped being able to produce a credible book. What I’d missed was the other side of the equation.

How small could the audience get?

Publishing has costs that someone has to recover or be willing to carry. A commercial publisher needs enough sales. An author doing this because he wants a particular thing to exist may accept a loss, but even he eventually has to pay the bill. The amount changes what he is willing to attempt.

There is nothing especially new about this. Printing, desktop publishing, the web, print-on-demand, and digital books have all changed the math. People were publishing obscure, particular, commercially questionable things well before a chatbot offered to help them with their metadata. I am not claiming that human expression spent five hundred years waiting for my method.

What happened in my case was that another set of costs fell far enough to make the project reasonable to me.

Nothing turned twenty readers into twenty thousand. Twenty simply became potentially enough. Economically, I mean. My ego is another matter.

But those twenty aren’t simply a consolation prize. They are the people whose place and work I wanted to write about. Reaching them would have been enough. Once I could afford to make the book for them, I didn’t have to pretend I’d established a market among strangers before proceeding. I could answer the question I actually had enough information to answer: was this worth doing for the people I already knew it was for?

Finding out who else might care could happen afterward.

That change in sequence is easy to miss, especially if you’re the person in the middle of it, trying to figure out why Amazon KDP keeps rejecting your cover file.

* * *

When I started sending the book to other Chamber people, I was trying to find out whether the experience traveled. What deserves a little more attention is how I chose the people.

I selected for resemblance. I looked for organizations serving places where tourism, local business, community identity, and civic life sit on top of one another the way they do here. I preferred Chambers serving several communities rather than one tidy town. I went looking where recognition seemed most likely, because those were the people whose answers I thought would mean something.

That’s a reasonable way to find somebody worth talking to. It is also a reasonable way to find somebody who will tell you what you were hoping to hear.

If a vendor handed me that evidence in a procurement review, I would hand it back with questions. He has selected people likely to recognize his problem, shown them his account of it, and returned with encouraging responses from the ones willing to answer. Wonderful. What about the people who didn’t reply? What about the organizations he didn’t approach? What exactly does an encouraging response establish?

Unfortunately, I know the vendor in this case, and he would like a little more credit for having written an entire book.

The Vermont Chamber director’s recognition mattered to me enormously. It didn’t establish that anything changed at his Chamber, that anybody avoided burnout, or that a lesson from our organization survived being put to work in his.

Recognition is not reform. I’d be embarrassed to confuse them at work, but this is my essay, okay?

The Washington Chamber director’s question about staffing put a boundary around the comparison. We might encounter similar problems. We did not have the same arrangements for dealing with them. In one organization, somebody’s responsibilities remain part of a paid job. In the other, the Chamber work has to fit around whatever else everybody’s life requires.

And while drawing that boundary, she gave me “pancake and parade chambers,” which improved immediately on my own “party chamber” nickname. I told her I might steal it. Only later did I notice that the exchange had supplied its own small demonstration. In the reply that made me more careful about what our organizations had in common, she handed me language useful enough to bring straight home. I was busy examining whether my experience traveled while hers was already making itself comfortable in my vocabulary.

The man in Vermont supplied a demonstration of his own, although it wasn’t the one I’d asked for. He had just bought a Kobo, a brand of e-reader I hadn’t known existed until I started researching ebook distribution, and I sent him the Kobo edition with the cheerful warning that he would be my Rakuten quality control, since I didn’t own one and had never tested the file. He took the job more seriously than I had. A few hours after telling me what the book had meant to him, he wrote again to say that the file had worked beautifully and that he’d found a couple of typos, although he wasn’t sure whether that was the file or his being new to Kobo. It was neither, as it turned out, because I had sent him an older file.

So I had asked a stranger to check whether our lessons traveled, and he had also checked whether my files did, which, to be fair, I had asked him to do in a sentence I’d written mostly as a joke. All that coordination I described earlier, the kind that doesn’t happen automatically because you have made a spreadsheet, had slipped at exactly the moment I most wanted to look like a publisher, and it took a reader in Vermont to notice.

There’s a more troublesome objection to the argument I’m making about what all this cost: I didn’t need to publish a book to send an email. I could’ve written two pages about what I thought we’d learned and sent those to the same people. I could’ve sent excerpts, an essay, or a list of questions. Any of those might have produced a useful conversation, the difference between a staffed Chamber and an all-volunteer one doesn’t require 150-some-odd pages to explain.

I don’t know whether a shorter account would have produced equally useful responses. The man in Vermont did get a page of passages before he got the book, and the page was enough for him to see the resemblance, although it took the book to carry that resemblance into a conversation with his board president, and one reader in one order is an anecdote rather than an experiment. I didn’t try the alternatives in any way that would let me compare them.

What I did give people was enough of our experience to wander around in without me standing beside them pointing at the conclusions. They could follow a decision into its consequences, notice something I hadn’t thought especially important, or stop a few chapters in to ask what kind of organization I was actually running. Some of them did.

I also hadn’t written the book solely to investigate whether Chamber problems were portable. I wanted to tell the story. I’ve grown to love this town, our Chamber, and the people who keep it all afloat. The book had a very personal reason to exist before I started asking it to perform a second job.

So I can’t claim that cheaper publishing made checking possible. People have been asking other people whether they’re full of shit for a very long time. But it made my story affordable enough for me to finish and circulate. Circulating it then taught me things I hadn’t known when I decided it was worth making.

I would prefer a larger conclusion. I have this one.

* * *

The first experiment gave me an answer I wanted and made me limit what it proved, while the second gave me an answer I didn’t want and made me do something about it.

My second experiment cost me a Monday and ended with me dismantling a thing I’d hoped to keep.

La Société de pensée has always had a transportation problem. I schlep material and arguments between models by hand. Copy, paste, explain what the other one said, carry the response back. The systems may be artificial intelligences, but their communications department is a middle-aged man at a keyboard.

One day I decided to build a transport mechanism, an elegant little ferry. All four models could be open in their own Chrome tabs, and ChatGPT and Claude could interact with those tabs. So I had an assistant carry the conversation between them from inside its own client, doing the copying and pasting I’d been doing myself. It worked, and I was extremely pleased with it.

It was a really elegant solution. Then, like the Boy Scout I usually am not, I thought maybe I should read the terms of service. I wish I’d done that earlier. I also briefly wished I hadn’t done it at all, which tells you something about how the next part is going to go.

The trouble was the browser route. OpenAI’s consumer terms prohibited automated extraction of output. Anthropic’s restricted scraping and automated access, with exceptions for its API and other explicitly permitted uses. I had treated the chat windows as something an assistant could operate on my behalf because, technically, it could.

There was an asymmetry here I found difficult not to call hypocrisy. These assistants could reach into other browser tabs and do work for me, but directing that ability back into the chat services ran into restrictions on automation. Other sites have their own rules, of course. Being annoyed by the distinction did not grant me an exception, although I was prepared to be quite annoyed. I was also less entitled to the indignation than I would have liked, since my own copyright page tells the machines to keep their hands off my book, and here were the machines’ owners telling me to keep my machines off theirs.

Some of what I wanted to automate was prohibited. Some depended on the route or account involved. Some was murky enough that I wasn’t willing to pretend ambiguity meant permission. The important part is that the system worked, and I still couldn’t use it as designed without violating the rules I’d agreed to.

I sat there and seriously considered whether my elegant solution might be fine actually, because it had worked so well and the alternative was starting over. Then I spent real energy looking for an interpretation under which it survived. Surely there was a distinction I had missed. An exception. A different understanding available to a sufficiently thoughtful person who had already built the thing and would very much like to keep it.

It is remarkable how much intellectual energy becomes available when the alternative is admitting you should have checked something earlier.

Eventually I took it apart and spent the rest of the day pouting. Then I went back to CTRL-C, CTRL-V, which has never once asked me to read its terms of service.

I knew perfectly well that “does it work?” and “can I use it?” were different questions. Knowing that had not caused me to ask them in the right order. By the time I got to the second one, I had an investment in the answer.

The tools made it easier to build the experiment. They also made it easier to get attached to a working result before I’d finished examining the constraints. Lowering the cost of making things does not automatically improve the order in which you think about them.

Apparently I still have to do that part myself.

* * *

Set those experiences beside each other and I’m tempted to say the cheaper thing was being wrong.

Writing the book still took substantial effort. Sending it to strangers required some chutzpah. Reading it took other people’s time, which they gave me generously and which doesn’t become free simply because it doesn’t have a line item in my accounting spreadsheet. My elegant ferry consumed a day, and noticing the problem after building it was more expensive than noticing it before.

I don’t get to call the whole process cheap merely because nobody sent me an invoice for reading. But enough of the surrounding work came within reach that I could make something, find out more about it, and absorb an answer I hadn’t wanted. The book brought encouragement and a more precise understanding of where my experience might apply. The automation brought a reason to abandon an arrangement I was proud of.

Neither gave me certainty. Both changed what I was prepared to do next.

That’s the part I’m interested in. I could afford to proceed without having settled every question in advance, provided I remained willing to do something with the answers when they arrived. Otherwise I would just be producing things more efficiently and defending them with greater enthusiasm, a field in which humanity already has considerable expertise.

There are enormous categories of work whose first plausible audience is small. Local history. Family history. Community memory. Obscure scholarship. Narrow technical craft. The story of some volunteers running a Chamber along a twenty-mile stretch of rural highway at the end of nowhere.

Those things have never depended entirely on commercial publishers. Archives, correspondence, historical societies, specialty presses, newsletters, and people who keep telling the story have carried particular knowledge a very long way. Some of it has traveled widely. Some has mattered intensely to twenty people, half of whom owe me a favor, which is also a way for something to matter.

What I’m wondering is how often the audience we can see determines whether we bother preparing the work for anybody else.

Before I circulated the book, I knew some people here would care. I had very little evidence about anyone beyond them. Those facts looked an awful lot like an estimate of the total audience, especially when I put them beside the estimated bill. They weren’t. I knew where the audience began.

Making the book affordable let me proceed on that basis and learn a little about what lay beyond it. There may be plenty of places where our experience remains entirely foreign. There may be people who recognize the problem and think I’ve misunderstood it. I would like to hear from them, although I reserve the right to be irritated before becoming grateful. Have you met me?

The same qualification probably applies to my elegant ferry. While I eventually accepted what I found in the various TOS agreements, the road to that acceptance was colored with quite a lot of profanity.

I began with twenty readers and some idea of what I was willing to lose. I now have a few more readers, better questions, and a suspicion that some useful work stays close to home because getting it into a form strangers can examine costs more than its first audience seems to justify.

That happened to be a threshold I could cross. I don’t know how many other people can cross theirs, what they would make if they did, or whether anyone beyond the people they already know would find it useful.

I know a little more than I did before I sent the book.

Check my work.