I’ve been vibe coding! That’s where I’ve been... or at least that’s what’s been eating up the afterhours I usually devote to writing this newsletter. Vibe coding and travelling: to Bangkok for the African–Asian University Press Collaborative Workshop, which I helped organise on the sidelines of the Charleston Asia conference, and then on to Taipei for my niece’s wedding and the Taipei International Book Exhibition. Back home to Singapore to celebrate Chinese New Year.
Best wishes to all of you for the Year of the Fire Horse.
Vibe coding is great, but I’ve not gone as far as business consultant and podcaster Azeem Azhar who talked about how his sleep hours (as measured on his Oura Ring sleep scores) have absolutely collapsed as he gets deeper into Claude Code.
Azeem mentions this in a very interesting discussion with Anthropic’s Head of Economics Peter McCrory, on trying to sort out just what the tools are being used for and what that might mean for the economy and employment. Azeem has been doing way more vibe coding than me, and is now fully deep into agents, including running his own instance of the quickly infamous OpenClaw. By this month, Azeem was using up to 100 million tokens in a single day, (“the equivalent of reading and writing roughly 75 million words”), for analysis and research tasks assigned to his agents, and to create software tools to act as dashboards, helping him prioritise and organise all that research. Creating an agent means setting up software to act as a kind of “harness” for LLMs, to allow them to complete high-level tasks, given high levels of autonomy, access to tools, software environments, databases, and so on. Set up a few to talk to each other, test each other. This is the agentic web.
In the podcast, Azeem describes how he wakes up every morning, fires up a number of parallel agents, goes for a morning walk and comes back to see what they’ve done. He shares his excitement and it is infectious.
Well, think of me as your “AI (and publishing) and copyright agent”, and now that you are back from your walk, let me tell you some things I’ve learned from my time in vibe code land, or a bit more dramatically, out on the jagged frontier.
From the jagged edges
First off, I mostly love it there. (Yes, there, not here. This is writing-with-my-own-brain land, a place which processes a lot fewer tokens!) With the help of Claude Code I can leverage my script-kiddy coding knowledge and half-assed web development skills creating new offerings in minutes or a few hours. I’ve used it to speed up development of static content websites mostly, but now I find myself writing more and more tools. A few of these offerings are actually in production, deployed on NUS Press’ open access reference site at epress.nus.edu.sg. I can tell myself that I have saved at least a few thousands in software development fees for my publishing house.
But it is more than pretty scenery here. If there is some joy and productivity gain going on, there are some perils lying in wait as well:
Beware the sharp blades and short-lived euphoria of what I like to call the Kanna Effect, after Will Manidis’ great essay. As he says there, the market for feeling productive is orders of magnitude larger than the market for actually being productive. I can spend hours thinking I’m vibe coding something great, and end down a dead end, with little in hand at the end of the session.
Even if you build up your immunity to that risk, you may still fall victim to the The Jevons Paradox: greater efficiency leads to more work! Jevons noticed that back in the industrial revolution, using coal more efficiently lead to a huge increase in, well, the use of coal. If these agents are so great, why is Azeem getting less sleep? What happened to fully automated luxury communism? Why have I suddenly discovered a whole long list of software I want to write?
Beware the terrain: the jagged frontier is treacherous. “Jagged frontier” is a concept popularised by Ethan Mollick, from a paper by him and colleagues suggesting that AI capability improvement is hard to map: LLMs perform surprisingly well on some complex tasks while struggling with seemingly simpler ones. And so the frontier of their abilities is “jagged”, discontinuous. You can spend a lot of time at a laggard frontier, get super-frustrated, and wonder if all those folks claiming great usefulness are just smoking the Kool-Aid.
And then there are the times when the jagged edge is so jagged it opens into a fractal gate of hell…
…like me with Microsoft Word
My experience trying to integrate AI tools into Microsoft Word was a visit to hell is one reason I tend to think that predictions of AI-powered economic takeoff are way-exaggerated. Adoption is not a question just of what a model can do, it’s a question of integration into a huge network of legacy business processes, old code and ways of thinking. Now I’m no expert in Word and text processing. I wandered foolishly and arrogantly into this challenge, thinking it might be a good stretch for my vibe coding skills.
But I had motivation! Four years have passed since I asked GPT-3 beta to rewrite a paragraph of gnarly text for me, and was astounded by the results. (The aha moment that led to this newsletter). Two-and-a-half years have passed since I gave Chat GPT the NUS Press proofreader test, which it passed with flying colors. I expected that by now there would be dozens of tools to choose from that would allow me to integrate LLM power into our editors’ use of MS Word...
You know, a plugin (or agent) that would allow me to describe NUS Press editorial style, run through my word document and quickly make edits using MS Word “track changes”, to bring the doc into conformance with our style. Why track changes? So editors can review the work, and then share the proposed edits with authors, who want to know every change we make to their manuscript. No rewriting to a different grade level, or in the style of Ernest Hemingway, just some help with style corrections: punctuation, spelling choices, should it be % or per cent or percent? And the holy grails —1 correcting the hyphens to n-dashes in number ranges and fixing bibliography style.
You’d think Microsoft would have crafted tools like this into MS Word long ago, soon after their OpenAI deal. Nope. We can’t even do this in Google Docs, though we get closer (and you end up sharing all your content with Google for further training so stay away from that!) And 99% of my author audience works in MS Word.
Surely one among you is an expert in MS Word, pandoc, automation of editing tasks and can just contact me and tell me that actually there is a single solution that I just missed? I don’t mind! I welcome this!
When Microsoft launched Copilot, and Singapore publishers were offered grants for adoption, publisher friends paid up only to learn that even Microsoft’s own Copilot couldn’t make those sorts of changes inside MS Word that they expected. It offered helpful advice on a window outside of the document and expected you to do some fancy cutting and pasting. A cut above the animated paper clip, but definitely not ideal. Microsoft had the user base, but not the advantage you would have expected in integrating AI into everyday workflows.
I paid up for third party plug-ins and web-based products that offered a similar proposition, including “conformance to Chicago”. None of them could quite get there, although we did get close in few instances. One of these products could make the style changes in a web version of a document, preserve the track changes when the web update was saved back to Word, but then lose all the footnotes along the way. So close...
Once I started really vibe coding, I thought, ok, maybe I’m the only one who wants this capability, I’ll just vibe code up an MS Word plugin. First I had to learn some basic lessons about vibe-coding. If a coding agent is like an intern, it can especially be like the intern in the video linked below. Please click and watch!
You have to learn how to guide the agent, ask questions, ask it to plan and consult before taking action. It can go so wrong, so quickly. It turns out to really help if you know something about the code you are vibing on …
I worked away on Replit, then Cline, then Claude Code, which promised so much more, including better integration with local files and Microsoft software. After trying these approaches I realised a couple of things:
First, how dramatically nasty is the code inside a .docx file. It’s just text and some formatting tags, right? All in XML? How bad can it be? Answer: Oh, so so bad…
Secondly, how so much of what chatbots do to fulfil a general task is actually to write good old deterministic software to do your tasks for you.
I’ll mostly spare you the details on the first point. Perhaps I shouldn’t have been surprised to see that the LLM’s strategy to make corrections inside a docx file was to write software. But I was. Write software? Boring old deterministic software?
Remember when Gary Marcus and others used to make fun of LLMs for not being able to do simple math? One response was tool use: training LLMs to recognize math calculations, extract the calculation and shunt it over to a software calculator, then incorporate the result back into the LLM reply. Much more reliable than a searching semantic space for the tokens to come after “345.56 x 6703.5609...”
Well this principle, of packaging LLMs with tools (including for good old information retrieval, ie RAG) has been carried to out to ever deeper levels, such that when I ask a chatbot to tell me if my Word doc was written in British or American or — more usually in my world — some unholy mix of the two, Claude will go off and (initially unbeknownst to me) look up lists of word-pairs on the web, create a script to search for those pairs, as well as “ise” vs “ize” patterns, analyse the results and give me a answer based on the resulting score. (I skipped over the software steps that Claude takes to root around in an MS Word doc... opening it up in python, stripping out the text and converting it to json of all things, and doing all its search and counting operations on the json file with python, before writing a long bespoke javascript file to convert the json to Word…)
Hey, if it works, well, that could be OK with me! If we can use software to solve style questions, why not? But as I tested the routines against new Word docs, I started seeing errors, oversights. Mostly because Claude had written its software routines based on the patterns it saw in my first document. [update Mar 1st: I should be clear here, mostly patterns in the MS Word tagged text, not the text itself as visible to the editorial eye. Specifically this means the relationship of “runs” to the changes I want to make.] I would need to go and dig deeper into the actual testing logic, and develop edge case tests. Some among you will chuckle knowingly…
So I started designing a workflow that combined diagnostics and corrections in software, spending time on all the details of each style rule and how it might be captured and fixed programmatically. At one point somewhere in here, late one night, I thought I should build into my software process a determination of which language the docx file used in its MS Word settings, surely that would be useful. No author would have their dictionary set to one language while spelling in another? Surely that info was found somewhere in the complex metadata of a docx file.
But even this simple lookup turned out to be quite difficult because every single “run” or span of text in a docx file (some as short as a single letter) could potentially have a different dictionary attached... if it was cut and pasted from another Word doc, for example. This is why it is not always so easy to make a global change to the language setting of a Word doc someone has sent you, even from within MS Word itself. What I was saying about MS Word complexity...? So Claude got geared up to try and develop a set of routines to parse all the different runs of text, identify the language settings at document, section, paragraph and run level, and find an algorithm to answer my simple(?) question. We got out just in time…
I spare you all the twists and turns. Here is the current endpoint of all my development efforts. I may or may not return to try and wrap this up. I’m hoping someone will write to me and point out the solution I missed…(by which I do not mean an editorial services offering… sorry guys!)
I’m also totally prepared that Gemini 3.2 or Claude 4.7 or an update on its MS Word skill will make my problems disappear. But then, I’ve been expecting that moment every day for three years at least…
Claude helped me write a series of MS Word macros that correct the style problems that can be reliably identified and fixed programmatically, ie using fancy search and replace, with maybe a tiny bit of logic. Great... except I’m still chasing edge cases... Also, some of you will note that this could have been done without an LLM. And many of you (and your outsourced edit houses) already do have nice helpful macros I am told! But I was always prevented from writing my own by the challenge of VBA (the language in which one write macros). So I have the option of delegating the VBA to Claude.
I settled for the lesser goal of writing reports on issues that can’t be decided in software. There are a lot! Here for example you find punctuation patterns — whether the full stop is outside the quotation mark or within. A regex search can’t automatically decide if it needs fixing against our guidelines, but they can list cases, maybe highlighting them in the document. This should help an editor as they run through the entire manuscript.
But long story short, most editors I’ve show the work in progress to are saying variations of “Thanks Peter I’ll look at that report but I still have to go through the whole file carefully anyway...” Which they do. Thank goodness.
So enough. On out on the jagged frontier, I was tempted into a cave, looking for “track changes” gold and the dream of a robot copy-editor. I found only a monster in the darkness. But I wrestled a bit and lived to tell the tale.
On emerging, blinking into the sun, looking down on the landscape below, I wondered how long had I been away. Was this some sort of Rip van Winkle time slip? I saw only devastation.
The end of the internet. The subject of my next newsletter...
That’s a 100% organic m-dash. House style at NUS Press, though we leave out the spaces either side. I find them better for online reading.