That AI Temptation

I continue low-intensity work on Quamina; among other things, it benefits my my mental health. At the moment there’s this little voice at the back of my brain that keeps saying “This would be a great job for Claude.” So far I’m ignoring it, but the internal conversation is worth sharing.

Disclosure · I’m not a GenAI expert, have never relied on it for any substantial piece of work. However, I’ve seen plenty of PRs with LLM fingerprints, and I follow a whole lot of indie devs who are in it up to their elbows. It seems to me that a pretty clear picture emerges of what it can and can’t do, where the wins and losses are. This essay reflects that picture but maybe I’m just wrong, in which case I’ll confess my errors.

Negative externalities · For me (and I’m not alone), the biggest issue isn’t what GenAI can and can’t do, it’s the ethical and economic discomforts. This technology is being peddled by terrible people who are rich and whose goal is to get richer. It’s bought and deployed by terrible people who want to slash their payrolls by throwing thought workers onto the street. It facilitates the oncoming climate catastrophe and devalues human creative effort.

Notice that I said “ethical and economic”? That second issue is all about the bubble and how nobody knows what GenAI technology will really cost, post-explosive-decompression. It wouldn’t surprise me if we ended up in a future where most software devs are using beefy “desktop” computers running open-weight models; or else they’re using AWS/GCloud/Azure (but still open-weight). Or something completely different. Or we’ve learned that the cost of AI, as measured in mental fitness and organizational health, is simply too high.

The stage has been set; let’s talk about bits on the wire.

Quamina and JSON and … · At the moment Quamina does its super-fast-matching magic on JSON blobs. But it should be perfectly possible to work with any other data format that has nested fields and boolean/numeric/string values at the leaves. Oh, and arrays of the above. There’s no shortage of candidates: Thrift, Protobufs, Apache Avro, and CBOR come to mind immediately.

Quamina has the built-in notion of a “Flattener”, that turns input data into a sorted list of “Field” structures. For example, here’s a JSON blob:

{
  "X: {
    "Z": 22,
    "Y": 21,
  },    
  "A": 33
}

The output of the flattener, what the matcher works on, would be something like this:

{"A": 33}, {"X\nY": 21}, {"X\nZ": 22}

In Quamina, flattening eats more time than matching. It used to eat dramatically more time, but the JSON flattener has been heavily optimized by me and other contributors.

Maybe the most important optimization is maintaining a list of which fields are actually up for matching and skipping over the ones that aren’t. Obviously this means that the parser can’t be the typical thing that turns your JSON blob into a fully-serialized tree of structures and arrays in a one-shot call. In fact, you need a parser that’s a tokenizer and lets the calling code decide how to deal with each little piece of it, one at a time.

Protobufs · Someone out there asked Claude to build a Flattener for Protobufs. It seemed to partly work but was kind of buggy and I found the code hard to read. Hmm. Of course the usual GenAI “That was six months ago, it’s much better now” narrative applies.

CBOR · For those who don’t know, CBOR is JSON but binary, which is claimed to make it more compact and faster to process. It’s also got a slightly richer type repertoire, but whenever I’ve run across it in the field, it’s basically being used as JSON-only-faster.

So, I’ve started working on a CBOR Flattener for Quamina.

The simplest thing that could possibly work · The default CBOR processor for Go, from Faye Amacker, looks very polished. But it’s not a tokenizer, it’s a “deserialize your whole blob in one call” thing. So the simplest thing would be to make that one call on each incoming blob, then run around the deserialized tree and compute the flattening. Ignore all that stuff about bypassing for efficiency. Because it might be plenty fast enough and that optimization would have been premature. After all, if you read Faye’s page linked above, they seem to be something of an optimization fanatic.

And you know what, I bet if I gave Claude the problem of turning CBOR into Fields, I’d get OK code back; tell it find a way to fuzz-test the process and it’d be surprising if it didn’t work.

Right at this moment in history, I could make a case for that being a good engineering decision. Code, in the LLM-pre-bubble-pop era, is cheap. Time is still expensive. This task has clear, clean success criteria, easy to point an agent at. The Simplest Thing That Could Possibly Work is a powerful design principle, one that I’ve forced down the throat of more than just one or two over-aspirational engineering team leads.

But I’m not gonna do that. I mean, there’s the nauseating state of current AI culture and finance. But also, I’m doing this for fun. Writing high-performance tokenizers is fun! [Really? -Ed.][Serious fun. -T.] I’ve started poking around inside Faye’s library, raised an issue, and I’m confident that there is plenty of code and tests in there that I can steal.

Testing · Quamina has 14K lines of test code and 47M of of test data. A high proportion of the tests amount to “Run the code against this JSON and require these results.” Obviously, now I want to run 100% of those tests with equivalent CBOR input. It seems to me that Claude could do a great job of automating plenty of this work, which I predict will be grinding, tedious, and not fun at all. On top of which I tend to be lax and forgiving about test-code quality.

My internal engineering-leader mind has no doubt, it’d tell the devs to automate the hell out of replicating the test suite and not be picky about how. But I bet that it’d burn a whole lot of tokens, which means routing money to bad people with bad goals. So once again, I’m not going to do it with LLMs.

Post-bubble, it might be somewhere between forgivable and praiseworthy. But not as I write this.