The Assay

Avoiding Fool's Gold

The Athanor is now in transition to a practice as much as a publication. We work with founder-led organisations, individual leaders and small groups on the thinking that has to be done as we reach the edges of the maps we have been using, and enter territory defined by febrile geopolitics, changing climate and a technology that we do not yet trust.

For those who want to go to source, I have included links at the end of the post

The Road Ahead

At the end of last week, Indy Johar published a provocation called What If the Future Cannot and Shouldn't Be Controlled? His claim is that the central failure of our systems is a wrong theory of change. The modern world still works from a control-based imagination of progress: futures designed from above, intelligence gathered near the centre, and people and places expected to adapt to plans made elsewhere.

Complexity defeats this, he argues; no centre can sense the whole. His alternative is distributed agency, resting on four conditions (capacity, capability, intention, imagination), with justice reframed as the fair distribution of future-making power, and sovereignty exploded outward to wherever reality is made and remade.

"Growth is not simply the enlargement of the system. Growth is the deepening of agency within it."

His essay is welcome company here. It is our October diagnosis conducted at the scale of the polity: the architecture is the problem, control is a spent instrument, and what matters now is that the potential capability of the people inside a system is accelerating faster than their employer's. His pairing of capacity and capability will be familiar to readers here.

Sitting with it a while, though, a gap opens: the context in which we work. Johar's essay is about conditions; it can say what a society must distribute, but not how an individual, carrying real expectations, in the middle of a working life, becomes someone able to use that agency once it arrives. In a volatile system, agency is sometimes thrust upon us whether we are ready or not. Distribution can be legislated; our individual and community deepening cannot. That happens slowly in small groups as we feel our way forward, with opportunities to fail in private before we exercise it in public. Johar's essay is a powerful call, with a logic that requires work it does not yet describe.

For the Athanor, that work is a practice we have spent ten months feeling our way into.

The sealed vessel

The vas hermeticum was the alchemists' vessel, a sealed container inside which change took place out of sight. Our vessel is AI. The practice has been learning how to think with Claude, to draw on its capabilities without surrendering agency; to keep it as emissary, and not allow it to aspire to becoming master when it comes to our thinking. That has been true since the first experiment, as the spring's posts recorded what we did and learned because, in the words of Stephen Covey, we needed first to understand, before seeking to be understood. Working with these systems is alchemical, in the strict sense that we work with a technology whose inner nature we cannot observe.

For a couple of decades during my career, I spent a lot of time working in cultures that were strange to me, in languages I had limited capability in. The lesson learned was not to let ego get in the way; the gap between what we think we have said and what we assume has been understood is often a chasm that is easy to fall into. Spend money on the best translator you can find.

Whatever language you use to engage with a large language or reasoning model, it is working with us, in effect, in a second language. We can now say this with more confidence, because researchers have been trying to look inside. They traced prompts through the layers of a large model as the words ceased to be words in any language; the middle of the model works in an abstract geometry of concepts, and only in the final layers is the result dressed again in the language of the question. Ask in French and French is applied at the end, the way a translator drafts in thought and renders in speech. Whatever language the machine speaks to us in, it speaks it as a second language, translated out of a place that has no mother tongue at all. Every exchange is a translation twice over, and important parts of what generated the intent of our question can get lost in translation. We choose what goes into the vessel; we read what comes out; we caanot watch the working between.

The old alchemists knew this predicament in their bones. The vessel was sealed; and the transformation happened out of sight; all she or he possessed were their inputs, their tools, and whatever emerged when the vessel was opened.

From that position the old craft built two disciplines we can learn from.

The first was the preparation of the materia prima: the endless washing and grinding of what went in, on the understanding that the work could be no better than its beginnings, and AI turns out to be more sensitive than its fluent confidence would suggest. The robustness research keeps finding that changes we pass over in our haste; formatting, punctuation, the order of examples, swing results by margins that should embarrass everyone involved; a rephrasing that preserves every word of our meaning can flip the answer. Clarity of input is a material condition of the work, and a slow hour spent preparing it is where a good deal of the outcome is decided.

When it comes to AI, the thinking we do before we start matters a great deal.

The second was the assay. The touchstone, the cupel, the acids that part gold from what merely gleams. An entire craft built on the hard-won knowledge that outputs lie, and that the ones which lie best look most like gold, especially when gold was what was being hoped for. The translation across time is exact, because it has now been shown, formally, that fluent falsehood cannot be engineered out of these systems; whatever the architecture or the training, they will sometimes produce confident error, and it arrives in the same polished costume as the truth. If we measure the quality of the output by its fluency, and nothing else, we are exposing ourselves to a real risk of Fools Gold.

Constructive scepticism

Scepticism, in this practice, is not suspicion of the tool but the stance of an assayer toward any material we have brought out from an AI.

We need to think as critically about what we accept from an AI as we do about what it we send it, because the AI will always assume there is value present in what it offers. If we challenge what it offers, it will often apologise and offer another version, because it is designed to flatter and please. That does not diminish its astonishing capabilities, but it is wise to treat it as a political animal, and make sure we have a strong enough understanding of what we ask of it to be able to appraise it.

It distinguishes the assayer's stance from the two easier positions on offer: refusal to engage and learn, which gives us temporary respite from the work of engaging with something that will not go away, and credulity, which abjures agency and delegates responsibility to others. Both are ways of avoiding judgement, which is the one thing the situation will not let us outsource. This is Pye's workmanship of risk in an unfamiliar setting: the quality of the results we get depends on the judgement we exercise at every moment in working with AI, and the moment of the assay most of all. If we are not confident of the quality of what went in, we are vulnerable to what comes out. As those of us around in the early days of programming remember: GIGO. Garbage in, garbage out.

One more thing. AI does not deal in meaning, or mētis. The relation between an expression we use and what we intend by it, the purpose a piece of work serves, the person it is for; none of that survives the prompt, because none of it was ever in the training material. AI supplies structure and unexpected adjacency, drawn from an inhuman geography of everything we have said to one another. Responsibility for who the work we produce is for, and who it will affect, remains ours.

These models are the most widely distributed instrument of thought since print, and distribution is what this argument calls for. I am working on the basis that using AI in creative pursuits, including leadership, strategy, and anything that involves thinking more than process, is a craft.

AI will deepen agency only in those who have learned how to use it. For those who prepare what goes in and assay what comes out, the vessel extends their reach; for someone who takes fluency for truth, it produces dependency wearing the costume of capability. The difference lies wholly in craft. If the future is to be made by many, as Johar wants and I do too, then the assayer's disciplines are part of the justice he describes, and they are learned the way craft has always been learned: beside someone, on real material, with the possibility of failure kept constantly in view.

And so,

Going forward: the writing continues, and will keep it open as public inquiry and observation, with additional material for paid subscribers.

Alongside that, the practice is developing at the speed of trust: individuals and small groups, working on their own questions in their own contexts, with the unlike mind that is AI as third participant, and the assayer's craft as the discipline. Selecting what we put in with care, testing what comes out before we build on it. Judgement always ours.

What we cannot yet say is what fluency with AI does to the one acquiring it; whether structure without meaning proves a poor substitute for thought or, handled with the right scepticism, something useful, an inexhaustible supply of new material on which our own intent can be worked. The alchemists laboured for centuries ahead of the theories that would explain their material, and their disciplines were what kept their integrity in the face of scepticism.

Using AI is an inexact science and emergent craft. We are all Alchemists now.

Sources

For readers who would rather test the assertions than take them on trust. Each entry notes what it carries in the piece and how hard it can be leaned on.

Indy Johar, "What If the Future Cannot and Shouldn't Be Controlled?" (5 July 2026). The provocation the opening section engages. The quotations are verbatim; "exploded sovereignty" and the four conditions of agency (capacity, capability, intention, imagination) are his terms.

Wendler, Veselovsky, Monea and West, "Do Llamas Work in English? On the Latent Language of Multilingual Transformers" (ACL 2024). The traced prompt. Intermediate layers of a large model work in an abstract concept space, with the language of the question applied only in the final layers, and the space leans toward English semantics. Peer-reviewed at a top venue; the finding is on one model family, though later work has broadly confirmed it.

Wu et al., "The Semantic Hub Hypothesis" (ICLR 2025). Extends the shared inner space across languages and modalities (code, arithmetic, vision, audio), and shows that a nudge applied inside the space moves the output in every surface form, so the space is used rather than incidental.

One caution belongs here rather than in a footnote: "no mother tongue" is our reading of what these two papers describe, not a claim either of them makes. The evidence shows a shared non-linguistic representation space with an English tilt; the interpretation is ours to defend.

Bender and Koller, "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data" (ACL 2020). The argument that a system trained only on the form of language cannot recover what a speaker intends by it; this carries the claim that meaning, and mētis, stay on our side of the table. It is the careful version of the argument for which "stochastic parrots" is the weaker slogan. The philosophical dispute remains live; the interpretability work above complicates it without settling it.

Sclar, Choi, Tsvetkov and Suhr, "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design" (ICLR 2024). Formatting alone produced performance spreads of up to 76 accuracy points in few-shot settings, and the sensitivity survives model scale and instruction tuning. Later robustness work (for example arxiv.org/abs/2504.06969) confirms that meaning-preserving rephrasings can flip answers. The direction is consistent across studies; the magnitudes are not.

Xu, Jain and Kankanhalli, "Hallucination is Inevitable: An Innate Limitation of Large Language Models" (2024). The formal result behind the claim that fluent falsehood cannot be engineered out of these systems, whatever the architecture, training or prompting. The paper's definition of hallucination is broad; the safe practical reading is that confident error cannot be eliminated entirely, only managed. OpenAI's "Why Language Models Hallucinate" (2025) reaches a compatible conclusion by a statistical route.

David Pye, The Nature and Art of Workmanship (Cambridge, 1968). The workmanship of risk: quality that depends on judgement exercised while the work is in motion, as against the workmanship of certainty, where the outcome is fixed before the work begins.