It's Not a Function — A Statistical Account of Why AI Doesn't Repeat Itself
Three SSL renewals across three sites yielded three different procedural recipes from the same model. Why this isn't a bug but a property of the architecture — examined through probability, information theory, and the numerics of GPU matmuls.
a
aiiqlabs academy
·10 min read
I host three small sites. Earlier this week, all three SSL certificates needed renewing within the same hour, and I did what one does in 2026: I opened three tabs of the same model, pasted the same one-line description of the task — "renew the SSL on this Hostinger-backed Nginx VPS, here are the credentials" — and let three independent sessions go to work.
Three sessions. One model. Identical prompts. Three completely different procedural flows.
The first session insisted I run certbot --nginx interactively on the box and then walked me through an eight-step verification dance. The second session generated a CSR locally, asked me to paste the certificate body back into the chat, and proposed I drop it into /etc/ssl/certs/somecert.cert. The third session — same model, same week — used acme.sh, named the file somecert.pem, and configured a cron job for auto-renewal that the other two never mentioned. All three jobs ended in working HTTPS. None of the three paths agreed on anything else: not the tool, not the file extension, not even who was doing what.
If your first instinct is "the AI is unreliable," you have the wrong mental model. The AI is doing exactly what it was built to do. The model your instinct came from — that the same input should yield the same output — is borrowed from classical software, and it does not survive contact with the actual mathematical object you are talking to.
This article is about what that object actually is.
It is not a bug. It is the architecture. Asking an LLM to repeat itself is asking a probability distribution to behave like a function.
§1 — Definition: The Function That Wasn't
In undergraduate mathematics, a function $f : X \to Y$ is a relation that assigns to each $x \in X$ exactly one $y \in Y$. Single-valued. Deterministic. The same input maps to the same output, every time, by definition.
When you wrote your first line of code — print("hello") — and got hello back, every time, you were operating inside this paradigm. Forty years of software engineering culture (testing, CI, "if it worked yesterday it will work today") rests on it. Bugs, in classical software, are deviations from this single-valuedness; they are violations of the function contract.
A modern LLM is not a function in this sense. It is a conditional probability distribution over output sequences:
where $\mathcal{V}$ is the vocabulary, $\theta$ are the model parameters (weights), and the symbol $\sim$ denotes that $y$ is drawn from the distribution, not equal to it. What you receive when you "call the model" is not $p_\theta$ itself. It is a single sample. A draw. One realisation of a random variable whose entropy
The model does not return an answer. It returns a sample. The distinction is the entire article.
The SSL story is, in this light, banal. Three samples from $p_\theta(\cdot \mid \text{"renew SSL on Nginx VPS"})$ landed at three different but high-probability points of a high-entropy posterior. They had to. The posterior contains every plausible procedural flow weighted by training-set frequency and reinforcement-learning preference; sampling picks one each time.
You do not have a contract with $f$. You have a contract with $p$.
§2 — The Distribution
To sharpen the picture, expand the joint probability autoregressively:
At each timestep $t$, the model emits a logit vector $z_t \in \mathbb{R}^{|\mathcal{V}|}$ (the pre-softmax activations of the final linear layer), which is converted into a categorical distribution via the temperature-scaled softmax:
$$p_\theta(y_t = v \mid x, y_{
The decoding algorithm then samples — or in some special cases picks the argmax of — this distribution to produce $y_t$. The chosen token is appended, the model conditions on it, the next $z_{t+1}$ is computed, and the loop runs to completion.
Two structural facts follow.
Fact 1: The output is a trajectory, not a point. Each generation is a path through a tree where every node is a probability simplex over $\mathcal{V}$. The tree has branching factor $|\mathcal{V}| \approx 10^5$. Two trajectories that diverge at $t=3$ may rejoin in semantic content but never in literal token sequence — the cardinality of the leaf set is, for any non-trivial $n$, astronomical.
Fact 2: Divergence compounds. Suppose two runs produce identical tokens for the first $k$ steps. The probability that they continue to agree at step $k+1$ is $\sum_v p(v)^2$ — the Rényi entropy exponential, which for a typical mid-stream distribution is well below $1$. Disagreement at any single step rewrites the conditioning context for every step that follows. This is, in dynamical-systems terminology, sensitive dependence on initial conditions: a one-token perturbation in early position will, with very high probability, produce a globally different trajectory.
The model has not "made a different choice." It has walked a different path through the same forest. Both paths were always there.
§3 — Decoding (And the Temperature Myth)
The temperature parameter $T$ in the softmax is the most-misunderstood knob in the LLM API. In the limit $T \to 0$, the distribution collapses onto the argmax:
It is therefore widely — and, as we will see in §4, incorrectly — believed that setting $T = 0$ ("greedy decoding") yields a deterministic model. The reasoning runs: if at every step we pick the unique highest-logit token, and the logits are a deterministic function of the inputs, then the output sequence must also be deterministic.
This argument has two leaks.
Leak A: the argmax is not always unique. When two logits are equal, $\arg\max$ is ill-defined; the convention is set by the underlying tensor library (PyTorch, JAX, cuDNN), and that convention can depend on memory layout, batch size, or kernel version. Empirically, near-ties (within machine epsilon) at production scale are common — far more common than naive intuition suggests, because the softmax distribution is heavy-tailed and the argmax-runner-up gap is often $< 10^{-7}$.
Leak B: the logits are not, in practice, a deterministic function of the inputs. This is the deeper leak, and it is the entire subject of §4.
Beyond pure greedy and pure sampling, production systems use richer decoding regimes — top-$k$, top-$p$ (nucleus), beam search, contrastive decoding, speculative decoding — each of which restricts or transforms the distribution before drawing:
These transformations preserve — and in some cases concentrate — variance. None of them turn a sampler into a function.
"Deterministic mode" is a marketing artifact. The math admits no such mode.
§4 — Sources of Variance, Even at T = 0
Even if you set $T = 0$, deduplicate logits with deterministic tie-break, fix the random seed, pin every library version, and pray to the appropriate compute deity, you will still observe non-determinism in the output of a real production LLM. The reasons are mechanical and mostly invisible to the API consumer.
§4.1 — Floating-point non-associativity
IEEE 754 floating-point addition is not associative:
$$(a + b) + c \neq a + (b + c) \quad \text{in general}$$
A transformer's matmul is, at its core, a sum of products. The order of the partial sums is determined by the GPU's tile shape, reduction-tree depth, and warp-level scheduling — all of which are functions of the batch the request happens to land in. Two requests with identical inputs that are batched differently will compute logits whose values differ at the $\epsilon$ level. Pass those logits through softmax and argmax, and on near-tied tokens you get different outputs.
(Higham, Accuracy and Stability of Numerical Algorithms) — small, but non-zero, and aggregated over hundreds of layers. The argmax operation amplifies this error precisely at the points the model is least confident.
§4.2 — Mixed precision
In production, the activations are typically held in BF16 or FP16 (16-bit) and reductions are performed in FP32. Both reduce $\epsilon_{\text{mach}}$ by orders of magnitude relative to FP64. The geometry of "how close are the top two logits" is the geometry of where the model gets coerced into a discrete choice; reduce precision and you increase the rate at which that choice flips.
§4.3 — Mixture-of-Experts routing
Frontier production models are increasingly Mixture-of-Experts (MoE). For each token, a learned router $r_\phi$ selects (typically) the top-$k$ of $E$ experts:
Crucially, in modern load-balanced implementations (Switch, GShard, DeepSeek-MoE), the routing decision depends not only on the token but on the composition of the batch — to avoid sending every token to the same overloaded expert. Same prompt, different batch-mates, different expert assignment, different logits, different output.
This is not a bug. It is the cost-efficient serving regime. There is no commercial LLM at scale today that promises batch-invariant routing.
§4.4 — Speculative decoding
To accelerate inference, many production stacks use a small draft model to propose multiple tokens at once, then verify with the main model:
The acceptance pattern is stochastic by construction and depends on the draft model's confidence — which depends on the draft's batch, which depends on routing in the draft model. Variance compounds at every layer.
§4.5 — Hardware heterogeneity
A typical large-model serving fleet contains H100s and A100s and (still) some V100s and B200s and increasingly TPUs. Different generations have different reduction implementations, different precision defaults, different cuDNN/cuBLAS versions. Your request goes to whichever GPU the load balancer picks. The same prompt routed to a different generation can produce subtly different logits — and at the argmax boundary, "subtly different" cashes out as "different output."
Reproducibility is a property of the system, not of the model. To reproduce, you must fix the seed, the batch, the kernel, the version, the hardware, and the routing. Customers of public APIs control none of these.
§5 — Corollary: Engineering with a Sampler
If the theoretical content of §1–§4 is correct — and it is, modulo notation — then a competent engineering practice has to absorb three corollaries.
Corollary 1: Determinism is not the right metric. Stability is.
Treat the output of a model as $y \sim p_\theta(\cdot \mid x)$. Characterise its distribution, not its values. Useful proxies include:
where $\text{sim}$ is a domain-specific equivalence (embedding cosine, exact-match-after-canonicalisation, schema validity for structured output). For my SSL story, $\text{sim}(\text{"working HTTPS"}, \text{"working HTTPS"}) = 1$ regardless of which .cert vs .pem path got there. The distribution was stable on the right axis.
Corollary 2: Contracts must live above the text.
If you build a system that consumes LLM output, do not build it on the assumption that the output is a fixed string. Build it on a schema — JSON, a typed object, a constrained grammar. Validate. Re-prompt on schema failure. The text is the medium; the schema is the contract. Constrained decoding (e.g., FSM-guided generation, JSON-schema-aware sampling) is the strongest tool here, and it works precisely because it forces the sampler into a measure-zero region of $\mathcal{V}^*$ where the contract holds.
Corollary 3: Sample. Aggregate. Vote.
For decisions that matter, do not call the model once. Draw $N$ samples and aggregate:
or, in the literature on chain-of-thought, use self-consistency: sample $N$ reasoning paths, marginalise over the reasoning, and take the modal answer. The variance of $\hat y$ decays as $1/N$ for the bias of the underlying distribution. You buy correctness at the cost of latency and tokens — which is, often, the right trade.
You are not querying a function. You are estimating a distribution. Build accordingly.
A final note on the SSL story
Three sessions. Three procedures. Three working sites.
If any of those three sessions had failed at the SSL step, I would have called the model unreliable. Because all three succeeded — by different paths — I called it weird. Both reactions are wrong. The right reaction is the one this article has been arguing toward: the model did not give me three different answers. It gave me three samples from the same distribution, all of which happened to lie inside the manifold of correct procedures. That is precisely what a well-trained sampler is supposed to do.
The frustration is real. The expectation behind the frustration is, mathematically, a category error.
If your team is building on top of these models, your QA process needs to grow up with the math. Test the distribution, not the output. Constrain the contract, not the text. And when someone shows you three different solutions to the same problem, take a moment before you call the system broken — because in the only formal sense that applies, the system is doing exactly what it promised.
From theory to product.
Build with the math, not against it.
Live, instructor-led cohorts on AI engineering, Python for AI, and LLM APIs — including the production engineering needed to ship reliable systems on top of stochastic models.
● LIVE COHORTS● CERTIFICATE OF COMPLETION● PRIVATE DISCORD COMMUNITY● 1-ON-1 MENTORING
Disclaimer
This article uses standard notation from probability theory, information theory, numerical analysis, and the deep-learning literature. Where conventions differ across communities, I have chosen the most common form and named it explicitly. The math is simplified for narrative — production transformer architectures contain layer normalisation, residual connections, attention-with-causal-masking, and many more sources of compounded numerical error than fit in a single essay.
Statements about specific commercial systems' use of MoE routing, speculative decoding, and serving heterogeneity are based on publicly available technical reports, model cards, and architecture papers. Implementations evolve; specific claims should be checked against the relevant provider's current documentation before being relied upon for procurement or compliance decisions.
Nothing in this article is investment, legal, security, or operational advice. Examples — including the SSL renewal anecdote — are illustrative; do not generalise from a single workflow to a recommendation about your own production systems without independent review.
The author writes about AI engineering and adjacent topics; views expressed are personal and reflect the state of the field at the time of writing. The mathematics, however, is older and less likely to date.
