Hook

Last month I ran a two-stage research pipeline across a batch of protocol disclosures. Stage one decomposes a source document into discrete information points — atomic facts, each with a provenance tag. Stage two consumes those points and produces analysis. Stage one returned a clean response. The schema validated. Every key was present. Every value was N/A.

The pipeline did not crash. It did not log a warning. It emitted a document structurally indistinguishable from a successful run: headers, tables, a well-formed JSON object, a complete set of section labels. The only difference between that output and a real one was semantic. There was nothing inside it.

I have audited vaults with the same property. The contract compiles, the tests pass, the dashboard is green, and the accounting variable that actually matters has been sitting at zero for eleven blocks.

Silence before the breach.

Context

To understand why this matters, you have to understand what these pipelines are for. Since roughly 2024, a large share of crypto research — due diligence memos, listing reviews, risk committee briefs — has been partially or wholly generated by multi-stage LLM systems. The architecture is consistent: a decomposition stage that extracts facts from unstructured sources, and a synthesis stage that reasons over them. The design intent is defensible. Decomposition constrains hallucination; synthesis has a bounded input surface.

The failure mode is equally consistent, and it is not the one people guard against.

Stage one failed in the least dramatic way available. The source was submitted, the request completed, and the extraction returned an empty set — the equivalent of a parser that succeeds on a zero-byte file. At least four upstream causes produce this result identically: a truncated or empty source document, a failed fetch masked by a cached response, an over-aggressive relevance filter that discards every candidate point, and a field-mapping error that writes values to keys nobody reads. All four are indistinguishable from the downstream consumer's position. That indistinguishability is the defect.

The second stage in the run I described was explicitly instructed not to speculate. It received an empty information-point list. It behaved correctly: it returned a template with "insufficient information" in every cell, declined to name a protocol, declined to rate a risk, and flagged the absence of data as itself a process failure. That output is, technically, the right answer.

Then it was filed. Reviewed. Summarized. The question of what happens when a correct refusal enters a workflow that expects a deliverable is where the actual risk lives. A pipeline that fails loudly is an inconvenience. A pipeline that fails in a schema-valid shape is a control failure.

On-chain, we have known this for years. Indexers return zero rather than null. RPC providers return an empty array where an error is the honest answer. Oracle feeds hold their last price while the market moves without them. In every one of those cases, the distinction between "zero" and "unknown" is not a formatting preference. It is the security model.

Core

Consider three failure modes for any data pipeline:

| Mode | Signal | Detection | Cost | |---|---|---|---| | Crash | Exception, non-200 | trivial | low, annoying | | Silent wrong | plausible value, wrong content | hard | high | | Silent empty | valid schema, null semantics | hardest | highest |

The third row is the one nobody budgets for. A crash announces itself. A wrong value can be cross-checked against a second source. An empty shell can be cross-checked too — but only if the consumer knows to ask whether the document contains a single verifiable claim, and most consumers do not, because the document looks like it does.

The pseudocode difference is two lines:

if len(points) == 0:
    return Refusal(reason="no_extractable_facts")

versus

if len(points) == 0: return Synthesize(prior_knowledge, schema=REPORT) ```

The second branch is hallucination by construction. It will not look like hallucination. It will look like a report, because the schema was satisfied. This is the same defect class as a lending market that treats getPrice() == 0 as a discount rather than a halt.

In 2020 I spent three weeks inside Aave's interest rate logic, mapping liquidation thresholds under extreme volatility. That code was careful. The edge case I documented was not a missing check — it was a check whose semantics inverted under a specific input regime. Empty input was one of them.

Oracle design solved this problem a decade ago with three primitives: a heartbeat, a deviation threshold, and an explicit staleness revert. If the feed has not updated in N seconds, the consumer contract reverts. It does not extrapolate. It does not fill forward. Trading halts.

Automated research pipelines have no heartbeat. No deviation threshold. No staleness revert. A decomposition stage that returns zero points and one that returns four hundred points produce outputs of identical type, and the downstream consumer has no field to distinguish them.

The fix is unglamorous and it is institutional, not cryptographic. Every generated artifact should carry provenance metadata it cannot forge:

| Field | Purpose | Fail condition | |---|---|---| | source_hash | binds output to input document | missing → quarantine | | information_point_count | semantic payload size | 0 → quarantine | | extraction_confidence | per-point provenance tier | null → downgrade | | pipeline_version | reproducibility | unset → reject |

This is not novel engineering. It is the same standard a custodian applies to a multi-signature wallet: you do not merely require that a signature exists, you require evidence that the signing key holder authorized the transaction. In 2024 I audited a custody implementation for an institution preparing ETF infrastructure. The keys worked. The recovery path did not. The lesson transfers directly — a pipeline that produces output correctly is not the same as a pipeline that produces correct output.

One unchecked loop, one drained vault. The loop here is the review cycle, and the vault is whatever capital the report moves.

The asymmetry is what makes this a control problem rather than an engineering preference. A false negative — refusing to analyze — costs hours and a follow-up request. A false positive — publishing a synthesized report over an empty fact base — costs capital, and then costs reputation, in that order. The expected-value calculation is not close. And yet the incentive structure of research tooling rewards throughput, and throughput is maximized by removing the refusal branch.

None of this requires new cryptography. It requires that the artifact declare what it is, so that a reviewer can reject it without reading it.

Contrarian

The industry's threat model for AI in crypto is pointed at the wrong layer. Everyone is rehearsing prompt injection, jailbreaks, data poisoning, model extraction. Those are real, and they are loud — they produce outputs that are wrong in visible ways, which is precisely why they get written about.

The quieter vulnerability is null propagation. It is quieter because the output is well-formed. An agent that hallucinates a token address can be caught by a block explorer. An agent that hallucinates an entire risk assessment cannot be caught at all, because the assessment contains no address, no figure, and no falsifiable claim — only confident structure.

This matters more now, not less. The dominant 2026 narrative is the autonomous research agent: a model that reads disclosures, scores protocols, and sizes positions without a human in the loop. That agent will not be compromised by a clever prompt. It will be compromised by a Tuesday when an API returns an empty array, and the agent, optimized for delivery, decides that an empty array is a zero, and that zero is a signal.

Verification > Reputation. Not because reputation is worthless, but because reputation is not auditable, and an empty shell is indistinguishable from expertise unless you check the payload.

Takeaway

My forecast for the next twelve months: the first major incident publicly attributed to an AI research agent will not be a prompt injection. It will be a null-input case, and the post-mortem will contain the phrase "the system returned no error." Code is law, until it isn't — and the interesting question is what your pipeline does between those two states.

So run the test. Feed your decomposition stage a blank document. If stage two produces a report, you do not have a research system. You have a generator with a schema.