Why LLMs Get Stock Data Wrong: A Failure Taxonomy

Four failures, not one bug

"The AI got the stock wrong" is usually treated as a single problem called hallucination. In practice it is at least four problems that fail differently, get detected differently, and need different fixes. If you build agents that touch market data, separate them, because a mitigation that solves one can leave the others intact. A retrieval layer fixes staleness but does nothing for ticker confusion if retrieval itself is keyed on ambiguous symbols. A strict citation policy catches fabricated numbers but will happily cite a real filing for the wrong share class.

Here is the taxonomy:

The first three are data-modeling failures that would bite any system built on messy inputs; only the fourth is specific to generative models. Most public discussion focuses on the fourth, which is backwards: the silent identifier errors are often harder to catch, because the output contains a real number. It is just attached to the wrong thing.

The training cutoff: answering from a frozen market

A model's weights encode the world as of its training cutoff; nothing after that date exists for it unless a tool supplies it. For market data the problem is structural, because the disclosure pipeline itself runs on lags the model cannot see past. Institutional 13F holdings are due 45 days after each quarter ends; in 2026 the deadlines fall on February 17, May 15, August 14, and November 16, so the freshest possible picture of a fund's quarter-end book is already six weeks old on the day it becomes public (full calendar: 13F filing deadlines for 2026). Insider data moves much faster: a Form 4 is due within 2 business days of the trade, a Schedule 13D within 5 business days, a Form 3 within 10 days of becoming an insider, and a Form 5 within 45 days of fiscal year end.

A model trained even a few months ago has missed entire filing cycles, and it does not know it missed them. Ask about a fund's largest position and it answers from the last quarter it absorbed during training, in the same confident register as a fresh answer. The number was once true, which makes the failure invisible.

"Use a model with a later cutoff" is therefore a treadmill, not a fix: whatever the cutoff, Form 4 data goes stale within days of it. The structural answer is retrieval at question time, the core argument in our piece on whether ChatGPT can access live market data.

Ticker confusion: the symbol is not the security

Tickers feel like primary keys because humans use them that way. They are not. A ticker is a short, exchange-scoped, reusable label. Symbols get recycled after delistings, change in rebrandings, collide across exchanges, and differ between a foreign listing and its US depositary receipt. The internet text models train on is saturated with these ambiguities, so models inherit all of them.

The pattern predates AI entirely. In one well-known episode, traders bid up a dormant shell company whose ticker resembled the name of a then-popular video-conferencing firm that traded under a different symbol. Humans confused the label with the security; a model trained on that same discourse is at least as exposed. Academic and practitioner work on financial NLP has generally found that entity resolution, mapping a name or symbol in raw text to the actual legal security, is one of the most error-prone steps in the pipeline.

The fix is to stop using the ticker as the join key. US securities carry a CUSIP, the identifier 13F filers must report holdings under, and SEC filers carry a CIK in EDGAR. (Our explainer on what a CUSIP is covers the identifier stack in depth.) When data is keyed on CUSIP and CIK, a renamed company keeps its history and a recycled ticker cannot silently splice two unrelated companies together. Arkolith's stock pages and API resolve through these identifiers first, so an agent asking about "NVDA" gets the security, not the string.

Share-class merging: one issuer, several securities

Many large issuers list more than one share class. Alphabet trades as both GOOGL (Class A, voting) and GOOG (Class C, non-voting). Berkshire Hathaway's Class A and Class B shares trade at wildly different per-share prices because each represents a different economic slice of the company. To a clean data model these are separate securities: separate CUSIPs, prices, share counts, and filing rows. To an LLM trained on prose, they are mostly the same word: "Google stock," "Berkshire shares."

The result is quiet blending. Ask a model for a per-share figure and you can get Class A's price paired with Class B's share count, or an ownership percentage computed against the wrong class's float. Punctuation chaos makes it worse: the same Class B share appears in text as BRK.B, BRK-B, and BRK/B depending on the venue, and a model may treat those as three things or merge them into the Class A line.

13F data is unforgiving here because filers report holdings per CUSIP. A fund that owns both classes shows two distinct rows, and naive aggregation by company name either double-counts the position or drops one class entirely. Class-level rows are handled correctly on Berkshire's fund page, which is assembled from CUSIP-keyed positions rather than name matching. If your data layer does not distinguish share classes, no downstream prompt engineering will repair the answer; the error happened before generation started.

Fabricated numbers: fluency without retrieval

The most discussed failure is the model simply inventing a figure: a holdings value, a share count, a filing date, sometimes a fake accession number formatted exactly like a real one. This is not deception in any intentional sense; it is what a next-token predictor does when the prompt demands a number and no retrieval path exists. The output is optimized to look right, with plausible magnitude, correct units, and confident framing, which is precisely what makes it dangerous: a wrong number that passes the eyeball test is worse than no number at all.

We covered the mechanics and the grounding fix in How to Stop Your AI from Hallucinating Numbers, so two taxonomy points suffice here. First, fabrication compounds with the other three failures: a model with stale training and fuzzy entity boundaries fabricates more, because it has more gaps to paper over. Second, fabrication includes blending, where each component is individually real but the combination is not: one quarter's position value divided by a different quarter's share count, for example. Spot checks that verify single numbers miss blended ones; only source-level citation, where every figure traces to one specific filing, catches both. Even the primary data has documented limits, which is its own topic: see how accurate 13F data really is.

What actually fixes it: identifiers, tools, provenance

Map each failure to its fix and a pattern emerges: every fix moves work out of the model and into the data layer. Staleness needs retrieval at answer time. Ticker confusion needs CUSIP and CIK resolution. Share-class merging needs per-security keys. Fabrication needs provenance the agent is required to cite.

That is the design behind Arkolith's API and MCP server. The dataset is built from primary SEC filings: for Q1 2026 that means 1,824 institutional filers, 1.87 million long positions worth $53.7 trillion in reported value, plus more than 51,000 tracked insider transactions, with every row carrying the EDGAR accession number it came from. An agent resolves the entity first:

curl -H "Authorization: Bearer YOUR_KEY" \
  "https://arkolith.com/api/v1/search?q=berkshire"

then pulls CUSIP-keyed holdings for the resolved CIK:

curl -H "Authorization: Bearer YOUR_KEY" \
  "https://arkolith.com/api/v1/funds/1067983/holdings"

The same surface is exposed over MCP, so Claude or another agent calls these as tools instead of hand-writing HTTP; setup takes a few minutes via the quickstart. None of this makes the model smarter; it makes the model's job smaller, which is the point: the model handles language and reasoning, while identifiers, freshness, and citations come from a layer built for them.

To verify the primary sources yourself, the SEC's Form 13F FAQ covers what managers above the $100 million threshold must report, and EDGAR full-text search serves the original filings any sourced number should trace to.

Frequently asked questions about why LLMs get stock data wrong

Can a model with a newer training cutoff fix stale stock answers?

Only briefly. Form 4 filings land within 2 business days of trades and 13Fs arrive 45 days after quarter end, so any frozen snapshot decays immediately. A later cutoff shrinks the gap on release day, then the treadmill resumes. Retrieval at answer time is the structural fix.

Why do LLMs mix up GOOG and GOOGL?

Training text mostly says "Google" or "Alphabet" without distinguishing the Class A and Class C lines, so the model learns the symbols as near-synonyms. They are separate securities with separate CUSIPs, prices, and share counts. Any per-share or float-based figure has to be computed against one specific class.

Are tickers reliable identifiers for financial data pipelines?

No. Tickers are recycled after delistings, change in rebrandings, and collide across exchanges. Pipelines should resolve to stable identifiers, CUSIP for securities and CIK for filers, and treat the ticker as a display label.

How does provenance reduce hallucinated market data?

Provenance attaches a verifiable origin, such as an SEC EDGAR accession number, to every figure the agent receives. The agent can then be instructed to state only numbers it can cite, and a human can audit any claim back to the primary filing. Fabricated and blended numbers fail that check because no single source produces them.

This article explains public filings and data concepts. It is not investment advice.