What Is LLM Tool Calling? How It Actually Works

The loop: schema in, call out, result back

Tool calling is not magic, and it is not the model "running code". It is a tightly scripted exchange between three parties: the developer, the model, and a runtime that actually executes things.

The sequence runs like this. First, you send the model a list of tool definitions alongside the user's prompt. Each definition has a name, a natural-language description, and a JSON schema for its parameters. Second, when the model decides a tool would help, it stops generating prose and emits a structured block: the tool name plus a JSON object of arguments that, ideally, validates against your schema. Third, your runtime (your application, an agent framework, or an MCP client) executes the real work: an HTTP request, a database query, a calculation. The model never touches the network itself. Fourth, the runtime appends the result to the conversation, and the model continues, now able to quote real numbers instead of reconstructing them from training data.

Modern models are trained specifically for this format, which is why argument quality has improved so much. The model learns to read a schema roughly the way a developer reads an API reference. And the loop can repeat: an agent might call a search tool to resolve a fund's identifier, then a holdings tool with the identifier it just learned, then a third tool to cross-check a position. Multi-step chains like that are what people actually mean by "agentic" behavior: a loop of tool calls with reasoning between them, not a single oracle answer.

Tool schemas are the contract

A tool schema does two jobs at once. It is a validation contract, so the runtime can reject malformed arguments before anything executes. And it is documentation the model actually reads at answer time. A typical definition looks like this:

{
  "name": "get_fund_holdings",
  "description": "Return the latest reported 13F long positions for one institutional filer, identified by SEC CIK. Values are as reported to the SEC, with the source accession number attached to every row.",
  "input_schema": {
    "type": "object",
    "properties": {
      "cik": {
        "type": "string",
        "description": "SEC Central Index Key, e.g. 1067983 for Berkshire Hathaway. Resolve via the search tool if unknown."
      },
      "limit": { "type": "integer", "minimum": 1, "maximum": 500 }
    },
    "required": ["cik"]
  }
}

Everything in that block is prompt engineering in disguise. The description tells the model when to reach for the tool and what the data means. Field descriptions stop it from guessing identifier formats. Constraints like enums, minimums, and required fields cut off whole classes of bad calls before they run. In practice, the description is where most tool-calling quality is won or lost: a vague "gets fund data" produces vague calls, while a description that names the identifier system and the data's limitations produces calls a careful engineer would write.

There is a sizing discipline too. Every tool you expose consumes context and adds a wrong-choice opportunity, so a focused set of well-described tools usually outperforms an exhaustive one. That is why Arkolith keeps its MCP tool surface deliberately small and pushes complexity into parameters; the quickstart shows the live tool list and how an agent discovers it.

Why tool calling beats RAG for live structured data

RAG and tool calling solve different problems, and conflating them costs accuracy. RAG retrieves chunks of text that are semantically similar to a query and hopes the answer sits inside one of them. That works well for unstructured prose: policies, research notes, documentation. It works badly for structured, numeric, time-sensitive data, where "semantically similar" is not the same thing as "the correct row".

Filing data makes the gap concrete. 13F filings arrive on a hard calendar: 45 days after quarter end, which in 2026 means February 17, May 15, August 14, and November 16. A vector index built in April simply does not contain the Q1 wave that lands on May 15. A tool call against a live API does, the same day it is ingested. Scale matters as well: Arkolith's Q1 2026 dataset covers 1,824 institutional filers and 1.87 million long positions worth $53.7 trillion as reported. Embedding all of that as text and hoping cosine similarity surfaces the right position is the wrong instrument. A parameterized query ("holdings for this CIK, sorted by value") is exact by construction, and each returned row carries its EDGAR accession number, the grounding approach we unpack in how to stop AI hallucinating market data.

What this looks like in practice

There are two common transports for the same loop. Under an MCP connection, the agent discovers Arkolith's tools and calls them directly. Under REST, your own code is the runtime that executes whatever call the model requested. Same data, different caller, a distinction we cover in MCP vs REST API. Either way, the executed requests look like this:

# Resolve an entity the model only knows by name
curl -H "Authorization: Bearer YOUR_KEY" "https://arkolith.com/api/v1/search?q=berkshire+hathaway"

# Browse the institutional filer universe
curl -H "Authorization: Bearer YOUR_KEY" "https://arkolith.com/api/v1/funds"

# Pull holdings for the CIK the search step returned
curl -H "Authorization: Bearer YOUR_KEY" "https://arkolith.com/api/v1/funds/1067983/holdings"

Note the shape: search first, then fetch by identifier. This two-step pattern is the single most useful tool-calling design rule for financial data, because identifiers (CIKs, CUSIPs, accession numbers) are exactly the strings models hallucinate most confidently. Give the model a search tool and it never has to invent one. The output of the search becomes a verified argument for the next call, and every figure in the final answer can be traced to a real filing in EDGAR full-text search.

Two practical details round this out. Metering: each call debits prepaid credits, so a multi-step agent loop has a visible, per-call cost rather than a surprise bill. And verifiability for humans: the same data the agent queried is browsable on pages like NVDA's ownership page, so a user can check what their agent just reported. Wiring an agent like Claude to all of this is a one-line MCP connection, walked through in connecting market data to Claude.

Where tool calling still fails

Tool calling moves the failure surface; it does not remove it. Four classes show up constantly in production.

First, argument hallucination. The model picks the right tool but fabricates an identifier: a CIK that does not exist, a plausible-looking CUSIP. Schema validation catches the malformed ones; the well-formed-but-wrong ones need the search-first pattern above, plus APIs that fail loudly on unknown identifiers instead of returning a best-effort guess.

Second, call discipline. Models sometimes answer from memory when they should have called, presenting stale data confidently, and sometimes call five times when once would do, burning latency and credits. Tool descriptions that explicitly say "always use this for current holdings; training data is stale" measurably shift behavior. A model with no tools at all cannot answer these questions well at any temperature, as we show in can ChatGPT access live market data.

Third, the data has semantics the model must respect. A 13F is a quarterly snapshot filed up to 45 days late by managers above a $100M threshold in covered US equities (investor.gov has a plain-English overview), while insider trades on Form 4 arrive within two business days. A tool result is only as current as the disclosure rules allow, a nuance examined in how accurate is 13F data. Arkolith tracks 51,000+ insider transactions precisely because the fast Form 4 stream complements the slow 13F one.

Fourth, treat tool output as untrusted input. Results are injected back into the model's context, so a source that returns junk, or worse, embedded instructions, corrupts everything downstream. Per-datapoint provenance exists so both the agent and the human can audit the chain instead of trusting it.

Frequently asked questions about LLM tool calling

Is tool calling the same as function calling?

Effectively yes. "Function calling" was the early vendor term; "tool calling" or "tool use" is the broader umbrella that now covers functions, MCP tools, and computer-use actions. The mechanism is identical: the model emits a structured request, and a runtime executes it.

Does tool calling replace RAG entirely?

No. RAG remains the right approach for large bodies of unstructured text where semantic similarity is the actual goal. Tool calling wins when the data is structured, live, and queryable by precise parameters, and many production agents use both, sometimes exposing the retrieval step itself as a tool.

How does MCP relate to tool calling?

MCP (Model Context Protocol) is a standard transport for tool calling: a server publishes tool schemas that any compatible client, such as Claude, can discover and call without custom glue code. Instead of hand-wiring definitions into every application, you connect one server and the agent gets the whole toolset. We cover the protocol in what is an MCP server.

Can the model execute the tool itself?

No, and this is a security feature rather than a limitation. The model only emits a request; your runtime executes it with its own credentials, permissions, rate limits, and metering. API keys never enter the model's context, and a bad call can be rejected before it touches anything real.

This article explains public filings and data concepts. It is not investment advice.