The Real Cost of an LLM Call vs a SQL Query

I keep meeting the same architecture decision dressed up in new clothes. A team has data sitting in Postgres, a question they could answer with a WHERE clause, and somewhere in the design review someone suggests sending it to a language model instead. The query would finish in about a millisecond. The model takes a second or more, and it bills you for the privilege.

The reflex is understandable. LLMs feel free in development, where you’re making ten calls a day and the invoice rounds to nothing. So the cost question never really comes up until the feature ships, traffic arrives, and someone in finance asks why the inference line grew faster than the user count.

Here’s the point I want to make, and where it stops. An LLM call and a SQL query aren’t interchangeable tools with slightly different price tags. They’re different orders of magnitude, in money and in time, and most of the time that gap should decide which one you reach for. The exception matters too, and I’ll get to exactly where the model earns its cost.

Read More : The boring tools already in your stack 

How much does a single LLM call actually cost?

A single LLM call usually costs somewhere between a tenth of a cent and a few cents, and a reasoning model can quietly push one answer past a dollar. The range is the whole story.

Run the arithmetic on published rates from mid-2026. A mid-tier model sits around $2.50 per million input tokens and $15 per million output tokens. A typical call of roughly 1,000 input tokens and 500 output tokens works out to about a cent. Drop to a budget model at around $0.25 input and $1.50 output, and the same call costs a tenth of that. So far, so cheap.

The reasoning tier is where it stops being cheap. Reasoning models bill their hidden thinking tokens at the output rate, and a single call can burn tens of thousands of output tokens before it produces a one-paragraph answer. At flagship output prices, one such call lands near a dollar and a half. That’s not a typo. One answer, more than a hundred times the cost of the mid-tier call above, for a question you might have been able to answer with a join.

Read More : How these calls quietly inflate a cloud bill

It feels free because at ten calls a day, a cent is nothing, and a dollar fifty once in a while is nothing either. That’s the trap. The unit cost is small enough to ignore right up until the unit count isn’t.

Where’s the line? At genuinely low volume, an LLM call is cheap, and optimizing it is a waste of your attention. The cost turns real at scale, and it turns alarming inside agent loops, where one user action can trigger a dozen calls before anyone sees a result.

What does a SQL query cost next to that?

Next to an LLM call, a well-indexed SQL query is effectively free. You’re paying for a database instance by the hour whether it serves ten queries a second or ten thousand, so the marginal cost of one more indexed lookup rounds to zero.

That’s the structural difference, and it’s worth saying plainly. An LLM call is metered: every request is a fresh charge against a per-token meter. A database query is amortized: you’ve already bought the capacity, and the query rides on a cost you’re paying regardless. One pricing model scales with usage. The other doesn’t.

The answer-already-in-a-column case is the cleanest example:

-- the answer is a column, fetched with one indexed lookup
SELECT plan_tier, seats_remaining
FROM accounts
WHERE id = $1;

That returns in single-digit milliseconds or less, for a fraction of a microcent of compute you were going to spend anyway. There’s no model to wait on, no tokens to count, and the result is exact rather than approximate.

Where’s the line? The query is cheap only when the work is indexed and the data is already there. A sequential scan over a hundred million rows, or an N+1 pattern firing one query per row inside a loop, throws the advantage straight in the bin. Cheap is a property of the access pattern, not the technology. A bad query can be slow and wasteful too. It just won’t send you an invoice for it.

Why does the latency gap matter as much as the cost?

Because latency compounds, and the model sits on the wrong side of the gap. The fastest mainstream models reach their first token in a few hundred milliseconds. Reasoning models can take tens of seconds end to end. A SQL lookup often finishes before the network round trip to the LLM has even completed.

The numbers from mid-2026 benchmarks are stark. The quickest mainstream models reach their first token in under about 600 milliseconds, while slower ones sit closer to 2,400. Reasoning models, because they think before they answer, can run from ten seconds to well over a minute for a full response. An indexed database lookup, by contrast, is a sub-millisecond to low-single-digit-millisecond operation.

Now watch it compound. A four-step agent that makes one model call per step, at 600 milliseconds to first token each, spends 2.4 seconds just getting started. Swap in a slower model at 2,400 milliseconds per step and you’re near ten seconds before the agent produces anything useful. Every step you could have answered with a query is latency you didn’t have to pay.

Read More : When a specialized tool is genuinely worth it

Where’s the line? For a single, user-initiated answer, a few hundred milliseconds is invisible, and the model’s latency is a non-issue. In a tight loop, a hot request path, or anything a user is sitting and waiting through, it’s the whole experience.