Lesson 1 of 5

Your First API Call

"The most important property of a program is whether it accomplishes the intention of its user." — C. A. R. Hoare, An Axiomatic Basis for Computer Programming (1969)

Part I was about understanding. This part is about typing.

You now hold an accurate model of what sits behind the endpoint: a stateless function over tokens, priced by the token, positioned in a vector space, shaped by a training process that ended on a date, optimising plausibility rather than truth. That understanding is worth a great deal, and it is about to start paying out — because almost every quirk of the API you are about to meet is a direct consequence of something in Part I, and engineers who skipped Part I experience those quirks as arbitrary.

Here is the good news, and it genuinely is good news: calling a language model is an HTTP request. Not a special protocol, not an SDK you must adopt, not a runtime to install. You POST some JSON to an endpoint with an Authorization header and you get JSON back. If you have ever called Paystack, Stripe, or any third-party REST API, you already know ninety per cent of the mechanics. The SDKs that providers publish are convenience wrappers over exactly that request, and there will be moments in your career — debugging a proxy, writing a Bash one-liner, working in a language with no official SDK — when knowing the raw shape underneath is the thing that unblocks you.

The remaining ten per cent is where the interesting engineering lives, and it is where this chapter spends most of its time. This is an HTTP call with an unusual cost profile (you pay per token, and Chapter 2 taught you how quickly that compounds), an unusual latency profile (seconds, not milliseconds, and the first byte and the last byte are separate problems), an unusual failure profile (429s are routine rather than exceptional, and a 200 response can still contain a truncated answer), and an unusual determinism profile (the same request can return different bytes). Every one of those pushes back on the way you would ordinarily structure a service, and this chapter is about structuring it correctly from the first line.

We are going to build the call up in layers: the raw request, the response and the fields that matter, the wrapper you should actually ship, and the operational concerns — timeouts, cost accounting, logging — that separate a demo from a service. There is no network access in the playground, so every block here simulates the transport faithfully rather than reaching out. That is a feature: it means you can reason about the shape of the exchange without an API key, and everything you build will transfer directly.

The big question

If calling a model is just an HTTP POST, why can't you treat it like any other API client — and what does the answer tell you about where the abstraction in your codebase should sit?

The request, in full

Strip away every SDK and this is what goes over the wire. One endpoint, one header set, one JSON body.

JavaScript — Try it Yourself

Loading editor…

Console

Runs automatically. Edit the code to see it change.

That is the entire surface for a basic call. Providers differ in details — the endpoint path, whether the system prompt is a top-level field or the first message, the exact name of max_tokens — but the shape is close to universal, and the differences are the reason for the wrapper we build later in this chapter.

The key never touches the browser

Before anything else, the security point, because it is the most common serious mistake in this entire volume and it is made most often by strong frontend engineers.

Your API key must live on your server. Not in a NEXT_PUBLIC_ environment variable, not in a build-time constant, not in a config object the client bundle imports, not "temporarily for the demo". Anything that reaches the browser is public — the network tab shows every request, the bundle is downloadable, and automated scanners crawl deployed JavaScript looking for exactly these strings. A leaked key is not merely a security embarrassment; it is a metered billing account that a stranger now controls, and the bill arrives before the alert does.

The correct architecture is a route in your own backend that the browser calls, which then calls the provider:

Rendering diagram…

Look at what that intermediary buys you beyond key safety, because each item is a real requirement you would otherwise have nowhere to put: authentication (is this a logged-in user?), authorisation (are they entitled to this feature?), per-user rate limiting (one user cannot exhaust your quota), cost attribution (which user, which feature, how many tokens), prompt privacy (your carefully-built system prompt is not shipped to the client), input validation, and a single place to swap providers. There is no serious production architecture where the browser calls the provider directly.

The engineer's notebook

Set a hard spending limit in the provider's dashboard on the day you create the key — before you write a line of code, while you still remember. Every provider offers one. The failure this protects against is not sophisticated attack; it is a retry loop with no backoff, a test suite accidentally pointed at production, or a leaked key found by a scanner over a weekend. I have seen all three, and in every case the technical fix took ten minutes and the invoice conversation took considerably longer. Set the limit, set a budget alert at half the limit, and use a separate key per environment so you can revoke one without taking down the others.

The response, and the fields that matter

The response is JSON, and most engineers read exactly one field out of it. That habit costs them money and hides bugs.

Tests — Run & Watch Them Pass

Run to execute the tests…

Three habits come out of that block, and I would hold you to all three in code review.

Always read stop_reason. A response that hit max_tokens is a successful HTTP call containing an incomplete answer. If you are parsing JSON out of it, you get a cryptic syntax error at a random position; if you are showing prose, the user gets a sentence that stops mid-w. Neither failure announces its cause. One field read at the boundary turns both into a clear, actionable error.

Always record usage. Not sometimes, not in staging — every call, logged with the user, the feature, the model and the prompt version. This is the only way you will ever answer "why did the bill triple?", and it is the difference between an AI feature you can reason about economically and one you simply hope stays cheap.

Never assume content[0].text. Content is an array of typed blocks precisely because a response may contain tool calls, several text segments, or other block types you have not met yet. Filter by type; do not index by position. This one will bite you the day you enable tools in Part V.

Everyday analogy

Think of ordering food from a kitchen through a hatch. You pass in a written order (the messages), the kitchen cooks (inference), and a plate comes back with a docket attached. Most engineers take the plate and throw the docket away. The docket says how much of each ingredient was used, and whether the chef stopped because the dish was finished or because they ran out of pan. Throw it away and you can still eat — you simply have no idea what anything costs, and you cannot tell a finished dish from an interrupted one until someone complains.

The wrapper you should actually ship

Here is the mistake that shows up in nearly every codebase's first AI feature: the provider's SDK gets called directly from a route handler, then from a second route handler, then from a background job, and within a month the model name, the temperature and the retry policy are scattered across fourteen files with three different values.

Put one module between your application and the provider. Not a heavy abstraction layer attempting to be provider-agnostic in the abstract — that is over-engineering, and providers differ in ways that leak. One narrow module that owns the concerns your application should not know about.

Tests — Run & Watch Them Pass

Run to execute the tests…

Notice what injecting the transport bought: every one of those tests runs in milliseconds, offline, deterministically. Your application logic — message assembly, clamping, truncation handling, cost accounting, validation — is ordinary code with ordinary tests. Only the thin transport function actually touches the network, and that is the only piece needing an integration test. This is the same dependency-inversion discipline Volume 10 taught you, applied to a dependency that happens to be probabilistic, and it is what makes an AI codebase testable at all.

Model tiers are an architectural decision

That MODELS map deserves more than a passing glance. Providers offer a range: small-fast-cheap through large-slow-capable, often a twelve-fold price difference between the ends.

The instinct is to use the best model everywhere. The better practice is to make the tier an explicit, per-task decision. Classification, routing, short extraction, tagging and simple rewriting are frequently indistinguishable between tiers — and on those tasks the small model is not merely cheaper but faster, which your users feel. Reserve the large model for genuinely hard reasoning, long-context synthesis, and anything user-facing where quality is the product.

Because the tier lives in one map behind one wrapper, you can change your mind cheaply, and — this is the part that matters — you can measure the change, once Part VI gives you an eval set. Without the wrapper, "let's try the cheaper model for tagging" is a fourteen-file refactor nobody schedules.

In the real world

A team I advised ran every request through their provider's largest model because that is what the tutorial used. Their monthly bill was uncomfortable and growing linearly with signups. We looked at the traffic and found that roughly seventy per cent of calls were a single classification step — routing a support message into one of nine categories — and the rest were the actual drafting work. We moved the classifier to the small tier, kept the drafting on the large one, and measured both against a hand-labelled set of four hundred real messages. Classification accuracy moved by well under a point. The bill fell by more than half, and median latency on the routing step dropped from about two seconds to under four hundred milliseconds, which users noticed before the finance team did. The lesson is not "use small models" — it is that the tier is a decision per task, and you cannot make it at all if the model name is hard-coded in fourteen files.

Timeouts, and why the default is wrong

Every HTTP client you have ever configured had a timeout in the region of thirty seconds, and for a conventional API that is generous. For a language model it is a coin toss, because generation time scales with output length — Part I's decode phase is sequential, one token at a time.

A 2,000-token answer at 50 tokens per second takes forty seconds when everything is healthy. Your thirty-second timeout aborts a perfectly good request, your retry logic fires, and you now pay twice for a response the user never sees. Meanwhile a genuinely hung connection needs to be caught quickly, not after two minutes.

The resolution is that a single timeout is the wrong instrument. You need two, and they measure different failures:

Tests — Run & Watch Them Pass

Run to execute the tests…

A time-to-first-token timeout of five to ten seconds catches the real failure — a connection that is going nowhere — quickly. A generous overall ceiling then lets a legitimately long answer finish. Streaming, which the next chapter is devoted to, makes this arrangement natural, because you can watch tokens arrive and reset your patience with each one.

And note the last-but-one test: capping max_tokens is a latency control as much as a cost control. If your product needs answers inside ten seconds, that is a token budget before it is a prompt-writing problem, and no amount of instructing the model to "be brief" enforces it the way the parameter does.

Common Mistakes

Putting the API key in the client. NEXT_PUBLIC_, a build constant, a config object the bundle imports — all public. Scanners find deployed keys within hours. Route every call through your own backend, set a spending cap the day you create the key, and use a separate key per environment.
Ignoring stop_reason. A truncated answer arrives as HTTP 200 with valid-looking content. If you are parsing JSON you get a baffling syntax error; if you are showing prose the user gets half a sentence. Check the field at the boundary and turn it into an explicit error.
Throwing away usage. Without per-call token logging tied to user, feature, model and prompt version, you cannot explain a bill, cannot attribute cost, and cannot tell whether a prompt change made things cheaper or dearer. Log it on every call from the first day.
Calling the SDK from everywhere. Model name, temperature, retry policy and timeouts scattered across the codebase means you can never change any of them safely. One wrapper module, injected transport, everything else calls that.
Using one timeout. Thirty seconds kills healthy long generations and is far too patient for a stuck connection. Use a short time-to-first-token deadline and a generous overall ceiling.
Defaulting to the largest model. A large share of production calls are classification or short extraction, where a small model is as accurate, several times faster and an order of magnitude cheaper. Make the tier a per-task decision behind the wrapper, then measure it.
Assuming content[0].text. Content is an array of typed blocks. The day you turn on tools, position-indexing breaks in a way that looks like a model problem and is not. Filter by type.

Interview Corner

Interview Corner — the same idea at three levels. Answer yours; recognise the one above it.

Warm-up

Walk me through what happens when your app calls an LLM API.

Model answer — The browser calls my own backend, never the provider directly, because the API key has to stay server-side. My server authenticates the user, then POSTs JSON to the provider with an Authorization header — the body carries the model name, max_tokens, temperature and the full messages array, since the model is stateless and only knows what's in that array. The response comes back with the generated content, a stop_reason telling me whether it finished or hit the token ceiling, and a usage object with input and output token counts. I read all three: the text for the user, stop_reason to detect truncation, and usage to log the cost.

Intermediate

Your AI endpoint is timing out for some users but not others. How do you investigate?

Model answer — My first hypothesis is that it's not a fault at all — generation time scales with output length, so users whose requests produce long answers are simply exceeding a timeout that was set for a normal API. I'd check whether the failing requests correlate with output length rather than with user or region, and I'd look at whether we're using one timeout for everything, which is the usual cause. The fix is two deadlines: a short time-to-first-token timeout, five to ten seconds, which catches a genuinely stuck connection fast, and a much more generous overall ceiling that lets a long answer finish. Then I'd cap max_tokens to bound the worst case, because that's the only real enforcement — instructing the model to be brief isn't one. And I'd move the endpoint to streaming, which makes the whole problem better: the user sees tokens immediately, so the perceived latency collapses even though total time doesn't change. One thing I'd verify early is whether our retry logic is firing on these timeouts, because if it is, we're paying twice for responses nobody sees, and that shows up on the bill before it shows up in the logs.

Advanced

Design the service layer for a product where several features call LLMs.

Model answer — One module owns provider contact; nothing else in the codebase imports the SDK. It exposes a narrow task-shaped interface rather than a thin passthrough — callers ask for a summary or a classification, not for a chat completion — because that's what lets me change model, temperature or even provider per task without touching callers. Transport is injected so the whole layer is unit-testable offline; only the transport itself needs an integration test. Inside it: a model tier registry so the tier is a per-task decision I can measure and change in one place, prompt templates versioned in code with the version stamped on every log line, structured logging of user, feature, model, prompt version, token counts and computed cost on every call, retry with exponential backoff and jitter on 429s and 5xx only — never on 4xx — plus a circuit breaker so a provider outage degrades us instead of queueing until we fall over. Per-user and per-feature rate limits live here too, since the provider's quota is a shared resource and one user must not be able to exhaust it. On top of that I'd want a response cache keyed on the exact request for deterministic calls, which is free money on repeated identical prompts, and a kill switch per feature so we can disable one AI surface without a deploy. The organising principle is that the model is an unreliable, metered, slow dependency, and everything in that list is something you'd build for any dependency with those three properties — we've been doing this for third-party payment APIs for years.

Company lens — how the same topic gets asked

micro1Often a live build: wire up a working call, handle the error path, don't leak the key. They're watching whether you check stop_reason and usage or just grab the text — that single habit reads as production experience.

GoogleSystems reasoning about the dependency: timeout strategy, retry semantics, idempotency, and what your service does when the provider is degraded rather than down. Be precise about which failures are retryable.

MetaScale and cost: what happens to your architecture and your bill at ten million calls a day, where caching and tiering apply, and how you'd attribute spend to features and teams.

AmazonOwnership and operations: what you log, what you alarm on, what the runbook says when the provider starts 429ing at peak, and how the customer experience degrades gracefully. Frugality plays well here — talk about tiering and caching as engineering, not penny-pinching.

Hands-on

Build createClient properly, with the transport injected so it is testable offline. It should validate input before spending anything, assemble the messages array with the system prompt first when present, clamp max_tokens to the tier ceiling, extract text by filtering content blocks on type rather than indexing, surface truncation as a boolean, compute the cost of every call from the usage object and the tier's pricing, and accumulate a running total. Then extend it in the direction that pays for itself immediately: add a response cache keyed on the exact request body, active only when temperature is 0 — and prove with a test that a repeated identical call costs nothing the second time.

Tests — Run & Watch Them Pass

Run to execute the tests…

Summary

Calling a model is an HTTP POST: a JSON body with model, max_tokens, temperature and the full messages array, an Authorization header, and a JSON response. SDKs are convenience over exactly this.
The key never reaches the browser. Route through your own backend, which also gives you authentication, per-user rate limiting, cost attribution, prompt privacy and a single place to swap providers.
Read stop_reason on every response. A truncated answer is an HTTP 200 containing an incomplete result, and it will surface as a mystifying JSON parse error rather than an obvious failure.
Record usage on every call, tied to user, feature, model and prompt version. It is the only way to explain a bill or evaluate whether a change helped.
Content is an array of typed blocks. Filter by type; never index position zero, or enabling tools in Part V will break you.
One wrapper module owns the provider, with the transport injected so your logic is unit-testable offline and only the transport needs an integration test.
Model tier is a per-task architectural decision. Classification and short extraction rarely need the largest model, and the small tier is both faster and far cheaper.
Use two timeouts: a short time-to-first-token deadline to catch stuck connections, and a generous overall ceiling so healthy long generations are not killed and retried at double cost.

In the next chapter, Roles & Message Design, we take the messages array seriously. System, user and assistant are not decorative labels — they carry different weight with the model, they are the mechanism behind persona and instruction persistence, and the boundary between them is the seam that prompt injection attacks. Getting message design right is what makes everything in Part III possible.

Editorial reflection

I have watched a lot of engineers meet this API for the first time, and the reaction I hope for is mild disappointment. Not because the technology is unimpressive, but because the interface should feel utterly ordinary — a POST, a header, some JSON — and that ordinariness is the good news. It means the enormous body of knowledge you have built over ten volumes about calling unreliable, metered, slow dependencies transfers wholesale. Timeouts, retries, circuit breakers, caching, cost attribution, dependency inversion for testability: none of that is AI engineering, all of it is engineering, and all of it is required here. The engineers who struggle with this material are rarely the ones who find the model mysterious. They are the ones who treat it as so special that ordinary discipline seems not to apply — who skip the wrapper because it is only one call, skip the usage logging because they will add it later, skip the timeout because the demo was fast. The model is the novel part of your system. Almost nothing else about it should be.

Enrol to track progress

Q&A · 0

Enrol to ask questions and join the discussion.

No questions yet — be the first to ask.