What Is an LLM API? Requests, Messages, Tokens and Errors

An LLM API is a web endpoint that accepts text and returns a model's reply. Your program sends an HTTP request with a model name, a list of messages and a token limit. The service sends back the generated text, a reason it stopped, and a count of the tokens it read and wrote. A chat app does the same work behind a text box. The API gives your code the raw parts, so your code decides what goes in and what happens to the answer.

#What an LLM API is

In a chat app, the model feels like a conversation partner. Underneath, each message you send is a complete HTTP request. The app gathers the earlier messages, adds the new one, sends everything to a server, and shows the text that comes back. The model keeps no memory of your last message unless the app sends the history again.

An API skips the interface. Your code builds the message list, sets the token limit and handles the answer. The service does not care whether the caller is a browser, a script or a scheduled job. It checks the request format and the key.

Three terms matter from the start:

• Model: the named model that generates the reply. Each service publishes its own model IDs, and the list changes over time.
• Messages: an ordered list of turns, each with a role such as user or assistant.
• Tokens: the chunks of text the model reads and writes. A token is often shorter than a word. Limits, usage counts and prices are all measured in tokens.

#Why it matters for developers

A chat window lets one person ask questions. Code needs repeatable inputs, outputs it can parse and errors it can catch. Once a script calls a model, the key, the bill and the failure modes become your responsibility. Most first-integration problems come from a malformed body, a missing or wrong key, a limit nobody checked, or a response handled as if it always succeeded.

#How a request works, step by step

#1. Your code sends an HTTP request

Nearly every LLM API you will meet is a web API over HTTP. The request has a method, a URL, headers and usually a JSON body. The model call is almost always a POST. RFC 9110, the current HTTP semantics standard, defines what those parts mean and how a server must answer. The status codes and headers you meet later come from it.

#2. The body names the model and the messages

The body is a JSON object. At minimum it names the model and sends the conversation so far. This is the Messages format, one of the two request styles compared below. The values are placeholders, and nothing here is executed.

{
  "model": "<MODEL_ID_FROM_CURRENT_CATALOG>",
  "max_tokens": 300,
  "system": "You explain security concepts in plain English. Keep answers under 120 words.",
  "messages": [
    {
      "role": "user",
      "content": "What is the difference between hashing and encryption?"
    }
  ],
  "stream": false
}

Annotations:

• model is the ID of the model that generates the reply. Copy it from the provider's current list, not from an old tutorial.
• max_tokens caps how many tokens the model may write. It does not cap your prompt.
• system holds standing instructions. In this format it sits outside the message list.
• messages holds the conversation. Here it has one user turn.
• stream is false, so the server waits for the full reply before answering.

#3. Messages have roles

Each message has a role that says who is speaking:

• user: text from your end user or your program.
• assistant: a reply the model wrote earlier. You send it back to continue the conversation.
• system or developer: standing instructions. The name depends on the format.
• tool: the result of a tool call, covered in a later article.

To continue a conversation, append the model's last reply as an assistant message, then add the new user message. If you forget earlier turns, the model answers the new question with no context, and nothing reports an error. That is one of the most common bugs.

#4. The token limit caps the reply

max_tokens (or its equivalent) limits how much the model writes. If the reply reaches the limit, the text stops mid-sentence and the response says so. Check the stop reason, covered in step 6, instead of assuming every reply is complete. The limit does not reduce what you pay for the prompt, because input tokens still count toward usage.

#5. Streaming changes delivery, not meaning

With "stream": true, the server sends the reply in pieces as it generates them, using server-sent events. Users see words appear one after another. Your code reads the events and joins the pieces, and the usage figures and stop reason arrive in the closing events. Stream when a person is watching. For a batch job that stores the reply, a normal request is simpler to write and test.

#6. The response carries the text, the reason and the counts

A typical response contains:

• An ID for the request or message, useful when you report a problem.
• The generated content. In the Messages format it is a list of content blocks. In the Chat Completions format it sits in a choices array.
• A stop reason explaining why generation ended, such as a natural end of turn, the token limit, or a tool call.
• Usage counts for input and output tokens.

Read the stop reason before you trust the text. A reply that stopped at the token limit is incomplete. A reply that ends in a tool call is not the final answer, and your code must run the tool and send the result back.

#7. Log usage, not content

The usage object drives your bill. Log the counts, the model ID, the time, the stop reason and the status code. Do not log prompts or replies by default, because they can contain customer data, private code or secrets. Metadata is enough to find cost spikes and failures.

#Two request formats side by side

Two request styles are widely used in public documentation, and many services copy one of them so existing client code keeps working with a different base URL. Knowing both helps you read examples from any source.

The first is the Messages format, documented in the first reference under Sources. It takes system as a top-level field, requires max_tokens, returns content blocks with a stop_reason, and reports input_tokens and output_tokens.

The second is the Chat Completions format, documented in the second reference under Sources. It puts system or developer instructions inside messages, accepts max_completion_tokens, and returns a choices array with a finish_reason and a message in each item. Its usage object reports prompt_tokens, completion_tokens and total_tokens.

The differences are in shape, not meaning. Both send a model, a conversation and a limit, and both return text, a reason and counts. When an example uses the other format, map the fields by their role in this table rather than copying the JSON.

The same question in the Chat Completions format:

{
  "model": "<MODEL_ID_FROM_CURRENT_CATALOG>",
  "max_completion_tokens": 300,
  "messages": [
    {
      "role": "developer",
      "content": "You explain security concepts in plain English. Keep answers under 120 words."
    },
    {
      "role": "user",
      "content": "What is the difference between hashing and encryption?"
    }
  ],
  "stream": false
}

A reply in this format has the following shape. The values are placeholders, not an observed response:

{
  "id": "<RESPONSE_ID>",
  "object": "chat.completion",
  "model": "<MODEL_ID_FROM_CURRENT_CATALOG>",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "<GENERATED_TEXT>" },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

Your code reads choices[0].message.content, choices[0].finish_reason and then the usage object. The Messages format keeps the same information in content, stop_reason and usage, under different names.

#Authentication with API keys

An API key identifies the account that pays for a call and decides which models and limits apply. The two formats send it differently. The Chat Completions format uses an Authorization header with the Bearer scheme. The Messages format uses a dedicated key header and also requires an API version header, so the provider can keep older behavior stable.

Header names change with the provider and the version, so copy them from the current documentation. The shapes look like this:

# Placeholder shape only. Do not run this with a real key in a shared shell history.
curl https://<API_HOST>/<MESSAGES_PATH> \
  -H "x-api-key: <YOUR_API_KEY>" \
  -H "anthropic-version: <VERSION_FROM_CURRENT_DOCS>" \
  -H "content-type: application/json" \
  -d '{"model":"<MODEL_ID>","max_tokens":300,"messages":[{"role":"user","content":"Hello"}]}'

# Placeholder shape only. Same caution as above.
curl https://<API_HOST>/<CHAT_COMPLETIONS_PATH> \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"<MODEL_ID>","messages":[{"role":"user","content":"Hello"}]}'

Annotations:

• The key is a credential. Anyone who holds it can spend your balance and read what the account can read.
• Authorization: Bearer is the standard HTTP scheme for tokens. RFC 9110 defines the Authorization header in its authentication section.
• Typed commands can land in shell history. Use an environment variable, as shown below, instead of a literal key.

#Keeping keys out of code and Git

This is where most avoidable costs and leaks begin. The rules below follow the OWASP Secrets Management Cheat Sheet.

Keep keys out of source code. A key written into a Python file exists in every copy of that file: your laptop, your teammates' machines, the CI system and every commit that touched it. OWASP names hardcoded secrets in source and configuration files as a risky practice.

Never commit a key. Git keeps history. Deleting the key in a later commit does not remove it from the old one. OWASP notes that removing a secret from history may require a rewrite, which has side effects. Treat a key that reached a shared repository as exposed. Revoke it, create a new one and check the account for use you did not make.

Keep keys out of logs, screenshots and browser code. Anything in a log line, an error message or a web page script can be read by others. Calls that need a key should run on a server you control.

Do not bake keys into container images. OWASP warns against secrets in Docker ENV or ARG instructions, because the value ends up in image metadata and build history.

Prefer a secret store. OWASP says environment variables are generally accessible to all processes and may appear in logs or dumps, so it recommends them only when better options are not possible. On a laptop, a password manager or the operating system's credential store works. On servers, use a managed secret service or mounted files with restricted permissions.

Limit and rotate. Give each key only the access it needs, and replace keys on a schedule. OWASP says secrets must have the minimum privileges assigned to them and recommends automated rotation where possible.

A safe local pattern keeps the value in the environment and reads it by name:

# Read the key from the environment. Fail early if it is missing.
import os

api_key = os.environ.get("LLM_API_KEY")
if not api_key:
    raise SystemExit("LLM_API_KEY is not set. Set it in your shell or secret store.")

Add local settings files to .gitignore before you create them. Commit a template such as .env.example that lists the variable names with blank values, and never commit the real file.

#Rate limits and quotas

Providers limit how many requests and tokens an account may use in a time window. These are often called rate limits, and some accounts also have spending caps. The numbers vary by account, model and time, so check your provider's console and current documentation. This article gives no figures.

When you hit a limit, the server typically returns 429 Too Many Requests. RFC 6585 defines that code, which extends the base HTTP set. The response may include Retry-After, which RFC 9110 defines in its response-fields section and which says how long to wait.

Your code should read the status before the body, wait before retrying (using Retry-After when present, otherwise an increasing delay with some random jitter), and stop after a few attempts instead of looping forever. Count tokens before sending very large prompts, because an oversized request fails whatever your balance.

#Common HTTP errors

The first digit of a status code tells you the class: a client-side problem, a server failure, or something else. RFC 9110 defines the classes and the individual codes. These are the errors new integrations meet most often.

A 4xx error means the request needs a change, so sending the same request again will not help. A 5xx or 429 may succeed later, so backoff makes sense there. Log the status code, the model ID and the request ID. Never log the key or the prompt.

#Common mistakes

Trusting the first response. A reply can be cut off by the token limit or stop for another reason. Check the stop reason every time.

Forgetting earlier turns. A stateless API knows nothing about the previous request unless you send it again. Build the message list from your own storage.

Hardcoding the model ID. Model IDs get retired and renamed. Keep the ID in configuration, so a change is a one-line edit.

Mixing the two formats. Sending a Chat Completions body to a Messages endpoint, or the reverse, produces 400 errors. Confirm which format the endpoint documents before you copy an example.

#How to go further

Read one provider's reference from top to bottom. The request, response and error sections are short and answer most first-week questions. Then read about streaming and context windows, which decide how much conversation you can send before a request fails.

NotHoly's gateway documents its own fields and error codes. Compare them with the two shapes above, and treat its documentation as the authority for its behavior.

If you want the model to call your code, the next step is tool use: an assistant turn whose stop reason asks your program to run something and send back the result. The next article covers that loop.

#Sources

• Messages API reference. Request fields, response fields and stop reasons for the Messages format.
• Chat Completions API reference. Request fields, message roles, response fields and finish reasons for the Chat Completions format.
• RFC 9110, HTTP Semantics. Methods, headers, authentication, status codes and Retry-After.
• RFC 6585, Additional HTTP Status Codes. Defines 429 Too Many Requests.
• OWASP Secrets Management Cheat Sheet. Where secrets should not live and how to store, scope and rotate them.