Large language model (LLM)

How it works

An LLM reads text as tokens, which are pieces of words (in English, 1,000 tokens is roughly 750 words, per OpenAI's rule of thumb (opens in a new tab)), and predicts the most likely next token, one at a time. Trained on huge volumes of text and code, then tuned to follow instructions, that single skill turns into drafting, extraction, classification, translation and multi-step reasoning. The instruction-tuning step, described by Ouyang and colleagues (2022) (opens in a new tab), is what turns a text predictor into an assistant that does what a request asks.

The models most businesses use come from OpenAI (GPT), Anthropic (Claude) and Google (Gemini), reached through an API and billed per token. Open-weight models, such as Meta's Llama family or Mistral's models, can run on your own servers when data cannot leave your network.

A model only knows what was in its training data, up to a cutoff date, plus whatever you put in the context window for that request. It knows nothing about your company unless you give it that information.

How businesses use it

An LLM is rarely useful on its own. It becomes useful as one part of a larger system:

• An automation that reads inbound email and extracts order details into your ERP
• A RAG system that answers questions from your policies
• An agent that calls tools to finish a task
• A voice line that books appointments

Choosing a model is a cost and quality trade. A small, cheap model often handles classification well. A larger model is worth the extra cost on long documents and judgment calls. On our builds, model usage is billed at cost, so you see the real per-token bill.

Where it goes wrong

LLMs produce fluent text whether or not it is true; see AI hallucination. They are unreliable at exact arithmetic without a calculator tool, and the same prompt can return different answers on different runs. Providers also retire model versions, so a prompt tuned for one model can behave differently after an upgrade. Pin model versions and rerun a fixed test set before every switch.

Where a rule or a lookup table gives the right answer every time, use that and skip the model. It is cheaper and it does not drift.