What does LLM stand for? Large language models explained in plain English

You keep seeing "LLM" in news stories, job ads and product pages, usually with no explanation. Most definitions either stop at the acronym or jump straight into math, so you finish reading knowing the words but not what is actually happening inside the thing.

This guide fixes that. You will get the short answer, a word-by-word breakdown, a plain explanation of how LLMs work, a tiny Python demo of next-token prediction that we ran ourselves, current examples as of October 2026, and the limits you should know before you rely on one.

What does LLM stand for?

LLM stands for large language model. It is a type of AI model trained on enormous amounts of text so that it can predict, and therefore generate, language. When people say "LLMs" in the plural, they mean the whole category: the models behind ChatGPT, Claude, Gemini and many others.

You will also see "language learning model" online. That is a common mix-up rather than a real term. The name used by researchers, Google's own machine learning course (opens in a new tab) and every major AI company is large language model.

What is a large language model? A plain definition

Start with the second half of the name. Google's Machine Learning Crash Course defines it like this:

In everyday terms, a language model is a very sophisticated autocomplete. Given "The capital of France is", it estimates that "Paris" is far more likely to come next than "purple". A large language model does the same thing at enormous scale: it was trained on a vast amount of text, it has billions of parameters, and it can look at a long stretch of context at once. Google's course describes LLMs as models that predict a token or sequence of tokens, sometimes many paragraphs worth.

The surprising part is how much falls out of that one skill. To predict the next word well across billions of examples, a model has to pick up grammar, facts, writing styles, reasoning patterns and the structure of code. That is why the same model can draft an email, explain a contract clause and fix a bug.

How do LLMs work? The five steps

Every time you send a message to an LLM, the same basic cycle runs. Here it is without the math.

1. Your text is split into tokens. A token can be a whole word, part of a word or a single character. The model never sees letters, only token IDs.
2. Each token becomes a list of numbers. These numbers, called embeddings, place tokens with similar meanings near each other, so "cat" and "kitten" end up closer than "cat" and "carburetor".
3. The model weighs the context. Layers of a neural network called a Transformer work out how each token relates to every other token in your message, using a mechanism called attention.
4. It predicts the next token. The output is a probability for every token in its vocabulary: for example, 92 percent "Paris", 1 percent "Lyon", and tiny amounts for everything else.
5. It picks one and repeats. The chosen token is added to the text, and the whole cycle runs again for the next one, until the model produces a stop signal or hits a length limit.

That last step is why answers appear word by word on screen. The model is not retrieving a stored answer; it is generating one token at a time, each based on everything that came before.

Tokens: the unit LLMs read and write

Tokens matter because they are how LLMs measure everything: how much text they can read at once and, for paid APIs, what you are billed for. OpenAI's guide to tokens (opens in a new tab) gives these rules of thumb for English text:

• 1 token is about 4 characters.
• 1 token is about three-quarters of a word.
• 100 tokens are about 75 words.

OpenAI calls these estimates, not exact counts. Other languages can split very differently, and even capitalization and a leading space change how a word is tokenized. If you want to see real splits, OpenAI points to its online Tokenizer tool and its open-source tiktoken library.

Transformers and attention: how LLMs use context

Older language models read text one piece at a time and struggled to connect words far apart. The breakthrough came in 2017, when a team of eight researchers published "Attention Is All You Need" (opens in a new tab). It introduced the Transformer, an architecture "based solely on attention mechanisms" that drops the step-by-step processing of earlier designs. Google's crash course now calls Transformers the state-of-the-art architecture for a wide variety of language model applications.

That question is what self-attention (opens in a new tab) answers, for every token, in every layer. It is how a model works out that "it" in "The trophy didn't fit in the suitcase because it was too big" refers to the trophy, not the suitcase. Because attention looks at the whole context at once rather than word by word, distant words can still shape the meaning of each other.

See next-token prediction in about 20 lines of Python

The easiest way to understand the core idea is to build the world's smallest language model. The script below uses only Python's standard library. It counts which word follows which in a tiny training text, turns those counts into probabilities, and generates text by repeatedly picking the most likely next word. This is a bigram model, the simplest kind of n-gram model (opens in a new tab) that Google's course uses to introduce the topic.

from collections import Counter, defaultdict

# A tiny training corpus. Real LLMs train on trillions of tokens.
corpus = (
    "the cat sat on the mat . "
    "the cat ate the fish . "
    "the dog sat on the rug . "
    "the dog chased the cat ."
)

# 1. Tokenize: here, one token per word.
tokens = corpus.split()

# 2. Train: count which token follows which (a bigram model).
follows = defaultdict(Counter)
for current, nxt in zip(tokens, tokens[1:]):
    follows[current][nxt] += 1

# 3. Predict: turn counts into probabilities for the next token.
def next_token_probs(token):
    counts = follows[token]
    total = sum(counts.values())
    return {t: round(c / total, 2) for t, c in counts.most_common()}

print("after 'the':", next_token_probs("the"))
print("after 'sat':", next_token_probs("sat"))

# 4. Generate: repeatedly append the most likely next token.
text = ["the", "dog"]
for _ in range(5):
    probs = next_token_probs(text[-1])
    text.append(max(probs, key=probs.get))
print("generated:", " ".join(text))

Running it with python3 toy_lm.py prints exactly this:

after 'the': {'cat': 0.38, 'dog': 0.25, 'mat': 0.12, 'fish': 0.12, 'rug': 0.12}
after 'sat': {'on': 1.0}
generated: the dog sat on the cat sat

Three things are worth noticing, because they map directly onto real LLMs:

• Training is counting patterns. "the" was followed by "cat" 3 times out of 8 in the training text, so the model gives "cat" a 0.38 probability. Real LLMs learn far richer patterns with billions of parameters instead of a lookup table, but the goal is the same: assign high probability to likely continuations.
• Generation is prediction on repeat. The model only ever answers one question, "what comes next?", and builds a sentence by asking it again and again.
• Too little context produces nonsense. "the dog sat on the cat sat" is locally plausible, because every pair of words appeared in training, but the whole sentence is wrong. This model only looks one word back. Google's course makes the same point about n-grams: a 3-gram's only context is the previous two words. Attention exists to fix exactly this, by letting the model consider thousands of tokens at once.

How LLMs are trained

Turning a next-token predictor into a helpful assistant usually takes more than one stage. The details differ between companies, but the broad shape is well documented.

OpenAI's 2020 GPT-3 paper (opens in a new tab) showed how far pretraining alone could go: the model could pick up new tasks from a few examples written into the prompt, "without any gradient updates or fine-tuning". But raw pretrained models are awkward to use, because they continue text rather than follow instructions.

The fix came from the InstructGPT paper (opens in a new tab) in 2022, which opens with a line that still matters: "Making language models bigger does not inherently make them better at following a user's intent." Using reinforcement learning from human feedback (RLHF), the team found that people preferred outputs from a 1.3 billion parameter InstructGPT model over the 175 billion parameter GPT-3. Training method, not just size, decides how useful a model feels.

What makes an LLM "large"?

"Large" refers to two things: the amount of training data and the number of parameters, the adjustable numbers inside the network that store what the model learned. When GPT-3 (opens in a new tab) launched in 2020, its 175 billion parameters were 10 times more than any previous non-sparse language model. Google's crash course now describes Transformers with hundreds of billions or even trillions (opens in a new tab) of parameters, and notes that larger models tend to outperform smaller ones but need more computing resources to run.

Bigger is not automatically better for your task, though. Smaller models are faster and much cheaper, and for narrow jobs such as classification or extraction they are often good enough. Our guide to small language models vs LLMs walks through when to use each, with the cost math.

Examples of LLMs (as of October 2026)

Model names change every few months, so treat this as a snapshot. The table lists current models as named on each vendor's own documentation when we checked in October 2026. Note the difference between an app (ChatGPT, the Claude apps, the Gemini app) and the models that power it.

Sources: OpenAI's models page (opens in a new tab), Anthropic's models overview (opens in a new tab) and Google's Gemini API models page (opens in a new tab). Each vendor offers a ladder from a large, most capable model down to small, fast, cheap ones, for the cost reasons covered above. Many open-weight LLMs also exist that you can download and run on your own hardware.

What LLMs can do

Because almost everything people do at a computer involves text, LLMs turn up in a wide range of tasks:

• Writing and editing: drafts, rewrites, summaries and translations.
• Answering questions: explaining concepts, especially when you paste in the document the answer should come from.
• Coding: writing, explaining and debugging code, which is just another kind of text.
• Extracting structure: pulling names, dates and amounts out of emails, invoices or contracts.
• Acting as the brain of an agent: deciding which tools to call to finish a multi-step task, as our guide to building an AI agent shows.

How much an LLM can work with at once is set by its context window. Anthropic's docs describe it as all the text a model can reference when generating a response, a "working memory" (opens in a new tab) that is separate from what it learned in training. As of October 2026, Anthropic lists a 1 million token context window for its current Claude models, very roughly 750,000 English words by OpenAI's rule of thumb. To give a model access to live data or tools beyond that window, developers connect it to outside systems, for example through the Model Context Protocol.

The limits of LLMs: what they get wrong

Knowing how LLMs work makes their failure modes easy to predict. They are built to produce likely text, which is not the same as true text.

• Hallucinations. OpenAI defines these as "plausible but false statements generated by language models." Its September 2025 research (opens in a new tab) explains that pretraining has no true or false labels, so "arbitrary low-frequency facts, like a pet's birthday, cannot be predicted from patterns alone." It also argues that common training and evaluation methods reward guessing over admitting uncertainty.
• A knowledge cutoff. A model only knows what was in its training data. Vendors publish a training or knowledge cutoff date for each model, as Anthropic does in its models overview (opens in a new tab). Anything after that has to be supplied in the prompt or fetched with a tool.
• Long context is not perfect recall. Anthropic's context window docs note that more context isn't automatically better: as the token count grows, accuracy and recall can degrade, which they call context rot.
• No actions on their own. An LLM only outputs text. Sending an email or updating a database requires software around it that turns that text into actions, with permissions and checks.
• Sensitivity to wording. Small changes in how you ask can change the answer. Clear instructions, examples and supplied context help a lot, as our Claude prompt engineering guide explains.

LLM vs AI vs generative AI

These terms nest inside each other. AI is the broad field of making computers do tasks that normally need human intelligence. Generative AI is the part of AI that creates new content, such as text, images, audio or video. LLMs are the generative AI models that specialize in language. So every LLM is generative AI, but an image generator is generative AI without being an LLM. If you have also seen "agentic AI", that describes systems that use models like LLMs to take actions toward a goal; our explainer on agentic AI vs generative AI covers the difference.

The bottom line

LLM stands for large language model: a model trained on huge amounts of text to predict the next token, then used to generate answers one token at a time. That single idea, scaled up with Transformers, billions of parameters and careful training, explains both what LLMs are good at and why they sometimes state false things confidently. Use them for drafting, explaining, extracting and coding, give them the context they need, and check the facts that matter. For more plain-English explainers on models, agents and tools, browse our AI guides.