BYD TechThe short answer
A large language model, or LLM, is an AI model trained on very large amounts of text to predict what comes next, one small piece of text, called a token, at a time. That simple skill, at scale, lets it answer questions, write, summarize, translate, write code and follow instructions. Models such as Claude, GPT and Gemini are LLMs. They are powerful but can state wrong things confidently, know nothing after their training data unless given it, and only see what fits in their context window.
Key points
- An LLM predicts the next token, based on patterns learned from huge amounts of text.
- Tokens are words or pieces of words; pricing and limits are counted in them.
- The context window is everything the model can see at once.
- LLMs can be confidently wrong, so important answers need grounding and checks.
- Products rarely train their own; they call a provider’s model through an API.
How an LLM works
Google’s machine learning course describes a language model as one that estimates the probability of a token or sequence of tokens occurring within a longer sequence. A token can be a word, part of a word or a single character. Generation is repetition: the model picks a likely next token, adds it, and predicts again.
Modern LLMs are built on the transformer architecture, introduced in the 2017 paper "Attention Is All You Need". Transformers can look at the whole context at once and weigh which earlier tokens matter for the next one. After training on large amounts of text, models are further trained to follow instructions and behave helpfully.
Terms worth knowing
| Term | Meaning |
|---|---|
| Token | The unit of text the model reads and writes; usage is priced in tokens |
| Context window | How many tokens the model can consider at once, including your instructions and documents |
| Prompt | The input you give the model |
| Training cutoff | The point after which the model learned nothing new |
| Hallucination | A fluent but false statement |
| Tool use | The model asking your code to run a function, such as a search, and using the result |
What LLMs are good and bad at
| Good at | Weak at, without help |
|---|---|
| Summarizing and rewriting | Facts after the training cutoff |
| Answering questions over documents you provide | Facts about your business it was never given |
| Extracting structured data from text | Exact arithmetic over many steps |
| Drafting and translating | Knowing when it does not know |
| Writing and explaining code | Guaranteeing the same answer every time |
Most weaknesses are handled in the product design: give the model your data at request time, let it call tools for calculations and lookups, and check outputs before they are used. See How to reduce AI hallucinations and RAG vs fine-tuning.
Using an LLM in a product
Almost no product trains its own LLM. You call a provider’s model through an API, from your server, and build the feature around it. When the model is given tools and allowed to take steps toward a goal, it becomes an agent. See How to integrate AI into your app and What is an AI agent?. We build these features in AI agent development.
Sources
Checked when this guide was written.
