large language model

LLMs are built on the transformer architecture introduced by Google researchers in 2017, which dropped older recurrent models in favor of a self-attention mechanism that reads a whole passage at once instead of word by word. Text is split into tokens (in English, roughly 4 characters ≈ 1 token), and the model learns by a simple game: given the tokens up to some position, predict the one that comes next. The correct answer already sits in the training text, so no human labels are needed. That is why the method is called self-supervised.

Scale is the point. Modern models carry billions of parameters (the weights tuned during training) and read trillions of tokens, and at that size they start handling tasks nobody trained them on directly, like translation, summarization, and code.

Why it matters for search

Every AI answer engine runs on an LLM. ChatGPT, Perplexity, Google’s AI Overviews, and Gemini all generate replies token by token, sometimes grounded in retrieved documents (see retrieval-augmented generation), sometimes from the model’s memory alone. That second case is where hallucination comes from: the model picks a plausible-sounding token, not a verified fact. Knowing that mechanism is the starting point for getting your brand named in AI answers instead of left out of them.