What Actually Happens the Moment You Send an AI Prompt

You type a question, hit enter, and a fluent answer appears within a couple of seconds. It feels instantaneous, almost like a trick. There's no trick to it, though - it's a fast process that repeats itself dozens of times a second, and once you can see the individual steps, none of it feels mysterious anymore.

First, your prompt gets broken into tokens - the small chunks of text models actually operate on, typically shorter than a whole word. Each of those tokens then gets turned into a numerical form the model's internal math can work with. That's the moment where, as far as the model is concerned, language stops resembling language and turns into an organized pile of numbers.

Step One: Chopping Words Into Tokens

Tokenization happens to everything you type before anything else does, and it doesn't cleanly follow spaces the way you might assume. A short, common word can be a single token on its own. A longer or rarer one often splits into two or three pieces, and punctuation, digits, or even lone characters can end up as tokens in their own right, depending on the model's particular vocabulary.

• Tokens, not words: the unit the model actually reads and writes in is the token, and tokens don't map one-to-one onto words.
• A fixed vocabulary: each model ships with a defined list of possible tokens - typically tens of thousands - locked in during training rather than adjusted per chat.
• Numbers, not text: once tokenized, your prompt turns into a sequence of numerical IDs, and then into vector representations the model's math can actually process.
• The context window: everything so far - your prompt plus the rest of the conversation - has to fit inside a fixed token budget, which is why a long-running chat can start losing track of earlier details.

The Loop That Does All the Work: Predict, Pick, Repeat

From there, the model surveys everything up to that point - your entire prompt, now in numeric form - and works out, across its whole vocabulary, how well each candidate token would continue things. One gets chosen, usually the highest-probability option though not strictly always, and gets appended to the response so far. The whole calculation then runs again, now accounting for the token just added, to pick the next one. And again. And again, until the model signals it's finished.

That detail carries more weight than it seems to at first glance. The model isn't composing a complete answer internally and then revealing it in pieces - at every step, it genuinely only has access to the tokens it's generated so far. There's no stored plan sitting behind the scenes; whatever plan exists is implicit in the pattern of tokens already laid down.

The Sequence, Roughly

• 1. Tokenize: your prompt gets split into tokens and mapped onto numerical IDs.
• 2. Encode: those IDs turn into vector representations that carry meaning and position.
• 3. Process: the model pushes the full sequence through its layers to build up a picture of context.
• 4. Predict: it produces a probability score for every possible next token across its vocabulary.
• 5. Sample: a token gets picked, shaped by settings such as temperature.
• 6. Append and repeat: the chosen token joins the sequence and the whole process fires again for the next one.
• 7. Stop: generation wraps up once the model emits an end-of-response signal or hits a length cap.

Why the Exact Same Question Won't Always Get the Exact Same Answer

A setting typically called "temperature" decides how much randomness is allowed into that choice. Set it low, and the model almost always reaches for the single most probable token, which produces steady, fairly predictable results. Turn it up, and it more frequently opts for a lower-probability but still reasonable alternative, which produces more varied, occasionally more inventive output. That's the entire reason asking the same thing twice can hand back two different answers that are each perfectly sound on their own.

What That Word-by-Word Text on Screen Is Really Showing

That word-by-word reveal you watch on screen isn't a cosmetic effect layered on afterward - it's about as direct a view of the real process as you'll get. Since the model is already generating one token at a time, most interfaces simply display each one the instant it's produced rather than waiting to show the finished result.

That's also the reason a response can seem to pause mid-sentence, or why a longer answer visibly takes more time than a short one - that's not a rendering lag, it's the underlying token-by-token math genuinely taking that long, run once for every single token in the reply.

The Delay Before Anything Shows Up at All

There's also a short stretch of time before any generation appears on screen at all. Your prompt has to reach a server, wait in a queue if the system is under load, and get routed to available computing capacity before the token-by-token process even begins. That setup delay is separate from generation itself - it's overhead, not the model "thinking" about anything, and it's usually the smallest slice of the whole round trip.