How to control how AI chooses its next words

How to control how AI chooses its next words
Image: Intellect (generated)

In short: Large language models don't just pick words; they navigate a landscape of probabilities. By adjusting sampling strategies, you can transform an AI from a rigid, precise tool into a creative storyteller.

When a model prepares to generate text, it starts with raw probabilities for every word in its vocabulary, known as logits. These represent the model's initial gut feelings about what should come next. To turn these raw scores into actual text, we use different sampling methods to guide the selection process.

One way to influence this is through temperature. A high temperature (greater than 1.0) makes choices more random and creative, while a low temperature (less than 1.0) makes the output more focused and deterministic. You can also limit the options by using Top-k filtering, which looks only at the k most likely words, or Top-p (Nucleus) sampling, which considers only the most likely words that meet a specific probability threshold, such as the top 90%.

Based on: Hugging Face LLM Course — Deep dive into Text Generation Inference with LLMs · Licence: Apache License 2.0 · View source →

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/technology-how-to-control-how-ai-chooses-its-next-words.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…