ChatGPT doesn't think; it calculates the probability of your next word
In short: When you prompt an AI, it isn't searching a database for answers like a search engine. Instead, it is performing massive statistical calculations to predict which words should follow your query based on billions of patterns it observed during its training.
The core architecture of ChatGPT is the Generative Pre-trained Transformer (GPT). The three core roles of this title are its ability to create content, its human-led pre-training phase, and how it converts provided data into results. Developed by OpenAI, the model uses neural networking combined with two pillars of modern machine learning: supervised learning and reinforcement learning.
At its most fundamental level, ChatGPT is a sophisticated prediction engine. Unlike simple predictive text on a smartphone that guesses words based on preceding letters, this model attempts to create fully coherent sentences and paragraphs. It does this by analyzing context and meaning rather than just literal text strings. To handle a prompt, the software determines the subsequent word's likelihood within its training set, creating an ordered sequence of chances for each potential follow-up.
To achieve human-like nuance, the model undergoes a multi-stage training process. First, human contractors act as both users and ideal chatbots to teach the model how to conduct conversations. This supervised learning helps the AI move beyond simple sentence completion to understand patterns of intent. However, because humans cannot anticipate every possible question, the model then utilizes reinforcement learning.
This third stage is a form of unsupervised learning where no specific output is tied to an input. Instead, the model learns underlying contexts from a massive dataset—approximately 45 terabytes of data including webpages and books. By identifying patterns across these near-limitless subjects, it can respond appropriately to novel queries while occasionally randomizing outputs to ensure variety in its responses.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: So How Does ChatGPT really work? Behind the screen!
To cite this card, link to https://intellectplus.co.uk/app/science-so-how-does-chatgpt-really-work-behind-the-screen.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Science · Latest · A–Z · All topics
