\n\n\n\n
When ChatGPT was introduced last fall, it sent shockwaves through the technology industry and the larger world. Machine learning researchers had been experimenting with large language models (LLMs) for a few years by that point, but the general public had not been paying close attention and didn\u2019t realize how powerful they had become.<\/p>\n\n\n\n
Today almost everyone has heard about LLMs, and tens of millions of people have tried them out. But, still, not very many people understand how they work.<\/p>\n\n\n\n
If you know anything about this subject, you\u2019ve probably heard that LLMs are trained to \u201cpredict the next word,\u201d and that they require huge amounts of text to do this. But that tends to be where the explanation stops. The details of how<\/em> they predict the next word is often treated as a deep mystery.<\/p>\n\n\n\n One reason for this is the unusual way these systems were developed. Conventional software is created by human programmers who give computers explicit, step-by-step instructions. In contrast, ChatGPT is built on a neural network that was trained using billions of words of ordinary language.<\/p>\n\n\n\n As a result, no one on Earth fully understands the inner workings of LLMs. Researchers are working to gain a better understanding, but this is a slow process that will take years\u2014perhaps decades\u2014to complete.<\/p>\n\n\n\n Still, there\u2019s a lot that experts do<\/em> understand about how these systems work. The goal of this article is to make a lot of this knowledge accessible to a broad audience. We\u2019ll aim to explain what\u2019s known about the inner workings of these models without resorting to technical jargon or advanced math.<\/p>\n\n\n\n We\u2019ll start by explaining word vectors, the surprising way language models represent and reason about language. Then we\u2019ll dive deep into the transformer, the basic building block for systems like ChatGPT. Finally, we\u2019ll explain how these models are trained and explore why good performance requires such phenomenally large quantities of data.Subscribe<\/p>\n\n\n\n To understand how language models work, you first need to understand how they represent words. Human beings represent English words with a sequence of letters, like C-A-T for cat<\/strong>. Language models use a long list of numbers called a word vector. For example, here\u2019s one way<\/a> to represent cat <\/strong>as a vector:<\/p>\n\n\n\n [0.0074, 0.0030, -0.0105, 0.0742, 0.0765, -0.0011, 0.0265, 0.0106, 0.0191, 0.0038, -0.0468, -0.0212, 0.0091, 0.0030, -0.0563, -0.0396, -0.0998, -0.0796, \u2026, 0.0002] <\/p>\n\n\n\n (The full vector is 300 numbers long\u2014to see it all click here<\/a> and then click \u201cshow the raw vector.\u201d)<\/p>\n\n\n\n Why use such a baroque notation? Here\u2019s an analogy. Washington DC<\/strong> is located at 38.9 degrees North and 77 degrees West. We can represent this using a vector notation:<\/p>\n\n\n\n This is useful for reasoning about spatial relationships. You can tell New York<\/strong> is close to Washington DC<\/strong> because 38.9 is close to 40.7 and 77 is close to 74. By the same token, Paris<\/strong> is close to London<\/strong>. But Paris<\/strong> is far from Washington DC<\/strong>.<\/p>\n\n\n\n Language models take a similar approach: each word vector<\/p>\n\n\n\n 1<\/a> represents a point in an imaginary \u201cword space,\u201d and words with more similar meanings are placed closer together. For example, the words closest to <\/a>cat<\/a><\/strong> in vector space include dog<\/strong>, kitten<\/strong>, and pet<\/strong>. A key advantage of representing words with vectors of real numbers<\/em> (as opposed to a string of letters, like \u201cC-A-T\u201d) is that numbers enable operations that letters don\u2019t. <\/p>\n\n\n\n Words are too complex to represent in only two dimensions, so language models use vector spaces with hundreds or even thousands of dimensions. The human mind can\u2019t envision a space with that many dimensions, but computers are perfectly capable of reasoning about them and producing useful results.<\/p>\n\n\n\n Researchers have been experimenting with word vectors for decades, but the concept really took off when Google announced its word2vec project<\/a> in 2013. Google analyzed millions of documents harvested from Google News to figure out which words tend to appear in similar sentences. Over time, a neural network trained to predict which words co-occur with which other words learned to place similar words (like dog<\/strong> and cat<\/strong>) close together in vector space.<\/p>\n\n\n\n Google\u2019s word vectors had another intriguing property: you could \u201creason\u201d about words using vector arithmetic. For example, Google researchers took the vector for biggest<\/strong>, subtracted big<\/strong>, and added small<\/strong>. The word closest to the resulting vector was smallest<\/strong>.<\/p>\n\n\n\n You can use vector arithmetic to draw analogies! In this case big<\/strong> is to biggest<\/strong> as small<\/strong> is to smallest<\/strong>. Google\u2019s word vectors captured a lot of other relationships:<\/p>\n\n\n\n Because these vectors are built from the way humans use words, they end up reflecting many of the biases that are present in human language<\/a>. For example, in some word vector models, doctor<\/strong> minus man<\/strong> plus woman<\/strong> yields nurse<\/strong>. Mitigating biases like this is an area of active research.<\/p>\n\n\n\n Nevertheless, word vectors are a useful building block for language models because they encode subtle but important information about the relationships between words. If a language model learns something about a cat<\/strong> (for example: it sometimes goes to the vet), the same thing is likely to be true of a kitten<\/strong> or a dog<\/strong>. If a model learns something about the relationship between Paris<\/strong> and France<\/strong> (for example: they share a language) there\u2019s a good chance that the same will be true for Berlin<\/strong> and Germany<\/strong> and for Rome <\/strong>and Italy<\/strong>.<\/p>\n\n\n\n A simple word vector scheme like this doesn\u2019t capture an important fact about natural language: words often have multiple meanings.<\/p>\n\n\n\n For example, the word bank<\/strong> can refer to a financial institution or<\/em> to the land next to a river. Or consider the following sentences:<\/p>\n\n\n\n The meanings of magazine <\/strong>in these sentences are related but subtly different. John picks up a physical<\/em> magazine, while Susan works for an organization that publishes<\/em> physical magazines. <\/p>\n\n\n\n When a word has two unrelated meanings, as with bank<\/strong>, linguists call them homonyms. When a word has two closely related meanings, as with magazine<\/strong>, linguists call it polysemy. <\/p>\n\n\n\n LLMs like ChatGPT are able to represent the same word with different vectors depending on the context in which that word appears. There\u2019s a vector for bank<\/strong> (financial institution) and a different vector for bank<\/strong> (of a river). There\u2019s a vector for magazine<\/strong> (physical publication) and another for magazine<\/strong> (organization). As you might expect, LLMs use more similar vectors<\/a> for polysemous meanings than for homonymous meanings.<\/p>\n\n\n\n So far we haven\u2019t said anything about how<\/em> language models do this\u2014we\u2019ll get into that shortly. But we\u2019re belaboring these vector representations because it\u2019s fundamental to understanding how language models work.<\/p>\n\n\n\n Traditional software is designed to operate on data that\u2019s unambiguous. If you ask a computer to compute \u201c2 + 3,\u201d there\u2019s no ambiguity about what 2,<\/strong> +,<\/strong> or 3<\/strong> mean. But natural language is full of ambiguities that go beyond homonyms and polysemy:<\/p>\n\n\n\n People resolve ambiguities like this based on context, but there are no simple or deterministic rules for doing this. Rather, it requires understanding facts about the world. You need to know that mechanics typically fix customers\u2019 cars, that students typically do their own homework, and that fruit typically doesn\u2019t fly.<\/p>\n\n\n\n Word vectors provide a flexible way for language models to represent each word\u2019s precise meaning in the context of a particular passage. Now let\u2019s look at how they do that.Subscribe<\/p>\n\n\n\n GPT-3, the model behind the original version of ChatGPT<\/p>\n\n\n\n 2<\/a>, is organized into dozens of layers. Each layer takes a sequence of vectors as inputs\u2014one vector for each word in the input text\u2014and adds information to help clarify the meaning of that word and better predict which word might come next.<\/p>\n\n\n\n Let\u2019s start by looking at a stylized example:<\/p>\n\n\n\n Each layer of an LLM is a transformer, a neural network architecture that was first introduced by Google in a landmark 2017 paper<\/a>.<\/p>\n\n\n\n The model\u2019s input, shown at the bottom of the diagram, is the partial sentence \u201cJohn wants his bank to cash the.\u201d These words, represented as word2vec-style vectors, are fed into the first transformer. <\/p>\n\n\n\n The transformer figures out that wants<\/strong> and cash<\/strong> are both verbs (both words can also be nouns). We\u2019ve represented this added context as red text in parentheses, but in reality the model would store it by modifying the word vectors in ways that are difficult for humans to interpret. These new vectors, known as a hidden state, are passed to the next transformer in the stack.<\/p>\n\n\n\n The second transformer adds two other bits of context: it clarifies that bank<\/strong> refers to a financial institution rather than a river bank, and that his<\/strong> is a pronoun that refers to John<\/strong>. The second transformer produces another set of hidden state vectors that reflect everything the model has learned up to that point.<\/p>\n\n\n\n The above diagram depicts a purely hypothetical LLM, so don\u2019t take the details too seriously. We\u2019ll take a look at research into real language models shortly. Real LLMs tend to have a lot more than two layers. The most powerful version of GPT-3, for example, has 96 layers.<\/p>\n\n\n\n Research suggests<\/a> that the first few layers focus on understanding the syntax of the sentence and resolving ambiguities like we\u2019ve shown above. Later layers (which we\u2019re not showing to keep the diagram a manageable size) work to develop a high-level understanding of the passage as a whole.<\/p>\n\n\n\n For example, as an LLM \u201creads through\u201d a short story, it appears to keep track of a variety of information about the story\u2019s characters: sex and age, relationships with other characters, past and current location, personalities and goals, and so forth.<\/p>\n\n\n\n Researchers don\u2019t understand exactly how LLMs keep track of this information, but logically speaking the model must be doing it by modifying the hidden state vectors as they get passed from one layer to the next. It helps that in modern LLMs, these vectors are extremely large.<\/p>\n\n\n\n For example, the most powerful version of GPT-3 uses word vectors with 12,288 dimensions\u2014that is, each word is represented by a list of 12,288 numbers. That\u2019s 20 times larger than Google\u2019s 2013 word2vec scheme. You can think of all those extra dimensions as a kind of \u201cscratch space\u201d that GPT-3 can use to write notes to itself about the context of each word. Notes made by earlier layers can be read and modified by later layers, allowing the model to gradually sharpen its understanding of the passage as a whole.<\/p>\n\n\n\n So suppose we changed our diagram above to depict a 96-layer language model interpreting a 1,000-word story. The 60th layer might include a vector for John<\/strong> with a parenthetical comment like \u201c(main character, male, married to Cheryl, cousin of Donald, from Minnesota, currently in Boise, trying to find his missing wallet).\u201d Again, all of these facts (and probably a lot more) would somehow be encoded as a list of 12,288 numbers corresponding to the word John<\/strong>. Or perhaps some of this information might be encoded in the 12,288-dimensional vectors for Cheryl<\/strong>, Donald<\/strong>, Boise<\/strong>, wallet<\/strong>, or other words in the story.<\/p>\n\n\n\n The goal is for the 96th and final layer of the network to output a hidden state for the final word that includes all of the information necessary to predict the next word.<\/p>\n\n\n\n Now let\u2019s talk about what happens inside each transformer. The transformer has a two-step process for updating the hidden state for each word of the input passage:<\/p>\n\n\n\n Of course it\u2019s the network, not the individual words, that performs these steps. But we\u2019re phrasing things this way to emphasize that transformers treat words, rather than entire sentences or passages, as the basic unit of analysis. This approach enables LLMs to take full advantage of the massive parallel processing power of modern GPU chips. And it also helps LLMs to scale to passages with thousands of words. These are both areas where earlier language models<\/a> struggled.<\/p>\n\n\n\n You can think of the attention mechanism as a matchmaking service for words. Each word makes a checklist (called a query vector) describing the characteristics of words it is looking for. Each word also makes a checklist (called a key vector) describing its own characteristics. The network compares each key vector to each query vector (by computing a dot product<\/a>) to find the words that are the best match. Once it finds a match, it transfers information from the word that produced the key vector to the word that produced the query vector.<\/p>\n\n\n\n For example, in the previous section we showed a hypothetical transformer figuring out that in the partial sentence \u201cJohn wants his bank to cash the,\u201d his<\/strong> refers to John.<\/strong> Here\u2019s what that might look like under the hood. The query vector for his<\/strong> might effectively say \u201cI\u2019m seeking: a noun describing a male person.\u201d The key vector for John<\/strong> might effectively say \u201cI am: a noun describing a male person.\u201d The network would detect that these two vectors match and move information about the vector for John<\/strong> into the vector for his<\/strong>.<\/p>\n\n\n\n Each attention layer has several \u201cattention heads,\u201d which means that this information-swapping process happens several times (in parallel) at each layer. Each attention head focuses on a different task:<\/p>\n\n\n\n And so forth.<\/p>\n\n\n\n Attention heads frequently operate in sequence, with the results of an attention operation in one layer becoming an input for an attention head in a subsequent layer. Indeed, each of the tasks we just listed above could easily require several attention heads rather than just one.<\/p>\n\n\n\n The largest version of GPT-3 has 96 layers with 96 attention heads each, so GPT-3 performs 9,216 attention operations each time it predicts a new word.Subscribe<\/p>\n\n\n\n In the last two sections we presented a stylized version of how attention heads work. Now let\u2019s look at research on the inner workings of a real language model. Last year scientists at Redwood Research studied<\/a> how GPT-2, a predecessor to ChatGPT, predicted the next word for the passage \u201cWhen Mary and John went to the store, John gave a drink to.\u201d<\/p>\n\n\n\n GPT-2 predicted that the next word was Mary<\/strong>. The researchers found that three types of attention heads contributed to this prediction:<\/p>\n\n\n\n In short, these nine attention heads enabled GPT-2 to figure out that \u201cJohn gave a drink to John\u201d doesn\u2019t make sense and choose \u201cJohn gave a drink to Mary\u201d instead.<\/p>\n\n\n\n We love this example because it illustrates just how difficult it will be to fully understand LLMs. The five-member Redwood team published a 25-page paper<\/a> explaining how they identified and validated these attention heads. Yet even after they did all that work, we are still far from having a comprehensive explanation for why GPT-2 decided to predict Mary<\/strong> as the next word.<\/p>\n\n\n\n For example, how did the model know the next word should be someone\u2019s name and not some other kind of word? It\u2019s easy to think of similar sentences where Mary<\/strong> wouldn\u2019t be a good next-word prediction. For example, in the sentence \u201cwhen Mary and John went to the restaurant, John gave his keys to,\u201d the logical next words would be \u201cthe valet.\u201d<\/p>\n\n\n\n Presumably, with enough research computer scientists could uncover and explain additional steps in GPT-2\u2019s reasoning process. Eventually, they might be able to develop a comprehensive understanding of how GPT-2 decided that Mary<\/strong> is the most likely next word for this sentence. But it could take months or even years of additional effort just to understand the prediction of a single word.<\/p>\n\n\n\n The language models underlying ChatGPT\u2014GPT-3.5 and GPT-4\u2014are significantly larger and more complex than GPT-2. They are capable of more complex reasoning than the simple sentence-completion task the Redwood team studied. So fully explaining how these systems work is going to be a huge project that humanity is unlikely to complete any time soon.<\/p>\n\n\n\n After the attention heads transfer information between word vectors, there\u2019s a feed-forward network<\/p>\n\n\n\n 3<\/a> that \u201cthinks about\u201d each word vector and tries to predict the next word. No information is exchanged between words at this stage: the feed-forward layer analyzes each word in isolation. However, the feed-forward layer does have access to any information that was previously copied by an attention head. Here\u2019s the structure of the feed-forward layer in the largest version of GPT-3:<\/p>\n\n\n\n The green and purple circles are neurons: mathematical functions that compute a weighted sum of their inputs.<\/p>\n\n\n\n 4<\/a><\/p>\n\n\n\n What makes the feed-forward layer powerful is its huge number of connections. We\u2019ve drawn this network with three neurons in the output layer and six neurons in the hidden layer, but the feed-forward layers of GPT-3 are much larger: 12,288 neurons in the output layer (corresponding to the model\u2019s 12,288-dimensional word vectors) and 49,152 neurons in the hidden layer.<\/p>\n\n\n\n So in the largest version of GPT-3, there are 49,152 neurons in the hidden layer with 12,288 inputs (and hence 12,288 weight parameters) for each neuron. And there are 12,288 output neurons with 49,152 input values (and hence 49,152 weight parameters) for each neuron. This means that each feed-forward layer has 49,152 * 12,288 + 12,288 * 49,152 = 1.2 billion weight parameters. And there are 96 feed-forward layers, for a total of 1.2 billion * 96 = 116 billion parameters! This accounts for almost two-thirds of GPT-3\u2019s overall total of 175 billion parameters.<\/p>\n\n\n\nWord vectors<\/h1>\n\n\n\n
\n
![\"\"\/](\"https:\/\/substackcdn.com\/image\/fetch\/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1610e32f-6e98-4d6b-85e4-14f9abda902c_1600x656.png\")
<\/a><\/figure>\n\n\n\n\n
Word meaning depends on context<\/h1>\n\n\n\n
\n
\n
Transforming word vectors into word predictions<\/h1>\n\n\n\n
![\"\"\/](\"https:\/\/substackcdn.com\/image\/fetch\/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04891f5c-6869-4413-a32f-b1176d5a232f_1600x1001.png\")
<\/a><\/figure>\n\n\n\nCan I have your attention please<\/h1>\n\n\n\n
\n
\n
A real-world example<\/h1>\n\n\n\n
\n
The feed-forward step<\/h1>\n\n\n\n
![\"\"\/](\"https:\/\/substackcdn.com\/image\/fetch\/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3e159bd-1228-4205-b1eb-5898ab9172d3_1600x856.png\")
<\/a><\/figure>\n\n\n\n