How does GPT-like transformers utilize only the decoder to do sequence generation?

In natural language processing and understanding, GPT-like transformers have become some of the most influential and versatile language models. The capabilities of the GPT (Generative Pre-trained Transformer) family, which includes GPT-2 and GPT-3, in various NLP tasks, such as text creation, translation, summarization, and more, have drawn a lot of interest. Have you ever wondered, though, how these models work? What is their trade secret? The fact that GPT-like transformers solely use the decoder for sequence generation is a crucial characteristic. We will examine this intriguing architecture in this blog, including its operation, necessity, and revolutionary effects on the language model community.

Traditional transformers, the architectural blueprint upon which GPT-like models are based, consist of both an encoder and a decoder. While the decoder creates output sequences, the encoder processes input data. But this binary division begs the question: Why do GPT-like transformers generate sequences using only the decoder? What problems does this method solve, and what are the advantages?

Understanding the (Decoder-Only) Transformer Model:

To comprehend the decoder-only transformer, it's crucial first to understand its input and output:

Input: The model receives a prompt, often called context, and processes it as a whole. There's no sequential recurrence in this process.

Output: The model's production depends on its specific task. For GPT models, the output is a probability distribution of the next token or word that should follow the input. In essence, it produces a single prediction for the entire input.

Now, let's dissect the essential components that make up the decoder-only transformer:

• The Embedding: A prompt is an input for the transformer model, and it needs to be contained so that the model can be understood and used efficiently.

• The Blocks: These are the primary components of the model's complexity. Every block comprises multiple-layer normalization processes, a feedforward network, and a masked multi-head attention submodule. The successive arrangement of these components increases the model's depth.

• The Output: The final output of the model is obtained by processing the output of the last block via an extra linear layer. This output can be a classification or a forecast for the following token or word.

Multi-Head Attention:

In essence, multi-head attention is an arrangement of separate attention heads layered on each other. Every head has its own distinct set of weights, even though all heads receive the same input. To restore the original input's dimensionality, the outputs of each head are concatenated and passed through a linear layer after the input has been processed through all of the heads.

The Self-Attention Mechanism:

The self-attention mechanism is the secret sauce that empowers the transformer. This enables the model to concentrate on the most pertinent segments of the input. We refer to each self-attention mechanism as a "head."

This is the functioning of a self-attention head:

Three distinct linear layers are applied to the input. Using a softmax activation function, two of these layers—the queries (Q) and the keys (K)—are multiplied, scaled, and converted into a probability distribution. To determine which words in the prompt are significant for predicting the next word, this distribution highlights the most important indices for the output.

The output is then multiplied by values (V) to get V multiplied by the weight given to each token in V. Associated with the three linear layers are the parameters that comprise the self-attention head.

Masked Self-Attention:

Sequence padding is what masked self-attention is in the context of the decoder-only transformer. The term "masking" is a holdover from the original encoder-decoder transformer module, in which the decoder could only see the translated portion of the sentence and the encoder could access the full sentence in the source language. As a result, they called it "masking."

The Block:

The decoder-only transformer consists of two layer-normalization operations, two skip connections, a feedforward network, and a multi-head attention submodule in each block. The feedforward network, which typically consists of a fully connected layer, a ReLU activation, another fully connected layer, and a dropout layer, is a multi-layer perceptron.

Layer normalization is applied after the "add & norm" blocks add the output of the multi-head attention and feedforward submodule to the input of these modules. This method called a skip connection, is essential for overcoming the difficulties presented by vanishing or exploding gradients.

Positional Embedding:

Unlike RNNs, which process input sequentially, transformers process entire prompts at once. The issue here is that the model needs to know by default what the word order in a sentence is. To solve this, positional embedding is included, enabling the model to ascertain each word's location within the sentence. Different embedding layers can be used to learn positional embeddings, though the original authors suggested a more involved approach that does not require parameter learning.

Output:

The output is routed through one last linear layer, following the prompt's sequential passage through each block. By mapping the model's output back to the vocabulary size, this layer enables the model to predict the next word or token.

Training:

The training process for transformer models involves self-supervised learning. Large volumes of text are used to create training data, with each text segment being divided into multiple samples. For instance, the sentence "This is a sample" can be separated into samples like ["This"], ["This", "is"], ["This", "is", "a"], with padding added to match the maximum sequence length. The model is trained to predict the last word in each sample.

In addition to basic training, fine-tuning or transfer learning is crucial in adapting the model to specific applications. During fine-tuning, the model is provided with prompts and generates different answers, which human evaluators then rank. The model's scores are backpropagated, enhancing its performance for specific tasks.

Inference (Answer Generation):

Using a transformer model for inference closely resembles training. The model generates the next word or token upon receiving a prompt. Each predicted token is added to the prompt for the following prediction in an iterative process for GPT models. The model's output can be made probabilistic or deterministic during inference by choosing a token with the highest probability or by sampling from the probability distribution.

These models can be used for various tasks beyond text generation by fine-tuning. Comprehending the internal mechanisms of these models illuminates their adaptability and capacity to revolutionize the domain of natural language processing and comprehension.

Thank you for reading the article.