Encoders and decoders

A transformer can be built to do one of two jobs, and the difference comes down to what each word is allowed to look at.

An encoder reads a whole sentence at once. Every word can look at every other word, including the ones that come after it. This is good for understanding, because to know what a word means you often need the words around it, on both sides. Models that classify text are usually encoders. Answering questions about a passage used to be encoder work too; in 2026 it is mostly done by decoder-only LLMs, while encoder-decoder models survive mainly in translation and specialised pipelines.

A decoder generates text one word at a time, left to right. When it is producing a word, the words after it do not exist yet, they have not been written. So each word can only look at itself and the words before it, never ahead. This backward-only rule is called causal masking. Models that write text, like the GPT family, are decoders.

The figure shows this as a grid. Each row is a word doing the looking; each column is a word being looked at. A filled cell means this word is allowed to look at that word.

In encoder mode, every cell is filled. Every word sees the whole sentence. In decoder mode, only the lower triangle is filled. Each word sees itself and the words before it, and the empty upper triangle is everything it is not allowed to see yet.

Click a word in decoder mode and you will see it can only reach back to earlier words, never forward.

Some models are encoder-only, good at understanding. Some are decoder-only, good at generating. Some use both together, an encoder to read an input and a decoder to write an output, which is how translation works.

The grid shows what each position is allowed to look at, the masking rule, not the actual attention weights. The rule is real and computed live: in decoder mode a word can see a column only if that column is at or before its own position. This masking is the core difference between encoders and decoders. Real models also differ in other ways, like how a decoder can additionally attend to an encoder's output in translation, which this single grid does not show.

What the grid shows