Chat Overview

Encoder-Decoder Models and Seq2Seq Learning

How encoder-decoder models map one variable-length sequence to another, from the 2014 seq2seq paper to modern systems.

By Chat Overview Published Updated
The encoder builds a representation of the input; the decoder turns that representation into an output sequence.
The encoder builds a representation of the input; the decoder turns that representation into an output sequence.

An encoder-decoder model learns a mapping from one structured input to another. The encoder reads the input and builds an internal representation. The decoder uses that representation to produce the output.

The pattern sounds obvious now because it appears everywhere: translation, summarization, speech recognition, image captioning, and text-to-image generation. In 2014, it solved a specific and awkward problem. Earlier neural language systems could score or generate fixed kinds of sequences, but translating a sentence required a model that could accept one length and return another. Sequence-to-sequence learning supplied a clean end-to-end design.

Two papers established the pattern in 2014. Ilya Sutskever, Oriol Vinyals, and Quoc Le described a multilayer LSTM system in Sequence to Sequence Learning with Neural Networks. Kyunghyun Cho and colleagues developed a related recurrent encoder-decoder and introduced the gated recurrent unit. This article uses "encoder-decoder" for the general pattern and "seq2seq" for its sequence form.

Historical Context

Statistical machine translation usually consisted of several separately designed parts: translation tables, language models, alignment models, and a search procedure. Neural networks had entered parts of that pipeline, but an entire sentence pair was not yet commonly learned by one model.

Recurrent neural networks could process variable-length input, but ordinary recurrent units had trouble preserving information across many steps. LSTMs made longer dependencies more practical. The 2014 seq2seq work arranged two LSTMs back to back. One read the source sentence. The other generated the translation.

The model was trained jointly with backpropagation . It did not need a separate alignment table or hand-written translation rules. One practical detail in the paper was to reverse the word order of each source sentence. That shortened the effective distance between corresponding source and target words and improved optimization.

The Encoder

Suppose the input is a sequence of tokens x1, x2, ..., xn. The encoder converts those tokens into vectors and processes them into a representation. In the original LSTM system, the final hidden state and cell state carried the information needed by the decoder.

The encoder is not meant to reproduce the input literally. It learns features that help the output task. For translation, those features may capture word identity, order, grammatical role, and sentence meaning. Exactly what is retained depends on the objective and the capacity of the model.

In a classic fixed-vector seq2seq model, the final state is a bottleneck: every relevant part of the input must fit into one vector. This works surprisingly well for short sentences and becomes increasingly difficult as sequences grow.

The Decoder

The decoder starts from the encoder representation and predicts one output token at a time. At step t, it conditions on its current state and the preceding output tokens. A special end token tells it when to stop.

During training, the correct preceding token is commonly supplied to the decoder. This is called teacher forcing. At inference time, the correct future is unavailable, so the decoder must use its own previous prediction. Errors can therefore compound: one poor token changes the context for every later token.

The model assigns a probability to candidate output sequences. Choosing the most likely token at every step is greedy decoding. Beam search keeps several partial sequences, extends each one, and retains the most promising candidates. A larger beam explores more alternatives but costs more computation and does not guarantee a better human judgment.

The Learning Objective

For a paired example (source, target), training maximizes the probability of the target sequence given the source. This is usually implemented as the sum of token-level cross-entropy losses. Each target token contributes a signal, and backpropagation distributes those signals through the decoder and into the encoder.

The important shift was architectural and operational. Instead of training separate pipeline stages against separate objectives, the complete mapping could be trained end to end against the desired output sequence.

Attention Removes the Fixed-Vector Bottleneck

The fixed context vector soon became the obvious weak point. Bahdanau, Cho, and Bengio let the decoder inspect all encoder states. At each output step, the model calculated a weighted combination of the source representations.

This made the internal path between a source word and its likely translation much shorter. It also produced a soft alignment that could be inspected. The decoder no longer depended on one final state to preserve an entire sentence.

The attention-based encoder-decoder remained recurrent at first. The Transformer later replaced recurrence with self-attention while keeping the encoder-decoder division. Its encoder creates contextual source representations; its decoder combines causal self-attention with cross-attention over the encoder output.

Three Common Variants

The original design developed into three broad families:

  • Encoder-decoder: reads one input and generates a separate output. Translation and summarization fit naturally.
  • Encoder-only: builds contextual representations for classification, retrieval, or token labeling. BERT is the best-known example.
  • Decoder-only: predicts the continuation of a context. GPT uses this design and can express many tasks as text continuation.

These labels describe information flow, not a rigid list of products. Multimodal systems may combine a vision encoder with a language decoder. In text-conditioned diffusion systems, a text encoder supplies representations that guide a denoising network as it generates an image. Speech systems may place a feature encoder in front of a sequence decoder.

Impact and Legacy

The 2014 work showed that a single differentiable system could learn complex mappings between variable-length sequences. That result helped replace feature-heavy language pipelines with representation learning.

Encoder-decoder systems now appear in:

  • machine translation and summarization;
  • automatic speech recognition and speech translation;
  • image captioning and visual question answering;
  • code transformation and structured-data generation;
  • text-conditioned image and audio systems.

Decoder-only language models have absorbed many tasks by placing the input and output in one token stream. That does not make the encoder-decoder pattern obsolete. A separate encoder can be more efficient when an input is reused, and cross-attention provides a clear boundary between source evidence and generated output.

Practical Limits

An encoder-decoder does not guarantee faithful transformation. A summarizer can omit evidence or invent details. A translation system can produce fluent but incorrect text. The training data and objective determine what errors are rewarded or ignored.

Autoregressive decoding is also sequential. Tokens are generated one after another, which limits inference speed. Beam search and length penalties introduce additional choices. Evaluating the system therefore requires task-specific checks, not just a low training loss.

Further Reading