Sequence-to-Sequence Models

1. The task shape

Many NLP tasks are naturally sequence-to-sequence: translation, summarization, question answering, data-to-text generation, and grammatical correction.

2. Why it mattered

Before Transformers, encoder-decoder RNNs made neural machine translation practical. Attention improved them by letting the decoder look back at source states instead of relying on one fixed vector.

3. Hugging Face practice

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

name = "Helsinki-NLP/opus-mt-en-no"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForSeq2SeqLM.from_pretrained(name)

4. My research connection

English—Norwegian MT is a seq2seq problem. Domain adaptation asks whether the model can generate technically correct target text when the source contains rare petroleum terminology and formal document style.

Takeaway

Seq2seq is the task grammar behind MT: input text becomes output text, but faithfulness must be evaluated carefully.

Interview pattern

My interview answer would usually be:

1. define the concept in one sentence;
2. explain the data flow;
3. name the main failure mode;
4. connect it to evaluation, multilinguality, or fine-tuning.

References

• Hugging Face Course
• Hugging Face Transformers documentation
• Hugging Face tokenizer summary
• Hugging Face fine-tuning guide
• Hugging Face PEFT
• The Illustrated Transformer
• Speech and Language Processing, Jurafsky & Martin
• Stanford CS224N readings
• Attention Is All You Need
• COMET: A Neural Framework for MT Evaluation