· Xiaojing Yang · Machine Learning · 2 min read
中文Feature Engineering vs Representation Learning
How traditional ML features connect to embeddings, neural networks, and modern NLP systems.
Core idea
Feature engineering designs inputs by hand; representation learning lets models learn useful inputs from data.
1. The bridge
Traditional ML depends heavily on feature engineering: counts, TF-IDF, metadata, ratios, and handcrafted signals. Deep learning shifts part of that work into the model by learning representations.
Text, image, table
Counts, rules, ratios
Dense learned vectors
Uses representation
Prediction, retrieval, generation
2. Comparison
| Aspect | Feature engineering | Representation learning |
|---|---|---|
| Control | high | lower |
| Data need | often lower | often higher |
| Interpretability | often clearer | often harder |
| Power | limited by design | can learn complex patterns |
| NLP example | TF-IDF | Transformer embeddings |
3. sklearn example
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
pipe = Pipeline([
("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
("clf", LogisticRegression(max_iter=1000)),
])4. AI/NLP connection
Modern NLP did not make feature thinking obsolete. It changed where features live. Tokenization, embeddings, prompts, retrieved contexts, and fine-tuning data all shape the representation the model can use.
Interview answer
Features are input variables; representations are learned features useful for downstream tasks.
Research answer
The choice of representation changes generalization, fairness, retrieval behavior, and error patterns.
Takeaway
Feature engineering and representation learning are not enemies. They are two ways of deciding what information a model can see.
Interview pattern
When this appears in an interview, I would answer in four layers:
- give the short definition;
- explain the intuition;
- name the common failure mode;
- connect it to a real evaluation or deployment decision.