· Xiaojing Yang · Machine Learning · 2 min read
中文Train / Validation / Test Splits
A practical guide to splitting data so model evaluation stays honest.
Core idea
The test set is not for making decisions; it is for checking the decision after it has been made.
1. Why splitting matters
Machine learning is not only about fitting a model. It is about estimating how the model will behave on examples it has not seen. If training and evaluation share information, the score becomes too optimistic.
Fit parameters
Choose features, models, thresholds, and hyperparameters
Improve using validation evidence
Final check
Monitor real data
2. The roles
| Split | Role | What not to do |
|---|---|---|
| Training set | Learn parameters | Report it as final performance |
| Validation set | Make development choices | Treat it as untouched evidence |
| Test set | Final estimate | Reuse it for tuning |
Google MLCC has a very useful phrase: validation and test sets can effectively wear out when repeatedly used for decisions. That is a wonderful intuition for interviews.
3. sklearn example
from sklearn.model_selection import train_test_split
X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.30, random_state=42, stratify=y)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.50, random_state=42, stratify=y_temp)4. AI/NLP connection
For NLP, random splitting can be unsafe. Near-duplicate documents, translated versions, same authors, same topics, or same source documents can leak across splits. In domain MT, a sentence pair duplicated across train and test can make a system look better than it is.
Good split
Representative, deduplicated, and aligned with the real deployment population.
Bad split
Random-looking but contaminated by duplicates, time leakage, or source overlap.
Takeaway
Splitting is experimental design. A clean split protects the meaning of every score that comes later.
Interview pattern
When this appears in an interview, I would answer in four layers:
- give the short definition;
- explain the intuition;
- name the common failure mode;
- connect it to a real evaluation or deployment decision.