· Xiaojing Yang · Statistics
Statistical Testing for NLP and LLM Evaluation
A practical map for choosing statistical tests in NLP, MT, RAG, and LLM evaluation.
Blog
Foundations and research notes on mathematics, statistics, machine learning, NLP, LLMs, model evaluation, and applied AI systems.
- · Xiaojing Yang · NLP and LLMs
Tokenization and Subwords
Tokenization is the first modeling decision in NLP: it decides what units a model can see, how rare terms are represented, and how multilingual systems handle domain language.
- · Xiaojing Yang · Machine Learning
Train / Validation / Test Splits
A practical guide to splitting data so model evaluation stays honest.
- · Xiaojing Yang · NLP and LLMs
Transformers for NLP
How Transformers combine self-attention, feed-forward layers, residuals, and positional information.
- · Xiaojing Yang · Statistics
Why Statistics Matters for AI Research
Statistics is not just a set of formulas. It is a way to reason about uncertainty, evidence, and trust in AI experiments.
- · Xiaojing Yang · NLP and LLMs
Word Embeddings and Sentence Embeddings
How discrete language becomes vector space, and why sentence embeddings matter for retrieval and evaluation.