ML/DL Foundations: The Complete Guide for Exams & Interviews
Everything you need to explain these concepts clearly: What, Why, How, Pros & Cons.
📝 About this blog This post covers 32 topics in Machine Learning and Deep Learning, from Linear Regression to Quantum ML. Each section gives you the intuition, key formulas, pros/cons, and interview-style takeaways so you can speak confidently in exams or technical interviews. No fluff, just the structure that works. Updated November 2026: new sections on JEPA, flow matching, LoRA, RAG, mixture of experts, DPO and GRPO, quantization, speech models and more, all in simple words and simple maths.
Table of Contents
Where to start: Basics (1–4) → Classical ML (5–8) → Sequences (9) → Deep Learning (10–13) → Transformers (14–16) → Learning Without Labels (17–18) → Generative (19–21) → LLMs (22–25) → RL and Alignment (26–27) → Beyond Transformers (28–30) → Speech (31) → Frontier (32).
Part I: Basics
1. Linear Regression
The simplest predictive model, where it all begins.
What it is: Predicts a continuous value (house price, temperature, salary) by fitting a straight line \(\hat{y} = w^{\top}x + b\).
How it works:
- Assume \(Y = wX + b\); start with random \(w\), \(b\).
- Predict on training data, compute the MSE \(= \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2\).
- Update \(w\), \(b\) with Gradient Descent (or use the Normal Equation).
- Repeat until loss is minimized.
Key formula:
Squaring penalizes large errors more.
| ✅ Pros | ❌ Cons |
|---|---|
| Simple, fast, interpretable | Only linear relationships |
| No hyperparameters; closed-form exists | Sensitive to outliers |
| Coefficients = feature importance | Multicollinearity hurts |
💡 Interview: "When would you NOT use Linear Regression?" When the relationship is non-linear, when there are outliers, or when features are highly correlated. "Regularization?" L1 (Lasso) → sparsity; L2 (Ridge) → small weights.
⚡ Quick recall: Continuous output → MSE → Gradient Descent or Normal Equation. Regularization: L1/L2.
2. Logistic Regression
Classification disguised as regression, predicting probabilities.
What it is: Despite the name, it's for classification. Outputs probability via sigmoid \(\sigma(z) = \frac{1}{1+e^{-z}} \in (0,1)\). If \(\sigma(z) > 0.5\) → Class 1, else Class 0.
How it works:
- Compute \(z = wX + b\); apply \(\sigma(z)\).
- Loss = Binary Cross-Entropy: \(\mathcal{L} = -\left[\,y\log p + (1-y)\log(1-p)\,\right]\).
- Update weights with Gradient Descent.
Key formula:
The decision boundary is linear in feature space.
| ✅ Pros | ❌ Cons |
|---|---|
| Outputs probabilities; interpretable | Linear decision boundary only |
| Fast; good for linearly separable data | Can't solve XOR-type problems |
| Multi-class via Softmax | Sensitive to outliers |
💡 Interview: "Why not MSE for classification?" MSE with sigmoid is non-convex; cross-entropy is convex and penalizes confident wrong answers more.
⚡ Quick recall: Binary classification; sigmoid → [0,1]; loss = Cross-Entropy; boundary is linear.
3. Overfitting, Bias-Variance and Regularization
Why a model can look perfect on training data and still fail on new data.
What it is: Overfitting means the model memorizes the training data, noise included, so it does badly on new data. Underfitting means the model is too simple to learn the pattern at all. Compare the training score with the test score to see which one you have.
Bias and variance in plain words:
- Bias: the model is too simple and misses the real pattern (underfitting).
- Variance: the model reacts too strongly to the exact training set, so small changes in the data change it a lot (overfitting).
- Noise: randomness in the data that nobody can predict.
Key formula:
Simple models have high bias and low variance. Complex models have low bias and high variance. You want the sweet spot in the middle.
How to fight overfitting:
- Get more data, or create more with augmentation.
- Use regularization: add a penalty for large weights so the model prefers simpler solutions.
- Use dropout, early stopping, or a smaller model.
- Check with k-fold cross-validation: split the data into \(k\) parts, train on \(k-1\) of them, test on the last one, rotate, and average the scores.
Regularization formulas:
\(\lambda\) sets the strength. A bigger \(\lambda\) gives a simpler model.
| ✅ Pros | ❌ Cons |
|---|---|
| Reduces overfitting with very little code | Adds a setting (\(\lambda\)) that you must tune |
| L1 sets some weights to exactly zero (feature selection) | Too much regularization causes underfitting |
| Cross-validation gives a more honest score | Cross-validation costs \(k\) times more training |
💡 Interview: "What is the bias-variance trade-off?" A more complex model lowers bias but raises variance. You look for the point where the error on new data is smallest. "L1 vs L2?" L1 can push weights to exactly zero (a sparse model). L2 shrinks all weights smoothly but rarely to zero.
⚡ Quick recall: High train score and low test score means overfitting. Fixes: more data, L1/L2, dropout, early stopping, cross-validation.
4. Evaluation Metrics
How do you know if your model is actually good?
What it is: Accuracy alone can fool you. If 99% of emails are not spam, a model that always says "not spam" gets 99% accuracy and is useless. So we count four things: true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN).
In plain words:
- Precision: of everything I flagged as positive, how much was really positive?
- Recall: of everything that was really positive, how much did I find?
- F1: one number that balances precision and recall.
- ROC-AUC: how well the model separates the two classes across all thresholds. 0.5 is random guessing, 1.0 is perfect.
Key formulas:
Accuracy is \(\frac{TP+TN}{TP+TN+FP+FN}\). For regression use MAE (average absolute error), RMSE (square root of the MSE, punishes big errors) and \(R^2\) (the share of variation the model explains).
| ✅ Pros | ❌ Cons |
|---|---|
| Accuracy is simple to explain | Misleading when classes are imbalanced |
| Precision and recall show the type of mistake | They trade off against each other |
| F1 gives a single balanced number | Hides which of the two is weak |
| ROC-AUC does not depend on a threshold | Can look too good on very imbalanced data (use PR-AUC) |
💡 Interview: "Cancer screening: precision or recall?" Recall, because missing a sick patient is worse than a false alarm. "Spam filter?" Precision, because sending a real email to spam is worse. "Why not just accuracy?" It hides failure on rare classes.
⚡ Quick recall: Precision = right among flagged. Recall = found among real. F1 balances both. Imbalanced data: avoid accuracy alone.
Part II: Classical ML
5. SVM
Maximum margin classifier, the widest street between classes.
What it is: Finds the hyperplane that maximizes the margin (distance to nearest points). Those points are support vectors; only they define the boundary. The kernel trick maps data to higher dimensions for non-linear boundaries without computing the mapping explicitly.
How it works:
- Maximize the margin \(\frac{2}{\lVert w\rVert}\) subject to \(y_i(w^{\top}x_i + b) \ge 1\).
- For non-linear data: use a kernel \(K(x,x') = \phi(x)^{\top}\phi(x')\), e.g. Linear, RBF, Polynomial.
- C controls trade-off (regularization): high C = less regularization, risk of overfitting.
Key formula:
Common kernels: RBF, Polynomial.
| ✅ Pros | ❌ Cons |
|---|---|
| Effective in high dimensions | Slow on large data \(O(n^2)\)–\(O(n^3)\) |
| Only support vectors matter | No direct probability output |
| Kernel trick → non-linear boundaries | Kernel choice is critical |
💡 Interview: "What are support vectors?" The points closest to the decision boundary; remove others, boundary stays the same. "Kernel trick?" Dot product in high-dimensional space without explicitly transforming data.
⚡ Quick recall: Max margin; support vectors; kernel trick; C = regularization.
6. Decision Trees and Random Forest
If-then rules learned from data, highly interpretable.
What it is: A Decision Tree is a flowchart: each node is a question on a feature, branches are answers, leaves are predictions. Random Forest = many trees on bootstrap samples + random feature subsets; aggregate by voting (classification) or averaging (regression).
How it works:
- Split criterion: Gini \(= 1 - \sum_i p_i^2\) or Entropy \(= -\sum_i p_i \log_2 p_i\). Maximize Information Gain.
- Random Forest: Bagging (bootstrap samples) + random feature subset at each split → decorrelated trees, lower variance.
- ~37% of data is out-of-bag per tree → free validation.
Key formulas: \(\text{Gini} = 0\) means a pure node.
Information Gain is the parent's entropy minus the weighted average entropy of the children.
| ✅ Pros | ❌ Cons |
|---|---|
| No scaling; handles non-linearity | Single tree overfits easily |
| Mixed feature types; feature importance | RF loses interpretability |
| RF very hard to overfit | Can be slow at inference |
💡 Interview: "How does RF reduce overfitting?" Bagging reduces variance; random features decorrelate trees so errors don't add up.
⚡ Quick recall: Gini/Entropy → split; RF = Bagging + random features; OOB ≈ 37%.
7. Gradient Boosting (XGBoost)
Many small trees, each one fixing the mistakes of the last.
What it is: Boosting builds models one after another. Start with a simple guess, such as the average. Then each new small tree learns the errors (residuals) that the current model still makes, and gets added with a small step size. Random Forest builds trees side by side and averages them. Boosting builds them in a row, each one correcting the last. Popular libraries: XGBoost, LightGBM, CatBoost.
How it works:
- Start with \(F_0(x)\) = the average of \(y\).
- Compute the errors: \(r_i = y_i - F_{m-1}(x_i)\).
- Train a small tree \(h_m\) to predict those errors.
- Add it to the model with a small learning rate.
- Repeat for \(M\) rounds.
Key formula:
\(\eta\) (eta) is the learning rate, usually between 0.01 and 0.3. For squared error, the residual is exactly the negative gradient of the loss, which is why it is called gradient boosting.
| ✅ Pros | ❌ Cons |
|---|---|
| Often the best model on tabular data | Trees are built in sequence, so training is slower than Random Forest |
| Handles mixed feature types and missing values (XGBoost) | Many settings to tune |
| Gives feature importance | Overfits with too many rounds or noisy data |
💡 Interview: "Random Forest vs boosting?" Random Forest averages independent trees to cut variance. Boosting adds trees in sequence to cut bias. "Why a small learning rate?" Each tree helps a little, so the model learns slowly and overfits less, but you need more trees.
⚡ Quick recall: Start with the average, fit trees to the errors, add them with step \(\eta\). Best default for tabular data.
8. k-Means and PCA
Two classic ways to find structure when you have no labels.
k-Means (clustering): group points into \(k\) clusters.
- Pick \(k\) random centers.
- Assign each point to its nearest center.
- Move each center to the average of its points.
- Repeat until nothing changes.
k-Means tries to make this total squared distance to the centers as small as possible. Choose \(k\) with the elbow method or the silhouette score.
PCA (dimensionality reduction): find the directions in which the data varies the most, keep the top few, and project the data onto them.
- Subtract the mean from every feature.
- Compute the covariance matrix.
- Find its eigenvectors. The ones with the biggest eigenvalues are the principal components.
- Keep the top \(d\) and project.
Here \(W\) holds the top \(d\) eigenvectors as columns. The share of variance kept by a component is its eigenvalue divided by the sum of all eigenvalues.
| ✅ Pros | ❌ Cons |
|---|---|
| k-Means is simple and fast | You must choose \(k\), and it assumes round clusters of similar size |
| PCA removes noise and speeds up other models | PCA is linear only, and the components are hard to interpret |
| Both work without labels | Both need features on the same scale |
💡 Interview: "Is k-Means supervised?" No, it uses no labels. "PCA vs t-SNE?" PCA is linear, fast and keeps global structure. t-SNE and UMAP are non-linear and mainly for visualization. "Why scale before PCA?" A feature with big numbers would dominate the variance.
⚡ Quick recall: k-Means = assign to nearest center, move centers, repeat. PCA = keep the directions of biggest variance. Scale features first.
Part III: Sequences (Pre–Deep Learning)
9. Hidden Markov Models
Sequences with hidden states, the pre–deep-learning way.
What it is: States are hidden; we only see observations (emissions). Markov property: next state depends only on current state. Components: States \(S\), Observations \(O\), Transition \(A\), Emission \(B\), Initial \(\pi\).
Three problems:
- Evaluation: \(P(\text{observations})\) → Forward algorithm.
- Decoding: Most likely state sequence → Viterbi.
- Learning: Estimate \(A\), \(B\), \(\pi\) → Baum–Welch (EM).
| ✅ Pros | ❌ Cons |
|---|---|
| Solid probabilistic framework | Only one-step memory (Markov) |
| Efficient dynamic programming | States and structure hand-designed |
| Handles variable length | Largely superseded by RNNs/Transformers |
💡 Interview: "HMM vs RNN?" HMM: discrete states, limited memory. RNN: continuous hidden state, learns features, longer memory (e.g. LSTM).
⚡ Quick recall: Hidden states + emissions + Markov; Forward, Viterbi, Baum–Welch.
Part IV: Deep Learning Core
10. Neural Networks (MLP)
The foundation of deep learning, learning features automatically.
What it is: Layers of neurons: \(\text{output} = \text{activation}(w \cdot \text{inputs} + b)\). Non-linear activation (ReLU, sigmoid, tanh) is essential. Without it, stacking layers is just one linear map. Backpropagation = chain rule to compute gradients; Gradient Descent updates weights.
Key ideas:
- Vanishing gradient: Deep nets with sigmoid/tanh → tiny gradients in early layers. ReLU mitigates this.
- Regularization: Dropout, L2, BatchNorm, Early Stopping.
- Optimizers: Adam/AdamW (adaptive learning rate) are default choices.
Key formulas: a single neuron, and the weight update.
| ✅ Pros | ❌ Cons |
|---|---|
| Learns features; universal approximator | Needs lots of data |
| Works for any modality | Many hyperparameters; can overfit |
| Transfer learning | "Black box" |
💡 Interview: "Why non-linear activation?" Without it, any number of layers = one linear transformation. "Dropout at test time?" No; use all units, scale if needed.
⚡ Quick recall: MLP = layers + activation; Backprop = chain rule; ReLU; Dropout, BatchNorm, Adam.
11. Autoencoders and VAE
Compression and reconstruction, the bridge to generative models.
What it is: Autoencoder: Encoder compresses \(x \to z\) (bottleneck), Decoder reconstructs \(z \to \hat{x}\). Loss \(= \lVert x - \hat{x}\rVert^2\). VAE: Encoder outputs a distribution \((\mu, \sigma)\); sample \(z = \mu + \sigma \cdot \epsilon\) (reparameterization); Decoder reconstructs. Loss = Reconstruction + KL\(\left(q(z\mid x)\,\Vert\,p(z)\right)\) so the latent stays near \(\mathcal{N}(0,1)\).
Key idea: VAE's latent space is continuous; sampling gives generation. Standard AE does not generate meaningfully.
| ✅ Pros | ❌ Cons |
|---|---|
| Unsupervised; dimensionality reduction | Reconstructions often blurry (VAE) |
| Anomaly detection (high recon error) | Posterior collapse risk in VAE |
| VAE: generative; smooth latent space | Tuning bottleneck size |
💡 Interview: "Why are VAE outputs blurry?" They optimize average likelihood; GANs optimize adversarially and often look sharper but less stable.
⚡ Quick recall: AE = encode + decode; VAE = encode to \((\mu,\sigma)\), sample \(z\), decode; loss = Recon + KL.
12. CNNs
The architecture that revolutionized vision.
What it is: Convolution = slide small filters over the image; weight sharing + local connectivity → far fewer parameters than a fully connected MLP. Stack: Conv → ReLU → Pooling → … → FC for classification.
Concepts: Stride, padding, receptive field. Pooling (e.g. max) downsamples and adds translation invariance. ResNet = skip connections \(F(x)+x\) so gradients flow and very deep nets train.
Key formula:
| ✅ Pros | ❌ Cons |
|---|---|
| Parameter efficient; translation invariant | Not rotation/scale invariant by default |
| Hierarchical features; strong transfer learning | Pooling loses some spatial info |
| ResNet enables 100+ layers | ViT competing in some settings |
💡 Interview: \"Why \(3\times 3\) filters?\" Two \(3\times 3\) filters have the same receptive field as one \(5\times 5\), fewer params, more non-linearity. "ResNet skip?" Gradient flows through identity; layer learns the residual \(F(x)\).
⚡ Quick recall: Conv → ReLU → Pool → FC; weight sharing; ResNet = skip connection; transfer learning.
13. RNN, LSTM, GRU
Neural networks with memory, processing sequences.
What it is: RNN: Hidden state \(h_t = \tanh(W_{hh}h_{t-1} + W_{xh}x_t + b)\); carries information across time. LSTM: Cell state + 3 gates (forget, input, output) so gradients can flow over long sequences. GRU: 2 gates (reset, update), fewer parameters, often similar performance.
Limitation: Sequential processing → can't parallelize over time; slow for long sequences. This is why Transformers (parallel + attention) took over.
| ✅ Pros | ❌ Cons |
|---|---|
| Variable-length sequences; parameter reuse | Sequential → slow; vanishing gradient (vanilla RNN) |
| LSTM/GRU handle longer dependencies | Replaced by Transformers in most NLP |
| Can generate sequences | Hard to interpret hidden state |
💡 Interview: "Why did Transformers replace RNNs?" Parallelization; attention gives direct long-range connections; no vanishing gradient along sequence.
⚡ Quick recall: RNN: \(h_t = f(h_{t-1}, x_t)\); LSTM: 3 gates + cell state; GRU: 2 gates; sequential = bottleneck.
Part V: Transformers
14. Attention Mechanism
"Look at what matters", the breakthrough behind Transformers.
What it is: Instead of compressing the whole sequence into one vector (like RNN), attention lets each position look at all others and form a weighted sum of values. Query (what I need), Key (what I have), Value (content to return). Scores \(= \dfrac{QK^{\top}}{\sqrt{d_k}}\); softmax → weights; weighted sum of V.
Formula:
Scaling by \(\sqrt{d_k}\) prevents softmax saturation.
Types: Self-attention (Q,K,V from same sequence); cross-attention (Q from one, K,V from another); causal/masked (no looking ahead, for generation). Multi-head = several attention heads in parallel.
| ✅ Pros | ❌ Cons |
|---|---|
| Parallel; direct long-range links | \(O(n^2)\) time and memory |
| No vanishing gradient along sequence | Need positional encoding |
| Interpretable attention weights | Costly for very long sequences |
💡 Interview: \"Why \(\sqrt{d_k}\)?\" Dot products grow with dimension; scaling keeps logits in a reasonable range so softmax doesn't saturate.
⚡ Quick recall: Attention \(= \text{softmax}(QK^{\top}/\sqrt{d_k})V\); self vs cross; causal for decoder; \(O(n^2)\).
15. Transformers
"Attention is all you need", the backbone of modern AI.
What it is: Stack of Multi-Head Self-Attention + Feed-Forward (two linear layers with ReLU) + Residual + LayerNorm. Positional encoding (sinusoidal or learned) injects order. No recurrence, no convolution.
Variants:
- Encoder-only (e.g. BERT): bidirectional, Masked LM; understanding, classification.
- Decoder-only (e.g. GPT): causal attention, next-token prediction; generation, chat.
- Encoder–decoder (e.g. T5): both + cross-attention; translation, summarization.
Key formula:
The inner dimension is often \(4\times\) the model dimension.
| ✅ Pros | ❌ Cons |
|---|---|
| Fully parallel; long-range; scales well | \(O(n^2)\) attention |
| Same architecture for text, vision, audio | Needs lots of data; context length limited |
| Emergent abilities at scale | Position encoding limits extrapolation |
💡 Interview: "BERT vs GPT?" BERT: bidirectional, MLM, understanding. GPT: causal, next-token, generation; can do both with prompting.
⚡ Quick recall: Transformer = Self-Attention + FFN + Residual + LayerNorm; positional encoding; BERT vs GPT vs T5.
16. Vision Transformers (ViT)
Treat image patches like words.
What it is: A Transformer needs a sequence. ViT turns an image into one by cutting it into square patches (for example \(16\times16\) pixels). Each patch is flattened and projected into a vector, a position embedding is added, and a special [CLS] token is put in front. Then a standard Transformer encoder runs, and the [CLS] output is used for classification.
Key formula: the number of patches for an image of height \(H\) and width \(W\) with patch size \(P\):
A \(224\times224\) image with \(P=16\) gives \(N = 196\) patches.
| ✅ Pros | ❌ Cons |
|---|---|
| Scales very well with lots of data | Needs a lot of data or pretraining (no built-in sense of locality) |
| Same architecture as text, so images and text mix easily | Attention cost grows as \(O(N^2)\) with the number of patches |
| Every patch can see every other patch from layer 1 | High-resolution images are expensive |
💡 Interview: "ViT vs CNN?" A CNN has locality and translation invariance built in, so it works well with less data. ViT learns these from data, so it wins when data and compute are large. "Why position embeddings?" Attention alone has no sense of order or place.
⚡ Quick recall: Image to patches to embeddings, plus position and [CLS], then a Transformer. Great at scale, hungry for data.
Part VI: Learning Without Labels
17. Self-Supervised and Contrastive Learning
Learning from data with no human labels, by inventing the task.
What it is: Labels are expensive, but raw data is everywhere. In self-supervised learning the data provides its own "labels". There are three main styles:
- Fill in the missing part: BERT hides words, GPT predicts the next word, MAE hides image patches.
- Contrastive: pull things that belong together close, push other things apart. SimCLR treats two augmented views of one image as a pair. CLIP treats an image and its caption as a pair.
- Predict in embedding space: see JEPA in the next section.
Key formula: the contrastive loss (InfoNCE). For one anchor \(z\), one true partner \(z^{+}\) and many others \(z_k\):
In words: make the right match score higher than all the wrong matches. \(\text{sim}\) is cosine similarity and \(\tau\) (tau) is a temperature that controls how sharp the comparison is.
CLIP in one line: trained on hundreds of millions of image and caption pairs. For zero-shot classification, compare the image embedding with text embeddings such as "a photo of a dog" and pick the closest.
| ✅ Pros | ❌ Cons |
|---|---|
| No human labels needed | Contrastive methods need many negatives (large batches) |
| Features can be reused for many tasks | Results depend on the choice of augmentations |
| CLIP gives zero-shot classification and search | Can collapse (all outputs identical) if badly set up |
💡 Interview: "What is representation collapse?" The model outputs the same vector for everything, which makes the loss easy but the features useless. "Why a temperature?" A small \(\tau\) makes the model focus on the hardest wrong matches. "How does CLIP do zero-shot?" It compares an image embedding with embeddings of text prompts.
⚡ Quick recall: Make your own labels. Contrastive = pull pairs together, push others apart. CLIP = image plus caption pairs.
18. JEPA (Joint Embedding Predictive Architecture)
Predict the idea of what is hidden, not every pixel.
What it is: Yann LeCun proposed JEPA in 2022 as a building block for "world models". The idea: from the visible part \(x\) of an image or video, predict the embedding (the abstract summary) of the hidden part \(y\), not its pixels or words. Pixels contain details nobody can predict, like leaf wiggles and noise. Predicting abstract features lets the model ignore those and focus on meaning.
Three parts:
- Context encoder: reads the visible part and gives \(s_x\).
- Target encoder: reads the hidden part and gives the true embedding \(s_y\).
- Predictor: guesses \(\hat{s}_y\) from \(s_x\).
Key formulas:
The first is the loss: how far the guess is from the true embedding. The second says the target encoder \(\bar{\theta}\) slowly follows the context encoder \(\theta\) (moving average, with \(m\) close to 1). This helps prevent collapse.
The main danger is collapse: if both encoders output the same vector for everything, the loss is zero but nothing is learned. Fixes: use the slow-moving target encoder with stop-gradient (I-JEPA, V-JEPA), or add a regularizer that keeps embeddings spread out (LeJEPA, called SIGReg).
The JEPA family:
- I-JEPA: images.
- V-JEPA (2024): video.
- V-JEPA 2 (2025): pretrained on over a million hours of video. A version trained further on under 62 hours of robot video could plan pick-and-place tasks on real robot arms without extra training.
- V-JEPA 2.1 (2026): better dense (per-patch) features.
- LeJEPA: a simpler, better-grounded training objective with one main setting to tune.
- VL-JEPA: vision-language model that predicts the embedding of the answer text instead of generating it token by token.
| ✅ Pros | ❌ Cons |
|---|---|
| Ignores unpredictable detail, so training is efficient | Does not produce pixels or text by itself (needs a decoder) |
| Learns meaningful features for video, robots and planning | Collapse must be handled carefully |
| Needs no negatives or hand-made augmentations | Still mostly research, far less used than LLMs |
💡 Interview: "JEPA vs generative models?" Generative models rebuild the raw data (pixels or tokens). JEPA predicts only the abstract embedding. Whether this will replace generative models is still debated. "How do you avoid collapse?" A slow-moving target encoder with stop-gradient, or an explicit regularizer like SIGReg. "Why predict in embedding space?" It skips details that cannot be predicted anyway.
⚡ Quick recall: See part, predict the embedding of the hidden part. Loss = distance between embeddings. Danger = collapse. Family: I-JEPA, V-JEPA 1/2/2.1, LeJEPA, VL-JEPA.
Part VII: Generative Models
19. GANs
Generator vs Discriminator, adversarial training.
What it is: Generator \(G(z)\) maps noise → fake data; Discriminator \(D(x)\) says real vs fake. Minimax: \(D\) maximizes correct classification; \(G\) minimizes \(D(G(z))\) (fool \(D\)). The generator never sees real data directly, only gradients from \(D\).
Issues: Mode collapse (\(G\) outputs few modes); training instability; no explicit likelihood.
| ✅ Pros | ❌ Cons |
|---|---|
| Sharp, realistic samples | Unstable training; mode collapse |
| Fast sampling (one forward pass) | No likelihood; hard to evaluate |
| StyleGAN for faces; DCGAN baseline | Largely superseded by Diffusion for images |
💡 Interview: "Mode collapse?" \(G\) finds a few outputs that fool \(D\) and keeps producing only those. "GAN vs Diffusion?" Diffusion: stable, diverse, slower. GAN: fast, sharp, unstable.
⚡ Quick recall: \(G\): \(z \to\) fake; \(D\): real/fake; minimax; mode collapse; FID/IS for evaluation.
20. Diffusion Models
Denoise step by step, from noise to image.
What it is: Forward: Gradually add Gaussian noise to data until pure noise. Reverse (learned): Network predicts the added noise; subtract it step by step to get back to data. Network input: noisy image + timestep \(t\). Latent diffusion (e.g. Stable Diffusion) runs in VAE latent space → faster.
Training: minimize the noise-prediction error:
Sampling: Start from noise; denoise \(T\) steps (e.g. 50–1000). Classifier-free guidance amplifies conditioning (e.g. text) at inference.
| ✅ Pros | ❌ Cons |
|---|---|
| State-of-the-art quality; stable training | Slow sampling (many steps) |
| High diversity; no mode collapse | High compute and memory |
| Strong theory; good conditioning | Many hyperparameters |
💡 Interview: "Why predict noise?" Easier and more stable than predicting the image directly. "Classifier-free guidance?" Train with and without condition; at inference, extrapolate toward conditional output.
⚡ Quick recall: Forward = add noise; reverse = denoise; predict ε; latent diffusion; classifier-free guidance.
21. Flow Matching
Draw a straight line from noise to data, and learn which way to move.
What it is: A newer alternative to diffusion, used in Stable Diffusion 3 and FLUX. Instead of learning to remove noise step by step, the model learns a velocity: at each point and time, which direction to move to go from noise toward a real image.
How it works:
- Take a noise sample \(x_0\) and a real data sample \(x_1\).
- Pick a random time \(t\) between 0 and 1.
- Place a point on the straight line between them.
- The right velocity is the same everywhere on that line: \(x_1 - x_0\).
- Train the network to predict that velocity.
Key formulas:
Sampling: start from pure noise and take small steps along the predicted velocity (Euler steps):
Straight paths are easy to follow, so far fewer steps are needed than with classic diffusion (a handful instead of dozens or hundreds).
| ✅ Pros | ❌ Cons |
|---|---|
| Very simple training loss (plain regression) | Still needs several steps unless distilled |
| Fewer sampling steps than diffusion | Needs large compute and data for top quality |
| Works with the same Transformer backbones | Real paths are only approximately straight on complex data |
💡 Interview: "Diffusion vs flow matching?" Diffusion predicts the noise along a curved, random path. Flow matching predicts a velocity along a straight, deterministic path and samples by solving an ODE. "Why fewer steps?" A solver can cross a straight line in a few big steps.
⚡ Quick recall: \(x_t=(1-t)x_0+tx_1\), target velocity \(x_1-x_0\), sample with Euler steps. Used in SD3 and FLUX.
Part VIII: Large Language Models
22. How LLMs Are Trained
From raw text to a helpful assistant, in stages.
What it is: A large language model is trained in steps:
- Tokenization: split text into small pieces called tokens (often word parts).
- Pretraining: predict the next token over trillions of tokens. The model learns language and a lot of facts.
- SFT (instruction tuning): train on examples of a prompt and a good answer, so it follows instructions.
- Preference and RL training: DPO, RLHF or GRPO (see Part IX) to make answers helpful, safe and better at reasoning.
Key formula: the pretraining loss is the average of how surprised the model is by each next token:
Scaling laws: loss falls smoothly as you add model size \(N\) and data \(D\):
In words: an unavoidable floor \(E\), plus one term that shrinks as the model grows, plus one that shrinks as data grows. The Chinchilla result says that for a fixed compute budget, model size and data should grow together (roughly 20 tokens per parameter).
KV cache: when generating, the model stores the keys and values of earlier tokens so it does not recompute them for every new token. It makes generation fast, but memory grows with context length.
| ✅ Pros | ❌ Cons |
|---|---|
| One model handles many tasks | Pretraining is extremely expensive |
| Improves in a predictable way with scale | Can repeat errors and biases from its data |
| Can be adapted cheaply (SFT, LoRA, RAG) | Can hallucinate, and knowledge stops at the training cutoff |
💡 Interview: "Pretraining vs fine-tuning?" Pretraining teaches general language and knowledge from huge raw text. Fine-tuning teaches a specific behavior or task from a small curated dataset. "What is hallucination?" The model writes fluent text that is false, because it predicts likely words, not verified facts. "Why a KV cache?" To avoid recomputing attention over the whole past for every new token.
⚡ Quick recall: Tokens, pretrain (next token), SFT, preference or RL. Scaling laws: bigger model plus more data lowers loss. KV cache speeds generation.
23. LoRA and Efficient Fine-Tuning
Teach a big model a new skill by training only a tiny add-on.
What it is: Full fine-tuning updates every weight, which needs a lot of memory. LoRA (Low-Rank Adaptation) freezes the original weight matrix \(W\) and learns only a small change \(\Delta W\), written as the product of two thin matrices:
The rank \(r\) is small (for example 8 to 64). The layer output becomes:
\(\alpha\) is a scaling constant. \(B\) starts at zero, so training begins exactly at the original model.
How small is it? For \(d=k=4096\) and \(r=8\), LoRA trains \(r(d+k) = 65{,}536\) numbers instead of \(dk \approx 16.8\) million, about 0.4% of the layer.
After training: merge it back with \(W' = W + \frac{\alpha}{r}BA\), so there is no extra cost at inference. Or keep adapters separate and swap them per task.
QLoRA: store the base model in 4-bit and train LoRA adapters on top, so large models can be fine-tuned on a single GPU.
| ✅ Pros | ❌ Cons |
|---|---|
| Trains a tiny fraction of the weights | Can be slightly worse than full fine-tuning on hard tasks |
| Low memory, small files (megabytes) | You must choose the rank and which layers to adapt |
| Can be merged, or swapped per task | Not great at teaching large amounts of new knowledge |
💡 Interview: "Why low rank?" The changes needed to adapt a pretrained model tend to lie in a small number of directions. "Does LoRA slow inference?" No, if you merge the adapter into \(W\). "What does QLoRA add?" A 4-bit base model, so the memory need drops further.
⚡ Quick recall: Freeze \(W\), learn \(BA\) with small rank \(r\). Merge after training. QLoRA = 4-bit base plus LoRA.
24. RAG (Retrieval-Augmented Generation)
Let the model look things up before it answers.
What it is: An LLM only knows what it saw in training, and it can be wrong or out of date. RAG fetches relevant documents at question time and puts them into the prompt.
How it works:
- Index: cut documents into chunks, turn each chunk into an embedding vector, and store them in a vector database.
- Retrieve: embed the question and find the top-\(k\) most similar chunks.
- Augment: place those chunks and the question in the prompt.
- Generate: the model answers using the chunks, ideally with citations.
Key formula: similarity between a question vector \(q\) and a chunk vector \(d\) (cosine similarity):
Common upgrades: mix keyword search (BM25) with embeddings, rerank the results with a stronger model, chunk documents carefully, and rewrite the user's query before searching.
| ✅ Pros | ❌ Cons |
|---|---|
| Uses fresh or private data without retraining | Answer quality is limited by retrieval quality |
| Fewer hallucinations, and sources can be cited | More moving parts and extra latency |
| Cheap to update (just re-index) | Bad chunking can lose important context |
💡 Interview: "RAG vs fine-tuning?" Use RAG for knowledge that changes or is private. Use fine-tuning for style, format or skills. "Why not paste everything into the prompt?" It costs more, is slower, hits context limits, and models can miss facts buried in the middle of long contexts.
⚡ Quick recall: Chunk, embed, retrieve top-\(k\) by cosine similarity, add to the prompt, generate. RAG for facts, fine-tuning for behavior.
25. Mixture of Experts (MoE)
A huge model where only a few parts work on each token.
What it is: In a Transformer, replace each feed-forward block with many small feed-forward networks called experts, plus a small router. For each token, the router picks the top-\(k\) experts (for example 2 or 8). The token goes through only those, and their outputs are mixed using the router's weights. The model has a very large number of total parameters, but only a small number are active for each token, so compute per token stays low.
Key formula:
\(E_i\) is expert \(i\) and \(g_i(x)\) is the router's weight for it. A load-balancing loss keeps the router from sending everything to the same few experts, and many models also add a shared expert that every token always uses.
Real examples (2026): Qwen3-Next has about 80B total parameters with only about 3B active per token. NVIDIA's Nemotron 3 Super has 120B total with 12B active.
| ✅ Pros | ❌ Cons |
|---|---|
| More knowledge and capacity for the same compute | All experts still have to fit in memory |
| Fast per token, since few parameters are active | Routing can be unstable or unbalanced |
| Makes trillion-parameter models practical | Harder to train and serve across many GPUs |
💡 Interview: "Total vs active parameters?" Total is the size of the whole model. Active is how many are actually used for one token, and that sets the compute cost. "What does the router do?" It scores the experts for each token and picks the top-\(k\). "Why a load-balancing loss?" Otherwise a few experts get all the tokens and the rest are wasted.
⚡ Quick recall: Many experts, router picks top-\(k\), mix outputs. Big total size, small active size. Needs load balancing.
Part IX: Reinforcement Learning and Alignment
26. Reinforcement Learning
Learning by trial and error, and the basis of RLHF.
What it is: Agent in environment takes actions, gets rewards. Goal: maximize cumulative discounted reward \(\sum_{t} \gamma^{t} r_t\). MDP: states, actions, rewards, policy \(\pi(a\mid s)\), value \(V(s)\), \(Q(s,a)\).
Families: Value-based (e.g. DQN): learn \(Q\), act by argmax. Policy-based (e.g. REINFORCE): learn \(\pi\) directly. Actor–Critic (e.g. PPO): policy + value; PPO clips updates for stability → used in RLHF.
RLHF pipeline: (1) SFT on demonstrations, (2) Reward model from human preferences, (3) RL (e.g. PPO) to maximize reward with KL penalty toward SFT so the model doesn't "hack" the reward.
| ✅ Pros | ❌ Cons |
|---|---|
| No labels; long-term optimization | Sample inefficient |
| AlphaGo, games; RLHF for LLMs | Reward design and hacking risk |
| PPO stable, used in ChatGPT training | Sensitive to hyperparameters |
💡 Interview: "Why PPO for RLHF?" Stable, works with large action spaces (tokens), clipping avoids catastrophic updates. "Why KL penalty?" Keeps output close to SFT; reduces reward hacking.
⚡ Quick recall: Agent, environment, reward; value vs policy; PPO; RLHF = SFT → Reward model → PPO + KL.
Next: how RL is used to train language models (DPO, GRPO), in the next section.
27. RLHF, DPO and GRPO (RL for Language Models)
How to teach a language model what "good" means.
What it is: After pretraining and SFT, a model can still give unhelpful or unsafe answers, or fail at reasoning. Three recipes use feedback to improve it:
- RLHF with PPO: learn a reward model from human preferences, then use PPO to raise the reward, with a KL penalty to stay close to the SFT model (see the Reinforcement Learning section).
- DPO (Direct Preference Optimization, 2023): skip the reward model and the RL loop. Train directly on pairs of a chosen answer \(y_w\) and a rejected answer \(y_l\).
- GRPO (Group Relative Policy Optimization): introduced in DeepSeekMath (2024) and used in DeepSeek-R1 (2025). For each prompt, sample a group of answers, score them, and compare each one to the group average. No separate critic (value) network is needed.
DPO loss:
In words: make the model like the chosen answer more than the reference model does, and the rejected answer less. \(\beta\) controls how far it may move from the reference.
GRPO advantage: for a group of \(G\) answers (often 8 to 16) with rewards \(r_1,\dots,r_G\):
Tiny example: 4 answers, rewards 1, 0, 0, 1 (1 = correct). The mean is 0.5 and the standard deviation is 0.5, so the advantages are \(+1, -1, -1, +1\). Correct answers are pushed up and wrong ones pushed down.
RLVR (RL with Verifiable Rewards): the reward comes from a checker instead of a human or a learned reward model. Example: a math answer matches the right one, or code passes its tests. GRPO plus verifiable rewards is a common recipe for reasoning models.
| PPO (RLHF) | DPO | GRPO | |
|---|---|---|---|
| Reward model | Learned reward model plus a critic | None, uses preference pairs | Reward function or verifier, no critic |
| Data | Generated online | Fixed set of preference pairs | Generated online, groups per prompt |
| Best for | General alignment | Cheap, stable preference tuning | Reasoning with checkable answers |
| ✅ Pros | ❌ Cons |
|---|---|
| DPO is simple and stable (no RL loop) | DPO is limited by its fixed data |
| GRPO is cheaper than PPO (no critic) | GRPO needs many samples per prompt, which is costly |
| RLVR needs no human labels | Verifiable rewards exist mainly for math and code, not essays |
💡 Interview: "DPO vs RLHF?" RLHF trains a reward model and then runs RL. DPO folds both into one simple loss on preference pairs. "Why is GRPO cheaper than PPO?" It uses the group average as the baseline, so there is no value network to train. "What is reward hacking?" The model finds ways to score high without being truly good, like exploiting a weak checker.
⚡ Quick recall: RLHF = reward model plus PPO. DPO = preference pairs, one loss. GRPO = compare answers within a group, no critic. RLVR = reward from a checker.
Part X: Beyond Transformers and Compression
28. State Space Models (Mamba)
Linear complexity alternative, \(O(n)\) instead of \(O(n^2)\).
What it is: SSM: Hidden state \(h' = Ah + Bx\); output \(y = Ch + Dx\). Discretize and compute with parallel scan (training) or recurrence (inference). Mamba: Selective SSM, where \(A\), \(B\), \(C\) depend on the input so the model can choose what to remember/forget.
Why it matters: Transformers are \(O(n^2)\) in sequence length; SSMs are \(O(n)\) → longer contexts, less memory.
2026 update: hybrids. Many new open models do not pick one side. Most of their layers are cheap linear layers (Mamba-2 or Gated DeltaNet) and a few are full attention. For example, the Qwen3-Next and Qwen3.5 family uses 3 linear layers for every 1 full-attention layer, and NVIDIA's Nemotron 3 Nano mixes 23 Mamba-2 layers, 23 mixture-of-experts layers and only 6 attention layers. You get long-context speed and keep attention's strong recall.
| ✅ Pros | ❌ Cons |
|---|---|
| \(O(n)\) complexity; efficient memory | Pure SSMs are weaker at exact recall, so hybrids add attention |
| Mamba matches Transformer quality in many settings | May not capture all attention patterns |
| Good for long context and streaming | Scaling laws less established |
💡 Interview: \"How does Mamba get \(O(n)\)?\" No full \(n\times n\) attention matrix; state recurrence or parallel scan over sequence.
⚡ Quick recall: SSM: \(h'=Ah+Bx\), \(y=Ch\); Mamba = selective (input-dependent); \(O(n)\); two modes: parallel vs recurrent.
29. Knowledge Distillation
Teaching small models to mimic large ones.
What it is: Teacher (large model) produces soft labels (e.g. class probabilities). Student (small model) is trained to match both hard labels and teacher soft labels. Temperature \(T>1\) softens softmax so "dark knowledge" (relationships between classes) is visible. The loss combines both signals:
Why: Deploy on edge, reduce latency and cost. E.g. DistilBERT: ~40% smaller, ~60% faster, ~97% of BERT performance.
| ✅ Pros | ❌ Cons |
|---|---|
| Big size/speed gain, small quality loss | Need a trained teacher |
| Works for any architecture | Some knowledge always lost |
| Can combine with quantization | \(T\), \(\alpha\), student architecture to tune |
💡 Interview: "Why soft labels?" They encode similarity between classes (e.g. cat vs dog vs fox); hard labels don't. "Why temperature?" Softer distribution reveals more structure.
⚡ Quick recall: Teacher → soft labels; student matches hard + soft; temperature \(T\); \(T^2\) in loss; DistilBERT-style.
30. Quantization
Store numbers with fewer bits, so models get smaller and faster.
What it is: Model weights are usually stored as 16-bit numbers. Quantization stores them as 8-bit or 4-bit integers plus a scale, which cuts memory by 2 to 4 times.
Key formula: uniform quantization with scale \(s\):
For 8-bit symmetric quantization, a common choice is \(s = \max\lvert x\rvert / 127\). The rounding error \(x - \hat{x}\) is small when the scale fits the data.
Example: a 7B-parameter model in 16-bit takes \(7\times10^9\times2\) bytes = 14 GB. In 4-bit it takes about 3.5 GB (plus a small overhead).
Two main ways:
- PTQ (post-training quantization): quantize a trained model using a little calibration data. No retraining. Examples: GPTQ, AWQ.
- QAT (quantization-aware training): simulate rounding during training so the model learns to cope with it.
The hard part is outliers: a few huge values stretch the scale and hurt accuracy. Modern methods handle them with care.
| ✅ Pros | ❌ Cons |
|---|---|
| Large memory savings, runs on small GPUs and phones | Some accuracy loss, especially at 4-bit or below |
| Faster, because LLM decoding is limited by memory bandwidth | Needs hardware and software support |
| Cheaper to serve | Activations are harder to quantize than weights |
💡 Interview: "PTQ vs QAT?" PTQ is quick and needs no training, QAT is slower but usually more accurate. "Why does quantization speed up LLMs?" Generating tokens is mostly limited by how fast weights can be read from memory, and fewer bytes means faster reads. "Quantization vs distillation?" Quantization shrinks the numbers in the same model. Distillation trains a smaller model.
⚡ Quick recall: \(q=\text{round}(x/s)\), \(\hat{x}=sq\). 16-bit to 4-bit is about 4 times smaller. PTQ = no retraining, QAT = trained with rounding.
Part XI: Speech and Audio
31. Speech Recognition: CTC, wav2vec 2.0 and Whisper
How machines turn sound into text.
What it is: Automatic speech recognition (ASR) maps audio to text. The audio becomes features (for example a log-mel spectrogram) or is read directly as a waveform, and an encoder produces about 50 vectors per second. The problem is that there are far more audio frames than letters, and we do not know which frame belongs to which letter.
CTC (Connectionist Temporal Classification): the model outputs a letter or a special blank at every frame. Then repeats are merged and blanks removed. Example: h h _ e e l l _ l l o becomes hello. The blank between the two l's keeps them as two letters.
The probability of the text \(Y\) is the sum over all frame-by-frame alignments \(A\) that collapse to \(Y\). Training minimizes \(-\log P(Y\mid X)\).
wav2vec 2.0 (2020): first learn from lots of unlabeled audio. Parts of the internal speech features are masked, and the model must pick the true one among distractors (a contrastive task, like in self-supervised learning). Then fine-tune with CTC on a small labeled set. This made good ASR possible with very little labeled data.
Whisper (OpenAI, 2022): an encoder-decoder Transformer trained on 680,000 hours of weakly labeled, multilingual audio from the web. The decoder writes the text token by token, with special tokens for the language and the task (transcribe or translate). It is robust to noise and many accents.
How we measure it: word error rate (WER):
\(S\) = substituted words, \(D\) = deleted words, \(I\) = inserted words, \(N\) = number of words in the reference. Lower is better. WER is often noticeably higher for accents that are rare in the training data.
| ✅ Pros | ❌ Cons |
|---|---|
| wav2vec 2.0 plus CTC needs little labeled data | Needs large unlabeled pretraining, and CTC treats frames as independent |
| Whisper is robust to noise, accents and many languages | Can invent text on silence or noise, and is not built for live streaming |
| WER is simple and standard | Treats all word errors equally |
💡 Interview: "Why a blank token in CTC?" It lets the model say "no new letter here" and separates repeated letters. "How does wav2vec 2.0 use unlabeled audio?" It masks parts of the speech features and learns to pick the true ones, so it learns speech sounds without transcripts. "How is WER computed?" Substitutions plus deletions plus insertions, divided by the number of reference words.
⚡ Quick recall: CTC = letter or blank per frame, merge repeats, drop blanks. wav2vec 2.0 = self-supervised pretraining then CTC fine-tuning. Whisper = Transformer trained on 680k hours. WER = \((S+D+I)/N\).
Part XII: Frontier
32. Quantum ML
Where quantum computing meets ML, still early.
What it is: Qubits in superposition \(\alpha|0\rangle + \beta|1\rangle\); entanglement and interference enable different kinds of computation. NISQ = Noisy Intermediate-Scale Quantum (50–1000 qubits, high error). QML: Variational quantum circuits, quantum kernels, etc.; in practice mostly hybrid (quantum + classical). Not "trying all answers at once": algorithms use interference to amplify correct and cancel wrong.
Reality: No demonstrated practical advantage for real ML tasks yet; useful for some optimization/simulation problems. Worth understanding for the long term.
| ✅ Potential | ❌ Current |
|---|---|
| Exponential speedup for some problems | Noisy; limited qubits; NISQ |
| Exponential feature spaces (e.g. kernels) | Barren plateaus; data loading bottleneck |
| Natural for quantum simulation | No proven ML advantage yet |
💡 Interview: "Is quantum faster for ML?" Not yet for practical tasks; theoretical speedups need fault-tolerant hardware we don't have. "NISQ?" Noisy Intermediate-Scale Quantum; limited, noisy devices today.
⚡ Quick recall: Qubit \(= \alpha|0\rangle+\beta|1\rangle\); superposition, entanglement, interference; NISQ; variational circuits; no practical ML advantage yet.
Quick Decision Guide
| If you have... | A good first choice in 2026 |
|---|---|
| Tabular data (rows and columns) | Gradient boosting (XGBoost, LightGBM) |
| Images with few labels | A pretrained ViT or CNN, then fine-tune (LoRA) |
| No labels at all | Self-supervised pretraining (contrastive or JEPA), then k-Means or PCA to explore |
| Text and a knowledge base that changes | An LLM with RAG |
| A fixed style or skill to teach an LLM | SFT with LoRA, then DPO if you have preference pairs |
| Math or code answers you can check | GRPO with verifiable rewards (RLVR) |
| Speech to text | Whisper, or wav2vec 2.0 plus CTC for low-resource languages |
| Image or video generation | Flow matching or diffusion with a Transformer backbone |
| A model that is too big or slow to deploy | Quantization, distillation, or a mixture-of-experts model |
| Very long sequences | A hybrid with linear layers (Mamba-2 or Gated DeltaNet) plus some attention |
Wrap-Up
You now have a structured map from Linear Regression to Quantum ML. For exams and interviews, keep this frame in mind:
What is it? → Why does it exist? → How does it work? → Pros & Cons → When to use it?
Understanding intuition and trade-offs beats memorizing formulas. Good luck with your exams and interviews. 🚀
This guide is a living document. Found an error or want to suggest improvements? Leave a comment.
Post a Comment