AI Signal - Mastery Calibration

Adam Optimizer

L1

What is the primary advantage of the Adam optimizer over standard SGD?

It uses adaptive learning rates for each parameter individually It requires less memory than SGD It always converges faster than SGD It doesn't require gradient computation

L2

Why does Adam use both first moment (m) and second moment (v) estimates of gradients?

First moment tracks gradient direction, second moment normalizes by gradient magnitude for adaptive scaling First moment prevents overfitting, second moment speeds up convergence Both moments are needed to compute the exact Hessian matrix First moment handles positive gradients, second moment handles negative gradients

Batch Normalization

L1

What does Batch Normalization normalize across?

The feature dimension within each sample The batch dimension for each feature All dimensions simultaneously Only the output layer activations

L2

Why is Batch Normalization problematic during inference compared to Layer Normalization?

It requires storing the entire training dataset It depends on batch statistics that may not be representative with small/single inference batches It only works with convolutional layers It increases computational cost exponentially

Scaling Laws

L1

According to neural scaling laws, what is the relationship between model performance and compute/data/parameters?

Performance improves linearly with each factor Performance follows a predictable power law relationship Performance plateaus after a certain threshold Performance is random and unpredictable

L2

What was the key insight from the Chinchilla paper regarding optimal compute allocation?

Larger models always perform better regardless of training data Model size and training data should be scaled proportionally for optimal compute efficiency Training data is irrelevant compared to model parameters Compute should be spent entirely on increasing model depth

Learning Rate Schedules

L1

What is the typical pattern for learning rate schedules in modern deep learning?

Keep learning rate constant throughout training Start high and immediately decay Warmup period followed by decay Randomly vary the learning rate

L2

Why is learning rate warmup particularly important for large batch training?

Large batches require more epochs to converge Large batches create noisy gradients that need smoothing Early gradients are unreliable when parameters are randomly initialized, and large batches amplify bad updates Warmup reduces memory usage with large batches

Cross-Entropy Loss

L1

What does cross-entropy loss measure in language modeling?

The difference between predicted and actual word embeddings How far the model's predicted probability distribution is from the true next-token distribution The computational cost of generating tokens The similarity between input and output sequences

L2

Why does cross-entropy loss become very large when the model assigns low probability to the correct token?

The loss function is linear in probability Cross-entropy uses -log(p), so as p approaches 0, -log(p) approaches infinity Low probabilities indicate the model is overfitting The gradient becomes unstable with low probabilities

Overfitting & Regularization

L1

What is overfitting in machine learning?

When the model trains too slowly When the model memorizes training data instead of learning generalizable patterns When the model uses too much computational resources When the model's architecture is too simple

L2

How does dropout regularization work during training versus inference?

Dropout is applied identically during both training and inference During training, randomly zero neurons; during inference, scale all neurons by the dropout probability During training, scale neurons; during inference, randomly zero neurons Dropout is only used during inference to add robustness

Agent Memory Systems

L1

What are the three main types of memory systems used in AI agents?

Short-term (context window), long-term (vector store), and episodic (interaction summaries) Cache memory, RAM memory, and disk memory Working memory, semantic memory, and procedural memory Input memory, processing memory, and output memory

L2

Why do agents need episodic memory when they already have long-term vector storage for retrieval?

Episodic memory is faster to access than vector databases Vector stores only work with text, not with interaction patterns Episodic memory captures the temporal flow and context of conversations that pure semantic retrieval misses Episodic memory is required by the transformer architecture

Tool Use

L1

What is the primary mechanism that enables LLMs to use external tools?

Fine-tuning the model on tool-specific datasets Generating structured output (like JSON) that specifies function calls Directly executing code within the model's neural network Using reinforcement learning to learn tool interactions

L2

Why is tool use considered 'the bridge from language to action' rather than just another form of text generation?

Because tools require special tokens that regular text doesn't use Because tool calls must be executed synchronously with text generation Because it transforms the LLM's text predictions into executable operations that can modify the external world Because tools use a different neural network architecture than language modeling

Planning & Reasoning Loops

L1

What is the key difference between ReAct and simple chain-of-thought prompting?

ReAct uses larger language models than chain-of-thought ReAct interleaves reasoning steps with action execution and observation ReAct only works with mathematical problems ReAct requires fine-tuning while chain-of-thought doesn't

L2

Why do planning loops often lead to better performance than single-pass generation, even when the total compute is similar?

Planning loops use more advanced neural architectures The iterative process allows error correction and adaptation based on intermediate results Planning loops can access larger context windows Multiple passes always generate more creative solutions

Context Engineering

L1

What are the main components that context engineering carefully orchestrates in an LLM's input?

Training data, validation data, and test data System prompts, retrieved documents, conversation history, and tool results Tokens, embeddings, attention weights, and gradients Input layer, hidden layers, and output layer

L2

Why is the ordering and placement of information within the context window crucial for performance?

Because transformers can only process information in sequential order Because attention patterns and recency bias affect how the model weighs different parts of the context Because earlier tokens use less computational resources Because the model's training data was ordered in a specific way

Transformer Block (Full Assembly)

L1

What is the correct order of operations in a standard transformer block?

Multi-Head Attention -> LayerNorm -> FFN -> LayerNorm -> Residual Connections LayerNorm -> Multi-Head Attention -> Residual Add -> LayerNorm -> FFN -> Residual Add FFN -> Multi-Head Attention -> LayerNorm -> Residual Add Residual Add -> Multi-Head Attention -> LayerNorm -> FFN

L2

Why are residual connections essential for training deep transformer networks?

They reduce the computational cost of matrix multiplications They allow gradients to flow directly through the network, preventing vanishing gradients They increase the model's memory capacity They enable parallel processing of attention heads

Causal Masking

L1

What does causal masking do in decoder-only transformer models?

It removes irrelevant tokens from the input sequence It prevents tokens from attending to future tokens by setting their attention scores to negative infinity It masks out padding tokens in variable-length sequences It randomly masks tokens during training for better generalization

L2

Why is setting masked attention scores to negative infinity (rather than zero) crucial for the masking mechanism?

Negative infinity is computationally more efficient than zero Zero would still allow some attention weight after softmax, while -∞ ensures exactly zero attention Negative infinity prevents numerical overflow in the attention computation The transformer architecture requires negative values for proper gradient flow

Tokenization (BPE)

L1

What is the primary purpose of Byte-Pair Encoding (BPE) in language models?

To compress the model weights for faster inference To split text into subword tokens that balance vocabulary size and sequence length To encode positional information about word order in sentences To convert text directly into numerical embeddings

L2

Why does BPE iteratively merge the most frequent character pairs during training, rather than using a fixed set of subword rules?

To ensure all tokens have exactly the same length for efficient processing To create a vocabulary that adapts to the statistical patterns in the training data To guarantee that every possible word can be represented with a single token To minimize the computational cost of the tokenization process

Positional Encoding

L1

Why do transformer models need positional encodings?

To reduce the computational complexity of attention mechanisms Because attention operations are permutation-invariant and don't inherently understand sequence order To compress the input sequence into a fixed-size representation To prevent overfitting during training

L2

What is the key advantage of Rotary Positional Encoding (RoPE) over traditional sinusoidal positional encodings?

RoPE requires less memory and computational resources RoPE can handle sequences of any length without retraining RoPE encodes relative positions directly in the attention computation through rotation RoPE eliminates the need for learned position embeddings entirely

Self-Attention Score Computation

L1

In the self-attention mechanism, what does the formula QK^T / sqrt(d_head) compute?

The final output values for each token The attention scores measuring how much each token should attend to every other token The gradients for backpropagation through the attention layer The positional relationships between tokens in the sequence

L2

Why is the scaling factor sqrt(d_head) crucial in the attention score computation?

It normalizes the scores to have zero mean and unit variance It prevents the dot products from growing too large and pushing softmax into saturation It ensures that attention weights sum to exactly 1.0 across all tokens It makes the computation more efficient by reducing numerical precision requirements

Top-k and Top-p (Nucleus) Sampling

L1

What is the main purpose of top-k and top-p sampling in language model inference?

To speed up inference by reducing the number of forward passes To truncate the vocabulary to likely candidates before sampling, avoiding low-probability tokens To ensure that generated text always follows grammatical rules To compress the model's output probabilities for storage efficiency

L2

Why might top-p (nucleus) sampling be preferred over top-k sampling for creative text generation?

Top-p is computationally faster than top-k for large vocabularies Top-p adapts the candidate set size based on the probability distribution's shape Top-p guarantees that exactly p% of tokens will be novel or creative Top-p eliminates the need for temperature scaling entirely

Temperature & Sampling

L1

How does temperature affect the sampling behavior in language models?

Higher temperature makes the model generate longer sequences Higher temperature makes the probability distribution more uniform, increasing randomness Higher temperature improves the model's factual accuracy Higher temperature reduces the computational cost of generation

L2

What happens to the probability distribution when temperature approaches 0, and why is this mathematically significant?

The distribution becomes uniform, giving equal probability to all tokens The distribution approaches a one-hot vector, making sampling deterministic The distribution becomes undefined due to division by zero The distribution inverts, making low-probability tokens more likely

Speculative Decoding

L1

What is the core idea behind speculative decoding?

Use multiple large models in parallel to generate different candidate sequences Use a small draft model to generate candidate tokens, then verify them with the large model Cache previous computations to avoid recomputing attention for repeated sequences Compress the large model's weights to reduce memory usage during inference

L2

Why can speculative decoding maintain the same output quality as standard autoregressive decoding?

The draft model is trained to exactly match the large model's predictions The verification step ensures that accepted tokens follow the large model's probability distribution Multiple draft models vote on the best tokens to reduce individual model errors The technique only speeds up generation without changing the underlying sampling process

Quantization

L1

What is the primary purpose of quantization in neural network inference?

To reduce model weight precision from higher bit formats (fp32/fp16) to lower bit formats (int8/int4) for faster inference To increase model accuracy by using higher precision arithmetic during training To convert continuous model outputs into discrete categorical predictions To compress training data by reducing the number of input features

L2

Why can quantization from fp16 to int8 often maintain model performance despite the significant reduction in numerical precision?

Because neural networks inherently require high precision arithmetic to function properly Because the quantization process adds noise that acts as regularization during inference Because neural networks are naturally robust to small perturbations, and careful calibration can map the reduced precision range to preserve the most important weight distributions Because int8 arithmetic is mathematically equivalent to fp16 arithmetic for matrix operations

Retrieval-Augmented Generation (RAG)

L1

What does RAG (Retrieval-Augmented Generation) do to improve LLM responses?

It fine-tunes the model on domain-specific data to improve accuracy It retrieves relevant documents from an external knowledge base and includes them in the prompt context It generates multiple responses and selects the best one using a separate ranking model It augments the training data with synthetic examples generated by other models

L2

Why is RAG particularly valuable for addressing the knowledge cutoff problem in LLMs, compared to simply fine-tuning on new data?

RAG is computationally cheaper than fine-tuning in all scenarios Fine-tuning always leads to catastrophic forgetting while RAG never does RAG allows dynamic access to up-to-date information without retraining, while fine-tuning creates a static knowledge snapshot that becomes outdated RAG provides better accuracy than fine-tuning for all types of questions

Guardrails & Safety Layers

L1

What are guardrails in the context of LLM deployment?

Hardware safety mechanisms that prevent GPU overheating during inference Input/output filters and validation systems designed to keep LLM behavior within acceptable bounds Backup models that take over when the primary model fails Rate limiting mechanisms that control how many requests a model can process

L2

Why do guardrails typically operate as separate layers around the LLM rather than being built directly into the model weights?

Because it's impossible to train safety directly into neural network weights Because separate guardrails are always more computationally efficient than integrated safety Because external guardrails can be updated, customized, and debugged independently without retraining the core model, and they provide interpretable safety decisions Because LLMs cannot learn to recognize harmful content during training

Hallucination

L1

What is hallucination in the context of large language models?

When a model generates text that is grammatically incorrect or poorly formatted When a model produces confident, fluent responses that contain factually incorrect information When a model fails to generate any output due to computational errors When a model generates the same response repeatedly in a loop

L2

Why is hallucination considered a fundamental limitation of the next-token prediction training objective rather than just a training data quality issue?

Because hallucination only occurs when models are trained on low-quality datasets Because next-token prediction optimizes for fluency and coherence rather than factual accuracy, and models learn to generate plausible-sounding text even when uncertain about facts Because the transformer architecture is inherently prone to generating false information Because hallucination can be completely eliminated by using larger training datasets

LLM Evaluation Metrics

L1

Which of the following represents different categories of LLM evaluation approaches?

Training loss, validation loss, and test loss Perplexity (intrinsic metrics), standardized benchmarks (MMLU, HumanEval), and human/LLM-as-judge evaluation Accuracy, precision, and recall BLEU score, ROUGE score, and cosine similarity

L2

Why might a model with excellent perplexity scores still perform poorly on downstream tasks like code generation or reasoning?

Because perplexity is calculated incorrectly for these types of tasks Because perplexity measures how well a model predicts the next token in training-like text, but doesn't capture whether the model can perform complex reasoning or generate functionally correct outputs Because code generation tasks require different tokenization schemes than language modeling Because perplexity only works for models trained on English text

Fine-Tuning & LoRA

L1

What is LoRA (Low-Rank Adaptation) designed to accomplish in fine-tuning?

To increase the rank of weight matrices for better model capacity To add small trainable rank-decomposed matrices to frozen pretrained weights, avoiding updates to the original parameters To reduce the learning rate during fine-tuning for more stable training To compress the model size by removing unnecessary parameters

L2

Why does LoRA's low-rank decomposition approach (W + AB) work effectively for adaptation, given that it uses far fewer parameters than full fine-tuning?

Because low-rank matrices can represent any possible weight update with perfect accuracy Because task-specific adaptations often lie in a lower-dimensional subspace than the full parameter space, and the pretrained weights already capture most of the necessary representations Because LoRA matrices are initialized with better values than random initialization Because the low-rank constraint acts as regularization that always improves generalization

Implement micrograd from scratch

L1

In micrograd's Value class, what is the primary purpose of the `_backward` function that gets stored during forward operations?

To store the gradient value for this node To define how to propagate gradients from this node to its children during backpropagation To cache the forward pass result for efficiency To track which operations were performed on this Value

L2

When implementing the backward pass for multiplication (c = a * b) in micrograd, why must you accumulate gradients using `+=` rather than direct assignment when updating a.grad and b.grad?

Because multiplication is not differentiable and requires special handling Because a Value node might be used multiple times in the computation graph, so its gradient contributions must be summed Because the chain rule requires addition of partial derivatives Because it prevents numerical overflow during gradient computation