Machine Learning to CNNs

What learning from data means: generalization, overfitting, the train/validation/test discipline, and metrics you can defend.

Study

• Géron ch. 1 (the landscape), ch. 2 (end-to-end project) - actually run the ch. 2 project.
• Géron ch. 3 (classification): precision, recall, ROC, confusion matrices.

Exercises

• Do the ch. 2 project on a different dataset (any tabular set from OpenML) so you cannot copy the book's answers.
• Train a classifier on MNIST (per ch. 3); produce and interpret its confusion matrix.
• Deliberately overfit a model, then show the overfitting in a train-vs-validation curve.

Checkpoint

You can say precisely why a model with 99 percent accuracy can still be useless (class imbalance) and which metric you would use instead.

Study

• Géron ch. 4 (linear/logistic regression, gradient descent, regularization).
• Skim ch. 5-7 (SVMs, trees, ensembles) - one evening; know what they are, when they beat neural nets on small tabular data, and move on.

Exercises

• Implement logistic regression with batch gradient descent from scratch in NumPy; match scikit-learn's coefficients on the same data.
• Show the effect of L2 regularization on your from-scratch model's weights and validation accuracy.

Checkpoint

You can derive the gradient of the logistic loss on paper.

The week the magic becomes mechanism. You will never fear a deep learning bug the same way after writing backprop yourself.

Watch

Exercises

• Build a two-layer fully connected network in pure NumPy - forward pass, softmax cross-entropy loss, manual backprop, SGD loop - and train it on MNIST to over 95 percent test accuracy.
• Verify every gradient in your network with numerical gradient checking (your Week 2 tool).

Checkpoint

Your from-scratch network trains, and you can trace where each gradient term comes from.

Study

• Official PyTorch tutorials: Learn the Basics series (tensors, datasets and dataloaders, autograd, building models, optimization, saving).
• Géron ch. 10-11 as the conceptual companion (MLPs, training deep nets) - the book uses Keras; translate mentally, it is good for you.

Exercises

• Rebuild your Week 7 MLP in PyTorch in under 100 lines; confirm it matches your from-scratch accuracy.
• Write a reusable training loop with validation, checkpointing, and early stopping - this becomes your template for every later project, including the quantum ones.
• Break autograd on purpose: detach a tensor mid-graph, observe the gradient go silent, explain why.

Checkpoint

You can build and train a new architecture from a blank file without looking at a tutorial.

The architecture this whole course is named after. Convolution as learned filters, weight sharing, receptive fields, pooling.

Study

• CS231n lecture 5 (CNNs) and lecture 9 (architectures: LeNet, AlexNet, VGG, ResNet) - same playlist as Week 7.
• CS231n notes on convolutional networks at cs231n.github.io - work the shape-arithmetic examples by hand.
• Géron ch. 14 (computer vision with CNNs) as a second pass.

Exercises

• Implement 2D convolution in NumPy (single channel, stride, padding); verify against torch.nn.functional.conv2d.
• Train a small CNN on CIFAR-10 to at least 80 percent test accuracy; log experiments properly.
• Visualize first-layer filters and feature maps of your trained model.

Checkpoint

Given input 32x32x3, a 5x5 conv with 16 filters, stride 1, pad 2 - you can state the output shape and parameter count in your head.

Where practitioners are separated from tutorial-followers: initialization, normalization, optimizers, schedules, augmentation, and reading learning curves like an engineer.

Study

• CS231n lectures 6-8 (training neural networks I and II, deep learning software).
• Deep Learning (Goodfellow et al.) ch. 8 - optimization, as theory backup. Reference, not cover-to-cover.

Exercises

• On your CIFAR-10 CNN, run a controlled ablation: with/without batch norm, with/without augmentation, three learning rates. One change at a time; table of results in your lab notebook.
• Push your CIFAR-10 accuracy to 90 percent or higher using what you learned (a small ResNet is fair game).

Checkpoint

Shown a loss curve that plateaus immediately, or one that diverges, you can name the two most likely causes of each.

Radio data is 1D, complex-valued, and time-structured. This week bridges everything to Capstone A.

Study

• 1D convolutions: torch.nn.Conv1d docs; think of I/Q as a 2-channel 1D signal of length 128.
• Read O'Shea, Corgan, Clancy - Convolutional Radio Modulation Recognition Networks (2016). Your first proper paper read: three passes (skim, read, reproduce-in-your-head).
• Input representations: raw I/Q vs amplitude/phase vs spectrogram - note the trade-offs; you will test them in the capstone.

Exercises

• Using your Week 4 signal generator, create a small synthetic dataset (BPSK, QPSK, 8PSK, 16QAM at several SNRs) and train a 1D CNN to classify it.
• Plot accuracy as a function of SNR - the signature plot of this entire field. You will make this plot many more times.

Checkpoint

You can explain why weight sharing makes convolution the right prior for signals (translation invariance in time).

No new material. Close gaps, clean up your lab notebook, and take the gate seriously - Capstone A assumes all of it.

Exercises

• Reimplement a small ResNet (two residual blocks) from memory, 1D variant, on your synthetic signal dataset.
• Write a one-page summary in your repo: "How I train a model that works" - your personal checklist, in your own words.