Overview

The MNIST dataset is the "Hello World" of machine learning — 70,000 grayscale images of handwritten digits (0–9), each 28×28 pixels. This project was my deep dive into convolutional neural networks, moving beyond simple dense layers to understand how CNNs extract spatial features from images.

The Problem

While MNIST is considered a solved problem in the research community, my goal wasn't to break records. It was to understand every layer, every parameter, and every design decision. I wanted to answer: Why does a convolutional layer work better than a fully connected one for images?

Approach

I built the model in three stages:

  • Baseline: A simple dense network with two hidden layers to establish a performance floor (~97%).
  • CNN v1: Two convolutional blocks with max pooling, reaching ~98.2%.
  • CNN v2: Added dropout, batch normalization, and data augmentation, pushing to 98.7%.

Model Architecture

Model: "mnist_cnn"
__________________________________________________________
Layer (type)                 Output Shape              Param #
==========================================================
conv2d_1 (Conv2D)            (None, 26, 26, 32)        320
max_pooling2d_1              (None, 13, 13, 32)        0
conv2d_2 (Conv2D)            (None, 11, 11, 64)        18496
max_pooling2d_2              (None, 5, 5, 64)          0
flatten (Flatten)            (None, 1600)              0
dense_1 (Dense)              (None, 128)               204928
dropout (Dropout)            (None, 128)               0
dense_2 (Dense)              (None, 10)                1290
==========================================================

Key Learnings

  • Data augmentation (small rotations and shifts) significantly improved generalization.
  • Batch normalization stabilized training and allowed higher learning rates.
  • Visualizing feature maps helped me understand what each layer was actually learning.
  • Overfitting is real — even on MNIST, without regularization, validation loss diverges quickly.

Results

The final model achieved 98.7% accuracy on the test set. While this isn't state-of-the-art (which sits around 99.8%), the knowledge gained from building and tuning every component from scratch was far more valuable than the number itself.

Related

I wrote about my approach to learning machine learning in this blog post. You might also be interested in my Housing Price Prediction project.