Emotion Recognition

What I Built#

I built this project as a term project for my Deep Learning course at FAST-NUCES. The objective was to train a computer vision model that classifies human facial expressions into 7 categories (Angry, Disgust, Fear, Happy, Neutral, Sad, Surprise) using the FER-2013 benchmark dataset (about 35,000 images).

I developed the pipeline in a Google Colab notebook using TensorFlow/Keras, adapting a pre-trained ResNet50V2 backbone using transfer learning.

Technical Hurdles & Practical Solutions#

1. Preserving Facial Geometry Across Downsampling

The FER-2013 dataset contains tiny 48x48 grayscale crops. ResNet50V2 downsamples input dimensions by 32x across its residual blocks. If you pass a 48x48 image directly into ResNet, the final feature map collapses down to a 1x1 spatial grid, completely losing subtle details like lip curvature and eyebrow shape.

How I handled it:

• Upscaled the input images to 96x96x3 using bilinear interpolation before feeding them into the network, preserving enough spatial resolution for the deeper convolutional layers to extract meaningful expression features.

2. Preventing Overfitting with Global Average Pooling

Flattening the final convolutional block into a standard dense layer would introduce millions of parameters, causing severe overfitting on a dataset this small.

How I handled it:

• Replaced Flatten() with GlobalAveragePooling2D (GAP), which averages each feature channel into a single value, enforcing translation invariance.
• Added a regularized classification head: BatchNormalization -> Dense(256, ReLU) -> Dropout(0.40) -> Dense(7, Softmax).

Model Pipeline#

FER-2013 Dataset (48x48 Grayscale)
             │
             ▼
┌──────────────────────────────────────┐
│  Bilinear Upscaling to 96x96x3       │ (Preserves spatial resolution)
└────────────┬─────────────────────────┘
             │
             ▼
┌──────────────────────────────────────┐
│  Data Augmentation & Normalization   │ (15° rotation, zoom, horizontal flip)
└────────────┬─────────────────────────┘
             │
             ▼
┌──────────────────────────────────────┐
│     ResNet50V2 Backbone (ImageNet)   │ (Frozen lower stages, top blocks fine-tuned)
└────────────┬─────────────────────────┘
             │
             ▼
┌──────────────────────────────────────┐
│        GlobalAveragePooling2D        │ (Replaces parameter-heavy Flatten layer)
└────────────┬─────────────────────────┘
             │
             ▼
┌──────────────────────────────────────┐
│  Dense(256) + Dropout(0.4) + Softmax │ (7 emotion classes)
└──────────────────────────────────────┘

What I Learned#

• Check your backbone's downsampling ratio: When using transfer learning with models originally designed for 224x224 images, small inputs (like 48x48) will collapse unless you upscale or adjust earlier stride lengths.
• Global Average Pooling beats Flattening: GAP drastically reduces total parameter count while helping the network generalize better to slight variations in face positioning.
• Dynamic learning rate scheduling: Using ReduceLROnPlateau (decaying learning rate by 80% when validation loss stalls) helped the model settle smoothly into a stable minimum without oscillating.