Lab 7: Transfer Learning Project

Part 1: What is Transfer Learning?

Imagine you want to become a world-class Italian chef. You have two options:

Start from Scratch: Learn what an oven is, how heat works, what a knife is, basic knife skills, what flour is, etc. This would take years.
Transfer Learning: Hire a world-class French chef who already knows all the basics of cooking (heat, knife skills, textures, flavors). You just need to teach them the specifics of Italian cuisine (pasta, tomato sauces, etc.). This is much faster!

In AI, building a CNN from scratch is like Option 1. **Transfer Learning** is Option 2. We take a powerful model (like `VGG16`, `ResNet`, or `EfficientNet`) that has already been trained on millions of diverse images (like the French chef) and adapt it for our specific task (like Italian cooking).

The Two Main Transfer Learning Strategies:

Feature Extraction: We use the pre-trained model's convolutional layers as a fixed "feature extractor." We freeze its weights and only train a new classifier head that we add on top. This is fast and works well with small datasets.
Fine-Tuning: We start with a pre-trained model, but we "unfreeze" the top few layers of the base model and train them with a very low learning rate, along with our new classifier head. This allows the model to adapt its specialized filters to our new dataset.

Today, we'll use the "Cats vs. Dogs" dataset to explore both of these powerful techniques.

Part 2: Setup and Data Preparation

First, we need to download and prepare the "Cats vs. Dogs" dataset from Kaggle. In Colab, you can use the Kaggle API for this.

Setting up the Kaggle API

You'll need to upload your `kaggle.json` API token to your Colab environment. Go to your Kaggle account -> Settings -> API -> Create New Token. Then upload the downloaded file to your Colab session.

                # Install Kaggle API and set up credentials

                !pip install -q kaggle

                !mkdir -p ~/.kaggle

                !cp kaggle.json ~/.kaggle/

                !chmod 600 ~/.kaggle/kaggle.json

                # Download and unzip the dataset

                !kaggle competitions download -c dogs-vs-cats

                !unzip -q dogs-vs-cats.zip

                !unzip -q train.zip

                import tensorflow as tf

                from tensorflow import keras

                from tensorflow.keras import layers

                import numpy as np

                import matplotlib.pyplot as plt

                # Define image size and batch size

                IMG_SIZE = (180, 180)

                BATCH_SIZE = 32

                # Create training and validation datasets from the 'train' directory

                # This utility automatically labels images based on their folder
                    structure.

                # But since dogs-vs-cats doesn't have subfolders, we'll need to sort that
                    out.

                # First, let's create the subfolders.

                import os, shutil

                os.makedirs("data/cats", exist_ok=True)

                os.makedirs("data/dogs", exist_ok=True)

                for fname in os.listdir('train'):

                  if fname.startswith('cat'):

                    shutil.move(os.path.join('train', fname), "data/cats")

                  elif fname.startswith('dog'):

                    shutil.move(os.path.join('train', fname), "data/dogs")

                train_dataset = tf.keras.utils.image_dataset_from_directory(

                  "data",

                  validation_split=0.2,

                  subset="training",

                  seed=1337, # Must be the same seed for
                    both datasets

                  image_size=IMG_SIZE,

                  batch_size=BATCH_SIZE,

                )

                validation_dataset = tf.keras.utils.image_dataset_from_directory(

                  "data",

                  validation_split=0.2,

                  subset="validation",

                  seed=1337,

                  image_size=IMG_SIZE,

                  batch_size=BATCH_SIZE,

                )

💡 Your Turn: Inspect the Data

Our `train_dataset` object is a `tf.data.Dataset`. Let's peek inside. Use the `.take(1)` method to grab one batch of images and labels, then print their shapes. This is a great way to debug your data pipeline and confirm everything is working as expected.

                    for images, labels in
                    train_dataset.take(1):

                      print("Images shape:",
                    images.shape)

                      print("Labels shape:",
                    labels.shape)
                

Part 3: Strategy 1 - Feature Extraction

We'll load the `VGG16` model, which was trained on the ImageNet dataset. We'll chop off its classifier head and replace it with our own.

                # Load the VGG16 base model, excluding its top classification layer

                base_model = keras.applications.VGG16(

                  weights='imagenet', # Load weights
                    pre-trained on ImageNet.

                  input_shape=(180, 180, 3),

                  include_top=False) # Do not include the
                    ImageNet classifier at the top.

                # Freeze the base model

                base_model.trainable = False

                # Create a new model on top

                inputs = keras.Input(shape=(180, 180, 3))

                x = base_model(inputs, training=False) # Set
                    training=False so it runs in inference mode

                x = layers.GlobalAveragePooling2D()(x) # Efficient alternative to
                    Flatten

                x = layers.Dense(256, activation='relu')(x)

                x = layers.Dropout(0.5)(x)

                outputs = layers.Dense(1, activation='sigmoid')(x) # Sigmoid for binary
                    classification

                model = keras.Model(inputs, outputs)

                model.summary()

... Total params: 15,243,329 Trainable params: 524,545 Non-trainable params: 14,718,784

Notice that the vast majority of parameters are "Non-trainable." We are only training our small classifier head.

💡 Your Turn: Classifier Head Complexity

Look at the `model.summary()` again. The trainable parameters come from our new `Dense` layers. Try changing the number of units in the `Dense(256, ...)` layer to `512`. Re-run `model.summary()`. How does this impact the number of trainable parameters? It's a good way to see how quickly a dense head can become complex.

Train the Feature Extraction Model

                model.compile(optimizer=keras.optimizers.Adam(),
                loss=keras.losses.BinaryCrossentropy(),
                metrics=[keras.metrics.BinaryAccuracy()])

                epochs = 10

                history = model.fit(train_dataset, epochs=epochs, validation_data=validation_dataset)

💡 Your Turn: Plot the Loss

We plotted the accuracy in previous labs, which is great. But loss tells a different story about how confident the model is. Copy the plotting code from a previous lab and adapt it to plot `'loss'` and `'val_loss'`. A good model will have both accuracy increasing and loss decreasing.

💡 Your Turn: Try a Different Base Model

Keras offers many pre-trained models. `VGG16` is old and heavy. Repeat the steps above, but this time use a more modern, efficient model like `EfficientNetB0`. Do you get better results? Does it train faster?

                    # Example of loading a different model

                    base_model = keras.applications.EfficientNetB0(include_top=False,
                    weights='imagenet')
                

Part 4: Strategy 2 - Fine-Tuning

We've trained our new head. Now, let's unfreeze the top layers of the base model to allow it to specialize on cat and dog features.

                # First, let's get the base_model back to its initial state

                # And unfreeze the base model

                base_model.trainable = True

💡 Your Turn: Verify the Change

Before you re-compile and start fine-tuning, run `model.summary()` one more time. Notice how the number of "Trainable params" has dramatically increased now that `base_model.trainable` is `True`. Confirm this matches the "Total params".

                # It's important to re-instantiate the optimizer and use a very low learning
                    rate

                model.compile(optimizer=keras.optimizers.Adam(1e-5), #
                    Low learning rate!
                loss=keras.losses.BinaryCrossentropy(),
                metrics=[keras.metrics.BinaryAccuracy()])

                # Continue training for a few more epochs

                fine_tune_epochs = 10

                total_epochs = epochs + fine_tune_epochs

                history_fine = model.fit(train_dataset,
                epochs=total_epochs,
                initial_epoch=history.epoch[-1],
                validation_data=validation_dataset)

By training for a few more epochs with the top layers unfrozen, the model should be able to eke out a few more percentage points of accuracy, often reaching 97-98% on this dataset!

💡 Your Turn: Deeper Fine-Tuning

The code above unfreezes the entire `VGG16` model. This can be risky. A better practice is to only unfreeze the top convolutional block. Look at `base_model.summary()` to find the names of the layers. Use a `for` loop to freeze all layers up to `block5_conv1`, then unfreeze the rest. Does this provide more stable training?

                    # Hint:

                    for layer in base_model.layers:

                      if layer.name == 'block5_conv1':

                        break

                      layer.trainable = False

Part 5: Your Mission - Flower Classifier

Assignment: Classify 5 Types of Flowers

Your goal is to build a high-accuracy classifier for the `tf_flowers` dataset, which is available directly in TensorFlow Datasets. You must apply both feature extraction and fine-tuning.

Your Tasks:

Load Data: Use `tensorflow_datasets` to load the `tf_flowers` dataset. This will require a slightly different loading pipeline than the one we used for Kaggle.
Choose a Base Model: Start with a modern base model like `Xception` or `InceptionV3`.
Data Augmentation: Since the dataset is small, data augmentation is crucial. Add a `Sequential` model with `RandomFlip` and `RandomRotation` as the first layer of your model.
Feature Extraction: Freeze the base model and train your classifier head (which should have 5 outputs with `'softmax'` activation) until validation accuracy plateaus.
Fine-Tuning: Unfreeze the top 20% of the layers in your base model and continue training with a very low learning rate (`1e-5`).
Report: What is the highest validation accuracy you can achieve?

Part 6: Bonus - Medical Image Classification

Transfer learning is incredibly powerful in specialized domains like medical imaging, where labeled data is scarce and expensive.

Kaggle: Histopathologic Cancer Detection

The goal is to identify metastatic cancer in small image patches taken from lymph node sections. This is a binary classification task of critical importance.

Your Challenge:

Apply the full feature extraction and fine-tuning pipeline you learned in this lab to this dataset. Key challenges will be:

Handling a very large dataset that may not fit in memory. You will need to use the `image_dataset_from_directory` utility effectively.
Choosing a powerful and appropriate base model. `EfficientNet` models are often a good starting point.
Carefully choosing how many layers to unfreeze during fine-tuning to maximize performance without overfitting.

Part 7: Submission Guidelines

Complete all "Your Turn" tasks and the main "Lab Assignment" (Flower Classifier) in a single Google Colab notebook. The Kaggle project is a bonus.
For the assignment, clearly separate your feature extraction and fine-tuning steps. Show the `model.summary()` after freezing and after unfreezing layers.
Show the code used to train your model and plot its full training/validation history (including both phases).
Add a Text Cell at the end reporting the final validation accuracy and summarizing the choices you made (base model, augmentation, fine-tuning strategy) to get there.
Ensure all your code cells have been run so that their outputs and plots are visible.
When you are finished, generate a shareable link. In Colab, click "Share" and set access to "Anyone with the link" can "Viewer".
Click "Copy link" and submit this link as your assignment.