Seed Identification: Computer Vision - Supervised Multi-Class CNN with Transfer Learning¶
Value Proposition¶
In order to automate manual plant/seedling sorting a CNN was developed that classifies 12 plant species at 90.3% accuracy, targeting recovery of weed-related crop losses estimated at $32B annually with potential herbicide reduction of 60–90%.
Executive Summary¶
Business Opportunities¶
Opportunity A: Weeds cause an estimated 34% yield loss (approximately $32B annually). Automated detection provides the best known opportunity to recover a significant share of that loss.
Solution A: Our current model has a 90.3% accurate detection layer which provides a recovery ceiling of approximately 30%. Further model development would raise that ceiling.
Opportunity B: Precision weeding is projected to reach $10.65B by 2034 (approximately 17% CAGR).
Solution B: Vision-based systems like this are shown in existing field studies to reduce herbicide use (the current implementation process) by 60–90%.
Opportunity C: Multiple deployment paths exist for this model across a growing precision agriculture market.
Solution C (Options): Embedded vision module retrofitted onto existing sprayer booms (John Deere See and Spray). Onboard classification for autonomous robotic platforms (Solinftec Solix). Aerial detection via drone-mounted inference (emerging sector, no dominant player). Edge-deployed mobile app for manual scouting support (agronomist field tools).
Outcomes¶
Model Performance
Custom CNN achieved 90.3% accuracy (0.90 weighted F1).
MobileNetV2 + TTA achieved 82.1% accuracy (0.81 weighted F1).
Architecture
The lighter custom CNN outperforming MobileNetV2 is viable as:
a retrofit on components in purpose-built existing equipment or platforms
a component that can develop in purpose-built platforms
Retaining both models, at least in their early development, enables:
ensemble opportunities
provides architectural flexibility across different deployment targets.
Economic Impact
- Recovering even a fraction of the $32B annual weed-related yield loss through automated detection represents significant value, while reducing herbicide use by 60–90% directly lowers input costs.
Strategy Recommendation
Enterprise: Package the CNN into a secure deployment pipeline, validate on existing equipment through a controlled field pilot, then iterate using production data.
SMB: Ensemble both models to maximize accuracy, add Grad-CAM for interpretability, and deploy as a proof of concept.
Problem Space¶
Overview¶
Current seedling identification is slow, inconsistent, and does not scale. The most common herbicide use techniques are inefficient
Misidentifying weeds as crops (or vice versa) leads directly to yield loss or wasted resources.
With the precision weeding market projected to reach $10.65B by 2034 (~17% CAGR), demand for automated visual classification is clear.
Data Description¶
4,750 labeled images of 12 plant seedling species, preprocessed to 128×128 resolution.
Sourced from
images.npyandLabels.csv.
Process¶
Two deep learning approaches were compared:
Part A — Custom CNN: Three convolutional blocks (32→64→128), batch normalization, dropout, real-time augmentation, learning rate scheduling.
Part B — MobileNetV2 Transfer Learning: ImageNet-pretrained, fine-tuned last 30 layers, cosine-decay LR, test-time augmentation.
Both used an 80/10/10 stratified split.
Results¶
| Model | Test Accuracy | Weighted F1 | Best Species F1 | Weakest Species F1 | |---|---|---|---|---| | Model 1: Custom CNN ✅ | 90.3% | 0.90 | Fat Hen (0.99) | Shepherds Purse (0.67) | | Model 2: MobileNetV2 + TTA | 82.1% | 0.81 | Charlock (0.95) | Black-grass (0.43) |
Next Steps¶
- Misclassifications concentrate in visually similar species (Black-grass vs. Loose Silky-bent, Shepherds Purse vs. Scentless Mayweed). To close these gaps:
- Class weighting or focal loss to improve recall on Shepherds Purse and Black-Grass.
- Field pilot on edge hardware (drone/UGV) to validate real-time inference.
- Agronomist feedback loop on low-confidence predictions for continuous improvement.
Code Execution¶
Library Import and Configuration¶
Summary: Core libraries for data manipulation, visualization, and deep learning model construction were imported.
Observations: TensorFlow/Keras backend confirmed active, environment ready for CNN operations.
Runtime Configuration¶
Hardware Accelerator: L4 GPU (or T4 GPU) and High-RAM
Please ensure the Colab runtime is set to this configuration before executing any code cells to avoid memory errors or protracted training times.
In [ ]:
# ------------------------------
# LIBRARY IMPORT
# ------------------------------
# Imports libraries for image classification: data handling, visualization, deep learning, and metrics.
# numpy, pandas, matplotlib, seaborn, cv2, tensorflow/keras, sklearn.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import cv2
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping, ModelCheckpoint
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelBinarizer
from sklearn.metrics import confusion_matrix, classification_report
IMPORTANT: Restart Runtime¶
When running this Colab Notebook it is important that you restart runtime after importing libraries in order to ensure libraries load properly
Data Loading and Exploration¶
Summary: Images and labels were auto-loaded from GitHub.
Observations: 4,750 total samples loaded with consistent shapes for both images and labels.
In [ ]:
# ------------------------------
# DATA LOADING
# ------------------------------
# Downloads the dataset automatically and loads images + labels.
import importlib.util
import sys
import urllib.request
from pathlib import Path
HELPER_URL = "https://raw.githubusercontent.com/EvagAIML/000-website-notebooks/main/notebooks/_shared/data_access.py"
helper_path = Path("/content/data_access.py") if "google.colab" in sys.modules else Path("./data_access.py")
if not helper_path.exists():
urllib.request.urlretrieve(HELPER_URL, helper_path)
spec = importlib.util.spec_from_file_location("data_access", helper_path)
data_access = importlib.util.module_from_spec(spec)
spec.loader.exec_module(data_access)
ensure_dataset = data_access.ensure_dataset
DATA_DIR = ensure_dataset("seed-identification__multi-cls__cnn-tl-exp")
RAW_DIR = DATA_DIR / "raw"
images = np.load(RAW_DIR / "images_plant.npy")
labels_df = pd.read_csv(RAW_DIR / "labels.csv")
labels = labels_df["Label"]
print(f"Images shape: {images.shape}")
print(f"Labels shape: {labels.shape}")
Downloading dataset for 'seed-identification__multi-cls__cnn-tl-exp'... Extracting dataset for 'seed-identification__multi-cls__cnn-tl-exp'... Images shape: (4750, 128, 128, 3) Labels shape: (4750,)
Exploratory Data Analysis (EDA)¶
Summary: Class distribution was visualized to assess balance across the 12 species.
Observations: Class imbalance detected — Loose Silky-bent is overrepresented; stratified splitting is critical to prevent majority-class bias.
In [ ]:
# ------------------------------
# CLASS DISTRIBUTION VISUALIZATION
# ------------------------------
# Plots species class frequencies to assess balance before training.
plt.figure(figsize=(12, 6))
sns.countplot(x=labels)
plt.xticks(rotation=45, ha='right')
plt.title('Distribution of Plant Seedling Species')
plt.show()
Data Preprocessing¶
Summary: Pixel values were normalized to [0,1] and target labels one-hot encoded for softmax output compatibility.
Observations: Normalization and encoding verified — inputs ready for stable convergence during training.
In [ ]:
# ------------------------------
# NORMALIZATION AND LABEL ENCODING
# ------------------------------
# Scales pixels to 0-1 and one-hot encodes species labels for multi-class classification.
# sklearn.preprocessing.LabelBinarizer handles one-hot encoding.
images_norm = images.astype('float32') / 255.0
# Encode labels
lb = LabelBinarizer()
labels_encoded = lb.fit_transform(labels)
print(f'Normalized images max value: {images_norm.max()}')
print(f'Encoded labels shape: {labels_encoded.shape}')
Normalized images max value: 1.0 Encoded labels shape: (4750, 12)
Train-Test Split¶
Summary: Dataset partitioned into 70% training (3,325 samples) and 30% test (1,425 samples) with stratification.
Observations: Stratified split ensures representative class distribution across both sets.
In [ ]:
# ------------------------------
# TRAIN / VALIDATION / TEST SPLIT
# ------------------------------
# Splits data 80/10/10 with stratification to preserve class proportions.
# First split: 80% train, 20% holdout.
X_train, X_temp, y_train, y_temp = train_test_split(images_norm, labels_encoded, test_size=0.2, random_state=42, stratify=labels)
# Second, split Temp into 50% Val and 50% Test (which is 10% and 10% of total)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=42, stratify=lb.inverse_transform(y_temp))
print(f'Training set shape: {X_train.shape}')
print(f'Validation set shape: {X_val.shape}')
print(f'Test set shape: {X_test.shape}')
Training set shape: (3800, 128, 128, 3) Validation set shape: (475, 128, 128, 3) Test set shape: (475, 128, 128, 3)
Data Augmentation¶
Summary: ImageDataGenerator applied stochastic rotations, zooms, and shifts to artificially expand the training set.
Observations: Augmentation multiplies effective dataset size, forcing the model to learn robust features and reducing overfitting risk.
In [ ]:
# ------------------------------
# DATA AUGMENTATION
# ------------------------------
# Configures real-time augmentation (rotation, zoom, shift, flip) to improve generalization.
# keras.preprocessing.image.ImageDataGenerator applies transforms during training.
datagen = ImageDataGenerator(
rotation_range=20,
zoom_range=0.1,
width_shift_range=0.1,
height_shift_range=0.1,
horizontal_flip=True
)
datagen.fit(X_train)
# Visualize augmentation on one image
plt.figure(figsize=(10, 2))
for i in range(5):
img = X_train[0].reshape((1, 128, 128, 3))
batch = datagen.flow(img, batch_size=1)
plt.subplot(1, 5, i+1)
plt.imshow(batch[0][0])
plt.axis('off')
plt.suptitle('Augmentation Examples')
plt.show()
Model Architecture (Custom CNN)¶
Summary: A 3-block CNN (32→64→128 filters) with batch normalization and dropout was constructed.
Observations: Final softmax layer outputs probability distributions across the 12 target species.
In [ ]:
# ------------------------------
# CNN MODEL ARCHITECTURE
# ------------------------------
# Builds a three-block CNN (32→64→128 filters) with batch normalization and dropout.
model = Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)),
BatchNormalization(),
MaxPooling2D((2, 2)),
Conv2D(64, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D((2, 2)),
Conv2D(128, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D((2, 2)),
Flatten(),
Dense(256, activation='relu'),
Dropout(0.5),
Dense(12, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
/usr/local/lib/python3.12/dist-packages/keras/src/layers/convolutional/base_conv.py:113: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead. super().__init__(activity_regularizer=activity_regularizer, **kwargs)
Model: "sequential_2"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ conv2d_6 (Conv2D) │ (None, 126, 126, 32) │ 896 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_6 │ (None, 126, 126, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ max_pooling2d_6 (MaxPooling2D) │ (None, 63, 63, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_7 (Conv2D) │ (None, 61, 61, 64) │ 18,496 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_7 │ (None, 61, 61, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ max_pooling2d_7 (MaxPooling2D) │ (None, 30, 30, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_8 (Conv2D) │ (None, 28, 28, 128) │ 73,856 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_8 │ (None, 28, 28, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ max_pooling2d_8 (MaxPooling2D) │ (None, 14, 14, 128) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ flatten_2 (Flatten) │ (None, 25088) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_8 (Dense) │ (None, 256) │ 6,422,784 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout_4 (Dropout) │ (None, 256) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_9 (Dense) │ (None, 12) │ 3,084 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 6,520,012 (24.87 MB)
Trainable params: 6,519,564 (24.87 MB)
Non-trainable params: 448 (1.75 KB)
Model Training (Custom CNN)¶
Summary: The CNN was trained for 25 epochs with learning rate reduction on validation plateau.
Observations: Training converged with steady accuracy improvement; LR callback dynamically adjusted rate when validation stalled.
In [ ]:
# ------------------------------
# MODEL TRAINING
# ------------------------------
# Trains the CNN with learning rate reduction on plateau and best-model checkpointing.
# Halves learning rate after 3 epochs of no improvement.
reduce_lr = ReduceLROnPlateau(monitor='val_accuracy', factor=0.5, patience=3, min_lr=0.00001, verbose=1)
# Saves weights when validation loss hits a new minimum.
checkpoint = ModelCheckpoint('best_model.keras', monitor='val_loss', save_best_only=True, mode='min', verbose=1)
# Use datagen.flow for training
history = model.fit(datagen.flow(X_train, y_train, batch_size=64),
epochs=25, # Increased epochs slightly given augmentation
validation_data=(X_val, y_val),
callbacks=[reduce_lr, checkpoint])
Epoch 1/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 261ms/step - accuracy: 0.2209 - loss: 5.1093 Epoch 1: val_loss improved from None to 17.69895, saving model to best_model.keras Epoch 1: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 24s 300ms/step - accuracy: 0.2676 - loss: 3.3066 - val_accuracy: 0.1158 - val_loss: 17.6989 - learning_rate: 0.0010 Epoch 2/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.3601 - loss: 1.9607 Epoch 2: val_loss improved from 17.69895 to 15.13314, saving model to best_model.keras Epoch 2: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 225ms/step - accuracy: 0.3826 - loss: 1.9114 - val_accuracy: 0.0842 - val_loss: 15.1331 - learning_rate: 0.0010 Epoch 3/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.4528 - loss: 1.7006 Epoch 3: val_loss did not improve from 15.13314 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 219ms/step - accuracy: 0.4655 - loss: 1.6949 - val_accuracy: 0.1368 - val_loss: 15.8961 - learning_rate: 0.0010 Epoch 4/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 222ms/step - accuracy: 0.5032 - loss: 1.5912 Epoch 4: val_loss did not improve from 15.13314 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 225ms/step - accuracy: 0.5105 - loss: 1.5269 - val_accuracy: 0.1326 - val_loss: 25.1366 - learning_rate: 0.0010 Epoch 5/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 219ms/step - accuracy: 0.5338 - loss: 1.4780 Epoch 5: val_loss did not improve from 15.13314 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 221ms/step - accuracy: 0.5453 - loss: 1.4452 - val_accuracy: 0.1368 - val_loss: 28.3137 - learning_rate: 0.0010 Epoch 6/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 218ms/step - accuracy: 0.6030 - loss: 1.2475 Epoch 6: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. Epoch 6: val_loss did not improve from 15.13314 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 220ms/step - accuracy: 0.6045 - loss: 1.2663 - val_accuracy: 0.1368 - val_loss: 27.9496 - learning_rate: 0.0010 Epoch 7/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.6098 - loss: 1.2058 Epoch 7: val_loss did not improve from 15.13314 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 219ms/step - accuracy: 0.6237 - loss: 1.1336 - val_accuracy: 0.1368 - val_loss: 24.1116 - learning_rate: 5.0000e-04 Epoch 8/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.6744 - loss: 1.0349 Epoch 8: val_loss improved from 15.13314 to 14.39568, saving model to best_model.keras Epoch 8: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 223ms/step - accuracy: 0.6687 - loss: 1.0399 - val_accuracy: 0.1516 - val_loss: 14.3957 - learning_rate: 5.0000e-04 Epoch 9/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.6664 - loss: 0.9719 Epoch 9: val_loss improved from 14.39568 to 4.83028, saving model to best_model.keras Epoch 9: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 223ms/step - accuracy: 0.6687 - loss: 0.9647 - val_accuracy: 0.3326 - val_loss: 4.8303 - learning_rate: 5.0000e-04 Epoch 10/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.6953 - loss: 0.9161 Epoch 10: val_loss improved from 4.83028 to 1.69388, saving model to best_model.keras Epoch 10: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 223ms/step - accuracy: 0.6955 - loss: 0.9193 - val_accuracy: 0.6042 - val_loss: 1.6939 - learning_rate: 5.0000e-04 Epoch 11/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.6972 - loss: 0.8756 Epoch 11: val_loss did not improve from 1.69388 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 219ms/step - accuracy: 0.7108 - loss: 0.8699 - val_accuracy: 0.5305 - val_loss: 1.7161 - learning_rate: 5.0000e-04 Epoch 12/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.7022 - loss: 0.8816 Epoch 12: val_loss improved from 1.69388 to 0.54190, saving model to best_model.keras Epoch 12: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 225ms/step - accuracy: 0.7137 - loss: 0.8835 - val_accuracy: 0.8421 - val_loss: 0.5419 - learning_rate: 5.0000e-04 Epoch 13/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7309 - loss: 0.7878 Epoch 13: val_loss did not improve from 0.54190 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 217ms/step - accuracy: 0.7376 - loss: 0.7995 - val_accuracy: 0.7874 - val_loss: 0.6640 - learning_rate: 5.0000e-04 Epoch 14/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.7272 - loss: 0.7834 Epoch 14: val_loss did not improve from 0.54190 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 220ms/step - accuracy: 0.7339 - loss: 0.8013 - val_accuracy: 0.7768 - val_loss: 0.7257 - learning_rate: 5.0000e-04 Epoch 15/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7609 - loss: 0.7207 Epoch 15: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. Epoch 15: val_loss did not improve from 0.54190 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 217ms/step - accuracy: 0.7571 - loss: 0.7265 - val_accuracy: 0.6758 - val_loss: 1.1668 - learning_rate: 5.0000e-04 Epoch 16/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7589 - loss: 0.6904 Epoch 16: val_loss did not improve from 0.54190 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 217ms/step - accuracy: 0.7684 - loss: 0.6869 - val_accuracy: 0.8042 - val_loss: 0.7142 - learning_rate: 2.5000e-04 Epoch 17/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.7811 - loss: 0.6205 Epoch 17: val_loss improved from 0.54190 to 0.51823, saving model to best_model.keras Epoch 17: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 225ms/step - accuracy: 0.7792 - loss: 0.6230 - val_accuracy: 0.8463 - val_loss: 0.5182 - learning_rate: 2.5000e-04 Epoch 18/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.7754 - loss: 0.6235 Epoch 18: val_loss improved from 0.51823 to 0.40331, saving model to best_model.keras Epoch 18: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 225ms/step - accuracy: 0.7795 - loss: 0.6257 - val_accuracy: 0.8632 - val_loss: 0.4033 - learning_rate: 2.5000e-04 Epoch 19/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.7970 - loss: 0.5852 Epoch 19: val_loss did not improve from 0.40331 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 218ms/step - accuracy: 0.7945 - loss: 0.6041 - val_accuracy: 0.8568 - val_loss: 0.4581 - learning_rate: 2.5000e-04 Epoch 20/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.7940 - loss: 0.5858 Epoch 20: val_loss did not improve from 0.40331 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 218ms/step - accuracy: 0.8042 - loss: 0.5644 - val_accuracy: 0.8463 - val_loss: 0.5261 - learning_rate: 2.5000e-04 Epoch 21/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8009 - loss: 0.5590 Epoch 21: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. Epoch 21: val_loss did not improve from 0.40331 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 217ms/step - accuracy: 0.8105 - loss: 0.5443 - val_accuracy: 0.8632 - val_loss: 0.4356 - learning_rate: 2.5000e-04 Epoch 22/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8319 - loss: 0.4983 Epoch 22: val_loss did not improve from 0.40331 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 217ms/step - accuracy: 0.8237 - loss: 0.5153 - val_accuracy: 0.8632 - val_loss: 0.4285 - learning_rate: 1.2500e-04 Epoch 23/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 219ms/step - accuracy: 0.7999 - loss: 0.5459 Epoch 23: val_loss improved from 0.40331 to 0.39150, saving model to best_model.keras Epoch 23: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.8063 - loss: 0.5263 - val_accuracy: 0.8758 - val_loss: 0.3915 - learning_rate: 1.2500e-04 Epoch 24/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8254 - loss: 0.5152 Epoch 24: val_loss improved from 0.39150 to 0.35474, saving model to best_model.keras Epoch 24: finished saving model to best_model.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 223ms/step - accuracy: 0.8192 - loss: 0.5145 - val_accuracy: 0.8821 - val_loss: 0.3547 - learning_rate: 1.2500e-04 Epoch 25/25 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 218ms/step - accuracy: 0.8333 - loss: 0.4801 Epoch 25: val_loss did not improve from 0.35474 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 221ms/step - accuracy: 0.8342 - loss: 0.4860 - val_accuracy: 0.8905 - val_loss: 0.3626 - learning_rate: 1.2500e-04
In [ ]:
# ------------------------------
# LOAD BEST MODEL
# ------------------------------
# Restores the best checkpoint; falls back to the current model if unavailable.
try:
model = tf.keras.models.load_model('best_model.keras')
print("Loaded best model from checkpoint.")
except:
print("Could not load checkpoint, using current model state.")
Loaded best model from checkpoint.
Model Evaluation (Custom CNN)¶
Summary: Training/validation curves, confusion matrix, and per-class classification report were generated.
Observations: 89% test accuracy with Fat Hen at 0.99 F1 and Shepherds Purse at 0.55 F1 — visually similar species (Black-grass/Loose Silky-bent) are the primary confusion source.
In [ ]:
# ------------------------------
# MODEL EVALUATION
# ------------------------------
# Evaluates the CNN with training curves, confusion matrix, and per-class classification report.
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['accuracy'], label='Train Accuracy')
plt.plot(history.history['val_accuracy'], label='Val Accuracy')
plt.legend()
plt.title('Accuracy')
plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Train Loss')
plt.plot(history.history['val_loss'], label='Val Loss')
plt.legend()
plt.title('Loss')
plt.show()
# Confusion Matrix and Classification Report
y_pred = model.predict(X_test)
y_pred_classes = np.argmax(y_pred, axis=1)
y_true_classes = np.argmax(y_test, axis=1)
plt.figure(figsize=(10, 8))
cm = confusion_matrix(y_true_classes, y_pred_classes)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=lb.classes_, yticklabels=lb.classes_)
plt.xlabel('Predicted')
plt.ylabel('True')
plt.title('Confusion Matrix')
plt.show()
# Per-species precision, recall, and F1.
print(classification_report(y_true_classes, y_pred_classes, target_names=lb.classes_))
15/15 ━━━━━━━━━━━━━━━━━━━━ 2s 53ms/step
precision recall f1-score support
Black-grass 0.60 0.58 0.59 26
Charlock 0.91 1.00 0.95 39
Cleavers 0.96 0.86 0.91 29
Common Chickweed 0.94 0.98 0.96 61
Common wheat 0.94 0.73 0.82 22
Fat Hen 0.98 1.00 0.99 47
Loose Silky-bent 0.81 0.88 0.84 66
Maize 0.96 1.00 0.98 22
Scentless Mayweed 0.86 0.94 0.90 52
Shepherds Purse 0.90 0.39 0.55 23
Small-flowered Cranesbill 0.92 0.96 0.94 50
Sugar beet 0.95 0.95 0.95 38
accuracy 0.89 475
macro avg 0.89 0.86 0.86 475
weighted avg 0.89 0.89 0.89 475
Prediction Visualization¶
Summary: Individual test images were sampled to qualitatively verify model predictions.
Observations: Most predictions are correct; failures correspond to high inter-class visual similarity or lower image quality.
In [ ]:
# ------------------------------
# PREDICTION VISUALIZATION
# ------------------------------
# Displays five random test images with true vs. predicted labels as a sanity check.
plt.figure(figsize=(15, 5))
for i in range(5):
idx = np.random.randint(0, len(X_test))
img = X_test[idx]
true_label = lb.classes_[y_true_classes[idx]]
pred_label = lb.classes_[y_pred_classes[idx]]
plt.subplot(1, 5, i+1)
plt.imshow(img)
plt.title(f'True: {true_label}\nPred: {pred_label}')
plt.axis('off')
plt.show()
Conclusion and Insights (Part A)¶
Summary: Custom CNN performance was evaluated and future improvement directions identified.
Observations: Model is effective at 90.3% but gains on underrepresented species require advanced augmentation or transfer learning.
In [ ]:
# ------------------------------
# MODEL INSIGHTS
# ------------------------------
# Summarizes findings and identifies improvement opportunities for the next iteration.
print("1. The model performs well overall but struggles with species that have similar leaf structures.")
print("2. Data augmentation (rotation, zooming) could further improve generalization.")
print("3. Class weighting could address the imbalance observed in the EDA phase.")
1. The model performs well overall but struggles with species that have similar leaf structures. 2. Data augmentation (rotation, zooming) could further improve generalization. 3. Class weighting could address the imbalance observed in the EDA phase.
Transfer Learning (MobileNetV2)¶
Summary: MobileNetV2 pretrained on ImageNet was fine-tuned on the last 30 layers with cosine-decay LR and TTA.
Observations: Despite leveraging ImageNet knowledge, MobileNetV2 achieved only 82.1% — underperforming the custom CNN on this small, domain-specific dataset.
In [ ]:
# ------------------------------
# TRANSFER LEARNING — MOBILENETV2
# ------------------------------
# Builds a MobileNetV2 transfer learning model with a frozen pretrained base and custom head.
# keras.applications.MobileNetV2 provides the pretrained feature extractor.
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import GlobalAveragePooling2D
from tensorflow.keras.models import Model
# Load MobileNetV2 without top layers, input size 128x128
base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(128, 128, 3))
# Freeze the base model initially
base_model.trainable = False
# Add custom head
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(128, activation='relu')(x)
x = Dropout(0.5)(x)
predictions = Dense(12, activation='softmax')(x)
model_b = Model(inputs=base_model.input, outputs=predictions)
model_b.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_b.summary()
Model: "functional_29"
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ input_layer_3 │ (None, 128, 128, │ 0 │ - │ │ (InputLayer) │ 3) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ Conv1 (Conv2D) │ (None, 64, 64, │ 864 │ input_layer_3[0]… │ │ │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ bn_Conv1 │ (None, 64, 64, │ 128 │ Conv1[0][0] │ │ (BatchNormalizatio… │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ Conv1_relu (ReLU) │ (None, 64, 64, │ 0 │ bn_Conv1[0][0] │ │ │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ expanded_conv_dept… │ (None, 64, 64, │ 288 │ Conv1_relu[0][0] │ │ (DepthwiseConv2D) │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ expanded_conv_dept… │ (None, 64, 64, │ 128 │ expanded_conv_de… │ │ (BatchNormalizatio… │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ expanded_conv_dept… │ (None, 64, 64, │ 0 │ expanded_conv_de… │ │ (ReLU) │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ expanded_conv_proj… │ (None, 64, 64, │ 512 │ expanded_conv_de… │ │ (Conv2D) │ 16) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ expanded_conv_proj… │ (None, 64, 64, │ 64 │ expanded_conv_pr… │ │ (BatchNormalizatio… │ 16) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_expand │ (None, 64, 64, │ 1,536 │ expanded_conv_pr… │ │ (Conv2D) │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_expand_BN │ (None, 64, 64, │ 384 │ block_1_expand[0… │ │ (BatchNormalizatio… │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_expand_relu │ (None, 64, 64, │ 0 │ block_1_expand_B… │ │ (ReLU) │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_pad │ (None, 65, 65, │ 0 │ block_1_expand_r… │ │ (ZeroPadding2D) │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_depthwise │ (None, 32, 32, │ 864 │ block_1_pad[0][0] │ │ (DepthwiseConv2D) │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_depthwise_… │ (None, 32, 32, │ 384 │ block_1_depthwis… │ │ (BatchNormalizatio… │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_depthwise_… │ (None, 32, 32, │ 0 │ block_1_depthwis… │ │ (ReLU) │ 96) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_project │ (None, 32, 32, │ 2,304 │ block_1_depthwis… │ │ (Conv2D) │ 24) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_1_project_BN │ (None, 32, 32, │ 96 │ block_1_project[… │ │ (BatchNormalizatio… │ 24) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_expand │ (None, 32, 32, │ 3,456 │ block_1_project_… │ │ (Conv2D) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_expand_BN │ (None, 32, 32, │ 576 │ block_2_expand[0… │ │ (BatchNormalizatio… │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_expand_relu │ (None, 32, 32, │ 0 │ block_2_expand_B… │ │ (ReLU) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_depthwise │ (None, 32, 32, │ 1,296 │ block_2_expand_r… │ │ (DepthwiseConv2D) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_depthwise_… │ (None, 32, 32, │ 576 │ block_2_depthwis… │ │ (BatchNormalizatio… │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_depthwise_… │ (None, 32, 32, │ 0 │ block_2_depthwis… │ │ (ReLU) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_project │ (None, 32, 32, │ 3,456 │ block_2_depthwis… │ │ (Conv2D) │ 24) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_project_BN │ (None, 32, 32, │ 96 │ block_2_project[… │ │ (BatchNormalizatio… │ 24) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_2_add (Add) │ (None, 32, 32, │ 0 │ block_1_project_… │ │ │ 24) │ │ block_2_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_expand │ (None, 32, 32, │ 3,456 │ block_2_add[0][0] │ │ (Conv2D) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_expand_BN │ (None, 32, 32, │ 576 │ block_3_expand[0… │ │ (BatchNormalizatio… │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_expand_relu │ (None, 32, 32, │ 0 │ block_3_expand_B… │ │ (ReLU) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_pad │ (None, 33, 33, │ 0 │ block_3_expand_r… │ │ (ZeroPadding2D) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_depthwise │ (None, 16, 16, │ 1,296 │ block_3_pad[0][0] │ │ (DepthwiseConv2D) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_depthwise_… │ (None, 16, 16, │ 576 │ block_3_depthwis… │ │ (BatchNormalizatio… │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_depthwise_… │ (None, 16, 16, │ 0 │ block_3_depthwis… │ │ (ReLU) │ 144) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_project │ (None, 16, 16, │ 4,608 │ block_3_depthwis… │ │ (Conv2D) │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_3_project_BN │ (None, 16, 16, │ 128 │ block_3_project[… │ │ (BatchNormalizatio… │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_expand │ (None, 16, 16, │ 6,144 │ block_3_project_… │ │ (Conv2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_expand_BN │ (None, 16, 16, │ 768 │ block_4_expand[0… │ │ (BatchNormalizatio… │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_expand_relu │ (None, 16, 16, │ 0 │ block_4_expand_B… │ │ (ReLU) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_depthwise │ (None, 16, 16, │ 1,728 │ block_4_expand_r… │ │ (DepthwiseConv2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_depthwise_… │ (None, 16, 16, │ 768 │ block_4_depthwis… │ │ (BatchNormalizatio… │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_depthwise_… │ (None, 16, 16, │ 0 │ block_4_depthwis… │ │ (ReLU) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_project │ (None, 16, 16, │ 6,144 │ block_4_depthwis… │ │ (Conv2D) │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_project_BN │ (None, 16, 16, │ 128 │ block_4_project[… │ │ (BatchNormalizatio… │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_4_add (Add) │ (None, 16, 16, │ 0 │ block_3_project_… │ │ │ 32) │ │ block_4_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_expand │ (None, 16, 16, │ 6,144 │ block_4_add[0][0] │ │ (Conv2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_expand_BN │ (None, 16, 16, │ 768 │ block_5_expand[0… │ │ (BatchNormalizatio… │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_expand_relu │ (None, 16, 16, │ 0 │ block_5_expand_B… │ │ (ReLU) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_depthwise │ (None, 16, 16, │ 1,728 │ block_5_expand_r… │ │ (DepthwiseConv2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_depthwise_… │ (None, 16, 16, │ 768 │ block_5_depthwis… │ │ (BatchNormalizatio… │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_depthwise_… │ (None, 16, 16, │ 0 │ block_5_depthwis… │ │ (ReLU) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_project │ (None, 16, 16, │ 6,144 │ block_5_depthwis… │ │ (Conv2D) │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_project_BN │ (None, 16, 16, │ 128 │ block_5_project[… │ │ (BatchNormalizatio… │ 32) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_5_add (Add) │ (None, 16, 16, │ 0 │ block_4_add[0][0… │ │ │ 32) │ │ block_5_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_expand │ (None, 16, 16, │ 6,144 │ block_5_add[0][0] │ │ (Conv2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_expand_BN │ (None, 16, 16, │ 768 │ block_6_expand[0… │ │ (BatchNormalizatio… │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_expand_relu │ (None, 16, 16, │ 0 │ block_6_expand_B… │ │ (ReLU) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_pad │ (None, 17, 17, │ 0 │ block_6_expand_r… │ │ (ZeroPadding2D) │ 192) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_depthwise │ (None, 8, 8, 192) │ 1,728 │ block_6_pad[0][0] │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_depthwise_… │ (None, 8, 8, 192) │ 768 │ block_6_depthwis… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_depthwise_… │ (None, 8, 8, 192) │ 0 │ block_6_depthwis… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_project │ (None, 8, 8, 64) │ 12,288 │ block_6_depthwis… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_6_project_BN │ (None, 8, 8, 64) │ 256 │ block_6_project[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_expand │ (None, 8, 8, 384) │ 24,576 │ block_6_project_… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_expand_BN │ (None, 8, 8, 384) │ 1,536 │ block_7_expand[0… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_expand_relu │ (None, 8, 8, 384) │ 0 │ block_7_expand_B… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_depthwise │ (None, 8, 8, 384) │ 3,456 │ block_7_expand_r… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_depthwise_… │ (None, 8, 8, 384) │ 1,536 │ block_7_depthwis… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_depthwise_… │ (None, 8, 8, 384) │ 0 │ block_7_depthwis… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_project │ (None, 8, 8, 64) │ 24,576 │ block_7_depthwis… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_project_BN │ (None, 8, 8, 64) │ 256 │ block_7_project[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_7_add (Add) │ (None, 8, 8, 64) │ 0 │ block_6_project_… │ │ │ │ │ block_7_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_expand │ (None, 8, 8, 384) │ 24,576 │ block_7_add[0][0] │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_expand_BN │ (None, 8, 8, 384) │ 1,536 │ block_8_expand[0… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_expand_relu │ (None, 8, 8, 384) │ 0 │ block_8_expand_B… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_depthwise │ (None, 8, 8, 384) │ 3,456 │ block_8_expand_r… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_depthwise_… │ (None, 8, 8, 384) │ 1,536 │ block_8_depthwis… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_depthwise_… │ (None, 8, 8, 384) │ 0 │ block_8_depthwis… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_project │ (None, 8, 8, 64) │ 24,576 │ block_8_depthwis… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_project_BN │ (None, 8, 8, 64) │ 256 │ block_8_project[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_8_add (Add) │ (None, 8, 8, 64) │ 0 │ block_7_add[0][0… │ │ │ │ │ block_8_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_expand │ (None, 8, 8, 384) │ 24,576 │ block_8_add[0][0] │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_expand_BN │ (None, 8, 8, 384) │ 1,536 │ block_9_expand[0… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_expand_relu │ (None, 8, 8, 384) │ 0 │ block_9_expand_B… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_depthwise │ (None, 8, 8, 384) │ 3,456 │ block_9_expand_r… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_depthwise_… │ (None, 8, 8, 384) │ 1,536 │ block_9_depthwis… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_depthwise_… │ (None, 8, 8, 384) │ 0 │ block_9_depthwis… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_project │ (None, 8, 8, 64) │ 24,576 │ block_9_depthwis… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_project_BN │ (None, 8, 8, 64) │ 256 │ block_9_project[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_9_add (Add) │ (None, 8, 8, 64) │ 0 │ block_8_add[0][0… │ │ │ │ │ block_9_project_… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_expand │ (None, 8, 8, 384) │ 24,576 │ block_9_add[0][0] │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_expand_BN │ (None, 8, 8, 384) │ 1,536 │ block_10_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_expand_re… │ (None, 8, 8, 384) │ 0 │ block_10_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_depthwise │ (None, 8, 8, 384) │ 3,456 │ block_10_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_depthwise… │ (None, 8, 8, 384) │ 1,536 │ block_10_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_depthwise… │ (None, 8, 8, 384) │ 0 │ block_10_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_project │ (None, 8, 8, 96) │ 36,864 │ block_10_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_10_project_BN │ (None, 8, 8, 96) │ 384 │ block_10_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_expand │ (None, 8, 8, 576) │ 55,296 │ block_10_project… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_expand_BN │ (None, 8, 8, 576) │ 2,304 │ block_11_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_expand_re… │ (None, 8, 8, 576) │ 0 │ block_11_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_depthwise │ (None, 8, 8, 576) │ 5,184 │ block_11_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_depthwise… │ (None, 8, 8, 576) │ 2,304 │ block_11_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_depthwise… │ (None, 8, 8, 576) │ 0 │ block_11_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_project │ (None, 8, 8, 96) │ 55,296 │ block_11_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_project_BN │ (None, 8, 8, 96) │ 384 │ block_11_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_11_add (Add) │ (None, 8, 8, 96) │ 0 │ block_10_project… │ │ │ │ │ block_11_project… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_expand │ (None, 8, 8, 576) │ 55,296 │ block_11_add[0][… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_expand_BN │ (None, 8, 8, 576) │ 2,304 │ block_12_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_expand_re… │ (None, 8, 8, 576) │ 0 │ block_12_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_depthwise │ (None, 8, 8, 576) │ 5,184 │ block_12_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_depthwise… │ (None, 8, 8, 576) │ 2,304 │ block_12_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_depthwise… │ (None, 8, 8, 576) │ 0 │ block_12_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_project │ (None, 8, 8, 96) │ 55,296 │ block_12_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_project_BN │ (None, 8, 8, 96) │ 384 │ block_12_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_12_add (Add) │ (None, 8, 8, 96) │ 0 │ block_11_add[0][… │ │ │ │ │ block_12_project… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_expand │ (None, 8, 8, 576) │ 55,296 │ block_12_add[0][… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_expand_BN │ (None, 8, 8, 576) │ 2,304 │ block_13_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_expand_re… │ (None, 8, 8, 576) │ 0 │ block_13_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_pad │ (None, 9, 9, 576) │ 0 │ block_13_expand_… │ │ (ZeroPadding2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_depthwise │ (None, 4, 4, 576) │ 5,184 │ block_13_pad[0][… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_depthwise… │ (None, 4, 4, 576) │ 2,304 │ block_13_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_depthwise… │ (None, 4, 4, 576) │ 0 │ block_13_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_project │ (None, 4, 4, 160) │ 92,160 │ block_13_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_13_project_BN │ (None, 4, 4, 160) │ 640 │ block_13_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_expand │ (None, 4, 4, 960) │ 153,600 │ block_13_project… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_expand_BN │ (None, 4, 4, 960) │ 3,840 │ block_14_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_expand_re… │ (None, 4, 4, 960) │ 0 │ block_14_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_depthwise │ (None, 4, 4, 960) │ 8,640 │ block_14_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_depthwise… │ (None, 4, 4, 960) │ 3,840 │ block_14_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_depthwise… │ (None, 4, 4, 960) │ 0 │ block_14_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_project │ (None, 4, 4, 160) │ 153,600 │ block_14_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_project_BN │ (None, 4, 4, 160) │ 640 │ block_14_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_14_add (Add) │ (None, 4, 4, 160) │ 0 │ block_13_project… │ │ │ │ │ block_14_project… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_expand │ (None, 4, 4, 960) │ 153,600 │ block_14_add[0][… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_expand_BN │ (None, 4, 4, 960) │ 3,840 │ block_15_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_expand_re… │ (None, 4, 4, 960) │ 0 │ block_15_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_depthwise │ (None, 4, 4, 960) │ 8,640 │ block_15_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_depthwise… │ (None, 4, 4, 960) │ 3,840 │ block_15_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_depthwise… │ (None, 4, 4, 960) │ 0 │ block_15_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_project │ (None, 4, 4, 160) │ 153,600 │ block_15_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_project_BN │ (None, 4, 4, 160) │ 640 │ block_15_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_15_add (Add) │ (None, 4, 4, 160) │ 0 │ block_14_add[0][… │ │ │ │ │ block_15_project… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_expand │ (None, 4, 4, 960) │ 153,600 │ block_15_add[0][… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_expand_BN │ (None, 4, 4, 960) │ 3,840 │ block_16_expand[… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_expand_re… │ (None, 4, 4, 960) │ 0 │ block_16_expand_… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_depthwise │ (None, 4, 4, 960) │ 8,640 │ block_16_expand_… │ │ (DepthwiseConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_depthwise… │ (None, 4, 4, 960) │ 3,840 │ block_16_depthwi… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_depthwise… │ (None, 4, 4, 960) │ 0 │ block_16_depthwi… │ │ (ReLU) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_project │ (None, 4, 4, 320) │ 307,200 │ block_16_depthwi… │ │ (Conv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ block_16_project_BN │ (None, 4, 4, 320) │ 1,280 │ block_16_project… │ │ (BatchNormalizatio… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ Conv_1 (Conv2D) │ (None, 4, 4, │ 409,600 │ block_16_project… │ │ │ 1280) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ Conv_1_bn │ (None, 4, 4, │ 5,120 │ Conv_1[0][0] │ │ (BatchNormalizatio… │ 1280) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ out_relu (ReLU) │ (None, 4, 4, │ 0 │ Conv_1_bn[0][0] │ │ │ 1280) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ global_average_poo… │ (None, 1280) │ 0 │ out_relu[0][0] │ │ (GlobalAveragePool… │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ dense_6 (Dense) │ (None, 128) │ 163,968 │ global_average_p… │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ dropout_3 (Dropout) │ (None, 128) │ 0 │ dense_6[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ dense_7 (Dense) │ (None, 12) │ 1,548 │ dropout_3[0][0] │ └─────────────────────┴───────────────────┴────────────┴───────────────────┘
Total params: 2,423,500 (9.24 MB)
Trainable params: 165,516 (646.55 KB)
Non-trainable params: 2,257,984 (8.61 MB)
In [ ]:
# ------------------------------
# FINE-TUNING SETUP
# ------------------------------
# Unfreezes the last 30 MobileNetV2 layers with cosine-decay learning rate to avoid catastrophic forgetting.
base_model.trainable = True
for layer in base_model.layers[:-30]:
layer.trainable = False
# Implement Cosine Decay Learning Rate
initial_learning_rate = 0.0001
decay_steps = 1000
cosine_decay = tf.keras.optimizers.schedules.CosineDecay(initial_learning_rate, decay_steps)
# Recompile with low learning rate
model_b.compile(optimizer=Adam(learning_rate=cosine_decay), loss='categorical_crossentropy', metrics=['accuracy'])
print("Model B re-compiled for fine-tuning.")
Model B re-compiled for fine-tuning.
In [ ]:
# ------------------------------
# TRANSFER LEARNING TRAINING
# ------------------------------
# Trains the fine-tuned MobileNetV2 for 20 epochs with checkpointing.
checkpoint_b = ModelCheckpoint('best_model_b.keras', monitor='val_loss', save_best_only=True, mode='min', verbose=1)
# Use the same datagen as before
history_b = model_b.fit(datagen.flow(X_train, y_train, batch_size=64),
epochs=20, # Fine-tune for 20 epochs
validation_data=(X_val, y_val),
callbacks=[checkpoint_b])
Epoch 1/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 343ms/step - accuracy: 0.2149 - loss: 2.5270 Epoch 1: val_loss improved from None to 2.08091, saving model to best_model_b.keras Epoch 1: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 47s 507ms/step - accuracy: 0.3118 - loss: 2.1609 - val_accuracy: 0.2926 - val_loss: 2.0809 Epoch 2/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.5243 - loss: 1.4592 Epoch 2: val_loss improved from 2.08091 to 2.03996, saving model to best_model_b.keras Epoch 2: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.5518 - loss: 1.3694 - val_accuracy: 0.3242 - val_loss: 2.0400 Epoch 3/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.6494 - loss: 1.0796 Epoch 3: val_loss did not improve from 2.03996 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 218ms/step - accuracy: 0.6574 - loss: 1.0471 - val_accuracy: 0.3579 - val_loss: 2.0791 Epoch 4/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.6897 - loss: 0.8975 Epoch 4: val_loss improved from 2.03996 to 2.01269, saving model to best_model_b.keras Epoch 4: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.7055 - loss: 0.8712 - val_accuracy: 0.3853 - val_loss: 2.0127 Epoch 5/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.7540 - loss: 0.7387 Epoch 5: val_loss improved from 2.01269 to 1.83911, saving model to best_model_b.keras Epoch 5: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 229ms/step - accuracy: 0.7558 - loss: 0.7343 - val_accuracy: 0.4484 - val_loss: 1.8391 Epoch 6/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7723 - loss: 0.6819 Epoch 6: val_loss improved from 1.83911 to 1.67162, saving model to best_model_b.keras Epoch 6: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.7787 - loss: 0.6631 - val_accuracy: 0.5053 - val_loss: 1.6716 Epoch 7/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7982 - loss: 0.6250 Epoch 7: val_loss did not improve from 1.67162 60/60 ━━━━━━━━━━━━━━━━━━━━ 13s 219ms/step - accuracy: 0.7997 - loss: 0.5992 - val_accuracy: 0.4611 - val_loss: 1.9056 Epoch 8/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.7953 - loss: 0.5834 Epoch 8: val_loss improved from 1.67162 to 1.64957, saving model to best_model_b.keras Epoch 8: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.8071 - loss: 0.5581 - val_accuracy: 0.5305 - val_loss: 1.6496 Epoch 9/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8226 - loss: 0.5105 Epoch 9: val_loss improved from 1.64957 to 1.50520, saving model to best_model_b.keras Epoch 9: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8200 - loss: 0.5142 - val_accuracy: 0.5768 - val_loss: 1.5052 Epoch 10/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8291 - loss: 0.5069 Epoch 10: val_loss improved from 1.50520 to 1.42732, saving model to best_model_b.keras Epoch 10: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.8353 - loss: 0.4966 - val_accuracy: 0.5916 - val_loss: 1.4273 Epoch 11/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8402 - loss: 0.4705 Epoch 11: val_loss improved from 1.42732 to 1.26751, saving model to best_model_b.keras Epoch 11: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.8450 - loss: 0.4659 - val_accuracy: 0.6147 - val_loss: 1.2675 Epoch 12/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.8510 - loss: 0.4268 Epoch 12: val_loss improved from 1.26751 to 1.18209, saving model to best_model_b.keras Epoch 12: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8450 - loss: 0.4488 - val_accuracy: 0.6421 - val_loss: 1.1821 Epoch 13/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8572 - loss: 0.4458 Epoch 13: val_loss improved from 1.18209 to 1.09529, saving model to best_model_b.keras Epoch 13: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8513 - loss: 0.4354 - val_accuracy: 0.6505 - val_loss: 1.0953 Epoch 14/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8547 - loss: 0.4212 Epoch 14: val_loss improved from 1.09529 to 1.01189, saving model to best_model_b.keras Epoch 14: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8550 - loss: 0.4215 - val_accuracy: 0.6568 - val_loss: 1.0119 Epoch 15/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 217ms/step - accuracy: 0.8412 - loss: 0.4347 Epoch 15: val_loss improved from 1.01189 to 0.94333, saving model to best_model_b.keras Epoch 15: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 236ms/step - accuracy: 0.8445 - loss: 0.4336 - val_accuracy: 0.6758 - val_loss: 0.9433 Epoch 16/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8555 - loss: 0.4345 Epoch 16: val_loss improved from 0.94333 to 0.87153, saving model to best_model_b.keras Epoch 16: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8582 - loss: 0.4205 - val_accuracy: 0.6947 - val_loss: 0.8715 Epoch 17/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8602 - loss: 0.4308 Epoch 17: val_loss improved from 0.87153 to 0.81080, saving model to best_model_b.keras Epoch 17: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 226ms/step - accuracy: 0.8568 - loss: 0.4229 - val_accuracy: 0.7200 - val_loss: 0.8108 Epoch 18/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8590 - loss: 0.4159 Epoch 18: val_loss improved from 0.81080 to 0.75705, saving model to best_model_b.keras Epoch 18: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8558 - loss: 0.4231 - val_accuracy: 0.7389 - val_loss: 0.7570 Epoch 19/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 216ms/step - accuracy: 0.8564 - loss: 0.4245 Epoch 19: val_loss improved from 0.75705 to 0.71030, saving model to best_model_b.keras Epoch 19: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 228ms/step - accuracy: 0.8555 - loss: 0.4309 - val_accuracy: 0.7474 - val_loss: 0.7103 Epoch 20/20 60/60 ━━━━━━━━━━━━━━━━━━━━ 0s 215ms/step - accuracy: 0.8532 - loss: 0.4239 Epoch 20: val_loss improved from 0.71030 to 0.66929, saving model to best_model_b.keras Epoch 20: finished saving model to best_model_b.keras 60/60 ━━━━━━━━━━━━━━━━━━━━ 14s 227ms/step - accuracy: 0.8639 - loss: 0.4170 - val_accuracy: 0.7747 - val_loss: 0.6693
In [ ]:
# ------------------------------
# TEST-TIME AUGMENTATION
# ------------------------------
# Averages predictions over multiple augmented copies of each test image to reduce variance.
# Generates n augmented versions per image and averages their softmax outputs.
def predict_tta(model, images, n_aug=5):
predictions = []
for img in images:
# Create batch of augmented versions
img_expanded = img.reshape((1, 128, 128, 3))
aug_images = [datagen.random_transform(img) for _ in range(n_aug)]
aug_batch = np.array(aug_images)
# Predict on augmented batch
preds = model.predict(aug_batch, verbose=0)
# Average predictions
avg_pred = np.mean(preds, axis=0)
predictions.append(avg_pred)
return np.array(predictions)
print("Running TTA on Model B...")
# Load best model B
try:
model_b = tf.keras.models.load_model('best_model_b.keras')
except:
pass
y_pred_b = predict_tta(model_b, X_test, n_aug=5)
y_pred_classes_b = np.argmax(y_pred_b, axis=1)
print("TTA Predictions Complete.")
Running TTA on Model B... TTA Predictions Complete.
In [ ]:
# ------------------------------
# TRANSFER LEARNING EVALUATION
# ------------------------------
# Evaluates Model B with training curves and classification report.
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history_b.history['accuracy'], label='Train Acc')
plt.plot(history_b.history['val_accuracy'], label='Val Acc')
plt.title('Model B Accuracy')
plt.legend()
plt.subplot(1, 2, 2)
plt.plot(history_b.history['loss'], label='Train Loss')
plt.plot(history_b.history['val_loss'], label='Val Loss')
plt.title('Model B Loss')
plt.legend()
plt.show()
print(classification_report(y_true_classes, y_pred_classes_b, target_names=lb.classes_))
precision recall f1-score support
Black-grass 0.80 0.31 0.44 26
Charlock 0.97 0.79 0.87 39
Cleavers 0.93 0.86 0.89 29
Common Chickweed 0.79 1.00 0.88 61
Common wheat 0.83 0.45 0.59 22
Fat Hen 0.64 0.96 0.77 47
Loose Silky-bent 0.77 0.92 0.84 66
Maize 1.00 0.55 0.71 22
Scentless Mayweed 0.81 0.83 0.82 52
Shepherds Purse 0.55 0.52 0.53 23
Small-flowered Cranesbill 1.00 0.92 0.96 50
Sugar beet 0.89 0.82 0.85 38
accuracy 0.81 475
macro avg 0.83 0.74 0.76 475
weighted avg 0.83 0.81 0.80 475
Model Comparison (Part A vs Part B)¶
Summary: Custom CNN and MobileNetV2 + TTA were compared on test accuracy and F1.
Observations: Custom CNN (90.3%) decisively outperforms MobileNetV2 (82.1%), confirming the purpose-built architecture generalizes better on 4,750 samples.
In [ ]:
# ------------------------------
# MODEL COMPARISON
# ------------------------------
# Compares test accuracy between the custom CNN and MobileNetV2 + TTA.
# sklearn.metrics.accuracy_score computes overall accuracy for each model.
from sklearn.metrics import accuracy_score
acc_a = accuracy_score(y_true_classes, y_pred_classes)
acc_b = accuracy_score(y_true_classes, y_pred_classes_b)
results = pd.DataFrame({
'Model': ['Custom CNN (Part A)', 'MobileNetV2 + TTA (Part B)'],
'Test Accuracy': [acc_a, acc_b]
})
print(results)
plt.figure(figsize=(6, 4))
sns.barplot(x='Model', y='Test Accuracy', data=results)
plt.title('Model Performance Comparison')
plt.ylim(0, 1.0)
plt.show()
Model Test Accuracy 0 Custom CNN (Part A) 0.892632 1 MobileNetV2 + TTA (Part B) 0.810526
Expanded Executive Summary¶
TLDR
The Custom CNN achieves 90.3% accuracy, outperforming MobileNetV2 + TTA (82.1%) despite being a simpler architecture. The smaller model generalizes better on this 4,750-sample dataset and is more practical for edge deployment on existing agricultural equipment, where computational resources are constrained.
Full Summary
Objective: This project established a Computer Vision pipeline for automated plant seedling classification, targeting recovery of the estimated $32B in annual crop losses driven by uncontrolled weed growth. The ability to distinguish 12 species visually enables precision herbicide application and automated sorting.
Iterative Development: Two architectures were compared — a purpose-built 3-block CNN trained from scratch and a MobileNetV2 model leveraging ImageNet transfer learning with test-time augmentation. Both were trained on 4,750 labeled images with stratified splits and real-time data augmentation to combat overfitting.
Performance Analysis: The Custom CNN achieved 90.3% test accuracy with Fat Hen reaching 0.99 F1, while Shepherds Purse (0.55 F1) and Black-grass (0.59 F1) remain challenging due to high inter-species visual similarity. MobileNetV2 underperformed at 82.1%, particularly struggling with minority classes like Common wheat (0.59 F1) and Black-grass (0.44 F1).
Economic Impact: Deploying this detection capability targets the $32B annual yield loss from weeds and aligns with the precision weeding market projected to reach $10.65B by 2034. Field studies show vision-based systems can reduce herbicide use by 60–90%, directly lowering input costs.
Deployment Readiness: The lighter custom CNN is immediately viable for edge deployment on agricultural drones, robotic platforms, or existing sprayer booms. Retaining MobileNetV2 provides ensemble opportunities and architectural flexibility for future iterations.