# -*- coding: utf-8 -*-
"""seed-identification__multi-cls__cnn-tl-exp.ipynb

Automatically generated by Colab.

Original file is located at
    https://colab.research.google.com/github/EvagAIML/000-website-notebooks/blob/main/notebooks/seed-identification__multi-cls__cnn-tl-exp/seed-identification__multi-cls__cnn-tl-exp.ipynb

# **Seed Identification**: Computer Vision - Supervised Multi-Class CNN with Transfer Learning
---

## **Value Proposition**

In order to automate manual plant/seedling sorting a CNN was developed that classifies 12 plant species at 90.3% accuracy, targeting recovery of weed-related crop losses estimated at $32B annually with potential herbicide reduction of 60–90%.

---

## **Executive Summary**


### Business Opportunities

* **Opportunity A:** Weeds cause an estimated 34% yield loss (approximately $32B annually). Automated detection provides the best known opportunity to recover a significant share of that loss.

* **Solution A:** Our current model has a 90.3% accurate detection layer which provides a recovery ceiling of approximately 30%. Further model development would raise that ceiling.

* **Opportunity B:** Precision weeding is projected to reach $10.65B by 2034 (approximately 17% CAGR).

* **Solution B:** Vision-based systems like this are shown in existing field studies to reduce herbicide use (the current implementation process) by 60–90%.

* **Opportunity C:** Multiple deployment paths exist for this model across a growing precision agriculture market.

* **Solution C** (Options): Embedded vision module retrofitted onto existing sprayer booms (John Deere See and Spray). Onboard classification for autonomous robotic platforms (Solinftec Solix). Aerial detection via drone-mounted inference (emerging sector, no dominant player). Edge-deployed mobile app for manual scouting support (agronomist field tools).


### Outcomes

* **Model Performance**

  * Custom CNN achieved 90.3% accuracy (0.90 weighted F1).

  * MobileNetV2 + TTA achieved 82.1% accuracy (0.81 weighted F1).

* **Architecture**

  * The lighter custom CNN outperforming MobileNetV2 is viable as:

    * a retrofit on components in purpose-built existing equipment or platforms

    * a component that can develop in purpose-built platforms

  * Retaining both models, at least in their early development, enables:

     *  ensemble opportunities

     *  provides architectural flexibility across different deployment targets.

* **Economic Impact**

  * Recovering even a fraction of the $32B annual weed-related yield loss through automated detection represents significant value, while reducing herbicide use by 60–90% directly lowers input costs.

* **Strategy Recommendation**

  * Enterprise: Package the CNN into a secure deployment pipeline, validate on existing equipment through a controlled field pilot, then iterate using production data.

  * SMB: Ensemble both models to maximize accuracy, add Grad-CAM for interpretability, and deploy as a proof of concept.


---

## **Problem Space**


### Overview

* Current seedling identification is slow, inconsistent, and does not scale. The most common herbicide use techniques are inefficient

* Misidentifying weeds as crops (or vice versa) leads directly to yield loss or wasted resources.

* With the precision weeding market projected to reach $10.65B by 2034 (~17% CAGR), demand for automated visual classification is clear.


### Data Description

* 4,750 labeled images of 12 plant seedling species, preprocessed to 128×128 resolution.

* Sourced from `images.npy` and `Labels.csv`.

### Process

* Two deep learning approaches were compared:

  * **Part A — Custom CNN:** Three convolutional blocks (32→64→128), batch normalization, dropout, real-time augmentation, learning rate scheduling.
  
  * **Part B — MobileNetV2 Transfer Learning:** ImageNet-pretrained, fine-tuned last 30 layers, cosine-decay LR, test-time augmentation.

* Both used an 80/10/10 stratified split.


### Results
>| Model | Test Accuracy | Weighted F1 | Best Species F1 | Weakest Species F1 |
>|---|---|---|---|---|
>| **Model 1: Custom CNN ✅** | **90.3%** | **0.90** | **Fat Hen (0.99)** | **Shepherds Purse (0.67)** |
>| Model 2: MobileNetV2 + TTA | 82.1% | 0.81 | Charlock (0.95) | Black-grass (0.43) |


### Next Steps
* Misclassifications concentrate in visually similar species (Black-grass vs. Loose Silky-bent, Shepherds Purse vs. Scentless Mayweed). To close these gaps:
  *  Class weighting or focal loss to improve recall on Shepherds Purse and Black-Grass.
  * Field pilot on edge hardware (drone/UGV) to validate real-time inference.
  * Agronomist feedback loop on low-confidence predictions for continuous improvement.

## **Code Execution**

### **Library Import and Configuration**

**Summary:** Core libraries for data manipulation, visualization, and deep learning model construction were imported.

**Observations:** TensorFlow/Keras backend confirmed active, environment ready for CNN operations.

### **Runtime Configuration**

> **Hardware Accelerator:** **L4 GPU** (or T4 GPU) and **High-RAM**
>
> Please ensure the Colab runtime is set to this configuration before executing any code cells to avoid memory errors or protracted training times.
"""

# ------------------------------
# LIBRARY IMPORT
# ------------------------------
# Imports libraries for image classification: data handling, visualization, deep learning, and metrics.
# numpy, pandas, matplotlib, seaborn, cv2, tensorflow/keras, sklearn.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import cv2
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping, ModelCheckpoint
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelBinarizer
from sklearn.metrics import confusion_matrix, classification_report

"""### **IMPORTANT: Restart Runtime**
When running this Colab Notebook it is important that you restart runtime after importing libraries in order to ensure libraries load properly

### **Data Loading and Exploration**

**Summary:** Images and labels were auto-loaded from GitHub.

**Observations:** 4,750 total samples loaded with consistent shapes for both images and labels.
"""

# ------------------------------
# DATA LOADING
# ------------------------------
# Downloads the dataset automatically and loads images + labels.

import importlib.util
import sys
import urllib.request
from pathlib import Path

HELPER_URL = "https://raw.githubusercontent.com/EvagAIML/000-website-notebooks/main/notebooks/_shared/data_access.py"

helper_path = Path("/content/data_access.py") if "google.colab" in sys.modules else Path("./data_access.py")

if not helper_path.exists():
    urllib.request.urlretrieve(HELPER_URL, helper_path)

spec = importlib.util.spec_from_file_location("data_access", helper_path)
data_access = importlib.util.module_from_spec(spec)
spec.loader.exec_module(data_access)

ensure_dataset = data_access.ensure_dataset

DATA_DIR = ensure_dataset("seed-identification__multi-cls__cnn-tl-exp")
RAW_DIR = DATA_DIR / "raw"

images = np.load(RAW_DIR / "images_plant.npy")

labels_df = pd.read_csv(RAW_DIR / "labels.csv")
labels = labels_df["Label"]

print(f"Images shape: {images.shape}")
print(f"Labels shape: {labels.shape}")

"""### **Exploratory Data Analysis (EDA)**

**Summary:** Class distribution was visualized to assess balance across the 12 species.

**Observations:** Class imbalance detected — Loose Silky-bent is overrepresented; stratified splitting is critical to prevent majority-class bias.

"""

# ------------------------------
# CLASS DISTRIBUTION VISUALIZATION
# ------------------------------
# Plots species class frequencies to assess balance before training.

plt.figure(figsize=(12, 6))
sns.countplot(x=labels)
plt.xticks(rotation=45, ha='right')
plt.title('Distribution of Plant Seedling Species')
plt.show()

"""### **Data Preprocessing**

**Summary:** Pixel values were normalized to [0,1] and target labels one-hot encoded for softmax output compatibility.

**Observations:** Normalization and encoding verified — inputs ready for stable convergence during training.

"""

# ------------------------------
# NORMALIZATION AND LABEL ENCODING
# ------------------------------
# Scales pixels to 0-1 and one-hot encodes species labels for multi-class classification.
# sklearn.preprocessing.LabelBinarizer handles one-hot encoding.

images_norm = images.astype('float32') / 255.0

# Encode labels
lb = LabelBinarizer()
labels_encoded = lb.fit_transform(labels)

print(f'Normalized images max value: {images_norm.max()}')
print(f'Encoded labels shape: {labels_encoded.shape}')

"""### **Train-Test Split**

**Summary:** Dataset partitioned into 70% training (3,325 samples) and 30% test (1,425 samples) with stratification.

**Observations:** Stratified split ensures representative class distribution across both sets.

"""

# ------------------------------
# TRAIN / VALIDATION / TEST SPLIT
# ------------------------------
# Splits data 80/10/10 with stratification to preserve class proportions.

# First split: 80% train, 20% holdout.
X_train, X_temp, y_train, y_temp = train_test_split(images_norm, labels_encoded, test_size=0.2, random_state=42, stratify=labels)

# Second, split Temp into 50% Val and 50% Test (which is 10% and 10% of total)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=42, stratify=lb.inverse_transform(y_temp))

print(f'Training set shape: {X_train.shape}')
print(f'Validation set shape: {X_val.shape}')
print(f'Test set shape: {X_test.shape}')

"""### **Data Augmentation**

**Summary:** ImageDataGenerator applied stochastic rotations, zooms, and shifts to artificially expand the training set.

**Observations:** Augmentation multiplies effective dataset size, forcing the model to learn robust features and reducing overfitting risk.

"""

# ------------------------------
# DATA AUGMENTATION
# ------------------------------
# Configures real-time augmentation (rotation, zoom, shift, flip) to improve generalization.
# keras.preprocessing.image.ImageDataGenerator applies transforms during training.

datagen = ImageDataGenerator(
    rotation_range=20,
    zoom_range=0.1,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True
)

datagen.fit(X_train)

# Visualize augmentation on one image
plt.figure(figsize=(10, 2))
for i in range(5):
    img = X_train[0].reshape((1, 128, 128, 3))
    batch = datagen.flow(img, batch_size=1)
    plt.subplot(1, 5, i+1)
    plt.imshow(batch[0][0])
    plt.axis('off')
plt.suptitle('Augmentation Examples')
plt.show()

"""### **Model Architecture (Custom CNN)**

**Summary:** A 3-block CNN (32→64→128 filters) with batch normalization and dropout was constructed.

**Observations:** Final softmax layer outputs probability distributions across the 12 target species.

"""

# ------------------------------
# CNN MODEL ARCHITECTURE
# ------------------------------
# Builds a three-block CNN (32→64→128 filters) with batch normalization and dropout.

model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)),
    BatchNormalization(),
    MaxPooling2D((2, 2)),

    Conv2D(64, (3, 3), activation='relu'),
    BatchNormalization(),
    MaxPooling2D((2, 2)),

    Conv2D(128, (3, 3), activation='relu'),
    BatchNormalization(),
    MaxPooling2D((2, 2)),

    Flatten(),
    Dense(256, activation='relu'),
    Dropout(0.5),
    Dense(12, activation='softmax')
])

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()

"""### **Model Training (Custom CNN)**

**Summary:** The CNN was trained for 25 epochs with learning rate reduction on validation plateau.

**Observations:** Training converged with steady accuracy improvement; LR callback dynamically adjusted rate when validation stalled.

"""

# ------------------------------
# MODEL TRAINING
# ------------------------------
# Trains the CNN with learning rate reduction on plateau and best-model checkpointing.

# Halves learning rate after 3 epochs of no improvement.
reduce_lr = ReduceLROnPlateau(monitor='val_accuracy', factor=0.5, patience=3, min_lr=0.00001, verbose=1)
# Saves weights when validation loss hits a new minimum.
checkpoint = ModelCheckpoint('best_model.keras', monitor='val_loss', save_best_only=True, mode='min', verbose=1)

# Use datagen.flow for training
history = model.fit(datagen.flow(X_train, y_train, batch_size=64),
                    epochs=25,  # Increased epochs slightly given augmentation
                    validation_data=(X_val, y_val),
                    callbacks=[reduce_lr, checkpoint])

# ------------------------------
# LOAD BEST MODEL
# ------------------------------
# Restores the best checkpoint; falls back to the current model if unavailable.

try:
    model = tf.keras.models.load_model('best_model.keras')
    print("Loaded best model from checkpoint.")
except:
    print("Could not load checkpoint, using current model state.")

"""### **Model Evaluation (Custom CNN)**

**Summary:** Training/validation curves, confusion matrix, and per-class classification report were generated.

**Observations:** 89% test accuracy with Fat Hen at 0.99 F1 and Shepherds Purse at 0.55 F1 — visually similar species (Black-grass/Loose Silky-bent) are the primary confusion source.

"""

# ------------------------------
# MODEL EVALUATION
# ------------------------------
# Evaluates the CNN with training curves, confusion matrix, and per-class classification report.

plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['accuracy'], label='Train Accuracy')
plt.plot(history.history['val_accuracy'], label='Val Accuracy')
plt.legend()
plt.title('Accuracy')

plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Train Loss')
plt.plot(history.history['val_loss'], label='Val Loss')
plt.legend()
plt.title('Loss')
plt.show()

# Confusion Matrix and Classification Report
y_pred = model.predict(X_test)
y_pred_classes = np.argmax(y_pred, axis=1)
y_true_classes = np.argmax(y_test, axis=1)

plt.figure(figsize=(10, 8))
cm = confusion_matrix(y_true_classes, y_pred_classes)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=lb.classes_, yticklabels=lb.classes_)
plt.xlabel('Predicted')
plt.ylabel('True')
plt.title('Confusion Matrix')
plt.show()

# Per-species precision, recall, and F1.
print(classification_report(y_true_classes, y_pred_classes, target_names=lb.classes_))

"""### **Prediction Visualization**

**Summary:** Individual test images were sampled to qualitatively verify model predictions.

**Observations:** Most predictions are correct; failures correspond to high inter-class visual similarity or lower image quality.

"""

# ------------------------------
# PREDICTION VISUALIZATION
# ------------------------------
# Displays five random test images with true vs. predicted labels as a sanity check.

plt.figure(figsize=(15, 5))
for i in range(5):
    idx = np.random.randint(0, len(X_test))
    img = X_test[idx]
    true_label = lb.classes_[y_true_classes[idx]]
    pred_label = lb.classes_[y_pred_classes[idx]]

    plt.subplot(1, 5, i+1)
    plt.imshow(img)
    plt.title(f'True: {true_label}\nPred: {pred_label}')
    plt.axis('off')
plt.show()

"""### **Conclusion and Insights (Part A)**

**Summary:** Custom CNN performance was evaluated and future improvement directions identified.

**Observations:** Model is effective at 90.3% but gains on underrepresented species require advanced augmentation or transfer learning.

"""

# ------------------------------
# MODEL INSIGHTS
# ------------------------------
# Summarizes findings and identifies improvement opportunities for the next iteration.

print("1. The model performs well overall but struggles with species that have similar leaf structures.")
print("2. Data augmentation (rotation, zooming) could further improve generalization.")
print("3. Class weighting could address the imbalance observed in the EDA phase.")

"""### **Transfer Learning (MobileNetV2)**

**Summary:** MobileNetV2 pretrained on ImageNet was fine-tuned on the last 30 layers with cosine-decay LR and TTA.

**Observations:** Despite leveraging ImageNet knowledge, MobileNetV2 achieved only 82.1% — underperforming the custom CNN on this small, domain-specific dataset.

"""

# ------------------------------
# TRANSFER LEARNING — MOBILENETV2
# ------------------------------
# Builds a MobileNetV2 transfer learning model with a frozen pretrained base and custom head.
# keras.applications.MobileNetV2 provides the pretrained feature extractor.

from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import GlobalAveragePooling2D
from tensorflow.keras.models import Model

# Load MobileNetV2 without top layers, input size 128x128
base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(128, 128, 3))

# Freeze the base model initially
base_model.trainable = False

# Add custom head
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(128, activation='relu')(x)
x = Dropout(0.5)(x)
predictions = Dense(12, activation='softmax')(x)

model_b = Model(inputs=base_model.input, outputs=predictions)

model_b.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_b.summary()

# ------------------------------
# FINE-TUNING SETUP
# ------------------------------
# Unfreezes the last 30 MobileNetV2 layers with cosine-decay learning rate to avoid catastrophic forgetting.

base_model.trainable = True
for layer in base_model.layers[:-30]:
    layer.trainable = False

# Implement Cosine Decay Learning Rate
initial_learning_rate = 0.0001
decay_steps = 1000
cosine_decay = tf.keras.optimizers.schedules.CosineDecay(initial_learning_rate, decay_steps)

# Recompile with low learning rate
model_b.compile(optimizer=Adam(learning_rate=cosine_decay), loss='categorical_crossentropy', metrics=['accuracy'])
print("Model B re-compiled for fine-tuning.")

# ------------------------------
# TRANSFER LEARNING TRAINING
# ------------------------------
# Trains the fine-tuned MobileNetV2 for 20 epochs with checkpointing.

checkpoint_b = ModelCheckpoint('best_model_b.keras', monitor='val_loss', save_best_only=True, mode='min', verbose=1)

# Use the same datagen as before
history_b = model_b.fit(datagen.flow(X_train, y_train, batch_size=64),
                        epochs=20,  # Fine-tune for 20 epochs
                        validation_data=(X_val, y_val),
                        callbacks=[checkpoint_b])

# ------------------------------
# TEST-TIME AUGMENTATION
# ------------------------------
# Averages predictions over multiple augmented copies of each test image to reduce variance.

# Generates n augmented versions per image and averages their softmax outputs.
def predict_tta(model, images, n_aug=5):
    predictions = []
    for img in images:
        # Create batch of augmented versions
        img_expanded = img.reshape((1, 128, 128, 3))
        aug_images = [datagen.random_transform(img) for _ in range(n_aug)]
        aug_batch = np.array(aug_images)

        # Predict on augmented batch
        preds = model.predict(aug_batch, verbose=0)

        # Average predictions
        avg_pred = np.mean(preds, axis=0)
        predictions.append(avg_pred)
    return np.array(predictions)

print("Running TTA on Model B...")
# Load best model B
try:
    model_b = tf.keras.models.load_model('best_model_b.keras')
except:
    pass

y_pred_b = predict_tta(model_b, X_test, n_aug=5)
y_pred_classes_b = np.argmax(y_pred_b, axis=1)
print("TTA Predictions Complete.")

# ------------------------------
# TRANSFER LEARNING EVALUATION
# ------------------------------
# Evaluates Model B with training curves and classification report.

plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history_b.history['accuracy'], label='Train Acc')
plt.plot(history_b.history['val_accuracy'], label='Val Acc')
plt.title('Model B Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(history_b.history['loss'], label='Train Loss')
plt.plot(history_b.history['val_loss'], label='Val Loss')
plt.title('Model B Loss')
plt.legend()
plt.show()

print(classification_report(y_true_classes, y_pred_classes_b, target_names=lb.classes_))

"""### **Model Comparison (Part A vs Part B)**

**Summary:** Custom CNN and MobileNetV2 + TTA were compared on test accuracy and F1.

**Observations:** Custom CNN (90.3%) decisively outperforms MobileNetV2 (82.1%), confirming the purpose-built architecture generalizes better on 4,750 samples.

"""

# ------------------------------
# MODEL COMPARISON
# ------------------------------
# Compares test accuracy between the custom CNN and MobileNetV2 + TTA.
# sklearn.metrics.accuracy_score computes overall accuracy for each model.

from sklearn.metrics import accuracy_score

acc_a = accuracy_score(y_true_classes, y_pred_classes)
acc_b = accuracy_score(y_true_classes, y_pred_classes_b)

results = pd.DataFrame({
    'Model': ['Custom CNN (Part A)', 'MobileNetV2 + TTA (Part B)'],
    'Test Accuracy': [acc_a, acc_b]
})

print(results)

plt.figure(figsize=(6, 4))
sns.barplot(x='Model', y='Test Accuracy', data=results)
plt.title('Model Performance Comparison')
plt.ylim(0, 1.0)
plt.show()

"""### **Expanded Executive Summary**

**TLDR**

The Custom CNN achieves 90.3% accuracy, outperforming MobileNetV2 + TTA (82.1%) despite being a simpler architecture. The smaller model generalizes better on this 4,750-sample dataset and is more practical for edge deployment on existing agricultural equipment, where computational resources are constrained.

**Full Summary**

**Objective:** This project established a Computer Vision pipeline for automated plant seedling classification, targeting recovery of the estimated $32B in annual crop losses driven by uncontrolled weed growth. The ability to distinguish 12 species visually enables precision herbicide application and automated sorting.

**Iterative Development:** Two architectures were compared — a purpose-built 3-block CNN trained from scratch and a MobileNetV2 model leveraging ImageNet transfer learning with test-time augmentation. Both were trained on 4,750 labeled images with stratified splits and real-time data augmentation to combat overfitting.

**Performance Analysis:** The Custom CNN achieved 90.3% test accuracy with Fat Hen reaching 0.99 F1, while Shepherds Purse (0.55 F1) and Black-grass (0.59 F1) remain challenging due to high inter-species visual similarity. MobileNetV2 underperformed at 82.1%, particularly struggling with minority classes like Common wheat (0.59 F1) and Black-grass (0.44 F1).

**Economic Impact:** Deploying this detection capability targets the \$32B annual yield loss from weeds and aligns with the precision weeding market projected to reach $10.65B by 2034. Field studies show vision-based systems can reduce herbicide use by 60–90%, directly lowering input costs.

**Deployment Readiness:** The lighter custom CNN is immediately viable for edge deployment on agricultural drones, robotic platforms, or existing sprayer booms. Retaining MobileNetV2 provides ensemble opportunities and architectural flexibility for future iterations.


"""
