UMPSA STEM Lab – Edge Intelligence (Image Processing) on ESP32 2026

Deploying artificial intelligence directly on compact microcontrollers—often termed Edge Intelligence—represents a major leap in embedded engineering. This UMPSA STEM Lab module provides a hands-on roadmap for transitioning from basic software-based computer vision to standalone, hardware-integrated AI models capable of performing real-time image processing, feature extraction, and object classification on low-power ESP microcontroller hardware.

The Shift to Edge Intelligence

Traditional computer vision solutions rely on cloud servers or heavy laptop GPUs to process video streams. Edge Intelligence moves processing power directly to the physical sensor level. In this module, participants move beyond pre-packaged datasets to create, train, and deploy bespoke computer vision pipelines designed specifically for embedded micro-processors.

Image Processing & Data Engineering Discipline

A fundamental lesson of embedded machine learning is that an AI model is only as effective as the data used to train it.

Figure 1: The compact ESP32 camera board featuring an integrated 2 megapixel camera sensor and Wi-Fi capability.

The Thumbnail Test & Image Resolution

      1. Pixel Reduction Raw images captured by the sensor are downscaled to 96 x 96 pixel grayscale or RGB arrays.
      2. Visual Clarity Before training, images are evaluated via the “thumbnail test” to ensure visual features remain recognizable when shrunk to stamp-sized dimensions.

Essential Rules for Dataset Quality

        1. Single Variable Changes Change only one condition (angle, lighting, position) between consecutive photo captures to maximize information density.
        2. Background Separation Avoid shooting all target objects on identical surfaces. Otherwise, the model learns background cues rather than object features.
        3. Consistent Label Formatting Labels must follow strict naming rules (lower case, no spaces, e.g., red_chili) to prevent duplicate target classes.

3. Feature Extraction & Impulse Pipeline Design

Once images are collected and labeled, raw pixel values are mapped into numerical feature vectors inside Edge Impulse.

Figure 2: The four-block impulse design connecting input image data to digital signal processing (DSP) and neural network classifiers.

      1. Raw Image Preprocessing Images are normalized and structured into uniform dimensions.
      2. DSP Feature Generation Spatial characteristics (edges, textures, color distributions) are extracted, generating distinct visual clusters in 3D feature space.
      3. Classifier Training Neural networks process the extracted features to assign probabilities across classes.
      4. Inference Speed Real-time classification executes in approximately 1 ms per frame, enabling fluid live monitoring.

4. Bounding Boxes, Confidence Thresholds, and the “Nothing” Class

A key challenge in real-world vision deployment is handling unexpected inputs—a scenario highlighted by the “chili problem“.

Challenge Cause Embedded Solution
False Positives The model forces unseen objects into known classes. Add an explicit background or “nothing” class.
Fluctuating Predictions Minor shifts in lighting or tilt alter confidence scores. Apply confidence thresholds (e.g., ignore results below 0.80).
Spatial Localization Image classification lacks coordinate location. Implement bounding box object detection for region tracking.

5. Microcontroller Integration on ESP32 Hardware

The culmination of the module is flashing the trained impulse directly onto an ESP microcontroller processor.

Figure 3: Real-time classification outputs streamed over the Arduino Serial Monitor at 115200 baud.

Hardware Deployment Steps

      1. Firmware Setup Configure the Arduino IDE with necessary ESP32 board definitions and upload the custom collection sketch (ESP32_datacollection.ino).
      2. Port Selection & Cabling Ensure high-speed data cables are used, as charge-only cables will fail to establish COM communication.
      3. Network Configurations Connect via self-generated Wi-Fi access points (Route A) or local network infrastructure (Routes B/C).
      4. Serial Verification Initialize the Serial Monitor at 115200 baud and press the hardware reset (RST) button to verify initialization and monitor live inference logs.

6. Key Takeaways

Through this module, participants develop end-to-end expertise in embedded machine learning:

    1. Moving from cloud-dependent AI to standalone micro-processor execution.
    2. Mastering raw image preprocessing, feature extraction, and dataset hygiene.
    3. Deploying real-time computer vision models on low-cost hardware platforms.