Ir. Dr. Nurul Hazlina Noordin

Read Article

The palm oil industry plays a critical role in Malaysia’s agricultural economy, where accurate and timely harvesting of Fresh Fruit Bunches (FFB) directly impacts oil yield and quality. Traditionally, ripeness assessment relies heavily on manual inspection, which is subjective, labor-intensive, and inconsistent under varying field conditions.

To address these challenges, our research introduces a real-time, IoT-enabled Edge AI system for automatic palm oil FFB ripeness detection using a YOLOv8n deep learning model deployed on a Raspberry Pi 4. The system enables on-site intelligence, minimizes data transmission latency, and supports smarter plantation decision-making.

 Figure 1: Block Diagram of the Proposed System

The proposed system integrates computer vision, edge computing, and IoT services into a compact and deployable architecture. The core components include:

1. 1. 1. Camera Module: Captures real-time images of palm oil FFB either on-tree or post-harvest (on the ground).

      2. Edge Device (Raspberry Pi 4): Executes the YOLOv8n model locally for real-time inference.

      3. Deep Learning Model (YOLOv8n): Detects and classifies palm fruit ripeness stages.

      4. IoT Communication Layer: Transmits detection results and metadata to the cloud.

      5. Web-Based Dashboard: Visualizes ripeness distribution and system performance for plantation managers.

By processing data at the edge, the system significantly reduces dependence on cloud computing while maintaining high detection accuracy.

Model Training and Testing Workflow

Figure 2: Training and Testing Workflow

A supervised deep learning approach was adopted for model development:

1. Dataset Preparation
   Palm oil FFB images were collected under real plantation conditions and annotated according to ripeness stages.

2. Annotation and Augmentation
   Image labeling and dataset management were performed using Roboflow.
   Data augmentation techniques such as:

   • Image flipping

   • Rotation

   • Brightness and exposure adjustment

   were applied to improve generalization and prevent overfitting.

3. Model Fine-Tuning
   A pre-trained YOLOv8n model was fine-tuned on the custom dataset to balance accuracy and computational efficiency.

4. Training and Validation
   The dataset was split into training, validation, and testing subsets to ensure reliable performance evaluation.

This workflow ensures that the trained model is robust to real-world lighting variations and plantation environments.

Model Evaluation Indicators

Figure 3: Model Evaluation Metrics (Precision, Recall, mAP)

To evaluate the effectiveness of the proposed model, several standard performance metrics were used:

1. 1. 1. Precision – Measures the accuracy of positive detections.

      2. Recall – Indicates the model’s ability to detect all relevant ripe fruit instances.

      3. Mean Average Precision (mAP) – Summarizes detection performance across confidence thresholds.

The YOLOv8n model achieved high precision and recall, demonstrating reliable detection while maintaining real-time inference capability on a resource-constrained edge device.

Compared to earlier YOLOv4 and YOLOv5 implementations reported in the literature, the proposed approach delivers a balanced trade-off between accuracy, speed, and deployment efficiency, making it well-suited for Edge AI applications.

Real-Time Deployment Scenarios

The system was evaluated under two practical plantation scenarios:

1. 1. 1. On-tree FFB detection (pre-harvest)

      2. On-ground FFB detection (post-harvest)

Both scenarios reflect real operational conditions and validate the system’s adaptability for field deployment.

IoT Dashboard for Plantation Monitoring

Figure 4: Web-Based Monitoring Dashboard

A web-based dashboard was developed to support estate plantation management, providing:

• • 1. Real-time visualization of detected FFB ripeness levels

    2. Historical data tracking

    3. Summary statistics for harvesting planning

    4. System status monitoring

The dashboard enables plantation managers to make data-driven decisions without requiring technical expertise in AI or computer vision.

Why Edge AI Matters for Agriculture

Deploying AI directly on edge devices offers several advantages, among them:

1. 1. 1. Low latency – Immediate decision-making without cloud delays

      2. Reduced bandwidth usage – Only essential data is transmitted

      3. Improved privacy – Images remain local to the device

      4. Scalability – Suitable for large plantation deployments

This makes Edge AI particularly attractive for remote and resource-limited agricultural environments.

STEM Learning Impact 

This project has been a highly engaging learning experience, providing students and researchers with hands-on exposure to image processing, Artificial Intelligence (AI), and Edge Computing. By working with YOLO-based object detection, participants gained practical insight into how AI models learn visual patterns, process real-world images, and make intelligent decisions. YOLO, as a deep learning–based computer vision algorithm, represents a core component of modern AI applications used in industry today.

Through this project, learners developed essential 21st-century STEM skills, including data annotation, model training and evaluation, system integration, and deployment on resource-constrained edge devices. Importantly, participants were able to see how AI moves beyond theory into real-world problem solving, particularly within the context of precision agriculture.

Looking ahead, this work provides a strong foundation for expanding AI-driven characterization methods, such as fruit size estimation, defect detection, and maturity analysis. The system can also be adapted for wider agricultural applications, including crop monitoring, yield estimation, and decision support for plantation management.

From a technology perspective, future efforts will explore deployment on diverse processors and edge platforms, enabling comparisons across low-power embedded systems and AI accelerators. This approach supports the development of cost-effective, scalable, and energy-efficient AI solutions, making advanced technology more accessible to local communities and industries.

Overall, this project demonstrates how STEM-based AI initiatives can nurture innovation, strengthen digital competencies, and contribute to sustainable agricultural practices—aligning with national priorities in Industry 4.0, smart farming, and digital transformation.

Reflections and Future Directions

Working on this project has been both technically enriching and intellectually exciting, as it allowed us to deepen our understanding of image processing and Artificial Intelligence (AI) through hands-on experimentation. In particular, implementing YOLO-based object detection, a deep learning approach within the broader AI domain, provided valuable insights into how modern computer vision models learn visual features, make real-time predictions, and operate under resource constraints on edge devices.

Beyond ripeness classification, this project opens opportunities to expand AI-driven characterization methods, including size estimation, defect detection, maturity grading, and yield prediction. We also look forward to extending these techniques to broader agricultural applications, such as crop health monitoring, disease detection, and automated harvesting support.

From a systems perspective, future work will explore optimization and deployment across different processors and edge computing platforms, including alternative single-board computers, AI accelerators, and low-power embedded systems. This aligns with the growing need for scalable, cost-effective, and energy-efficient Edge AI solutions in real-world agricultural environments.

Overall, this project highlights how integrating image processing, deep learning, and edge computing can drive practical innovation in precision agriculture while serving as a strong learning platform for AI and embedded system development.

Acknowledgement

This work was conducted at the UMPSA STEM Lab, Faculty of Electrical and Electronics Engineering Technology, Universiti Malaysia Pahang Al-Sultan Abdullah (UMPSA), with support from academic, industry, and student collaborators.

understanding the impact of reach, engagement and behavior