In pursuit of an accurate and dependable image-based fruit detection system to support advanced agricultural tasks such as yield mapping and robotic harvesting, the authors of the ACFR Orchard Fruit Dataset (ACFR is Australian Centre for Robotics) dataset introduce the application of an object detection framework, Faster R-CNN. This framework is employed within the context of orchard fruit detection, encompassing various fruit types, including mangoes, almonds, and apples.

In agricultural automation, vision-based fruit detection stands as a pivotal element. It provides the foundation for in-field automation, enabling tasks such as yield estimation, mapping, and ultimately contributing to efficient resource utilization and enhanced returns per unit area and time. Additionally, precise fruit localization is an indispensable aspect of automated robotic harvesting systems, offering relief from one of the most labor-intensive challenges in orchard management.

Data Acquisition

The dataset under evaluation in this study encompasses three distinct fruit varieties: apple, almond, and mango. These data were collected during daylight hours at orchards located in Victoria and Queensland, Australia. For data collection, apple and mango data were acquired through sensors integrated into a versatile research ground vehicle developed at the Australian Centre for Field Robotics. This vehicle traversed different orchard rows to collect tree image data. Apple trees, which are trellised, allow the ground vehicle to maintain close proximity to the fruit. In contrast, the greater distance between mangoes and the ground vehicle was compensated for with a higher resolution sensor. In the mango orchard, external strobe lighting with a brief exposure time (approximately 70 µs) was employed to mitigate variable illumination artifacts. Almond trees, characterized by larger canopies and the potential to host between 1000 to 10,000 almonds, necessitated high-resolution imagery, which was acquired using a handheld DSLR camera. The images captured for each orchard encompassed entire trees, aligning with the primary research goal of efficient yield estimation and mapping. The fruit detection aspect presented in this paper constitutes a pivotal component of this overarching project.

The tree image data exhibited variations in size, ranging from 2 to 17 megapixels, with each image containing approximately 100 fruit for apples and mangoes, and over 1000 fruit for almonds. However, hardware constraints limited the usage of large images. For instance, a 0.25-megapixel image requires around 2.5 gigabytes of GPU memory with the VGG16 network. Additionally, labeling a substantial number of small objects within large images proved to be a perceptually challenging task. To address these challenges, the authors mitigated them by randomly sampling smaller sub-image patches from the pool of larger images obtained across the farm. This approach yielded smaller images, akin in size to the PASCAL-VOC dataset used with Faster R-CNN, featuring lower fruit counts while effectively covering data variability across the farm.

Labeling Process

The ground truth fruit annotations for almonds and mangoes were established through rectangular annotations, while circular annotations were better suited for apples. However, since Faster R-CNN operates on bounding box predictions, circular annotations were initially converted to rectangles with equal width and height. Note, that DatasetNinja converted circle bounding boxes to rectangular ones. In practice, labeling apples and mangoes was relatively more straightforward compared to almonds, primarily due to the size and contrast of the fruit in relation to the surrounding foliage and the canopy’s complexity. To facilitate fruit-background differentiation in shadowed regions of the image, the annotation software incorporated sliders for contrast and brightness adjustments.

Finally, each fruit’s labeled dataset was subdivided into training, validation, and testing sets. This division was carefully structured to ensure that each set comprised data collected from different sections of the orchard block, minimizing potential biases in the results. Images in the training set lacking any fruit presence were excluded from consideration.