|
Blackberry crop production is an essential sector of high-value specialty crops. Blackberries are delicate and easy to be damaged during harvest process. Besides, the blackberries in an orchard are not ripe at the same time so that multiple passes of harvesting are often needed. Therefore, the production is highly labor intensive and could be addressed using robotic solutions while maintaining the post-harvest berry quality for desired profitability. To further empower the developed tendon-driven soft robotic gripper specifically designed for berries, this study aims at investigating a state-of-the-art deep-learning YOLOv7 for accurately detecting the blackberries at multi-ripeness level in field conditions. In-field blackberry localization is a challenging task since blackberries are small objects and differ in color due to various levels of ripeness. Furthermore, the outdoor light condition varies depending on the time of day/location. Our study focused on detecting in-field blackberries at multi-ripeness levels using the state-of-the-art YOLOv7 model. In total, 642 RGB images were acquired targeting the plant canopies in several commercial orchards in Arkansas. The images were augmented to increase the diversity of data set using various methods. There are mainly three ripeness levels of blackberries that can present simultaneously in individual plants, including ripe (in black color), ripening (in red color), and unripe berries (in green color). The differentiation of ripeness levels can help the system to specifically harvest the ripe berries, and to keep track of the ripening/unripe berries in preparation for the next harvesting pass. The aggregation of total number of berries at all ripeness levels can also help estimate the crop-load for growers. The YOLOv7 model with seven configurations and six variants were trained and validated with 431 and 129 images, respectively. Overall, results of the test set (82 images) showed that YOLOv7-base was the best configuration with mean average precision (mAP) of 91.4% and F1-score of 0.86. YOLOv7-base also achieved 94% of mAP and 0.93 of True Positives (TPs) for ripe berries, 91% and 0.88 for ripening berries, and 88% and 0.86 for unripe berries under the Intersection-over-Union (IoU) of 0.5. The inference speed for YOLOv7-base was 21.5 ms on average per image with 1,024x1,024 resolution.
|
