Sweet Pepper Detection Using Fast Point Features Histogram and Unsupervised Learning

Document Type : Research Article-en

Authors

• O. Doosti Irani
• M. H. Aghkhani
• M. R. Golzarian

Department of Biosystems Engineering, Faculty of Agriculture, Ferdowsi University of Mashhad, Mashhad, Iran

https://doi.org/10.22067/jam.2023.83054.1174

Abstract

Robotic harvesting in agriculture is an effective method for producing healthy fruit, reducing costs, and increasing productivity. Detecting and harvesting sweet peppers, however, remains a challenging task. This study aims to develop an unsupervised machine vision algorithm to recognize colored sweet peppers using a combination of geometric features (Fast Point Feature Histogram- FPFH) and color features (H, S, and V). Depth images were captured using a Kinect v2 sensor, and a 3D model was reconstructed. After extracting the geometric and color features, data preprocessing involved undersampling to ensure balance and applying the Z-score criterion to eliminate outliers. Principal component analysis (PCA) was used to reduce the feature dimensions, and the K-means clustering model was implemented to categorize the data using six geometric features and three color features. The silhouette coefficient was employed to evaluate clustering quality, and human evaluation demonsterated that the algorithm achieved a detection accuracy of 95.10% for sweet peppers.

Keywords

• Fast point feature histogram
• Fruit detection
• Machine vision
• Sweet pepper
• Unsupervised learning

Main Subjects

• Image Processing

©2025 The author(s). This is an open access article distributed under Creative Commons Attribution 4.0 International License (CC BY 4.0)

References

1. Adams, T., Nolen, S., Sweezy, J., Zukaitis, A., Campbell, J., Goorley, T., ... & Aulwes, R. (2015). Monte Carlo application toolkit (MCATK). Annals of Nuclear Energy, 82, 41-47. https://doi.org/10.1016/j.anucene.2014.08.047
2. Ball, D., Upcroft, B., Wyeth, G., Corke, P., English, A., Ross, P., ... & Bate, A. (2016). Vision‐based obstacle detection and navigation for an agricultural robot. Journal of Field Robotics, 33(8), 1107-1130. https://doi.org/10.1002/rob.21644
3. Behley, J., Steinhage, V., & Cremers, A. B. (2012, May). Performance of histogram descriptors for the classification of 3D laser range data in urban environments. In 2012 IEEE international conference on robotics and automation (pp. 4391-4398). IEEE. https://doi.org/10.1109/icra.2012.6225003
4. Chidambaranathan, C. M., Handa, S. S., Ramanamurthy, M. V., & Ramanamurthy, M. V. (2018). Development of smart farming-a detailed study. International Journal of Engineering & Technology, 7(2.4), 56. https://doi.org/10.14419/ijet.v7i2.4.10042
5. Dharmaraj, V., & Vijayanand, C. (2018). Artificial intelligence (AI) in agriculture. International Journal of Current Microbiology and Applied Sciences, 7(12), 2122-2128. https://doi.org/10.20546/ijcmas.2018.712.241
6. Doosti-Irani, O., Golzarian, M. R., & Aghkhani, M. H. (2023). Automatic recognition of sweet peppers based on the fast point features histogram (FPFH) 3-D descriptor and machine learning. Journal of Researches in Mechanics of Agricultural Machinery, 12(1), 27-40. https://doi.org/22034/jrmam.2023.13863.587
7. Fu, L., Majeed, Y., Zhang, X., Karkee, M., & Zhang, Q. (2020). Faster R–CNN–based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosystems Engineering, 197, 245-256. https://doi.org/10.1016/j.biosystemseng.2020.07.007
8. Gan, H., Lee, W. S., Alchanatis, V., Ehsani, R., & Schueller, J. K. (2018). Immature green citrus fruit detection using color and thermal images. Computers and Electronics in Agriculture, 152, 117-125. https://doi.org/10.1016/j.compag.2018.07.011
9. Gongal, A., Amatya, S., Karkee, M., Zhang, Q., & Lewis, K. (2015). Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture, 116, 8-19. https://doi.org/10.1016/j.compag.2015.05.021
10. Gongal, A., Silwal, A., Amatya, S., Karkee, M., Zhang, Q., & Lewis, K. (2016). Apple crop-load estimation with over-the-row machine vision system. Computers and Electronics in Agriculture, 120, 26-35. https://doi.org/10.1016/j.compag.2015.10.022
11. Han, X. F., Sun, S. J., Song, X. Y., & Xiao, G. Q. (2018). 3D point cloud descriptors in hand-crafted and deep learning age: State-of-the-art. arXiv preprint arXiv:1802.02297. https://doi.org/10.48550/arXiv.1802.02297
12. Hemming, J., Bac, C. W., & Van Tuijl, B. A. J. (2011). CROPS project deliverable 5.1: Report with design objectives and requirements for sweet-pepper harvesting. Wageningen, The Netherlands: Wageningen UR Greenhouse Horticulture.
13. Javidan, S. M., Banakar, A., Vakilian, K. A., & Ampatzidis, Y. (2023). Diagnosis of grape leaf diseases using automatic K-means clustering and machine learning. Smart Agricultural Technology, 3, 100081. https://doi.org/10.1016/j.atech.2022.100081
14. Kurtulmus, F., Lee, W. S., & Vardar, A. (2014). Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network. Precision Agriculture, 57-79. https://doi.org/10.1007/s11119-013-9323-8
15. Lachat, E., Macher, H., Mittet, M. A., Landes, T., & Grussenmeyer, P. (2015). First experiences with kinect v2 sensor for close range 3d modelling. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences. https://org/10.5194/isprsarchives-XL-5-W4-93-2015
16. Moghimi, A., Aghkhani, M. H., & Golzarian, M. R. (2015). Desigining of Computer Vision Algorithm to Detect Sweet Pepper for Robotic Harvesting Under Natural Light. Journal of Agricultural Machinery, 5(1), 82-91. https://doi.org/10.22067/jam.v5i1.23528
17. Mohamadzamani, D., Javidan, S. M., Zand, M., & Rasouli, M. (2023). Detection of Cucumber Fruit on Plant Image Using Artificial Neural Network. Journal of Agricultural Machinery, 13(1), 27-39. https://doi.org/10.22067/jam.2022.73827.1077
18. Mohammadi, P., Massah, J., & Asefpour Vakilian, K. (2023). Robotic date fruit harvesting using machine vision and a 5‐DOF manipulator. Journal of Field Robotics. https://doi.org/10.1002/rob.22184
19. Muja, M., Rusu, R. B., Bradski, G., & Lowe, D. G. (2011, May). Rein-a fast, robust, scalable recognition infrastructure. In 2011 IEEE international conference on robotics and automation (pp. 2939-2946). IEEE. https://doi.org/10.1109/icra.2011.5980153
20. Nan, Y., Zhang, H., Zeng, Y., Zheng, J., & Ge, Y. (2023). Faster and accurate green pepper detection using NSGA-II-based pruned YOLOv5l in the field environment. Computers and Electronics in Agriculture, 205, 107563. https://doi.org/10.1016/j.compag.2022.107563
21. Nguyen, T. T., Vandevoorde, K., Kayacan, E., De Baerdemaeker, J., & Saeys, W. (2014, July). Apple detection algorithm for robotic harvesting using a RGB-D camera. In International Conference of Agricultural Engineering, Zurich, Switzerland.
22. Ning, Z., Luo, L., Ding, X., Dong, Z., Yang, B., Cai, J., ... & Lu, Q. (2022). Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards. Computers and Electronics in Agriculture, 196, 106878. https://doi.org/10.1016/j.compag.2022.106878
23. Ringdahl, O., Kurtser, P., & Edan, Y. (2019). Evaluation of approach strategies for harvesting robots: Case study of sweet pepper harvesting: Category:(5). Journal of Intelligent & Robotic Systems, 95(1), 149-164. https://doi.org/10.1007/s10846-018-0892-7
24. Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53-65. https://doi.org/10.1016/0377-0427(87)90125-7
25. Rusu, R. B., & Cousins, S. (2011, May). 3d is here: Point cloud library (pcl). In 2011 IEEE international conference on robotics and automation (pp. 1-4). IEEE. https://doi.org/10.1109/icra.2011.5980567
26. Rusu, R. B., Blodow, N., & Beetz, M. (2009, May). Fast point feature histograms (FPFH) for 3D registration. In 2009 IEEE international conference on robotics and automation (pp. 3212-3217). IEEE. https://doi.org/10.1109/robot.2009.5152473
27. Rusu, R. B., Marton, Z. C., Blodow, N., & Beetz, M. (2008, July). Persistent point feature histograms for 3D point clouds. In Proc 10th Int Conf Intel Autonomous Syst (IAS-10), Baden-Baden, Germany (pp. 119-128). https://doi.org/10.3233/978-1-58603-887-8-119
28. Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., & McCool, C. (2016). Deepfruits: A fruit detection system using deep neural networks. Sensors, 16(8), 1222. https://doi.org/10.3390/s16081222
29. Sa, I., Lehnert, C., English, A., McCool, C., Dayoub, F., Upcroft, B., & Perez, T. (2017). Peduncle detection of sweet pepper for autonomous crop harvesting—combined color and 3-D information. IEEE Robotics and Automation Letters, 2(2), 765-772. https://doi.org/1109/LRA.2017.2651952
30. Shamshiri, R. R., Hameed, I. A., Karkee, M., & Weltzien, C. (2018). Robotic harvesting of fruiting vegetables: A simulation approach in V-REP, ROS and MATLAB. Proceedings in Automation in Agriculture-Securing Food Supplies for Future Generations, 126, 81-105. https://doi.org/5772/intechopen.73861
31. Shen, D., Wu, G., & Suk, H. I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221-248. https://doi.org/10.1146/annurev-bioeng-071516-044442
32. Stein, M., Bargoti, S., & Underwood, J. (2016). Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors, 16(11), 1915. https://doi.org/10.3390/s16111915
33. Tang, Y., Chen, M., Wang, C., Luo, L., Li, J., Lian, G., & Zou, X. (2020). Recognition and localization methods for vision-based fruit picking robots: A review. Frontiers in Plant Science, 11, 510. https://doi.org/10.3389/fpls.2020.00510
34. Tao, Y., & Zhou, J. (2017). Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking. Computers and Electronics in Agriculture, 142, 388-396. https://doi.org/10.1016/j.compag.2017.09.019
35. Wan, Y., Li, Y., Jiang, J., & Xu, B. (2020, March). Edge Voxel Erosion for Noise Removal in 3D Point Clouds Collected by Kinect. In Proceedings of the 2020 2nd International Conference on Image, Video and Signal Processing (pp. 59-63). https://doi.org/10.1145/3388818.3388821
36. Zhao, X., Li, H., Zhu, Q., Huang, M., Guo, Y., & Qin, J. (2020). Automatic sweet pepper detection based on point cloud images using subtractive clustering. International Journal of Agricultural and Biological Engineering, 13(3), 154-160. https://doi.org/10.25165/j.ijabe.20201303.5460