Deep learning based Real-time GPU-accelerated tracking and counting of cotton bolls under field conditions using a moving camera
Loading...
Date
2018-08
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
2018 ASABE Annual International Meeting
Abstract
Robotic harvesting involves navigation and environmental perception as first operations before harvesting
of the bolls can commence. Navigation is the distance required for a harvester’s arm to reach the cotton boll while
perception is the position of the boll relative to surrounding environment. These two operations give a 3D position of the
cotton boll for picking and can only be achieved by detection and tracking of the cotton bolls in real-time. It means
detection, tracking and counting of cotton bolls using a moving camera allows the robotic machine to harvest easily.
GPU-accelerated deep neural networks were used to train the convolution networks for detection of cotton bolls. It was
achieved by using pretrained tiny yolo weights and DarkFlow, a framework which translates YOLOv2 darknet neural
networks to TensorFlow. A method to connect tracklets using vectors that are predicted using Lucas-Kanade algorithm
and optimized using robust L-estimators and homography transformation is proposed. The system was tested in defoliated
cotton plants during the spring of 2018. Using three video treatments, the counting performance accuracy was around
93% with standard deviation 6%. The system average processing speed was 21 fps in desktop computer and 3.9 fps in
embedded system. Detection of the system achieved an accuracy and sensitivity of 93% while precision was 99.9% and
F1 score was 1. The Tukey’s test showed that the system accuracy and sensitivity was the same when the plants were
rearranged. This performance is crucial for real-time robot decisions that also measure yield while harvesting.
Description
Conference Paper
Keywords
Boll counting, Cotton Bolls, Cotton counting, Cotton harvesting, DarkFlow, Darknet, Deep Learning, GPU, machine vision, TensorFlow, YOLO