Application of unmanned aerial systems and deep learning in high-throughput plant phenotyping

Plant phenotyping plays an essential role in decision support in precision agriculture and plant breeding. However, traditional phenotyping methods are typically through manually measuring the target traits, which is time-consuming and labor-intensive with sampling bias. Technological innovations in unmanned aerial systems (UAS) with various sensors provide high-resolution data for high-throughput plant phenotyping. UAS image-based machine learning algorithms have been applied in plant phenotyping. However, only limited studies have evaluated the performance of using deep learning and UAS imaging in cotton or plant phenotyping for breeding. Therefore, the goal of this study was to evaluate the performance of applications of using UAS images and deep learning algorithms in sorghum (Sorghum bicolor L. Moench) and cotton (Gossypium hirsutum L.) phenotyping. The objectives were to 1) develop a deep learning CNN image segmentation algorithm using UAS imagery to detect and quantify sorghum panicles; 2) to assess the application of MobileNet and CenterNet models in cotton stand counting at the seedling stage; 3) to develop an algorithm for detecting and counting open cotton bolls and assess the performance in relation to image acquisition altitudes and camera angles; 4) to develop a method for cotton boll detection using point cloud data derived from LiDAR and RGB data and to compare the performance of RGB images and LiDAR point in cotton boll detection. A set of 1000 UAS images, acquired at 10 m height, were randomly selected, and a mask was developed for each by manually delineating sorghum panicles for training the U-Net model for sorghum panicle detection. The algorithm performed the best with 1000 training images, with an accuracy of 95.5% and a root mean square error (RMSE) of 2.5, which showed the accuracy had a general increasing trend with the number of training images. The results indicated that the integration of image segmentation and the U-Net CNN model is an accurate and robust method for sorghum panicle counting and offers an opportunity for enhanced sorghum breeding efficiency and accurate yield estimation. UAS images were collected at 20 m flight height at the seedling stages on two dates in 2020 for cotton stand counting. The CenterNet model had a better overall performance for cotton plant detection and counting with 900 training images. More training images are required when applying object detection models on images with different dimensions from training datasets. Both MobileNet and CenterNet models have the potential to accurately and timely detect and count cotton plants based on high-resolution UAS images at the seedling stage. UAS imagery was collected from two angles, 60° to the land surface at 15 m and 90° at 10 m and 15 m flight height for open cotton boll detection in 2020. Open cotton bolls were detected with UAS images using the CenterNet model. Cotton boll count had more accurate predictions with UAS images taken with 60° camera angles. Models trained with images containing leaves performed more accurately than models trained with images only containing open bolls. UAS imagery acquired at oblique angles is effective for boll counting and offers an opportunity for enhanced cotton breeding efficiency and accurate yield estimation. LiDAR data was collected at 12 m and UAS imagery was collected with 90° and 60° cameras at 20 m height in 2021 for open cotton boll detection. Open cotton bolls were detected in LiDAR and UAS RGB-based point cloud data using the density-based spatial clustering of applications with noise (DBSCAN) algorithm. LiDAR-based point cloud data performed more accurately on cotton boll detection than RGB image-based point cloud data. Also, cotton boll count had more accurate predictions with small-sized plants. Point cloud data from LiDAR and UAS RGB imagery offer an opportunity for enhanced cotton breeding efficiency and accurate yield estimation. This study offers useful guidance for choosing appropriate deep learning models, remote sensing sensors, and the quantity of training data for computer vision tasks in agricultural applications. The proposed algorithms support decision-making in precision agriculture and plant breeding. Additional research is required to evaluate how image resolution, flight height, and environmental factors like soil background and light conditions affect plant phenotyping.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Crop Science

Grantor

Texas Tech University

Year dc:date.issued

2022

Author dc:creator

• Lin, Zhe

Chair dc:contributor.committeechair

• Guo, Wenxuan

Committee members dc:contributor.committeemember

• Ritchie, Glen
• Kelly, Brendan
• Song, Xiaopeng

• UAS
• Deep learning
• High-Throughput Plant Phenotyping

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/2346/90039

OAI identifier oai:identifier

oai:ttu-ir.tdl.org:2346/90039