Optimizing crop management with reinforcement learning, imitation learning, and crop simulations

Crop management, including nitrogen (N) fertilization and irrigation management, has a significant impact on crop yield, economic profit, and the environment. Although management guidelines exist, finding the optimal management practices is challenging given a specific planting environment and a crop. This thesis presents an intelligent crop management system that optimizes N fertilization and irrigation simultaneously via reinforcement learning (RL), imitation learning (IL), and crop simulations using the Decision Support System for Agrotechnology Transfer (DSSAT). The thesis formulates the crop management problem as an RL problem, and uses RL algorithms to train management policies that require all state variables from the simulator as observations (denoted as full observation). We then invoke IL to train management policies that only need a limited amount of state variables that can be easily obtained or measured in the real world (denoted as partial observation) by mimicking the actions of the RL-trained policies under full observation. This thesis includes three case studies in total. The first one focuses on optimizing N fertilization under full observation only. The second one optimizes N fertilization and irrigation simultaneously under full observation, and the last one considers N fertilization and irrigation under partial observation. For the case study focusing on N fertilization, experiments are conducted in simulations on the maize crop in Iowa, where farmers usually do not irrigate, and N management policies are trained with two deep RL algorithms, deep Q-network (DQN) and soft actor-critic (SAC). For the case studies on optimizing N fertilization and irrigation simultaneously, we conduct experiments in simulations on maize crop in Florida, where both irrigation and fertilization are critical for the crop growth. Deep Q-network is applied in the RL-based training for finding management policies under full observation, and the RL-trained policies are then used as the expert to train management policies under partial observation with IL. For each case study, we compare the trained policies with baseline methods, which follow a maize production guideline in the corresponding location. The trained policies under both full and partial observations achieve better outcomes than the baseline methods, resulting in a higher profit or yield with less management input or a smaller environmental impact. Moreover, the partial-observation management policies are directly deployable in the real world as they use readily available information, which paves the way for validating the performance of the framework with field tests.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Mechanical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Tao, Ran

Contributors dc:contributor

• Hovakimyan, Naira

• Reinforcement Learning
• Imitation Learning
• Intelligent Crop Management
• Sustainable Agriculture

Statement dc:rights

• Copyright 2022 Ran Tao

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/117756

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F117756