{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/90676"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/90676","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Developing near- and mid-infrared spectroscopy analysis methods for rapid assessment of soil quality in Illinois","abstract":"Interest in use of near- (NIR) and mid-infrared (MIR) spectroscopic methods for soil quality assessment has grown rapidly. Ideally robust calibration of models can be developed to rapidly and affordably predict soil quality (SQ) indicators that are more costly to measure. Refinement of data libraries and standardization of data processing steps are needed to improve prospects for reuse of predictive models. To explore the potential for a regional library to predict SQ, this study used 468 topsoil samples collected from Illinois grain farms with loess-derived soils where, fields were managed using conventional-tillage, conservation-tillage or organic practices and so were expected to vary in SQ. Partial Least Squares Regression (PLSR) and Random Forest (RF) algorithms were used to predict SQ indicators, including soil organic carbon (SOC), total N (TN), soil C and N ratio (C: N), soil pH, particulate organic matter (POM), potentially mineralizable nitrogen (PMN), fluorescein diacetate (FDA) hydrolysis, soil nutrient abundance, and productivity proxy Normalized Difference Vegetation Index (NDVI), using the whole NIR or MIR spectra or reduced data sets comprised of spectral features associated with organic functional groups. Monte Carlo feature selection (MCFS) was used as a variable selection tool for PLSR spectra refinement. Overall, NIR models slightly outperformed MIR models and, both NIR and MIR methods were better able to predict SOC, Ca, TN, Mg, and PMN than other SQ indicators. While RF models slightly outperformed PLSR models when estimating a range of SQ indicators in NIR regions that fell within the midrange of the data set; PLSR performed better than RF for most SQ indicators using MIR spectra and had a better estimation on high or low soil property values. Neither NIR nor MIR model performance was improved when spectral ranges primarily associated with organic functional groups were used, but variable selection did significantly improve MIR model performance (p < 0.01). Unfortunately, soil region, not management, explained most differences among samples, suggesting that within IL croplands, spectral features associated with mineralogy overwhelm information about SQ obtained from this technique. Our evaluation suggests that development of robust prediction models should rely not only on careful interpretation of statistical techniques used to select peaks retained, but also careful consideration of their physical meaning.","abstract_html":"Interest in use of near- (NIR) and mid-infrared (MIR) spectroscopic methods for soil quality assessment has grown rapidly. Ideally robust calibration of models can be developed to rapidly and affordably predict soil quality (SQ) indicators that are more costly to measure. Refinement of data libraries and standardization of data processing steps are needed to improve prospects for reuse of predictive models. To explore the potential for a regional library to predict SQ, this study used 468 topsoil samples collected from Illinois grain farms with loess-derived soils where, fields were managed using conventional-tillage, conservation-tillage or organic practices and so were expected to vary in SQ. Partial Least Squares Regression (PLSR) and Random Forest (RF) algorithms were used to predict SQ indicators, including soil organic carbon (SOC), total N (TN), soil C and N ratio (C: N), soil pH, particulate organic matter (POM), potentially mineralizable nitrogen (PMN), fluorescein diacetate (FDA) hydrolysis, soil nutrient abundance, and productivity proxy Normalized Difference Vegetation Index (NDVI), using the whole NIR or MIR spectra or reduced data sets comprised of spectral features associated with organic functional groups. Monte Carlo feature selection (MCFS) was used as a variable selection tool for PLSR spectra refinement. Overall, NIR models slightly outperformed MIR models and, both NIR and MIR methods were better able to predict SOC, Ca, TN, Mg, and PMN than other SQ indicators. While RF models slightly outperformed PLSR models when estimating a range of SQ indicators in NIR regions that fell within the midrange of the data set; PLSR performed better than RF for most SQ indicators using MIR spectra and had a better estimation on high or low soil property values. Neither NIR nor MIR model performance was improved when spectral ranges primarily associated with organic functional groups were used, but variable selection did significantly improve MIR model performance (p &lt; 0.01). Unfortunately, soil region, not management, explained most differences among samples, suggesting that within IL croplands, spectral features associated with mineralogy overwhelm information about SQ obtained from this technique. Our evaluation suggests that development of robust prediction models should rely not only on careful interpretation of statistical techniques used to select peaks retained, but also careful consideration of their physical meaning.","abstract_has_math":false,"creators":["Xia, Yushu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Natural Res & Env Sciences","degree_department":null,"school":null,"contributors":["Wander, Michelle M."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2016,"date_issued":"2016-07-07T19:58:16Z","date_published":"2016-07-07T19:58:16Z","updated_at":"2026-07-22T22:26:34Z","subjects":["soil quality","near infrared","mid infrared","Partial Least Squares Regression","Random Forest","Monte Carlo feature selection"],"languages":["en"],"rights":["Copyright 2016 Yushu Xia"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/90676","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Wander, Michelle M."]},{"key":"dc:creator","label":"Author","values":["Xia, Yushu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2016-07-07T19:58:16Z","2016-04-28","2016-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Natural Res & Env Sciences"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["soil quality","near infrared","mid infrared","Partial Least Squares Regression","Random Forest","Monte Carlo feature selection"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2016 Yushu Xia"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/90676"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Interest in use of near- (NIR) and mid-infrared (MIR) spectroscopic methods for soil quality assessment has grown rapidly. Ideally robust calibration of models can be developed to rapidly and affordably predict soil quality (SQ) indicators that are more costly to measure. Refinement of data libraries and standardization of data processing steps are needed to improve prospects for reuse of predictive models. To explore the potential for a regional library to predict SQ, this study used 468 topsoil samples collected from Illinois grain farms with loess-derived soils where, fields were managed using conventional-tillage, conservation-tillage or organic practices and so were expected to vary in SQ. Partial Least Squares Regression (PLSR) and Random Forest (RF) algorithms were used to predict SQ indicators, including soil organic carbon (SOC), total N (TN), soil C and N ratio (C: N), soil pH, particulate organic matter (POM), potentially mineralizable nitrogen (PMN), fluorescein diacetate (FDA) hydrolysis, soil nutrient abundance, and productivity proxy Normalized Difference Vegetation Index (NDVI), using the whole NIR or MIR spectra or reduced data sets comprised of spectral features associated with organic functional groups. Monte Carlo feature selection (MCFS) was used as a variable selection tool for PLSR spectra refinement. Overall, NIR models slightly outperformed MIR models and, both NIR and MIR methods were better able to predict SOC, Ca, TN, Mg, and PMN than other SQ indicators. While RF models slightly outperformed PLSR models when estimating a range of SQ indicators in NIR regions that fell within the midrange of the data set; PLSR performed better than RF for most SQ indicators using MIR spectra and had a better estimation on high or low soil property values. Neither NIR nor MIR model performance was improved when spectral ranges primarily associated with organic functional groups were used, but variable selection did significantly improve MIR model performance (p < 0.01). Unfortunately, soil region, not management, explained most differences among samples, suggesting that within IL croplands, spectral features associated with mineralogy overwhelm information about SQ obtained from this technique. Our evaluation suggests that development of robust prediction models should rely not only on careful interpretation of statistical techniques used to select peaks retained, but also careful consideration of their physical meaning.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2016-07-07 without embargo terms","The student, Yushu Xia, accepted the attached license on 2016-04-27 at 10:12.","The student, Yushu Xia, submitted this Thesis for approval on 2016-04-27 at 11:12.","This Thesis was approved for publication on 2016-04-28 at 08:34.","DSpace SAF Submission Ingestion Package generated from Vireo submission #9541 on 2016-07-07 at 13:33:39","Made available in DSpace on 2016-07-07T19:58:16Z (GMT). No. of bitstreams: 2 XIA-THESIS-2016.pdf: 12845345 bytes, checksum: 2749732a5a7cc4d939ffcd497b928a51 (MD5) LICENSE.txt: 4206 bytes, checksum: 4f3a1a43686763e6ed4a16accffb49c1 (MD5) Previous issue date: 2016-04-28"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Developing near- and mid-infrared spectroscopy analysis methods for rapid assessment of soil quality in Illinois"]}]}],"canonical_facts":{"dc:contributor":["Wander, Michelle M."],"dc:creator":["Xia, Yushu"],"dc:date":["2016-07-07T19:58:16Z","2016-04-28","2016-05"],"dc:description":["Interest in use of near- (NIR) and mid-infrared (MIR) spectroscopic methods for soil quality assessment has grown rapidly. Ideally robust calibration of models can be developed to rapidly and affordably predict soil quality (SQ) indicators that are more costly to measure. Refinement of data libraries and standardization of data processing steps are needed to improve prospects for reuse of predictive models. To explore the potential for a regional library to predict SQ, this study used 468 topsoil samples collected from Illinois grain farms with loess-derived soils where, fields were managed using conventional-tillage, conservation-tillage or organic practices and so were expected to vary in SQ. Partial Least Squares Regression (PLSR) and Random Forest (RF) algorithms were used to predict SQ indicators, including soil organic carbon (SOC), total N (TN), soil C and N ratio (C: N), soil pH, particulate organic matter (POM), potentially mineralizable nitrogen (PMN), fluorescein diacetate (FDA) hydrolysis, soil nutrient abundance, and productivity proxy Normalized Difference Vegetation Index (NDVI), using the whole NIR or MIR spectra or reduced data sets comprised of spectral features associated with organic functional groups. Monte Carlo feature selection (MCFS) was used as a variable selection tool for PLSR spectra refinement. Overall, NIR models slightly outperformed MIR models and, both NIR and MIR methods were better able to predict SOC, Ca, TN, Mg, and PMN than other SQ indicators. While RF models slightly outperformed PLSR models when estimating a range of SQ indicators in NIR regions that fell within the midrange of the data set; PLSR performed better than RF for most SQ indicators using MIR spectra and had a better estimation on high or low soil property values. Neither NIR nor MIR model performance was improved when spectral ranges primarily associated with organic functional groups were used, but variable selection did significantly improve MIR model performance (p < 0.01). Unfortunately, soil region, not management, explained most differences among samples, suggesting that within IL croplands, spectral features associated with mineralogy overwhelm information about SQ obtained from this technique. Our evaluation suggests that development of robust prediction models should rely not only on careful interpretation of statistical techniques used to select peaks retained, but also careful consideration of their physical meaning.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2016-07-07 without embargo terms","The student, Yushu Xia, accepted the attached license on 2016-04-27 at 10:12.","The student, Yushu Xia, submitted this Thesis for approval on 2016-04-27 at 11:12.","This Thesis was approved for publication on 2016-04-28 at 08:34.","DSpace SAF Submission Ingestion Package generated from Vireo submission #9541 on 2016-07-07 at 13:33:39","Made available in DSpace on 2016-07-07T19:58:16Z (GMT). No. of bitstreams: 2 XIA-THESIS-2016.pdf: 12845345 bytes, checksum: 2749732a5a7cc4d939ffcd497b928a51 (MD5) LICENSE.txt: 4206 bytes, checksum: 4f3a1a43686763e6ed4a16accffb49c1 (MD5) Previous issue date: 2016-04-28"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/90676"],"dc:language":["en"],"dc:rights":["Copyright 2016 Yushu Xia"],"dc:subject":["soil quality","near infrared","mid infrared","Partial Least Squares Regression","Random Forest","Monte Carlo feature selection"],"dc:title":["Developing near- and mid-infrared spectroscopy analysis methods for rapid assessment of soil quality in Illinois"],"dc:type":["text"],"thesis:degree_discipline":["Natural Res & Env Sciences"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:26:34Z"}