Criteria for evaluating model deficiency for groundwater models and the effects of eliminating deficient models on multi-model analysis using AICC, KIC, AIC, and BIC
type: CreativeWork
description: Multi-model analysis (MMA) considers multiple model interpretations of a system. MMA provides a more realistic assessment of uncertainty associated with model predictions because both uncertainty of individual models and uncertainty associated with different model structures are considered. Models are evaluated for the strength of evidence that they represent an unknown system using different Information Criteria (IC) equations. IC equations are designed to assess the likelihood that a model in a set of models represents the true but unknown system. IC equations do not include a component which identifies a deficient model. Therefore, inclusion of deficient models in the set of models leads to poor model-averaged results. Evaluation of models to assess whether available observation data sufficiently support the model structure is an important step in MMA. Measures for evaluating models include: 1) failure to reach proper convergence during non-linear regression; 2) unreasonable parameter estimates; 3) unreasonable confidence intervals on parameters or a coefficient of variation greater than ten for one or more parameters; 4) high correlations between parameters; 5) determinant of the correlation matrix less than 1x10-12; 6) condition number of the Jacobian matrix greater than 2000; and 7) unreasonable confidence intervals on predictions. Experiments presented herein are designed to evaluate how components of AIC, AICc, BIC, and KIC rank models and assign model probabilities, and to demonstrate how removing deficient models improves MMA results. Synthetic models are used to represent true but unknown systems in contrast to experimental models that are created to simulate a simplified version of the unknown system based on observation data taken from the synthetic models. AIC, AICc, BIC, and KIC generally assign high probability to deficient models. AICc generally assigns high probability to deficient models if 1) there are many observation data or 2) there are few ob…
abstract: Multi-model analysis (MMA) considers multiple model interpretations of a system. MMA provides a more realistic assessment of uncertainty associated with model predictions because both uncertainty of individual models and uncertainty associated with different model structures are considered. Models are evaluated for the strength of evidence that they represent an unknown system using different Information Criteria (IC) equations. IC equations are designed to assess the likelihood that a model in a set of models represents the true but unknown system. IC equations do not include a component which identifies a deficient model. Therefore, inclusion of deficient models in the set of models leads to poor model-averaged results. Evaluation of models to assess whether available observation data sufficiently support the model structure is an important step in MMA. Measures for evaluating models include: 1) failure to reach proper convergence during non-linear regression; 2) unreasonable parameter estimates; 3) unreasonable confidence intervals on parameters or a coefficient of variation greater than ten for one or more parameters; 4) high correlations between parameters; 5) determinant of the correlation matrix less than 1x10-12; 6) condition number of the Jacobian matrix greater than 2000; and 7) unreasonable confidence intervals on predictions. Experiments presented herein are designed to evaluate how components of AIC, AICc, BIC, and KIC rank models and assign model probabilities, and to demonstrate how removing deficient models improves MMA results. Synthetic models are used to represent true but unknown systems in contrast to experimental models that are created to simulate a simplified version of the unknown system based on observation data taken from the synthetic models. AIC, AICc, BIC, and KIC generally assign high probability to deficient models. AICc generally assigns high probability to deficient models if 1) there are many observation data or 2) there are few ob…
author: Schenk, Judith A.
language: eng
language: English
keywords: KIC, BIC, information theory, modeling, AICc, groundwater, multi-model analysis, model deficiency, AIC
identifier: T 8256
date published: 2017
date modified: 2026-07-24T01:44:05Z
provider: Colorado School of Mines
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: Colorado School of Mines. Arthur Lakes Library

PropertyValue
property ID: Handle
value: https://hdl.handle.net/11124/170981