An integrated performance model learning and planning approach for optimal infrastructure facility maintenance under partial observability

Infrastructure inspection and maintenance decision making is a stochastic and partially observable problem. This thesis presents a learning and decision-making approach for developing optimal joint inspection and maintenance policies for civil infrastructure facilities under performance model uncertainty and partially observable condition state. Joint inspection and maintenance decision making refers to the cost-effective policies that capture the interactions between the inspections, and the maintenance and rehabilitation decisions. Performance model uncertainty is the uncertainty about the actual facility deterioration model. Partially observation condition states imply that the true or latent condition of the facility is not captured by the measurement technology used for inspecting the facility due to measurement errors. The proposed approach models single facility inspection and maintenance decision-making problem as a partially observable Markov decision process. The data collected during the agency-facility interaction is used to learn the maximum likelihood estimate of performance model using the Baum-Welch algorithm. Both offline and online versions of the learning algorithm are presented. The probing-optimizing dichotomy, also known as exploration-exploitation dilemma, in choosing between the best strategy based on the past knowledge of deterioration and the strategy that provides information leading to better performance model is also illustrated. Once a set of performance models is learned, each of the models is quantitatively compared with the corresponding true model using an information theoretic measure known as asymmetric Kullback-Leibler divergence to determine its goodness. It is also shown that given a set of new observation sequences gathered after the model learning, each of the sequences or the entire set of sequences can be assigned the model that gives the least log-likelihood on that sequence or the set. This can help in classifying the facility deterioration into different deterioration modes like slow, medium, or fast. Finally, an exact algorithm for arriving at optimal inspection and maintenance decision-making policies is described. It is argued that the proposed approach scores over other recently developed adaptive control approaches in that it directly learns the best model from historical data instead of maintaining and updating beliefs over a set of pre-selected models. Furthermore, since the proposed learning approach is model-based, it makes more efficient use of costly data as compared to the model-less learning approaches. Keywords: Infrastructure Inspection and Maintenance, Baum-Welch Algorithm, Kullback-Liebler Divergence, Sequence Classification, Partially Observable Markov Decision Process, Reinforcement Learning.

Author dc:creator

• Amin, Saurabh

Contributors dc:contributor

• Zhang, Zhanmin

• Sequence classification
• Reinforcement learning
• Infrastructure inspection and maintenance

Statement dc:rights

• Open
• Copyright © is held by the author. Presentation of this material on the Libraries' web site by University Libraries, The University of Texas at Austin was made possible under a limited license grant from the author who has retained all copyrights in the works.

Language dc:language

eng

Identifier

http://dx.doi.org/10.26153/tsw/1553

OAI identifier oai:identifier

oai:tdl-ir.tdl.org:2152/74433