Multi-task semantic segmentation of damage and materials for visual inspection of civil infrastructure

Manual visual inspection is the most common means of assessing the condition of civil infrastructure in the United States but can be exceedingly laborious, time consuming, and dangerous. Research has focused on automating parts of the inspection process using unmanned aerial vehicles for image acquisition, followed by deep learning techniques for damage identification. Existing deep learning methods and datasets for inspections have typically been developed for a single damage type. However, most guidelines for inspections require the identification of multiple damage types and describe evaluating the significance of the damage based on the associated material type. Thus, the identification of material type is important in understanding the meaning of the identified damage. Training separate networks for the tasks of material and damage identification fails to incorporate this intrinsic interdependence between them. We hypothesize that a network that incorporates such interdependence directly will have a better accuracy in material and damage identification. To this end, a deep neural network, termed the Material-and-Damage-Network (MaDnet), is proposed to simultaneously identify material type (concrete, steel, asphalt), as well as fine (cracks, exposed rebar) and coarse (spalling, corrosion) structural damage. In this approach, semantic segmentation (i.e., assignment of each pixel in the image with a material and damage label) is employed, where the interdependence between material and damage is incorporated through shared filters learned through multi-objective optimization. A new dataset with pixel-level labels identifying the material and damage type is developed and made available to the research community. Finally, the dataset is used to evaluate MaDnet and demonstrate the improvement in pixel-accuracy over employing independent networks.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Hoskere, Vedhus A

Contributors dc:contributor

• Lazebnik, Svetlana

• multi-task learning, deep learning, convolutional neural networks, damage identification, visual inspection

Statement dc:rights

• Copyright 2020 Vedhus A. Hoskere

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/108191

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/108191