Interpretable Knowledge Transfer in Communicative Neural-based Swarm-Guidance Agents

Transparency ensures that the decision-making logic used by autonomous agents is available in a form comprehensible to human teammates. It enhances a system's performance, reliability and trustworthiness. Transparency in human-autonomy teaming is a key success factor in the communication between humans and autonomous agents. This thesis focuses on the problem of knowledge transfer between two neural network (NN) agents in a swarm-guidance task in the presence of a human observer. A novel three-module knowledge transfer framework is proposed to interpret non-symbolic knowledge of communicative autonomous agents into a transparent human-friendly form. The knowledge interpretation module transforms the NN into a rule-based knowledge representation. The relevant knowledge is then chosen by a knowledge selection module to transfer it to the other agent. Finally, the knowledge fusion module combines the newly incoming knowledge with the receiver agent's existing knowledge. Two algorithms are introduced to transform NNs into a rule-based knowledge representation. The first algorithm, the Exact-Convertible Decision Tree (EC-DT), rewrites the relationships between nodes and weights of the NN into a multivariate decision tree. The second algorithm, the Extended C-Net, leverages the training data to learn the association between the NN's nodes at the final hidden layer and the outputs and then uses recursive back-projections to derive the rules regulating input-output relationships. Performance is assessed using three measures: fidelity, compactness and transparency. EC-DT has higher fidelity, while Extended C-Net produces more compact rule sets. The fusion module then evaluates the effectiveness of transmitted rule-based knowledge in new environments. The rule-based representation is projected back into forms that can be integrated with the NN representation at the receiver agent's end. A retraining strategy, Priority on Weak State Areas (PoWSA), is introduced to help speed up the learning process in novel scenarios. Analyses of the proposed methodology in a swarm-guidance problem show higher training stability and chances of success in return for a slight increase in computational costs. The framework provides a more transparent knowledge representation that could be visualised to complement the verbal rule-based representation.

Grantor dc:publisher

UNSW, Sydney

Year dc:date

2022

Author dc:creator

• Nguyen, Tung ; https://orcid.org/0000-0001-8437-1113

• Interpretability
• Transparency
• Explainable Artificial Intelligence
• Knowledge Representation
• Knowledge Transfer
• Artificial Neural Network
• Multivariate Decision Tree
• Multi-Agent Systems
• Reinforcement Learning
• Swarm Guidance
• Shepherding
• anzsrc-for: 460202 Autonomous agents and multiagent systems
• anzsrc-for: 460205 Intelligent robotics
• anzsrc-for: 460206 Knowledge representation and reasoning
• anzsrc-for: 460805 Fairness, accountability, transparency, trust and ethics of computer systems
• anzsrc-for: 461104 Neural networks

Statement dc:rights

• open access
• CC BY 4.0
• free_to_read

Licence

• https://purl.org/coar/access_right/c_abf2
• https://creativecommons.org/licenses/by/4.0/

Language dc:language

en

Identifier

https://doi.org/10.26190/unsworks/24375

OAI identifier oai:identifier

oai:unsworks.library.unsw.edu.au:1959.4/100668