{"id":{"repo_id":"unr","oai_identifier":"oai:scholarwolf.unr.edu:11714/10949"},"canonical_url":"https://search.dev.ndltd.org/etd/unr/oai:scholarwolf.unr.edu:11714/10949","repository":{"repo_id":"unr","name":"University of Nevada - Reno","base_url":"https://scholarwolf.unr.edu/server/oai/request"},"display":{"title":"Attention-Enabled Reinforcement Learning for Control of Scalable Multi-Agent Systems","abstract":"Multi-agent reinforcement learning has been the subject of considerable interest and effort for its potential as a means of specifying behavior policies for multi-agent systems. Specifically, on-policy algorithms based on gradient estimation have achieved state-of-the-art performance on end-to-end control problems once thought beyond the scope of machine learning methods. In seeking to apply the benefits of MARL to practical control of physical autonomous systems, we must begin to account for three factors: (1) the presence of other autonomous elements in the environment configuration space, which may or may not be amenable to coordination; (2) non-idealities in sensing the configuration of the environment (e.g. locality and limited observability); and (3) variability in the number of sensed dynamical elements. The attention head, a relational ML structure originally designed for extraction of abstract natural language features, is structurally well suited to addressing these challenges. This work presents a systematic argument and framework for the use of attention as an input layer to enable learning of neural policy models in changing multi-agent environments which are not well-suited to other representations. In benchmark physical simulations, it is shown that such models achieve competitive performance on cooperative and mixed cooperative/competitive MAS control tasks as the agent cohort is arbitrarily changed. Prospective advantages of attention-based architectures for physical autonomous systems in select applications are discussed, as well as drawbacks associated with explainability and potential for emergent behavior.","abstract_html":"Multi-agent reinforcement learning has been the subject of considerable interest and effort for its potential as a means of specifying behavior policies for multi-agent systems. Specifically, on-policy algorithms based on gradient estimation have achieved state-of-the-art performance on end-to-end control problems once thought beyond the scope of machine learning methods. In seeking to apply the benefits of MARL to practical control of physical autonomous systems, we must begin to account for three factors: (1) the presence of other autonomous elements in the environment configuration space, which may or may not be amenable to coordination; (2) non-idealities in sensing the configuration of the environment (e.g. locality and limited observability); and (3) variability in the number of sensed dynamical elements. The attention head, a relational ML structure originally designed for extraction of abstract natural language features, is structurally well suited to addressing these challenges. This work presents a systematic argument and framework for the use of attention as an input layer to enable learning of neural policy models in changing multi-agent environments which are not well-suited to other representations. In benchmark physical simulations, it is shown that such models achieve competitive performance on cooperative and mixed cooperative/competitive MAS control tasks as the agent cohort is arbitrarily changed. Prospective advantages of attention-based architectures for physical autonomous systems in select applications are discussed, as well as drawbacks associated with explainability and potential for emergent behavior.","abstract_has_math":false,"creators":["Dailey, Joseph A."],"institution":"University of Nevada, Reno","degree_name":"Master of Science","degree_level":"Master's Degree","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Xu, Hao"],"committee_chairs":[],"committee_members":["Fadali, M. Sami","La, Hung"],"year":2024,"date_issued":"2024","date_published":"2024","updated_at":"2026-07-27T21:45:34Z","subjects":["Attention","Autonomous Systems","Intelligent Control","Multi-agent Systems","Reinforcement Learning"],"languages":[],"rights":["Creative Commons Attribution-ShareAlike 4.0 International"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://scholarwolf.unr.edu/handle/11714/10949","outbound_label":"Repository record","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Xu, Hao"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Fadali, M. Sami","La, Hung"]},{"key":"dc:creator","label":"Author","values":["Dailey, Joseph A."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-08-22T17:09:42Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-08-22T17:09:42Z"]},{"key":"dc:date.issued","label":"Date","values":["2024"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Master's Degree"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Nevada, Reno"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Attention","Autonomous Systems","Intelligent Control","Multi-agent Systems","Reinforcement Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["Creative Commons Attribution-ShareAlike 4.0 International"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://scholarwolf.unr.edu/handle/11714/10949"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Multi-agent reinforcement learning has been the subject of considerable interest and effort for its potential as a means of specifying behavior policies for multi-agent systems. Specifically, on-policy algorithms based on gradient estimation have achieved state-of-the-art performance on end-to-end control problems once thought beyond the scope of machine learning methods. In seeking to apply the benefits of MARL to practical control of physical autonomous systems, we must begin to account for three factors: (1) the presence of other autonomous elements in the environment configuration space, which may or may not be amenable to coordination; (2) non-idealities in sensing the configuration of the environment (e.g. locality and limited observability); and (3) variability in the number of sensed dynamical elements. The attention head, a relational ML structure originally designed for extraction of abstract natural language features, is structurally well suited to addressing these challenges. This work presents a systematic argument and framework for the use of attention as an input layer to enable learning of neural policy models in changing multi-agent environments which are not well-suited to other representations. In benchmark physical simulations, it is shown that such models achieve competitive performance on cooperative and mixed cooperative/competitive MAS control tasks as the agent cohort is arbitrarily changed. Prospective advantages of attention-based architectures for physical autonomous systems in select applications are discussed, as well as drawbacks associated with explainability and potential for emergent behavior."]},{"key":"dc:format","label":"Dc Format","values":["PDF"]},{"key":"dc:title","label":"Title","values":["Attention-Enabled Reinforcement Learning for Control of Scalable Multi-Agent Systems"]}]}],"canonical_facts":{"dc:contributor.advisor":["Xu, Hao"],"dc:contributor.committeemember":["Fadali, M. Sami","La, Hung"],"dc:creator":["Dailey, Joseph A."],"dc:date.accessioned":["2024-08-22T17:09:42Z"],"dc:date.available":["2024-08-22T17:09:42Z"],"dc:date.issued":["2024"],"dc:description.abstract":["Multi-agent reinforcement learning has been the subject of considerable interest and effort for its potential as a means of specifying behavior policies for multi-agent systems. Specifically, on-policy algorithms based on gradient estimation have achieved state-of-the-art performance on end-to-end control problems once thought beyond the scope of machine learning methods. In seeking to apply the benefits of MARL to practical control of physical autonomous systems, we must begin to account for three factors: (1) the presence of other autonomous elements in the environment configuration space, which may or may not be amenable to coordination; (2) non-idealities in sensing the configuration of the environment (e.g. locality and limited observability); and (3) variability in the number of sensed dynamical elements. The attention head, a relational ML structure originally designed for extraction of abstract natural language features, is structurally well suited to addressing these challenges. This work presents a systematic argument and framework for the use of attention as an input layer to enable learning of neural policy models in changing multi-agent environments which are not well-suited to other representations. In benchmark physical simulations, it is shown that such models achieve competitive performance on cooperative and mixed cooperative/competitive MAS control tasks as the agent cohort is arbitrarily changed. Prospective advantages of attention-based architectures for physical autonomous systems in select applications are discussed, as well as drawbacks associated with explainability and potential for emergent behavior."],"dc:format":["PDF"],"dc:identifier.uri":["https://scholarwolf.unr.edu/handle/11714/10949"],"dc:rights":["Creative Commons Attribution-ShareAlike 4.0 International"],"dc:subject":["Attention","Autonomous Systems","Intelligent Control","Multi-agent Systems","Reinforcement Learning"],"dc:title":["Attention-Enabled Reinforcement Learning for Control of Scalable Multi-Agent Systems"],"dc:type":["Thesis"],"thesis:degree_level":["Master's Degree"],"thesis:degree_name":["Master of Science"],"thesis:institution_name":["University of Nevada, Reno"]},"updated_at":"2026-07-27T21:45:34Z"}