{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/158481"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/158481","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Building Strategic AI Agents for Human-centric Multi-agent Systems","abstract":"This thesis addresses the challenge of developing strategic AI agents capable of effective decision-making and communication in human-centric multi-agent systems. While significant progress has been made in AI for strategic decision-making, creating agents that can seamlessly interact with humans in multi-agentic settings remains a challenge. This research explores the limitations of current approaches, such as self-play reinforcement learning (RL) and imitation learning (IL), and proposes novel methods to overcome these constraints. Modeling human-like communication and decision making is a crucial first step toward building effective strategic agents. The initial part of the thesis addresses this through two approaches. We start by developing a regret minimization algorithm for modeling actions of strong and human-like agents called piKL, which incorporates a cost term proportional to the KL divergence between a search policy and a humanimitation learned policy. This approach improves reward while keeping behavior close to a human-imitation learned policy, producing agents that predict human actions accurately while improving performance in the benchmark game of no-press Diplomacy. Then, we develop a procedure for modeling populations of agents that communicate with humans using natural language. Our sample-efficient multitask training scheme for latent language policies (LLPs) improves the reward obtained by these policies while preserving the semantics of language in a complex real-time strategy game. Building on these foundations, the second part of the thesis focuses on building strategic agents for human-centric multi-agent domains. The research introduces the DiL-piKL planning algorithm and its extension, RL-DiL-piKL, which regularize self-play reinforcement learning and search towards a human imitation-learned policy. These algorithms enable the training of Diplodocus, an agent achieving expert human-level performance in no-press Diplomacy. A significant milestone is reached with Cicero, the first AI agent to achieve human-level performance in full-press Diplomacy, integrating a language model (LM) with planning and reinforcement learning algorithms based on piKL. The final part of the thesis revisits language generation tasks, applying piKL to model pragmatic communication and improving LM truthfulness. It presents Regularized Conventions (ReCo), a model of pragmatic language understanding that outperforms existing best response and rational speech act models across several datasets. Furthermore, a novel approach to LM decoding is introduced, casting it as a regularized imperfect-information sequential signaling game. This results in the equilibrium-ranking algorithm, which consistently improves performance over existing language model decoding procedures.","abstract_html":"This thesis addresses the challenge of developing strategic AI agents capable of effective decision-making and communication in human-centric multi-agent systems. While significant progress has been made in AI for strategic decision-making, creating agents that can seamlessly interact with humans in multi-agentic settings remains a challenge. This research explores the limitations of current approaches, such as self-play reinforcement learning (RL) and imitation learning (IL), and proposes novel methods to overcome these constraints. Modeling human-like communication and decision making is a crucial first step toward building effective strategic agents. The initial part of the thesis addresses this through two approaches. We start by developing a regret minimization algorithm for modeling actions of strong and human-like agents called piKL, which incorporates a cost term proportional to the KL divergence between a search policy and a humanimitation learned policy. This approach improves reward while keeping behavior close to a human-imitation learned policy, producing agents that predict human actions accurately while improving performance in the benchmark game of no-press Diplomacy. Then, we develop a procedure for modeling populations of agents that communicate with humans using natural language. Our sample-efficient multitask training scheme for latent language policies (LLPs) improves the reward obtained by these policies while preserving the semantics of language in a complex real-time strategy game. Building on these foundations, the second part of the thesis focuses on building strategic agents for human-centric multi-agent domains. The research introduces the DiL-piKL planning algorithm and its extension, RL-DiL-piKL, which regularize self-play reinforcement learning and search towards a human imitation-learned policy. These algorithms enable the training of Diplodocus, an agent achieving expert human-level performance in no-press Diplomacy. A significant milestone is reached with Cicero, the first AI agent to achieve human-level performance in full-press Diplomacy, integrating a language model (LM) with planning and reinforcement learning algorithms based on piKL. The final part of the thesis revisits language generation tasks, applying piKL to model pragmatic communication and improving LM truthfulness. It presents Regularized Conventions (ReCo), a model of pragmatic language understanding that outperforms existing best response and rational speech act models across several datasets. Furthermore, a novel approach to LM decoding is introduced, casting it as a regularized imperfect-information sequential signaling game. This results in the equilibrium-ranking algorithm, which consistently improves performance over existing language model decoding procedures.","abstract_has_math":false,"creators":["Jacob, Athul Paul"],"institution":"Massachusetts Institute of Technology","degree_name":"Doctoral","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Andreas, Jacob"],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-09","date_published":"2024-09","updated_at":"2026-07-22T22:22:08Z","subjects":[],"languages":[],"rights":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"],"rights_urls":["https://rightsstatements.org/page/InC-EDU/1.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/158481","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Andreas, Jacob"]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"]},{"key":"dc:creator","label":"Author","values":["Jacob, Athul Paul"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-03-12T16:54:53Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-03-12T16:54:53Z"]},{"key":"dc:date.issued","label":"Date","values":["2024-09"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctoral","Doctor of Philosophy"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://rightsstatements.org/page/InC-EDU/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/158481"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This thesis addresses the challenge of developing strategic AI agents capable of effective decision-making and communication in human-centric multi-agent systems. While significant progress has been made in AI for strategic decision-making, creating agents that can seamlessly interact with humans in multi-agentic settings remains a challenge. This research explores the limitations of current approaches, such as self-play reinforcement learning (RL) and imitation learning (IL), and proposes novel methods to overcome these constraints. Modeling human-like communication and decision making is a crucial first step toward building effective strategic agents. The initial part of the thesis addresses this through two approaches. We start by developing a regret minimization algorithm for modeling actions of strong and human-like agents called piKL, which incorporates a cost term proportional to the KL divergence between a search policy and a humanimitation learned policy. This approach improves reward while keeping behavior close to a human-imitation learned policy, producing agents that predict human actions accurately while improving performance in the benchmark game of no-press Diplomacy. Then, we develop a procedure for modeling populations of agents that communicate with humans using natural language. Our sample-efficient multitask training scheme for latent language policies (LLPs) improves the reward obtained by these policies while preserving the semantics of language in a complex real-time strategy game. Building on these foundations, the second part of the thesis focuses on building strategic agents for human-centric multi-agent domains. The research introduces the DiL-piKL planning algorithm and its extension, RL-DiL-piKL, which regularize self-play reinforcement learning and search towards a human imitation-learned policy. These algorithms enable the training of Diplodocus, an agent achieving expert human-level performance in no-press Diplomacy. A significant milestone is reached with Cicero, the first AI agent to achieve human-level performance in full-press Diplomacy, integrating a language model (LM) with planning and reinforcement learning algorithms based on piKL. The final part of the thesis revisits language generation tasks, applying piKL to model pragmatic communication and improving LM truthfulness. It presents Regularized Conventions (ReCo), a model of pragmatic language understanding that outperforms existing best response and rational speech act models across several datasets. Furthermore, a novel approach to LM decoding is introduced, casting it as a regularized imperfect-information sequential signaling game. This results in the equilibrium-ranking algorithm, which consistently improves performance over existing language model decoding procedures."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:title","label":"Title","values":["Building Strategic AI Agents for Human-centric Multi-agent Systems"]}]}],"canonical_facts":{"dc:contributor.advisor":["Andreas, Jacob"],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"],"dc:creator":["Jacob, Athul Paul"],"dc:date.accessioned":["2025-03-12T16:54:53Z"],"dc:date.available":["2025-03-12T16:54:53Z"],"dc:date.issued":["2024-09"],"dc:description.abstract":["This thesis addresses the challenge of developing strategic AI agents capable of effective decision-making and communication in human-centric multi-agent systems. While significant progress has been made in AI for strategic decision-making, creating agents that can seamlessly interact with humans in multi-agentic settings remains a challenge. This research explores the limitations of current approaches, such as self-play reinforcement learning (RL) and imitation learning (IL), and proposes novel methods to overcome these constraints. Modeling human-like communication and decision making is a crucial first step toward building effective strategic agents. The initial part of the thesis addresses this through two approaches. We start by developing a regret minimization algorithm for modeling actions of strong and human-like agents called piKL, which incorporates a cost term proportional to the KL divergence between a search policy and a humanimitation learned policy. This approach improves reward while keeping behavior close to a human-imitation learned policy, producing agents that predict human actions accurately while improving performance in the benchmark game of no-press Diplomacy. Then, we develop a procedure for modeling populations of agents that communicate with humans using natural language. Our sample-efficient multitask training scheme for latent language policies (LLPs) improves the reward obtained by these policies while preserving the semantics of language in a complex real-time strategy game. Building on these foundations, the second part of the thesis focuses on building strategic agents for human-centric multi-agent domains. The research introduces the DiL-piKL planning algorithm and its extension, RL-DiL-piKL, which regularize self-play reinforcement learning and search towards a human imitation-learned policy. These algorithms enable the training of Diplodocus, an agent achieving expert human-level performance in no-press Diplomacy. A significant milestone is reached with Cicero, the first AI agent to achieve human-level performance in full-press Diplomacy, integrating a language model (LM) with planning and reinforcement learning algorithms based on piKL. The final part of the thesis revisits language generation tasks, applying piKL to model pragmatic communication and improving LM truthfulness. It presents Regularized Conventions (ReCo), a model of pragmatic language understanding that outperforms existing best response and rational speech act models across several datasets. Furthermore, a novel approach to LM decoding is introduced, casting it as a regularized imperfect-information sequential signaling game. This results in the equilibrium-ranking algorithm, which consistently improves performance over existing language model decoding procedures."],"dc:description.degree":["Ph.D."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/158481"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"],"dc:rights.uri":["https://rightsstatements.org/page/InC-EDU/1.0/"],"dc:title":["Building Strategic AI Agents for Human-centric Multi-agent Systems"],"dc:type":["Thesis"],"thesis:degree_name":["Doctoral","Doctor of Philosophy"]},"updated_at":"2026-07-22T22:22:08Z"}