{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/158964"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/158964","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"From Data, to Models, and Back: Making Machine Learning Predictably Reliable","abstract":"Machine learning systems exhibit impressive performance, but we currently lack scalable ways to anticipate their successes, failure modes, and biases. This position limits our ability to deploy these systems in the appropriate contexts, and to build systems which we can confidently deploy in high-risk settings. Motivated by this state of affairs, this thesis aims to develop design principles for predictably reliable machine learning. Our ultimate goal is to enable developers to know when their models will work, anticipate when they will fail, and understand “why” in both cases. In pursuit of this goal, this thesis combines large-scale experiments with theoretical analysis to form a precise understanding of the ML “pipeline,” from training data (and the way we collect it), to learning algorithms, to deployment. Fully realized, such an understanding would allow us to build ML systems the same way we build buildings or airplanes—safely, scalably, and with a robust grasp of the underlying principles. In this thesis, we focus on four design choices within this pipeline: model deployment (Part I), dataset creation (Part II), data collection (Part III), and algorithm selection (Part IV). For each of these design choices, we use targeted experiments to uncover the corresponding principles that actually underlie the behavior of ML systems. We distill these principles into concise conceptual models which allow us to both reason about existing systems and design improved ones. Along the way, we will revisit, challenge, and refine various aspects of conventional wisdom surrounding ML model development.","abstract_html":"Machine learning systems exhibit impressive performance, but we currently lack scalable ways to anticipate their successes, failure modes, and biases. This position limits our ability to deploy these systems in the appropriate contexts, and to build systems which we can confidently deploy in high-risk settings. Motivated by this state of affairs, this thesis aims to develop design principles for predictably reliable machine learning. Our ultimate goal is to enable developers to know when their models will work, anticipate when they will fail, and understand “why” in both cases. In pursuit of this goal, this thesis combines large-scale experiments with theoretical analysis to form a precise understanding of the ML “pipeline,” from training data (and the way we collect it), to learning algorithms, to deployment. Fully realized, such an understanding would allow us to build ML systems the same way we build buildings or airplanes—safely, scalably, and with a robust grasp of the underlying principles. In this thesis, we focus on four design choices within this pipeline: model deployment (Part I), dataset creation (Part II), data collection (Part III), and algorithm selection (Part IV). For each of these design choices, we use targeted experiments to uncover the corresponding principles that actually underlie the behavior of ML systems. We distill these principles into concise conceptual models which allow us to both reason about existing systems and design improved ones. Along the way, we will revisit, challenge, and refine various aspects of conventional wisdom surrounding ML model development.","abstract_has_math":false,"creators":["Ilyas, Andrew"],"institution":"Massachusetts Institute of Technology","degree_name":"Doctoral","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Daskalakis, Constantinos","Mądry, Aleksander"],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-02","date_published":"2025-02","updated_at":"2026-07-22T22:21:20Z","subjects":[],"languages":[],"rights":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"],"rights_urls":["https://rightsstatements.org/page/InC-EDU/1.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/158964","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Daskalakis, Constantinos","Mądry, Aleksander"]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"]},{"key":"dc:creator","label":"Author","values":["Ilyas, Andrew"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-03-27T17:00:55Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-03-27T17:00:55Z"]},{"key":"dc:date.issued","label":"Date","values":["2025-02"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctoral","Doctor of Philosophy"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://rightsstatements.org/page/InC-EDU/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/158964"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Machine learning systems exhibit impressive performance, but we currently lack scalable ways to anticipate their successes, failure modes, and biases. This position limits our ability to deploy these systems in the appropriate contexts, and to build systems which we can confidently deploy in high-risk settings. Motivated by this state of affairs, this thesis aims to develop design principles for predictably reliable machine learning. Our ultimate goal is to enable developers to know when their models will work, anticipate when they will fail, and understand “why” in both cases. In pursuit of this goal, this thesis combines large-scale experiments with theoretical analysis to form a precise understanding of the ML “pipeline,” from training data (and the way we collect it), to learning algorithms, to deployment. Fully realized, such an understanding would allow us to build ML systems the same way we build buildings or airplanes—safely, scalably, and with a robust grasp of the underlying principles. In this thesis, we focus on four design choices within this pipeline: model deployment (Part I), dataset creation (Part II), data collection (Part III), and algorithm selection (Part IV). For each of these design choices, we use targeted experiments to uncover the corresponding principles that actually underlie the behavior of ML systems. We distill these principles into concise conceptual models which allow us to both reason about existing systems and design improved ones. Along the way, we will revisit, challenge, and refine various aspects of conventional wisdom surrounding ML model development."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:title","label":"Title","values":["From Data, to Models, and Back: Making Machine Learning Predictably Reliable"]}]}],"canonical_facts":{"dc:contributor.advisor":["Daskalakis, Constantinos","Mądry, Aleksander"],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"],"dc:creator":["Ilyas, Andrew"],"dc:date.accessioned":["2025-03-27T17:00:55Z"],"dc:date.available":["2025-03-27T17:00:55Z"],"dc:date.issued":["2025-02"],"dc:description.abstract":["Machine learning systems exhibit impressive performance, but we currently lack scalable ways to anticipate their successes, failure modes, and biases. This position limits our ability to deploy these systems in the appropriate contexts, and to build systems which we can confidently deploy in high-risk settings. Motivated by this state of affairs, this thesis aims to develop design principles for predictably reliable machine learning. Our ultimate goal is to enable developers to know when their models will work, anticipate when they will fail, and understand “why” in both cases. In pursuit of this goal, this thesis combines large-scale experiments with theoretical analysis to form a precise understanding of the ML “pipeline,” from training data (and the way we collect it), to learning algorithms, to deployment. Fully realized, such an understanding would allow us to build ML systems the same way we build buildings or airplanes—safely, scalably, and with a robust grasp of the underlying principles. In this thesis, we focus on four design choices within this pipeline: model deployment (Part I), dataset creation (Part II), data collection (Part III), and algorithm selection (Part IV). For each of these design choices, we use targeted experiments to uncover the corresponding principles that actually underlie the behavior of ML systems. We distill these principles into concise conceptual models which allow us to both reason about existing systems and design improved ones. Along the way, we will revisit, challenge, and refine various aspects of conventional wisdom surrounding ML model development."],"dc:description.degree":["Ph.D."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/158964"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["In Copyright - Educational Use Permitted","Copyright retained by author(s)"],"dc:rights.uri":["https://rightsstatements.org/page/InC-EDU/1.0/"],"dc:title":["From Data, to Models, and Back: Making Machine Learning Predictably Reliable"],"dc:type":["Thesis"],"thesis:degree_name":["Doctoral","Doctor of Philosophy"]},"updated_at":"2026-07-22T22:21:20Z"}