{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/114034"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/114034","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Parameterized sequential decision making problems","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-12-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-12-01","abstract_has_math":false,"creators":["Srivastava, Amber"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Mechanical Engineering","degree_department":null,"school":null,"contributors":["Salapaka, Srinivasa","Ferriera, Placid","West, Matthew","Milenkovic, Olgica","Srikant, Rayadurgam"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-04-29T21:58:12Z","date_published":"2022-04-29T21:58:12Z","updated_at":"2026-07-22T22:24:54Z","subjects":["Sequential Decisions","Markov Decision Processes","Reinforcement Learning","Maximum Entropy Principle","Clustering","Markov chain"],"languages":["en","eng"],"rights":["Copyright 2021 Amber Srivastava"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/114034","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Salapaka, Srinivasa","Ferriera, Placid","West, Matthew","Milenkovic, Olgica","Srikant, Rayadurgam"]},{"key":"dc:creator","label":"Author","values":["Srivastava, Amber"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-04-29T21:58:12Z","2024-04-29T21:58:46Z","2021-12","2021-07-29"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Mechanical Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Sequential Decisions","Markov Decision Processes","Reinforcement Learning","Maximum Entropy Principle","Clustering","Markov chain"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2021 Amber Srivastava"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/114034"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-12-01","The student, Amber Srivastava, accepted the attached license on 2021-07-27 at 12:27.","The student, Amber Srivastava, submitted this Dissertation for approval on 2021-07-27 at 12:35.","This Dissertation was approved for publication on 2021-07-29 at 11:37.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17020 on 2022-04-29 at 16:08:45","Made available in DSpace on 2022-04-29T21:58:12Z (GMT). No. of bitstreams: 2 SRIVASTAVA-DISSERTATION-2021.pdf: 7899287 bytes, checksum: 83d70f27c984fadf34024092b2bb5925 (MD5) LICENSE.txt: 4213 bytes, checksum: b6d1006ccf9d059577a8857060760380 (MD5) Previous issue date: 2021-07-29","Embargo set by: Seth Robbins for item 123399 Lift date: 2024-04-29T21:58:46Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited","This thesis addresses a class of optimization problems that deals with the two-fold objective of making sequential decisions, and simultaneously determining the unknown problem parameters such that the associated cost function gets minimized. We refer to these problems as parameterized Sequential Decision Making (para-SDM) problems. The application areas are plenty; for instance, network design problems, job-shop scheduling, industrial process optimization, last mile delivery, vehicle routing, sensor networks, clustering and classification. In this work, we develop a combinatorial optimization viewpoint for these problems - where the viewpoint is facilitated by the combinatorially large number of possible sequences of decisions - and use Maximum Entropy Principle (MEP) based framework to address them. The optimization problems considered in this thesis have been shown to be NP-hard, accompanied by a non-convex cost function whose surface that is riddled by multiple poor local minima. The combinatorially large number of possible sequences of decisions on top of the above mentioned challenges render para-SDM as a difficult class of optimization problems. Our proposed MEP-based framework is designed to overcome the aforesaid challenges in para-SDM. For instance, we employ annealing from a suitable convex function to the non-convex cost function to avoid getting stuck in a poor local minima. Additionally, we utilize the problem structures (such as the law of optimality of the paths) to represent the combinatorial number of possibilites using much smaller decision variable space. The proposed framework is flexible to incorporating application-specific capacity, inclusion-exclusion, and dynamic constraints. Our framework also extends to the class of problems where the information about the underlying model is lacking, and we develop suitable stochastic iterative updates that interacts with the underlying system to simultaneously learn the sequences and the parameter values. A peculiar characteristic of the annealing process in our MEP-based frameworks is the phase transition phenomenon. In particular, these are the specific instances in the annealing procedure at which the solution undergoes significant changes. We demonstrate the utility of these phase transitions in determining certain design hyperparamters in para-SDMs, and in general, in combinatorial optimization problems; for instance, estimating the true number of clusters in a data set, or determining the appropriate choice of the sparsity level in sparse linear regression problems."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Parameterized sequential decision making problems"]}]}],"canonical_facts":{"dc:contributor":["Salapaka, Srinivasa","Ferriera, Placid","West, Matthew","Milenkovic, Olgica","Srikant, Rayadurgam"],"dc:creator":["Srivastava, Amber"],"dc:date":["2022-04-29T21:58:12Z","2024-04-29T21:58:46Z","2021-12","2021-07-29"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-12-01","The student, Amber Srivastava, accepted the attached license on 2021-07-27 at 12:27.","The student, Amber Srivastava, submitted this Dissertation for approval on 2021-07-27 at 12:35.","This Dissertation was approved for publication on 2021-07-29 at 11:37.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17020 on 2022-04-29 at 16:08:45","Made available in DSpace on 2022-04-29T21:58:12Z (GMT). No. of bitstreams: 2 SRIVASTAVA-DISSERTATION-2021.pdf: 7899287 bytes, checksum: 83d70f27c984fadf34024092b2bb5925 (MD5) LICENSE.txt: 4213 bytes, checksum: b6d1006ccf9d059577a8857060760380 (MD5) Previous issue date: 2021-07-29","Embargo set by: Seth Robbins for item 123399 Lift date: 2024-04-29T21:58:46Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited","This thesis addresses a class of optimization problems that deals with the two-fold objective of making sequential decisions, and simultaneously determining the unknown problem parameters such that the associated cost function gets minimized. We refer to these problems as parameterized Sequential Decision Making (para-SDM) problems. The application areas are plenty; for instance, network design problems, job-shop scheduling, industrial process optimization, last mile delivery, vehicle routing, sensor networks, clustering and classification. In this work, we develop a combinatorial optimization viewpoint for these problems - where the viewpoint is facilitated by the combinatorially large number of possible sequences of decisions - and use Maximum Entropy Principle (MEP) based framework to address them. The optimization problems considered in this thesis have been shown to be NP-hard, accompanied by a non-convex cost function whose surface that is riddled by multiple poor local minima. The combinatorially large number of possible sequences of decisions on top of the above mentioned challenges render para-SDM as a difficult class of optimization problems. Our proposed MEP-based framework is designed to overcome the aforesaid challenges in para-SDM. For instance, we employ annealing from a suitable convex function to the non-convex cost function to avoid getting stuck in a poor local minima. Additionally, we utilize the problem structures (such as the law of optimality of the paths) to represent the combinatorial number of possibilites using much smaller decision variable space. The proposed framework is flexible to incorporating application-specific capacity, inclusion-exclusion, and dynamic constraints. Our framework also extends to the class of problems where the information about the underlying model is lacking, and we develop suitable stochastic iterative updates that interacts with the underlying system to simultaneously learn the sequences and the parameter values. A peculiar characteristic of the annealing process in our MEP-based frameworks is the phase transition phenomenon. In particular, these are the specific instances in the annealing procedure at which the solution undergoes significant changes. We demonstrate the utility of these phase transitions in determining certain design hyperparamters in para-SDMs, and in general, in combinatorial optimization problems; for instance, estimating the true number of clusters in a data set, or determining the appropriate choice of the sparsity level in sparse linear regression problems."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/114034"],"dc:language":["en","eng"],"dc:rights":["Copyright 2021 Amber Srivastava"],"dc:subject":["Sequential Decisions","Markov Decision Processes","Reinforcement Learning","Maximum Entropy Principle","Clustering","Markov chain"],"dc:title":["Parameterized sequential decision making problems"],"dc:type":["text"],"thesis:degree_discipline":["Mechanical Engineering"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:54Z"}