{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/108701"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/108701","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Efficient learning of temporal dynamics with first-order methods","abstract":"Temporal dynamical systems are pervasively used in data science to model high-dimensional data generating processes. For instance, event data are often modeled with point processes, while time series data are often captured by autoregressive models or differential equations. In this dissertation, we design algorithms for such models that enable efficient learning on large datasets. We address several key challenges that rise from real-world applications on learning structured temporal dynamics in the following aspects: • how to enable efficient nonparametric learning for large datasets? • how to learn positive-valued intensity functions for point processes? • how to learn from complex systems with implicit likelihood? • how to learn from aggregated observations of temporal dynamics? Our main focus in this dissertation will be on designing algorithms that are statistically and computationally efficient, by harnessing the power of first-order optimization methods. In particular, we provide solutions to the above challenges, by developing the following distinct, yet closely related algorithms: • First, we introduce an online learning framework for nonparametric maximum likelihood estimation of multivariate Hawkes processes, significantly outperforming the existing nonparametric learning algorithms in run time and at the same time, achieving better prediction accuracy compared to existing parametric algorithms. • Second, we develop a general framework, named “pseudo mirror descent”, to address the challenge in efficient handling of the positivity constraint when learning the intensity function of point processes. This framework greatly alleviates the burden of expensive projections required by existing nonparametric approaches without compromising the convergence guarantees. • Third, we develop a saddle point optimization approach for efficient posterior estimation in large datasets when the likelihood is not available in closed form. The proposed framework outperforms existing benchmarks significantly in terms of learning accuracy and allows us to efficiently learn sophisticated dynamics such as the evolution of a population in an ecological system. • Lastly, to learn with aggregated observations, we propose a novel learning framework based on conditional stochastic optimization, as well as a provably convergent algorithm based on gradient descent and random search for finding the optimal solution. Compared to the plug-in supervised learning setting which uses only aggregated or pre-aggregated observations, our proposed framework achieves superior performances in various applications, including the prediction of Medicare data and COVID-19 infection data. For each of the proposed algorithms and frameworks, we provide both theoretical guarantees, as well as extensive numerical comparisons with the state-of-the-art benchmarks. Our experimental results demonstrate clear advantage of our proposed algorithms, both in terms of computational efficiency and statistical accuracy when compared to the state-of-the-art.","abstract_html":"Temporal dynamical systems are pervasively used in data science to model high-dimensional data generating processes. For instance, event data are often modeled with point processes, while time series data are often captured by autoregressive models or differential equations. In this dissertation, we design algorithms for such models that enable efficient learning on large datasets. We address several key challenges that rise from real-world applications on learning structured temporal dynamics in the following aspects: • how to enable efficient nonparametric learning for large datasets? • how to learn positive-valued intensity functions for point processes? • how to learn from complex systems with implicit likelihood? • how to learn from aggregated observations of temporal dynamics? Our main focus in this dissertation will be on designing algorithms that are statistically and computationally efficient, by harnessing the power of first-order optimization methods. In particular, we provide solutions to the above challenges, by developing the following distinct, yet closely related algorithms: • First, we introduce an online learning framework for nonparametric maximum likelihood estimation of multivariate Hawkes processes, significantly outperforming the existing nonparametric learning algorithms in run time and at the same time, achieving better prediction accuracy compared to existing parametric algorithms. • Second, we develop a general framework, named “pseudo mirror descent”, to address the challenge in efficient handling of the positivity constraint when learning the intensity function of point processes. This framework greatly alleviates the burden of expensive projections required by existing nonparametric approaches without compromising the convergence guarantees. • Third, we develop a saddle point optimization approach for efficient posterior estimation in large datasets when the likelihood is not available in closed form. The proposed framework outperforms existing benchmarks significantly in terms of learning accuracy and allows us to efficiently learn sophisticated dynamics such as the evolution of a population in an ecological system. • Lastly, to learn with aggregated observations, we propose a novel learning framework based on conditional stochastic optimization, as well as a provably convergent algorithm based on gradient descent and random search for finding the optimal solution. Compared to the plug-in supervised learning setting which uses only aggregated or pre-aggregated observations, our proposed framework achieves superior performances in various applications, including the prediction of Medicare data and COVID-19 infection data. For each of the proposed algorithms and frameworks, we provide both theoretical guarantees, as well as extensive numerical comparisons with the state-of-the-art benchmarks. Our experimental results demonstrate clear advantage of our proposed algorithms, both in terms of computational efficiency and statistical accuracy when compared to the state-of-the-art.","abstract_has_math":false,"creators":["Yang, Yingxiang"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Kiyavash, Negar","He, Niao","Srikant, Rayadurgam","Raginsky, Maxim","Liu, Han"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-10-07T22:49:58Z","date_published":"2020-10-07T22:49:58Z","updated_at":"2026-07-22T22:24:48Z","subjects":["machine learning","temporal dynamics","first-order optimization","point processes","multivariate Hawkes processes","Poisson processes","positive functions","approximate Bayesian computation","macroscopic learning"],"languages":["en"],"rights":["Copyright 2020 Yingxiang Yang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/108701","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kiyavash, Negar","He, Niao","Srikant, Rayadurgam","Raginsky, Maxim","Liu, Han"]},{"key":"dc:creator","label":"Author","values":["Yang, Yingxiang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-10-07T22:49:58Z","2020-07-17","2020-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["machine learning","temporal dynamics","first-order optimization","point processes","multivariate Hawkes processes","Poisson processes","positive functions","approximate Bayesian computation","macroscopic learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Yingxiang Yang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/108701"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Temporal dynamical systems are pervasively used in data science to model high-dimensional data generating processes. For instance, event data are often modeled with point processes, while time series data are often captured by autoregressive models or differential equations. In this dissertation, we design algorithms for such models that enable efficient learning on large datasets. We address several key challenges that rise from real-world applications on learning structured temporal dynamics in the following aspects: • how to enable efficient nonparametric learning for large datasets? • how to learn positive-valued intensity functions for point processes? • how to learn from complex systems with implicit likelihood? • how to learn from aggregated observations of temporal dynamics? Our main focus in this dissertation will be on designing algorithms that are statistically and computationally efficient, by harnessing the power of first-order optimization methods. In particular, we provide solutions to the above challenges, by developing the following distinct, yet closely related algorithms: • First, we introduce an online learning framework for nonparametric maximum likelihood estimation of multivariate Hawkes processes, significantly outperforming the existing nonparametric learning algorithms in run time and at the same time, achieving better prediction accuracy compared to existing parametric algorithms. • Second, we develop a general framework, named “pseudo mirror descent”, to address the challenge in efficient handling of the positivity constraint when learning the intensity function of point processes. This framework greatly alleviates the burden of expensive projections required by existing nonparametric approaches without compromising the convergence guarantees. • Third, we develop a saddle point optimization approach for efficient posterior estimation in large datasets when the likelihood is not available in closed form. The proposed framework outperforms existing benchmarks significantly in terms of learning accuracy and allows us to efficiently learn sophisticated dynamics such as the evolution of a population in an ecological system. • Lastly, to learn with aggregated observations, we propose a novel learning framework based on conditional stochastic optimization, as well as a provably convergent algorithm based on gradient descent and random search for finding the optimal solution. Compared to the plug-in supervised learning setting which uses only aggregated or pre-aggregated observations, our proposed framework achieves superior performances in various applications, including the prediction of Medicare data and COVID-19 infection data. For each of the proposed algorithms and frameworks, we provide both theoretical guarantees, as well as extensive numerical comparisons with the state-of-the-art benchmarks. Our experimental results demonstrate clear advantage of our proposed algorithms, both in terms of computational efficiency and statistical accuracy when compared to the state-of-the-art.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2022-08-01","The student, Yingxiang Yang, accepted the attached license on 2020-07-14 at 11:39.","The student, Yingxiang Yang, submitted this Dissertation for approval on 2020-07-14 at 11:45.","This Dissertation was approved for publication on 2020-07-17 at 11:11.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15604 on 2020-10-02 at 15:50:43","Made available in DSpace on 2020-10-07T22:49:58Z (GMT). No. of bitstreams: 3 YANG-DISSERTATION-2020.pdf: 3465908 bytes, checksum: 0e738e0025a3d8c1a6c32a8e3cd2b537 (MD5) LICENSE.txt: 4211 bytes, checksum: 7c6d516fa944fc9037ffef71801bfdc4 (MD5) PROQUEST_LICENSE.txt: 4557 bytes, checksum: 387e83b6ba248595ffd5a5dbd8c22e2a (MD5) Previous issue date: 2020-07-17","Embargo set by: Seth Robbins for item 116330 Lift date: 2022-10-07T22:50:13Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Open Restriction set for Item 116330 on 2022-01-18T16:15:19Z with date null by eliasbh2@illinois.edu.","Open Restriction set for Item 116330 on 2022-01-18T16:15:21Z with date null by eliasbh2@illinois.edu.","Open"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Efficient learning of temporal dynamics with first-order methods"]}]}],"canonical_facts":{"dc:contributor":["Kiyavash, Negar","He, Niao","Srikant, Rayadurgam","Raginsky, Maxim","Liu, Han"],"dc:creator":["Yang, Yingxiang"],"dc:date":["2020-10-07T22:49:58Z","2020-07-17","2020-08"],"dc:description":["Temporal dynamical systems are pervasively used in data science to model high-dimensional data generating processes. For instance, event data are often modeled with point processes, while time series data are often captured by autoregressive models or differential equations. In this dissertation, we design algorithms for such models that enable efficient learning on large datasets. We address several key challenges that rise from real-world applications on learning structured temporal dynamics in the following aspects: • how to enable efficient nonparametric learning for large datasets? • how to learn positive-valued intensity functions for point processes? • how to learn from complex systems with implicit likelihood? • how to learn from aggregated observations of temporal dynamics? Our main focus in this dissertation will be on designing algorithms that are statistically and computationally efficient, by harnessing the power of first-order optimization methods. In particular, we provide solutions to the above challenges, by developing the following distinct, yet closely related algorithms: • First, we introduce an online learning framework for nonparametric maximum likelihood estimation of multivariate Hawkes processes, significantly outperforming the existing nonparametric learning algorithms in run time and at the same time, achieving better prediction accuracy compared to existing parametric algorithms. • Second, we develop a general framework, named “pseudo mirror descent”, to address the challenge in efficient handling of the positivity constraint when learning the intensity function of point processes. This framework greatly alleviates the burden of expensive projections required by existing nonparametric approaches without compromising the convergence guarantees. • Third, we develop a saddle point optimization approach for efficient posterior estimation in large datasets when the likelihood is not available in closed form. The proposed framework outperforms existing benchmarks significantly in terms of learning accuracy and allows us to efficiently learn sophisticated dynamics such as the evolution of a population in an ecological system. • Lastly, to learn with aggregated observations, we propose a novel learning framework based on conditional stochastic optimization, as well as a provably convergent algorithm based on gradient descent and random search for finding the optimal solution. Compared to the plug-in supervised learning setting which uses only aggregated or pre-aggregated observations, our proposed framework achieves superior performances in various applications, including the prediction of Medicare data and COVID-19 infection data. For each of the proposed algorithms and frameworks, we provide both theoretical guarantees, as well as extensive numerical comparisons with the state-of-the-art benchmarks. Our experimental results demonstrate clear advantage of our proposed algorithms, both in terms of computational efficiency and statistical accuracy when compared to the state-of-the-art.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2022-08-01","The student, Yingxiang Yang, accepted the attached license on 2020-07-14 at 11:39.","The student, Yingxiang Yang, submitted this Dissertation for approval on 2020-07-14 at 11:45.","This Dissertation was approved for publication on 2020-07-17 at 11:11.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15604 on 2020-10-02 at 15:50:43","Made available in DSpace on 2020-10-07T22:49:58Z (GMT). No. of bitstreams: 3 YANG-DISSERTATION-2020.pdf: 3465908 bytes, checksum: 0e738e0025a3d8c1a6c32a8e3cd2b537 (MD5) LICENSE.txt: 4211 bytes, checksum: 7c6d516fa944fc9037ffef71801bfdc4 (MD5) PROQUEST_LICENSE.txt: 4557 bytes, checksum: 387e83b6ba248595ffd5a5dbd8c22e2a (MD5) Previous issue date: 2020-07-17","Embargo set by: Seth Robbins for item 116330 Lift date: 2022-10-07T22:50:13Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Open Restriction set for Item 116330 on 2022-01-18T16:15:19Z with date null by eliasbh2@illinois.edu.","Open Restriction set for Item 116330 on 2022-01-18T16:15:21Z with date null by eliasbh2@illinois.edu.","Open"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/108701"],"dc:language":["en"],"dc:rights":["Copyright 2020 Yingxiang Yang"],"dc:subject":["machine learning","temporal dynamics","first-order optimization","point processes","multivariate Hawkes processes","Poisson processes","positive functions","approximate Bayesian computation","macroscopic learning"],"dc:title":["Efficient learning of temporal dynamics with first-order methods"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:48Z"}