{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/127478"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/127478","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Can LLMs implicitly learn numeric parameter constraints in data science APIs?","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","abstract_has_math":false,"creators":["Deng, Yinlin"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Zhang, Lingming"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-12-05","date_published":"2024-12-05","updated_at":"2026-07-22T22:25:04Z","subjects":["Program Synthesis","Evaluation","Dataset","Data Science","Llm4code"],"languages":["en","eng"],"rights":["Copyright 2024 Yinlin Deng"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/127478","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhang, Lingming"]},{"key":"dc:creator","label":"Author","values":["Deng, Yinlin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-12-05","2024-12"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Program Synthesis","Evaluation","Dataset","Data Science","Llm4code"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Yinlin Deng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/127478"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","The student, Yinlin Deng, accepted the attached license on 2024-12-03 at 15:46.","The student, Yinlin Deng, submitted this Thesis for approval on 2024-12-03 at 15:52.","This Thesis was approved for publication on 2024-12-05 at 10:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21430 on 2025-03-28 at 14:55:28","Data science (DS) programs, typically built on popular DS libraries (such as PyTorch and NumPy) with thousands of APIs, serve as the cornerstone for various mission-critical domains such as financial systems, autonomous driving software, and coding assistants. Recently, large language models (LLMs) have been widely applied to generate DS programs across diverse scenarios, such as assisting users for DS programming or detecting critical vulnerabilities in DS frameworks. Such applications have all operated under the assumption, that LLMs can implicitly model the numerical parameter constraints in DS library APIs and produce valid code. However, this assumption has not been rigorously studied in the literature. In this paper, we empirically investigate the proficiency of LLMs to handle these implicit numerical constraints when generating DS programs. We studied 28 widely used APIs from PyTorch and NumPy, and scrutinized the LLMs’ generation performance in different levels of granularity: full programs and all parameters completion. The results show that LLMs are great at generating simple DS programs, particularly those that follow common patterns seen in training data. However, as we increase the difficulty by providing more complex/unusual inputs, the performance of LLMs drops significantly."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Can LLMs implicitly learn numeric parameter constraints in data science APIs?"]}]}],"canonical_facts":{"dc:contributor":["Zhang, Lingming"],"dc:creator":["Deng, Yinlin"],"dc:date":["2024-12-05","2024-12"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","The student, Yinlin Deng, accepted the attached license on 2024-12-03 at 15:46.","The student, Yinlin Deng, submitted this Thesis for approval on 2024-12-03 at 15:52.","This Thesis was approved for publication on 2024-12-05 at 10:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21430 on 2025-03-28 at 14:55:28","Data science (DS) programs, typically built on popular DS libraries (such as PyTorch and NumPy) with thousands of APIs, serve as the cornerstone for various mission-critical domains such as financial systems, autonomous driving software, and coding assistants. Recently, large language models (LLMs) have been widely applied to generate DS programs across diverse scenarios, such as assisting users for DS programming or detecting critical vulnerabilities in DS frameworks. Such applications have all operated under the assumption, that LLMs can implicitly model the numerical parameter constraints in DS library APIs and produce valid code. However, this assumption has not been rigorously studied in the literature. In this paper, we empirically investigate the proficiency of LLMs to handle these implicit numerical constraints when generating DS programs. We studied 28 widely used APIs from PyTorch and NumPy, and scrutinized the LLMs’ generation performance in different levels of granularity: full programs and all parameters completion. The results show that LLMs are great at generating simple DS programs, particularly those that follow common patterns seen in training data. However, as we increase the difficulty by providing more complex/unusual inputs, the performance of LLMs drops significantly."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/127478"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Yinlin Deng"],"dc:subject":["Program Synthesis","Evaluation","Dataset","Data Science","Llm4code"],"dc:title":["Can LLMs implicitly learn numeric parameter constraints in data science APIs?"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:04Z"}