{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/90776"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/90776","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Optimal entropy estimation on large alphabet: fundamental limits and fast algorithms","abstract":"Consider the problem of estimating the Shannon entropy of a distribution over k elements from n independent samples. We obtain the minimax mean- square error within universal multiplicative constant factors if n exceeds a constant factor of k/log(k); otherwise there exists no consistent estimator. This refines the recent result of Valiant and Valiant (2011) that the mini- mal sample size for consistent entropy estimation scales. The apparatus of best polynomial approximation plays a key role in both the construction of optimal estimators and, via a duality argument, the minimax lower bound.","abstract_html":"Consider the problem of estimating the Shannon entropy of a distribution over k elements from n independent samples. We obtain the minimax mean- square error within universal multiplicative constant factors if n exceeds a constant factor of k/log(k); otherwise there exists no consistent estimator. This refines the recent result of Valiant and Valiant (2011) that the mini- mal sample size for consistent entropy estimation scales. The apparatus of best polynomial approximation plays a key role in both the construction of optimal estimators and, via a duality argument, the minimax lower bound.","abstract_has_math":false,"creators":["Yang, Pengkun"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Wu, Yihong"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2016,"date_issued":"2016-07-07T20:27:34Z","date_published":"2016-07-07T20:27:34Z","updated_at":"2026-07-22T22:26:34Z","subjects":["entropy estimation","large alphabet"],"languages":["en"],"rights":["Copyright 2016 Pengkun Yang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/90776","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Wu, Yihong"]},{"key":"dc:creator","label":"Author","values":["Yang, Pengkun"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2016-07-07T20:27:34Z","2018-07-08T09:15:36Z","2016-04-19","2016-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["entropy estimation","large alphabet"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2016 Pengkun Yang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/90776"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Consider the problem of estimating the Shannon entropy of a distribution over k elements from n independent samples. We obtain the minimax mean- square error within universal multiplicative constant factors if n exceeds a constant factor of k/log(k); otherwise there exists no consistent estimator. This refines the recent result of Valiant and Valiant (2011) that the mini- mal sample size for consistent entropy estimation scales. The apparatus of best polynomial approximation plays a key role in both the construction of optimal estimators and, via a duality argument, the minimax lower bound.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2018-05-01","The student, Pengkun Yang, accepted the attached license on 2016-04-18 at 09:01.","The student, Pengkun Yang, submitted this Thesis for approval on 2016-04-18 at 09:10.","This Thesis was approved for publication on 2016-04-19 at 11:16.","DSpace SAF Submission Ingestion Package generated from Vireo submission #9287 on 2016-07-07 at 13:49:50","Made available in DSpace on 2016-07-07T20:27:34Z (GMT). No. of bitstreams: 2 YANG-THESIS-2016.pdf: 437966 bytes, checksum: 56a247801d3c1d9b592a498347b02e4b (MD5) LICENSE.txt: 4209 bytes, checksum: 45523ffcb7e2b421561eb00e4bfccfa7 (MD5) Previous issue date: 2016-04-19","Embargo set by: Seth Robbins for item 93128 Lift date: 2018-07-07T20:28:14Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 93128 Lift date: 2018-07-07T20:35:34Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 93128 on 2018-07-08T09:15:36Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Optimal entropy estimation on large alphabet: fundamental limits and fast algorithms"]}]}],"canonical_facts":{"dc:contributor":["Wu, Yihong"],"dc:creator":["Yang, Pengkun"],"dc:date":["2016-07-07T20:27:34Z","2018-07-08T09:15:36Z","2016-04-19","2016-05"],"dc:description":["Consider the problem of estimating the Shannon entropy of a distribution over k elements from n independent samples. We obtain the minimax mean- square error within universal multiplicative constant factors if n exceeds a constant factor of k/log(k); otherwise there exists no consistent estimator. This refines the recent result of Valiant and Valiant (2011) that the mini- mal sample size for consistent entropy estimation scales. The apparatus of best polynomial approximation plays a key role in both the construction of optimal estimators and, via a duality argument, the minimax lower bound.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2018-05-01","The student, Pengkun Yang, accepted the attached license on 2016-04-18 at 09:01.","The student, Pengkun Yang, submitted this Thesis for approval on 2016-04-18 at 09:10.","This Thesis was approved for publication on 2016-04-19 at 11:16.","DSpace SAF Submission Ingestion Package generated from Vireo submission #9287 on 2016-07-07 at 13:49:50","Made available in DSpace on 2016-07-07T20:27:34Z (GMT). No. of bitstreams: 2 YANG-THESIS-2016.pdf: 437966 bytes, checksum: 56a247801d3c1d9b592a498347b02e4b (MD5) LICENSE.txt: 4209 bytes, checksum: 45523ffcb7e2b421561eb00e4bfccfa7 (MD5) Previous issue date: 2016-04-19","Embargo set by: Seth Robbins for item 93128 Lift date: 2018-07-07T20:28:14Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 93128 Lift date: 2018-07-07T20:35:34Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 93128 on 2018-07-08T09:15:36Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/90776"],"dc:language":["en"],"dc:rights":["Copyright 2016 Pengkun Yang"],"dc:subject":["entropy estimation","large alphabet"],"dc:title":["Optimal entropy estimation on large alphabet: fundamental limits and fast algorithms"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:26:34Z"}