{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/122241"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/122241","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Evaluation of the smoothing activation function in neural networks for business applications","abstract":"With vast improvements in computational power, increased accessibility to big data, and rapid innovations in computing algorithms, the use of neural networks for both engineering and business purposes was met with a renewed interest beginning in early 2000s. Amidst substantial development, the Softplus and Rectified Linear Unit (ReLU) activation functions were introduced in 2000 and 2001 respectively, with the latter emerging as the more popular choice of activation function in neural networks. Notably, the ReLU activation function maintains a high degree of gradient propagation while presenting greater model sparsity and computational efficiency over Softplus. As an alternative to the ReLU, a family of a modified Softplus activation function - the \"Smoothing\" activation function of the form g(z) = [mu] log(1 + e[superscript z/[mu]) has been proposed. Theoretically, the Smoothing activation function will leverage the high degree of gradient propagation and model simplicity characteristic of the ReLU function, while eliminating possible issues associated with the non-differentiability of ReLU about the origin. In this research, the performance of the Smoothing family of activation functions vis-à-vis the ReLU activation function will be examined.","abstract_html":"With vast improvements in computational power, increased accessibility to big data, and rapid innovations in computing algorithms, the use of neural networks for both engineering and business purposes was met with a renewed interest beginning in early 2000s. Amidst substantial development, the Softplus and Rectified Linear Unit (ReLU) activation functions were introduced in 2000 and 2001 respectively, with the latter emerging as the more popular choice of activation function in neural networks. Notably, the ReLU activation function maintains a high degree of gradient propagation while presenting greater model sparsity and computational efficiency over Softplus. As an alternative to the ReLU, a family of a modified Softplus activation function - the "Smoothing" activation function of the form g(z) = [mu] log(1 + e[superscript z/[mu]) has been proposed. Theoretically, the Smoothing activation function will leverage the high degree of gradient propagation and model simplicity characteristic of the ReLU function, while eliminating possible issues associated with the non-differentiability of ReLU about the origin. In this research, the performance of the Smoothing family of activation functions vis-à-vis the ReLU activation function will be examined.","abstract_has_math":false,"creators":["Ang, Jun Siong."],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Engineering and Management Program","school":null,"contributors":[],"advisors":["Robert Freund."],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019","date_published":"2019","updated_at":"2026-07-22T22:20:55Z","subjects":["Engineering and Management Program.","System Design and Management Program."],"languages":["eng"],"rights":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."],"rights_urls":["http://dspace.mit.edu/handle/1721.1/7582"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/122241","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Robert Freund."]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Engineering and Management Program","SysDes","System Design and Management Program"]},{"key":"dc:contributor.other","label":"Dc Contributor Other","values":["Massachusetts Institute of Technology. Engineering and Management Program.","System Design and Management Program."]},{"key":"dc:creator","label":"Author","values":["Ang, Jun Siong."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2019-09-17T19:49:40Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2019-09-17T19:49:40Z"]},{"key":"dc:date.issued","label":"Date","values":["2019"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Engineering and Management Program.","System Design and Management Program."]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://dspace.mit.edu/handle/1721.1/7582"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/122241"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis: S.M. in Engineering and Management, Massachusetts Institute of Technology, System Design and Management Program, 2019","Cataloged from PDF version of thesis.","Includes bibliographical references (pages R-1 to R-2)."]},{"key":"dc:description.abstract","label":"Abstract","values":["With vast improvements in computational power, increased accessibility to big data, and rapid innovations in computing algorithms, the use of neural networks for both engineering and business purposes was met with a renewed interest beginning in early 2000s. Amidst substantial development, the Softplus and Rectified Linear Unit (ReLU) activation functions were introduced in 2000 and 2001 respectively, with the latter emerging as the more popular choice of activation function in neural networks. Notably, the ReLU activation function maintains a high degree of gradient propagation while presenting greater model sparsity and computational efficiency over Softplus. As an alternative to the ReLU, a family of a modified Softplus activation function - the \"Smoothing\" activation function of the form g(z) = [mu] log(1 + e[superscript z/[mu]) has been proposed. Theoretically, the Smoothing activation function will leverage the high degree of gradient propagation and model simplicity characteristic of the ReLU function, while eliminating possible issues associated with the non-differentiability of ReLU about the origin. In this research, the performance of the Smoothing family of activation functions vis-à-vis the ReLU activation function will be examined."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["S.M. in Engineering and Management"]},{"key":"dc:title","label":"Title","values":["Evaluation of the smoothing activation function in neural networks for business applications"]}]}],"canonical_facts":{"dc:contributor.advisor":["Robert Freund."],"dc:contributor.department":["Massachusetts Institute of Technology. Engineering and Management Program","SysDes","System Design and Management Program"],"dc:contributor.other":["Massachusetts Institute of Technology. Engineering and Management Program.","System Design and Management Program."],"dc:creator":["Ang, Jun Siong."],"dc:date.accessioned":["2019-09-17T19:49:40Z"],"dc:date.available":["2019-09-17T19:49:40Z"],"dc:date.issued":["2019"],"dc:description":["Thesis: S.M. in Engineering and Management, Massachusetts Institute of Technology, System Design and Management Program, 2019","Cataloged from PDF version of thesis.","Includes bibliographical references (pages R-1 to R-2)."],"dc:description.abstract":["With vast improvements in computational power, increased accessibility to big data, and rapid innovations in computing algorithms, the use of neural networks for both engineering and business purposes was met with a renewed interest beginning in early 2000s. Amidst substantial development, the Softplus and Rectified Linear Unit (ReLU) activation functions were introduced in 2000 and 2001 respectively, with the latter emerging as the more popular choice of activation function in neural networks. Notably, the ReLU activation function maintains a high degree of gradient propagation while presenting greater model sparsity and computational efficiency over Softplus. As an alternative to the ReLU, a family of a modified Softplus activation function - the \"Smoothing\" activation function of the form g(z) = [mu] log(1 + e[superscript z/[mu]) has been proposed. Theoretically, the Smoothing activation function will leverage the high degree of gradient propagation and model simplicity characteristic of the ReLU function, while eliminating possible issues associated with the non-differentiability of ReLU about the origin. In this research, the performance of the Smoothing family of activation functions vis-à-vis the ReLU activation function will be examined."],"dc:description.degree":["S.M. in Engineering and Management"],"dc:identifier.uri":["https://hdl.handle.net/1721.1/122241"],"dc:language.iso":["eng"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."],"dc:rights.uri":["http://dspace.mit.edu/handle/1721.1/7582"],"dc:subject":["Engineering and Management Program.","System Design and Management Program."],"dc:title":["Evaluation of the smoothing activation function in neural networks for business applications"],"dc:type":["Thesis"],"thesis:degree_name":["Master"]},"updated_at":"2026-07-22T22:20:55Z"}