{"id":{"repo_id":"brazil-ufpe","oai_identifier":"oai:repositorio.ufpe.br:123456789/38541"},"canonical_url":"https://search.dev.ndltd.org/etd/brazil-ufpe/oai:repositorio.ufpe.br:123456789/38541","repository":{"repo_id":"brazil-ufpe","name":"Brazil UFPE","base_url":"https://repositorio.ufpe.br/oai/request"},"display":{"title":"Time Aware Sigmoid Optimization : a new learning rate scheduling method","abstract":"The correct choice of hyperparameters for the training of a deep neural network is a critical step to achieve a good result. Good hyperparameters would give rise to faster training and a lower error rate, while bad choices could make the network not even converge, rendering the whole training process useless. Among all the existing hyperparameters, perhaps the one with the greatest importance is the learning rate, which controls how the weights of a neural network are going to change at each interaction. In that context, by analyzing some theoretical findings in the area of information theory and topology of the loss function in deep learning, the author was able to come up with a new training rate decay method called Training Aware Sigmoid Optimization (TASO), which proposes a dual-phase during training. The proposed method aims to improve training, achieving a better inference performance in a reduced amount of time. A series of tests were done to evaluate this hypothesis, comparing TASO with different training methods such as Adam, ADAGrad, RMSProp, and SGD. Results obtained on three datasets (MNIST, CIFAR10, and CIFAR100) and with three different architectures (Lenet, VGG, and RESNET) have shown that TASO presents, in fact, an overall better performance than the other evaluated methods.","abstract_html":"The correct choice of hyperparameters for the training of a deep neural network is a critical step to achieve a good result. Good hyperparameters would give rise to faster training and a lower error rate, while bad choices could make the network not even converge, rendering the whole training process useless. Among all the existing hyperparameters, perhaps the one with the greatest importance is the learning rate, which controls how the weights of a neural network are going to change at each interaction. In that context, by analyzing some theoretical findings in the area of information theory and topology of the loss function in deep learning, the author was able to come up with a new training rate decay method called Training Aware Sigmoid Optimization (TASO), which proposes a dual-phase during training. The proposed method aims to improve training, achieving a better inference performance in a reduced amount of time. A series of tests were done to evaluate this hypothesis, comparing TASO with different training methods such as Adam, ADAGrad, RMSProp, and SGD. Results obtained on three datasets (MNIST, CIFAR10, and CIFAR100) and with three different architectures (Lenet, VGG, and RESNET) have shown that TASO presents, in fact, an overall better performance than the other evaluated methods.","abstract_has_math":false,"creators":["LEUCHTENBERG, Pedro Henrique Dreyer"],"institution":"Universidade Federal de Pernambuco","degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["ZANCHETTIN, Cleber"],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-09-06","date_published":"2019-09-06","updated_at":"2026-07-24T01:18:32Z","subjects":["Inteligência computacional","Aprendizagem de máquinas","Redes neurais profundas","Taxa de aprendizado"],"languages":["eng"],"rights":["openAccess","Attribution-NonCommercial-NoDerivs 3.0 Brazil"],"rights_urls":["http://creativecommons.org/licenses/by-nc-nd/3.0/br/"],"identifier_entries":[]},"links":{"outbound_url":"https://repositorio.ufpe.br/handle/123456789/38541","outbound_label":"Repository record","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["ZANCHETTIN, Cleber"]},{"key":"dc:creator","label":"Author","values":["LEUCHTENBERG, Pedro Henrique Dreyer"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2020-11-09T13:57:38Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2020-11-09T13:57:38Z"]},{"key":"dc:date.issued","label":"Date","values":["2019-09-06"]},{"key":"dc:publisher","label":"Institution","values":["Universidade Federal de Pernambuco"]},{"key":"dc:type","label":"Dc Type","values":["masterThesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Inteligência computacional","Aprendizagem de máquinas","Redes neurais profundas","Taxa de aprendizado"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["openAccess","Attribution-NonCommercial-NoDerivs 3.0 Brazil"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by-nc-nd/3.0/br/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://repositorio.ufpe.br/handle/123456789/38541"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["LEUCHTENBERG, Pedro Henrique Dreyer​, também é conhecido em citações bibliográficas por: ​DREYER, Pedro Henrique"]},{"key":"dc:description.abstract","label":"Abstract","values":["The correct choice of hyperparameters for the training of a deep neural network is a critical step to achieve a good result. Good hyperparameters would give rise to faster training and a lower error rate, while bad choices could make the network not even converge, rendering the whole training process useless. Among all the existing hyperparameters, perhaps the one with the greatest importance is the learning rate, which controls how the weights of a neural network are going to change at each interaction. In that context, by analyzing some theoretical findings in the area of information theory and topology of the loss function in deep learning, the author was able to come up with a new training rate decay method called Training Aware Sigmoid Optimization (TASO), which proposes a dual-phase during training. The proposed method aims to improve training, achieving a better inference performance in a reduced amount of time. A series of tests were done to evaluate this hypothesis, comparing TASO with different training methods such as Adam, ADAGrad, RMSProp, and SGD. Results obtained on three datasets (MNIST, CIFAR10, and CIFAR100) and with three different architectures (Lenet, VGG, and RESNET) have shown that TASO presents, in fact, an overall better performance than the other evaluated methods."]},{"key":"dc:title","label":"Title","values":["Time Aware Sigmoid Optimization : a new learning rate scheduling method"]}]}],"canonical_facts":{"dc:contributor.advisor":["ZANCHETTIN, Cleber"],"dc:creator":["LEUCHTENBERG, Pedro Henrique Dreyer"],"dc:date.accessioned":["2020-11-09T13:57:38Z"],"dc:date.available":["2020-11-09T13:57:38Z"],"dc:date.issued":["2019-09-06"],"dc:description":["LEUCHTENBERG, Pedro Henrique Dreyer​, também é conhecido em citações bibliográficas por: ​DREYER, Pedro Henrique"],"dc:description.abstract":["The correct choice of hyperparameters for the training of a deep neural network is a critical step to achieve a good result. Good hyperparameters would give rise to faster training and a lower error rate, while bad choices could make the network not even converge, rendering the whole training process useless. Among all the existing hyperparameters, perhaps the one with the greatest importance is the learning rate, which controls how the weights of a neural network are going to change at each interaction. In that context, by analyzing some theoretical findings in the area of information theory and topology of the loss function in deep learning, the author was able to come up with a new training rate decay method called Training Aware Sigmoid Optimization (TASO), which proposes a dual-phase during training. The proposed method aims to improve training, achieving a better inference performance in a reduced amount of time. A series of tests were done to evaluate this hypothesis, comparing TASO with different training methods such as Adam, ADAGrad, RMSProp, and SGD. Results obtained on three datasets (MNIST, CIFAR10, and CIFAR100) and with three different architectures (Lenet, VGG, and RESNET) have shown that TASO presents, in fact, an overall better performance than the other evaluated methods."],"dc:identifier.uri":["https://repositorio.ufpe.br/handle/123456789/38541"],"dc:language.iso":["eng"],"dc:publisher":["Universidade Federal de Pernambuco"],"dc:rights":["openAccess","Attribution-NonCommercial-NoDerivs 3.0 Brazil"],"dc:rights.uri":["http://creativecommons.org/licenses/by-nc-nd/3.0/br/"],"dc:subject":["Inteligência computacional","Aprendizagem de máquinas","Redes neurais profundas","Taxa de aprendizado"],"dc:title":["Time Aware Sigmoid Optimization : a new learning rate scheduling method"],"dc:type":["masterThesis"]},"updated_at":"2026-07-24T01:18:32Z"}