Pattern recognition problems span a broad range of applications, where each application has its own tolerance on classification error. The varying levels of risk associated with many pattern recognition applications indicate the need for a versatile algorithm with the ability to measure its own reliability. In this work, the supervised incremental learning algorithm Learn++ [1, 2], which exploits the synergistic power of an ensemble of classifiers, is further developed to add the capability of assessing its own confidence. Estimation of the true generalization performance of the classifier as well as the confidences on classification of individual data instances is investigated separately. Several confidence estimation techniques are explored such as majority voting, variance based confidence estimation, and the weighted exponential method.
Experiments for incorporating confidence estimation techniques for evaluating the confidence of decisions made by Learn++ produced promising results, with weighted exponential based confidence estimation providing the best performance. The objective of the confidence experiments was to evaluate the algorithm's ability to assess the confidence in its own decisions. In addition to the ability of Learn++ to assess its own confidence in generalization performance and individual data instances, several additional desirable traits were also observed. Confidence estimates on individual instances exhibit increased values on the correctly classified instances of the test dataset as additional subsets of training data become available. Furthermore, \"decreasing\" confidence levels for misclassified test data instances is observed, which indicates that the algorithm has the potential ability to detect its own misclassifications, effectively warning the user that the classifier outputs for those instances need to be evaluated with caution. Finally, the confidences of those instances, which were correctly classified, were generally high or very high after multiple incremental learning sessions.
Learn++ is in essence a procedure to sequentially generate and combine an ensemble of classifiers to achieve its desired properties of incremental learning and estimation of confidence levels. It was found that the algorithm itself is independent of the specific classifier architectures used in the ensemble, and has been shown to work with a versatile set of supervised classifiers, including the multilayer perceptron (MLP), radial basis function (RBF), and probabilistic neural networks (PNN). In addition to its versatility with respect to the classifier used, it can be used on a wide variety of applications. The small subset of the applications that were explored in this work includes volatile organic compound identification, nondestructive evaluation of a nuclear plant's pipings and tubings, handwritten character recognition, glass identification, iris plant recognition, and identifying the existence of structures within the ionosphere.
","abstract_html":"<p>Pattern recognition problems span a broad range of applications, where each application has its own tolerance on classification error. The varying levels of risk associated with many pattern recognition applications indicate the need for a versatile algorithm with the ability to measure its own reliability. In this work, the supervised incremental learning algorithm Learn++ [1, 2], which exploits the synergistic power of an ensemble of classifiers, is further developed to add the capability of assessing its own confidence. Estimation of the true generalization performance of the classifier as well as the confidences on classification of individual data instances is investigated separately. Several confidence estimation techniques are explored such as majority voting, variance based confidence estimation, and the weighted exponential method.</p> <p>Experiments for incorporating confidence estimation techniques for evaluating the confidence of decisions made by Learn++ produced promising results, with weighted exponential based confidence estimation providing the best performance. The objective of the confidence experiments was to evaluate the algorithm's ability to assess the confidence in its own decisions. In addition to the ability of Learn++ to assess its own confidence in generalization performance and individual data instances, several additional desirable traits were also observed. Confidence estimates on individual instances exhibit increased values on the correctly classified instances of the test dataset as additional subsets of training data become available. Furthermore, "decreasing" confidence levels for misclassified test data instances is observed, which indicates that the algorithm has the potential ability to detect its own misclassifications, effectively warning the user that the classifier outputs for those instances need to be evaluated with caution. Finally, the confidences of those instances, which were correctly classified, were generally high or very high after multiple incremental learning sessions.</p> <p>Learn++ is in essence a procedure to sequentially generate and combine an ensemble of classifiers to achieve its desired properties of incremental learning and estimation of confidence levels. It was found that the algorithm itself is independent of the specific classifier architectures used in the ensemble, and has been shown to work with a versatile set of supervised classifiers, including the multilayer perceptron (MLP), radial basis function (RBF), and probabilistic neural networks (PNN). In addition to its versatility with respect to the classifier used, it can be used on a wide variety of applications. The small subset of the applications that were explored in this work includes volatile organic compound identification, nondestructive evaluation of a nuclear plant's pipings and tubings, handwritten character recognition, glass identification, iris plant recognition, and identifying the existence of structures within the ionosphere.</p>","abstract_has_math":false,"creators":["Byorick, Jeffrey T."],"institution":null,"degree_name":"M.S. in Electrical Engineering","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engineering","degree_department":null,"school":null,"contributors":["Polikar, Robi"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2003,"date_issued":"2003-04-03T08:00:00Z","date_published":"2003-04-03T08:00:00Z","updated_at":"2026-07-24T04:14:13Z","subjects":["Pattern recognition systems; Machine learning","Electrical and Computer Engineering"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://rdw.rowan.edu/etd/1268","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Polikar, Robi"]},{"key":"dc:creator","label":"Author","values":["Byorick, Jeffrey T."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2016-05-02T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S. in Electrical Engineering"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Pattern recognition systems; Machine learning","Electrical and Computer Engineering"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://rdw.rowan.edu/etd/1268"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Pattern recognition problems span a broad range of applications, where each application has its own tolerance on classification error. The varying levels of risk associated with many pattern recognition applications indicate the need for a versatile algorithm with the ability to measure its own reliability. In this work, the supervised incremental learning algorithm Learn++ [1, 2], which exploits the synergistic power of an ensemble of classifiers, is further developed to add the capability of assessing its own confidence. Estimation of the true generalization performance of the classifier as well as the confidences on classification of individual data instances is investigated separately. Several confidence estimation techniques are explored such as majority voting, variance based confidence estimation, and the weighted exponential method.
Experiments for incorporating confidence estimation techniques for evaluating the confidence of decisions made by Learn++ produced promising results, with weighted exponential based confidence estimation providing the best performance. The objective of the confidence experiments was to evaluate the algorithm's ability to assess the confidence in its own decisions. In addition to the ability of Learn++ to assess its own confidence in generalization performance and individual data instances, several additional desirable traits were also observed. Confidence estimates on individual instances exhibit increased values on the correctly classified instances of the test dataset as additional subsets of training data become available. Furthermore, \"decreasing\" confidence levels for misclassified test data instances is observed, which indicates that the algorithm has the potential ability to detect its own misclassifications, effectively warning the user that the classifier outputs for those instances need to be evaluated with caution. Finally, the confidences of those instances, which were correctly classified, were generally high or very high after multiple incremental learning sessions.
Learn++ is in essence a procedure to sequentially generate and combine an ensemble of classifiers to achieve its desired properties of incremental learning and estimation of confidence levels. It was found that the algorithm itself is independent of the specific classifier architectures used in the ensemble, and has been shown to work with a versatile set of supervised classifiers, including the multilayer perceptron (MLP), radial basis function (RBF), and probabilistic neural networks (PNN). In addition to its versatility with respect to the classifier used, it can be used on a wide variety of applications. The small subset of the applications that were explored in this work includes volatile organic compound identification, nondestructive evaluation of a nuclear plant's pipings and tubings, handwritten character recognition, glass identification, iris plant recognition, and identifying the existence of structures within the ionosphere.
"]},{"key":"dc:title","label":"Title","values":["Assessing classification confidence using a weighted exponential based technique with the Learn++ incremental learning algorithm"]}]}],"canonical_facts":{"dc:contributor":["Polikar, Robi"],"dc:creator":["Byorick, Jeffrey T."],"dc:date.available":["2016-05-02T07:00:00Z"],"dc:description.abstract":["Pattern recognition problems span a broad range of applications, where each application has its own tolerance on classification error. The varying levels of risk associated with many pattern recognition applications indicate the need for a versatile algorithm with the ability to measure its own reliability. In this work, the supervised incremental learning algorithm Learn++ [1, 2], which exploits the synergistic power of an ensemble of classifiers, is further developed to add the capability of assessing its own confidence. Estimation of the true generalization performance of the classifier as well as the confidences on classification of individual data instances is investigated separately. Several confidence estimation techniques are explored such as majority voting, variance based confidence estimation, and the weighted exponential method.
Experiments for incorporating confidence estimation techniques for evaluating the confidence of decisions made by Learn++ produced promising results, with weighted exponential based confidence estimation providing the best performance. The objective of the confidence experiments was to evaluate the algorithm's ability to assess the confidence in its own decisions. In addition to the ability of Learn++ to assess its own confidence in generalization performance and individual data instances, several additional desirable traits were also observed. Confidence estimates on individual instances exhibit increased values on the correctly classified instances of the test dataset as additional subsets of training data become available. Furthermore, \"decreasing\" confidence levels for misclassified test data instances is observed, which indicates that the algorithm has the potential ability to detect its own misclassifications, effectively warning the user that the classifier outputs for those instances need to be evaluated with caution. Finally, the confidences of those instances, which were correctly classified, were generally high or very high after multiple incremental learning sessions.
Learn++ is in essence a procedure to sequentially generate and combine an ensemble of classifiers to achieve its desired properties of incremental learning and estimation of confidence levels. It was found that the algorithm itself is independent of the specific classifier architectures used in the ensemble, and has been shown to work with a versatile set of supervised classifiers, including the multilayer perceptron (MLP), radial basis function (RBF), and probabilistic neural networks (PNN). In addition to its versatility with respect to the classifier used, it can be used on a wide variety of applications. The small subset of the applications that were explored in this work includes volatile organic compound identification, nondestructive evaluation of a nuclear plant's pipings and tubings, handwritten character recognition, glass identification, iris plant recognition, and identifying the existence of structures within the ionosphere.
"],"dc:identifier":["https://rdw.rowan.edu/etd/1268"],"dc:subject":["Pattern recognition systems; Machine learning","Electrical and Computer Engineering"],"dc:title":["Assessing classification confidence using a weighted exponential based technique with the Learn++ incremental learning algorithm"],"thesis:degree_discipline":["Electrical & Computer Engineering"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S. in Electrical Engineering"]},"updated_at":"2026-07-24T04:14:13Z"}