Natural Language Processing (NLP) systems are included everywhere on the internet from search engines, language translations to more advanced systems like voice assistant and customer service. Since humans are always on the receiving end of NLP technologies, it is very important to analyze whether or not the Large Language Models (LLMs) in use have bias and are therefore unfair. The majority of the research in NLP bias has focused on societal stereotype biases embedded in LLMs. However, our research focuses on all types of biases, namely model class level bias, stereotype bias and domain bias present in LLMs. Model class level bias happens when a model tends to favor some classification labels or outputs compared to the others. We investigate how a classification model hugely favors one class with respect to another. We propose a bias evaluation technique called \\textit{directional pairwise class confusion bias} that highlights an LLM's bias on pairs of classes. Unfavorable kind of stereotype bias takes place when LLMs cause significant injustice or harm to disadvantaged or marginalized group of people. Although the most advanced deep LLMs claim to mimic human responses via powerful and sophisticated algorithms, the capabilities that such models offer have shown to possess bias. Quantifying such stereotype biases appropriately is essential so that the bias measures can be used to calibrate potential harm the models can cause. On the other hand, domain biases are desired for the model because it indicates the model is learning necessary facts for it to be powerful. We devise techniques to measure class level, stereotype, and domain biases appropriately.
","abstract_html":"<p>Natural Language Processing (NLP) systems are included everywhere on the internet from search engines, language translations to more advanced systems like voice assistant and customer service. Since humans are always on the receiving end of NLP technologies, it is very important to analyze whether or not the Large Language Models (LLMs) in use have bias and are therefore unfair. The majority of the research in NLP bias has focused on societal stereotype biases embedded in LLMs. However, our research focuses on all types of biases, namely model class level bias, stereotype bias and domain bias present in LLMs. Model class level bias happens when a model tends to favor some classification labels or outputs compared to the others. We investigate how a classification model hugely favors one class with respect to another. We propose a bias evaluation technique called \\textit{directional pairwise class confusion bias} that highlights an LLM's bias on pairs of classes. Unfavorable kind of stereotype bias takes place when LLMs cause significant injustice or harm to disadvantaged or marginalized group of people. Although the most advanced deep LLMs claim to mimic human responses via powerful and sophisticated algorithms, the capabilities that such models offer have shown to possess bias. Quantifying such stereotype biases appropriately is essential so that the bias measures can be used to calibrate potential harm the models can cause. On the other hand, domain biases are desired for the model because it indicates the model is learning necessary facts for it to be powerful. We devise techniques to measure class level, stereotype, and domain biases appropriately.</p>","abstract_has_math":false,"creators":["Sayenju, Sudhashree"],"institution":null,"degree_name":"Doctor of Philosophy in Analytic and Data Science","degree_level":"Dissertation","degree_discipline":"Statistics and Analytical Sciences","degree_department":null,"school":null,"contributors":["Dr. Ramazan Aygun","Dr. Ying Xie","Dr. Yifan Zhang"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-04-28T07:00:00Z","date_published":"2023-04-28T07:00:00Z","updated_at":"2026-07-24T02:44:06Z","subjects":["Natural Language Processing","Bias evaluation","Bias mitigation","Artificial Intelligence and Robotics","Data Science","Numerical Analysis and Scientific Computing","Risk Analysis","Statistical Models","Statistics and Probability"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.kennesaw.edu/dataphd_etd/16","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Dr. Ramazan Aygun","Dr. Ying Xie","Dr. Yifan Zhang"]},{"key":"dc:creator","label":"Author","values":["Sayenju, Sudhashree"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2024-05-10T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Statistics and Analytical Sciences"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy in Analytic and Data Science"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Natural Language Processing","Bias evaluation","Bias mitigation","Artificial Intelligence and Robotics","Data Science","Numerical Analysis and Scientific Computing","Risk Analysis","Statistical Models","Statistics and Probability"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.kennesaw.edu/dataphd_etd/16"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Natural Language Processing (NLP) systems are included everywhere on the internet from search engines, language translations to more advanced systems like voice assistant and customer service. Since humans are always on the receiving end of NLP technologies, it is very important to analyze whether or not the Large Language Models (LLMs) in use have bias and are therefore unfair. The majority of the research in NLP bias has focused on societal stereotype biases embedded in LLMs. However, our research focuses on all types of biases, namely model class level bias, stereotype bias and domain bias present in LLMs. Model class level bias happens when a model tends to favor some classification labels or outputs compared to the others. We investigate how a classification model hugely favors one class with respect to another. We propose a bias evaluation technique called \\textit{directional pairwise class confusion bias} that highlights an LLM's bias on pairs of classes. Unfavorable kind of stereotype bias takes place when LLMs cause significant injustice or harm to disadvantaged or marginalized group of people. Although the most advanced deep LLMs claim to mimic human responses via powerful and sophisticated algorithms, the capabilities that such models offer have shown to possess bias. Quantifying such stereotype biases appropriately is essential so that the bias measures can be used to calibrate potential harm the models can cause. On the other hand, domain biases are desired for the model because it indicates the model is learning necessary facts for it to be powerful. We devise techniques to measure class level, stereotype, and domain biases appropriately.
"]},{"key":"dc:title","label":"Title","values":["Quantification of Various Types of Biases in Large Language Models"]}]}],"canonical_facts":{"dc:contributor":["Dr. Ramazan Aygun","Dr. Ying Xie","Dr. Yifan Zhang"],"dc:creator":["Sayenju, Sudhashree"],"dc:date.available":["2024-05-10T07:00:00Z"],"dc:description.abstract":["Natural Language Processing (NLP) systems are included everywhere on the internet from search engines, language translations to more advanced systems like voice assistant and customer service. Since humans are always on the receiving end of NLP technologies, it is very important to analyze whether or not the Large Language Models (LLMs) in use have bias and are therefore unfair. The majority of the research in NLP bias has focused on societal stereotype biases embedded in LLMs. However, our research focuses on all types of biases, namely model class level bias, stereotype bias and domain bias present in LLMs. Model class level bias happens when a model tends to favor some classification labels or outputs compared to the others. We investigate how a classification model hugely favors one class with respect to another. We propose a bias evaluation technique called \\textit{directional pairwise class confusion bias} that highlights an LLM's bias on pairs of classes. Unfavorable kind of stereotype bias takes place when LLMs cause significant injustice or harm to disadvantaged or marginalized group of people. Although the most advanced deep LLMs claim to mimic human responses via powerful and sophisticated algorithms, the capabilities that such models offer have shown to possess bias. Quantifying such stereotype biases appropriately is essential so that the bias measures can be used to calibrate potential harm the models can cause. On the other hand, domain biases are desired for the model because it indicates the model is learning necessary facts for it to be powerful. We devise techniques to measure class level, stereotype, and domain biases appropriately.
"],"dc:identifier":["https://digitalcommons.kennesaw.edu/dataphd_etd/16"],"dc:subject":["Natural Language Processing","Bias evaluation","Bias mitigation","Artificial Intelligence and Robotics","Data Science","Numerical Analysis and Scientific Computing","Risk Analysis","Statistical Models","Statistics and Probability"],"dc:title":["Quantification of Various Types of Biases in Large Language Models"],"thesis:degree_discipline":["Statistics and Analytical Sciences"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Doctor of Philosophy in Analytic and Data Science"]},"updated_at":"2026-07-24T02:44:06Z"}