{"id":{"repo_id":"unitec","oai_identifier":"oai:researchbank.ac.nz:10652/6749"},"canonical_url":"https://search.dev.ndltd.org/etd/unitec/oai:researchbank.ac.nz:10652/6749","repository":{"repo_id":"unitec","name":"Unitec","base_url":"https://www.researchbank.ac.nz/server/oai/request"},"display":{"title":"Comparative research on code vulnerability detection: Open-source vs. proprietary large language models and LSTM neural network","abstract":"The reliance of industries such as banking, e-commerce, logistics, transportation, energy, and healthcare on computer systems has escalated the threat of cyberattacks. With the rapid development of the online ecosystem and the growth of sensitive data, cybercriminals have developed the ability to attack software systems. Safeguarding software has become an urgent yet challenging task despite advancements in technology. The contributions of this thesis show the incorporation of security into the Software Development Lifecycle through a focus on Static Code Analysis as a proactive approach for the discovery and mitigation of security flaws in the development cycle. This research focuses on enhancing vulnerability detection in source code through advanced machine learning techniques, including open-source fine-tuned and proprietary large language models. It compares the models, including CodeGen2, LLaMA 2, and GPT, developed by OpenAI to quantitative and qualitative how suitable they are for detecting vulnerabilities, their accuracy and efficiency. Using a zero-shot classification-based approach, the study examines these models’ capabilities in detecting security risks and compares them to Word2Vec+LSTM neural networks. The findings reveal that CodeGen2 emerges as the most reliable model for vulnerability detection, achieving near-perfect precision and balanced recall, leading to superior F1 and AUC scores. LLaMA2-7b delivers reasonable performance, particularly in precision, but falls short in recall. Conversely, GPT-4 Assistant excels in recall but suffers from a high false positive rate, limiting its effectiveness. Classical neural networks, such as Word2Vec+LSTM, demonstrate moderate capability but lag modern LLMs in precision and recall. Through this comparative analysis, the thesis underscores the importance of selecting tools aligned with organizational needs and constraints. The insights gained contribute to developing secure software by integrating machine learning into the SDLC, inspiring further progress in vulnerability detection and mitigating risks in an increasingly digital world.","abstract_html":"The reliance of industries such as banking, e-commerce, logistics, transportation, energy, and healthcare on computer systems has escalated the threat of cyberattacks. With the rapid development of the online ecosystem and the growth of sensitive data, cybercriminals have developed the ability to attack software systems. Safeguarding software has become an urgent yet challenging task despite advancements in technology. The contributions of this thesis show the incorporation of security into the Software Development Lifecycle through a focus on Static Code Analysis as a proactive approach for the discovery and mitigation of security flaws in the development cycle. This research focuses on enhancing vulnerability detection in source code through advanced machine learning techniques, including open-source fine-tuned and proprietary large language models. It compares the models, including CodeGen2, LLaMA 2, and GPT, developed by OpenAI to quantitative and qualitative how suitable they are for detecting vulnerabilities, their accuracy and efficiency. Using a zero-shot classification-based approach, the study examines these models’ capabilities in detecting security risks and compares them to Word2Vec+LSTM neural networks. The findings reveal that CodeGen2 emerges as the most reliable model for vulnerability detection, achieving near-perfect precision and balanced recall, leading to superior F1 and AUC scores. LLaMA2-7b delivers reasonable performance, particularly in precision, but falls short in recall. Conversely, GPT-4 Assistant excels in recall but suffers from a high false positive rate, limiting its effectiveness. Classical neural networks, such as Word2Vec+LSTM, demonstrate moderate capability but lag modern LLMs in precision and recall. Through this comparative analysis, the thesis underscores the importance of selecting tools aligned with organizational needs and constraints. The insights gained contribute to developing secure software by integrating machine learning into the SDLC, inspiring further progress in vulnerability detection and mitigating risks in an increasingly digital world.","abstract_has_math":false,"creators":["Don, Ravihansa Geekiyanage Geekiyanage"],"institution":"Unitec, Te Pūkenga – New Zealand Institute of Skills and Technology","degree_name":"Master of Applied Technologies (Computing)","degree_level":"Masters","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024","date_published":"2024","updated_at":"2026-07-27T21:45:34Z","subjects":["software development","vulnerability assessment","risk management framework","software security","computer security","Long Short-Term Memory (LSTM)","open source","neural networks","large language models"],"languages":["en"],"rights":["All rights reserved"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/10652/6749","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Don, Ravihansa Geekiyanage Geekiyanage"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-04-02T00:41:25Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-04-02T00:41:25Z"]},{"key":"dc:date.issued","label":"Date","values":["2024"]},{"key":"dc:type","label":"Dc Type","values":["Masters Thesis"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Masters"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Applied Technologies (Computing)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Unitec, Te Pūkenga – New Zealand Institute of Skills and Technology"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["software development","vulnerability assessment","risk management framework","software security","computer security","Long Short-Term Memory (LSTM)","open source","neural networks","large language models"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["All rights reserved"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10652/6749"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The reliance of industries such as banking, e-commerce, logistics, transportation, energy, and healthcare on computer systems has escalated the threat of cyberattacks. With the rapid development of the online ecosystem and the growth of sensitive data, cybercriminals have developed the ability to attack software systems. Safeguarding software has become an urgent yet challenging task despite advancements in technology. The contributions of this thesis show the incorporation of security into the Software Development Lifecycle through a focus on Static Code Analysis as a proactive approach for the discovery and mitigation of security flaws in the development cycle. This research focuses on enhancing vulnerability detection in source code through advanced machine learning techniques, including open-source fine-tuned and proprietary large language models. It compares the models, including CodeGen2, LLaMA 2, and GPT, developed by OpenAI to quantitative and qualitative how suitable they are for detecting vulnerabilities, their accuracy and efficiency. Using a zero-shot classification-based approach, the study examines these models’ capabilities in detecting security risks and compares them to Word2Vec+LSTM neural networks. The findings reveal that CodeGen2 emerges as the most reliable model for vulnerability detection, achieving near-perfect precision and balanced recall, leading to superior F1 and AUC scores. LLaMA2-7b delivers reasonable performance, particularly in precision, but falls short in recall. Conversely, GPT-4 Assistant excels in recall but suffers from a high false positive rate, limiting its effectiveness. Classical neural networks, such as Word2Vec+LSTM, demonstrate moderate capability but lag modern LLMs in precision and recall. Through this comparative analysis, the thesis underscores the importance of selecting tools aligned with organizational needs and constraints. The insights gained contribute to developing secure software by integrating machine learning into the SDLC, inspiring further progress in vulnerability detection and mitigating risks in an increasingly digital world."]},{"key":"dc:title","label":"Title","values":["Comparative research on code vulnerability detection: Open-source vs. proprietary large language models and LSTM neural network"]}]}],"canonical_facts":{"dc:creator":["Don, Ravihansa Geekiyanage Geekiyanage"],"dc:date.accessioned":["2025-04-02T00:41:25Z"],"dc:date.available":["2025-04-02T00:41:25Z"],"dc:date.issued":["2024"],"dc:description.abstract":["The reliance of industries such as banking, e-commerce, logistics, transportation, energy, and healthcare on computer systems has escalated the threat of cyberattacks. With the rapid development of the online ecosystem and the growth of sensitive data, cybercriminals have developed the ability to attack software systems. Safeguarding software has become an urgent yet challenging task despite advancements in technology. The contributions of this thesis show the incorporation of security into the Software Development Lifecycle through a focus on Static Code Analysis as a proactive approach for the discovery and mitigation of security flaws in the development cycle. This research focuses on enhancing vulnerability detection in source code through advanced machine learning techniques, including open-source fine-tuned and proprietary large language models. It compares the models, including CodeGen2, LLaMA 2, and GPT, developed by OpenAI to quantitative and qualitative how suitable they are for detecting vulnerabilities, their accuracy and efficiency. Using a zero-shot classification-based approach, the study examines these models’ capabilities in detecting security risks and compares them to Word2Vec+LSTM neural networks. The findings reveal that CodeGen2 emerges as the most reliable model for vulnerability detection, achieving near-perfect precision and balanced recall, leading to superior F1 and AUC scores. LLaMA2-7b delivers reasonable performance, particularly in precision, but falls short in recall. Conversely, GPT-4 Assistant excels in recall but suffers from a high false positive rate, limiting its effectiveness. Classical neural networks, such as Word2Vec+LSTM, demonstrate moderate capability but lag modern LLMs in precision and recall. Through this comparative analysis, the thesis underscores the importance of selecting tools aligned with organizational needs and constraints. The insights gained contribute to developing secure software by integrating machine learning into the SDLC, inspiring further progress in vulnerability detection and mitigating risks in an increasingly digital world."],"dc:identifier.uri":["https://hdl.handle.net/10652/6749"],"dc:language.iso":["en"],"dc:rights":["All rights reserved"],"dc:subject":["software development","vulnerability assessment","risk management framework","software security","computer security","Long Short-Term Memory (LSTM)","open source","neural networks","large language models"],"dc:title":["Comparative research on code vulnerability detection: Open-source vs. proprietary large language models and LSTM neural network"],"dc:type":["Masters Thesis"],"thesis:degree_level":["Masters"],"thesis:degree_name":["Master of Applied Technologies (Computing)"],"thesis:institution_name":["Unitec, Te Pūkenga – New Zealand Institute of Skills and Technology"]},"updated_at":"2026-07-27T21:45:34Z"}