{"id":{"repo_id":"queens","oai_identifier":"oai:queensu.scholaris.ca:1974/29913"},"canonical_url":"https://search.dev.ndltd.org/etd/queens/oai:queensu.scholaris.ca:1974/29913","repository":{"repo_id":"queens","name":"Queens University","base_url":"https://qspace.library.queensu.ca/server/oai/request"},"display":{"title":"NeuroYara: Learning to Rank for Yara Rules Generation through Deep Language Modeling & Discriminative N-gram Encoding","abstract":"Signature-based malware detection methods are simple, explainable, and efficient. One of the most ubiquitous tools is Yara. It is a widely-used syntax for writing malware signatures. Compared to machine learning models, Yara rules have a lower false-positive rate and better maintainability of the rules to incorporate new variants of malware. To produce a high-quality rule, one will need extensive experience in reverse engineering and malware analysis. However, it is resource- and time-consuming to train an experienced malware analyst. Only a few works have been conducted to automate the generation of high-quality signatures and generally perform worse than manually-generated ones. Moreover, they rely on a huge static and non-inclusive database of hard-coded byte n-grams. This database is used as the reference set for the automated Yara rules generator which aids in reducing the number of false-positive predictions. Hence, instead of storing a huge non-inclusive database to score byte n-grams, we propose a novel architecture utilizing two learning-to-rank neural networks to understand the underlying effectiveness of and correlations among n-grams extracted for rule construction. This approach provides better flexibility and coverage of possible n-grams while reducing the required storage size for this task from several GBs to only 10MBs. Combining these two models with a hierarchical density-based clustering method allows us to group multiple n-grams into logical conditions as Yara rules of a higher quality. Compared to state-of-the-art tools for automatic Yara rules generation, our experimental results show that our framework, NeuroYara, reduces the resources invested by the human analyst while generating rules with a low false-positive rate outperforming existing tools and rules manually generated by expert malware analysts.","abstract_html":"Signature-based malware detection methods are simple, explainable, and efficient. One of the most ubiquitous tools is Yara. It is a widely-used syntax for writing malware signatures. Compared to machine learning models, Yara rules have a lower false-positive rate and better maintainability of the rules to incorporate new variants of malware. To produce a high-quality rule, one will need extensive experience in reverse engineering and malware analysis. However, it is resource- and time-consuming to train an experienced malware analyst. Only a few works have been conducted to automate the generation of high-quality signatures and generally perform worse than manually-generated ones. Moreover, they rely on a huge static and non-inclusive database of hard-coded byte n-grams. This database is used as the reference set for the automated Yara rules generator which aids in reducing the number of false-positive predictions. Hence, instead of storing a huge non-inclusive database to score byte n-grams, we propose a novel architecture utilizing two learning-to-rank neural networks to understand the underlying effectiveness of and correlations among n-grams extracted for rule construction. This approach provides better flexibility and coverage of possible n-grams while reducing the required storage size for this task from several GBs to only 10MBs. Combining these two models with a hierarchical density-based clustering method allows us to group multiple n-grams into logical conditions as Yara rules of a higher quality. Compared to state-of-the-art tools for automatic Yara rules generation, our experimental results show that our framework, NeuroYara, reduces the resources invested by the human analyst while generating rules with a low false-positive rate outperforming existing tools and rules manually generated by expert malware analysts.","abstract_has_math":false,"creators":["Mansour, Ziad"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Computing","school":null,"contributors":[],"advisors":["Ding, Steven"],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-07-27T20:35:33Z","subjects":["Deep Learning","Malware Analysis","Malware Detection","Yara Rules","Discriminative Encoding","Language Modeling","Automatic Signatures Generation","Adversarial Malware"],"languages":["eng"],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/1974/29913","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.department","label":"Department","values":["Computing"]},{"key":"dc:contributor.supervisor","label":"Supervisor","values":["Ding, Steven"]},{"key":"dc:creator","label":"Author","values":["Mansour, Ziad"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-01-28T16:58:20Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-01-28T16:58:20Z"]},{"key":"dc:type","label":"Dc Type","values":["thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Deep Learning","Malware Analysis","Malware Detection","Yara Rules","Discriminative Encoding","Language Modeling","Automatic Signatures Generation","Adversarial Malware"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/1974/29913"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Signature-based malware detection methods are simple, explainable, and efficient. One of the most ubiquitous tools is Yara. It is a widely-used syntax for writing malware signatures. Compared to machine learning models, Yara rules have a lower false-positive rate and better maintainability of the rules to incorporate new variants of malware. To produce a high-quality rule, one will need extensive experience in reverse engineering and malware analysis. However, it is resource- and time-consuming to train an experienced malware analyst. Only a few works have been conducted to automate the generation of high-quality signatures and generally perform worse than manually-generated ones. Moreover, they rely on a huge static and non-inclusive database of hard-coded byte n-grams. This database is used as the reference set for the automated Yara rules generator which aids in reducing the number of false-positive predictions. Hence, instead of storing a huge non-inclusive database to score byte n-grams, we propose a novel architecture utilizing two learning-to-rank neural networks to understand the underlying effectiveness of and correlations among n-grams extracted for rule construction. This approach provides better flexibility and coverage of possible n-grams while reducing the required storage size for this task from several GBs to only 10MBs. Combining these two models with a hierarchical density-based clustering method allows us to group multiple n-grams into logical conditions as Yara rules of a higher quality. Compared to state-of-the-art tools for automatic Yara rules generation, our experimental results show that our framework, NeuroYara, reduces the resources invested by the human analyst while generating rules with a low false-positive rate outperforming existing tools and rules manually generated by expert malware analysts."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["M.Sc."]},{"key":"dc:title","label":"Title","values":["NeuroYara: Learning to Rank for Yara Rules Generation through Deep Language Modeling & Discriminative N-gram Encoding"]}]}],"canonical_facts":{"dc:contributor.department":["Computing"],"dc:contributor.supervisor":["Ding, Steven"],"dc:creator":["Mansour, Ziad"],"dc:date.accessioned":["2022-01-28T16:58:20Z"],"dc:date.available":["2022-01-28T16:58:20Z"],"dc:description.abstract":["Signature-based malware detection methods are simple, explainable, and efficient. One of the most ubiquitous tools is Yara. It is a widely-used syntax for writing malware signatures. Compared to machine learning models, Yara rules have a lower false-positive rate and better maintainability of the rules to incorporate new variants of malware. To produce a high-quality rule, one will need extensive experience in reverse engineering and malware analysis. However, it is resource- and time-consuming to train an experienced malware analyst. Only a few works have been conducted to automate the generation of high-quality signatures and generally perform worse than manually-generated ones. Moreover, they rely on a huge static and non-inclusive database of hard-coded byte n-grams. This database is used as the reference set for the automated Yara rules generator which aids in reducing the number of false-positive predictions. Hence, instead of storing a huge non-inclusive database to score byte n-grams, we propose a novel architecture utilizing two learning-to-rank neural networks to understand the underlying effectiveness of and correlations among n-grams extracted for rule construction. This approach provides better flexibility and coverage of possible n-grams while reducing the required storage size for this task from several GBs to only 10MBs. Combining these two models with a hierarchical density-based clustering method allows us to group multiple n-grams into logical conditions as Yara rules of a higher quality. Compared to state-of-the-art tools for automatic Yara rules generation, our experimental results show that our framework, NeuroYara, reduces the resources invested by the human analyst while generating rules with a low false-positive rate outperforming existing tools and rules manually generated by expert malware analysts."],"dc:description.degree":["M.Sc."],"dc:identifier.uri":["http://hdl.handle.net/1974/29913"],"dc:language.iso":["eng"],"dc:subject":["Deep Learning","Malware Analysis","Malware Detection","Yara Rules","Discriminative Encoding","Language Modeling","Automatic Signatures Generation","Adversarial Malware"],"dc:title":["NeuroYara: Learning to Rank for Yara Rules Generation through Deep Language Modeling & Discriminative N-gram Encoding"],"dc:type":["thesis"]},"updated_at":"2026-07-27T20:35:33Z"}