{"id":695,"date":"2022-11-30T20:50:16","date_gmt":"2022-11-30T20:50:16","guid":{"rendered":"https:\/\/practicalsecurityanalytics.com\/?page_id=695"},"modified":"2022-12-01T15:38:36","modified_gmt":"2022-12-01T15:38:36","slug":"yaratools","status":"publish","type":"page","link":"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/","title":{"rendered":"YaraTools"},"content":{"rendered":"\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Overview\" >Overview<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Features\" >Features<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Downloads\" >Downloads<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Background\" >Background<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Output\" >Output<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Usage\" >Usage<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Scanning_a_Single_File\" >Scanning a Single File<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/practicalsecurityanalytics.com\/tools\/yaratools\/#Scanning_Multiple_Files\" >Scanning Multiple Files<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Overview\"><\/span>Overview<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>This tool houses a large set of open-source YARA signatures that have been evaluated on a set of 284,181 legitimate and malicious portable executable files. The Get-YaraMatches PowerShell script can be used to scan new files and enrich the results with additional information such as information gain and the source text for the matching signature. This gives users more information to determine if a file is legitimate or malicious.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Features\"><\/span>Features<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Collection over over 100K open-source YARA signatures.<\/li>\n\n\n\n<li>PowerShell script to automatically compile signatures and scan files.<\/li>\n\n\n\n<li>Data enrichment statistics that provide insight into true and false positive rates for each signature.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Downloads\"><\/span>Downloads<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p><strong>Github:<\/strong> <a href=\"https:\/\/github.com\/pracsec\/YaraTools\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/github.com\/pracsec\/YaraTools<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Background\"><\/span>Background<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The YARA signatures that are bundled with this tool were collected from two primary sources: (1) conversion of ClamAV signatures to YARA and (2) open source signatures primarily found on Github. These were combined into several source files, duplicates were removed, and poor performing signatures were removed resulting in a set of 108,061YARA rules.<\/p>\n\n\n\n<p>These rules were then evaluated against a dataset of 284,181 legitimate and malicious files found here (<a href=\"https:\/\/practicalsecurityanalytics.com\/pe-malware-machine-learning-dataset\/\" data-type=\"post\" data-id=\"535\">PE Malware Machine Learning Dataset<\/a>). At the time of this writing, the dataset consisted of 104,621 legitimate PE files and 179,650 malicious PE files. The amount of legitimate and malicious binaries a signature fired on was counted for each YARA signature. From there, we could then determine information gain as a useful metric for evaluating the performance of each YARA rule.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Output\"><\/span>Output<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The Get-YaraMatches cmdlet will output a single object for each matching YARA signature that has the following fields:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Field Name<\/th><th>Description<\/th><\/tr><\/thead><tbody><tr><td>File<\/td><td>The full path to the file that was scanned.<\/td><\/tr><tr><td>RuleName<\/td><td>The name of the YARA rule that matched.<\/td><\/tr><tr><td>Ruleset<\/td><td>The name of the YARA file containing the rule that matched.<\/td><\/tr><tr><td>Rule<\/td><td>The source text of the YARA rule that matched.<\/td><\/tr><tr><td>Whitelist<\/td><td>The percentage of legitimate PE files that this signature matched.<\/td><\/tr><tr><td>Blacklist<\/td><td>The percentage of malicious PE files that this signature matched.<\/td><\/tr><tr><td>InfoGain<\/td><td>The information gain. This metric can be used to evaluate the performance of a signature as a discriminator between legitimate and malicious files. The higher the value, the more information this signature firing gives you.<\/td><\/tr><\/tbody><\/table><figcaption class=\"wp-element-caption\">Descriptions of each field in the output object.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Usage\"><\/span>Usage<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Scanning_a_Single_File\"><\/span>Scanning a Single File<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>This example shows how to use the Get-YaraMatches PowerShell cmdlet on a sample of NoPetya. This example assumes you are in the YaraTools directory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"powershell\" class=\"language-powershell\">#Import the PowerShell script from the powershell directory.\n. .\\powershell\\Get-YaraMatches.ps1\n#Scan the specified file using the Get-YaraMatches cmdlet.\n#This will automatically compile the YARA rules if they are not already compiled.\n$results = Get-YaraMatches -File \"C:\\malware\\027cc450ef5f8c5f653329641ec1fed91f694e0d229928963b30f6b0d7d3a745.bin\"\n#Print the results\n$results | Select Ruleset,RuleName,Whitelist,Blacklist,InfoGain | ft -a<\/code><\/pre>\n\n\n\n<p>You should get output similar to the following:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"adoc\" class=\"language-adoc\">Ruleset      RuleName                                     Whitelist   Blacklist   InfoGain\n-------      --------                                     ---------   ---------   --------\nAPT Set 1    Str_Win32_Winsock2_Library                   0.093833934 0.23927378  0.025520794\nAPT Set 1    DoublePulsarXor_Petya                        0           5.56917E-06 2.33072E-06\nAPT Set 1    DoublePulsarDllInjection_Petya               0           5.56917E-06 2.33072E-06\nAPT Set 1    ransomware_PetrWrap                          0           5.56917E-06 2.33072E-06\nAPT Set 1    FE_CPE_MS17_010_RANSOMWARE                   0           5.56917E-06 2.33072E-06\nAPT Set 1    petya_eternalblue                            0           5.56917E-06 2.33072E-06\nCapabilities escalate_priv                                0.106307529 0.17767877  0.006953452\nCapabilities cred_local                                   0.010131809 0.04436957  0.007570034\nCapabilities win_token                                    0.196012273 0.219993317 0.000583858\nCapabilities win_files_operation                          0.335267298 0.504912007 0.01984064\nCrypto       CRC32_poly_Constant                          0.106489137 0.223229004 0.016504966\nCrypto       CRC32_table                                  0.058993892 0.082930497 0.001445101\nOpen Source  IsPeFile                                     0           0           0\nOpen Source  sysinternals_not_signed                      0.000946273 0.000517933 4.38599E-05\nOpen Source  Generic_bitmask_table__32_lil_128_           0.003192476 0.000562486 0.000734221\nOpen Source  Windows_CryptAcquireContext__8_byt_STR_21_   0.012970627 0.008387169 0.000342743\nOpen Source  bitmask__32_lil_128_                         0.003154242 0.000562486 0.000719289\nOpen Source  Windows_CryptImportKey__8_byt_STR_15_        0.009424494 0.010091334 7.70059E-06\nOpen Source  PEiD_00071_Anti007____NsPacK_Private_        0.011947888 0.034924259 0.003873335\nOpen Source  PEiD_02191_tElock_0_99___1_0_private____tE__ 0.050028197 0.064435286 0.000640483\nOpen Source  misc_pe_signature                            0           0           0\nOpen Source  RansomImportDetect                           0           0           0\nOpen Source  DebuggerTiming__Ticks                        0.302300685 0.205613722 0.008413969\nOpen Source  research_pe_signed_outside_timestamp         0           0           0\nOpen Source  create_process                               0.195945365 0.225662731 0.000887301\nOpen Source  Win32_Ransomware_NotPetya                    0           5.56917E-06 2.33072E-06\nOpen Source  BadRabbit_Gen                                0           1.11383E-05 4.66145E-06\nOpen Source  NotPetya_Ransomware_Jun17                    0           5.56917E-06 2.33072E-06\nOpen Source  VBox_Detection                               0.00571587  0.006939185 3.96658E-05\nOpen Source  IsPE32                                       0           0           0\nOpen Source  IsDLL                                        0           0           0\nOpen Source  IsConsole                                    0           0           0\nOpen Source  IsPacked                                     0           0           0\nOpen Source  HasOverlay                                   0           0           0\nOpen Source  HasDigitalSignature                          0.250810067 0.082295611 0.037138875\nOpen Source  HasRichSignature                             0.361533535 0.304449766 0.002473488\nOpen Source  DLL_inject                                   0.094436107 0.134790599 0.002669484\nPEID         Microsoft_Visual_Cpp_v50v60_MFC              0.071352788 0.242531744 0.037452384<\/code><\/pre>\n\n\n\n<p>You can inspect individual matches with the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"powershell\" class=\"language-powershell\">$results[1] | fl *<\/code><\/pre>\n\n\n\n<p>You should get output similar to the following:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"adoc\" class=\"language-adoc\">Blacklist : 5.56917E-06\nRule      : rule DoublePulsarXor_Petya\n            {\n             meta:\n               description = \"Rule to hit on the XORed DoublePulsar shellcode\"\n               author = \"Patrick Jones\"\n               company = \"Booz Allen Hamilton\"\n               reference1 =\"https:\/\/www.boozallen.com\/s\/insight\/publication\/the-petya-ransomware-outbreak.html\"\n               reference2 = \"https:\/\/www.boozallen.com\/content\/dam\/boozallen_site\/sig\/pdf\/white-paper\/rollup-of-booz-allen-petya-research.pdf\"\n               date = \"2017-06-28\"\n               hash = \"027cc450ef5f8c5f653329641ec1fed91f694e0d229928963b30f6b0d7d3a745\"\n               hash = \"64b0b58a2c030c77fdb2b537b2fcc4af432bc55ffb36599a31d418c7c69e94b1\"\n             strings:\n               $DoublePulsarXor_Petya = { FD 0C 8C 5C B8 C4 24 C5 CC CC CC 0E E8 CC 24 6B CC CC CC 0F 24 CD CC CC CC 27 5C 97 75 BA CD CC CC C3 FE }\n             condition:\n               $DoublePulsarXor_Petya\n            }\nRuleset   : APT Set 1\nInfoGain  : 2.33072E-06\nRuleName  : DoublePulsarXor_Petya\nWhitelist : 0\nFile      : C:\\Users\\helpdesk\\Desktop\\Workspace\\malware\\027cc450ef5f8c5f653329641ec1fed91f694e0d229928963b30f6b0d7d3a745.bin<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Scanning_Multiple_Files\"><\/span>Scanning Multiple Files<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>This example shows how you can use the Get-ChildItem to select multiple files to scan with Get-YaraMatches.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"powershell\" class=\"language-powershell\">$results = gci C:\\Windows\\ -Filter \"*.exe\" -File | Get-YaraMatches<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Overview This tool houses a large set of open-source YARA signatures that have been evaluated on a set of 284,181 legitimate and malicious portable executable files. The Get-YaraMatches PowerShell script can be used to scan new files and enrich the results with additional information such as information gain and the source text for the matching [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":662,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":{"advgb_blocks_editor_width":"","advgb_blocks_columns_visual_guide":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"jetpack_post_was_ever_published":false,"footnotes":""},"class_list":["post-695","page","type-page","status-publish","hentry"],"coauthors":[],"author_meta":{"author_link":"https:\/\/practicalsecurityanalytics.com\/author\/michael-lester-main\/","display_name":"pracsec"},"relative_dates":{"created":"Posted 3 years ago","modified":"Updated 3 years ago"},"absolute_dates":{"created":"Posted on November 30, 2022","modified":"Updated on December 1, 2022"},"absolute_dates_time":{"created":"Posted on November 30, 2022 8:50 pm","modified":"Updated on December 1, 2022 3:38 pm"},"featured_img_caption":"","featured_img":false,"series_order":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/PbnFRW-bd","_links":{"self":[{"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/pages\/695","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/comments?post=695"}],"version-history":[{"count":5,"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/pages\/695\/revisions"}],"predecessor-version":[{"id":706,"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/pages\/695\/revisions\/706"}],"up":[{"embeddable":true,"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/pages\/662"}],"wp:attachment":[{"href":"https:\/\/practicalsecurityanalytics.com\/wp-json\/wp\/v2\/media?parent=695"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}