Write a YARA Rule Against a Benign Test File

#What a YARA rule does

YARA is a pattern matching tool. You write a rule that describes what a file looks like, and YARA lists the files that fit the description. Malware analysts use it most. They write rules for strings, file layouts and byte sequences, then scan folders, disk images and memory dumps. The same mechanism works on any file you own, for example a template that must never carry an internal marker, or a config file that must never contain a token.

YARA has no idea what is dangerous. It matches what the rule says. A rule is a claim about bytes, and the rule is only as good as that claim.

The project is hosted on GitHub under VirusTotal. The official writing guide on Read the Docs is the reference for syntax. VirusTotal also maintains YARA-X, a rewrite in Rust that keeps most of the rule language. The two overlap but are not identical, so read the compatibility notes before you move a rule from one to the other.

#Why rules matter

A file hash answers one narrow question: is this exact file already known? Change one byte and the hash changes. The file hashes article covers that in detail. A YARA rule asks a wider question: does this file contain the pieces we expect? One rule can match many files that share a string, a structure or a byte pattern, even when no two of them share a hash.

The wider question is also where the trouble starts. A hash almost never matches a file it should not. A careless rule can match half of a file server. So the skill to learn is less about syntax and more about writing a rule that matches what you want and nothing else, then proving it with files you control.

#Anatomy of a rule

Every rule has a name, then up to three sections inside braces:

rule Example_Rule {
  meta:
    description = "What this rule is for"

  strings:
    $text = "some text"

  condition:
    $text
}

Rule names use letters, digits and underscores, and cannot start with a digit. The name appears in every match report, so pick one that tells a reader what the rule is about.

#meta

Meta holds key-value pairs for people: a description, an author, a date, a link to the research behind the rule. YARA never uses meta to decide a match. Keep it accurate anyway, because the next person to open the file will trust it.

#strings

Each string gets an identifier that starts with $. The condition refers to strings by that identifier. There are three kinds.

Text strings go in double quotes:

$marker = "LAB-TRAINING-MARKER"

Modifiers change how the text is matched. nocase ignores case. wide matches UTF-16LE text, where each ASCII character is followed by a zero byte, which is common in Windows binaries. fullword requires the match to stand alone, not inside a longer word.

Hex strings go in braces and match raw bytes:

$mz = { 4D 5A }

Those two bytes are the MZ signature at the start of many Windows executables. ?? matches any single byte, [2-4] allows a gap of two to four unknown bytes, and ( 90 | 91 ) matches either byte.

Regular expressions go in slashes:

$version = /LAB-[A-Z]+ v[0-9]+/

Regex is the most flexible kind and the easiest to make too broad. Use a plain text or hex string whenever one can express the pattern.

#condition

The condition is a boolean expression. It combines strings and file properties into one answer: match or no match.

• $marker is true when the string appears at least once.
• any of them and all of them cover every string in the rule. 2 of ($a, $b, $c) requires two of three.
• $marker at 0 requires the string at the first byte. $marker in (0..512) requires it somewhere in the first 512 bytes.
• #marker > 3 counts occurrences.
• filesize < 2KB checks the file size.
• uint16(0) == 0x5A4D reads the first two bytes as a little-endian integer and checks for the MZ signature.

Every rule needs a condition. A rule with strings and no sensible condition is either broken or useless.

#How YARA reads a rule

1. YARA parses the rule file and compiles it. A syntax error stops the run before any file is read.
2. For each file it scans, YARA looks for every string in the strings section.
3. It evaluates the condition using the results from step 2.
4. If the condition is true, the file matches. With -s, YARA also prints each string that matched, with the byte offset where it was found.

The strings come first and the condition comes second. A condition such as $a and $b is strict, because both strings must be found before the rule can match.

#Lab: one positive file and one negative file

The expected output below is described rather than pasted, because exact formatting can differ between YARA versions. Compare what you see with those descriptions.

Work in an empty folder. On Windows, run the commands in Git Bash. The heredoc in step 3 does not work the same way in PowerShell.

#1. Install yara

On Debian and Ubuntu:

sudo apt install yara

On macOS with Homebrew:

brew install yara

On Windows, download the release zip from https://github.com/VirusTotal/yara/releases, extract it and put the executable on your PATH. The file may be called yara64.exe. Rename it to yara.exe, or copy it under that name, so the commands below work as written.

Check the install:

yara --version

You should see a version number. This article does not pin one, so use the latest release your install method gives you.

#2. Create a positive file and a negative file

mkdir yara-lab && cd yara-lab
printf 'LAB-TRAINING-MARKER v1\n' > match.txt
printf 'ordinary text file\n' > nomatch.txt

Both files are plain text you wrote yourself. The first should match the rule. The second should not.

#3. Write the rule

The quotes around EOF matter. Without them, the shell replaces $m with an empty variable before YARA sees the file, and the rule no longer compiles.

cat > lab.yar <<'EOF'
rule Lab_Training_Marker {
  meta: description = "Matches the lab training marker"
  strings: $m = "LAB-TRAINING-MARKER"
  condition: $m
}
EOF

Read it as three parts. The meta says what the rule is for. The string says the file must contain the marker text. The condition says a file matches when that string is found at least once.

#4. Run the positive test

yara -s lab.yar match.txt

YARA should print the rule name and the file path. Because of -s, it also prints a line for the matched string, with the byte offset and the text it found. The marker starts at the first byte of match.txt, so the offset should be 0x0.

#5. Run the negative test

yara lab.yar nomatch.txt

Expected output: nothing. YARA prints matches only, so silence means the file did not match.

Silence only means something if the positive test worked. Run step 4 first. If step 4 printed nothing, the rule or the install is wrong, and the silence in step 5 tells you nothing.

#6. Add a near miss

A near miss is the most useful negative file, because it fails for one narrow reason.

printf 'LAB-TRAINING\n' > near.txt
yara lab.yar near.txt

Expected output: nothing. The file has the start of the marker but not the full string, so the string is not found and the condition is false.

#7. Scan the whole folder

yara -r lab.yar .

The -r flag scans the folder recursively. Expect match.txt in the output. You will probably also see lab.yar, the rule file itself. The rule file contains the marker text, so the rule matches its own source. This is a common mistake, and the next section covers it. Keep rules in a different folder from the data you scan.

#Make the rule stricter

The lab rule is fine for a training marker. Real rules usually need a constraint that the string alone does not give you. Suppose you only want small files that carry the marker:

rule Lab_Training_Marker_Small {
  strings: $m = "LAB-TRAINING-MARKER"
  condition: $m and filesize < 1KB
}

Test it the same way. match.txt should match. nomatch.txt and near.txt should not. Every change to the strings or the condition is a new rule as far as testing goes, so run both test files again.

#Avoid rules that match everything

This rule looks reasonable:

rule Too_Broad {
  strings: $a = "text"
  condition: $a
}

It will match nomatch.txt, because the sentence ordinary text file contains the word text. You can check that with the negative file you already made. A rule that matches it has failed its own test.

Patterns that match too much:

• Short strings, or common words such as text, error, copyright or this program.
• A one-byte or very short hex pattern, such as { 00 } or { 90 }.
• nocase on a common word.
• any of them over a list of strings that each appear in ordinary files.
• A condition that is only a size check, such as filesize < 10KB, which matches most small files.
• A loose regex, such as /.+/.

Ways to tighten a rule:

• Pick longer, distinctive strings. Ask whether the string would show up in a document your organization writes every day. If it would, choose another one.
• Require more than one string with and, or with 2 of (...), so no single common string decides the result.
• Anchor the search with at or in when you know where the string should appear.
• Combine a content test with a header or size check, such as uint16(0) == 0x5A4D.

#Common mistakes

1. Testing only the positive file. A rule that matches the file you built it for proves little. It may match everything.
2. Skipping near misses. Ordinary files rarely fail in an interesting way. A file that almost matches is the test that catches an overly loose string.
3. Leaving the quotes off the heredoc. The shell replaces $m before YARA reads the file. See step 3.
4. Scanning the folder with the rules in it. The rule file matches its own text.
5. Trusting meta. A description that says "detects X" does not detect X. Only test files show that.
6. Editing without retesting. Any change to the strings or the condition can change the result. Rerun both tests after each edit.

#Positive and negative tests before anyone deploys a rule

This is the rule that matters most in this article. Every YARA rule needs both kinds of test file before anyone deploys it.

• Positive files are files the rule must match. In the lab, that is match.txt. In real work, they come from samples you are authorized to hold, under your organization's handling rules.
• Negative files are files the rule must not match. Include ordinary documents, the file types common in your environment, clean files from the systems you plan to scan, and near misses like near.txt.

A rule passes when it matches every positive file and no negative file. Keep both sets next to the rule, in version control if your team uses it, and rerun them after every edit. A rule that passed the positive test but was never run against negatives has not been tested.

#How to go further

• Read the writing rules guide from start to finish. It covers modules such as pe, which lets a rule test the headers and sections of a Windows executable instead of raw bytes.
• Run the same lab rule with YARA-X and compare the results. The YARA-X documentation lists what is and is not compatible.
• Build a clean corpus: a folder of ordinary files from machines you control. Every new rule must pass it before it goes anywhere.
• Read the SIEM article next. It covers the other side of detection: rules that run over logs instead of files.

#Sources

• YARA writing rules guide: https://yara.readthedocs.io/en/stable/writingrules.html
• YARA source code and releases: https://github.com/VirusTotal/yara
• YARA-X documentation: https://virustotal.github.io/yara-x/