tuning-detections-to-reduce-false-positives
Tunes noisy detections by analyzing historical alert data to quantify false-positive
v1.0.03 installs+3 in 7 days24 StarsApache-2.0synced 10 hours ago
Install
No NotHoly CLI yet? Install it first.
$ curl -fsSL https://notholy.com/install.sh | sh$ notholy skills install meltedinhex/tuning-detections-to-reduce-false-positivesWrites to ~/.notholy/skills after checking the signature and the SHA-256.
SignatureSigned, ed25519
Package90e69b0d…8e6220eb
90e69b0d1ace…701a8e6220eb4 files · 3.1 KBScanMedium2 findings
- Executable script shipped with the skillscripts/analyst.py
- Links to outside domains
Sourcegithub.com/meltedinhex/analyst-ai-pack/tree/main/skills/tuning-detections-to-reduce-false-positives@da7600a
On this page
When to Use
- A detection generates excessive false positives and you want a data-driven tuning plan: which fields/values drive the noise, what filters to add, and the precision impact.
- You are balancing recall against alert fatigue.
Do not use tuning that suppresses true positives — filters must target benign noise, not the malicious pattern. Always estimate the impact on true positives before applying.
Prerequisites
- Labeled alert history (CSV/JSON) with at least an outcome/label field (true_positive vs false_positive) and the alert's distinguishing fields.
Workflow
Step 1: Quantify false-positive sources
python scripts/analyst.py analyze alerts.csv --label verdict --field ParentImageRanks field values by how much false-positive volume they drive and computes current precision/false-positive rate.
Step 2: Recommend filters
Suggest allow-list filters for values that are overwhelmingly benign (high FP, no/low TP), with the projected precision gain.
Step 3: Estimate impact
Report how many true positives would be lost (should be zero/near-zero for safe filters).
Step 4: Apply and monitor
Add the safe filters to the rule's filter/exclusion and continue monitoring precision.
Validation
- FP-driving values are ranked by their false-positive contribution.
- Recommended filters target values with negligible true-positive loss.
- Projected precision and TP-loss are reported.
Pitfalls
- Filtering a value that also carries true positives, blinding the detection.
- Over-fitting to one time window's noise.
- Tuning by gut feel instead of measured FP/TP contribution.
References
- See
references/api-reference.mdfor the tuner. - Sigma spec and detection-engineering references (linked in frontmatter).
Files in this version
references/11.3 KB
- api-reference.md1.3 KB
scripts/13.1 KB
- analyst.py3.1 KB
- LICENSE340 B
- SKILL.md2.7 KB
Permissions
| Permission | Declared | Found by the scan |
|---|---|---|
| Reads files | No | No |
| Writes files | No | No |
| Runs shell commands | Yes | Yes |
| Uses the network | Yes | Yes |
| Uses connectors | No | No |
| Drives a browser | No | No |
Scan findings
Reviewed on Nov 13, 2026
- MediumFile · Executable script shipped with the skillscripts/analyst.py
- InfoLink · Links to outside domains
A clean scan lowers the risk but does not prove a package is safe. Read what you install.
| Version | Published | Commit | SHA-256 | Signature | Permission change |
|---|---|---|---|---|---|
| v1.0.0 | Nov 13, 2026 | da7600a | 90e69b0d | Signed | – |
More from meltedinhex
All from this publisher- writing-yara-rules-from-reversed-codemeltedinhex/writing-yara-rules-from-reversed-code4
- validating-detections-with-atomic-testsmeltedinhex/validating-detections-with-atomic-tests4
- writing-suricata-and-snort-rulesmeltedinhex/writing-suricata-and-snort-rules3
- writing-sigma-detection-rulesmeltedinhex/writing-sigma-detection-rules3
More in Reporting and compliance
See the category- skillduty1g/skill5
- rea-ci-tests-auditmorluto/rea-ci-tests-audit4
- tracking-threat-actor-infrastructuremukul975/tracking-threat-actor-infrastructure4
- performing-authenticated-scan-with-openvasmukul975/performing-authenticated-scan-with-openvas4
- slicing-code-contexttrailofbits/slicing-code-context4
- 1337aeondave/13374
Ask anything your scope allows.
Create an account, add credits and ask the first question in the chat or the terminal.
- 1/3
Create the account
Individual or company. You can switch to a company later without losing balance or keys.
IndividualCompany - 2/3
Add credits
By card. Pay per token, no subscription.
$5.00minimum deposit - 3/3
Ask
In the chat, or install the CLI and run notholy login in your terminal.
$ notholy login