Skip to content
New

DeepSeek V4.1 Flash free for 7 days: 10M tokens a day after a top-up of $10 or more. Until Nov 30.

How it works
meltedinhex/tuning-detections-to-reduce-false-positivesReporting and compliance

tuning-detections-to-reduce-false-positives

⁨Tunes noisy detections by analyzing historical alert data to quantify false-positive⁩

v1.0.03 installs+3 in 7 days24 StarsApache-2.0synced 10 hours ago

Install

No NotHoly CLI yet? Install it first.

$ curl -fsSL https://notholy.com/install.sh | sh
$ notholy skills install meltedinhex/tuning-detections-to-reduce-false-positives

Writes to ⁦~/.notholy/skills⁩ after checking the signature and the SHA-256.

SignatureSigned, ed25519
SQTFOv+nlY…2cChX4BA==The CLI checks the SHA-256 and the signature before it writes a file.
Package90e69b0d…8e6220eb
90e69b0d1ace…701a8e6220eb4 files · ⁦3.1 KB⁩
ScanMedium2 findings
  • Executable script shipped with the skillscripts/analyst.py
  • Links to outside domains
See the full scan
Sourcegithub.com/meltedinhex/analyst-ai-pack/tree/main/skills/tuning-detections-to-reduce-false-positives@da7600a

When to Use

  • A detection generates excessive false positives and you want a data-driven tuning plan: which fields/values drive the noise, what filters to add, and the precision impact.
  • You are balancing recall against alert fatigue.

Do not use tuning that suppresses true positives — filters must target benign noise, not the malicious pattern. Always estimate the impact on true positives before applying.

Prerequisites

  • Labeled alert history (CSV/JSON) with at least an outcome/label field (true_positive vs false_positive) and the alert's distinguishing fields.

Workflow

Step 1: Quantify false-positive sources

bash
python scripts/analyst.py analyze alerts.csv --label verdict --field ParentImage

Ranks field values by how much false-positive volume they drive and computes current precision/false-positive rate.

Step 2: Recommend filters

Suggest allow-list filters for values that are overwhelmingly benign (high FP, no/low TP), with the projected precision gain.

Step 3: Estimate impact

Report how many true positives would be lost (should be zero/near-zero for safe filters).

Step 4: Apply and monitor

Add the safe filters to the rule's filter/exclusion and continue monitoring precision.

Validation

  • FP-driving values are ranked by their false-positive contribution.
  • Recommended filters target values with negligible true-positive loss.
  • Projected precision and TP-loss are reported.

Pitfalls

  • Filtering a value that also carries true positives, blinding the detection.
  • Over-fitting to one time window's noise.
  • Tuning by gut feel instead of measured FP/TP contribution.

References

  • See references/api-reference.md for the tuner.
  • Sigma spec and detection-engineering references (linked in frontmatter).

Files in this version

Permissions

PermissionDeclaredFound by the scan
Reads filesNoNo
Writes filesNoNo
Runs shell commandsYesYes
Uses the networkYesYes
Uses connectorsNoNo
Drives a browserNoNo

Scan findings

Reviewed on Nov 13, 2026

  1. MediumFile · Executable script shipped with the skillscripts/analyst.py
  2. InfoLink · Links to outside domains

A clean scan lowers the risk but does not prove a package is safe. Read what you install.

VersionPublishedCommitSHA-256SignaturePermission change
v1.0.0Nov 13, 2026da7600a90e69b0dSigned–

Ask anything your scope allows.

Create an account, add credits and ask the first question in the chat or the terminal.

  1. 1/3

    Create the account

    Individual or company. You can switch to a company later without losing balance or keys.

    IndividualCompany
  2. 2/3

    Add credits

    By card. Pay per token, no subscription.

    $5.00minimum deposit
  3. 3/3

    Ask

    In the chat, or install the CLI and run notholy login in your terminal.

    $ notholy login