Dealing with False Positives in Gadriel Code

Every static analyzer produces false positives (FPs). What distinguishes Gadriel Code is that false-positive handling is a first-class part of the scoring model, not a bolt-on suppression list. A finding is never simply "on" or "off": it carries an explicit confidence tier, an effective-risk band that discounts severity by that confidence, a provenance record for its taint source, and a domain that decides whether it participates in the CI gate at all. This page explains each of those layers, how they combine, and the concrete commands you use to triage and dismiss a finding.

The governing principle behind all of it:

1. Confidence tiers

Every finding carries a confidence_tier of High, Medium, or Low. Confidence answers a different question from severity:

• Severity = how bad is this if it is real? (Critical … Info)
• Confidence = how sure is the engine that it is real? (High / Medium / Low)

Confidence is derived deterministically from the evidence the rule actually had — not from an LLM, not from a guess. The derivation (crates/code-security/gadriel-finding-types/src/confidence.rs, derive_confidence_tier) maps (scan_type, pattern_type, gates) to a tier with no I/O, so the same finding always gets the same tier.

1.1 How the tier is derived from evidence

The stronger the corroborating evidence a rule matched on, the higher the tier. A summary of the SAST derivation (the fullest table lives in confidence.rs):

Other scanners calibrate the same way:

Two bounded adjustments can move a tier by one band only, never more:

• High-signal floor (apply_high_signal_floor): some rules match a code fact that is the vulnerability — jwt.decode(..., verify=False), yaml.load RCE, md5 for hashing, random.choice for a token. These are structurally ast_call with no import gate, so the table above would band them Low and drop a confirmed-real bug out of the gate. A rule opts in via high_signal = true, which raises a structural Low → Medium (never higher). This is a floor, not an override.
• Per-rule calibration (derive_confidence_tier_calibrated): offline benchmark data can promote (Low→Medium, at n ≥ 20 fires with a Wilson-lower TP-rate ≥ 0.85) or demote (→Low, when Wilson-lower FP-rate ≥ 0.50) a rule within the same one-band clamp.

1.2 Low-confidence SAST findings are advisory by default

This is the single most important FP rule to understand:

Why: across the Round-2 50-repo benchmark, bare-pattern SAST matches (no import gate, no taint trace, no entropy check) ran at roughly 26% precision — three noise findings for every real one. Gating on them is what makes a scanner "cry wolf." Moving them out of the gate removed ~200 gate-blocking findings per benchmark run without demoting a single true positive — every TP in the corpus was either Medium/High confidence, or had a High/Critical band even after the confidence discount.

A high severity does not save a finding from this demotion. A Low-confidence SAST finding is routed to a non-gating advisory domain (see Scoring & Verdicts §4), so it never reaches the --fail-on severity comparison regardless of its raw severity. Its effective risk (severity × confidence, §2) has a band of Low, which is the intuition for why it should not block; severity alone never forces a block.

Scope guard (recall preservation): this gate applies to SAST only. It does not touch Secrets (a no-entropy-gate secret can still be a real committed credential) or SCA (always High confidence anyway).

Implementation: Finding::domain() in finding.rs.

2. Effective risk = severity × confidence

Showing raw severity alone makes a low-confidence false positive scream "CRITICAL" and erodes trust. Effective Risk is the number a human should actually triage on — the same idea as CVSS's Report Confidence temporal metric, which discounts an unconfirmed score.

effective_score = severity_base × confidence_weight

with (effective_risk.rs):

The score is then banded into Critical / High / Medium / Low / Info, with two invariants:

• De-escalation only. The effective band is capped at the raw severity band — confidence can only lower risk, never inflate it. (A High×High finding scores 7.5 but stays in the High band; it never crosses into Critical.)
• unverified flag. When raw severity is High/Critical but confidence is Low, the finding is stamped unverified so UIs render an UNVERIFIED badge — "needs triage", not "the sky is falling".

Worked examples:

How this reaches the gate today — via the finding domain. The --fail-on <severity> threshold is compared against the raw severity of gating-domain findings only. Confidence controls gating indirectly: a Low-confidence SAST finding is placed in a non-gating advisory domain, so a Low-confidence SAST High never reaches the --fail-on comparison at all. Raw severity and confidence are both preserved on the finding, so nothing is hidden. (The sub-10%-FP roadmap moves toward having the gate reason about the effective band directly; the effective-risk column above is the prioritization signal in the meantime.)

3. Source provenance

The second structural FP defense is where the tainted data came from. A rule that flags "attacker-controlled input reaches a dangerous sink" is only a real vulnerability if the input is actually attacker-controlled. Historically the taint engines could not tell getenv() from an HTTP body — every origin looked equally dangerous — which produced a large class of FPs (the NO_PROVENANCE class: ~11% of all FPs in the 31-repo audit).

Every taint origin now carries a provenance tier:

A rule declares which tiers it fires on via requires_source_provenance: { fire: [...], demote: [...] } (GVL schema, back-compatible: absent = legacy = every non-Const tier fires). A web-injection rule (SQLi / XSS / traversal / SSRF) declares fire: [remote] and demote: [local_cli, env]. When a finding's dominant source tier is not in the rule's fire set, it is demoted out of the gating stream — not deleted.

3.1 Why the SSLKEYLOGFILE debug hook does not gate

The 50-repo FP study's single worst gating false positive was this line in mongoose (mongoose.c:16172):

cCOPY
char *keylogfile = getenv("SSLKEYLOGFILE");   // Env-tier source
if (keylogfile == NULL) return;
FILE *f = fopen(keylogfile, "a");             // path-open sink

This is the industry-standard TLS-debug hook, compiled only under #ifdef MG_TLS_SSLKEYLOGFILE, reading the process's own trusted environment. It is not attacker-remote, yet it used to gate CI at High severity.

This is fixed with a sink taxonomy:

• CommandExec sinks (system, popen, execl…, posix_spawn) — a non-remote source is still dangerous here (setuid privilege escalation), so these keep the "High/Critical stay visible" carve-out. argv → system() in a setuid binary remains a gating finding.
• PathOrMemory sinks (fopen, open, malloc, memcpy, …) — a program opening its own env-configured path, or sizing a buffer from a CLI arg, is genuinely low-risk when the source is non-remote. For these, a non-remote source demotes the finding.

The demotion works by setting confidence_tier → Low, which then flows through the confidence→domain gate (§1.2) and reclassifies the finding to the non-gating advisory domain. The finding stays in findings.json at its original severity, carries provenance: {tier: env, demoted: true}, and no longer gates CI. Fail-open default: a sink in neither list is treated as CommandExec (keep-visible).

4. Dismissing a finding

When triage confirms a finding is wrong (or right), you record that decision with gadriel code fix.

4.1 Mark a false positive

bashCOPY
gadriel code fix <FINDING_ID> --false-positive --reason "debug-only endpoint, gated behind admin auth"

This is a zero-LLM, deterministic path. It:

1. Writes a durable waiver to .gadriel-waivers.toml at the repo root, keyed by (rule_id, path, line) — not by rule_id alone, so a second finding of the same rule at a different location is not covered.
2. Updates the per-repo Bayesian posterior (β += 1) as a soft learning signal.
3. Appends an audit event (confirmed_false_positive, agent_runner: human) to .security/audit-log.jsonl.

On future scans the finding is segregated, not deleted — it still appears in a separate "Waived (FP)" report section, but the security gate and KPIs ignore it.

The MCP tool dismiss_false_positive (used by AI coding agents on your behalf) writes the identical (rule_id, path, line) waiver schema.

4.2 Confirm real, or hand to an AI agent

bashCOPY
gadriel code fix <FINDING_ID> --confirm-real   # records α += 1, zero LLM, audit event
gadriel code fix <FINDING_ID>                  # routes to the per-pillar AI agent for
                                               # confirmation + a remediation suggestion

The bare gadriel code fix <id> form invokes the reasoning runner (per-pillar agent) to confirm the finding and propose a fix, writing its verdict to the audit log. Sending source to the model requires explicit consent (the send_source gate).

5. The waiver file

.gadriel-waivers.toml lives at the repo root and is committed with your code.

tomlCOPY
[[waiver]]
rule_id = "CODE-W1-AI-001"
path    = "src/health/echo.py"   # optional glob; default: any path
line    = 42                      # optional exact line
scope   = "echo_handler"          # optional symbol/scope substring
reason  = "debug-only endpoint, gated behind admin auth"
expires = "2026-12-31"            # optional; an expired waiver is ignored
mode    = "segregate"            # "segregate" (default) | "suppress"

Matching. A waiver matches a finding on rule_id plus any of the optional narrowing keys that are present (path glob, exact line, scope substring). Absent keys are unconstrained.

Modes.

Lifecycle & expiry.

• A missing waiver file is the common case and a cheap no-op.
• A malformed file logs a warning and yields an empty set — a broken waiver file must never crash a scan or silently disable all gating.
• An expired waiver (expires in the past) is ignored — the finding becomes active again and gates. Set expiries deliberately: a waiver with an expiry is a promise to revisit, and CI will start failing on the finding the day after it lapses. Prefer time-boxed waivers over permanent ones.

The waiver layer is independent of both the audit log (the history) and the posterior (the soft signal): the waiver is the durable disposition.

6. The FP-prevention program

Confidence, effective risk, and provenance are the runtime instruments. Behind them is a standing engineering program whose explicit target is a gating FP rate under 10%, measured and budgeted rather than discovered by users.

The recall-preservation principle governs everything. Every gate below defaults to keeping the finding and acts only on positive proof of benignity:

Engine-level gates that recognise provably-safe code:

• Sanitizer / guard recognition — a known sanitizer or bound-check that dominates the sink.
• Literal / const suppression — Const-tier sources never taint (hard invariant); constant-propagation for class-level string attributes.
• Test / tooling-path exclusion — findings in test and tooling paths are demoted (with an adversarial "a real vuln in a dir merely named test" must-still-fire fixture to prevent over-reach).
• Framework scoping — requires_import gates so a rule only fires when the dangerous library is actually in scope.

Standing benchmark & per-rule budgets:

• A consolidated gadriel-fp-benchmark corpus with a discovery tier and a standing precision/recall tier (SHA-pinned repos, labelled vuln_by_design | clean_hardened | mixed), run nightly.
• Per-rule FP budgets with Wilson-bounded gates and an auto-quarantine concept (reversible, per-rule demotion) rather than an all-or-nothing build break.
• Recall anchors are mandatory: at least one vuln_by_design target per language family before that family's rows count toward recall. C and Rust are flagged NEEDS_RECALL_ANCHOR until anchored — so a precision win can never be bought with an unmeasured recall loss.
• Every FP fixed becomes a ground_truth=F regression entry so a PR that re-introduces it fails.

Honest scope. This program raises precision; it does not add new detections, and recall on real CVEs is bootstrap-level. Suppression is always fail-open.

7. Tier-2 semantic verification (opt-in FP instrument)

Gadriel Code can optionally ask the host LLM to help refute a false positive. This is opt-in — it is off by default and enabled per scan via the .gadriel.toml [tier2] opt-in / the CLI flag. It is not a detector; it is a precision instrument with a deliberately narrow surface.

Eligibility is bounded by construction to one band: a Tier-2 refutation only changes anything for scan_type == Sast && confidence_tier == Medium. SAST-Low is already non-gating (§1.2) and SAST never emits High deterministically — so Medium is the only band where a refutation can flip the gating decision.

What a refutation does: when the host LLM is asked to actively try to prove finding {id} is a false positive and returns a concrete safety-mechanism citation, the fusion engine (tier2_fusion.rs) writes exactly one field — confidence_tier → Low — and stamps an ai_refutation audit block. The confidence→domain gate then reclassifies the finding as non-gating advisory. Severity is never touched, and a deterministic code-vuln can never be suppressed by any AI input (locked by tests such as never_suppress_deterministic_code_vuln_survives_all_ai_inputs).

The three-way LLM verdict maps onto existing, tested fusion arms:

Budget is spent only where it can flip a decision: a boundary-proximate Medium finding from a historically-noisy rule with ambiguous reachability is prioritised over one sitting deep inside "clearly Medium" (where an LLM read risks a hallucinated demotion for no gain).

8. Practical triage workflow

When a scan surfaces findings, work them in effective-risk order:

1. Read the gating stream first. Only CodeVuln and SupplyChain domains gate (§ scoring-and-verdicts.md). Everything in the advisory section (hardening, CI/config, coverage, AI advisory, and demoted Low-confidence SAST) is informational — triage it, but it will not block your build.
2. For each gating finding, look at effective risk and the unverified badge, not raw severity. An UNVERIFIED badge means the engine has not confirmed it — treat it as "needs a human look", starting with the highest effective band.
3. Check provenance. If the taint source is Env / LocalCli / Const on a path/memory sink, it is likely already demoted; if it somehow gates, that's a strong FP signal worth a --false-positive.
4. Decide: waive or fix.
   • Waive (gadriel code fix <id> --false-positive) when the code is genuinely safe for a reason the engine can't see (an auth gate upstream, a trusted operator-only path, a debug-only hook). Add a reason, and set an expires if the safety argument is temporary.
   • Fix the code when the finding is real, or run gadriel code fix <id> (bare) to get an AI-proposed remediation. Use --confirm-real to record a true positive without a code change (e.g. tracked in a ticket).
5. Commit .gadriel-waivers.toml alongside the code so the disposition is durable and reviewable, and re-run the scan to confirm a clean gate.

Rule of thumb: waive the finding, not the rule. A (rule_id, path, line) waiver disposes of one specific false positive; if a whole rule is systematically wrong, that belongs in the benchmark/quarantine program (§6), not in per-repo waivers.

Cross-references

• scoring-and-verdicts.md — the risk formula, pillar/overall scores, PASS/PARTIAL/FAIL verdicts, coverage status, and exit codes.
• Ground truth in source: crates/code-security/gadriel-finding-types/src/confidence.rs, .../effective_risk.rs, .../finding.rs (domain()), crates/gadriel-cli/src/commands/code/waivers.rs, crates/gadriel-cli/src/commands/code/fix.rs.