Skip to content

Systems and records

Security false positive outcome data: what a dismissal must record

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

A dismissed security alert should carry a disposition from a fixed list that says why it was closed (detection logic wrong, data wrong, authorized test, expected activity, duplicate or insufficient evidence), the evidence that rules out malicious activity, and the follow-on record such as a tuning ticket. A bare false positive label hides which of these happened, and those causes teach a model opposite lessons.

Grader rubric

Security false positive

This page covers security false positive outcome data: what a dismissal must record.Illustrative

Key takeaways

  • False positive and benign positive are different outcomes: one says the detection was wrong, the other says the activity was real but allowed.
  • Split false positives by cause, detection logic or inaccurate data, because the fixes and the lessons differ.
  • Every label needs minimum evidence an independent reviewer can check; a closing note that says fine is not evidence.
  • Insufficient evidence and unreviewed automatic closure are honest labels and belong in the data.
  • Keep reversals as a second dated decision instead of overwriting the original call with hindsight.

What reason and evidence should accompany a dismissed security alert?

A dismissed alert needs three things: a disposition from a fixed list that says why it was dismissed, the evidence that rules out the malicious explanation, and a pointer to what changed afterwards. Why matters more than the word false positive. A detection that fired on corrupted log data, a detection whose logic is wrong, and a detection that correctly caught an authorized penetration test are all closed without action, but they teach a model opposite lessons.

This guide gives security operations and managed security teams a disposition label guide they can apply to past alerts before those alerts become training or evaluation data. It covers the labels, the minimum evidence for each, a crosswalk to the values used by common SIEM and XDR tools, an illustrative worked example, the exception that breaks most label sets, and what to keep out of any dataset that leaves the company.

It deliberately does not cover how to investigate an alert. The question here is narrower: once a competent analyst has dismissed it, what must the record contain for someone else to trust that dismissal later?

Why is closed as false positive the weakest label in an alert history?

Because it hides whether the detection was wrong, the data was wrong, or the activity was real but allowed. NIST's glossary defines a false positive as benign activity or content incorrectly classified as malicious, which is a statement about the detection, not about the activity's harm.

The tools SOC teams already use make this split, but teams often collapse it at closing time. Microsoft Sentinel's incident API offers four classifications (true positive, benign positive, false positive and undetermined) and a separate classification reason that separates incorrect alert logic from inaccurate data and suspicious but expected activity. The Microsoft Graph security alert resource pairs a classification with a determination such as security testing, line-of-business application, confirmed activity, not malicious or not enough data to validate. Splunk Enterprise Security lists dispositions that distinguish false positives caused by incorrect analytic logic from those caused by inaccurate data, and benign positives that were suspicious but expected.

If your history only says closed or false positive, the distinction is lost and cannot be recovered from the label. It can sometimes be recovered from the notes and linked tickets, which is why the evidence fields below matter more than the label itself.

The disposition label guide: labels, minimum evidence and misuse

Apply one label per dismissed alert, chosen by the analyst who closed it or by a reviewer reconstructing the decision from the record. Each label has a minimum evidence requirement. An alert that cannot meet the minimum for any label gets insufficient evidence, which is itself a useful label, not a failure.

The disposition label guide: labels, minimum evidence and misuse
LabelUse whenMinimum evidence attachedDo not use when
False positive: detection logicThe rule fired on activity that does not match the behavior it was written to catchRule identifier and version, the matching events, and why they fall outside the rule's intentThe rule did what it was written to do and the activity was authorized
False positive: inaccurate dataThe alert came from wrong, duplicated, mis-parsed or mis-timed telemetryThe source record, the parsing or ingestion defect, and the corrected valueThe data was accurate and the analyst simply judged the activity harmless
Benign positive: authorized testingThe activity was real and malicious-looking, and it was part of an approved testThe test authorization or scope document, the tester identity as a role, and matching time window and targetsThe test scope does not cover the asset, account or time of the alert
Benign positive: expected administrative or business activityA known user, administrator or internal application performed the activity for a legitimate reasonThe change ticket, approved job or application inventory entry, plus the account and asset contextThe only evidence is the user saying it was them
DuplicateAnother alert or incident already covers the same activityThe identifier of the surviving alert or incidentThe two alerts share an indicator but describe different activity
Insufficient evidenceThe analyst could not confirm or rule out malicious activity with the data availableWhat was checked, what was missing, and the retention or access limit that stopped the checkThe analyst ran out of time but the data existed; record that as unreviewed instead
Unreviewed automatic closureA suppression rule, playbook or age-out closed the alert without a human decisionThe rule or playbook identifier and its version at closure timeA human reviewed and agreed with the automation; use the human's label

What evidence explains the dismissal beyond a closed ticket?

The evidence that matters is whatever an independent reviewer would need to reach the same conclusion without asking the original analyst. A closing comment such as checked with user, fine is a conclusion with no support. Google Security Operations, for example, requires a close reason and a root cause in its Close Case dialog and lets administrators add their own root causes, which is a good prompt, but a root-cause value is still a summary until it links to evidence.

At a glance

What evidence explains the dismissal beyond a closed ticket

  • Alert payload as raised: rule or model identifier, version, severity and the events that matched, with timestamps in one timezone.
  • The checks performed: queries run, enrichment consulted, endpoints or accounts examined, recorded as references rather than pasted secrets.
  • Authorization artifacts: the change ticket, test scope, job schedule or application inventory entry that explains the activity.
  • People involved, as roles: closing analyst, reviewer, and the system owner or user who confirmed the activity.
  • Runbook or playbook version in force when the alert was closed, because a later runbook may require checks the analyst was not asked to do.
  • Follow-on records: a tuning ticket, suppression rule or parser fix created because of the alert, with its date.
  • Reopen history: whether the alert or a related one was reopened later, and what the new evidence was.

How do tool-specific values map onto one disposition set?

Map each tool's closing values onto the guide's labels with a crosswalk, and keep the original value alongside the mapped one so nothing is lost if the mapping is later corrected. The table below uses values as published in each vendor's or project's own documentation. OCSF, the open cybersecurity schema, offers a normalized verdict list that includes false positive, benign, test, insufficient data, duplicate and managed externally, which makes it a convenient target field for the mapped value.

Where a tool has no value for a label, the gap is the finding: an analyst who could only choose false positive for an authorized test needed a better option, and a reviewer has to read the notes to recover the real reason. Google Security Operations' close reasons (malicious, not malicious, maintenance and inconclusive) are broader, so its root-cause field carries most of the detail.

How do tool-specific values map onto one disposition set?
Guide labelSentinel classification and reasonGraph security alert classification and determinationSplunk ES dispositionOCSF verdict
False positive: detection logicFalsePositive, IncorrectAlertLogicfalsePositive, notMaliciousFalse Positive - Incorrect Analytic LogicFalse Positive
False positive: inaccurate dataFalsePositive, InaccurateDatafalsePositive, notMalicious (detail in comments)False Positive - Inaccurate DataFalse Positive
Benign positive: authorized testingBenignPositive, SuspiciousButExpectedinformationalExpectedActivity, securityTestingBenign Positive - Suspicious But ExpectedTest
Benign positive: expected activityBenignPositive, SuspiciousButExpectedinformationalExpectedActivity, confirmedActivity or lineOfBusinessApplicationBenign Positive - Suspicious But ExpectedBenign
DuplicateNo dedicated value; record in the commentNo dedicated value; record in commentsNo default value; a custom disposition can be addedDuplicate
Insufficient evidenceUndeterminedfalsePositive, notEnoughDataToValidateUndeterminedInsufficient Data

How do you relabel a backlog of alerts that were closed with one generic value?

Relabel from the record, in batches grouped by detection rule, and let the evidence rather than the reviewer's memory decide. Grouping by rule works because a rule that fired on bad data or flawed logic usually did so many times, and one confirmed parser defect or rule fix can explain a whole batch once its dates are known.

For each rule, first collect the tuning tickets, parser fixes, rule version changes and test schedules from the same period. Then walk the closed alerts in date order and assign labels only where a linked record supports them; everything else becomes insufficient evidence. Have a second reviewer check a sample from every batch, and give any batch where the two reviewers disagree often a fresh look rather than settling it by majority vote.

Keep the original closing value in its own field and record who relabeled each alert, as a role, and when. A relabeled history is a reconstruction, and a buyer or auditor should be able to tell reconstructed labels from the ones analysts chose at closing time.

Worked example (illustrative): three dismissals from one week

The alerts, systems and details below are invented. A mid-sized company's SOC dismissed three alerts in one week. A reviewer preparing the history for a labeled dataset applies the guide and finds that only one of the three is usable as first recorded.

  • Only the parser case was correct as originally labeled; the testing case was mislabeled and the travel case was unsupported.
  • Relabeling did not need new investigation, only the records the team already had, which is the usual situation.
  • The impossible-travel case is still valuable: an honest insufficient-evidence label teaches a model when not to conclude.
Worked example (illustrative): three dismissals from one week
Alert as closedEvidence in the recordLabel after reviewUsable as is?
Credential dumping tool on a server, closed as false positiveTest scope document naming the server and window, tester role, matching timestampsBenign positive: authorized testingYes, after relabeling from false positive
Impossible travel sign-in, closed as false positiveNote says VPN; no VPN log reference, no user confirmationInsufficient evidenceOnly as an insufficient-evidence example
Spike in failed logons, closed as false positiveParser ticket showing a time-zone defect that duplicated events, with the fix dateFalse positive: inaccurate dataYes
Follow-up for the logon spikeParser fix deployed; no recurrence in the following monthOutcome recordedYes, adds the after state

When a dismissed alert turns out to be real

Keep both decisions. When later evidence shows that a dismissed alert was part of a real intrusion, the original dismissal and the reversal are the most informative records in the history, as long as they are kept as two dated decisions rather than one overwritten field.

Record the original label, the evidence available at the time, the new evidence, the date it appeared and the corrected label. Do not retroactively mark the first analyst's call as an error if the evidence at the time supported it; mark it as superseded. A training set that silently rewrites past labels with hindsight teaches a model to expect information it will not have when it runs.

Reversals also usually involve the most sensitive material, so they are the cases most likely to be held back under the rules in the next section.

Which dismissed alerts should stay out of a dataset that leaves the company?

Anything that would let a reader attack you or your clients, and anything your contracts keep confidential, stays out. Dismissed alerts feel low risk because nothing bad happened, but their evidence often contains exactly the detail an attacker wants: what you monitor, what you ignore and why.

  • Live secrets of any kind: passwords, tokens, keys, session cookies or connection strings found in alert payloads, which should be removed before export and treated as an incident if exposed.
  • Detection logic that would reveal blind spots, such as exact suppression conditions or allow-listed paths, unless it is generalized.
  • Client telemetry handled under managed service agreements, unless the client has agreed in writing to that use.
  • Alerts tied to HR, legal or insider investigations, and any open investigation.
  • Internal hostnames, IP plans and account names: replace them with consistent pseudonyms so relationships survive.
  • Penetration test reports themselves, which belong to the testing agreement; keep only the scope facts the label needs.

Licensing a labeled dismissal history

A SOC or managed security provider that can show why each alert was closed, with the evidence linked, has records few public datasets offer. SourceX helps security providers turn selected dismissal records into licensed datasets for AI, handling rights review, contracting, preparation and delivery.

You choose which records are in scope. No alert record is delivered until the company has signed off on scope and terms in a signed agreement, and the starting point is a short company assessment with no upload of alert data.

Frequently asked questions

What is the difference between a false positive and a benign positive alert?

A false positive means the detection was wrong: the activity did not match what the rule was meant to catch, or the data was faulty. A benign positive means the detection was right about the activity but the activity was allowed, such as an authorized test or a known administrator task. Microsoft Sentinel and Splunk Enterprise Security both keep these as separate values.

What should an analyst write when closing an alert as a false positive?

Name the reason (detection logic or inaccurate data), link the events and the rule version, and describe the check that ruled out malicious activity. Add the tuning or parser ticket if one was raised. A note that only says fine or checked is not enough for anyone else to rely on.

Should alerts closed by automation be used as training labels?

Only with a label that says so. A suppression rule or playbook closure records the rule's judgment, not an analyst's, so mark it as an unreviewed automatic closure and include the rule version. Mixing them with human decisions makes a model learn the automation's blind spots.

How should an insufficient evidence dismissal be labeled?

Label it as insufficient evidence and record what was checked, what was missing and why. Sentinel calls this undetermined; OCSF has an insufficient data verdict. These cases teach a model when the right answer is that it cannot decide.

What if a dismissed alert later turns out to be malicious?

Keep the original decision and add the reversal as a second dated decision with the new evidence. Mark the first label as superseded rather than wrong if the evidence at the time supported it. Overwriting it hides what an analyst could actually know when the alert fired.

Sources

  • NIST's glossary defines a false positive as benign activity or content incorrectly classified as malicious (SP 800-86 and SP 800-83 Rev. 1 definitions). Source
  • The Microsoft Sentinel incident API (2025-06-01) classifies closed incidents as Undetermined, TruePositive, BenignPositive or FalsePositive, with reasons SuspiciousActivity, SuspiciousButExpected, IncorrectAlertLogic and InaccurateData. Source
  • The Microsoft Graph security alert resource has a classification (falsePositive, truePositive, informationalExpectedActivity) and a determination such as securityTesting, confirmedActivity, lineOfBusinessApplication, notMalicious or notEnoughDataToValidate. Source
  • Splunk Enterprise Security 8.5 lists dispositions Undetermined, True Positive - Suspicious Activity, Benign Positive - Suspicious But Expected, False Positive - Incorrect Analytic Logic and False Positive - Inaccurate Data. Source
  • Splunk Enterprise Security dispositions default to Undetermined and administrators can configure custom dispositions. Source
  • Google Security Operations requires a close reason and a root cause in the Close Case dialog, and administrators can add custom root causes. Source
  • Google Security Operations case close reasons are Malicious, Not Malicious, Maintenance and Inconclusive. Source
  • The OCSF dictionary's verdict_id lists normalized incident verdicts including False Positive, True Positive, Benign, Test, Insufficient Data, Duplicate and Managed Externally. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies