Skip to content

Systems and records

Superseded document retrieval dataset: retired text that reads as live

By SourceX Editorial · Updated

10 sections

Short answer

A superseded document retrieval dataset labels which guidance was retired, what replaced it and when, using markers a retriever will actually read. MITRE ATT&CK shows the gap: all 446 retired entries carry a status flag and every revoked entry links its replacement, but only 14 say in their own text that they are retired, and 283 deprecated entries name no successor anywhere.

Retrieval

Superseded document

This page covers superseded document retrieval dataset: retired text that reads as live.Illustrative

Key takeaways

  • Superseded guidance needs its marker wherever a retriever looks: the entry's status, a link to the replacement, the text itself and a dated record.
  • In ATT&CK v20.0, all 157 revoked entries link a replacement that is still active, yet none says in its description that it was revoked.
  • 224 of 268 mitigations are deprecated and their advice still reads as current; no retired mitigation, data source, data component or detection strategy carries a notice in its text.
  • Only revoked entries carry a dated retirement record, and all 367 entries retired by July 2020 now show modified dates from after their retirement.
  • Removal is the quietest failure: 14 entries from the January 2018 release are gone from every file without a tombstone.

What makes guidance superseded, and how would a retriever know?

Guidance is superseded when its owner replaces it, withdrawn when the owner retires it with no replacement, and partly superseded when a later document changes some of it. A retriever knows only what the indexed text and its metadata say, so the question for a dataset is where each of those states is recorded and whether that place survives indexing.

A retirement marker can live in six places: a notice on the container, such as a retired wiki space or a deprecated folder; a status field on the entry; a link from the retired entry to its successor; a notice in the entry's own text; a dated record of when it stopped applying; and the links from live entries that should no longer point at it. A chunked text index keeps only the fourth. Curated knowledge bases tend to invest in the second and third, as the census below shows for one of them.

Retired guidance still answers questions. Which mitigation a team recommended in 2019, or which procedure applied on the day of an incident, has a retired answer. Deleting old guidance from the index loses that, so a superseded document retrieval dataset keeps retired entries, labels them and dates them.

Why ATT&CK stands in for an internal guidance library

MITRE ATT&CK is maintained the way a careful internal knowledge base is meant to be: a single organization curates the entries, publishes numbered releases and retires entries under a written policy instead of deleting them. Its repository describes it as a globally accessible knowledge base of adversary tactics and techniques built from real-world observations, and its mitigations and detection strategies are guidance in the plain sense of telling a defender what to do.

The retirement policy is written down. The repository's usage guide says entries no longer worth tracking are marked deprecated, entries replaced by another are marked revoked and given a revoked-by relationship pointing at the replacement, and both kinds are kept so that workflows relying on them do not break. It adds that relationships cannot be revoked or deprecated themselves; they count as retired when an entry at either end is.

ATT&CK is published in STIX, whose version 2.1 specification makes revocation permanent and says that marking an object revoked is an update, so its modified timestamp must change at the same moment. That gives a standard to measure the dates against. The census reads the Enterprise bundle for release v20.0 (commit 50be388, 13 October 2026) and the bundles for three earlier releases.

Which retirement markers does ATT&CK carry, and where?

The structured markers are complete, and the text markers are almost absent. Of 4,867 entries in v20.0, 446 are retired: 157 revoked and 289 deprecated, with no entry carrying both flags. All 446 carry their status flag, and every revoked entry has a revoked-by link that resolves to an active entry, 154 in one step and 3 by way of a second revoked entry.

Deprecated entries have no structured successor link by design, and their text rarely supplies one. Six of the 289 tell the reader which entry to use instead, each a technique whose description opens with a deprecation notice and a link. The other 283 name no replacement anywhere.

Read as plain text, the retired set is nearly invisible. Only 14 of the 446 retired entries (3%) say in their own description that they are retired: 12 techniques, one group and one software entry. None of the 157 revoked entries does. None of the 267 retired guidance entries does either: 224 deprecated mitigations out of 268, all 38 data sources, 3 data components and 2 detection strategies. A deprecated mitigation still reads as advice, telling the reader what to configure, with nothing in the text to say that advice was retired; all 224 were already deprecated in the July 2020 release.

Graph links are the one place ATT&CK cleans up after a retirement. Apart from the 157 revoked-by links, only 2 of 21,662 relationships touch a retired entry, each linking a retired sub-technique to its active parent, and no relationship carries a status flag of its own. A system that walks the graph from live entries will rarely land on retired guidance; one that searches the text will land on it whenever the words match. Marker by marker:

Which retirement markers does ATT&CK carry, and where?
MarkerRevoked entries (157)Deprecated entries (289)What a text-only retriever sees
Status flag on the entry157289Nothing: the flag is metadata
Machine-readable link to the replacement157None, by designNothing
Successor named in the description06A pointer, in 6 entries
Retirement notice in the description014A warning, in 14 entries
Dated retirement record157, the link's creation date0Nothing
Live relationships still attached11Not applicable
A grid of five retirement markers for revoked and deprecated ATT&CK entries: status flags on all 446, replacement links and dated records on the 157 revoked entries only, and successor names or retirement notices in the text of just 6 and 14 deprecated entries.
Where ATT&CK v20.0 records retirement for its 446 retired Enterprise entries (mitre/cti, commit 50be388). Counts from the article's census.

When did this guidance stop applying?

Only the revoked entries can say. Each revoked-by relationship carries its own creation date, so 157 entries have a dated retirement record and the 289 deprecated entries have none. The modified timestamp does not fill the gap, because later bulk edits move it: 386 of the 446 retired entries share their last-modified day with at least 50 other retired entries, and 226 were last modified on 18 April 2025.

Older releases show how far those timestamps drift. All 367 entries already retired in v7.0, released on 8 July 2020, now carry a modified date after that day, so none of them can date its own retirement. Among revoked entries, 17 were last modified on the day their revoked-by link was created and 140 later. STIX expects the timestamp to move when an object is revoked; nothing stops the next edit from moving it again.

The dependable dating source is the release history. The repository tags every release from v1.0 in January 2018 to v20.0, so the window in which an entry was retired can be bounded by the last release where it was active and the first where it was not. Company guidance needs the same discipline in a simpler form: an effective-from and effective-to date on each version, set when the status changes and never inferred from a last-edited stamp.

What disappeared without any marker?

A few entries were removed rather than retired, which no status field can reveal. Of the 620 entries in the January 2018 release, 14 appear nowhere in the v20.0 repository, in any domain folder: 12 mitigations, one group and one software entry. Of the 1,367 entries in v7.0, 4 are missing from the v20.0 Enterprise bundle; 2 moved to the Mobile domain and 2, a mitigation and a software entry, are gone. No entry present in v17.1 (May 2025) has been removed since.

For a retrieval dataset, removal is worse than an unmarked retirement because the historical answer goes with the entry. Ask for a tombstone for anything deleted: the identifier, the last version, the date and the reason, even when the content itself cannot be shared.

The supersession marking audit: seven checks for a guidance archive

The audit separates marking from cleanup. A library can score perfectly on status fields and still fail retrieval, as ATT&CK's text notices show, so each check is counted on its own rather than folded into a single score, and each produces a label the dataset keeps.

Run the checks on a company's archive before sampling it, and ask for the counts in this shape. ATT&CK's results are a reference point, not a target:

The supersession marking audit: seven checks for a guidance archive
CheckWhat to countATT&CK v20.0Label in the dataset
Container noticeRetired spaces, folders or repositories with a banner or statusNot applicable: one live bundleEvery entry inherits its container's status
Entry statusRetired entries with a status field446 of 446Superseded, withdrawn or partly superseded
Successor linkSuperseded entries linking an active replacement157 of 157 revoked; 6 of 289 deprecated name one in textSuccessor ID, or none recorded
Text noticeRetired entries whose own text says so14 of 446Text-marked yes or no, kept as a hard-negative flag
Effective datesRetired entries with a dated status change157 of 446Effective-to date, or a window bounded by releases
Inbound linksLive links still pointing at retired entries2 of 21,662 relationships, apart from the 157 revoked-by linksEach listed, none dropped silently
RemovalsIdentifiers in an older snapshot and gone now14 since January 2018; none since May 2025Tombstone with last version and date

How a buyer would use a superseded document retrieval dataset

As cases whose gold answer depends on the date and on the marker, not on text similarity. Each case pairs a question with an as-of date and the guidance set as it stood, retired entries included. For a current date the correct output cites the active successor; for a historical date it cites the entry that applied then, with its effective window; for an entry withdrawn with no successor, the correct output says so and does not substitute the nearest live entry.

Three measures come out of that design: how often a system cites retired guidance as current, how often it finds the successor when one exists, and how often it answers a dated question with the guidance in force on that date. Retired entries with no text notice make the strongest hard negatives, because nothing but metadata separates them from live ones. Ranking the controlling version among look-alike versions is a different test with its own guide; this design asks whether the retirement itself is visible.

Keep ambiguity as a label. A deprecated entry with no successor and no notice may have been retired because the topic stopped mattering or because it was merged somewhere unrecorded, and the archive cannot say which. The dataset should mark it unresolved rather than guess a successor.

The strongest objection: just filter on the status field

Filtering works when every copy of the guidance carries the field and every pipeline reads it, and neither holds for long. Text gets chunked, exported, pasted into tickets and summarized, while the status field stays behind in the source system. Only a notice in the text travels with the words, and ATT&CK puts one on 14 of its 446 retired entries.

Even an official filter can misbehave. ATT&CK's usage guide advises against relying on the built-in STIX filter for revoked objects, because its behavior differs between local data and the TAXII server, and supplies its own check of both flags instead. If a curated knowledge base needs a workaround for its own status field, an internal archive spread across three systems with no shared schema needs labeled examples where the field is missing or ignored.

The objection is right that status fields are the source of truth. The dataset should keep them, and test whether a system uses them, rather than assume the text will carry the marker.

What company guidance archives add, and how to request them

Company archives hold cases ATT&CK cannot: policies superseded mid-quarter, procedures replaced after an incident, and guidance withdrawn by one team while another kept linking to it. Policy portals, document libraries with version history, wiki page archives and knowledge base article versions usually keep status and dates; what they lose is the successor link and the notice in the text. Public administration offers one container-level model: OMB's 2007 Final Bulletin for Agency Good Guidance Practices told agencies to keep an online list of significant guidance in effect, with issuance and revision dates, and to identify documents issued, revised or withdrawn in the past year.

A useful sample exports every version with its status, effective dates, owner role and successor link, keeps retired entries and tombstones, and reports which of the seven checks the archive passes. SourceX does not keep guidance datasets on the shelf: buyers set out scope, volume, language, format, timing and licensing terms on the request form, and SourceX follows up about potential sources. Attaching the audit table as the acceptance test, with the share of retired entries that must carry a successor or an unresolved label, makes such a request specific.

Methods note: ATT&CK releases, counting rules and limits

Data: enterprise-attack.json from mitre/cti at commit 50be388 (tag ATT&CK-v20.0, committed 13 October 2026) and at the release commits for v1.0 (17 January 2018), v7.0 (8 July 2020) and v17.1 (6 May 2025), read on 28 October 2026 by a script (attack_supersession.py) that loads only those bundles and a file listing of the v20.0 tree; no repository code was run. Saved outputs hold STIX and ATT&CK identifiers, types, flags and dates, with no names or descriptions.

Rules: entries are objects of type attack-pattern, course-of-action, x-mitre-data-source, x-mitre-data-component, x-mitre-detection-strategy, x-mitre-analytic, x-mitre-tactic, intrusion-set, malware, tool and campaign; course-of-action, data source, data component, detection strategy and analytic entries count as guidance. Revoked means the revoked field is true; deprecated means x_mitre_deprecated is true and revoked is not. A successor link is a revoked-by relationship from the entry, followed to the first target that is not revoked. A text notice is a description containing this technique, this entry, this mitigation, this data source, this data component, this detection strategy, this group, this software or this object, or a link to an ATT&CK page, followed by has been, was or is and then deprecated, revoked, superseded, replaced or retired, ignoring case. A successor named in text is please use, replaced by, superseded by, merged into or in favor of, followed by a link to an ATT&CK page. Same-day clusters count retired entries whose modified day is shared by 50 or more other retired entries. A dropped entry is an identifier in an older bundle that is absent from the v20.0 Enterprise bundle, counted as moved when a file with that identifier exists in another domain folder. Percentages are rounded once, half up.

Limits: ATT&CK is a security knowledge base curated by one organization, not a company's internal guidance, and it publishes its retirement policy; its shares are a reference point, not a benchmark for enterprise archives. Every notice and successor match is printed by the script and was read, but a notice worded differently would be missed. Three earlier snapshots bound retirement windows only to those intervals. Not measured: how retrieval systems rank retired entries, the Mobile and ICS domains, and company archives, for which SourceX has published no measurements.

Frequently asked questions

What is a superseded document retrieval dataset?

A set of guidance documents in which each retired version is labeled with its status, its successor if one exists and the dates it applied, paired with questions whose correct answer depends on those labels. It tests whether a system cites retired guidance as current, finds the replacement and answers dated questions with the version in force at the time.

What is the difference between superseded and withdrawn guidance?

Superseded guidance has a named replacement; withdrawn guidance was retired with none. ATT&CK draws the same line: revoked entries link to the entry that replaced them, while deprecated entries were dropped from active tracking, and only 6 of its 289 deprecated entries name a successor in their text.

Why not delete superseded documents from the search index?

Because dated questions need them. An audit, an incident review or a dispute asks what applied on a past date, and the answer is often a retired version. Keep retired entries, label and date them, and record a tombstone for anything that has to be deleted.

How often does retired guidance say it is retired in its own text?

In MITRE ATT&CK v20.0, 14 of 446 retired entries do, about 3%, and none of the 267 retired mitigations, data sources, data components and detection strategies does. Their status lives in metadata fields that chunking and export usually drop.

How is this different from a document version benchmark?

A version benchmark asks whether retrieval ranks the controlling version above similar ones. This dataset asks whether the retirement itself is visible: whether each retired entry carries a status, a successor link, a text notice and a date, and whether a system uses them.

Sources

  • ATT&CK Enterprise bundle at release v20.0 (commit 50be388, 13 October 2026): the source of the census of 4,867 entries, 446 retired, revoked-by links, text notices, dates and relationships. Source
  • ATT&CK's usage guide says entries no longer worth tracking are deprecated, replaced entries are revoked with a revoked-by relationship, both are kept so workflows do not break, relationships count as retired when an attached entry is, and the built-in STIX revoked filter behaves inconsistently across access methods. Source
  • The mitre/cti README describes ATT&CK as a globally accessible knowledge base of adversary tactics and techniques based on real-world observations. Source
  • ATT&CK v1.0 release commit (17 January 2018): 620 Enterprise entries, used to find entries later removed without a tombstone. Source
  • ATT&CK v7.0 release commit (8 July 2020): used to establish which entries were already retired by then and which were later removed or moved. Source
  • ATT&CK v17.1 release commit (6 May 2025): used to confirm no entry has been removed since and which guidance entries were active then. Source
  • STIX 2.1 says a revoked object may not have future versions created and that marking an object revoked is an update, so its modified property must be updated at the same time. Source
  • OMB's 2007 Final Bulletin for Agency Good Guidance Practices asked agencies to keep an online list of significant guidance in effect, with issuance and revision dates, identifying documents issued, revised or withdrawn in the past year. Source
  • SourceX's request page asks buyers to share scope, volume, language, format, timing and licensing needs and says SourceX follows up about potential sources. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies