Skip to content

AI uses for records

Deployment stop decision dataset: what 90 release holds left behind

By SourceX Editorial · Updated

9 sections

Short answer

A deployment stop decision dataset treats each hold as a case: the candidate that was stopped, the evidence weighed, who had authority, the rule applied when nobody answered, and what shipped instead. OpenStack's public release record shows how often those parts go missing: across eight cycles, 89 of 622 deliverables were held, yet only 17 of 90 holds state a reason in words and none names its approver in git.

Agent run

Deployment stop decision

This page covers deployment stop decision dataset: what 90 release holds left behind.Illustrative

Key takeaways

  • A hold is a decision with an alternative, not an absence of deployment; its record must name the stopped candidate, the evidence and what shipped next.
  • In openstack/releases at commit 9da6d9c, 89 of 622 cycle-with-rc deliverables, 14.3 percent, were held at least once between 2023.1 and 2026.2, for 90 holds.
  • 32 of the 90 holds carry nothing beyond a subject line, 17 state a reason in words, and only 9 link to a bug, review or commit through a URL or a Depends-On/Related-Bug footer.
  • Approvals sit in the review system rather than git, and the rule for silence differed between generated hold requests and the written process.
  • Held deliverables later shipped stable fixes more often, 60 of 77 against 136 of 479, which shows where defects cluster, not that holds failed.

What counts as a decision not to deploy?

A decision not to deploy is a recorded judgment, made at a named decision point, that one specific candidate should not ship yet, together with the alternative chosen instead: fix it and cut a new candidate, wait for a later window, or ship with the issue written down. An empty deploy log is not a hold. A hold has a candidate, a reason and a consequence, and each of the three can be checked later.

It sits between two neighbors that already have their own records. A rejected patch is a review decision about one change, the subject of our page on why a working patch was rejected. A rollback undoes something users already received, the subject of our page on reverted deployments. A hold happens after the change is merged and before anyone runs it in production, which is why its evidence is scattered across release files, test results and approval threads rather than kept in one ticket.

Risk frameworks already expect the decision to exist. NIST's AI Risk Management Framework lists under MANAGE 1.1 that "A determination is made as to whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed," and it describes an initial go/no-go decision once the MAP function is complete. The framework asks for the decision; it does not say what record the decision should leave behind. That gap is what a dataset of stop decisions fills.

For an AI team the use is specific: training or evaluating an agent that recommends whether a release should go, and scoring it on cases where not yet was the right answer. Without hold cases in the data, every example teaches a model that merged, tested code ships.

Which seven fields make a release hold record usable?

Seven fields turn a hold into a case that a model can learn from and an auditor can rebuild: the identity of the stopped candidate, the trigger and its evidence, the reason in words, the decider and the authority they held, the rule that applies when nobody responds, the next candidate with what changed, and the artifact that finally shipped. Missing any one of them changes what the case can teach.

The decider and the silence rule are the easiest fields to overlook, and they matter most for training. A model that sees a stop without knowing who could order it, or whether anyone actively agreed, cannot tell a judgment from a default. The last column counts how many of OpenStack's 90 recent holds carry each field in the public release repository, using the script described in the methods note:

Which seven fields make a release hold record usable?
Trace fieldWhat it must holdWhere OpenStack keeps itPresent in the 90 holds
Candidate identityThe exact commit or artifact that was stopped and the one cut insteadDeliverable file: one version and commit hash per candidate90 of 90
Trigger and evidenceThe bug, test result or review that prompted the stopLinks and change lists in the request; bug tracker and code review hold the rest39 of 90 (9 with a URL or a Depends-On/Related-Bug footer)
Reason in wordsWhy the candidate was not good enoughFree text in the request's commit message17 of 90
Decider and authorityWho could say no, and under which ruleCode review votes and an approval check job, outside git0 of 90 in git
Rule on silenceWhat happens if the decider does not answerProcess document, sometimes repeated in the request23 of 90 (20 merge, 3 abandon)
Next candidate and deltaThe re-cut candidate and the changes between the twoNext version entry; a change list when one is pasted in90 of 90 (32 list changes)
What shippedThe artifact users receivedThe final release re-tags the last candidate90 of 90
Seven ring gauges in the order a release hold unfolds, showing how many of OpenStack's 90 release holds carry each field: stopped candidate 90, evidence or change list 39, reason in words 17, approver in git 0, silence rule 23, next candidate 90 and what shipped 90.
How much of a stop decision OpenStack's release repository keeps: 90 holds, cycles 2023.1 to 2026.2, counted at commit 9da6d9c on 2026-10-27.

What 90 OpenStack release holds put on the record

They put the decision on the record every time and the reasoning rarely. OpenStack, the open-source cloud platform, releases most of its services under a model it calls cycle-with-rc: each deliverable publishes one or more release candidates before a coordinated release, and on release day the release team re-tags "the commit used for the last RC," in the words of its process document. Every candidate after the first is therefore a dated decision that the previous candidate would not become the release.

We counted those decisions in the public openstack/releases repository at commit 9da6d9c of October 27, 2026, across the eight cycles from 2023.1 (Antelope, released March 22, 2023) to 2026.2 (Hibiscus, released September 30, 2026). Of 623 cycle-with-rc deliverable files, one per deliverable per cycle, 622 published at least one candidate. 89 of those, 14.3 percent, were held at least once: 88 cut a second candidate and one cut a third, 90 holds in all. All 622 final releases point at the same commit as their last candidate, so what shipped is never in doubt.

Why each hold happened is a different story. We read the commit that added each new candidate, after removing tool-generated sentences, process boilerplate, pasted change lists and links. 32 holds, 35.6 percent, carry nothing beyond a subject line naming the deliverable, the candidate and the cycle. 17, or 18.9 percent, state a reason in words, and only 7 of those also link to the bug, review or commit behind it.

The best of them read like the record a dataset needs. One names the fix for security advisory OSSA-2026-004. One explains that a library's minimum version "did not get bumped" as needed and a second candidate fixes it. One reports a blocker that stopped the previous release from uploading to the Python package index. Each tells a reader what was wrong, which artifact carried it and what changed. The full classification:

What 90 OpenStack release holds put on the record
What the hold's commit message carriesHoldsShare of 90Typical content
A reason in words1718.9%A security fix, a missed dependency bump, a failed package upload
Free text without a reason word88.9%Missing release notes, or a note that this is the final candidate
A change list or link only2730.0%Commits between the two candidates, or a link to one change
A generated sentence only66.7%A tool's standard line about picking up merged changes
The subject line only3235.6%Deliverable, candidate number and cycle, nothing else

Who said no, and what happened when nobody answered?

The repository cannot say who said no. OpenStack's reviewer guide states that "Releases from someone other than the PTL or release liaison must be acknowledged by one of them with a +1 vote in gerrit," naming the two team contacts it lists for each deliverable, and a check-release-approval job verifies that approval when a request is submitted. Those votes and job results live in the code review system. The public git mirror carries branch refs only, no review notes, so none of the 90 holds names its approver in git, and the author of a request is not necessarily the person with authority to stop a release.

The rule for silence is recorded unevenly too. For last-minute candidates the process document says "Patches that get no feedback by the deadline should be abandoned," a sentence that entered the document in an edit dated October 10, 2019 and has not been removed since. Yet 20 generated hold requests in the 2023.1 to 2024.1 cycles told teams that requests with no response would be merged by a stated Friday, and 3 requests in 2026.2 repeat the abandon rule. A merged hold from those earlier cycles may record an explicit yes or nobody objecting in time, and the release record cannot tell which.

Other projects split the decision from its reason in the same way. Python's release schedule for version 3.13, PEP 719, records an added third release candidate and moves the final release from October 1 to October 7, 2024, in commit c778ab9, titled "PEP 719: Add 3.13.0rc3, postpone 3.13.0 final." The commit has no message body. The schedule proves a hold happened; anyone who wants the reason has to join it from somewhere else.

For a dataset, the decider field therefore has to come from the review system's export, stored as a role rather than a name, and the silence rule has to be stored with each decision as it stood on that date, not inferred from today's process page.

How do you score a hold when the failure never happened?

Score the evidence for the catch, not the catch itself, because the failure a hold prevents is never observed. A hold that removed a real defect and a hold that delayed a healthy candidate look identical in production: both ship clean. What differs is the chain from reason to fix to a test on the shipped artifact, so the outcome label should grade that chain and name the link that is missing.

Six labels cover the cases a release record produces. Confirmed catch demands the most and is rare in public data: none of OpenStack's 90 holds can earn it from the release repository alone, because the failing tests and bug reports sit in other systems. Declined hold is the label most datasets lack. OpenStack's process says that a -1 on a last-minute candidate "will mean that the PTL prefers to wait for a post-release stable update," a decision to ship and patch later that never reaches git, because the request is not merged.

A dataset that cannot link the supporting systems should say so in the label rather than upgrade a plausible catch to a confirmed one. The rubric, with what each label requires and what a model can safely learn from it:

How do you score a hold when the failure never happened?
Outcome labelEvidence requiredWhat a model may learnSupportable from the OpenStack release repository alone?
Confirmed catchDefect reproduced on the held candidate, fix in the next one, test passing on what shippedThat this kind of evidence justifies a stopNo: tests and bug reports live elsewhere
Plausible catchA named defect or advisory linked to a fix in the next candidate, not reproducedThat a stated risk was acted on, not that it was realPartly: 7 holds pair a reason word with a link
Process holdA release-mechanics fault such as notes, packaging, metadata or uploadThat shipping mechanics can stop a releaseYes, where the message says so
Hygiene holdMerged changes picked up with no named defectLittle about risk, more about cadenceYes, where the message says so
Declined holdA stop that was proposed and refused, plus what happened after releaseThat shipping now and patching later was judged acceptableNo: refused requests never merge
UnscorableNo reason word, link or change listNothing; keep out of outcome labelsYes: 44 of 90 holds

Does a later stable patch prove a hold was wrong?

No, and reading it that way would teach a model the wrong lesson. In the seven cycles released at least 180 days before our snapshot, 60 of 77 held deliverables, 77.9 percent, later published a stable point release in the same cycle, against 136 of 479 deliverables that shipped their first candidate, 28.4 percent.

The tempting reading, that holds fail to stop defects, confuses where problems cluster with what holds achieve. Deliverables under active repair both trigger holds and ship patches; quiet deliverables do neither. The numbers fit that explanation at least as well as any other, and the release repository contains nothing that could separate the two, such as whether a later patch fixed the same issue that triggered the hold.

The same trap sits inside companies. A hold that ends with a green test suite and a closed ticket can still leave the originating problem in place if the fix treated a symptom. The record should tie any post-release fix back to the hold's stated reason, and the outcome label should change only when that tie is shown, not when a ticket closes.

Holds also cost something, and the dataset should keep the cost. OpenStack's process asks teams to avoid more than three release candidates "so we are not creating candidates that consumers are then trained to ignore." A model trained only on stops that turned out well learns to stop too often, so go decisions, declined holds and their outcomes belong in the same dataset as counterweights.

What should a buyer test before licensing hold records?

Test whether each case can be rebuilt by someone who was not in the room. Draw a random sample of holds with a published seed and a stated sort order, then check every sampled case against the list below before reading any summary statistic.

Then check the denominators. A hold rate needs the number of candidates that reached the decision point, not only those that stopped, and a dataset of stops alone cannot teach when to go. Ask how go decisions from the same releases are represented, how declined holds were recovered from the review system, and what the security scope removed: details of vulnerabilities not yet fixed, internal hostnames, customer names and personal data in approval threads. This acceptance test, written into the request, gives the supplier a target and the buyer a reason to refuse a delivery that misses it:

  • The stopped and the re-cut candidates each resolve to a commit or artifact digest.
  • The stated reason and its evidence carry timestamps earlier than the hold decision.
  • The decider appears as a role, with the authority rule and its source cited.
  • The silence rule in force on the decision date is stored with the case.
  • Go decisions and declined holds are present, with counts for each release.
  • Every outcome label comes from one fixed label set and names any missing evidence.
  • Vulnerability details, hostnames and personal data are removed under a written scope.

Where do stop decisions live inside a software company?

In four places that rarely point at each other: the release or deployment ticket that names the candidate, the CI and test results that prompted the stop, the approval thread in code review or chat where someone wrote not yet, and the next release entry showing what actually shipped. Postmortems add the after-effects when a hold was declined and something broke; our pages on incident timelines and on uncertain root causes cover those records.

Joining the four is the real work. A company with years of releases may have made hundreds of stop decisions, but they form a dataset only if the approval system and the release log share identifiers that a script can match. That join, more than the number of releases, decides whether the archive is worth preparing. A team that kept release notes but deleted its review history can still supply candidates and outcomes, and should say plainly that deciders are missing.

SourceX sources datasets for AI buyers on request rather than holding them in stock, and nothing is delivered until the company supplying the records has approved the scope and signed. A buyer that needs release-hold cases should describe the decision points, the outcome labels and the acceptance test above when it submits a request; whether a given company's records can meet that test is settled only by reviewing them.

Methods note: OpenStack repository, counting rules, limits

Data: openstack/releases, cloned from its GitHub mirror on October 27, 2026 at commit 9da6d9cae346 (committed 09:46 UTC that day). Scope: series in doc/source/series_status.yaml with release ids 2023.1 to 2026.2, and deliverable files under deliverables/<series>/ whose release-model is cycle-with-rc. A candidate is a version matching X.Y.Z.0rcN; a hold is every candidate numbered 2 or higher. The commit that introduced a hold is the earliest non-merge commit, read with git log --full-history and each commit compared with its first parent, that adds that version line to that file; every hold matched exactly one such commit, 78 distinct commits for 90 holds.

Classification: after dropping Change-Id, Signed-off-by, Co-Authored-By and Generated-By lines, the script removes change-list lines (git log commands, lines that begin with a 7 to 40 character hexadecimal hash, a change log header), two generated sentences, the ack sentence, the merge-on-silence and abandon-on-silence sentences, link footers and URLs, and prints how many holds each removal touched. A hold has a link when its message carries a URL on a bug tracker, code review or code hosting site, or a Depends-On, Closes-Bug, Partial-Bug or Related-Bug footer. A reason word is a whole-word, case-insensitive match on one of 21 words (bug, bugs, fix, fixes, fixed, fixing, blocker, regression, regressions, CVE, security, critical, broken, breaks, failing, failure, failures, issue, issues, vulnerability, crash) in the remaining text or the subject. Shares are computed from integers and rounded once, half up.

The later-release comparison counts any non-candidate version above the final in the same file, for the seven series released at least 180 days before the snapshot. Not measured: review votes, abandoned requests, bug reports, test results, the content of changes between candidates, and anything about any company's internal releases. The Python example comes from python/peps at commit 35d53a8. The scripts and a per-hold output file, which holds no names or message text, are kept for recount.

Frequently asked questions

What is a deployment stop decision dataset?

It is a collection of cases in which a team decided that a specific release candidate should not ship yet. Each case records the stopped candidate, the evidence weighed, who decided under which rule, and what shipped instead. It lets a model be trained and tested on when not yet is the right call, not only on code that shipped.

How is a release hold different from a rollback?

A hold stops a candidate before users receive it; a rollback undoes a change that already reached production. Holds leave weaker traces because nothing failed in production to open an incident, so their evidence has to be joined from release files, test results and approval threads.

Can the value of a release hold be measured?

Not directly, because the prevented failure never happens. What can be graded is the evidence chain: a defect reproduced on the held candidate, a fix in the next candidate and a passing test on the shipped artifact. Holds without that chain belong under a plausible or unscorable label, not counted as saves.

Why should a dataset about not deploying include decisions to ship?

A model shown only stops learns to stop. Go decisions from the same releases, plus holds that were proposed and declined, supply the denominators and counterexamples that let a model tell a justified stop from reflexive caution.

What has to come out of release hold records before they are licensed?

Details of vulnerabilities that are not yet fixed or disclosed, internal hostnames and infrastructure identifiers, customer names, and personal data such as names in approval threads. Deciders can be kept as roles. The supplying company's security team and counsel set the written scope before anything is shared.

Sources

  • OpenStack's release process: final cycle-with-rc releases re-tag the commit used for the last RC; teams should avoid more than 3 release candidates; last-minute candidate patches with no feedback by the deadline should be abandoned; a PTL -1 means waiting for a post-release stable update. Source
  • The cycle-with-rc model: a single release at the end of the cycle with one or more release candidates close to the end of the cycle. Source
  • Releases from someone other than the PTL or release liaison must be acknowledged by one of them with a +1 vote in Gerrit; a check-release-approval job checks that approval. Source
  • Deliverable files for the 2023.1 to 2026.2 cycles: 622 cycle-with-rc deliverables with candidates, 89 held at least once, 90 holds, every final on its last candidate's commit (counted by the article's script). Source
  • Release ids and initial release dates of the OpenStack series counted, from Antelope (2023.1, 2023-03-22) to Hibiscus (2026.2, 2026-09-30). Source
  • Python 3.13 release schedule (PEP 719) lists candidate 3 and the final release on 2024-10-07. Source
  • Commit c778ab9 to PEP 719, titled Add 3.13.0rc3, postpone 3.13.0 final, moved the final date from 2024-10-01 to 2024-10-07 and has no message body. Source
  • NIST AI RMF 1.0, MANAGE 1.1: a determination whether the AI system achieves its intended purposes and whether development or deployment should proceed; an initial go/no-go decision after MAP. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies