Insights
AI uses for records
531 guides · page 2 of 12
Dataset refresh incremental value study: set the cost bar first
A dataset refresh incremental value study design: cost ledger, value bar, task-category tests and six exhibits. SourceX has not yet published results.
Data pilot acceptance threshold: three numbers to sign before results
Set a data pilot acceptance threshold before results: the smallest gain worth paying for, a harm margin and a task count, with each outcome tied to a decision.
Repair or concession? Billing correction support labels that record who decided
Billing correction support labels that split fixed bills from refund concessions, with a 9,288-line ledger recount, a decision packet and a six-check rubric.
Incident fragmentation dataset sampling: draw incidents, not tickets
Incident fragmentation dataset sampling: set task boundaries, roll alerts and tickets up to incidents and size the sample by incident, not by fragment.
Dataset examples versus outcome labels: a 2x2 pilot before you buy
Dataset examples versus outcome labels: rule out non-data causes, fix the test set, then run a matched-budget 2x2 pilot to see which fix your model needs.
Customer concentration dataset bias: audit the task mix by customer tier
Customer concentration dataset bias hides in the task mix, not row counts. Audit tasks by de-identified customer tier, adjust for clustering, then pick a fix.
Enterprise answer citation benchmark: a claim-by-claim study protocol
A preregistered enterprise answer citation benchmark design: grade every material claim against the business record it cites. Results are not yet published.
Enterprise exception coverage benchmark: map the cases your evals skip
An enterprise exception coverage benchmark protocol: define four exception classes from workflows, count them against denominators, and brief the gaps.
Unseen policy agent evaluation: hold out rules, not just cases
Unseen policy agent evaluation needs held-out rules, not just held-out cases: a recount of two public benchmarks, a prior-intrusion score and a sealing plan.
Ask, condition or decline: a grant administration evidence dataset
A grant administration evidence dataset for eligibility conflicts: six labeled actions, nine scripted files and what 1,572 single-audit findings reveal.
Document version decision dataset: keep later edits out of the past
Build a document version decision dataset that pairs each decision with the text that existed then: two clocks, an edit-after test and labeled gap codes.
Dataset pilot hard case reporting: count every case before you average
Dataset pilot hard case reporting with a case-flow ledger, closed exclusion rules and a stratified result table that keeps difficult cases in the verdict.
Who ended the deal? Lost deal AI training labels by actor and timing
Lost deal AI training labels that separate a sound walk-away from a seller miss: an audit of four CRM defaults, a seven-class taxonomy and label rules.
AI data pilot severity metrics: when a better pass rate hides worse failures
AI data pilot severity metrics: a preregistered protocol and tier transition table that catch a dataset raising the pass rate while adding severe failures.
Counting the cleanup: an AI reconciliation review effort dataset
An AI reconciliation review effort dataset counts touches, overrides, reopens, open items and escaped errors per item; with a code audit, rules and a casebook.
Dataset expansion regression test: set loss budgets before you train
A dataset expansion regression test with loss budgets fixed before training, matched-compute arms and a decision grid: expand only if protected skills hold.
Enterprise question context experiment: count repairs, not replies
An enterprise question context experiment that removes one context field at a time and scores wrong actions and repair minutes, plus a public floor test.
Dataset pilot human time savings: time the reviewers, not the model
Measure dataset pilot human time savings by timing review, correction and redo work on blinded outputs, with a preset protocol and a table for the decision.
Dataset utility decay refresh decision: run a knee-and-ceiling test
Decide on automatic dataset refreshes with a decay replay: find where utility loss levels off and what each cadence costs, shown on 5,307 Kubernetes notes.
Incident dataset recovery experiment: held-out design and stop rules
An incident dataset recovery experiment tests whether past incident records speed safe recovery on held-out incidents. The full design, sized; no results.
Finished but forbidden: a procurement constraint dataset experiment
A procurement constraint dataset experiment with four matched data arms, a frozen held-out workflow test and a metric that counts only rule-abiding completions.
Two errors, one test set: a support escalation dataset experiment
A support escalation dataset experiment that counts missed escalations and needless handoffs apart on a protected test set, with paired tests and a rubric.
Missing attachment AI performance study: a paired three-arm protocol
A missing attachment AI performance study protocol: paired intact, missing and misleading-file arms, behavior metrics, and a recovery-cost worksheet.
Supplier clustered dataset sampling: count suppliers, not just records
Supplier clustered dataset sampling: why records from one company are not independent, how to compute effective sample size and a worksheet for coverage claims.
Specialist dataset pilot limits: which claims a small sample can carry
Specialist dataset pilot limits made explicit: define the sampled task population, preregister the rules and match every claim to the evidence it needs.
Failed test design decision dataset: what 2,219 slowdowns changed
A failed test design decision dataset ties each negative result to its decision and change. 2,219 Rust slowdowns show which links survive and which go missing.
Courtesy exception support training: one action, four reasons
Courtesy exception support training data must record why each concession was given. In 700 public support concessions, the action label never said.
Who could say yes? Labeling a sales commitment evidence dataset
A sales commitment evidence dataset should label each yes by its form and the speaker's authority: seven labels, an evidence map and an ERPNext field audit.
Accrued is not invoiced: a finance estimate uncertainty dataset
A finance estimate uncertainty dataset should mark each amount estimated or confirmed, date its evidence and classify the later difference. Casebook and rubric.
Collaboration intent dataset: four states between an idea and a yes
How to build a collaboration intent dataset that separates ideas from commitments, with counts from Ethereum upgrade stages and 210 meeting summaries.
Expert correction knowledge dataset: what 1,545 public fixes omit
An expert correction knowledge dataset links the original, the fix, its evidence and its approver. Public corrections name a checker in only 13% of cases.
Unresolved expert disagreement dataset: what a single label erases
An unresolved expert disagreement dataset keeps every vote. An audit of 7,675 public claim-evidence pairs shows what one label erases and how to test yours.
Real vs. synthetic exception benchmark: size the rare-case test first
Design a real vs. synthetic exception benchmark: matched compute and validation budgets, sealed real rare cases per family, and a case-count table for verdicts.
Build a reconciliation explanation dataset that catches forced balances
A reconciliation explanation dataset grades evidence, treatment and approval for every item, not totals. Five tools that book the difference show why.
What 3,648 CBO revisions teach a variance analysis evidence dataset
A variance analysis evidence dataset should test whether each explanation rebuilds from records. CBO's revisions show how bridges close, and where they break.
Support sentiment resolution dataset: 505 failed chats said thanks
In 1,980 public agent chats, failed tasks thanked the agent more often than completed ones. Build a support sentiment resolution dataset on five axes instead.
SourceX dataset refresh value: four tests before you request new records
SourceX dataset refresh value depends on what new records add. Four tests on change, coverage, maturity and repeats show what to request, or whether to skip.
Dataset pilot evaluation leakage: read the final test once
Dataset pilot evaluation leakage starts when tuning feedback touches the final test. A preregistered protocol, a feedback ledger and decision rules that hold.
One case, two problems: how a multi-issue support dataset keeps both
How a multi-issue support dataset records each problem's action and outcome, why stitched public multi-intent data fails, and a two-level grading rubric.
Outage support needs its own evaluation set, keyed to the incident clock
Outage support dataset evaluation: why incidents change the right reply, five contact types, a decision packet, a rubric and a count of 2,285 telecom tasks.
Cross-company dataset generalization: test held-out suppliers before scaling
Test cross-company dataset generalization before a bigger data order: hold out whole supplier companies, measure the transfer gap, then scale, broaden or stop.
Dataset value after a model upgrade: a matched retest of the gain
A dataset value model upgrade experiment in four arms: retest the gain on the new base model, measure what it absorbed, then keep, narrow or retire the data.
Dataset pilot vs current model baseline: the rival and its 10 settings
Your dataset pilot's current model baseline must be the strongest rival you could ship. Choose it, tune it as hard as the data arm and log its 10 settings.
Document lineage retrieval experiment: 73 spec questions, five rankers
A document lineage retrieval experiment with held-out families, five arms and stop rules, plus a public pilot where links beat recency on 73 spec questions.
Older records, worse odds? A CRM dataset forecast calibration experiment
A CRM dataset forecast calibration experiment, run on 41,188 public contacts: rolling holdouts, Brier scores and the baseline more history must beat.
Issue history code patch experiment: a design and a 237-task pilot
An issue history code patch experiment tests whether pre-fix issue comments help pick the right patch. Here is the design, and a public 237-task pilot.
Finance dataset error detection pilot: how to test correction history
A finance dataset error detection pilot design: matched records with and without correction history, a frozen test set, a reviewer rubric and reporting rules.
An accrual reversal temporal dataset for the month a reversal distorts
An accrual reversal temporal dataset links each accrual to its reversal and invoice across months, so a model reads the next month's figures correctly.