Skip to content

AI uses for records

Small branch representation in a dataset: measuring a local workflow

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Treat each branch as its own stratum. Define the population of cases the model will serve, sample or take every case from small offices instead of letting volume weighting drown them, then compare each branch's process variants, step presence, handoffs, timing and outcomes against the pooled pattern with intervals sized for small counts. Check what the branch never recorded, and test transfer by holding the branch out of training.

Agent run

Dataset small branch

This page covers small branch representation in a dataset: measuring a local workflow.Illustrative

Key takeaways

  • A pooled average weights every case equally, so a small office's different workflow can vanish inside a healthy company-wide number.
  • Stratify by the setting variables that change the work, such as system, role coverage, local rules and channel, and freeze the strata before looking at outcomes.
  • Compare structure first: process variants, step presence, conformance and handoffs say more than average duration.
  • Report counts and small-sample intervals beside every branch metric, and reconcile case counts against records the office cannot skip.
  • Hold the branch out during evaluation; a model that fails there has learned the large office.

How do you measure workflow differences at a small branch or office?

Measure them case by case, with the branch as a reporting unit, not as a footnote to a company-wide average. Pull every completed case for the small office over a fixed window, rebuild the sequence of steps for each case from its event history, and compare that office's mix of process variants, required steps, handoffs, durations and outcomes with the rest of the company. Then ask what the office did that never reached the system at all.

That last step matters more at small offices than large ones. A six-person office often runs part of its work on a shared inbox, a local spreadsheet or the phone, because the central workflow was designed around the volume of the big sites. The records that do reach the system can look tidy and still describe only half of what the office actually does.

The result is a branch profile: a short, evidence-backed statement of how this office's work differs, how confident you can be given its case count, and which of its cases a dataset can and cannot represent. Everything below is the method for producing that profile and deciding what to do with it.

Why does a company-wide average hide the small office?

A pooled metric weights every case equally, so a branch that handles a small share of cases contributes the same small share to every average. If the head office closes nine of every ten cases, its habits are the dataset's habits, and a model trained on the pool learns the head office. A small office that skips a step, adds a local approval or routes work through a different role can disappear inside a number that looks healthy.

Model reporting research made the same point for model performance. The Model Cards paper proposes reporting evaluation results disaggregated across groups and conditions, including intersections of them, because an aggregate can conceal poor performance for a smaller group. A branch is exactly that kind of condition: a place where the same task is done under different staffing, customers, systems and local rules.

For some systems the expectation is written into law. Article 10(4) of the EU AI Act requires training, validation and testing data for high-risk systems to take into account, to the extent the intended purpose requires, the characteristics particular to the geographical, contextual, behavioral or functional setting where the system will be used. Most business workflow models are not high-risk systems under that Act, but the test is a sensible one for any buyer: if the model will run at the small office, the data should show how the small office works.

Which offices deserve their own stratum, and which can be grouped?

Give an office its own stratum when the model will be used there and something about its setting could change the work; group offices only when they share every setting variable that matters. Branch name alone is a weak stratifier. Two small offices on the same system, with the same product mix and the same approval rules, can be pooled as one stratum of small offices. An office that joined through an acquisition and still runs its own system cannot.

Write the stratification variables down before looking at outcomes, so the grouping is not chosen to make a difference appear or vanish. The table lists the variables that most often separate one office's workflow from another's and the record that shows each.

Which offices deserve their own stratum, and which can be grouped?
Setting variableWhy it can change the workflowWhere the evidence usually sitsStratify or group?
Headcount and role coverageOne person may hold intake, approval and closing, so separation-of-duty steps collapse or moveHR roster joined to case assignee fieldsStratify when the office has fewer distinct roles than the workflow assumes
System of record and versionAn acquired office on a different CRM, ticketing or ERP instance records different steps and statusesSystem inventory, instance IDs on each caseAlways stratify; never pool cases from different systems without a mapping
Product or service mixA rural office may handle more of one case type and almost none of anotherCase type field by office and monthStratify by office and case type when the mix differs materially
Local rules and approvalsState, county or regional requirements add or remove stepsPolicy documents, approval matrix by locationStratify; a local rule is a different process, not noise
Customer channelWalk-in and phone work enters later or not at allChannel field, call logs, front-desk sign-in sheetsStratify when channel shares differ between offices
Hours and staffing patternPart-time coverage creates batching, delays and weekend gapsTimestamps by weekday and hourGroup offices with similar patterns; flag outliers
Data entry practiceBack-dated entry or end-of-day batching distorts timing metricsGap between event time and created timeStratify timing metrics; keep step-presence metrics pooled if unaffected

What steps does a branch stratification and local review follow?

The plan below is the asset this page promises: nine steps that take a team from a vague concern about a small office to a documented branch profile. Each step names the decision it forces and the output a reviewer should be able to inspect. Run it once per office or office group, and date the output, because staffing and systems change.

What steps does a branch stratification and local review follow?
StepQuestion it answersMethodOutput a reviewer checks
1. Fix the populationWhich cases will the model see in use?Name the case types, offices, channels and date range the model will serve; list exclusions with reasonsA one-paragraph population statement and exclusion list
2. Build the sampling frameWhich records could be drawn?List every system and side channel per office; reconcile counts against an independent source such as invoices or call logsFrame table with counts per office, system and channel
3. Set strataWhich offices are compared separately?Apply the setting variables chosen in advanceStratum definitions frozen before outcome analysis
4. Allocate the sampleHow many cases per stratum?Take every case from small strata when feasible; otherwise allocate a fixed minimum per stratum and record design weightsAllocation table with design weights
5. Rebuild case historiesWhat actually happened in each case?Order events by timestamp per case ID; resolve merges, reopens and transfersEvent log with case ID, activity, timestamp, actor role, office
6. Compare variants and stepsDoes the office follow a different path?Variant counts, step presence rates and conformance against the reference processVariant table and step-presence table per stratum
7. Compare timing and outcomesDoes the office reach different results or take different time?Durations, reopen and rework rates, outcome fields, with intervals for small countsMetric table with counts and interval bounds
8. Review locallyWhy is it different?Walk through a sample of divergent cases with the office lead and a central process ownerSigned review notes classifying each difference
9. Decide and recordWhat does the dataset do with this office?Keep as its own slice, reweight, exclude with a stated limit, or collect moreBranch profile with decision, limits and review date

How many cases does a small office need before its numbers mean anything?

Fewer than most teams fear, as long as you report the uncertainty instead of hiding it. A small office rarely gives you enough cases for precise rates, so the goal is to show whether a difference is large enough to survive a wide interval, not to estimate the office's rate to a decimal place.

For rates such as the share of cases that skip a step, report an interval designed for small counts. The statsmodels proportion_confint function, for example, offers Wilson score and Clopper-Pearson intervals; its documentation notes that the Clopper-Pearson exact interval has coverage of at least the nominal level but is generally conservative. Whichever method you choose, name it in the profile so the next reviewer can recompute it.

If the office's whole history is small, take all of it rather than sampling. Proportional allocation, where each office gets a share of the sample matching its share of cases, is what makes small offices vanish in the first place. A fixed minimum per stratum, with design weights recorded so pooled estimates can still be rebuilt, keeps both views available. When even a full census leaves too few cases of a given type, say so and stop: an interval that spans most of the range is a finding, and the right response is a scope limit or more collection, not a confident claim.

  • Report the count of cases behind every branch metric, next to the metric.
  • Name the interval method and confidence level once, at the top of the profile.
  • Keep design weights in the dataset so a buyer can rebuild pooled estimates.
  • Mark any metric with fewer cases than your stated minimum as descriptive only.

Which comparisons reveal a different workflow, and in what order?

Compare structure first and speed second. A different sequence of steps is the strongest sign of a different workflow; a slower average can come from staffing, batching or data entry habits and needs more care before it is called a process difference.

Process mining libraries make the structural comparison routine. In PM4Py, get_variants returns the distinct activity sequences in an event log with their counts, which gives each office's variant mix. Conformance checking by token-based replay matches each trace against a reference process model and reports where tokens were missing or left over; PM4Py's fitness function returns the share of traces that fit and an overall log fitness. Run both per stratum against the same reference model, so the comparison measures the office and not the model.

Which comparisons reveal a different workflow, and in what order?
MeasureHow it is computedWhat a difference suggestsCheck before concluding
Variant mixCount distinct step sequences per office; compare top variants and their sharesThe office follows its own path for some case typesVariants split by case type; a different mix of work can mimic a different process
Step presenceShare of cases in which each required step appearsA step is skipped, merged or done outside the systemWhether the step happens elsewhere: paper, email or another system
Conformance fitnessToken-based replay of each case against the reference modelThe office deviates from the documented processWhether the reference model reflects current policy
Handoff patternSequence of actor roles per caseOne person holds roles that are split elsewhereRole field quality; shared logins hide handoffs
DurationTime between defined start and end eventsLocal delays, batching or extra approvalsGap between event time and record creation time
Rework and reopen rateCases reopened or returned to an earlier stepQuality or rule differences at the officeWhether outcomes have matured; recent cases reopen later

Which of the office's cases never reach the archive?

Usually the cases the small office handles outside the system, and the cases that are too recent to have an outcome. The archive represents the work that reached it. At a small office, walk-in requests resolved at the counter, urgent jobs handled by phone and exceptions settled by the office manager by email often leave no case record, or a record created after the fact with half the steps missing.

The Datasheets for Datasets questions are a useful prompt here: they ask whether a dataset contains all possible instances or a sample of a larger set, and how the sample relates to that larger set. Answer them per office. Reconcile the office's case count against something it cannot skip, such as invoices, appointment books, call logs or shipment records. A gap between the two is the office's unrecorded work, and its size belongs in the branch profile.

Timing also biases the picture. Outcome fields mature at different speeds, so an office whose cases close slowly will look better than it is if the window ends too early. Set a maturity cut-off per case type and exclude cases younger than it from outcome metrics, while keeping them for step and variant analysis.

What does a branch profile show, and when is the gap just data entry?

Illustrative example: a property management company runs one large regional office and four small ones, and wants a model that drafts the next action on maintenance requests. All figures here are invented counts for illustration. Over twelve months the large office logged 9,400 requests and the smallest office 310. Pooled, about 4 percent of requests lacked a vendor-assignment step, which looked like data noise.

Profiled separately, the smallest office lacked that step on 41 percent of its requests. The local review explained why: its lead technician fixed minor issues on the spot and closed the request, and an owner-approval step required by a local rule appeared only there. Reconciliation against technician timesheets found roughly 60 visits with no request record at all. The office's interval was wide, but nowhere near the pooled rate. The decision was to keep the office as its own slice, add the local approval as a labeled variant, and document the unrecorded visits as a known gap.

The exception is a difference produced by recording rather than work. A second small office showed long durations, but the gap between event time and record creation time showed the manager entered the day's requests each evening. The work itself matched the large office. Treating that as a distinct workflow would have taught a model a delay that never happened; the right fix was to use event times and note the entry practice.

Does a model trained on other offices work at the small one?

Hold the office out and test on it. Group-aware splits make this mechanical: scikit-learn's LeaveOneGroupOut trains on every group except one, and GroupKFold keeps each group entirely within a single test fold, so no office's cases leak into both training and testing. Run the evaluation with offices as groups and report each office's result with its case count.

A model that does well on random splits and poorly with the small office held out has learned the large office. That is the signal to collect more of the small office's cases, add its variants as labeled examples, or restrict the model's stated scope to the settings it was tested on.

Stop and bring in a statistician and the office's process owner when strata are too thin to separate a difference from chance, when a difference traces to a local legal rule, or when reconciliation shows a large share of the office's work outside the system. Narrow the scope when the model will not run at that office anyway; a documented exclusion is better than an untested claim of coverage.

What should a request for branch-stratified records ask for?

A buyer who needs small offices represented should say so in the request: name the settings that matter, ask for an office or office-group identifier on every case, and ask the supplier to state per office what share of work reaches the system. A dataset that cannot carry an office field cannot support any of the analysis above.

Branch-stratified records are sourced to order: SourceX matches each buyer's request to U.S. businesses that hold such records, and the request is where scope, volume, format, timing and licensing needs are written down, an office identifier and per-office minimums included. Each contributing business approves its own scope and terms and signs before handing over any records.

Frequently asked questions

What is branch stratification in a training dataset?

It means treating each branch, or each group of similar branches, as a separate stratum when sampling and reporting, so every stratum has enough cases to describe. Pooled estimates can still be rebuilt with design weights, but small offices are no longer swamped by the largest site.

How do I know a small office's workflow is really different?

Look for structural evidence first: a different mix of step sequences, a required step that is missing or added, or a different handoff pattern. Confirm it with a local review of divergent cases, and check that timing differences are not caused by end-of-day data entry.

What if a branch has too few cases to analyze?

Take every case it has, report the counts and interval bounds, and say plainly when the interval is too wide to support a conclusion. Then either collect more history, group it with offices that share its settings, or exclude it from the model's stated scope.

Should small branches be oversampled?

Usually yes for analysis and evaluation, with design weights recorded so pooled rates can still be estimated. For training, whether to upweight depends on where the model will be used; match the deployment population and document the choice.

Does a branch need its own model?

Rarely. More often the shared model needs the branch's variants as labeled examples and an evaluation that holds the branch out. A separate model makes sense only when the office runs a different system or rules that the main workflow cannot express.

Sources

  • Article 10(4) of the EU AI Act requires data sets for high-risk systems to take into account the characteristics particular to the geographical, contextual, behavioral or functional setting of intended use. Source
  • The Model Cards paper proposes reporting model evaluation disaggregated across groups and conditions, including intersections, so aggregate results do not hide weaker subgroup performance. Source
  • Datasheets for Datasets asks whether a dataset contains all possible instances or is a sample of a larger set, alongside questions on composition and the collection process. Source
  • PM4Py's get_variants retrieves the distinct activity sequences (variants) from an event log with their counts. Source
  • PM4Py's token-based replay matches each trace against a Petri net and reports missing or remaining tokens; its fitness function returns the share of fitting traces and log fitness. Source
  • statsmodels proportion_confint offers Wilson score and Clopper-Pearson intervals, and notes the Clopper-Pearson exact interval has at least nominal coverage but is generally conservative. Source
  • scikit-learn's LeaveOneGroupOut trains on all samples except one group, and GroupKFold places each group in the test set exactly once with non-overlapping groups. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies