AI uses for records
Small branch representation in a dataset: measuring a local workflow
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Treat each branch as its own stratum. Define the population of cases the model will serve, sample or take every case from small offices instead of letting volume weighting drown them, then compare each branch's process variants, step presence, handoffs, timing and outcomes against the pooled pattern with intervals sized for small counts. Check what the branch never recorded, and test transfer by holding the branch out of training.
Agent run
Dataset small branch
Key takeaways
- A pooled average weights every case equally, so a small office's different workflow can vanish inside a healthy company-wide number.
- Stratify by the setting variables that change the work, such as system, role coverage, local rules and channel, and freeze the strata before looking at outcomes.
- Compare structure first: process variants, step presence, conformance and handoffs say more than average duration.
- Report counts and small-sample intervals beside every branch metric, and reconcile case counts against records the office cannot skip.
- Hold the branch out during evaluation; a model that fails there has learned the large office.
How do you measure workflow differences at a small branch or office?
Measure them case by case, with the branch as a reporting unit, not as a footnote to a company-wide average. Pull every completed case for the small office over a fixed window, rebuild the sequence of steps for each case from its event history, and compare that office's mix of process variants, required steps, handoffs, durations and outcomes with the rest of the company. Then ask what the office did that never reached the system at all.
That last step matters more at small offices than large ones. A six-person office often runs part of its work on a shared inbox, a local spreadsheet or the phone, because the central workflow was designed around the volume of the big sites. The records that do reach the system can look tidy and still describe only half of what the office actually does.
The result is a branch profile: a short, evidence-backed statement of how this office's work differs, how confident you can be given its case count, and which of its cases a dataset can and cannot represent. Everything below is the method for producing that profile and deciding what to do with it.
Why does a company-wide average hide the small office?
A pooled metric weights every case equally, so a branch that handles a small share of cases contributes the same small share to every average. If the head office closes nine of every ten cases, its habits are the dataset's habits, and a model trained on the pool learns the head office. A small office that skips a step, adds a local approval or routes work through a different role can disappear inside a number that looks healthy.
Model reporting research made the same point for model performance. The Model Cards paper proposes reporting evaluation results disaggregated across groups and conditions, including intersections of them, because an aggregate can conceal poor performance for a smaller group. A branch is exactly that kind of condition: a place where the same task is done under different staffing, customers, systems and local rules.
For some systems the expectation is written into law. Article 10(4) of the EU AI Act requires training, validation and testing data for high-risk systems to take into account, to the extent the intended purpose requires, the characteristics particular to the geographical, contextual, behavioral or functional setting where the system will be used. Most business workflow models are not high-risk systems under that Act, but the test is a sensible one for any buyer: if the model will run at the small office, the data should show how the small office works.
Which offices deserve their own stratum, and which can be grouped?
Give an office its own stratum when the model will be used there and something about its setting could change the work; group offices only when they share every setting variable that matters. Branch name alone is a weak stratifier. Two small offices on the same system, with the same product mix and the same approval rules, can be pooled as one stratum of small offices. An office that joined through an acquisition and still runs its own system cannot.
Write the stratification variables down before looking at outcomes, so the grouping is not chosen to make a difference appear or vanish. The table lists the variables that most often separate one office's workflow from another's and the record that shows each.
| Setting variable | Why it can change the workflow | Where the evidence usually sits | Stratify or group? |
|---|---|---|---|
| Headcount and role coverage | One person may hold intake, approval and closing, so separation-of-duty steps collapse or move | HR roster joined to case assignee fields | Stratify when the office has fewer distinct roles than the workflow assumes |
| System of record and version | An acquired office on a different CRM, ticketing or ERP instance records different steps and statuses | System inventory, instance IDs on each case | Always stratify; never pool cases from different systems without a mapping |
| Product or service mix | A rural office may handle more of one case type and almost none of another | Case type field by office and month | Stratify by office and case type when the mix differs materially |
| Local rules and approvals | State, county or regional requirements add or remove steps | Policy documents, approval matrix by location | Stratify; a local rule is a different process, not noise |
| Customer channel | Walk-in and phone work enters later or not at all | Channel field, call logs, front-desk sign-in sheets | Stratify when channel shares differ between offices |
| Hours and staffing pattern | Part-time coverage creates batching, delays and weekend gaps | Timestamps by weekday and hour | Group offices with similar patterns; flag outliers |
| Data entry practice | Back-dated entry or end-of-day batching distorts timing metrics | Gap between event time and created time | Stratify timing metrics; keep step-presence metrics pooled if unaffected |
What steps does a branch stratification and local review follow?
The plan below is the asset this page promises: nine steps that take a team from a vague concern about a small office to a documented branch profile. Each step names the decision it forces and the output a reviewer should be able to inspect. Run it once per office or office group, and date the output, because staffing and systems change.
| Step | Question it answers | Method | Output a reviewer checks |
|---|---|---|---|
| 1. Fix the population | Which cases will the model see in use? | Name the case types, offices, channels and date range the model will serve; list exclusions with reasons | A one-paragraph population statement and exclusion list |
| 2. Build the sampling frame | Which records could be drawn? | List every system and side channel per office; reconcile counts against an independent source such as invoices or call logs | Frame table with counts per office, system and channel |
| 3. Set strata | Which offices are compared separately? | Apply the setting variables chosen in advance | Stratum definitions frozen before outcome analysis |
| 4. Allocate the sample | How many cases per stratum? | Take every case from small strata when feasible; otherwise allocate a fixed minimum per stratum and record design weights | Allocation table with design weights |
| 5. Rebuild case histories | What actually happened in each case? | Order events by timestamp per case ID; resolve merges, reopens and transfers | Event log with case ID, activity, timestamp, actor role, office |
| 6. Compare variants and steps | Does the office follow a different path? | Variant counts, step presence rates and conformance against the reference process | Variant table and step-presence table per stratum |
| 7. Compare timing and outcomes | Does the office reach different results or take different time? | Durations, reopen and rework rates, outcome fields, with intervals for small counts | Metric table with counts and interval bounds |
| 8. Review locally | Why is it different? | Walk through a sample of divergent cases with the office lead and a central process owner | Signed review notes classifying each difference |
| 9. Decide and record | What does the dataset do with this office? | Keep as its own slice, reweight, exclude with a stated limit, or collect more | Branch profile with decision, limits and review date |
How many cases does a small office need before its numbers mean anything?
Fewer than most teams fear, as long as you report the uncertainty instead of hiding it. A small office rarely gives you enough cases for precise rates, so the goal is to show whether a difference is large enough to survive a wide interval, not to estimate the office's rate to a decimal place.
For rates such as the share of cases that skip a step, report an interval designed for small counts. The statsmodels proportion_confint function, for example, offers Wilson score and Clopper-Pearson intervals; its documentation notes that the Clopper-Pearson exact interval has coverage of at least the nominal level but is generally conservative. Whichever method you choose, name it in the profile so the next reviewer can recompute it.
If the office's whole history is small, take all of it rather than sampling. Proportional allocation, where each office gets a share of the sample matching its share of cases, is what makes small offices vanish in the first place. A fixed minimum per stratum, with design weights recorded so pooled estimates can still be rebuilt, keeps both views available. When even a full census leaves too few cases of a given type, say so and stop: an interval that spans most of the range is a finding, and the right response is a scope limit or more collection, not a confident claim.
- Report the count of cases behind every branch metric, next to the metric.
- Name the interval method and confidence level once, at the top of the profile.
- Keep design weights in the dataset so a buyer can rebuild pooled estimates.
- Mark any metric with fewer cases than your stated minimum as descriptive only.
Which comparisons reveal a different workflow, and in what order?
Compare structure first and speed second. A different sequence of steps is the strongest sign of a different workflow; a slower average can come from staffing, batching or data entry habits and needs more care before it is called a process difference.
Process mining libraries make the structural comparison routine. In PM4Py, get_variants returns the distinct activity sequences in an event log with their counts, which gives each office's variant mix. Conformance checking by token-based replay matches each trace against a reference process model and reports where tokens were missing or left over; PM4Py's fitness function returns the share of traces that fit and an overall log fitness. Run both per stratum against the same reference model, so the comparison measures the office and not the model.
| Measure | How it is computed | What a difference suggests | Check before concluding |
|---|---|---|---|
| Variant mix | Count distinct step sequences per office; compare top variants and their shares | The office follows its own path for some case types | Variants split by case type; a different mix of work can mimic a different process |
| Step presence | Share of cases in which each required step appears | A step is skipped, merged or done outside the system | Whether the step happens elsewhere: paper, email or another system |
| Conformance fitness | Token-based replay of each case against the reference model | The office deviates from the documented process | Whether the reference model reflects current policy |
| Handoff pattern | Sequence of actor roles per case | One person holds roles that are split elsewhere | Role field quality; shared logins hide handoffs |
| Duration | Time between defined start and end events | Local delays, batching or extra approvals | Gap between event time and record creation time |
| Rework and reopen rate | Cases reopened or returned to an earlier step | Quality or rule differences at the office | Whether outcomes have matured; recent cases reopen later |
Which of the office's cases never reach the archive?
Usually the cases the small office handles outside the system, and the cases that are too recent to have an outcome. The archive represents the work that reached it. At a small office, walk-in requests resolved at the counter, urgent jobs handled by phone and exceptions settled by the office manager by email often leave no case record, or a record created after the fact with half the steps missing.
The Datasheets for Datasets questions are a useful prompt here: they ask whether a dataset contains all possible instances or a sample of a larger set, and how the sample relates to that larger set. Answer them per office. Reconcile the office's case count against something it cannot skip, such as invoices, appointment books, call logs or shipment records. A gap between the two is the office's unrecorded work, and its size belongs in the branch profile.
Timing also biases the picture. Outcome fields mature at different speeds, so an office whose cases close slowly will look better than it is if the window ends too early. Set a maturity cut-off per case type and exclude cases younger than it from outcome metrics, while keeping them for step and variant analysis.
What does a branch profile show, and when is the gap just data entry?
Illustrative example: a property management company runs one large regional office and four small ones, and wants a model that drafts the next action on maintenance requests. All figures here are invented counts for illustration. Over twelve months the large office logged 9,400 requests and the smallest office 310. Pooled, about 4 percent of requests lacked a vendor-assignment step, which looked like data noise.
Profiled separately, the smallest office lacked that step on 41 percent of its requests. The local review explained why: its lead technician fixed minor issues on the spot and closed the request, and an owner-approval step required by a local rule appeared only there. Reconciliation against technician timesheets found roughly 60 visits with no request record at all. The office's interval was wide, but nowhere near the pooled rate. The decision was to keep the office as its own slice, add the local approval as a labeled variant, and document the unrecorded visits as a known gap.
The exception is a difference produced by recording rather than work. A second small office showed long durations, but the gap between event time and record creation time showed the manager entered the day's requests each evening. The work itself matched the large office. Treating that as a distinct workflow would have taught a model a delay that never happened; the right fix was to use event times and note the entry practice.
Does a model trained on other offices work at the small one?
Hold the office out and test on it. Group-aware splits make this mechanical: scikit-learn's LeaveOneGroupOut trains on every group except one, and GroupKFold keeps each group entirely within a single test fold, so no office's cases leak into both training and testing. Run the evaluation with offices as groups and report each office's result with its case count.
A model that does well on random splits and poorly with the small office held out has learned the large office. That is the signal to collect more of the small office's cases, add its variants as labeled examples, or restrict the model's stated scope to the settings it was tested on.
Stop and bring in a statistician and the office's process owner when strata are too thin to separate a difference from chance, when a difference traces to a local legal rule, or when reconciliation shows a large share of the office's work outside the system. Narrow the scope when the model will not run at that office anyway; a documented exclusion is better than an untested claim of coverage.
What should a request for branch-stratified records ask for?
A buyer who needs small offices represented should say so in the request: name the settings that matter, ask for an office or office-group identifier on every case, and ask the supplier to state per office what share of work reaches the system. A dataset that cannot carry an office field cannot support any of the analysis above.
Branch-stratified records are sourced to order: SourceX matches each buyer's request to U.S. businesses that hold such records, and the request is where scope, volume, format, timing and licensing needs are written down, an office identifier and per-office minimums included. Each contributing business approves its own scope and terms and signs before handing over any records.
Frequently asked questions
What is branch stratification in a training dataset?
It means treating each branch, or each group of similar branches, as a separate stratum when sampling and reporting, so every stratum has enough cases to describe. Pooled estimates can still be rebuilt with design weights, but small offices are no longer swamped by the largest site.
How do I know a small office's workflow is really different?
Look for structural evidence first: a different mix of step sequences, a required step that is missing or added, or a different handoff pattern. Confirm it with a local review of divergent cases, and check that timing differences are not caused by end-of-day data entry.
What if a branch has too few cases to analyze?
Take every case it has, report the counts and interval bounds, and say plainly when the interval is too wide to support a conclusion. Then either collect more history, group it with offices that share its settings, or exclude it from the model's stated scope.
Should small branches be oversampled?
Usually yes for analysis and evaluation, with design weights recorded so pooled rates can still be estimated. For training, whether to upweight depends on where the model will be used; match the deployment population and document the choice.
Does a branch need its own model?
Rarely. More often the shared model needs the branch's variants as labeled examples and an evaluation that holds the branch out. A separate model makes sense only when the office runs a different system or rules that the main workflow cannot express.
Sources
- Article 10(4) of the EU AI Act requires data sets for high-risk systems to take into account the characteristics particular to the geographical, contextual, behavioral or functional setting of intended use. Source
- The Model Cards paper proposes reporting model evaluation disaggregated across groups and conditions, including intersections, so aggregate results do not hide weaker subgroup performance. Source
- Datasheets for Datasets asks whether a dataset contains all possible instances or is a sample of a larger set, alongside questions on composition and the collection process. Source
- PM4Py's get_variants retrieves the distinct activity sequences (variants) from an event log with their counts. Source
- PM4Py's token-based replay matches each trace against a Petri net and reports missing or remaining tokens; its fitness function returns the share of fitting traces and log fitness. Source
- statsmodels proportion_confint offers Wilson score and Clopper-Pearson intervals, and notes the Clopper-Pearson exact interval has at least nominal coverage but is generally conservative. Source
- scikit-learn's LeaveOneGroupOut trains on all samples except one group, and GroupKFold places each group in the test set exactly once with non-overlapping groups. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.