Skip to content

AI uses for records

Customer concentration dataset bias: audit the task mix by customer tier

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Customer concentration dataset bias usually shows up in whose tasks a dataset teaches, not in how many rows one account owns. Tag each task with permitted, de-identified segment attributes such as plan tier, tenure band and industry, compare each segment's task mix with the population the model will serve, shrink the sample size for clustering within accounts, and fix only the gaps that would change a decision.

Checklist

Customer concentration

This page covers customer concentration dataset bias: audit the task mix by customer tier.Illustrative

Key takeaways

  • High-value accounts leave more, longer and better-documented records, so a whole tier can set a dataset's task mix even when no single account holds a large share of rows.
  • The right comparison depends on how the model is deployed, per task, per user or per account; choose the reference distribution before measuring anything.
  • Banded attributes such as plan tier and tenure band are enough for the audit; customer names are unnecessary, and thin segment cells should be merged or suppressed.
  • Tasks from one account are correlated: in the illustrative archive, 46,000 enterprise tickets from 80 accounts carry about the evidence of 1,550 independent ones.
  • Rule out exposure, repetition and export filters before calling a skew bias, then accept, stratify, reweight, supplement or narrow the claim.

Do your best customers dominate the dataset?

Often they do, and the dominance sits in the task mix more than in the row count. Accounts that pay the most tend to have more seats, longer tenure and premium support, so they generate more records, keep them for more years and get fuller notes from dedicated staff. A dataset cut from the archive inherits that weighting unless someone measures it.

The decision this audit supports is narrow: does the distribution of tasks in the dataset match the distribution the model will meet once deployed, and if not, which segments are over-taught or under-taught? A useful answer needs four pieces of evidence: a stated reference population, a task label on every sampled unit, a permitted segment attribute on every unit, and an estimate of how strongly tasks cluster within accounts. Without the first, no share is too high or too low.

Weighting toward valuable work is not wrong when it is declared. OpenAI built GDPval from 44 occupations in the nine industries contributing most to U.S. GDP, a frame that openly favors economically weighty work, and it documents that frame alongside a set of 1,320 tasks. The trouble in business archives is the undeclared version: a dataset that leans toward top-tier customers because of how records were created and kept, while its documentation implies it speaks for every customer.

Why row-count concentration misses the problem

A per-account share answers a different question. The guide to dataset customer concentration coverage measures how much of an archive each identified customer contributes, with top-one and top-five shares and a held-out customer test. Run it first; then ask what the accounts in each tier ask for, because a tier can dominate without any member dominating.

Picture a support archive in which no account exceeds 2 percent of tickets. If 80 enterprise accounts together supply 46 percent, and only enterprise customers file single sign-on, procurement portal and contract exception tickets, a model trained on the archive will treat those tasks as routine. Deployed to self-serve small businesses, it will suggest escalation paths and account manager handoffs those customers do not have.

The reverse also happens. One very large account can hold 15 percent of records and do no harm when its task mix mirrors everyone else's. Large record counts can also hide repetition: a single integration that opens a ticket for every failed sync can produce thousands of near-identical tasks that inflate one tier's share without adding any coverage.

Which customer attributes can the audit use without identifying anyone?

Use coarse, banded attributes that both the license and the privacy review permit, and no customer names. Plan tier, contract size band, tenure band, seat-count band, broad industry, region and sales channel are usually enough. The supplier can derive them inside its own environment and attach them to each task before anything leaves, so the buyer never learns who the customer was.

Clustering analysis needs one more field: a pseudonymous account key, generated by the supplier and meaningless outside the dataset, so tasks from the same account can be grouped. Whether even that key may travel is a contract question, not a technical one. De-identifying an attribute does not create a right to license it; the attribute has to sit inside the agreed scope.

Small cells can point to a specific business. If only three enterprise healthcare accounts exist in one region, a cell for that combination identifies them. Health data publishers handle the same problem with suppression rules: the data use terms of the Centers for Medicare and Medicaid Services bar displaying any cell of 10 or fewer, along with calculations that would reveal one. An audit report can borrow that logic by counting accounts rather than records and merging or suppressing any segment-task cell with fewer than 11 accounts.

The sampling worksheet for a customer concentration audit

The worksheet records every choice that can move the answer before anyone computes a share. Datasheets for Datasets, the documentation questionnaire proposed by Gebru and colleagues, asks whether a dataset is a sample of some larger set, whether the sample is representative of it and how that was checked; this worksheet answers those questions with numbers instead of adjectives.

Complete one row per line, treat the illustrative entries as a format guide only, and store the finished sheet with the dataset version it describes:

The sampling worksheet for a customer concentration audit
Worksheet lineWhat to recordIllustrative entryWhy it changes the decision
Reference populationWho or what the model will serve, and the unit it is deployed on: task, user or accountSupport copilot for small-business plans, deployed per ticketSets the denominator every share is compared with
Sampling unitThe unit counted: a task with defined inputs and a success criterion, not a message or a rowOne resolved ticket thread, merged duplicates removedCounting messages over-weights long enterprise threads
Segment attributesPermitted, banded attributes attached by the supplier, with the clause or approval behind eachPlan tier, tenure band, seat band, regionDefines which comparisons the audit can make at all
Selection rulesEvery filter the export applied: dates, statuses, completeness, language, channelClosed tickets 2023 to 2025 with notes over 50 wordsCompleteness filters often favor premium-tier records
Missing segment valuesShare of units with an unknown attribute, by year and by task type7 percent unknown tier, mostly from 2023Unknowns that cluster in one segment hide its true share
ClusteringAccounts per segment, tasks per account and an intraclass correlation estimate80 enterprise accounts, about 575 tickets each, ICC 0.05Drives the effective sample size of each segment
Reference distributionSegment and task shares in the deployed population, with source and dateBilling system: 70 percent of active accounts on small-business plansWithout it no share is too high or too low
Decision ruleLift thresholds and the action each triggers, fixed before resultsFlag lift above 1.5 or below 0.67; act if a flagged cell holds a target taskStops thresholds being chosen after seeing the data

How do you compare task mix across customer segments?

Compare shares on the unit the model is deployed on, then compute a lift for every segment and task family. Here lift is a cell's share of all tasks in the dataset divided by the same cell's share of all tasks in the reference population; a lift of 1 means the dataset presents that segment-task combination as often as deployment will.

The reference unit matters more than the formula. A support copilot answers tickets, so its reference is task-weighted, and a large enterprise share of tickets may be correct. A self-serve assistant sold per account serves each small business once, so its reference is account-weighted, and the same archive is badly skewed. When the deployed population is defined by users or accounts, normalize by exposure, such as tasks per active seat or per order.

Then separate two effects that pooled numbers blend. One is segment weight: a tier supplies too many tasks overall. The other is task composition: the mix inside a tier differs from what deployment needs. Reweighting repairs the first; only new or re-sampled records repair the second, because no weight creates a task type a segment never contributed.

In the illustrative grid below, every enterprise cell sits above the 1.5 flag and every small-business cell except how-to questions sits below 0.67. The archive over-teaches enterprise work in general, and single sign-on and contract exceptions most of all, while small-business billing questions appear at half the rate deployment will bring:

How do you compare task mix across customer segments?
Task family (illustrative lift)EnterpriseMid-marketSmall business
Single sign-on and access setup3.41.30.6
Contract and procurement exceptions3.01.20.4
Billing and plan questions1.81.00.5
How-to and configuration2.21.10.7
A lift grid of three customer tiers against four task families, with enterprise cells flagged above 1.5 and most small-business cells flagged below 0.67.
Task-mix lift by customer tier. Illustrative values from the article's example support archive, not measured data.

Why clustering shrinks the evidence from your biggest accounts

Tasks from the same account resemble each other, so 1,000 tickets from 10 accounts carry far less independent evidence than 1,000 tickets from 1,000 accounts. Survey statisticians measure this with the design effect. The United Nations handbook on household sample surveys gives its clustering component as roughly 1 + ICC × (m − 1), where ICC is the intraclass correlation and m the average cluster size, and notes that a small positive correlation is enough to push it above 1.

Apply it segment by segment. In the illustrative archive, 80 enterprise accounts average about 575 tickets each, 46,000 in all; with an intraclass correlation of 0.05 the design effect is about 29.7, so those tickets behave like roughly 1,550 independent ones. The 6,000 small-business accounts average 10 tickets, 60,000 in all; their design effect is 1.45 and their effective sample size about 41,400. The tier that looks dominant by volume carries the thinner evidence.

Two consequences follow. Report uncertainty by resampling accounts rather than tasks, so intervals reflect how few enterprise customers there really are. And treat a pattern seen in one tier as a property of a handful of accounts until enough distinct accounts show it. Where account sizes vary widely, the handbook's adjustment replaces the simple mean with a weighted average cluster size.

Estimate the intraclass correlation from a labeled pilot of a few hundred tasks spread across accounts. If no pilot exists, run the calculation at two or three plausible values and show whether the decision changes; if it does, the pilot comes before the purchase.

Is it dominance, or just exposure, repetition and export filters?

Rule out four alternative explanations before calling the skew bias. Each test takes hours rather than weeks, and each can reverse the conclusion.

If exposure explains the share and the remaining gaps are small, the archive represents the deployed population and the audit ends with a documented pass. NIST's bias guidance, SP 1270 from 2022, treats statistical bias as one of three categories alongside systemic and human bias and says zero risk of bias is not achievable; the realistic aim is a skew that is measured, bounded and written down.

The four checks, each with the test that settles it:

  • Exposure. If enterprise customers hold 46 percent of the seats the model will serve, a 46 percent share of tasks is accurate. Test: divide tasks by active seats or orders in each segment and compare with the deployed population on the same unit.
  • Repetition. Automated or templated tasks inflate a tier's count without adding coverage. Test: group near-duplicate tasks by template or normalized text and report distinct task patterns per segment next to raw counts.
  • Export filters. Rules such as a minimum note length favor tiers whose dedicated staff write fuller notes. Test: compute segment shares before and after each filter; a filter that moves a share by more than a few points is a selection rule, not cleaning.
  • Missing attributes and lost accounts. Churned small accounts often lose their tier or industry fields when CRM records are purged, so their tasks fall into an unknown bucket or out of the archive. Test: tabulate unknown-attribute shares by year and compare account counts with billing history for the same years.

What can the dataset claim after the audit?

The audit ends with a written list of limitations, each tied to the claim it blocks and the evidence that would lift it. Buyers can then decide whether the remaining skew matters for their use instead of discovering it in production.

Where an EU buyer is building a high-risk AI system, Article 10 of the EU AI Act requires that the data used to train, validate and test such a system be relevant and sufficiently representative for its intended purpose, and that it account, as far as that purpose demands, for the geographical, contextual, behavioural or functional setting where it is meant to operate. Those duties apply to high-risk systems only, but a table like this is the kind of record that answers them.

The completed table for the illustrative support archive:

What can the dataset claim after the audit?
LimitationEvidence that revealed itClaim it blocksNext evidence needed
Enterprise tasks over-taughtEvery enterprise cell above the 1.5 lift flagThe dataset represents small-business supportAccount-weighted resample or a small-business supplement
Thin evidence from large accountsEnterprise effective sample about 1,550 despite 46,000 ticketsEnterprise-specific behavior is well estimatedMore distinct enterprise accounts, not more tickets
Unknown tier for 7 percent of tasksUnknowns concentrated in 2023 exportsTrends across years are comparableTier re-derived from 2023 billing history
Churned accounts underrepresentedFewer accounts in the archive than in billing records for the same yearsCoverage of customers who leftRecords from the retention period or a disclosed exclusion
Single-region sampleAll accounts in North AmericaPerformance outside that regionA regional sample or a scoped intended use
Templated tasks inflate one tierDistinct-pattern count far below raw count for enterpriseRaw counts measure coverageDistinct patterns reported per segment

Accept, stratify, reweight, supplement or narrow: choosing the fix

Match the fix to the cause the audit found. Accept when exposure explains the share. Stratify evaluation sets so each tier is scored separately and then weighted to the deployment mix, which keeps rare enterprise tasks testable without letting them set the average. Reweight training data when segment weight is wrong but each segment's task mix is sound.

Supplement when a segment's task composition is missing, because weights cannot invent tasks a segment never contributed. Narrow the intended use when supplementing is impossible: a dataset that represents enterprise support well can be sold and bought as exactly that. Dropping top-tier records is rarely right; it throws away the complex, rare tasks that justify buying business data at all.

Keep the scoring unit fixed while you do it. Anthropic's guidance on agent evaluation counts each task as one test, with specified inputs and explicit criteria for success, and each attempt at it as a trial. If enterprise tasks end up with looser success criteria than small-business tasks, a gap between tiers may reflect grading rather than data.

Putting the audit into a dataset request

A request that names the reference population and segment attributes up front lets a supplier say early whether it can meet them. SourceX keeps no standing inventory for AI buyers: it looks for each dataset among U.S. businesses after the request lands, and its request form is where buyers set out the basics of scope, volume, language, format, timing and licensing needs.

For this audit, add four items: the deployment unit and reference distribution, the banded attributes you need with a minimum account count per reported cell, the selection rules you will accept, and an acceptance test such as no target-task cell outside the agreed lift band after weighting. Whether a given company's records can carry those attributes is settled in that dataset's rights and privacy review; nothing here implies that a dataset with a particular segment mix is available.

Frequently asked questions

What is customer concentration bias in a training dataset?

It is a skew in which a small group of customers, usually the highest-value accounts, supplies a share of tasks out of proportion to the population the model will serve. The skew can sit in the task mix even when no single customer holds a large share of records. It matters whenever the model is deployed to a different customer mix than the one that produced the archive.

Can a dataset lean toward big customers when no single account dominates?

Yes. Dozens of enterprise accounts can together supply most tasks of a kind only enterprise customers create, such as single sign-on setup or contract exceptions. A per-account concentration measure looks healthy while a segment-by-task comparison shows the tilt.

Which customer attributes are safe to use in a segment audit?

Coarse, banded attributes that the license and the privacy review permit: plan tier, tenure band, seat band, broad industry, region or channel. Customer names are unnecessary. Count accounts in every reported cell and merge or suppress thin cells, since a narrow combination can point to one business.

How many tasks per segment are enough?

Count effective tasks, not raw tasks. Divide each segment's task count by its design effect, roughly 1 + ICC × (m − 1) where m is tasks per account. A segment with tens of thousands of tickets from a few dozen accounts may carry the evidence of only a few thousand independent tasks.

Should you downsample the top tier or add small-customer data?

Downsampling discards rare, complex tasks and only corrects segment weight. If weight is the problem, reweight or stratify; if a segment's task types are missing, add records from that segment or narrow the dataset's stated intended use. Fix the rule before looking at results.

Sources

  • OpenAI's GDPval spans 44 occupations selected from the top nine industries contributing to U.S. GDP, with 1,320 tasks, 220 of them in the open gold set. Source
  • CMS data use terms state that no cell of 10 or less may be displayed, including through calculations that reveal it. Source
  • Datasheets for Datasets asks whether a dataset is a sample of a larger set, whether the sample is representative and how representativeness was validated. Source
  • The UN household survey handbook gives the clustering design effect as approximately 1 + rho(m - 1) and adapts it to unequal clusters with a weighted average cluster size. Source
  • NIST SP 1270 (2022) groups AI bias into systemic, statistical and human categories and states that zero risk of bias is not achievable. Source
  • EU AI Act Article 10(3) and 10(4) require data for high-risk AI systems to be relevant and sufficiently representative for the intended purpose and to reflect the setting of use as that purpose requires. Source
  • Anthropic describes an agent evaluation task as a single test with defined inputs and success criteria, with each attempt counted as a trial. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies