Skip to content

Deal economics

When does dataset qualification capital allocation beat more leads?

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Better qualification beats more lead generation when the expensive review stage is already full. Extra leads then only lengthen the queue, while a screen that removes a larger share of doomed cases than of good ones lifts accepted output at the same review capacity: by 14.2 percent in this article's illustrative model. When reviewers sit idle, buy leads instead, but price them at marginal yield, not the average.

Value drivers

Dataset qualification

This page covers when does dataset qualification capital allocation beat more leads.Illustrative

Key takeaways

  • The next supply dollar belongs at whichever stage is full: leads help only while the expensive review stage has spare hours.
  • With review full and leads plentiful, a screen adds accepted output whenever its catch rate of doomed cases exceeds its false-drop rate of good ones; the cost is paid in leads, not reviewer hours.
  • In Olist's public seller sample, the top 10 percent of 2,970 sellers delivered 67.6 percent of items while the median seller delivered 8 against a mean of 37.1, so the average supplier is a poor guide to the next one.
  • A randomized eBay study found paid search returns far below non-experimental estimates, a warning that attributed lead counts overstate what more spend buys.
  • Fund a screen only after a backtest on closed cases and a shadow run, and stop it if accepted output falls.

Which question does dataset qualification capital allocation answer?

It answers where the next increment of supply spending should go: into generating more leads, or into qualifying the leads a company already gets, earlier and more accurately. Accepted supply, meaning datasets that clear rights review and a buyer's fit test, is the output both options are trying to buy.

The choice is usually framed as growth against caution, which hides the real test. More leads add accepted supply only if every stage after intake has room for them and the new leads pass at a useful rate. Better qualification adds accepted supply only if the stage it protects is full and the screen removes more doomed cases than good ones.

So a useful answer needs four facts from a company's own records: which stage is running at capacity, the hours and pass rate at each stage, the pass rate of the newest lead channel measured on its own, and how a proposed screen would have sorted cases that have already closed. An investor who sees those four can tell a plan that buys supply from a plan that buys activity.

Where do a supply funnel's yield and cost sit?

In most data supply funnels the cheap steps sit at the top and the expensive ones near the bottom, so most hours go to cases that eventually fail. The illustrative funnel below makes that concrete; every input is an assumption chosen for the example, not a measurement.

One hundred leads cost 0.5 hours each to generate and one hour each at intake, where 40 percent pass. Rights and records review takes 6 specialist hours per case and passes half; sample preparation and a buyer-fit test take 16 specialist hours and pass half again. Out come 10 accepted datasets for 710 hours, or 71.0 hours each. Rights review and fit testing absorb 78.9 percent of the hours between them, and 66.9 percent of all hours are spent on cases that end rejected.

The 560 specialist hours this baseline uses are set as the review team's full capacity per period. That one assumption decides which plan in the sections below adds supply.

Where do a supply funnel's yield and cost sit?
Stage (Illustrative)Hours per casePass rateHours per 100 leadsShare of all hours
Lead generation0.5Not a gate507.0%
Intake call and questionnaire1.040%10014.1%
Rights and records review (specialist)6.050%24033.8%
Sample preparation and buyer-fit test (specialist)16.050%32045.1%

Is the next supplier like the average one?

Rarely, and planning with the average overstates what more intake buys. Public marketplace data show how far the typical supplier sits below the mean.

Olist, a Brazilian marketplace operator, published order, item and seller tables on GitHub (work-at-olist-data, commit d9e4980); its README says the data are samples taken directly from the transactional system without treatment. Counting only items on orders with the status delivered, 110,197 items were sold by 2,970 sellers. The top 10 percent of sellers, 297 of them, delivered 67.6 percent of the items; the bottom half delivered 4.1 percent. The mean seller delivered 37.1 items and the median seller 8, so 80.4 percent of sellers sat below the mean, and 16.4 percent delivered exactly one item.

Sellers who joined late had less time to sell, so the count was repeated on a fixed window: the 819 sellers whose first delivered item was bought in the first half of 2017, each followed for 365 days. The shape held. The top tenth delivered 61.0 percent of that cohort's items, the bottom half 4.8 percent, and the median seller 13 items against a mean of 48.2.

Data suppliers are not marketplace sellers, and nothing here measures a data company. The lesson transfers anyway: if the next supplier recruited looks like the median rather than the mean, a plan that multiplies average yield by more leads will overshoot, and a screen that tells head from tail early is worth more than the average suggests.

Lead counts can mislead in a second way. In a randomized field experiment at eBay (Blake, Nosko and Tadelis, NBER Working Paper 20171, published in Econometrica in 2015), brand-keyword search ads had no measurable short-term benefit, returns across keywords were much smaller than non-experimental estimates, and loyal users whose purchases the ads did not change accounted for most of the spending, leaving average returns negative. Leads that would have arrived anyway look like the product of spend until someone runs a holdout.

More leads or an earlier screen? Find the full stage first

If the specialist review stage is full, more leads add nothing this period and a screen adds supply; if it has spare hours, the order flips. Plan A adds 90 non-specialist hours a period and Plan B about 54, for the extra leads it needs and the screening itself.

Plan A buys 50 more leads from a weaker channel: 0.8 hours each to generate, with 30 percent passing intake. If the review team could absorb them, they would add 3.75 accepted datasets at 80.0 hours each, against the 71.0-hour average, and would need 210 more specialist hours. With review already full, accepted output stays at 10, the 90 extra lead and intake hours buy nothing this period, and 15 more cases join the review queue, so 55 cases compete for review instead of 40.

Plan B adds a half-hour document screen before rights review, checking facts that predict failure there. It removes 60 percent of the cases that would fail rights review and wrongly drops 5 percent of those that would pass. The pass rate at rights review rises from 50 to 70.4 percent, and specialist hours per accepted dataset fall from 56.0 to 49.1. At the same 560 hours the review team now produces 11.4 accepted datasets, 14.2 percent more. Getting there takes 48.1 screened cases a period, of which 32.4 reach review, 24.0 hours of screening and 10.5 leads per accepted dataset instead of 10.0.

Reverse the condition and the verdict reverses. With only the original 100 leads and an idle review team, the same screen lowers output from 10.0 to 9.5, because each wrongly dropped case is a lost dataset; it still saves 94 specialist hours for 20 hours of screening, which matters only if those hours have another use. When reviewers wait for work, the lead plan is the one to fund, priced at the 80.0-hour marginal cost rather than the 71.0-hour average.

Three funnel panels at the same 560-hour review capacity: today 40 cases reach review and 10 datasets are accepted; with 50 more leads 55 cases compete, 15 queue and 10 are accepted; with a screen first 48.1 cases are screened and 11.4 datasets are accepted, 14.2 percent more.
Where the full stage sits decides which plan adds supply. Illustrative model inputs.

The break-even rule: catch rate against false-drop rate

When the review stage is full and leads are plentiful, a screen raises accepted output exactly when it removes a larger share of doomed cases than of good ones. Call the share of doomed cases it catches c and the share of good cases it drops f; the screen pays whenever c exceeds f.

The reason is simple arithmetic. Before the screen, a share q of cases reaching review would pass. After it, the passing share becomes q times (1 - f), divided by q times (1 - f) plus (1 - q) times (1 - c). That ratio is larger than q precisely when c is larger than f, whatever q is. A higher pass share at review means fewer review hours per accepted dataset, and with review hours fixed, more datasets.

The screen's price is paid in leads rather than reviewer time: leads per accepted dataset rise by a factor of 1 / (1 - f), which is why Plan B needed 10.5 leads per dataset instead of 10.0. The grid below shows the change in accepted output at fixed review capacity for other screens, using the same illustrative funnel.

The break-even rule: catch rate against false-drop rate
False-drop rate of good casesCatches 30% of doomed casesCatches 60%Catches 80%
2%+6.5%+14.5%+20.6%
5%+6.0%+14.2%+20.4%
15%+3.9%+12.8%+19.6%
30%0.0%+10.1%+18.1%

What do published screens test, and what does SourceX publish?

Durable screens test facts that are cheap to check and that predict failure in the expensive review, and they leave judgments of quality to the reviewers. Two public examples from very different fields share that design.

The U.S. Food and Drug Administration runs an acceptance review on 510(k) submissions before substantive review. Its Refuse to Accept guidance, issued April 21, 2022, says the completeness of the 510(k) is assessed during acceptance review, while the quality of the submitted information is assessed during substantive review. FDA aims to finish the acceptance review within 15 calendar days of receipt; a refused submission goes on hold, the review clock does not start until the missing elements are provided, and without a complete response within 180 days FDA considers the 510(k) withdrawn.

Homebrew, the open-source package manager, screens new packages before maintainers spend review time on them. Its Package Acceptance Policy (last reviewed July 18, 2026) says official repositories accept software the project can verify, maintain and support for a broad user base, and that a new package must show public interest beyond its author: normally at least 30 forks, 30 watchers or 75 stars on GitHub, tripled to 90, 90 or 225 when the submission comes from the project's owner or from someone at its web domain, with repositories under 30 days old normally not eligible. The audit code applies the check only to new formulae, so it costs nothing once a package is in.

Note what both screens refuse to do. Neither tries to judge the science or the software at the gate; they check facts that make an expensive review unlikely to succeed, and they publish the criteria so applicants can screen themselves.

SourceX publishes a screen of the same kind, which an investor can read but not measure. Its referral hub's qualification page describes a profile of established companies in listed countries, with at least 90 percent English data required outside the U.S., Canada and the EU, roughly 50 or more full-time employees at peak and several years of operation, records the company created itself, the rights to license them, a sponsor with authority to approve the discussion and a fit with a current program; consumer data, scraped data and material belonging to a company's clients are usually excluded. On its how-it-works page, applicants list their systems, years of history and exports, SourceX checks fit and runs the rights check, qualifying companies list at least 5 systems, and no records change hands at that stage. Each of those checks targets a cause of failure in rights review or buyer fit before specialist hours are spent.

Decision tree and milestones for the next supply tranche

Answer the questions in order; the first one that settles the choice ends the review. Then release money in tranches, each opened by evidence rather than by the calendar. The tranche figures are illustrative and should be replaced with a company's own.

The tree comes first below, followed by the milestone table that turns its answers into tranches:

  • Q1. Is the expensive review stage running near capacity, with a queue that has grown for two or more periods? No: go to Q2. Yes: go to Q3.
  • Q2. Is the newest lead channel's pass rate measured on its own cohort? No: measure it before scaling. Yes: fund leads while marginal hours per accepted dataset stay under a ceiling set in advance, and stop when review fills.
  • Q3. What share of review failures trace to facts knowable at intake, such as company size, years of records, who holds the rights or whether a sponsor can approve? Low: fund review capacity or sharper buyer specifications, not a screen. High: go to Q4.
  • Q4. Run the proposed screen over closed cases. Does its catch rate beat its false-drop rate by more than sampling error? No: revise the criteria. Yes: go to Q5.
  • Q5. Are leads plentiful enough to absorb the false drops, which raise leads per accepted dataset by 1 / (1 - f)? No: pair the screen with a small, separately measured lead increase. Yes: fund the screen and rerun the tree next quarter.
Decision tree and milestones for the next supply tranche
Tranche (Illustrative)Evidence that releases itMoney goes toStop if
1. MeasureNone needed: this tranche creates the evidenceStage-level hours, pass rates, queue length and a reason code on every rejection, kept for at least eight weeksAny stage cannot be measured
2. Shadow screenA backtest on at least 100 closed cases with catch rate above false-drop rateRunning the screen on new intake without acting on itFalse drops exceed catches in the shadow run
3. Live screenShadow results confirm the backtest and the review pass rate risesMaking the screen a gate before rights reviewAccepted output falls for two periods in a row
4. New lead channelReview below full capacity for a whole period after the screenOne new channel with its cohort tracked separatelyMarginal hours per accepted dataset exceed the ceiling

What could make the qualification case wrong?

The strongest alternative is that neither plan fixes the real constraint, and that hiring or automation scales cost faster than accepted delivery whichever way the money goes. Adding intake staff to work more leads, or buying software to score them, raises cost at once, while accepted supply moves only if the full stage gets relief. A screen that starts as a half-hour check can also grow into a second review team. Watch accepted datasets per specialist hour and leads per accepted dataset by cohort; if both worsen while headcount or tooling grows, the plan is buying activity.

Other ways the case can fail:

  • False drops stay invisible. Rejected cases never show what they would have become, so a screen can look accurate while discarding good supply; periodic blind re-review of a sample of rejects is the only check.
  • Demand, not supply, may bind. If accepted datasets wait for buyers, both plans add inventory rather than licenses, and the money belongs on the buyer side.
  • Criteria can screen out rare, valuable supply. Size and volume thresholds miss small specialists whose records are scarce, which is why published screens, SourceX's included, leave room for exceptions.
  • The predictive facts can change. New rights rules, platform terms or buyer requirements can make yesterday's screen check the wrong things, so its backtest needs repeating.
  • Lead channels improve. A channel with a weak first cohort can mature, so a single bad cohort should pause a channel rather than end it.
  • The model is a sketch. Its pass rates, hours and capacity are illustrative, the Olist counts come from a sample of one marketplace, and the eBay result concerns consumer search ads, not business supply.

Questions for SourceX and any plan that funds supply

SourceX's published criteria show what its screen checks, not how well it works. Its funnel, pass rates, lead volumes and spending are not public; the company makes operating and commercial figures available to investors on request through its investor page, and any offering is made only to verified accredited investors under Rule 506(c). The same questions fit SourceX and any other data company asking for money to grow supply:

  • Which stage is at capacity today, and how long is its queue measured in cases and in weeks?
  • What are the pass rate and hours per case at each stage, by quarter and by lead channel?
  • What share of rejections at rights review or buyer fit trace to facts that were knowable at intake?
  • Has the qualification screen been backtested on closed cases, and what were its catch and false-drop rates?
  • How are rejected applicants re-reviewed to find good supply the screen dropped?
  • What is the marginal hours-per-accepted figure for the newest channel, next to the average?

Methods note: data, model and blind spots

The funnel model, decision tree and milestone table were built for this article on 2026-10-27. Model inputs (lead, intake, review and fit hours; 40, 50 and 50 percent pass rates; 560 specialist hours; a 0.8-hour, 30 percent marginal channel; a 0.5-hour screen catching 60 percent of doomed cases and dropping 5 percent of good ones) are illustrative and describe no company. Arithmetic uses exact fractions with one rounding at the end, half up.

The Olist counts come from olist_orders_dataset.csv and olist_order_items_dataset.csv at commit d9e4980 of github.com/olist/work-at-olist-data. Rules, as the script applies them: one item is one row of the item table; an item counts only if its order's status is exactly delivered, which drops 2,453 of 112,650 rows (shipped 1,185, canceled 542, invoiced 359, processing 357, unavailable 7, approved 3); sellers are ranked by delivered items with ties broken by seller id, which cannot change any sum; the top tenth is the first 297 of 2,970 sellers and the bottom half the last 1,485. The cohort check takes sellers whose first delivered item was purchased from 2017-01-01 00:00:00 through 2017-06-30 23:59:59, so 30 June counts in full, and counts their delivered items purchased within 365 days of that first purchase, compared as timestamps; with 819 sellers in the cohort, the top tenth is the first 81 (819 divided by 10, rounded down) and the bottom half the last 409. Only order ids, statuses, purchase timestamps and opaque seller ids were read, and no data were saved.

Other evidence: the eBay study through NBER, FDA's Refuse to Accept guidance, Homebrew's policy and audit code at commit a59d9fc, and SourceX's qualification and how-it-works pages, the last read directly on 2026-10-27; the NBER and FDA texts were checked through searches limited to each publisher's domain. Not measured: any data company's funnel, SourceX's included; the cost of leads in money; how accurate any real qualification screen is.

Frequently asked questions

Is more lead generation ever the better use of capital?

Yes, when the expensive review stage has spare hours and the new channel's own pass rate is known. Then each added lead can become accepted supply, and the right price is the marginal hours per accepted dataset, which in this article's illustrative model is 80.0 against a 71.0-hour average.

How do you know if a qualification screen is good enough to fund?

Run it over cases that have already closed and compare two rates: the share of eventually failed cases it would have removed and the share of eventually accepted cases it would have dropped. If the first clearly beats the second, test it in shadow mode on new intake before making it a gate.

Why does the average supplier mislead a funding plan?

Because output is concentrated. In Olist's public seller sample the median seller delivered 8 items against a mean of 37.1, and the top tenth delivered 67.6 percent of items. A plan that multiplies average output by more recruits assumes the next supplier looks like the mean, when it more often looks like the median.

What does a screen cost if the review team is not the bottleneck?

It costs supply. With idle reviewers, every good case the screen wrongly drops is a dataset lost, so output falls; in the illustrative model it fell from 10.0 to 9.5 while saving 94 specialist hours. Those saved hours only matter if the team has other work for them.

What should investors ask a data company about its qualification funnel?

Ask which stage is full, the pass rate and hours at each stage by channel, how many rejections trace to facts knowable at intake, and whether the screen was backtested. Then ask how rejected applicants are re-reviewed, because a screen's mistakes never show up on their own.

Sources

  • Olist's README says its published marketplace tables are samples extracted directly from the transactional environment without treatment. Source
  • Olist's order and item tables, from which this article counts delivered items per seller. Source
  • A large field experiment at eBay found brand-keyword ads had no measurable short-term benefit and paid search returns far smaller than non-experimental estimates. Source
  • FDA's Refuse to Accept guidance separates a completeness check at acceptance review from the substantive review of quality, with a 15-day acceptance target. Source
  • Homebrew requires new packages to show notability, normally 30 forks, 30 watchers or 75 stars, tripled for self-submissions. Source
  • Homebrew's audit code encodes the notability thresholds and a self-submission multiplier of 3. Source
  • Homebrew applies the repository notability audit only to new formulae. Source
  • SourceX's published screening profile: eligible countries, roughly 50 or more full-time employees at peak, original records, rights to license, an authorized sponsor and program fit. Source
  • SourceX's qualification step checks fit and rights from an inventory of systems before any records are shared. Source
  • SourceX shares company and commercial metrics with investors directly. Source

Request SourceX investor materials

SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.

See if your company qualifies