Skip to content

Getting started

Utility dataset pilot evidence: what an energy data pilot should prove

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

An energy workflow data pilot should prove, on a small bounded slice, that cases link asset, reading, tariff, contract and work order evidence; that every fact is dated to the event; that estimated and corrected readings keep their quality flags; that scope stays administrative; and that infrastructure, customer and field-safety boundaries held. Model usefulness is the buyer's own pre-registered test on held-out cases. This guide gives the method, not results.

Process

Utility dataset pilot

This page covers utility dataset pilot evidence: what an energy data pilot should prove.Illustrative

Key takeaways

  • Prove completeness, dating, reading quality, bounded scope and boundaries; leave model gains to the buyer's pre-registered test.
  • Join readings, rates and contract terms as of the event date, and keep later corrections as separate dated records.
  • Keep reading quality codes; dropping them turns estimates into apparent facts.
  • Exclude infrastructure detail, fine-grained customer usage and any field or switching procedure before selecting records.
  • Write the protocol, holdout and reporting terms down before any record is chosen.
  • Stop and narrow scope when a scan finds protected content; do not rely on case-by-case cleaning.

What should an energy workflow data pilot demonstrate?

An energy workflow data pilot should prove five things on a small, bounded slice of records: that cases link asset, account, reading, tariff, contract and work order evidence; that every fact is dated so the rule in force on the event date can be recovered; that corrected and estimated readings keep their quality flags; that the task scope is narrow and administrative; and that confidentiality and safety boundaries held throughout. Whether the records improve a model is the buyer's own test, run on a held-out set agreed in advance.

This guide is for utilities, energy retailers, energy service companies and large energy users weighing a pilot before licensing operational records. It covers administrative and back-office workflows such as billing exceptions, meter data corrections, tariff application, service orders and contract administration. It does not cover switching, clearances or any field procedure, which stay out of scope for reasons explained below.

No pilot results are reported here. The plan describes what to prove and how to show it, so the evidence can be reviewed before anyone draws conclusions.

Which records make an energy administrative task checkable?

A task is checkable when a reviewer can recompute or confirm the right answer from the case itself. In energy administration that almost always means joining several systems: the customer information and billing system, the meter data management system, the asset register or GIS, the work management system and the contract repository.

Identifiers are the first thing to test. Account, premise, service point and meter are different entities: an account can move between premises, a premise can have several service points, and a meter can be exchanged while the service point stays the same. A pilot should show that each case uses the right key for each join, and that history survives a move-in, move-out or meter exchange instead of being overwritten by the current mapping.

Meter readings are where most pilots go wrong, because the same interval can be read, estimated, corrected and re-billed. The Green Button ESPI schema published by the Green Button Alliance shows how much a reading can carry: its quality-of-reading codes distinguish valid data, manually edited values, values estimated from a reference day or by linear interpolation, questionable data that failed checks, raw data that has not been validated, validated data, and data that failed a check but was verified as actual usage. A pilot that drops those flags turns estimates into apparent facts.

Which records make an energy administrative task checkable?
Evidence linkTypical systemWhat it lets a reviewer check
Asset or meter identifier with install and removal datesAsset register, meter data managementWhich device produced a reading on a given date
Reading with revision history and quality codeMeter data managementWhether a bill used an estimate, a correction or an actual read
Tariff or rate with effective and end datesBilling rate tables, published tariffWhich rate applied to each day of a billing period
Contract term or special agreementContract repository, CRMWhether a negotiated term overrides the standard rate
Work order with status timestampsWork management systemWhether a meter exchange or field visit explains a reading gap
Approval and exception recordBilling exception queue, workflow toolWho authorized a rebill, write-off or adjustment and why
OutcomeBilling ledger, customer account historyThe final corrected bill and any customer contact

Why do corrected readings and tariff dates decide the label?

The correct answer to most billing and meter tasks depends on what was true on a date, not what is true now. If a rate changed mid-period, a bill must be prorated across both; if a reading was estimated and later replaced by an actual read, the rebill compares the two. A pilot that joins current tariffs and latest readings to historical bills will mark correct work as wrong.

Public rate data shows the pattern. The OpenEI Utility Rate Database API, which publishes U.S. utility rate structures, lets a caller ask for rates effective on a given date, a reminder that a rate is a dated record with a start and, once superseded, an end. Internal rate tables need the same treatment, and so do contract terms, which can start, renew or lapse inside a billing period.

So the pilot should demonstrate as-of joins: every reading, rate and contract term in a case is the version that applied on the event date, with the later corrections kept as separate, dated records. Reviewers can test this directly by recomputing a sample of bills from the case contents.

Where do the safety and confidentiality boundaries sit?

Three boundaries decide what a pilot may contain, and each should be written into the plan before any record is selected.

Infrastructure information comes first. FERC's current CEII regulation, 18 CFR 388.113, covers specific engineering, vulnerability or detailed design information about critical infrastructure that could be useful to someone planning an attack, beyond the general location of a facility. NERC's CIP-011 standard requires responsible entities to identify and protect BES Cyber System Information. Work orders, asset records and outage notes can carry exactly this kind of detail: one-line diagrams, protection settings, network addresses or access procedures. The pilot plan should exclude those fields and the documents that contain them, and the utility's CIP or security lead should confirm the exclusion list.

Customer energy usage is the second boundary. Green Button's Connect My Data model has a retail customer authorize a third party to receive usage information from the utility, which reflects how sensitive interval data is treated. Usage at fine intervals can reveal occupancy and routines, so the plan should state how account and premise identifiers are removed or tokenized and whether interval data is aggregated.

Field safety is the third. OSHA's 1910.269 standard covers the operation and maintenance of electric power generation, transmission and distribution lines and equipment. A dataset of administrative work should not train a model to plan or instruct that work. Keep switching orders, clearance and lockout records, and crew safety procedures out of the pilot, and say so in the scope.

What should the domain reviewed utility and risk acceptance plan include?

The plan below states each claim the pilot must support, the evidence that supports it, the test a reviewer runs, who reviews it and what counts as failure. Agree it with the prospective buyer before records are selected, so the result cannot be argued after the fact.

What should the domain reviewed utility and risk acceptance plan include?
Pilot claimEvidence to produceAcceptance testReviewerFails if
Cases are completeLinked records per case across the seven evidence linksTrace a random subset from request to outcomeData lead and billing operationsAny traced case is missing an outcome or a governing rule
Facts are datedEffective and end dates on rates, contracts and readingsRecompute a sample of bills from case contentsBilling analystRecomputed amounts differ without a recorded reason
Reading quality is preservedQuality codes and revision history on every readingCompare code distribution with the source systemMeter data leadEstimated or edited reads appear unflagged
Scope is boundedWritten list of task families, territories, periods and fieldsCheck every record against the scope listOperations ownerAny record falls outside the list
Infrastructure information is excludedExclusion list and scan results for diagrams, settings and addressesTargeted search of notes and attachmentsCIP or security leadAny protected detail appears
Customer data is protectedDe-identification method and re-identification testAttempt linkage on a sample with known identifiersPrivacy leadA premise or customer can be re-linked
Safety boundary holdsList of excluded field procedures and record typesSearch for switching, clearance and lockout contentSafety leadField procedure content is present
Rights are confirmedContracts, regulatory orders and third-party terms covering the recordsCounsel review against the scope listCounselA source is restricted or unclear
Usefulness is measured by the buyerBuyer's pre-registered test, metric and held-out setBuyer runs its test and reports against the planBuyer's technical leadThe test or metric changes after records are seen

How should the pilot be sized and run without overreaching?

Keep the pilot narrow enough that every case can be reviewed. One task family, one service territory or business unit, and one continuous period is a sensible shape; the period should cover at least one rate change or seasonal billing pattern so the dated-facts claim is actually tested.

At a glance

How should the pilot be sized and run without overreaching

Write the protocol down before selecting records: the question the pilot answers, the population it draws from, the selection rule, the variables carried in each case, the comparison the buyer will run, the metrics, how uncertainty will be reported and the known limits. The NIST AI Risk Management Framework's MEASURE function calls for documenting test sets, metrics and tool details used in testing, and for documenting limits on how far results generalize beyond the conditions tested. A pilot plan that does the same is easier to accept and harder to over-read.

Hold out a set of cases the buyer never sees until the test. Agree how the buyer will report results back, if at all, and what each party may do with the pilot records afterward; those terms belong in the pilot agreement.

  • Question: the single task the pilot records are meant to support, stated in one sentence.
  • Population and selection: the territory, account class, period and the rule used to draw cases, including how rare cases are handled.
  • Variables: the fields carried per case, with their source system and as-of date logic.
  • Comparison: what the buyer will compare against, such as its current data or a public dataset, on the same held-out cases.
  • Metrics and uncertainty: the metric, how it is computed and how its uncertainty will be reported, for example an interval rather than a single figure.
  • Limits: what the pilot cannot show, such as performance in other territories, account classes or periods.
  • Change control: what happens if the plan has to change after records are seen, and who must agree.

Which dated rule, asset fact or safety boundary changes the permissible next step?

Each case should let a reviewer say what the next step was allowed to be, and why. These rules flag the facts that most often change the answer.

Checklist · 6 items

  1. A rate or rider change inside the billing period changes the correct bill; the case must hold both versions and their dates.
  2. A meter exchange changes which device's readings apply; the case needs the work order and the install and removal dates.
  3. An estimated reading later replaced by an actual read changes the rebill; keep both values with their quality codes.
  4. A negotiated contract term can override the published rate; the case must show the term and its effective dates.
  5. An approval threshold changes who may authorize an adjustment; keep the approver's role and the threshold in force.
  6. Any step that would require field work, switching or a safety procedure is outside the dataset's task scope; the correct next step in the data is a hand-off to the responsible team, not an instruction.

0 of 6 ticked. Tick each item as you go.

Worked plan (illustrative): estimated-read rebills for commercial electric accounts

This plan is illustrative and uses counts and durations only. A utility proposes a pilot on rebills caused by estimated readings for small commercial electric accounts in one territory over 18 months, a window that includes one general rate change and one meter exchange program.

Each case would link the account (tokenized), the meter and its install dates, the estimated and actual readings with their quality codes, the rate versions in force, any special contract term, the exchange work order where relevant, the rebill approval and the final bill. The plan excludes outage records, protection or network details, switching and clearance records and all interval data below daily totals.

Acceptance would be run on a random draw of a few hundred cases, illustrative only, with billing analysts recomputing a subset and the security lead scanning notes. The buyer pre-registers a single task and metric, for example whether a model proposes the correct rebill basis, and evaluates on a held-out set. No result is reported in this guide.

The exception: accounts served under a negotiated contract with confidentiality terms that cover usage and pricing. Those cases are dropped from the pilot unless counsel confirms the contract allows the use, even though they are some of the most interesting rebills.

When should a pilot stop, narrow or bring in specialist review?

Stop and fix the source when protected infrastructure detail, customer identifiers or field procedure content turns up in a scan; narrow the scope rather than relying on case-by-case cleaning. Pause for counsel when records come from regulatory proceedings, joint ownership arrangements, wholesale or interconnection agreements, or contracts with confidentiality terms. Bring in a domain reviewer whenever recomputed bills disagree with recorded ones for reasons nobody can explain.

Narrowing is usually the better outcome than abandoning the pilot. If interval data proves too sensitive, move to billing-period totals; if one territory's work orders carry too much infrastructure detail, choose another; if a task family keeps pulling in field procedures, drop it and keep the purely administrative ones. Record each narrowing decision and its reason in the plan, because the same reasons will shape the full dataset.

Recheck the plan when tariffs, asset records, readings practices, contracts, operating conditions or the scope change. A pilot plan is tied to the systems and rules of its period; it is evidence about that slice, not a statement about every record the company holds.

From pilot evidence to a SourceX assessment

Energy companies that move from a pilot to a license can work through SourceX, which runs the sourcing and qualification, rights review, contracting, data preparation, delivery and payment for selected operational records licensed to AI developers. Eligibility is checked by assessing the company first; no records are uploaded for that step.

The acceptance plan above is useful input to that conversation because it states scope, exclusions and evidence in advance. Delivery waits for the company's approval of scope and terms and its signature, and archives too large to send over a network remain in the company's storage or travel on encrypted drives, because SourceX does not host them.

Frequently asked questions

What is utility dataset pilot evidence?

It is the set of records and checks that show a small slice of energy workflow data is complete, correctly dated, quality-flagged, tightly scoped and free of protected infrastructure, customer and safety-procedure content. It supports a decision to scope a larger dataset; it is not a claim that a model improved.

Should an energy data pilot report model accuracy?

Only if the buyer runs a test agreed before the records were seen, on a held-out set, and the parties agreed to share the result. The supplier's pilot evidence should focus on completeness, dating, quality flags and boundaries, which it can show directly.

Can interval meter data be part of a pilot?

Sometimes, but it needs care. Fine-grained usage can reveal occupancy and routines, so many pilots use daily or billing-period totals with tokenized accounts. Counsel and the privacy lead should decide based on the applicable rules and customer terms.

How do I keep critical infrastructure information out of a dataset?

Write an exclusion list before selecting records: diagrams, protection settings, network addresses, access procedures and similar detail, plus the documents that carry them. Have the CIP or security lead confirm it, then scan notes and attachments and narrow the scope if anything appears.

Why do reading quality codes matter for AI training data?

Because an estimate and an actual read look identical once the code is dropped. Standards such as Green Button ESPI distinguish estimated, edited, raw, validated and verified readings, and a model learning billing corrections needs those distinctions to learn the right rule.

Sources

  • The Green Button ESPI schema defines quality-of-reading codes including valid, manually edited, estimated using reference day, estimated using linear interpolation, questionable, raw, validated and verified. Source
  • Green Button Connect My Data lets a retail customer authorize a third party to access energy usage information directly from the data custodian. Source
  • The OpenEI Utility Rate Database API lets callers request rates effective on a specified date. Source
  • FERC's CEII definition covers specific engineering, vulnerability or detailed design information about critical infrastructure that could be useful to a person planning an attack and that gives more than the general location of the infrastructure. Source
  • NERC CIP-011 aims to prevent unauthorized access to BES Cyber System Information by requiring entities to identify and protect it. Source
  • OSHA 1910.269 covers the operation and maintenance of electric power generation, control, transformation, transmission and distribution lines and equipment. Source
  • The NIST AI RMF MEASURE function calls for documenting test sets, metrics and tool details used in testing and the limits of generalizing beyond tested conditions. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies