Skip to content

Privacy and preparation

Energy data supplier scope sheet: what an operating dataset should specify

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

An energy operating dataset scope sheet should specify the task, the asset population by stable ID, the date window with tariff and contract versions in force, how estimated and corrected readings are kept, the work orders and approvals linked to each event, and four boundaries: rights, contract confidentiality, critical infrastructure information and permitted use. Clear each boundary with evidence before any extract leaves the company.

License terms

Energy data supplier scope

This page covers energy data supplier scope sheet: what an operating dataset should specify.Illustrative

Key takeaways

  • Scope the task first; everything the task does not need is out of the dataset by default.
  • Keep original and corrected readings with their quality flags, because the correction is often what the model must learn.
  • Join every reading and charge to the tariff or contract version in force on its own date.
  • Exclude engineering, vulnerability and design detail about infrastructure unless the security or compliance owner approves.
  • Customer-level usage data needs its own consent or legal basis; tokenizing identifiers is the default.

What should an energy operating dataset scope sheet specify?

An energy data supplier scope sheet should pin down six things before any extract: the task the data will support, the asset population by stable asset ID, the date window with the tariff and contract versions in force, how corrected and estimated readings are handled, which work orders and approvals link to each event, and four boundaries that decide what may leave the company: rights, confidentiality, critical infrastructure information and permitted use.

The first five make the dataset coherent. The last four make it releasable. Utilities, generators, midstream operators and energy service companies hold records that are valuable precisely because they connect physical assets, measurements, prices and decisions, and those same connections are what create security and confidentiality exposure.

A scope sheet is also the cheapest place to say no. Narrowing the task on one page costs an afternoon; discovering after extraction that a feeder export contains protection settings or customer-identifiable interval data costs much more, and some of that exposure cannot be undone.

The scope sheet, field by field

Write each field as a decision someone can check, not a description. Values in the example column are illustrative and show the level of precision that works.

The scope sheet, field by field
FieldWhat to writeIllustrative entryWhy it bounds the dataset
TaskThe one decision or prediction the data supportsWhether an estimated meter read is later corrected after a field visitEverything not needed for this task is out of scope by default
Asset populationAsset classes and the ID scheme that joins themResidential and small commercial meters, joined by meter ID to premise IDStable IDs replace locations and network detail in the extract
Date windowStart and end dates and the reason for themThree full billing years after a meter system migrationPrevents mixing records from systems with different rules
Reading versionsWhich value is kept: original, estimated, corrected, finalOriginal and final value with a quality flag for eachCorrections are the label; dropping them removes the task
Rules by dateTariff, rate and contract versions joined by effective dateResidential rate versions with start and end datesA reading priced under the wrong rate version teaches the wrong rule
Linked actionsWork order, field visit and approval records that close each eventField service order ID, completion code and approver roleShows what was done and who decided, not only what was measured
ExclusionsRecord classes deliberately left outSubstation, transmission and protection records; customer names and addressesMakes the safety and confidentiality boundary visible to reviewers
Permitted useWhat the recipient may and may not do with the dataModel training and evaluation for meter-exception handling onlyTies the extract to a use that rights holders can approve

Which readings belong in the dataset: original, estimated or corrected?

Keep both the original and the final value of each reading, with the quality marker that explains the difference, because for most operating tasks the correction is the lesson. A dataset that holds only final billed values hides every estimate, edit and exception the business actually handled.

The Green Button data model shows why this field cannot be an afterthought. Its ReadingQuality element describes the quality of a specific reading or interval reading, notes that more than one quality may apply to the same reading, and says it is typically used only when problems or unusual conditions occur. In practice that means quality flags are sparse and easy to lose in a flat export, so the scope sheet should name the field and require it to survive extraction.

Record the revision sequence where you can: when the estimate was made, when a corrected value replaced it, which work order or validation step produced the correction and whether the customer was rebilled. Without timestamps on each version, a model cannot tell a quick correction from one that took several billing cycles.

Joining tariffs, contracts and work orders by the date they applied

Every reading, charge or decision should join to the rule version in force on its own date, not to the current one. Rates, riders, contract terms and operating procedures all change, and a dataset that joins history to today's rules will contain apparent errors that were correct at the time.

Public rate data shows the pattern. The U.S. Utility Rate Database on OpenEI added a supersedes field that links a rate to historical versions of the same rate, which is the structure an internal tariff table needs as well: one row per version with an effective date and an end date, and a link from each version to the one it replaced.

Contracts need the same treatment and a stronger confidentiality check. Supply agreements, interconnection agreements and service contracts often restrict disclosure of their terms. The scope sheet can usually keep a category label and effective dates while excluding the terms themselves.

Work orders close the loop. Join each event to the order that responded to it, the completion code, the time it closed and the role that approved it. Prefer the role over the person's name; the role is what the task needs.

The rights, confidentiality, critical infrastructure and use checklist

Run this checklist on the scope sheet before extraction and attach the evidence for each line. A line without evidence is an open question, not a pass.

The rights, confidentiality, critical infrastructure and use checklist
CheckQuestion to answerEvidence to attachNarrow or stop if
OwnershipDoes the company control these records, or do a parent, a pool operator, a joint owner or a vendor hold rights in them?Data ownership clauses in operating, joint ownership and software agreementsAnother party's consent is needed and has not been obtained
Customer dataDo records identify a customer or premise, alone or combined with account data?Field list showing which identifiers are removed or tokenizedCustomer-level interval data would leave without consent or a reviewed legal basis
Contract confidentialityDo any records reveal terms of agreements with confidentiality clauses?List of contracts touched and the clause reviewedTerms can only be shared with the counterparty's permission
Critical infrastructure informationDoes any record contain engineering, vulnerability or detailed design detail about generation, transmission, distribution or pipeline assets?Exclusion list reviewed by the security or CIP compliance leadRecords contain one-line diagrams, protection settings, network topology or cyber asset details
Regulated system informationIs any record subject to an information protection program, such as one required by NERC CIP-011?Confirmation from the compliance owner that no protected information is includedThe compliance owner cannot confirm the extract is outside the program
Location precisionCan locations be reduced to a level that still supports the task?Stated aggregation level, such as service territory or climate zoneThe task genuinely needs exact asset locations
Permitted useIs the intended use written down and approved by whoever holds the rights?Signed scope with the use statementThe recipient's use is open-ended or undefined
Safety boundaryCould the data be used to plan or execute field work without the company's procedures?Note confirming switching orders and safety procedures are excludedThe task requires operating procedures that are safety controls

Where the critical infrastructure line sits

Treat engineering and vulnerability detail about energy infrastructure as out of scope unless a security and compliance owner has reviewed it, even when it has not been formally designated. That rule keeps most operating datasets well clear of the line while leaving meter, billing, work order and customer service records available.

Federal rules describe what the line protects. Under FERC's regulation at 18 CFR 388.113, critical energy infrastructure information means specific engineering, vulnerability or detailed design information about proposed or existing critical infrastructure that relates to the production, generation, transportation, transmission or distribution of energy, could be useful to a person planning an attack, is exempt from mandatory disclosure under the Freedom of Information Act and does not simply give the general location of the infrastructure. FERC shares such material only with requesters who sign a non-disclosure agreement limiting its use and disclosure.

For bulk electric system entities, NERC's CIP-011 standard requires a documented information protection program for BES Cyber System Information, aimed at preventing unauthorized access to it. If any record in scope could fall under that program, the compliance owner decides, not the data team.

The definition is a useful test even where it does not formally apply: if a record would help someone understand how to disable or damage an asset, it does not belong in a training extract. General location alone is not the concern; detail is.

Customer-level usage data should only enter a dataset through a reviewed consent or legal basis, or after it has been reduced to a form that no longer identifies a customer or premise. Usage records joined to account details are personal information in most frameworks, and many state utility commissions set their own rules.

The Department of Energy's voluntary code of conduct for smart grid data, now called the DataGuard Energy Data Privacy Program, describes processes for customers to control access to their data for secondary purposes and is written for both utilities and third parties, without superseding federal, state or local rules. Green Button Connect My Data, which the Green Button Alliance describes as part of the NAESB REQ.21 standard, follows the same idea in practice: the customer authorizes a third party to obtain usage data from the utility.

For a scope sheet, the safe default is to keep the operational event, such as an estimated read and its correction, and drop or tokenize everything that identifies the account, the premise or the person. Counsel decides whether any customer-level detail can stay.

Illustrative example: scoping meter-exception records at a mid-size utility

This example is illustrative, not a real company. A combined gas and electric utility wants to license records that show how its meter operations team handles estimated and disputed reads. The first draft of the scope sheet asks for all meter, billing and field records for the full service territory.

The checklist narrows it in four moves. Ownership review finds that the meter data management system vendor contract gives the utility full rights in its data, so that line passes with the clause attached. The customer data line removes names, service addresses and account numbers and replaces meter and premise IDs with tokens. The critical infrastructure line removes feeder and substation identifiers that the field work orders had picked up, leaving climate zone as the only location field. The use line limits the extract to training and evaluation for meter-exception handling.

The meaningful exception: about one in ten work orders (an illustrative share) carry free-text technician notes that mention equipment settings at a nearby transformer. Rather than redact them line by line, the team excludes the notes field entirely and keeps the structured completion code. The dataset loses some nuance and gains a boundary everyone can verify.

Signals that the scope should shrink before anyone extracts data

Shrink the scope or pause when a check cannot be answered with evidence. Typical signals: nobody can say which agreement governs a source system's data, a field list includes topology, settings or vulnerability findings, customer identifiers cannot be removed without breaking the joins the task needs, or the intended use cannot be stated in one sentence.

Bring in specialist review in three cases: a security or CIP compliance owner for anything touching generation, transmission, distribution or pipeline engineering detail; privacy counsel for customer-level usage; and contract counsel where agreements with suppliers, joint owners or market operators are in play. These reviews are faster against a one-page scope sheet than against a raw export.

Using the scope sheet in a licensing conversation

A completed scope sheet and checklist are what a licensing partner needs to judge whether an energy dataset is worth pursuing, and they keep sensitive material from moving before that judgment is made.

For utilities and other energy operators, SourceX handles qualification, rights review, contracting, data preparation and delivery when they license selected operational records to AI developers. The process opens with a short company assessment that needs no uploads, and no meter, billing or work order record is delivered until the company has approved the scope and terms and signed. Interval data too large to send can stay on the utility's own servers or go out on encrypted drives; SourceX does not host multi-terabyte sets.

Frequently asked questions

What is an energy data supplier scope sheet?

It is a one-page definition of a dataset an energy company may license: the task, asset population, date window, reading versions, rule versions, linked actions, exclusions and permitted use. It is paired with a checklist covering rights, confidentiality, critical infrastructure information and use, so reviewers can approve or narrow the scope before any data moves.

Is utility operating data automatically critical energy infrastructure information?

No. Under FERC's rule, the category covers specific engineering, vulnerability or detailed design information that could help someone plan an attack, not general location or routine business records. Meter, billing and work order records are usually outside it, but anything with network topology, settings or vulnerability detail should be excluded unless your compliance owner approves.

Should a dataset keep estimated readings or only final values?

Keep both, with the quality flag and timestamps for each version. The correction from estimate to final is often what an AI system needs to learn, and final values alone hide the exceptions the business handled.

Can customer interval data be included if names are removed?

Not automatically. Interval data tied to a premise can still identify a household, and state rules and consent commitments may apply. Keep the operational event, tokenize or drop premise and account identifiers, and have privacy counsel decide whether any customer-level detail can stay.

Which tariff version should a historical reading join to?

The version in force on the reading's own date. Keep one row per rate version with an effective date, an end date and a link to the version it replaced, and join each reading or charge by date rather than to the current rate.

Sources

  • FERC's regulation defines critical energy infrastructure information as specific engineering, vulnerability or detailed design information about critical infrastructure that relates to energy production, generation, transportation, transmission or distribution, could be useful in planning an attack, is FOIA-exempt and does not simply give general location. Source
  • FERC requires requesters of critical energy infrastructure information to provide an executed non-disclosure agreement limiting use and disclosure of the information. Source
  • NERC's CIP-011 standard requires responsible entities to keep documented information protection programs for BES Cyber System Information to prevent unauthorized access. Source
  • The Department of Energy's voluntary code of conduct for smart grid data covers customer control of access for secondary purposes, is meant for utilities and third parties, and does not supersede federal, state or local rules. Source
  • The Green Button Alliance describes Connect My Data as part of the NAESB REQ.21 standard, used to share utility usage data with third parties that the customer authorizes. Source
  • The Green Button ESPI data model describes ReadingQuality as the quality of a specific reading value, notes that more than one quality may apply, and says it is typically used only when problems or unusual conditions occur. Source
  • OpenEI's U.S. Utility Rate Database added a supersedes field that links a rate to historical versions of the same rate. Source
  • The Department of Energy says its Voluntary Code of Conduct for smart grid data privacy was renamed the DataGuard Energy Data Privacy Program. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies