Proxy Variables in Structured Business Data: Finding Stand-ins for Protected Attributes

Why dropping the protected column does not remove the signal

Deleting a race or gender column removes the label, not the information, because other fields in operational data were generated by the same people and places. The working rule is simple: if the protected class can be predicted from the other attributes, a model trained on those attributes can reproduce disparate outcomes without ever seeing the class. Testing the data rather than the model makes the check usable before you have trained anything.

In business data the leakage is rarely one strong field. It is usually several weak ones that combine: a 5-digit ZIP, a preferred-language flag, a prepaid payment method and a weekday-evening contact pattern can each look harmless in a correlation matrix. Together they can identify a demographic group better than any single column. This is why pairwise screening alone gives false comfort.

Treat this as a data quality problem in the same family as label bias. If the outcome field reflects past human decisions, a proxy lets the model learn those decisions by another route; see historical decision bias in operational labels for the label side of the same issue.

Fields in operational records that most often act as proxies

Geography is the most common proxy in US structured data, and ZIP code is the textbook example for race and income. US fair lending analysis itself relies on this relationship: Bayesian Improved Surname Geocoding (BISG) estimates race and ethnicity probabilities from a surname combined with census geography where race is not collected. If an examiner can estimate race from surname plus location, a gradient-boosted tree can too.

The table below groups proxy carriers by where they appear in typical CRM, ERP, ticketing and billing exports.

Illustrative example: invented to show structure; it does not describe an available dataset.

Two fields deserve special handling. honorific is close to a direct encoding of sex and often marital status, so it should be treated as the attribute itself rather than a proxy. Free-text fields can restate the deleted attribute verbatim, so a structured proxy screen must be paired with text redaction review.

How to detect proxy features: three tests on a reference set

The core test is supervised: train a model to predict the protected attribute from every candidate feature, and measure how well it does. The balanced error rate of this predictor is a practical leakage measure: low error means the protected class is recoverable. Run it with the same feature engineering you plan to use downstream, because one-hot encoded ZIPs or target-encoded store IDs leak differently from raw values.

Use three layers, cheapest first:

1. Pairwise association. For each candidate field, compute Cramér's V (categorical vs. categorical), mutual information (any type) or point-biserial correlation (binary attribute vs. numeric feature). Rank fields and flag the top of the list. This catches single strong proxies such as postal_code or honorific.
2. Multivariate leakage model. Fit a gradient-boosted classifier (for example LightGBM or XGBoost) to predict the protected attribute from all candidate features using cross-validation. Report balanced accuracy and AUC per attribute and per group. Compare against a majority-class baseline so a skewed population does not make leakage look small.
3. Attribution on the leakage model. Use permutation importance or SHAP values on the leakage classifier to see which features carry the signal. Then re-run the classifier with the top feature removed: if AUC barely moves, the signal is spread across a cluster of fields and removing one does nothing.

Run the leakage model on intersections too, for example age band crossed with sex. A dataset can leak little about each attribute alone and a lot about a specific combination. For representation and intersectional coverage checks that complement this, see the dataset bias audit.

Where the protected attribute comes from when the dataset lacks it

Every proxy test needs a reference set where the protected attribute is known, and most licensed operational datasets will not contain it. The cleanest source is a consented, self-reported sample: an internal survey, an HR self-identification field, or a voluntary demographic module collected for fairness testing. Keep it separate from training data and restrict access.

When self-report is not available, estimated attributes are the fallback. BISG is the established method in US fair lending, but its individual-level accuracy has been publicly disputed. Use estimated probabilities for aggregate leakage measurement, never to label individuals, and report the estimation method next to every result.

Privacy rules also shape what geography you can keep. Under the HIPAA Safe Harbor method, geographic subdivisions smaller than a state are identifiers, with a limited exception for the first three digits of a ZIP code in sufficiently populous areas [2]. A health dataset de-identified this way may already have coarsened your strongest geographic proxy; see dates, ages and ZIP codes under HIPAA Safe Harbor for what that costs temporal and geographic models.

Proxies introduced by joins and feature engineering

Proxies are often created during preparation, not delivered in the raw extract. Joining a customer table to a geocoder adds census_tract and tract-level income; joining to a store table adds store_id, which inherits the demographics of the neighborhood. Run the leakage test after every join, not just on the supplier's original export.

Feature engineering amplifies weak signals. Target encoding of high-cardinality fields such as ZIP or branch can turn a categorical with roughly 40,000 levels into one dense, highly predictive number. Aggregates such as "share of customers in this ZIP who paid in cash" encode neighborhood composition directly. When records are linked across billing, CRM and support systems, the merged row carries more proxy fields than any single source; see record linkage quality in multi-system datasets.

Deciding what to do with each proxy

Not every proxy should be deleted; the decision depends on whether the field carries legitimate signal for the task and what removing it costs. A store ID may be essential for inventory forecasting and irrelevant for credit decisions. The decision table below maps findings to actions.

Illustrative example: invented to show structure; it does not describe an available dataset.

Feature transformation is the pre-processing option when dropping is too costly. Practitioner guides describe this pre-processing step as transforming features to reduce their correlation with sensitive attributes [3][4], so the protected class becomes harder to recover while as much task signal as possible is kept. Always re-run the leakage model after transformation, and measure the task metric so the trade-off is explicit.

Some sectors now expect this analysis. New York's Department of Financial Services, in Insurance Circular Letter No. 7 (2024), addressed insurers' use of external consumer data and AI systems in underwriting and pricing, including assessing whether such data acts as a proxy for protected classes [1]. Check sector rules and outcome tests, such as four-fifths ratio comparisons, with counsel before treating any threshold as a safe harbor.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Documenting proxies in a dataset register

A proxy finding that lives only in a notebook will be lost by the next retraining cycle, so record it in the dataset's documentation. Data Cards provide a structure for recording sources, collection methods, intended use and decisions that affect model performance [5], and the Croissant-RAI vocabulary makes responsible-AI metadata such as limitations and potential biases machine-readable alongside the dataset [6]. A per-field proxy register fits naturally inside either.

Illustrative example: invented to show structure; it does not describe an available dataset.

proxy_register:
  dataset: support_tickets_2021_2025_v3
  reference_set: consented_self_id_sample (n recorded in audit log; access: fairness team only)
  attribute_source: self-report; BISG probabilities used for aggregate checks only
  findings:
    - field: customer_postal_code
      attribute: race_ethnicity
      test: leakage_model_lightgbm
      auc_before: recorded
      action: coarsen_to_zip3
      auc_after: recorded
      task_metric_change: recorded
      owner: rai-lead
      decided: 2026-10-09
    - field: preferred_language
      attribute: national_origin
      action: keep_with_rationale
      rationale: routing model must assign interpreters
      monitoring: resolution_time_by_language_monthly
  rerun_triggers: [new_join, new_feature_encoding, supplier_refresh]

Set rerun triggers explicitly. A supplier refresh, a new geocoding join or a switch from one-hot to target encoding can reintroduce a proxy you already handled. Pair the register with your structured dataset validation checks so proxy tests run with schema and range checks on every delivery.

What to ask a data supplier before licensing records

Proxy analysis is only possible if the delivered data lets you see which fields exist, how they were derived and what was removed. Ask before the license is signed, because coarsening or dropping fields later can change what you paid for.

Illustrative example: invented to show structure; it does not describe an available dataset.

• Field inventory: a data dictionary listing every column, its source system and any derived or joined fields (geocodes, segment codes, scores).
• Removed fields: which protected or personal fields were dropped or replaced, and by what method.
• Geographic granularity: whether location is full ZIP, ZIP3, region or none, and whether that matches your task.
• Free text: whether notes and narratives are included, and how they were redacted.
• Reference data: whether any consented demographic data exists that could support a leakage test under the license, or whether you must bring your own.
• Allowed uses: whether fairness testing and derived-feature creation are permitted uses under the license.

When SourceX sources operational data, personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Diligence materials covering source, rights, preparation and allowed use are prepared per dataset, which can inform a proxy screen. Buyers can describe the structured records they need along with the fields they must audit. Broader quality criteria are covered in the training data quality assessment hub, and the AI data guide maps the other buyer topics.

Sourcing structured records you can audit for proxy variables

SourceX sources operational datasets, such as support and sales histories and finance and legal workflows, from US companies on request; categories are not inventory and a request does not guarantee a match. Every dataset is rights-reviewed and delivered under a license that defines records, uses, term and delivery, with each release approved by the supplying company. To scope a dataset with the field documentation your proxy tests require, start a buyer request.