Equipment Error Codes and HMI Screens

Why screens and codes come first

A technician arriving at a faulted machine usually reads before touching anything: a code on a panel, a banner on an HMI, a lamp pattern, a line in an alarm history. A robot doing the same work has to read the same cues, recognize which screen it is on, and map what it sees to a next step. Training that reading ability needs paired data: the screen or alarm as displayed, the documented meaning, and what was done about it.

This page covers sourcing that paired material. It is narrower than work-order histories, which record the repair, and troubleshooting guides and fault trees, which record the manufacturer's intended path. Here the object is the signal the machine gives off. It belongs to the maintenance cluster of the humanoid data guide.

The assets in this family

Plants and equipment makers hold the first four. Field service organizations, facilities teams, fleets and MRO providers hold the links between a displayed code and the repair outcome, which is the part that makes the data useful for diagnosis. Fault code documentation overlaps with OEM service documentation, where manufacturer permissions are the first question.

What each part can train

• Screen understanding. Screenshots paired with element boxes and text support reading and grounding of interface elements.
• Code interpretation. Code tables and logged codes support lookup, normalization and mapping of codes to causes in language models.
• Diagnosis ranking. Codes linked to repair outcomes support ranking likely causes. A published technical-support set illustrates the pairing of a described problem with a vetted answer: TechQA pairs forum questions with accepted answers from company technotes, with 600 training, 310 development and 490 evaluation question-answer pairs (Castelli et al., 2019). It is software support, not machine faults, but the structure carries over.
• Display reading in the field. Photos of segment displays and lamps taken at working angles support perception under glare, wear and oblique views. Dial and gauge reading is a separate problem; see analog gauge and meter reading.

What this data cannot support

• Physical demonstrations. No hands, tools, torque or motion appear in a code table or a screenshot.
• Control. A humanoid needs robot-specific observations and actions to press a reset or open a cabinet. Screens tell it what to do next, not how to do it.
• Live interface behavior. Static screenshots do not show response time, scrolling or multi-step menu paths. Screen recordings or logs do.
• General equipment knowledge. Codes are often specific to a make, model and firmware version. A code table for one controller may not apply to another family, even from the same maker.
• Causality by itself. A code often indicates a downstream symptom. Without the repair outcome, a code-to-cause mapping is a guess.

Illustrative record: code, screen and outcome

# Illustrative only - synthetic values, not an available dataset
record_id: ecr_00418
equipment: {class: injection_press, model_ref: model_C3, firmware: fw_4.2, asset_id: asset_0071}
event:
  code: "E-217"
  text_as_displayed: "HYD PRESS LOW"
  severity: stop
  timestamp: 2026-03-04T08:12:09Z
screen:
  image: scr_00418.png
  resolution: 1280x800
  elements: [alarm_banner, status_bar, reset_button]
  capture: {method: panel_photo, angle_deg: 20, glare: moderate}
documented_meaning: {source: service_manual_rev_D, section: "7.3", probable_causes: [filter_clogged, pump_wear]}
outcome: {work_order: wo_20931, fix: replace_filter, first_time_fix: true, repeat_within_30d: false}

Adapt the fields to the model you train. A text-only diagnosis model can drop the screen block; a visual agent needs capture method, resolution and element annotations.

Preparation before this data is usable

• Normalize text. The same alarm may appear as "E217", "E-217" and "217 - HYD PRESS LOW". Decide a canonical form and keep the original string.
• Link assets. Join each code to an asset, model and firmware version, ideally using the facility's asset hierarchy from a CMMS export.
• Redact customer names. Screens often show plant names, operator IDs, recipes or network addresses. Plan redaction, and check that it does not obscure the alarm area.
• Annotate screens. Label UI elements and alarm regions. Assign a second reviewer for a sample.
• Mark the split. Hold out whole sites or machine models for evaluation rather than random screens, so near-duplicate screens do not leak across the boundary. See troubleshooting evaluation sets.

Permissions and provenance

Rights in this family are mixed. Code tables and manuals are usually authored by the equipment maker; screenshots belong to the plant that captured them; the links between code and repair belong to the service organization, and may contain customer data. Treat these as separate rights questions rather than a single license. Redacting names does not by itself settle whether a customer allowed reuse. If you plan to train on manufacturer documentation, confirm what the maker permits, and see licensing terms for robotics training data.

Reviewing a sample before you commit

Ask for a sample of 50 to 100 records and test it against your use, not the supplier's description.

• Coverage matrix. Tabulate codes by equipment model and firmware. Gaps show up immediately, and so does over-representation of a few nuisance alarms.
• Root versus consequential alarms. One fault often triggers a cascade. Check whether the data marks the first alarm or lists all of them with equal weight; a model trained on the cascade may learn the symptom order, not the cause.
• Language and locale. Panels may display other languages or abbreviations. Confirm what the text field holds and whether the original is kept next to any translation.
• Duplicates and near-duplicates. Repeated screenshots of the same alarm inflate counts and leak across splits. Ask how they were removed.
• Free-text links. Compare a few logged codes with the maintenance logbook text for the same events, to see whether technicians wrote down what the machine displayed or a paraphrase.

If the sample passes on one equipment class, do not assume the rest of the supplier's holdings are equally clean. Sample each class you intend to buy.

Questions to ask suppliers

1. Which makes, models and firmware versions do the codes and screens cover, and how many records per model?
2. Are screens provided as original captures, or re-rendered from text?
3. Can each code be linked to an asset ID, and to a repair outcome such as a work order or a repeat-visit flag?
4. Is the alarm text kept as displayed, and in which languages?
5. Were screens captured as screenshots from the device, panel photos, or video, and at what resolution?
6. Do screens show customer names, operator IDs or network details, and how were they handled?
7. Who authored the code tables, and does the supplier have the right to license them for training?
8. How are codes that changed between firmware versions represented?
9. Are acknowledged, nuisance and repeated alarms flagged, or is every logged event treated alike?

Discuss your requirements

If your project needs fault-code or HMI data for a named equipment class, describe the models, modalities and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match, and nothing is contracted until a supplier agrees. For related record types, see the catalog entries on field service and maintenance records and screen recordings.

Explore sourcing and licensing options with SourceX

Sources

• The TechQA Dataset — Published technical-support QA set showing how problem descriptions pair with vetted resolution documents.