Short answer
Lab recognition data has to separate small, transparent and near-identical items: tubes, plates, tips, pipettes, reagent bottles and bench instruments. Lab suppliers often hold catalog images and CAD; labs may hold in-use photos that are harder to release. Recognition data supports identification and state labeling, but it does not teach a robot to pipette or seal a tube.
What makes lab objects hard to recognize
A lab bench concentrates the cases that general object datasets avoid. Consumables are small. Many are transparent or translucent, so the contents and the background show through. Families of items look alike and differ in a printed volume mark, a cap color or a thread. Instruments carry displays, indicator lights and doors whose state matters as much as the identity of the machine.
For a buyer, the practical question is which of those difficulties your use actually meets. A humanoid fetching racks of sealed tubes needs a different perception stack from one that reads an instrument's door state. Be specific in the request, because the image set that serves one rarely serves the other.
This page covers sourcing for recognition. Where image resolution is the main lever, the small object resolution requirements page covers capture parameters; for glass and plastic, see transparent and reflective objects. The cluster overview is the product and object data hub.
Object families worth listing in a request
- Tubes and vials. Cap type and color, rack position, fill level, label presence, open or closed.
- Plates and trays. Well count, lid on or off, seal film, orientation.
- Pipettes and tips. Pipette body type, tip box, tip attached or not, tip eject state.
- Bottles and carboys. Label text, hazard pictograms, cap, level.
- Bench instruments. Make and model, door or lid state, display content, indicator light.
- PPE and sundries. Gloves, wipes, waste bins, sharps containers.
Hazard pictograms and chemical labels are their own subject; see hazmat and GHS labels. Whether a model must tell individual units apart or only classes is treated in instance versus category recognition.
A published reference for lab scenes
FineBio recorded 32 participants performing mock biological experiments in multi-view video, 14.5 hours in total, with hierarchical annotation of protocols, steps, atomic operations, object locations and manipulation states, plus baselines for step segmentation, atomic operation detection and object detection (Yagi et al., 2024).
It covers a lab setting, but of a specific kind: mock experiments by participants, not production laboratories. Treat it as a reference for annotation structure and capture design. It does not tell you how consumables appear in your lab, and the quantities above describe that study, not what your project requires. Requirements depend on architecture, task and embodiment.
Where existing assets might come from
Lab suppliers and manufacturers hold product photography, spec sheets and sometimes 3D models of labware. These are clean and systematic, which suits catalog-style recognition and attribute grounding. They are packshots: no hands, no spills, no racks half full. For the packshot-versus-field difference, see packshot versus in-situ images.
Laboratories hold the realistic material: photos from audits, training, equipment logs and incident reports. Release is complicated. Images can show samples, patient or participant identifiers, proprietary protocols, unpublished instruments and staff faces. Ask the owner what review has been done, and note that blurring a label does not settle who may license the image. This is not legal advice.
Commissioned capture is the likely route for gaps. A short, designed capture of your own consumable set on your own bench, from the heights and angles of your robot's cameras, usually fits better than a large collection from elsewhere. It needs a supplier or lab to host it, plus consent from anyone in frame. Discuss feasibility with SourceX; the choice between routes is covered in existing assets, new collection or specialist partners.
Illustrative request specification
The block below is illustrative, with synthetic values. The numbers are arbitrary examples of fields a buyer might set, not recommendations or minimums.
# Illustrative lab-object request (synthetic values)
object_families: [tubes, plates, tip_boxes, bench_instruments]
views: [bench_height_oblique, top_down, handheld_closeup]
states_to_label:
- tube: [capped, uncapped, in_rack, loose]
- plate: [lid_on, lid_off, sealed]
- instrument: [door_open, door_closed, display_on]
transparent_items: include_with_background_variation
resolution_note: "set so the smallest printed mark you need is legible"
metadata: [site_id, bench_id, lighting_type, camera_height_cm]
split_rule: "hold out whole labs, not random frames"
What lab recognition data cannot support
Images identify objects and states. They do not show how a pipette thumb-wheel resists, how much force closes a screw cap, how a tube slips in a gloved hand or how a plate sloshes when carried. Manipulating lab consumables needs robot or teleoperation data, or instrumented capture with synchronized sensors; recognition sets and catalog photos cannot stand in for it.
Specific limits:
- Transparency. Standard depth sensors and detectors often struggle with clear plastic and glass. Ask what sensing the images were taken with.
- Similarity. Two tubes differing by a cap color might share a label in a catalog and need different treatment on the bench.
- Liquids. Fill level and meniscus are visible only under the right lighting and viewpoint.
- Safety. Recognizing a hazard label is not authorization to handle the material.
- Evaluation. Test on a lab, or a consumable vendor, not seen in training.
Acceptance checks you can set
- Each image carries object family, state labels and a bench or site ID.
- Labelers had a written glossary, and a sample was reviewed by someone with lab experience.
- Transparent items appear on varied backgrounds, not only a uniform white.
- Near-duplicate frames from one tray are grouped, not counted as independent samples.
- Sample, participant or patient identifiers are absent or covered by the license terms.
Two neighboring topics: object state recognition covers capped versus uncapped style labels in more depth, and object recognition versus manipulation data explains why neither kind of image set teaches handling.
Questions to ask suppliers
- Which labware families and brands appear, and which do not?
- Were images captured on benches in use, or in a photo studio?
- What camera heights, lighting and backgrounds are represented?
- Are state labels (capped, filled, in rack) present, and how were they checked?
- Do any images show samples, identifiers, protocols or people?
- Who owns the photographs and any 3D models, and are manufacturer logos or trademarks restricted?
- Is the supplier able to link catalog identifiers to the images?
- Could a bench or lab be held out for evaluation?
- If the set is incomplete, can a targeted capture be arranged with the owner's consent?
Discuss your requirements
If you are specifying lab-object perception data, describe the object families, states and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If no existing asset fits, SourceX can scope a commissioned collection of new recordings. A request does not guarantee a match; SourceX follows up about potential sources. The humanoid training data guide maps the broader routes, and the catalog entry for expert annotations and labels covers related labeling data.
Discuss your humanoid training-data requirements with SourceX
Sources
- FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation — mock-experiment video with object and manipulation-state annotation; a labeling and capture reference for lab scenes.