Short answer
Food recognition data for robots differs from packaged-goods data because no two apples, steaks or leaves look the same, and appearance changes with ripeness, cutting, cooking and plating. Useful sets show each item across stages and conditions, in the real kitchen or packhouse context, with labels for state as well as name. Catalog photos of packaged SKUs do not cover this.
The buyer's problem
A manufactured SKU has a fixed design: variation comes from packaging revisions and lighting. Natural food varies within a class: shape, color, size, blemish, moisture and orientation. It also transforms during a task, with onions that are whole, peeled, sliced or sauteed. A recognition model that knows "tomato" from clean produce photos may miss a half tomato under a sauce, or a cherry tomato in a cluster.
This page covers sourcing image data for food and produce recognition. It sits next to grocery and retail product recognition, which deals with packaged goods, and object state recognition, which covers state changes in general.
What published kitchen data shows
The most relevant large public reference cited here is EPIC-KITCHENS-100, a first-person video dataset of kitchen activity. It comprises 100 hours, 20 million frames, 90,000 actions and 700 variable-length videos from 45 environments, recorded with head-mounted cameras. Annotation began with a "pause-and-talk" narration stage, followed by transcription, parsing and temporal segmentation, and the release provides 54% more actions per minute than the previous version and six challenges (Damen et al., 2020).
What a buyer can take from it is the shape of useful kitchen data. It covers many environments rather than one. It records objects as they are used and transformed. It ties labels to actions, which produces implicit state labels such as "cut onion". It is video rather than still images. A commercial food-service or packhouse set may be more specific, with more produce, bulk containers and cleaning steps, and fewer rare objects.
Where the variability comes from
| Source of variation | Example | Why it matters |
|---|---|---|
| Within-class appearance | Ripe versus unripe, blemished versus clean | Class boundaries are fuzzy |
| Preparation state | Whole, peeled, diced, cooked | Same item, different look and handling |
| Packaging and container | Bulk crate, tray, bag, bowl | Context cues for the model |
| Mixing and occlusion | Items in a salad, in a pot, under sauce | Hard to segment |
| Lighting and surface | Wet, shiny, steamy surfaces | Specular highlights, condensation |
| Seasonality and region | Varieties change through the year | Coverage gaps in a single snapshot |
| Allergen or dietary relevance | Visually similar but meaningfully different items | Errors have consequences beyond appearance |
Seasonal and regional variety means one-time collection captures one slice. Ask for capture dates and sites, and plan refreshes if produce variety matters to you.
Supplier types
Food service and restaurants hold photographs and video of dishes, prep stations and plating. Images are often tied to a recipe or menu item, which gives a label, and sometimes to inventory records.
Grocers hold produce photography for online listings and for shelf audits. These are often clean and front-facing, with limited state variation.
Packhouses and processors hold quality-inspection images of produce along lines, where grading data (size, color class, defects) may be labeled. Those labels are valuable for variability. See defect image datasets for the inspection side.
Annotation providers add labels to images you already hold, and need a food vocabulary and a state schema.
Existing data is a good start for appearance. Commissioned capture is more relevant when you need a specific produce list photographed through states, or in-hand views; see object-in-hand images. For wider kitchen contexts, see kitchen equipment and utensils.
Illustrative label schema
# Illustrative: synthetic values only
item_record:
image_id: img_66021
items:
- {class: tomato, variety: roma, state: halved, ripeness: ripe, mask: poly_001}
- {class: basil, state: whole_leaf, ripeness: fresh, mask: poly_002}
container: bowl_steel
surface: wet_countertop
lighting: overhead_led
site_id: site_014
capture_date: 2026-08-14
allergen_tags: [none]
taxonomy: closed_vocab_v3 # map to your own or a published product classification
Closed vocabularies matter. "Pepper" can mean bell pepper, chili or the ground spice. Align class names to a taxonomy you control or to a published classification; the product taxonomies page discusses structured options.
A sampling plan for the long tail
Menus and produce lists have a long tail. A kitchen may handle a few dozen staple items daily and hundreds occasionally. Define the staple list first and require dense coverage of it across states and sites. Then choose a second tier of occasional items where a single clean example per state is enough for retrieval or few-shot methods, and a third tier of items the robot should flag as unknown. Ask suppliers to report counts per tier. Treat photographs from separate days as separate samples, because produce and plated dishes change. Keep items from some sites out of training so you can test whether the model is recognizing the food or recognizing the kitchen.
Acceptance criteria
- State labels present for at least the transformations your robot must handle.
- Within-class spread. Report the number of distinct instances per class, not only images; many photos of the same apple count as one.
- Site and date diversity stated.
- Held-out items. Keep varieties, sites or time periods out of training; see novel object generalization evaluation.
- Long-tail coverage. Ask for the list of rare items and their counts; see long-tail object coverage.
- Label consistency audited on a random sample, with disagreements by class.
What this data cannot support
Images identify items; they do not say how a ripe fruit yields to a grip, how slippery wet leaves are, or how much a soft item deforms. Those need physical-property and interaction data; see deformable product properties. Food safety, allergen handling and hygiene rules are procedural and regulatory matters that images do not capture. Edible goods change over time, so images taken once may not match items a week later. Appearance does not reveal spoilage reliably.
Permissions and provenance
Food images from restaurants and grocers can contain staff, customers, menus, brand packaging and screens. Ask who appears, what consent exists, and who can license training use. Kitchen video is especially likely to include workers. See worker consent and identifiable people. This is not legal advice.
Questions to ask suppliers
- How many distinct instances and varieties per class, not only image counts?
- Which preparation and ripeness states are labeled?
- How many sites and what period does the capture cover?
- Were the images taken in routine operation or staged?
- What vocabulary or taxonomy defines class names?
- Are allergen-relevant or look-alike items labeled separately?
- Do any images include people, and what consent applies?
- Can you provide a sample showing the same item across states?
Discuss your requirements
If you can name the food items, states and kitchens or packhouses your robot will work in, describe the requirement to SourceX. SourceX reviews it, follows up about potential US sources, and checks the data and the supplier's licensing permissions; new recordings are a separate path scoped case by case. A request does not guarantee a match. Start from the humanoid training data guide or the product and object data hub. See also the site's entries on egocentric video of manual work and images and inspection photos.
Discuss your humanoid training-data requirements with SourceX
Sources
- Rescaling Egocentric Vision — EPIC-KITCHENS-100: 100 hours, 45 kitchens, head-mounted cameras, narration-based action annotation.