Short answer
Catalog text pairs product titles, descriptions and attribute fields with images, which helps a vision-language model link words to objects. It gives names, categories, colors, sizes and materials in the seller's language. It does not describe handling, in-situ appearance or the commands people give a robot, so buyers typically add instruction-style language separately.
What the buyer is trying to solve
An instruction such as "bring me the blue cordless drill, not the gray one" works only if the model can connect words to things. Training that connection needs many examples of object names and descriptive words attached to images of the right object. Catalogs are one of the few large, commercially maintained places where a human has already written, for each item, a title, a description and a set of structured fields.
The question for a language-grounding team is how far that text carries. This guide covers what catalog text contains, how to evaluate a supply, what it will not give you, and what to ask before licensing. For the image side of the same assets, see product catalog data for robot perception.
What catalog text contains
A typical product record has several kinds of language, and they behave differently in training.
| Text type | Example (Illustrative) | Grounding value | Main weakness |
|---|---|---|---|
| Title | "Cordless drill 18V, 2 batteries, case" | Names the object; short; consistent | Includes brand and pack information not visible in images |
| Bullet features | "Keyless chuck; LED work light" | Names parts and functions | Marketing phrasing; some features invisible |
| Long description | Paragraph of use and benefits | Usage context, synonyms | Promotional, repetitive across variants |
| Structured attributes | color: blue; material: ABS | Clean, queryable labels | Value lists vary across suppliers |
| Category path | Tools > Power tools > Drills | Hierarchy for open-vocabulary tests | Each retailer has its own tree |
| Q and A or reviews | Customer wording | How people refer to items | Third-party text, own rights, noisy |
The scale of what a single marketplace corpus can hold is visible in published work. The Amazon Berkeley Objects collection pairs catalog images and metadata for 147,702 product listings and 398,212 unique catalog images, with up to 18 metadata attributes per product, and links 7,953 products to artist-made 3D models (Collins et al., 2021). That shows the shape of catalog language data, not what any particular supplier holds.
What this text can support, and what it cannot
Catalog text can support:
- Object naming and synonym learning. Different sellers call the same item different things, which exposes a model to varied wording.
- Attribute grounding. Color, size, material and quantity words tied to a specific image.
- Open-vocabulary retrieval and detection. Matching a free-text query to images of the right category.
- Pretraining of the language side of a vision-language model before task-specific fine-tuning.
It cannot support:
- Instruction following. Nobody writes "put the red mug on the top shelf" in a product description. Commands, spatial relations and disambiguation come from instruction or referring-expression data; see referring expression annotation and instruction-paired demonstration datasets.
- Handling knowledge. "Fragile" or "dishwasher safe" is a seller claim. It is not a measured property, and descriptions rarely record mass distribution, slipperiness or how a part opens.
- How the item looks at work. Catalog images are staged shots; see packshot vs in-situ images.
- Truth in every field. Marketing copy may name features that do not appear in the photo, and one description often covers several variants.
None of this converts catalog language into robot control data. It supplies object vocabulary that a separate control policy still has to act on.
Quality problems to expect
Catalog text has predictable defects. Variant collapse is common: one description shared across colors or sizes, so the words do not match the picture. Titles are stuffed with search keywords. Attribute values mix units and spellings. Descriptions are copied between sellers, which creates near-duplicates that inflate counts and leak across train and test splits. Language may be machine-translated. Text and images can be updated at different times, so pairs drift.
A normalization step usually pays for itself. Mapping category text to a shared scheme, such as the four-tier Segment, Family, Class and Brick hierarchy in GS1's Global Product Classification, gives you stable grouping across suppliers; see product taxonomies and the object attribute schema.
How to test a sample before you license
Ask for a sample of a few hundred records and run four checks yourself, none of which needs a trained model.
First, read 100 title-and-image pairs side by side and count how many titles describe something that is not visible: pack counts, bundled accessories, brand slogans. Second, group records by exact-description hash to measure how much text is shared across variants. Third, check that every attribute value appears in a controlled list, and count how many distinct spellings exist for the five most common colors. Fourth, pick 20 items you know from your own site and see whether the supplier's wording matches how your operators name them.
The last check is often the most revealing. Workplace names ("the 10-inch tray", "the gray totes") differ from retail names, and a model grounded only on retail wording may need a bridging layer of site vocabulary. That bridge is usually written by annotators or by the site itself, not found in a catalog.
Choosing between suppliers
Retailers hold broad coverage and customer-facing wording. Manufacturers hold the most accurate specifications and part names. Marketplaces hold the widest variety but the least consistent text, and the rights position is often the hardest. The trade-offs are compared in object data from retailers vs manufacturers. Industrial catalogs carry long technical attribute lists with little narrative; see industrial distributor catalogs.
Illustrative delivery package
# Illustrative only - fields and values are examples, not an available dataset
package: catalog_text_pairs_v0
records: one row per item_id and variant_id
text:
title: original language, plus locale code
features: list of strings
description: plain text, markup stripped
attributes: {color, material, size, unit, pack_count} # supplier values
category_path: supplier tree, plus mapped GPC brick code if available
images:
files: image_id list per variant, with shot_type (packshot, angle, detail, lifestyle)
provenance:
text_author: supplier_staff | manufacturer | marketplace_seller | customer
text_last_modified: ISO date
license_id: per record
exclusions: customer reviews, supplier pricing, personal data
Decide up front whether review text, price fields and seller contact data are excluded; they carry different rights and add little to grounding.
Permissions and provenance
Product text has authors, and who wrote it matters. A brand writes its own descriptions, a retailer may rewrite them, and a marketplace may hold seller-submitted text under platform terms. Reviews are written by customers. Ask the supplier which of these they can license. Copyright questions about training on written works are the subject of an official report from the U.S. Copyright Office on generative AI training, which discusses where copying occurs in development and licensing options. This is not legal advice; take counsel on your own use. Image rights are separate and are covered in product image rights.
Questions to ask suppliers
- Who wrote the descriptions: you, the manufacturer, a content agency or sellers on your platform?
- Is text unique per variant, or shared across colors and sizes?
- In which languages and locales does the text exist, and is any machine-translated?
- How are attribute values controlled, and can you supply the value lists?
- Can you provide text with modification dates and the matching image versions?
- Which fields can be licensed for model training, and which are excluded by third-party terms?
- Are historical versions of descriptions retained, including for discontinued items?
- What share of records have no attributes or only a title?
Discuss your requirements
If you can state the languages, categories and fields you need paired with images, describe the catalog text requirement to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match, and nothing is contracted until a supplier agrees. Start from the humanoid training data guide or the object data hub.
Explore sourcing and licensing options with SourceX
Sources
- ABO: Dataset and Benchmarks for Real-World 3D Object Understanding — scale and structure of catalog images and metadata paired with 3D models.
- GS1 Global Product Classification (GPC) — four-tier classification with attributes and values.
- Copyright and Artificial Intelligence, Part 3: Generative AI Training (pre-publication version) — official discussion of copyright and licensing in AI training.