Short answer
Rendered images from CAD or scanned models give exact labels, controlled variation and coverage of objects that are hard to photograph, but they miss real wear, packaging variation, lighting and clutter. Real images carry those properties and cost more to label. Perception programs typically need real images at least for validation, so decide the mix by testing on held-out real photos.
The buy-versus-render decision
A perception team that needs to recognize products has three sources of images: photographs licensed from a supplier, images captured by the team, and images rendered from 3D models. Rendering looks attractive when a manufacturer can supply CAD or when public 3D assets exist: labels are free, variation is a parameter and rare objects are available on demand.
The cost is the gap between a render and a camera frame. This page helps decide when that gap is acceptable. It is about images for recognition. For the separate question of whether recognition data can support handling, see object recognition data vs manipulation training data.
What rendering tools provide
Simulation platforms are designed to generate training data. NVIDIA's Isaac Sim, built on Omniverse, includes Replicator, which outputs annotated data including bounding boxes, segmentation masks and depth, and imports CAD and URDF into USD (NVIDIA). The labels come from the scene definition, so they are exact and free of annotator disagreement.
Public 3D assets can seed a rendering pipeline. Google Scanned Objects offers over one thousand scanned household items under a Creative Commons license, preprocessed for simulation (Downs et al., 2022). Objaverse collects more than 800,000 3D models with captions and tags, sourced from Sketchfab under Creative Commons licenses (Deitke et al., 2022). License terms vary by asset, so a pipeline using them needs per-asset records; see trademarks in training images and product image rights.
These sources describe what is available, not how well rendered training transfers to a particular camera and product set. A buyer should find that out with a test.
Where renders and real images differ
The table below lists properties to check on your own products. These are general considerations, not findings from the sources above.
| Property | Rendered from CAD or scans | Real photographs |
|---|---|---|
| Label accuracy | Exact by construction | Depends on annotators |
| Pose and viewpoint control | Full | Limited by capture protocol |
| Lighting and background variation | Parameterized; can look unlike real scenes | Real, but uneven coverage |
| Packaging and print detail | Only if the model carries textures | Present as shipped |
| Wear, dents, labels, shrink-wrap | Absent unless modeled | Present |
| Transparent and reflective items | Hard to render faithfully | Real, hard to label |
| Clutter, occlusion, hands | Must be simulated | Present when captured in context |
| Rights | Depend on the model's owner | Depend on the photographer and subject |
| Cost per labeled image | Low after setup | Higher; includes capture and annotation |
Setup cost for rendering is real: you need models of the right products, materials that look right and scene variation. Sparse CAD is not enough for texture-heavy items such as printed packaging.
The comparison above lists properties to test, not established findings; how large each difference is depends on the products, renderer and camera. Models of the right products come from several places, including manufacturer 3D models and CAD, scans, or public assets, and each has different rights and fidelity. A pipeline that renders only common items will under-serve the tail; see long-tail object coverage.
Cases where rendering is likely to help
- Rare or new products with no photographs yet, where a manufacturer has CAD.
- Pose coverage, filling in viewpoints that photography would miss.
- Pretraining or augmentation before fine-tuning on real images.
- Label types hard to produce by hand, such as pixel-exact masks, depth and 6D pose; see 6D pose annotation.
- Stress tests, where you vary one factor at a time.
Cases where real images are hard to replace
- Packaging variants and appearance changes across SKUs and seasons; see packaging variants and SKU appearance.
- Text, barcodes and labels that must be legible at real resolution.
- Cluttered shelves, bins and hands representative of the deployment.
- Evaluation. If the claim is about performance in the real world, the test set must be real, held out by site, camera or time. Rendered test images do not support that claim.
Real images need rights and consent review, and the cost of capture is higher. See packshot vs in-situ images for what each type shows.
Illustrative comparison test
# Illustrative only - design sketch with placeholder values
objective: "Choose a real/synthetic mix for 200-SKU recognition"
test_set: real photos, held out by store, camera and month
training_arms:
R: real images only (fixed budget)
S: rendered images only (CAD-based)
M1: S + fine-tune on a small real subset
M2: R + S as augmentation
report_by:
- per-SKU accuracy on test set
- accuracy for SKUs absent from real training data
- failure examples by lighting, occlusion, packaging variant
decision: "pick the cheapest arm meeting the buyer's threshold on the
worst-case slice, not only the mean"
The slice report matters. A mean accuracy can hide that rendered-only training fails on printed packaging or on glossy items.
Instance and category effects
Whether renders suffice also depends on the recognition level. Telling a mug from a bottle is a different problem from separating two SKUs that differ by a printed flavor name, which a render without faithful textures cannot show. Settle the level first; instance-level vs category-level recognition data sets out what each needs.
Limitations
Neither source gives manipulation information. Rendered or real, an image shows appearance; it does not show how an object behaves when handled. Physical properties belong to physical property annotations, and simulation requires more than geometry, as in simulation-ready object assets.
Rendering can also hide the wrong problems. A model that scores well on rendered validation images may have learned renderer artifacts. If your pipeline rests on manufacturer CAD, ask whether it is current, as geometry and finishes change between revisions, a concern covered in SKU churn and catalog refresh.
Questions to ask suppliers
- Do you hold CAD or scan models for these products, and are textures and materials included?
- If you supply real images, how were they captured: camera, lighting, backgrounds, viewpoints?
- Do the real images include the packaging variants we will see in deployment?
- Who owns the models and the photographs, and can both be licensed together for training?
- Do any images or models include third-party trademarks, people or logos?
- Can you supply a small paired set, with real photos and the matching model for the same SKUs, so we can measure the gap?
- How current are the models relative to product revisions?
- What labels come with real images, and how were they produced and checked?
- Can the rights for rendered derivatives be separated from those for the originals?
Explore sourcing and licensing options
If you are weighing a render pipeline against licensed photographs, describe the products, image types and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX follows up about potential sources. Start at the humanoid training data guide or the objects hub, and see the site's guide to licensed vs synthetic vs scraped data.
Explore sourcing and licensing options with SourceX
Sources
- Isaac Sim - Robotics Simulation and Synthetic Data Generation (with Isaac Lab) — annotated synthetic output and CAD import.
- Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items — scanned household assets under a Creative Commons license.
- Objaverse: A Universe of Annotated 3D Objects — large collection of 3D models with Creative Commons licenses.