Pilot release: what 30 photographed shelves held
Published 2026-10-06. A pilot on public photographs, named as such on every line. These are Wikimedia Commons photographs of fridges, freezers and pantries under open licences, labelled by hand against the reader's own output, and they stand in for household shelves until households supply their own. Nothing here is projected.
The set
| Measure | Count |
|---|---|
| Shelves photographed | 30 |
| Items labelled | 341 |
| Items with a legible brand | 89 (26%) |
| Shelves holding a container of leftovers | 13 (43%) |
What the shelves held
Share of the 30 shelves holding each ingredient, after mapping labels to a common name (milk from three cartons counts once).
| Ingredient | Shelves | Share |
|---|---|---|
| milk | 11 | 37% |
| peas | 10 | 33% |
| apples | 9 | 30% |
| hummus | 9 | 30% |
| lemons | 9 | 30% |
| eggs | 7 | 23% |
| yogurt | 7 | 23% |
| corn | 6 | 20% |
| mayonnaise | 5 | 17% |
| butter | 5 | 17% |
| oranges | 5 | 17% |
| cheese | 5 | 17% |
Brands read
Brands as printed on the label, by the number of shelves they appeared on. A brand the reader could not read is absent, so the list is bounded by legibility at arm's length.
| Brand | Shelves |
|---|---|
| Sabra | 6 |
| AH | 2 |
| H-E-B | 2 |
| Crush | 2 |
| Starbucks | 2 |
| Del Monte | 2 |
| Campbell's | 2 |
| Kraft | 2 |
| StarKist | 2 |
| Heinz | 2 |
How well the reader read
On these 30 photographs the reader found 88% of labelled items (299 of 341) and matched brand and size on 84% of them. Of the 89 items with a legible brand it returned the right brand on 63 as scored, 71%, and about 81% after scorer and label artefacts were removed; the misses sit on labels at distance or turned away. The capture rule that follows is one shelf per frame at arm's length. The scoring file is published with the engine's records.
Licences of the source photographs
| Licence | Photos |
|---|---|
| Public domain | 18 |
| CC BY-SA 4.0 | 5 |
| CC BY-SA 3.0 | 2 |
| Public Domain | 1 |
| CC BY 4.0 | 1 |
| CC0 | 1 |
| CC BY-SA 2.5 | 1 |
| CC BY 2.0 | 1 |
What comes next
The first household release publishes when 500 shelves photographed through the home page with the sharing box ticked are on file, by region and household type where the count supports it, each figure with its n. The series behind it is available to licensees under the terms on the privacy page.