# How accurate is search by photo?

We measured it, on photos the model had never seen, ranked against the whole species library. Below is the number, the interval around it, how it was obtained, and what it does not cover. Every figure on this page comes from one dated, reproducible run.

## Results

**Top-1 accuracy:** **72.5%** (95% CI 70.2–74.7%)

The single best match returned is the correct species.

Wilson interval on 1,531 photos. If the measurement were repeated many times, intervals computed this way would contain the true accuracy 95% of the time.

- **Top-3 accuracy:** 87.3% — The correct species is somewhere among the three best matches — a friendlier claim than top-1, and never the number we lead with.
- **Top-5 accuracy:** 91.2%

Measured over 1,531 held-out photos across 132 species, each ranked against the full 152-species library. That library snapshot (1,240 images) was pinned at 2026-08-31T08:18:30.295Z using model `bioclip-vit-b16-vision/onnx-fp32/v1`.

## What this measures

This is a 152-way identification, not a multiple-choice pick from a handful of candidates: every held-out photo is ranked against every species in the library, including species that contributed none of the held-out photos themselves — they still compete as candidates, exactly as they do for a real search. A uniform random guess on this task would score about 0.7%.

- **Counts as correct:** The first row is the right species.
- **Counts as wrong:** The second row is right. The genus is right and the species is not.

## The measurement path is the production path

Each held-out photo goes through the exact same identification path a real search uses — the same image-matching model, the same ranking — measured against a stored index pinned to one library snapshot. The harness refuses to produce a number at all if the model it runs against doesn't match the index it was given, and the figure is re-run whenever the library changes materially — though that re-run is an intent, not an automatic guarantee.

We also scored the run three ways, because an average can be chosen to flatter. They agree within 0.4 percentage points:

- **Micro** (72.5%) — Every photo counts once.
- **Macro** (72.9%) — Every species counts once, so common plants cannot carry the number.
- **No minimum-photo floor** (72.5%) — Recomputed with no minimum held-out photos required per species — a species that would otherwise be dropped still counts.

## The photo set

1,531 photos from 821 photographers, each independently identified rather than taken on the uploader's word alone. All are licensed for reuse: 1,229 CC-BY and 302 CC0.

41% show a cultivated, potted specimen rather than a wild one — the closest openly-licensed approximation of an actual houseplant photo available at this scale.

Every photo's species label is confirmed by at least one identifier independent of the person who took it, never the uploader's own claim alone; 31% carry two or more.

## Held out, verified at measurement time

None of these photos exist in the library the search ranks against. Every one is verified against the library at measurement time — against the exact snapshot these figures are computed from — rather than assumed to still be held out from when it was gathered.

## Accuracy tracks library depth

Split the measured species by how many reference images each one has in the library — the pattern is real, but weak.

| Library images | Species | Top-1 |
| --- | --- | --- |
| 4–6 | 2 | 100.0% |
| 7–9 | 102 | 69.7% |
| 10+ | 28 | 80.4% |

Species with 7–9 library images score 69.7%, and species with 10+ score 80.4%. The 4–6 bucket's 100% looks like the best of the three, but it's only 2 species on 24 photos — noise, not a reversal of the trend (Pearson r = 0.18 across 132 species). The depth effect shows up mainly as the 10+ bucket's 11pp lead over the 7–9 bucket.

## What the errors look like

| Confusion pair | Photos |
| --- | --- |
| *Adiantum capillus-veneris* → **Answered:** *Adiantum raddianum* | 10 (2.4% of errors) |
| *Philodendron erubescens* → **Answered:** *Philodendron hederaceum* | 9 (2.1% of errors) |
| *Coleus scutellarioides* → **Answered:** *Begonia rex-cultorum* | 6 (1.4% of errors) |
| *Adenium obesum* → **Answered:** *Impatiens hawkeri* | 5 (1.2% of errors) |
| *Cereus repandus* → **Answered:** *Pachycereus pringlei* | 5 (1.2% of errors) |
| *Dracaena trifasciata* → **Answered:** *Gasteria bicolor* | 5 (1.2% of errors) |
| *Mammillaria elongata* → **Answered:** *Mammillaria matudae* | 5 (1.2% of errors) |
| All other pairs | 376 · 89.3% |

1,110 of 1,531 photos were identified correctly at top-1; 421 (27.5%) were not. The largest single confusion is congeneric — *Adiantum capillus-veneris* mistaken for *Adiantum raddianum*, 10 times — but same-genus mix-ups are only 23% of all errors. The seven pairs shown above are the heaviest, together just 10.7% of errors (45 of 421); the remaining 305 of 312 distinct pairs spread the rest thinly rather than concentrated in a few pairs. 25 of the 132 measured species score 100% top-1; none score 0%.

## What this number does not cover

6 limits, each with the direction it pushes the figure. They are part of the claim: a bounded number you can argue with is worth more than a round one you cannot.

### 01 20 species were ranked against, never asked about

132 of the 152 species (87%) had enough independently identified held-out photos to score. The other 20 stay in the candidate list but contribute no questions.

**Effect:** Unmeasured, not excluded — they can still be a wrong answer, and are counted as one when they are. 10 are non-binomial or cultivar entries no public reference taxon resolves, 2 collide under a single taxonomic id, 5 fell below the minimum-photo floor, and 3 simply drew none. Genus-level and cultivar entries are exactly where a fine-grained model is weakest, so their absence flatters this figure.

### 02 This is a 152-way task, not the whole world

Every held-out photo is ranked only against the 152 species already in the library. A photo of a plant we don't carry at all is never in this test.

**Effect:** In production that photo still gets a top-1 guess from among these 152 species — silently wrong every time. This figure says nothing about how often that happens.

### 03 Licensed photographs, not your camera roll

These photos come from a nature-observation community, not from Verdalia's own users; 41% show a cultivated, potted specimen rather than a wild one.

**Effect:** **Treat this figure as an upper bound** on real-user accuracy, not a guarantee of it.

### 04 One photo, one answer, no second look

Each held-out photo is scored on its own — no averaging across multiple photos of the same plant, no retake, no second angle.

**Effect:** A real search often gets a second try — a retake, better light. This figure credits only the first frame, so it may understate what a persistent user experiences.

### 05 The figure is pinned to a snapshot, and will move

72.5% describes the library as it stood on Aug 31, 2026: 152 species, 1,240 reference images. Because accuracy tracks depth (69.7% against 80.4%), adding photos to species we already carry raises it, and adding thin new species lowers it before it recovers.

**Effect:** This page is dated on purpose. Re-running it against a newer snapshot is the intent, not an automatic guarantee — the harness refuses to mix a new model with an old index.

### 06 Single model, single run

The headline carries a confidence interval.

**Effect:** The per-bucket and per-species rates do not — they're small-n and noisy, so treat them as directional, not precise.

## Reproducing this

Everything behind the numbers is in the open: the photo set is published in full — every observation, its licence, its photographer and its independent identification — so the sample can be audited and re-fetched by anyone. Reproducing the ranking itself additionally needs the reference index of library embeddings, which is not published.

- [Method — how the run is built and scored](https://verdalia.app/accuracy/method.md)
- [Photo manifest — 1,531 rows, one per observation](https://verdalia.app/accuracy/heldout-manifest.csv)

Run Aug 31, 2026 · library snapshot 152 species / 1,240 images.

---

- [Русский](https://verdalia.app/ru/accuracy.md)
- [Species library](https://verdalia.app/en/species.md)