# Method — search-by-photo accuracy This is the method behind the figures published at [https://verdalia.app/en/accuracy](https://verdalia.app/en/accuracy). The photo manifest referenced throughout is published alongside this document at [https://verdalia.app/accuracy/heldout-manifest.csv](https://verdalia.app/accuracy/heldout-manifest.csv). ## Building the held-out photo set 1,531 photos are drawn from a public nature-observation community. Most of them — all but 75 — are sampled in random order, not the order the reference library itself was curated in, which would have biased the sample toward the best-composed, best-lit photos rather than the ones people actually take. The other 75 (4.9%) were carried over from an earlier fetch that ran in the library's own curation order; disclosed rather than dropped, because they score 66.7% against the random-sampled subset's 72.8% — keeping them in the published figure lowers it, rather than flattering it. Every photo is a distinct observation from a distinct upload; licences are restricted to those safe to publish (no NonCommercial-only terms). A photo is held out if it is not already present in the reference library used to answer a real search. The guarantee that carries for every held-out photo is checked at measurement time: each photo's own community-observation id is re-checked against the library's exclusion list, taken from the exact snapshot the published figures are computed from, and the measurement run refuses to complete on any match or on any photo whose id cannot be resolved. This is what turns "held out" from an assumption into something verified on every run, for every photo, rather than assumed to still hold from whenever a photo was gathered. Photos gathered after the exclusion list existed are additionally skipped before being fetched at all: an entire observation is excluded if any of its photos is already in the reference library — at the observation level, not the photo level, because other photos of the same observation are the same specimen in the same session. The 75 legacy-order photos above predate that collection-time step — which is why their row in the published manifest carries no exclusion-snapshot timestamp — and were verified held-out at measurement time along with everything else. ## The independent-identification bar Every photo's species label is confirmed by at least one identification independent of the person who took the photo — never the uploader's own claim alone; 31% carry two or more. 41% of the held-out photos are of a cultivated, potted specimen rather than a wild one — the closest openly-licensed approximation available at this scale of an actual houseplant photo, since a community identification bar that required research-grade, wild-specimen photos only would exclude cultivated plants (and therefore most houseplants) by definition. ## The ranking path Held-out photos are embedded and ranked by the exact same code path that answers a real search — the same image-matching model, the same ranking function, measured against a reference index pinned to one library snapshot. There is no evaluation-only model and no re-tuned threshold. The measurement run refuses to produce a number at all if the model it is given does not match the model the reference index was built with, since embeddings from different models are not comparable. Each held-out photo is ranked against the full 152-species library — including species that contributed none of the held-out photos themselves, which still compete as candidates exactly as they do for a real search. ## The three weightings The headline top-1 figure is a single number, but it is checked three ways so that no one weighting can flatter it on its own. All three are published, and they agree within 0.4 percentage points: - **Micro** (72.5%) — every held-out photo counts once. - **Macro** (72.9%) — every species counts once, so a commonly-photographed species cannot carry the number on its own. - **No minimum-photo floor** (72.5%) — recomputed with no minimum number of held-out photos required for a species to count, so species that would otherwise be dropped for having too little data still count. ## Taxonomic synonyms in the manifest The published manifest's `taxon_id` and `resolved_taxon_name` columns exist so a reader can check a mapping rather than take this page's word for it. Eight species (94 photos) are filed under a name the naturalist community has since reclassified — the same plant under a different name, not a substitution: - *Schefflera arboricola* → *Heptapleurum arboricola* - *Echinocactus grusonii* → *Kroenleinia grusonii* - *Dracaena trifasciata* → *Sansevieria trifasciata* - *Dracaena angolensis* → *Sansevieria cylindrica* - *Dypsis lutescens* → *Chrysalidocarpus lutescens* - *Ferocactus hamatacanthus* → *Hamatocactus hamatacanthus* - *Hatiora gaertneri* → *Rhipsalidopsis gaertneri* - *Sedum adolphii* → *Sedum adolphi* ## What is not covered here This document describes the method only. The published figures, their confidence interval, the per-depth-bucket breakdown and the confusion pairs are on the [method page](https://verdalia.app/en/accuracy) itself, stated with what each one was measured over. Publishing the photo manifest makes the **photo set** fully auditable and re-fetchable by anyone. Reproducing the **ranking** itself additionally needs the reference index — the embeddings for every image in the species library — which is not published.