Most site search projects start in the wrong place: a vendor demo. The demo always works, because the demo is run on the vendor's index with the vendor's queries. Meanwhile Baymard Institute's 2026 search benchmark finds that 56% of e-commerce sites have mediocre or worse search UX, and that the failures cluster in specific, nameable query types rather than in "relevance" as a whole (Baymard, updated April 2026). A new engine fixes some of those and none of the others.
This is a scoring rubric for the search you already have. Twenty checks, two points each, forty points total. You run it against your own catalog in an afternoon, you get a number, and the number tells you whether your problem is configuration, content, or the engine itself. It is written for e-commerce UX leads, growth managers and product owners who have been asked to justify a search budget.
What the rubric measures, and what it deliberately ignores
The rubric scores four things: whether search understands the kinds of queries people actually type, whether it survives messy input, whether it recovers when it finds nothing, and whether you can measure any of it. It does not score result ranking quality in the abstract. Ranking is the last thing to tune and the first thing everyone argues about, and you cannot tune it while queries are still failing at the tokenizer.
Scoring is 0, 1 or 2 per check. Two means it passes for every query on your test list. One means it passes for head terms but fails on the long tail. Zero means it fails on head terms. Build the test list first: take your top 50 queries from analytics, then add five queries you construct for each of the eight Baymard query types so the long tail is represented. Roughly 90 queries. Run them yourself, on mobile, logged out.
The 20 checks
| # | Check | Scores 2 when |
|---|---|---|
| Group A — Query coverage (Baymard's failure clusters) | ||
| 1 | Exact and model-number queries | The named product is the first result, not the fourth |
| 2 | Product-type queries | Returns a browsable, faceted list rather than a flat relevance dump |
| 3 | Feature queries ("red wool coat") | Attributes narrow the set instead of diluting it with OR-matching |
| 4 | Compatibility queries ("charger for model X") | Returns accessories for the named model, not the model itself |
| 5 | Non-product queries ("return policy") | Returns the content page; 66% of sites fail this one |
| Group B — Input handling | ||
| 6 | Typo tolerance | One- and two-character errors on head terms still match |
| 7 | Case and diacritic folding | Correct for every language the site serves (see below) |
| 8 | Morphology and plurals | Inflected and suffixed forms reach the same products as the stem |
| 9 | Synonyms and local vernacular | Customer words map to catalog words, both directions |
| 10 | Autocomplete on misspellings | Suggestions appear for misspelled prefixes; 69% of sites fail this |
| Group C — Results and recovery | ||
| 11 | Zero-results recovery | At least two live paths out: related categories and alternative queries |
| 12 | Query persistence | The query stays in the field after submission; 37% of sites drop it |
| 13 | Scope is visible | Result count and any auto-applied category scope are stated and removable |
| 14 | Facet parity | Search results offer the same facets as the equivalent category page |
| 15 | Relevance is explainable | Someone on the team can say why result one ranks above result two |
| Group D — Measurement | ||
| 16 | Every search path is tracked | view_search_results fires with search_term, including autocomplete jumps |
| 17 | Zero-results queries are isolable | You can list last month's no-result queries by volume, unprompted |
| 18 | Search-to-purchase is reportable | Searchers and non-searchers can be compared as segments |
| 19 | Query review has an owner | A named person reviews the top 50 and the zero-result tail on a cadence |
| 20 | Config changes are versioned | Synonym and boost changes are logged, reversible and testable |
Group A thresholds come from Baymard's published per-type failure rates: non-product searches fail on 66% of sites, abbreviation and symbol searches on 54%, compatibility on 44%, use-case on 43% (Baymard). Check 11 exists because nearly half of sites give users no effective way out of a dead end (Baymard, February 2025). Checks 10 and 12 come from the same benchmark's autocomplete and query-persistence findings (Baymard search benchmark).
Check 7 in detail: the folding trap that only shows up outside English
Check 7 is worth its own section because it is the one that silently zeroes out a market. Most search stacks lowercase the query and the index before matching. In English that is harmless. In Turkish it is not, and the same class of problem appears in any language with characters outside Basic Latin.
Unicode defines a locale-specific casing rule for Turkish and Azeri: capital I (U+0049) lowercases to dotless ı (U+0131), while capital İ (U+0130) lowercases to ordinary i (Unicode SpecialCasing.txt). A stack that lowercases with the default locale turns "İPHONE" into "i̇phone" with a stray combining dot, or turns a product title's capital I into the wrong letter. The query and the index then disagree about a word they both contain. Nothing errors. You just get zero results for a brand you stock.
The two fixes are independent and you need both. First, apply locale-correct lowercasing at index and query time — Elasticsearch ships a turkish analyzer that chains apostrophe, turkish_lowercase, turkish_stop, turkish_keywords and turkish_stemmer for exactly this reason (Elastic, language analyzers). Second, decide separately what to do about diacritics, because Turkish shoppers routinely type kilif for kılıf and sarj for şarj on keyboards that make the accented form slower. The asciifolding filter converts non-Basic-Latin characters to ASCII equivalents, and its preserve_original option emits both the folded and the original token so exact matches still win (Elastic, asciifolding). Fold for recall, preserve the original for precision.
Outside Turkey the same check has local shapes: German compound nouns that need decompounding before a two-word query can match, French and Spanish accents typed inconsistently, Greek final sigma. Score check 7 as a 2 only if you have tested it in every language your storefront actually serves — not the language your team speaks.
Fixing measurement first: checks 16 to 20
If Group D scores under 6, stop and fix it before touching anything else, because you cannot verify a fix you cannot see. In GA4, enhanced measurement fires view_search_results with a search_term parameter when a results page carries one of five default query parameters — q, s, search, query or keyword — and you can add your own (Google Analytics Help). Two failure modes are common. If your results page uses a path segment rather than a query string, default tracking sees nothing. And if autocomplete navigates straight to a product, that search never fires the event at all, which quietly biases your query log toward searches that failed to find anything on the first try.
Reading your score
| Score | Verdict | Next move |
|---|---|---|
| 32–40 | Tune, do not replace | Spend on synonyms, merchandising rules and zero-result triage |
| 22–31 | Configuration problem | Fix Group B and check 11 first; re-score in six weeks |
| 12–21 | Probably structural | Re-score after fixing Group D, then scope a replacement with these 20 checks as acceptance criteria |
| 0–11 | Search is not a feature yet | Treat as a build, not an optimization |
The rule that matters: never sign a replacement contract on a score you took before Group D worked. A vendor evaluated against unmeasured search is evaluated against a demo.
A note on the number everyone quotes
You will be shown a statistic saying site-search users convert two or three times better than non-searchers, usually to justify the budget. The figure circulates widely through vendor roundups (example) and is rarely traceable to a controlled study. It is also a selection effect: people who search are people who already have purchase intent. Use it as a reason to look at search, never as a projected lift. The defensible business case is built from your own zero-results volume and your own search-to-purchase rate — both of which check 17 and check 18 exist to produce. [INTERNAL DATA NEEDED: Switas client zero-results rates before/after remediation, if any are shareable.]
Where this rubric breaks down
It is built for catalog retail. Marketplaces with many sellers of the same item need a duplicate-listing and seller-quality dimension this rubric has no column for. Content-heavy and public-sector sites need document-type and recency weighting instead of Group A. Sites under roughly 500 SKUs often score well and still lose money on search, because with a small catalog the real fix is better category navigation, not better search. The rubric also says nothing about search speed under load, and nothing about personalization — both deliberate, because both are expensive to test properly and neither is your problem at a score under 30. Finally, a score is a snapshot: a catalog import that changes attribute naming can drop Group A by six points overnight with no code deployed.
FAQ
How long does scoring take? Around half a day for one person on one storefront, once the 90-query test list exists. Building the list is the slow part; budget another two hours to pull top queries and construct the long-tail set.
Do I need engineering to run it? No. Checks 1 to 15 are run in a browser. Checks 16 to 20 need analytics access and, for check 20, a conversation with whoever owns the search configuration.
What if we do not have a search analytics history? Score Group D as zero, fix the tracking, and collect four weeks of data before scoring the rest. A test list built without a real query log will over-represent queries your team would type.
Is a score of 28 good? It is normal, and normal is mediocre — the benchmark says most sites sit there. The useful signal is not the total but which group is dragging it down.
Should we buy typo tolerance or build it? Buy it. Edit-distance matching with sensible thresholds is solved and cheap. Build effort belongs in synonyms and attribute structure, which are specific to your catalog and cannot be bought.
How does this relate to a category page audit? They share facets and filtering. If check 14 fails, the fix usually lives in the category templates rather than in search, so run the two audits in the same sprint.
Does any of this apply to AI shopping agents reading our catalog? Partly. Agents consume structured attributes rather than your search UI, so Groups A and B matter to them through your data quality, while Groups C and D do not apply at all. It is a separate audit.
How often should we re-score? Quarterly, and after any catalog migration, replatform or search configuration change. Keep the same 90-query list so the scores are comparable.
If you want the score without spending the afternoon: have us run this audit on your storefront and hand you the scored rubric, the failing queries and a prioritized fix list.
Sources
- Baymard Institute — Ecommerce Search UX: the 8 query types and per-type failure rates (updated April 2026)
- Baymard Institute — E-Commerce Search Usability: Report & Benchmark
- Baymard Institute — 5 Proven UX Strategies For "No Results" Pages (February 2025)
- Unicode Consortium — SpecialCasing.txt, Turkish and Azeri casing rules
- Elastic — Language analyzers, including the Turkish analyzer chain
- Elastic — ASCII folding token filter and preserve_original
- Google Analytics Help — Enhanced measurement, view_search_results and search_term
- Hello Retail — example of the circulating site-search conversion multiple
