Most teams decide their site search is broken and go shopping for a new engine. That is usually the wrong first move: the failures costing you orders are configuration, content and interface failures that follow you onto the new platform. Baymard's 2026 search benchmark, built from 10,000+ usability ratings across 170+ sites and apps, found that 56% of sites fail to adequately support users' search needs, with non-product queries failing on 66% of sites and abbreviation queries on 54% (Baymard, updated April 2026). Those are not engine-capability numbers. This guide gives you a 12-check, 48-point rubric you can run in an afternoon, a score band that says whether replacing the engine is justified, and the traps that hit Turkish and other non-English catalogues.
Why the engine is rarely the thing to fix first
Search engines are good at matching tokens to documents. Almost every failure we find sits outside that job: no catalogue attribute for the thing people type, an unpopulated synonym list, a dead-end zero-results page, a query dropped on filtering, a help centre that was never indexed. A new engine inherits all of it.
There is also a measurement problem. The most-quoted number in this field — a 43% conversion lift from site search optimisation — traces to one archived case study cited in a 2019 CXL article, bundles several changes together, and reports no sample size, baseline or control (Cobinyiu). Treat it as folklore. Score your own search, then measure your own lift.
The scale: score every check 0 to 4
Use one scale for all twelve checks, so totals are comparable between audits and sites.
- 0 — Fails. No results, or results that make the task impossible.
- 1 — Returns something, but the user cannot finish the task.
- 2 — Works for the obvious phrasing, fails on variants.
- 3 — Works across the variants you tested.
- 4 — Works across variants and you can show it in aggregate in analytics.
The gap between 3 and 4 is deliberate: a check you cannot measure quietly regresses after the next release.
The 12 checks
Build a query set first: 10 queries per check, from your search logs where possible, plus hand-written variants. Run each on mobile and desktop. Score each check on its worst query, not its average — users do not average their experience.
| # | Check | What you type | Scores 4 when |
|---|---|---|---|
| 1 | Exact product and SKU | Full product name, model code, SKU | First result every time, including with a trailing space or mixed case |
| 2 | Product-type queries | "chairs", "running shoes" | A filtered, ranked list, not a dump of several hundred items |
| 3 | Feature and spec | "waterproof", "cotton", "2 metre" | The attribute exists in the catalogue and is matched, not full-text guessed |
| 4 | Symptom and use case | "shoes for flat feet", "gift for a 5 year old" | A curated mapping to a real result set, or an honest guided route |
| 5 | Abbreviation, symbol, unit | "tv", "60 inch", "60"", "xl" | All forms return the same result set |
| 6 | Non-product queries | "returns", "shipping cost", "invoice", "contact" | Help, policy and legal pages are indexed and rank above products |
| 7 | Character and case normalisation | Queries with and without diacritics, all-caps, typos | Locale-correct casing and folding; one typo still matches |
| 8 | Synonyms and local names | Vernacular and competitor-brand terms for your products | A synonym list reviewed monthly from logs |
| 9 | Zero-results recovery | A deliberate nonsense query | Spelling suggestion, relaxed query, popular categories, contact route |
| 10 | Query persistence | Search, then filter, sort, paginate, go back | The query survives all of them, including the back button |
| 11 | Search box discoverability | Every template, including checkout and 404 | Visible (not icon-only), wide enough for a long query, query retained |
| 12 | Autosuggest accessibility and instrumentation | Keyboard, then screen reader, then analytics | Conformant combobox, 24px targets, search event firing |
A worked example: check 5
Baymard records a snapshot at Staples where "television", "tv", "60 television" and "60 inch television" returned 78, 174, 308 and 24 items (Baymard). Four phrasings of one intent, four result sets, two an order of magnitude apart. No engine upgrade fixes that; a synonym and unit-normalisation rule does. Score it 1: results appear, but nobody comparing phrasings can trust them.
Check 7: the casing bug that only appears in some locales
Most audits skip this check and most non-English catalogues fail it. In Turkish and Azerbaijani, i and I are not a case pair. Unicode encodes it as a locale-conditional mapping: in tr and az, I (U+0049) lowercases to dotless ı (U+0131) unless a combining dot follows, and i uppercases to dotted İ (U+0130) (Unicode SpecialCasing.txt).
If any layer of your stack lowercases with invariant or English rules, "ISITICI" becomes "isitici", which never matches the indexed token "ısıtıcı". The user sees zero results for a product you stock, and your logs record it as demand for something you do not sell. Lucene ships a dedicated TurkishLowerCaseFilter because, as its documentation puts it, "Turkish and Azeri have unique casing behavior for some characters" (Apache Lucene). The generic lowercase filter is a silent, catalogue-wide miss.
Verify three things, in order: the index analyser uses the locale-correct lowercase filter, the query analyser uses the same one, and no application code lowercases the query again before it reaches the engine. A mismatch at any one breaks matching.
The lesson generalises beyond Turkey. Lowercase is not a one-to-one map in German (ß), Greek (final sigma) or Lithuanian either, and folding diacritics on the index but not the query produces the same invisible miss. If your market writes anything other than unaccented English, weight this check heavily.
Check 12: autosuggest is a component with a specification
An autosuggest dropdown is an ARIA combobox with a listbox popup. The APG pattern specifies role="combobox" with aria-autocomplete="list", aria-controls pointing at the popup and aria-expanded reflecting its visibility; the popup is a listbox of options; DOM focus stays on the input while aria-activedescendant tracks the highlighted suggestion. Arrow keys move, Enter accepts, Escape closes and returns focus (W3C ARIA Authoring Practices). Most custom widgets we audit implement the visual behaviour and none of the semantics.
Add pointer size: WCAG 2.2 SC 2.5.8 (Level AA) requires targets of at least 24 by 24 CSS pixels, with a spacing exception based on a 24px-diameter circle centred on each target (W3C). Densely stacked mobile suggestion rows often fail it.
Then the instrumentation half. GA4's recommended event is search, with search_term as a required string parameter (Google). Without it you cannot compute a zero-results rate, so checks 1 to 9 can never score above 3.
Reading your score
| Total /48 | Verdict | Next action |
|---|---|---|
| 36–48 | The engine is not your constraint | Spend on merchandising rules, ranking, measurement |
| 24–35 | Configuration and catalogue gaps | Fix checks 3, 5, 6 and 8 before any procurement talk |
| 12–23 | Possible real engine limitation | Fix 9, 10 and 11 first: cheap, and interface rather than engine |
| Below 12 | Search is a dead end | Triage 9 and 11 this week; a replacement will not land safely |
One rule we hold to: never open a vendor evaluation while a check scores 0 or 1 on a cause you control. You will never know whether the new engine helped.
What this rubric does not cover, and where it breaks down
- Ranking quality. The rubric asks whether the right results appear, not whether they are well ordered. Ranking needs relevance judgements or an online test, not a checklist.
- Semantic retrieval. Scoring check 4 fairly on a semantic engine needs a larger, intent-labelled query set than ten queries.
- Multi-language storefronts. Run checks 7 and 8 per language; a combined score hides a broken locale.
- It is a snapshot. Synonym and attribute coverage decays with every catalogue change. Rerun quarterly, or wire checks 1, 5 and 9 into automated tests.
[INTERNAL DATA NEEDED: Switas zero-results-rate range across audited e-commerce and travel clients, to replace the folklore benchmark.]
[INTERNAL DATA NEEDED: before/after rubric scores and measured conversion delta from a completed search audit, to anchor the bands in outcomes.]
Frequently asked questions
How long does this audit take?
Half a day for a single-language store once the query set exists. Building that set from search logs is the slow part: budget another half day, more if search events are not instrumented yet.
Should I score on mobile or desktop?
Both, scored separately. Baymard's 2026 benchmark rates 58% of mobile sites mediocre or worse against 46% of desktop sites, so a blended score flatters mobile.
What zero-results rate should I aim for?
We do not publish a target. It depends on catalogue breadth and traffic mix, and the public figures for this metric are mostly vendor marketing with no stated methodology. Measure your own and reduce it against your own baseline.
Does Google still use my site search for the sitelinks search box?
No. Google removed the feature starting 21 November 2024, and said unsupported structured data of that kind "won't cause issues in Search, and won't trigger errors in Search Console reports" — so there is no urgency to strip the markup. Internal search is an on-site conversion asset now, not an SEO feature.
Why does non-product search matter for a commerce site?
Because people use the product search box to find policies. Baymard reports 34% of participants in its accounts and self-service testing searched for non-product content, and 66% of sites have issues with those queries. In the EU and Turkey, the return, delivery and complaint information they are hunting for is also information you are obliged to publish.
Can an AI or semantic engine skip checks 7 and 8?
Not safely. Embedding models are trained unevenly across languages, and a locale-incorrect lowercase step upstream corrupts the input before retrieval happens. Normalisation is a precondition for semantic search, not an alternative.
Run it, or have us run it
Score your own search against these twelve checks this week; the rubric is deliberately simple enough to run without us. If you would rather have it done with your query logs, catalogue attributes and locale handling examined together, Switas runs UX audits for e-commerce, travel, health and public-sector teams in Turkey and the region, and hands back a scored, prioritised defect list rather than a slide deck.
Sources
- Baymard Institute — Ecommerce Search UX Best Practices (2026 benchmark, updated April 2026)
- Unicode Character Database — SpecialCasing.txt (Turkish and Azeri conditional casing)
- Apache Lucene — TurkishLowerCaseFilter
- W3C — ARIA Authoring Practices, Combobox pattern
- W3C — Understanding SC 2.5.8 Target Size (Minimum)
- Google — GA4 recommended events (
search/search_term) - Google Search Central — Farewell, Sitelinks Search Box (October 2024)
- Search Engine Land — sitelinks search box dropped from 21 November 2024
- Cobinyiu — why the 43% site-search claim is not a benchmark







