Score Your Site Search Out of 48 Before You Pay for a New Search Engine

Score Your Site Search Out of 48 Before You Pay for a New Search Engine

Most teams decide their site search is broken and go shopping for a new engine. That is usually the wrong first move: the failures costing you orders are configuration, content and interface failures that follow you onto the new platform. Baymard's 2026 search benchmark, built from 10,000+ usability ratings across 170+ sites and apps, found that 56% of sites fail to adequately support users' search needs, with non-product queries failing on 66% of sites and abbreviation queries on 54% (Baymard, updated April 2026). Those are not engine-capability numbers. This guide gives you a 12-check, 48-point rubric you can run in an afternoon, a score band that says whether replacing the engine is justified, and the traps that hit Turkish and other non-English catalogues.

Why the engine is rarely the thing to fix first

Search engines are good at matching tokens to documents. Almost every failure we find sits outside that job: no catalogue attribute for the thing people type, an unpopulated synonym list, a dead-end zero-results page, a query dropped on filtering, a help centre that was never indexed. A new engine inherits all of it.

There is also a measurement problem. The most-quoted number in this field — a 43% conversion lift from site search optimisation — traces to one archived case study cited in a 2019 CXL article, bundles several changes together, and reports no sample size, baseline or control (Cobinyiu). Treat it as folklore. Score your own search, then measure your own lift.

The scale: score every check 0 to 4

Use one scale for all twelve checks, so totals are comparable between audits and sites.

  • 0 — Fails. No results, or results that make the task impossible.
  • 1 — Returns something, but the user cannot finish the task.
  • 2 — Works for the obvious phrasing, fails on variants.
  • 3 — Works across the variants you tested.
  • 4 — Works across variants and you can show it in aggregate in analytics.

The gap between 3 and 4 is deliberate: a check you cannot measure quietly regresses after the next release.

The 12 checks

Build a query set first: 10 queries per check, from your search logs where possible, plus hand-written variants. Run each on mobile and desktop. Score each check on its worst query, not its average — users do not average their experience.

#CheckWhat you typeScores 4 when
1Exact product and SKUFull product name, model code, SKUFirst result every time, including with a trailing space or mixed case
2Product-type queries"chairs", "running shoes"A filtered, ranked list, not a dump of several hundred items
3Feature and spec"waterproof", "cotton", "2 metre"The attribute exists in the catalogue and is matched, not full-text guessed
4Symptom and use case"shoes for flat feet", "gift for a 5 year old"A curated mapping to a real result set, or an honest guided route
5Abbreviation, symbol, unit"tv", "60 inch", "60"", "xl"All forms return the same result set
6Non-product queries"returns", "shipping cost", "invoice", "contact"Help, policy and legal pages are indexed and rank above products
7Character and case normalisationQueries with and without diacritics, all-caps, typosLocale-correct casing and folding; one typo still matches
8Synonyms and local namesVernacular and competitor-brand terms for your productsA synonym list reviewed monthly from logs
9Zero-results recoveryA deliberate nonsense querySpelling suggestion, relaxed query, popular categories, contact route
10Query persistenceSearch, then filter, sort, paginate, go backThe query survives all of them, including the back button
11Search box discoverabilityEvery template, including checkout and 404Visible (not icon-only), wide enough for a long query, query retained
12Autosuggest accessibility and instrumentationKeyboard, then screen reader, then analyticsConformant combobox, 24px targets, search event firing

A worked example: check 5

Baymard records a snapshot at Staples where "television", "tv", "60 television" and "60 inch television" returned 78, 174, 308 and 24 items (Baymard). Four phrasings of one intent, four result sets, two an order of magnitude apart. No engine upgrade fixes that; a synonym and unit-normalisation rule does. Score it 1: results appear, but nobody comparing phrasings can trust them.

Check 7: the casing bug that only appears in some locales

Most audits skip this check and most non-English catalogues fail it. In Turkish and Azerbaijani, i and I are not a case pair. Unicode encodes it as a locale-conditional mapping: in tr and az, I (U+0049) lowercases to dotless ı (U+0131) unless a combining dot follows, and i uppercases to dotted İ (U+0130) (Unicode SpecialCasing.txt).

If any layer of your stack lowercases with invariant or English rules, "ISITICI" becomes "isitici", which never matches the indexed token "ısıtıcı". The user sees zero results for a product you stock, and your logs record it as demand for something you do not sell. Lucene ships a dedicated TurkishLowerCaseFilter because, as its documentation puts it, "Turkish and Azeri have unique casing behavior for some characters" (Apache Lucene). The generic lowercase filter is a silent, catalogue-wide miss.

Verify three things, in order: the index analyser uses the locale-correct lowercase filter, the query analyser uses the same one, and no application code lowercases the query again before it reaches the engine. A mismatch at any one breaks matching.

The lesson generalises beyond Turkey. Lowercase is not a one-to-one map in German (ß), Greek (final sigma) or Lithuanian either, and folding diacritics on the index but not the query produces the same invisible miss. If your market writes anything other than unaccented English, weight this check heavily.

Check 12: autosuggest is a component with a specification

An autosuggest dropdown is an ARIA combobox with a listbox popup. The APG pattern specifies role="combobox" with aria-autocomplete="list", aria-controls pointing at the popup and aria-expanded reflecting its visibility; the popup is a listbox of options; DOM focus stays on the input while aria-activedescendant tracks the highlighted suggestion. Arrow keys move, Enter accepts, Escape closes and returns focus (W3C ARIA Authoring Practices). Most custom widgets we audit implement the visual behaviour and none of the semantics.

Add pointer size: WCAG 2.2 SC 2.5.8 (Level AA) requires targets of at least 24 by 24 CSS pixels, with a spacing exception based on a 24px-diameter circle centred on each target (W3C). Densely stacked mobile suggestion rows often fail it.

Then the instrumentation half. GA4's recommended event is search, with search_term as a required string parameter (Google). Without it you cannot compute a zero-results rate, so checks 1 to 9 can never score above 3.

Reading your score

Total /48VerdictNext action
36–48The engine is not your constraintSpend on merchandising rules, ranking, measurement
24–35Configuration and catalogue gapsFix checks 3, 5, 6 and 8 before any procurement talk
12–23Possible real engine limitationFix 9, 10 and 11 first: cheap, and interface rather than engine
Below 12Search is a dead endTriage 9 and 11 this week; a replacement will not land safely

One rule we hold to: never open a vendor evaluation while a check scores 0 or 1 on a cause you control. You will never know whether the new engine helped.

What this rubric does not cover, and where it breaks down

  • Ranking quality. The rubric asks whether the right results appear, not whether they are well ordered. Ranking needs relevance judgements or an online test, not a checklist.
  • Semantic retrieval. Scoring check 4 fairly on a semantic engine needs a larger, intent-labelled query set than ten queries.
  • Multi-language storefronts. Run checks 7 and 8 per language; a combined score hides a broken locale.
  • It is a snapshot. Synonym and attribute coverage decays with every catalogue change. Rerun quarterly, or wire checks 1, 5 and 9 into automated tests.

[INTERNAL DATA NEEDED: Switas zero-results-rate range across audited e-commerce and travel clients, to replace the folklore benchmark.]

[INTERNAL DATA NEEDED: before/after rubric scores and measured conversion delta from a completed search audit, to anchor the bands in outcomes.]

Frequently asked questions

How long does this audit take?
Half a day for a single-language store once the query set exists. Building that set from search logs is the slow part: budget another half day, more if search events are not instrumented yet.

Should I score on mobile or desktop?
Both, scored separately. Baymard's 2026 benchmark rates 58% of mobile sites mediocre or worse against 46% of desktop sites, so a blended score flatters mobile.

What zero-results rate should I aim for?
We do not publish a target. It depends on catalogue breadth and traffic mix, and the public figures for this metric are mostly vendor marketing with no stated methodology. Measure your own and reduce it against your own baseline.

Does Google still use my site search for the sitelinks search box?
No. Google removed the feature starting 21 November 2024, and said unsupported structured data of that kind "won't cause issues in Search, and won't trigger errors in Search Console reports" — so there is no urgency to strip the markup. Internal search is an on-site conversion asset now, not an SEO feature.

Why does non-product search matter for a commerce site?
Because people use the product search box to find policies. Baymard reports 34% of participants in its accounts and self-service testing searched for non-product content, and 66% of sites have issues with those queries. In the EU and Turkey, the return, delivery and complaint information they are hunting for is also information you are obliged to publish.

Can an AI or semantic engine skip checks 7 and 8?
Not safely. Embedding models are trained unevenly across languages, and a locale-incorrect lowercase step upstream corrupts the input before retrieval happens. Normalisation is a precondition for semantic search, not an alternative.

Run it, or have us run it

Score your own search against these twelve checks this week; the rubric is deliberately simple enough to run without us. If you would rather have it done with your query logs, catalogue attributes and locale handling examined together, Switas runs UX audits for e-commerce, travel, health and public-sector teams in Turkey and the region, and hands back a scored, prioritised defect list rather than a slide deck.

Sources


Inci Dindar
Written by

Inci Dindar

With a background in Software Development, she led design, UX, and product teams across fintech, media, e-commerce, and travel. For the past three years she has consulted full-time on UX and design — including as an external UX consultant for a global management consultancy helping brands and startups with audits, conversion optimization, and design processes.


Related Articles

Switas As Seen On

Magnify: Scaling Influencer Marketing with Engin Yurtdakul

Check Out Our Microsoft Clarity Case Study

We highlighted Microsoft Clarity as a product built with practical, real-world use cases in mind by real product people who understand the challenges companies like Switas face. Features such as rage clicks and JavaScript error tracking proved invaluable in identifying user frustrations and technical issues, enabling targeted improvements that directly impacted user experience and conversion rates.