The search index in crates/core/src/lib.rs (SearchDocument::search, lines ~360-376) is a naive substring matcher that assigns fixed bucket scores (100/75/25) and requires every term to appear via path.contains(term).
Problem
- Fixed bucket scoring (100/75/25) does not reflect actual relevance.
- Every term must appear via
path.contains(term), so partial words and typos never match.
split_whitespace over the raw query (lines ~317-322) produces crude tokens that won't match partial words.
Suggested direction
- Replace fixed buckets with term-frequency / proximity-based scoring.
- Support fuzzy and prefix matching.
- Add proper tokenization (stemming, word-boundary handling) instead of
split_whitespace over the raw query.
References
crates/core/src/lib.rs — SearchDocument::search (lines ~360-376)crates/core/src/lib.rs — query tokenization (lines ~317-322)
The search index in
crates/core/src/lib.rs(SearchDocument::search, lines ~360-376) is a naive substring matcher that assigns fixed bucket scores (100/75/25) and requires every term to appear viapath.contains(term).Problem
path.contains(term), so partial words and typos never match.split_whitespaceover the raw query (lines ~317-322) produces crude tokens that won't match partial words.Suggested direction
split_whitespaceover the raw query.References
crates/core/src/lib.rs—SearchDocument::search(lines ~360-376)crates/core/src/lib.rs— query tokenization (lines ~317-322)