Building a search that admits when it found nothing

My personal journaling dashboard has accumulated more than 1,500 entries, and I don’t expect that to slow down. Scrolling a thread used to be enough to find a thought. With more threads, and more entries inside each of them, it stopped being enough, and the whole point of capturing a thought is being able to get back to it later.

The obvious answer is embeddings: a vector representation of each entry in a space where similar concepts cluster together, which is what lets a search match on meaning rather than on the exact words I happened to use. If that concept is new to you, it’s worth reading up on separately, because everything below assumes it.

Meaning finds thoughts, words find names

Embeddings alone were never going to be enough, and that was the first real decision in this project.

A journal is an unusually bad corpus to search. It’s half German and half English, often inside the same entry. Nothing has a title. The same idea written four months apart uses none of the same words, because I wasn’t writing for a future search, I was writing a thought down. Meaning-based search handles all of that well.

What it handles badly is the case where I know exactly what I wrote. A name, a place, a product, one specific word. Those are the queries where I don’t want the system being clever about meaning at all, I want the entry containing that token. A vector search will happily return five thematically adjacent entries and not the one with the word in it.

So SearchEntriesHybrid runs three rankings in a single query and fuses them:

1. Vector similarity is the recall backbone. It carries meaning, and it’s the only arm that can answer a German query with an English entry. It’s deliberately ungated, because it has to keep producing candidates well past wherever the list eventually gets cut.
2. Full text covers exact tokens. It runs on the simple configuration rather than a language-specific one, since committing to German or English stemming would quietly break half the corpus. It ANDs its terms together, so on a long natural-language query it simply goes silent, which is correct: that isn’t the arm carrying that kind of query.
3. Trigram similarity catches typos and near-misses on names, which is most of what my own bad spelling produces.

Fusing them uses Reciprocal Rank Fusion, which has a smoothing constant conventionally set to 60. That value comes from fusing result lists over web-scale indexes, where gentle is what you want. On a corpus this size it’s too gentle: the fused scores for ranks 1 through 10 land within roughly 13% of each other, and a flat list like that leaves nothing for anything downstream to read. Dropping the constant to 10 spreads them out enough to be usable.

-- was: SUM(1.0 / (60 + rnk))
SUM(1.0 / (10 + arms.rnk))::float8 AS fused_score

It isn’t free. A smaller constant weights a single arm’s top hit more heavily against several arms agreeing further down, so the fusion gets slightly less democratic. For this corpus that’s an acceptable trade, and the exemption further down covers the case it would otherwise get wrong.

Running the model at home

Sending intimate thoughts to an API that has to read them in the clear was never on the table, so the embedder is self-hosted. There’s also a duller reason: this is something I intend to run for years, and a fixed cost I already own beats a per-call one I’d have to keep justifying to myself.

Google’s embeddinggemma-300m turned out to be the right fit. It’s small enough to run in its own llama.cpp container on the Raspberry Pi next to everything else, at its native 768 dimensions, with no published ports and no route in from the internet.

One thing cost me an evening. Pooled embeddings need the entire input inside a single physical batch, and llama.cpp has two batch flags, -b for the logical batch and -ub for the physical one, with the physical batch smaller by default. That’s fine for generation, where a prompt can be split across several passes, but wrong for embeddings, where the whole sequence has to be pooled into one vector in one pass. A long entry that spilled across two physical batches didn’t get a bad embedding, it got none at all: input is too large to process, thrown mid-batch. Setting both flags to the model’s context window fixed it. Three lines in the compose file, after a lot of staring at a stack trace that pointed at the wrong layer.

Vectors live in their own narrow table rather than as a column on entries. Keeping the entry row free of anything vector-shaped means swapping the model later is a table I truncate and refill, not a migration against the row that holds the actual text.

There’s no approximate index. pgvector offers HNSW and IVFFlat for exactly this, and I’m deliberately not using either: at this size an exhaustive scan is fast enough that an approximate index would only add a second, weaker source of truth. The CREATE INDEX sits commented out in the migration, waiting for the day the scan is actually the bottleneck rather than the embedding call in front of it. That decision turns out to matter for the next section, in a way I didn’t plan.

The hard part is returning nothing

Ranking was never the difficult part. A vector search always has a nearest neighbour, so ranked results are cheap. The difficult part is deciding when to say nothing here, instead of confidently handing over the least bad match to a question the journal has no answer to.

The first attempt was a fixed similarity floor: below some constant, don’t show the result. No constant works, and the reason is visible in the data. The median similarity across the corpus, which is roughly what an unrelated entry looks like, moves between about 0.06 and 0.22 depending on the query’s language and length. A floor tuned for one shape of query sits above the entire distribution for another. Short queries and long ones don’t produce comparable numbers in the first place, so comparing either to a constant compares them to nothing.

The floor has to be measured against the query’s own noise. Because there’s no approximate index, the scoring pass already touches every embedded entry, so the baseline is free: the median across that pass stands in for a typical unrelated entry, the 90th percentile marks the top of the noise, and the distance between them is the spread a real hit has to clear.

// 0 = typical noise, 1 = top of the noise band
func (b noiseBaseline) standing(cos float64) float64 {
return (cos - b.median) / (b.p90 - b.median)
}

The constants on top of that were found by trial and error rather than derived. Gibberish still returns a nearest entry, because a vector search can’t return nothing, only the least dissimilar thing, but it tends to land around 1.4 standings above baseline. A query the journal genuinely answers clears roughly 2.4. The cut sits at 1.8. That’s a hand-tuned number for one corpus and one model, and swapping either would mean finding it again.

Two exemptions keep the cut from being clever in the wrong direction:

1. A lexical hit is never cut by the noise floor. A shared rare token is its own evidence. An entry can mention Hafermilch once in passing, score badly on overall meaning, and still be precisely the entry I was looking for. The floor exists to catch meaning-only matches, not to overrule a match that’s already certain for a different reason.
2. Below ten embedded entries, the floor doesn’t apply at all. A percentile computed over nine rows is measuring itself rather than a corpus, so below that size the cut does nothing instead of something wrong.

A separate cap keeps the confident list to eight results. Past a handful, what’s happening is browsing rather than answering, and returning fifteen confident-looking results would undersell the two that mattered.

Nothing is thrown away, though. Results below the cut still come back, flagged, and the interface collapses them behind a fold instead of dropping them. A search should never make something it found unreachable, including the results its own ranking doesn’t trust.

The point of all of it is that the confident list is allowed to be empty.

The card explains, it doesn’t rate

The temptation once results are on screen is to show a relevance score. I didn’t want one. A number like 0.71 says nothing about why something matched, and it invites treating retrieval as more precise than it is.

Instead the result card highlights the words that matched literally, and gives meaning-only hits a quiet similar meaning label with no highlighting at all. Highlighting there would be a lie: the vector arm has no idea which words did the work, so the interface shouldn’t imply that it does.

That decision reached backwards into the query. The card needs to know which of the three arms produced each result, so the query returns that alongside the score. The interface determined the query’s return shape rather than the other way around, which is the correct direction and not the one I started in.

What the search is allowed to see

Two smaller rules, both of which exist because this is a journal and not a document store.

Threads can be marked private, and a private thread is invisible to corpus-wide search. It isn’t invisible everywhere, though. A search scoped to a single thread is exempt, because I’m already inside it, and so is a search scoped to one person, because filing an entry under someone is an explicit act and their page is an explicit context. Privacy here means don’t surface this when I’m not looking for it, not pretend it doesn’t exist.

The second is a “more like this” on every entry, which reuses that entry’s stored embedding as the query. No re-embedding, no query prompt, no typing. It runs through the same noise baseline and the same cut, so it’s equally willing to tell me an entry stands alone.

Closing thoughts

What surprised me is how much of the work went into the system’s willingness to decline. Every default in retrieval pushes the other way. A vector search structurally cannot return nothing, ranking always produces an order, and an interface will happily present the top of that order as an answer. Getting to nothing here meant adding a measurement whose only job is to disagree with the ranking.

I can also do this because it’s mine. A single user, one corpus, constants I tuned by hand until the results felt right, and no obligation to look useful to anyone else. A product would have to justify an empty results page. I only have to justify it to myself, and I’d rather have a search that occasionally says nothing than one that always dresses up its closest piece of noise and hands it over as an answer.