Home on Zenia Hew | zenshin

LLM-as-Judge For Contextual Evaluation

Wed, 30 Sep 2026 13:30:19 +0800

In my previous post, I documented moving from fuzzy keyword search (Fuse.js) to semantic vector search (all-MiniLM-L6-v2 + LanceDB). The upgrade felt huge: vector search looks past literal character matches to find conceptual intent.

However, as I ran deeper tests with emotional user inputs, I ran into a boundary that vector search cannot cross on its own: vector search evaluates distance, not appropriateness.

The Boundary: Vector Search is a Mirror

At their core, transformer-based Large Language Models aren’t just text generators. Their fundamental power lies in context processing and pattern recognition: analyzing relationships, organizing unstructured information, and evaluating constraints across high-dimensional token representations. When developers build with LLMs, the default pattern is almost always text generation. But when building a retrieval-backed app (like an anime quote finder), generation introduces severe tradeoffs:

Building A Retrieval System: Keywords vs. Natural Language

Mon, 28 Sep 2026 16:49:29 +0800

Imagine a foreign tourist who speaks limited English coming up, making a few hand gestures, and saying: “Train. Airport.”

You understand both words, but do you know what he wants to convey? He could be asking where to catch the train to the airport, when the next train leaves, or simply whether a train to the airport exists.

This short interaction highlights the fundamental challenge of building a text search engine: keywords carry specific terms, but they omit intent and context.

About

Sun, 22 Mar 2026 23:03:15 +0800

Coming soon!