Small decisions. Interesting possibilities.Submit contentSubmit

How it uses Jev

One state carries the query and up to 30 candidate documents keyed D00, D01 and so on. Each document gets its own noul question in the same request, and the calibrated probability works as both the sort key and an absolute pruning threshold.

What Jev decides

Source-defined relevance question (illustrative)Example answers · not a recorded Jev response · Source ↗
Question 1 · D00
YOUR APP
INSTRUCTION

Does this document contain information that answers or directly addresses the query?

STATE

The state is a query plus documents keyed D00, D01 and so on from a BM25 shortlist. The repository defines one domain-neutral noul question per document, with true and false criteria; probabilities above the threshold are kept.

JEV · NOUL
YesNo

App workflow

  1. Retrieve a shortlist

    A lexical retriever such as BM25 returns the top candidates for the query.

  2. Build the state

    The query and the shortlisted documents are packed into one state keyed D00, D01 and so on.

  3. Ask one question per document

    Every document receives the same relevance question, answered in parallel inside one request.

  4. Sort and prune

    The probabilities reorder the shortlist and a threshold removes weak candidates before they reach the next stage.

Why it is interesting

A reranker usually needs a trained scoring model; here one prompt and a schema turn a decision model into one, with a threshold that has meaning rather than a relative score. The repository reports nDCG@10 across three BEIR collections against hosted rerankers and LLM judges.