NewCelesto Jev PR Review Lab
Runnable PR-review lab that compares Jev and an LLM on candidate code-review findings.
Added to Jevfast
NewHow it uses Jev
Celesto's PR-review example sends a structured packet of pull-request metadata, candidate findings, test results, review recipe, and evidence to Jev. For each finding, it asks separate typed Choice questions about whether the patch introduced it, whether evidence supports it, and whether it is actionable, with `yes`, `no`, and `unknown` options.
What Jev decides
Assess candidate PR findings against the review evidence packetExample answers · not a recorded Jev response · Source ↗
Question 1 · {finding id} introduced
YOUR APP
INSTRUCTION
Judge whether the proposed patch introduced this finding, using the PR packet and cited evidence.
STATE
The state packet has pr {url, base, head}, findings [{id, title, location, claim, evidence_ids}], test_results, recipe, and evidence. It creates three questions per finding, keyed by finding ID plus introduced, supported, or actionable.
JEV · CHOICE
- yes
- no
- unknown
Question 2 · {finding id} supported
YOUR APP
INSTRUCTION
Judge whether the supplied evidence supports the finding claim.
STATE
The state packet has pr {url, base, head}, findings [{id, title, location, claim, evidence_ids}], test_results, recipe, and evidence. It creates three questions per finding, keyed by finding ID plus introduced, supported, or actionable.
JEV · CHOICE
- yes
- no
- unknown
Question 3 · {finding id} actionable
YOUR APP
INSTRUCTION
Judge whether the finding warrants an actionable review comment.
STATE
The state packet has pr {url, base, head}, findings [{id, title, location, claim, evidence_ids}], test_results, recipe, and evidence. It creates three questions per finding, keyed by finding ID plus introduced, supported, or actionable.
JEV · CHOICE
- yes
- no
- unknown
App workflow
- Analyze a public pull request
The example gathers PR metadata, runs its review investigation and checks, then packages findings with their evidence and the review recipe.
- Evaluate findings with Jev
The Jev evaluator submits the shared packet and dynamically generated questions to TypeSafe System One using the `jev-latest` model.
- Present the review result
The example combines the judgment results with the review output and saves a result artifact for inspection.
Why it is interesting
It applies Jev as an evidence-checking layer beside a code-review pipeline. The explicit `unknown` option preserves uncertainty when a finding is not established, and the example can compare typed judgments against a review run without claiming that the judgments prove correctness.