jevals
Agent evals and guardrails in one request. Built on Jev, Kev and Laya.
Added to Jevfast
How it uses Jev
jevals evaluates agent traces with a collection of typed checks, such as tool choice, use of tool results, grounding, scope, answer relevance, completeness, indirect injection, and PHI. Each evaluation derives state and questions from the sample; their requests are merged so a trace can be evaluated in one backend call.
What Jev decides
Illustrative Jev requestExample answers · not a recorded Jev response · Source ↗
Question 1 · action
YOUR APP
INSTRUCTION
Should this tool call proceed as proposed?
STATE
Proposed tool name and arguments, user goal, and the three most recent messages.
JEV · CHOICE
- approve
- escalate
- block
App workflow
- Provide an agent sample
Pass the trace messages and tool schemas; supported message representations can be normalized for evaluation.
- Build evaluation questions
Each eval extracts its required state and creates typed questions for judgment-dependent checks.
- Evaluate the trace
Pack questions from the requested evals into one backend request, using Jev or a configured compatible backend.
- Reduce and consume results
Combine returned judgments with deterministic checks into scores and pass/fail-style results for monitoring or gating.
Why it is interesting
It combines deterministic preprocessing and reduction code with typed judgments, allowing the same evaluation definition to run offline, as a production monitor, or as an agent gate. Backends can be swapped, with calibration required when model probabilities change.