Small decisions. Interesting possibilities.Submit contentSubmit

Starchild prompt routing

Starchild just became the first professional-grade app to integrate Jev from @typesafeai We use Jev to help route to LLMs which gave a 20x decrease in cost and 6x increase in speed for prompt classification Here it is classifying prompts in real-time (avg time was 140ms)

Added to Jevfast

How it uses Jev

Starchild sends a prompt to Jev to classify it and route it to a language model before the model turn runs.

App workflow

  1. Receive a prompt

    A user prompt enters Starchild.

  2. Classify and route

    Jev classifies the prompt to help select a language model.

  3. Run the selected model

    Routing takes place before the model turn begins.

Why it is interesting

Model choice happens just in time, allowing a lightweight classification step to direct each prompt before the more expensive model is invoked.