Small decisions. Interesting possibilities.Submit contentSubmit
A computer assistant that listens preview

A computer assistant that listens

building a computer assistant with Jev local whisper listens to everything I say which then gets classified by Jev to determine what actions to take it uses a small Swift app to provide the full accessibility tree to Jev (i.e. tell its what's on my screen, what can be clicked, etc) still super early, but promising and…

Added to Jevfast

How it uses Jev

Local Whisper transcribes speech, and Jev classifies it to determine what action the assistant should take. A Swift app provides Jev with the screen's accessibility tree, including visible click targets.

App workflow

  1. Listen and transcribe

    Local Whisper listens to speech and supplies its transcription.

  2. Provide screen context

    A Swift app shares the accessibility tree and available click targets.

  3. Choose an action

    Jev classifies the spoken input to determine what action to take.

Why it is interesting

The assistant combines spoken intent with structured information about the current interface to choose actions in real time.