Sound-to-Action Voice Commands

Sound-to-Action Voice Command System

A local system that turns spoken audio into actions. It is evaluated not only on direct commands but also on paraphrases, compound commands, negation and speech that should trigger no action at all, which is where simple command recognizers usually fail.

The model combines temporal convolution with a GRU. A voice identity step lets the system enrol a speaker and verify them. A multilingual benchmark workflow covers Russian and English.

Category :Audio Machine Learning
Stack :Python, temporal convolution, GRU
Mode :Local audio-to-action
Evaluation :Command, paraphrase, compound, negation, NO_ACTION
Identity :Voice enrolment and verification
Languages :Russian / English benchmark
Status :Working prototype

Voice command ring with a microphone and command results

Core Features

Most of the work is in defining what counts as a correct response, including the cases where the right answer is to do nothing.

A temporal convolution and GRU model over audio features.
Evaluation of commands, paraphrases and compound commands.
Negation handling: a sentence that says not to do something must not trigger it.
A NO_ACTION class for speech that needs no action.
Voice identity enrolment and verification.
A multilingual Russian-English benchmark workflow.

“A voice system must know when not to act.”

Design Note - NO_ACTION

Architecture

Audio is converted to frames and processed by a temporal model that outputs an action class or NO_ACTION. A separate voice identity step enrols a speaker and verifies them before an action is allowed. The system runs locally. The diagrams are conceptual illustrations of the pipeline.