Arm examines streaming speech recognition on a phone
An ExecuTorch case study separates audio collection delay from processing time.
Arm engineers’ October 26 case study runs NVIDIA’s Nemotron 3.5 speech recognition model through ExecuTorch on a Pixel 11. Streaming transcription turns arriving speech into text incrementally. Longer audio chunks postpone the first text and subsequent updates; preprocessing and inference add further delay. The study therefore evaluates chunk size and numerical precision together. Enabling optimized processor instructions is insufficient evidence of their use: native profiling identifies which calculation routines actually ran.
NVIDIA’s model documentation explains the input choices behind such evaluation. Audio must be mono, and developers can specify the language or select automatic detection. A chunk contains the current audio frame plus the configured future context. Output includes punctuation and capitalization. NVIDIA separates languages into different readiness groups, so support does not imply uniform transcription quality. It also calls for testing with data specific to the intended application before deployment.
For checking execution, the ExecuTorch tools documentation distinguishes output validation from timing. BundledProgram packages representative inputs and expected outputs with a model, allowing runtime results to be compared against those expectations. ETDump records performance and debugging data during execution; it is sufficient for basic timing inspection. ETRecord, produced during export, adds graph metadata needed to connect measurements to original model code. The Inspector interfaces combine these records. The tools also expose intermediate numerical discrepancies between export and runtime, helping locate where outputs diverge.
