Research Engineer, Evaluation

About the Role

Join our cutting-edge Voice AI team as a Research Engineer, where you'll bridge the gap between research, product, and engineering to define and measure the success of our industry-leading speech-to-text models.

Responsibilities

• Own end-to-end and integration-level model evaluation across accuracy, latency, and feature-specific metrics.
• Build and maintain competitive benchmarking pipelines against market competitors.
• Design and run systematic experiments to measure the impact of model changes.
• Onboard, curate, and maintain evaluation datasets, including public benchmarks and internal test sets.
• Define evaluation metrics that capture real-world performance and translate qualitative customer feedback into quantifiable evaluation criteria.
• Work with customer-facing teams to understand pain points and convert them into research priorities.
• Reduce friction for researchers by maintaining clean evaluation pipelines and clear documentation.

Requirements

• Strong understanding of ML fundamentals, including model training and evaluation.
• Proficient Python skills for writing evaluation scripts, working with data pipelines, and comfortable with SQL and cloud infrastructure.
• Strong metric intuition, understanding of statistical rigor, and ability to discern relative vs. absolute improvements.
• Familiarity with the voice agent stack (VAD, ASR, turn detection, LLM, TTS) and their interactions.
• A tinkerer mentality, comfortable with shipping and iterating quickly.
• Excellent communication skills to explain technical results and translate customer feedback.
• An ownership mindset, proactively identifying and filling evaluation gaps.
• Ability to overlap at least 3-4 hours with the Eastern US Time Zone.

Nice-to-Haves

• Experience with speech/audio ML or real-time systems.
• Hands-on experience with voice agent orchestrators (e.g., LiveKit, Pipecat, Vapi).
• Familiarity with standard ML evaluation practices and benchmarks.
• Experience working with customer-facing or product teams.
• Background in QA, data science, or applied ML roles.