AbsoluteJS

@absolutejs/voice-benchmarks

@absolutejs/voice-benchmarksv0.0.0betaDev Tools

Performance & accuracy benchmarks for @absolutejs/voice (STT, TTS, duplex, telephony, sessions) vs. Vapi.

#Installation

BASH
# Private workspace project: ~/abs/benchmarks/voice

#Capabilities

Overview

Performance & accuracy benchmarks for @absolutejs/voice — STT, TTS, duplex, telephony, and session soak tests, plus head-to-head comparisons against Vapi.

These consume the published @absolutejs/voice package and provider adapters, so they double as a real-world consumer of the public API.

For intake transcription, run bun run bench:intake:accuracy. It compares Deepgram Flux with OpenAI on the domain-heavy fixture set and reports both WER and expected-term recall, so a provider cannot pass by getting filler words right while missing the names and phrases that populate durable profile fields.

Running

Benchmarks are exposed as bench: scripts. Examples:

Run bun run to list every benchmark target.

Results are written to benchmark-results/ (gitignored). Some accent/code-switch suites read the multilingual corpus from a sibling voice-fixtures-multilingual checkout (../../voice-fixtures-multilingual).

Vapi comparison

benchmarks/vapi-baseline.example.json is a template — copy it to benchmarks/your-vapi-metrics.json (gitignored) with your own Vapi numbers to compare against.

Outcomes

What you can build

Build on the supported package contract

Use @absolutejs/voice-benchmarks through its supported public entry points.

Hardening checklist

Production guidance

Make every external boundary explicitPin the deployed @absolutejs/voice-benchmarks version, replace example or memory-backed dependencies with durable implementations, bound external calls, protect credentials, and emit enough evidence to retry or recover safely.

Follow in order

Troubleshooting path

1
Trace from the first failed boundary
Reproduce the smallest canonical @absolutejs/voice-benchmarks example, confirm the supported entry point and version in the API explorer, then inspect the first boundary that did not produce its documented result.

#Setup

Partial snippet

Working example for Setup.

BASH
bun install
cp .env.example .env   # add provider API keys for live benchmarks

#Running

Partial snippet

Benchmarks are exposed as bench: scripts. Examples:

BASH
bun run bench:tts                 # TTS latency across providers
bun run bench:stt all telephony   # STT accuracy on telephony fixtures
bun run bench:duplex              # full-duplex turn-taking
bun run bench:telephony:run       # telephony suite -> benchmark-results/
Private workspace project
This project is maintained inside the workspace and is not published as a standalone npm package.