Align

Timestamps that land on the word.

Word-timestamp refinement for Apple's SpeechAnalyzer pipeline.

SDKs, install and examples: https://github.com/Desert-Ant-Labs/desert-ant-core/blob/main/docs/models/align.md

Corrects the word-level timings that Apple's SpeechTranscriber and SpeechAnalyzer return, without replacing them. Align observes the same audio the analyzer already receives, runs a small Core ML cascade on the CPU and Neural Engine, and returns the familiar result surface with tightened audioTimeRange values. The models are tiny (700KB compiled Core ML) and refine a typical result in a few milliseconds on device.

Apple: "world" 2.61-3.04s ➜ Align: "world" 2.57-2.98s

Try it


Platforms	iOS, macOS, tvOS, visionOS
Weights	main

Install

Swift (requirements)

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "3.1.0")

Then add the Align product to your target.

Files

File	Format	Size	Contents
`align_coarse.mlmodelc`	Compiled Core ML (FP16)	300KB	Coarse stage: searches a 241-frame (2.4 s) context, fixed batch-16
`align_fine.mlmodelc`	Compiled Core ML (FP16)	300KB	Fine stage: searches an 81-frame (0.8 s) crop centered on the coarse prediction
`mel_filters.bin`	Float32 filter bank	40KB	Log-mel filter bank the runtime frontend needs
`calibrator.bin`	Gradient-boosted trees	70KB	Correction calibrator over coarse/fine uncertainty features
`refiner_config.json`	JSON	tiny	Frontend, lexical, and language config the runtime needs
`coarse.pt`	PyTorch checkpoint	500KB	Coarse-stage weights (for retraining / other runtimes)
`fine.pt`	PyTorch checkpoint	500KB	Fine-stage weights (for retraining / other runtimes)

The compiled .mlmodelc stages, mel_filters.bin, calibrator.bin, and refiner_config.json are exactly what the Swift SDK bundles. The .pt checkpoints are the training-run weights.

Architecture

A two-stage coarse-to-fine cascade over a log-mel spectrogram, refining one boundary at a time:

Frontend: an Accelerate/vDSP log-mel spectrogram of the same audio Apple transcribes.
Coarse stage: a compact convolutional model searches a 2.4 s context around Apple's proposed boundary and predicts a distribution over frames.
Fine stage: a second model re-searches a 0.8 s crop recentered on the coarse prediction for a tighter estimate.
Lexical conditioning: UTF-8 byte features of the neighboring words plus a language id let a single model cover all nine languages.
Calibrator: a small gradient-boosted-tree policy maps coarse/fine uncertainty features to a final correction, fit only on the validation split to reduce large regressions.
Structural fallback: boundaries whose correction would be invalid, hit the search-window edge, or lack streaming context keep Apple's original timestamp.

Each stage runs fixed batch-16 on CPU + Neural Engine. Total parameters are about 117k per stage.

Inputs and outputs

Input: mono audio plus Apple's recognized words with their proposed start/end times.
Output: the same words with corrected start/end times, or Apple's original time when a correction is not structurally safe.

Accuracy

Evaluated on the exact Swift runtime and these bundled Core ML models over 223 clean and 210 noisy group-held-out recordings across all nine languages, against forced-alignment references.

Condition	Apple raw error	Align error	Reduction	Median	Within 50ms
Clean	113.5ms	44.9ms	60%	28.2ms	75.1%
Noisy	124.4ms	50.1ms	60%	32.0ms	69.4%

Error is mean absolute distance from the reference boundary. Align roughly halves Apple's typical error and removes most of its large mistakes.

Languages

English, Spanish, French, Italian, Portuguese, German, Japanese, Korean, and Chinese. A locale outside this set is passed through unchanged.

Limitations

References are machine forced-alignment estimates, not human annotations, so the figures show a large, consistent reduction of Apple's timing error rather than sample-accurate ground truth.
A learned correction is not guaranteed to improve every boundary; the structural fallback keeps Apple's timestamp when a correction looks unsafe but cannot catch every plausible-looking error.
English, Italian, Japanese, and Korean are the weakest languages under the current reference convention.

Built on

FLEURS (CC BY 4.0): multilingual training audio.
Qwen3-ForcedAligner-0.6B (Apache-2.0): primary word-boundary references for all nine languages.
OWSM-CTC v4 1B (CC BY 4.0): gross alignment-outlier check where validation agreement is stable.
Genuine Apple SpeechAnalyzer proposals collected on macOS 26.

See THIRD_PARTY_NOTICES.md. None of these systems are redistributed here.

License

Desert Ant Labs Source-Available License. Free for most apps, and a commercial license is required at scale. Full terms are at the link. Licensing: licensing@desertant.com.

See THIRD_PARTY_NOTICES.md.

Citation

@software{align_2026,
  title  = {Align: Word-timestamp refinement for Apple's SpeechAnalyzer pipeline},
  author = {Desert Ant Labs},
  year   = {2026},
  url    = {https://huggingface.co/desert-ant-labs/align},
}

Downloads last month: -; Downloads are not tracked for this model. How to track