audio-transcription · @mitsuhiko

Use this skill whenever the user asks to transcribe an audio/video file, a Voice Memos export, dictation, lecture, meeting recording, or "bad audio".

Core rules

Preserve temporary inputs immediately. Voice Memo share-sheet paths under ~/Library/Containers/com.apple.VoiceMemos/Data/tmp/.com.apple.uikit.itemprovider... can disappear. Before probing or experimenting, copy the file to stable /private/tmp/audio-transcription-inputs/.
Use cached local models, not cloud APIs. Prefer MLX Whisper via uvx --from mlx-whisper mlx_whisper; Hugging Face models must be cached in ~/.cache/huggingface/hub/.
Force language when known. For Armin's own dictations this is usually English with an Austrian/German accent, even when the filename is German. Do not infer language from filename alone.
For bad audio, run a hallucination-resistant pass. Use --condition-on-previous-text False, --word-timestamps True, and --hallucination-silence-threshold 2.
Deliver a cleaned best-effort transcript. Compare model output with timestamps/JSON, remove obvious Whisper loops, and mark uncertain spans as [unclear] rather than inventing words.

Fast path

Run from this skill directory:

cd /Users/mitsuhiko/Development/agent-stuff/skills/audio-transcription
./transcribe-audio.py "/path/to/audio.m4a" --language en --quality balanced

The script:

stages a stable copy of the input under /private/tmp/audio-transcription-inputs/
ensures the selected model is cached (downloads only if missing)
writes txt, srt, vtt, tsv, and json to /private/tmp/audio-transcriptions/<name>-<timestamp>/
detects obvious hallucination loops and, in balanced mode, reruns with the full model if needed

Useful variants:

# Quick draft, fastest cached model
./transcribe-audio.py audio.m4a --language en --quality fast

# Bad/important audio, slower full model
./transcribe-audio.py audio.m4a --language en --quality best \
  --prompt "Armin Ronacher dictating about AI, data centers, Vienna, Donauinsel, shareholder value."

# Auto language detection when language is genuinely unknown
./transcribe-audio.py audio.m4a --language auto --quality balanced

Cached models

Default model IDs:

Fast/balanced: mlx-community/whisper-large-v3-turbo
Best fallback: mlx-community/whisper-large-v3-mlx

Pre-cache / refresh both models:

cd /Users/mitsuhiko/Development/agent-stuff/skills/audio-transcription
./precache-models.py

Verify cache manually:

find ~/.cache/huggingface/hub -maxdepth 1 -type d -name 'models--mlx-community--whisper-large-v3*' -print

If a model is already cached, mlx_whisper should say Fetching 4 files: 100% almost instantly.

Manual command template

If the helper script is not suitable, use this command directly:

mkdir -p /private/tmp/audio-transcriptions/manual
uvx --from mlx-whisper mlx_whisper "/stable/copy/of/audio.m4a" \
  --model mlx-community/whisper-large-v3-turbo \
  --language en \
  --condition-on-previous-text False \
  --word-timestamps True \
  --hallucination-silence-threshold 2 \
  --output-format all \
  --output-dir /private/tmp/audio-transcriptions/manual \
  --output-name transcript \
  --verbose False

For especially rough audio, replace the model with mlx-community/whisper-large-v3-mlx.

Quality checks

Inspect the generated .txt first, then the .srt/.json around suspicious areas.

Red flags that require rerun or cleanup:

repeated phrases for many lines (eg. in nature loops)
many zero-duration segments
avg_logprob is NaN or compression ratios are very high in JSON
text contradicts obvious context words supplied in the prompt

When finalizing, lightly punctuate and paragraph the transcript, but do not over-edit uncertain content.