PyAV Integration Guide

Use PyAV when Python needs direct access to FFmpeg libraries: containers, streams, packets, frames, codecs, filters, seeking, subtitles, and NumPy conversion. This SKILL is a lean orchestrator; the full API examples are preserved in references/pyav-recipes-and-api.md.

When to Use PyAV

Use PyAV for:

Frame-level video/audio decoding and encoding
Precise seeking and keyframe extraction
NumPy/Pillow/OpenCV frame access without shelling out per frame
Container inspection, packet-level remuxing, subtitle reads
RTSP/network stream handling with library-level control

Use FFmpeg CLI/subprocess instead for simple transcodes, production command parity, or heavy hardware-accelerated pipelines.

Quick Reference

Task	PyAV pattern	Notes
Open media	`av.open(path)`	Prefer context manager
Decode video	`container.decode(video=0)`	Yields `VideoFrame`
To NumPy	`frame.to_ndarray(format='rgb24')`	Convert to BGR for OpenCV
From NumPy	`av.VideoFrame.from_ndarray(arr, format='rgb24')`	Encode generated frames
Seek	`container.seek(offset)`	Usually keyframe-based
Encode	`stream.encode(frame)` then `container.mux(packet)`	Flush encoder at end
Close	`container.close()`	Critical in loops

Minimal Patterns

import av

with av.open("input.mp4") as container:
    stream = container.streams.video[0]
    stream.thread_type = "AUTO"
    for frame in container.decode(stream):
        rgb = frame.to_ndarray(format="rgb24")
        break

import av

with av.open("input.mp4") as container:
    stream = container.streams.video[0]
    container.seek(int(10.0 * av.time_base), backward=True)
    for frame in container.decode(stream):
        if frame.time >= 10.0:
            image = frame.to_ndarray(format="rgb24")
            break

Core Workflow

Install av from wheels unless you explicitly need a custom FFmpeg build.
Check PyAV and FFmpeg library compatibility before relying on new FFmpeg 8.x features.
Open containers with a context manager or try/finally close.
Select streams explicitly; decode only what you need.
Convert frame pixel/sample formats intentionally.
For encoding, set codec, dimensions, pixel format, rate, and options explicitly; flush encoders.
Disable verbose logging in threaded applications and prefer file paths over Python file objects.

Key Gotchas

PyAV frames can reference underlying buffers; call .copy() on NumPy arrays you keep after container close.
thread_type='AUTO' can greatly improve decode speed but may add frame delay.
PyAV hardware acceleration is limited compared with CLI FFmpeg.
Subtitle transcoding support is limited; use FFmpeg CLI for complex subtitle workflows.

Reference Map

references/pyav-recipes-and-api.md - Full preserved reference: installation, custom FFmpeg builds, FFmpeg 8 compatibility, decoding, NumPy conversion, encoding, audio, filters, seeking, remuxing, subtitles, RTSP, errors, memory, threading, performance, hardware caveats, common patterns.

Related Skills

ffmpeg-python-integration-reference - Type-safe FFmpeg parameter mapping
ffmpeg-opencv-integration - OpenCV pipelines and color conversion
ffmpeg-fundamentals-2025 - Core FFmpeg operations and codec choices

ffmpeg-pyav-integration