Skip to content

Overview

Frame extraction processor family.

Decode one or more still frames from a video at given timestamps.

Exported Classes

DeinterlaceMode

Bases: StrEnum

How frame extraction applies deinterlace (e.g. FFmpeg yadif).

Attributes:

Name Type Description
OFF

Never deinterlace.
FORCE

Always deinterlace (ignore field-order metadata).
AUTO

Deinterlace only when ffprobe reports an interlaced field order.

FFmpegFrameExtractionConfig

Bases: FrameExtractionConfig

FFmpeg-specific stable config (inherits shared FrameExtractionConfig).

Yadif fields apply when deinterlace is force or auto (and the probe selects yadif for auto).

Attributes:

Name Type Description
yadif_mode int

Yadif mode. Defaults to 0.
yadif_parity int

Field parity (-1=auto). Defaults to -1.
yadif_deint int

Deinterlace all (0) or flagged-only (1). Defaults to 0.
filter_override str | None

Full -vf string when set. Defaults to None.

yadif_filter()

Build the -vf filter string for deinterlaced extract.

Returns:

Name Type Description
str str

Filter graph for FFmpeg -vf.

FFmpegFrameExtractionProcessConfig

Bases: FrameExtractionProcessConfig

Per-invocation FFmpeg overrides (inherits shared process config).

FFmpegFrameExtractionProcessor(config=None)

Bases: BaseFFmpegProcessor, FrameExtractionProcessor

Extract image frames with FFmpeg (optional yadif).

Attributes:

Name Type Description
config FFmpegFrameExtractionConfig

Stable constructor configuration.

Initialize the FFmpeg frame extraction processor.

Parameters:

Name Type Description Default
config dict[str, Any] | FrameExtractionConfig | FFmpegFrameExtractionConfig | None

Shared or FFmpeg-specific config. Defaults to None.

 None

config_model() classmethod

Return the stable configuration model.

extract_frames(video_path, timestamps, *, params=None)

Extract encoded image bytes at each timestamp from a video path.

Parameters:

Name Type Description Default
video_path str

Source video path.

 required
timestamps list[float]

Non-empty non-negative timestamps.

 required
params FFmpegFrameExtractionConfig | None

Effective options. Defaults to self.config.

 None

Returns:

Type Description
list[bytes]

list[bytes]: Encoded frames in the same order as timestamps.

Raises:

Type Description
FileNotFoundError

If ffmpeg is unavailable.
RuntimeError

If any extraction fails.
ValueError

If timestamps are invalid.

process(attachment, **kwargs) async

Extract frames from one video attachment.

Parameters:

Name Type Description Default
attachment Attachment

Source video.

 required
**kwargs Any

May include process_config.

 {}

Returns:

Type Description
list[Attachment]

list[Attachment]: One image attachment per timestamp.

process_config_model() classmethod

Return the per-invocation configuration model.

set_timestamps(timestamps)

Update constructor-level default timestamps.

Parameters:

Name Type Description Default
timestamps list[float]

Non-empty non-negative timestamps.

 required

FrameExtractionConfig

Bases: BaseModel

Backend-agnostic stable configuration for frame extraction.

Attributes:

Name Type Description
output_format str

Image encode format (JPEG/PNG). Defaults to JPEG.
deinterlace DeinterlaceMode

Deinterlace policy (off / force / auto). Bool aliases True→force, False→off are accepted. Defaults to off.
default_timestamps list[float] | None

Used when process is called without process_config. Defaults to None (must pass per-call).

validate_default_timestamps(value) classmethod

Validate optional constructor-level timestamps.

Parameters:

Name Type Description Default
value list[float] | None

Candidate timestamps.

 required

Returns:

Type Description
list[float] | None

list[float] | None: Validated timestamps or None.

validate_deinterlace_mode(value) classmethod

Coerce bool / string deinterlace values.

Parameters:

Name Type Description Default
value Any

Candidate mode.

 required

Returns:

Name Type Description
DeinterlaceMode DeinterlaceMode

Normalized mode.

validate_output_format(value) classmethod

Reject blank output format strings.

Parameters:

Name Type Description Default
value str

Candidate format.

 required

Returns:

Name Type Description
str str

Normalized upper-case format.

Raises:

Type Description
ValueError

If blank.

FrameExtractionProcessConfig

Bases: ProcessorProcessConfig

Backend-agnostic per-invocation frame extraction config.

Attributes:

Name Type Description
timestamps list[float]

Required non-empty timestamps in seconds.
output_format str | None

Optional format override.
deinterlace DeinterlaceMode | None

Optional deinterlace override.

validate_optional_deinterlace_mode(value) classmethod

Coerce optional bool / string deinterlace overrides.

Parameters:

Name Type Description Default
value Any

Candidate mode or None.

 required

Returns:

Type Description
DeinterlaceMode | None

DeinterlaceMode | None: Normalized mode or None.

validate_optional_output_format(value) classmethod

Normalize optional format override.

Parameters:

Name Type Description Default
value str | None

Candidate format.

 required

Returns:

Type Description
str | None

str | None: Upper-case format or None.

validate_process_timestamps(value) classmethod

Validate per-call timestamps.

Parameters:

Name Type Description Default
value list[float]

Candidate timestamps.

 required

Returns:

Type Description
list[float]

list[float]: Validated timestamps.

FrameExtractionProcessor()

Bases: BackendSelectableProcessor[Attachment, list[Attachment]], ABC

Family base for extracting image frames at timestamps.

Why use this base class?

  • Portability: Swap FFmpeg vs GStreamer without changing call sites.
  • Batch I/O: Extract many keyframes in one process call.
  • Separation: Keyframe planning stays in extractors; decode is here.

Usage

from gllm_multimodal.media_toolkit.processor.frame_extraction_processor import (
    FrameExtractionProcessConfig,
    FrameExtractionProcessor,
)

processor = FrameExtractionProcessor.build(backend="ffmpeg")
frames = await processor.process(
    video_attachment,
    process_config=FrameExtractionProcessConfig(timestamps=[1.5, 4.0]),
)

set_timestamps(timestamps) abstractmethod

Configure default timestamps used when process_config is omitted.

Parameters:

Name Type Description Default
timestamps list[float]

Non-empty list of non-negative seconds.

 required

coerce_frame_extraction_config(config)

Normalize optional shared frame-extraction config.

Parameters:

Name Type Description Default
config FrameExtractionConfig | dict[str, object] | None

Raw config.

 required

Returns:

Name Type Description
FrameExtractionConfig FrameExtractionConfig

Shared configuration shape.