Skip to content

Overview

Frame sampling processor family.

This package provides processors for sampling frames from video streams at specified intervals or based on scene changes.

Exported Classes

FrameSamplingProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for resampling video attachments to a target frame rate.

This class serves as a unified entry point for frame sampling operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

  • Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
  • Simplicity: No need to handle fallback logic or conditional imports yourself.
  • Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = FrameSamplingProcessor.build(target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = FrameSamplingProcessor.build(backend="ffmpeg", target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Explicitly force the cv2 backend
processor = FrameSamplingProcessor.build(backend="cv2", target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

GstFrameSamplingConfig

Bases: GstBaseConfig

Stable configuration for GstFrameSamplingProcessor.

Attributes:

Name Type Description
min_size_mb int

Minimum video size in MB to process. 0 disables the check. Defaults to 0.
default_target_fps int

Default target FPS used when GstFrameSamplingProcessor.process is called without a process_config. Must be greater than 0. Defaults to 1.

validate_default_target_fps(default_target_fps) classmethod

Validate that default_target_fps is positive.

Parameters:

Name Type Description Default
default_target_fps int

The candidate default FPS to validate.

 required

Returns:

Name Type Description
int int

default_target_fps unchanged when valid.

Raises:

Type Description
ValueError

If default_target_fps is not greater than 0.

GstFrameSamplingProcessConfig

Bases: ProcessorProcessConfig

Per-invocation configuration for process.

Attributes:

Name Type Description
target_fps int

Target frames-per-second for the output video.

validate_target_fps(target_fps) classmethod

Validate that target_fps is positive.

Parameters:

Name Type Description Default
target_fps int

The candidate target FPS to validate.

 required

Returns:

Name Type Description
int int

target_fps unchanged when valid.

Raises:

Type Description
ValueError

If target_fps is not greater than 0.

GstFrameSamplingProcessor(target_fps=None, config=None)

Bases: BaseGstreamerProcessor, FrameSamplingProcessor

Resamples a video attachment to a target frame-rate using GStreamer.

The processor is format-agnostic: it handles MP4, MKV, MOV, and similar containers and preserves any audio track (passthrough by default).

Attributes:

Name Type Description
target_fps int

The target frames-per-second for the output video.
config GstFrameSamplingConfig

Runtime configuration.

Raises:

Type Description
RuntimeError

If GStreamer is not available or no suitable encoder is found.

Initialise and select encoders.

Deprecated

The target_fps parameter is deprecated. Pass config=GstFrameSamplingConfig(default_target_fps=target_fps) instead.

Parameters:

Name Type Description Default
target_fps int | None

Deprecated. Default target FPS applied at construction. Equivalent to passing config=GstFrameSamplingConfig(default_target_fps=target_fps). Defaults to None (no override; falls back to GstFrameSamplingConfig.default_target_fps).

 None
config dict[str, Any] | GstFrameSamplingConfig | None

Optional stable configuration.

 None

Raises:

Type Description
ValueError

If target_fps (or config.default_target_fps) is non-positive.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type Description
type[GstFrameSamplingConfig]

type[GstFrameSamplingConfig]: The stable configuration model for this processor.

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type Description
type[GstFrameSamplingProcessConfig]

type[GstFrameSamplingProcessConfig]: The configuration class
type[GstFrameSamplingProcessConfig]

accepted by process for per-call overrides.