Skip to content

Overview

Frame sampling processor family.

This package provides processors for sampling frames from video streams at specified intervals or based on scene changes.

Exported Classes

FrameSamplingProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for resampling video attachments to a target frame rate.

This class serves as a unified entry point for frame sampling operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

  • Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
  • Simplicity: No need to handle fallback logic or conditional imports yourself.
  • Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import (
    FrameSamplingProcessor,
    GstFrameSamplingConfig,
)
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = FrameSamplingProcessor.build(
    config=GstFrameSamplingConfig(default_target_fps=2)
)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = await processor.process(attachment)

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = FrameSamplingProcessor.build(backend="ffmpeg")

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = await processor.process(attachment)

GstFrameSamplingConfig

Bases: GstBaseConfig

Stable configuration for GstFrameSamplingProcessor.

Attributes:

Name Type Description
min_size_mb int

Minimum video size in MB to process. 0 disables the check. Defaults to 0.
default_target_fps int

Default target FPS used when GstFrameSamplingProcessor.process is called without a process_config. Must be greater than 0. Defaults to 1.

validate_default_target_fps(default_target_fps) classmethod

Validate that default_target_fps is positive.

Parameters:

Name Type Description Default
default_target_fps int

The candidate default FPS to validate.

 required

Returns:

Name Type Description
int int

default_target_fps unchanged when valid.

Raises:

Type Description
ValueError

If default_target_fps is not greater than 0.

GstFrameSamplingProcessConfig

Bases: ProcessorProcessConfig

Per-invocation configuration for [process][gllm_multimodal.media_toolkit.media_toolkit.MediaToolkit.process].

Attributes:

Name Type Description
target_fps int

Target frames-per-second for the output video.

validate_target_fps(target_fps) classmethod

Validate that target_fps is positive.

Parameters:

Name Type Description Default
target_fps int

The candidate target FPS to validate.

 required

Returns:

Name Type Description
int int

target_fps unchanged when valid.

Raises:

Type Description
ValueError

If target_fps is not greater than 0.

GstFrameSamplingProcessor(config=None)

Bases: BaseGstreamerProcessor[GstFrameSamplingConfig], FrameSamplingProcessor

Resamples a video attachment to a target frame-rate using GStreamer.

The processor is format-agnostic: it handles MP4, MKV, MOV, and similar containers and preserves any audio track (passthrough by default).

Attributes:

Name Type Description
config GstFrameSamplingConfig

Runtime configuration.

Raises:

Type Description
RuntimeError

If GStreamer is not available or no suitable encoder is found.

Initialise and select encoders.

Parameters:

Name Type Description Default
config dict[str, Any] | GstFrameSamplingConfig | None

Optional stable configuration.

 None

Raises:

Type Description
ValueError

If config.default_target_fps is non-positive.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type Description
type[GstFrameSamplingConfig]

type[GstFrameSamplingConfig]: The stable configuration model for this processor.

process(attachment, **kwargs) async

Resample one video attachment to the configured or requested FPS.

The method preserves the regular media-toolkit process contract while exposing this concrete processor's runtime semantics in API docs: constructor defaults establish baseline FPS behavior, and per-call overrides can be supplied through process_config.

Parameters:

Name Type Description Default
attachment Attachment

Input video attachment to resample.

 required
**kwargs Any

Runtime options, typically process_config as GstFrameSamplingProcessConfig or dict.

 {}
Notes
  1. Delegates shared mimetype validation and dispatch to MediaToolkit.process.

Returns:

Name Type Description
Attachment Attachment

Resampled video attachment, or the original attachment
Attachment

when the input is skipped by validation rules.
Example 
processor = GstFrameSamplingProcessor(
    config={"default_target_fps": 2},
)
sampled_attachment = await processor.process(
    attachment=video_attachment,
    process_config={"target_fps": 1},
)

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type Description
type[GstFrameSamplingProcessConfig]

type[GstFrameSamplingProcessConfig]: The configuration class
type[GstFrameSamplingProcessConfig]

accepted by
type[GstFrameSamplingProcessConfig]

process
type[GstFrameSamplingProcessConfig]

for per-call overrides.