Skip to content

Overview

Video clip processor family.

This package provides processors for extracting video clips from longer videos based on time ranges or keyframe boundaries.

Exported Classes

GstVideoClipConfig(**data)

Bases: GstBaseConfig

Stable configuration for GstVideoClipProcessor.

Set once at construction. Controls encoder selection, timeout, and other processor behaviour that does not change per attachment. Invalid options raise ComponentConfigError when the model is constructed.

Attributes:

Name Type Description
default_windows list[tuple[float, float]] | None

Default clipping windows used when GstVideoClipProcessor.process is called without a process_config. None (the default) means windows must be supplied per-call via GstVideoClipProcessConfig. When set, the value is validated by GstVideoClipProcessConfig's field_validator.

Validate the options and report a failure as a component configuration error.

Parameters:

Name Type Description Default
**data Any

The configuration options.

{}

Raises:

Type Description
ComponentConfigError

If an option is unknown or has an invalid value, such as empty default_windows or a window that is not finite, nonnegative, and well ordered.

validate_default_windows(default_windows) classmethod

Validate the constructor-level default windows, if any.

Parameters:

Name Type Description Default
default_windows list[tuple[float, float]] | None

The candidate default windows to validate.

required

Returns:

Type Description
list[tuple[float, float]] | None

list[tuple[float, float]] | None: default_windows unchanged

list[tuple[float, float]] | None

when None or when every window passes validation.

Raises:

Type Description
ValueError

If default_windows fails the checks of GstVideoClipProcessConfig's field_validator.

GstVideoClipProcessConfig(**data)

Bases: ProcessorProcessConfig

Per-invocation configuration for [process][gllm_multimodal.media_toolkit.media_toolkit.MediaToolkit.process].

Invalid options raise MediaOperationConfigError when the model is constructed.

Attributes:

Name Type Description
windows list[tuple[float, float]]

Ordered (start_time, end_time) windows in seconds. Timestamps must be finite, nonnegative, and less than (2**63 - 1) / 1_000_000_000 seconds so their nanosecond values fit GStreamer's signed integer range. Each window must satisfy end > start.

Validate the options and report a failure as a media operation configuration error.

Parameters:

Name Type Description Default
**data Any

The per-invocation options.

{}

Raises:

Type Description
MediaOperationConfigError

If an option is unknown or has an invalid value, such as empty windows or a window that is not finite, nonnegative, and well ordered.

validate_windows(windows) classmethod

Validate clipping window order and timestamp bounds.

Parameters:

Name Type Description Default
windows list[tuple[float, float]]

The windows to validate.

required

Returns:

Type Description
list[tuple[float, float]]

list[tuple[float, float]]: The validated windows.

Raises:

Type Description
ValueError

If windows are empty, contain nonfinite or negative timestamps, exceed the signed GStreamer nanosecond range, or have end <= start.

GstVideoClipProcessor(config=None)

Bases: BaseGstreamerProcessor[GstVideoClipConfig], VideoClipProcessor

Clips a video attachment to one or more [start_time, end_time] windows.

Stable settings belong in GstVideoClipConfig (constructor). Per-call clip windows belong in GstVideoClipProcessConfig (process(..., process_config=...)).

Attributes:

Name Type Description
config GstVideoClipConfig

Runtime configuration.

Raises:

Type Description
ComponentConfigError

If the constructor configuration is invalid.

MediaOperationConfigError

If the per-call process_config is invalid or no windows are available.

MediaInputInvalidError

If an attachment has no MIME type, a malformed one, or no media bytes.

MediaInputUnsupportedError

If an attachment is not a video.

MediaProfilingError

If the media format cannot be identified from the bytes.

MediaTrackMissingError

If the media has no video track.

MediaResultOperationError

If the pipeline completes without producing a video file.

MediaOperationTimeoutError

If the pipeline does not finish before config.timeout.

MediaOperationFailedError

If clipping fails with an error outside the GLLM taxonomy, for example when the pipeline fails, seeking fails, or a temporary file cannot be written. GLLM errors raised by the backend are re-raised unchanged.

MediaBackendUnavailableError

If GStreamer or a required GStreamer element is unavailable.

RuntimeError

If no suitable encoder is found.

Initialise the clip processor.

Parameters:

Name Type Description Default
config dict[str, Any] | GstVideoClipConfig | None

Optional stable configuration. None uses GstVideoClipConfig defaults.

None

Raises:

Type Description
ComponentConfigError

If config contains an unknown option or an invalid value, such as empty default_windows, or is not a GstVideoClipConfig, dict, or None.

MediaBackendUnavailableError

If GStreamer is unavailable.

RuntimeError

If no suitable encoder is found.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type Description
type[GstVideoClipConfig]

type[GstVideoClipConfig]: The stable configuration model for this processor.

process(attachment, **kwargs) async

Clip one video attachment using resolved time windows.

This method is the canonical caller-facing API for clipping. It accepts stable default windows set at construction time and also supports per-call window overrides through process_config.

Parameters:

Name Type Description Default
attachment Attachment

Input video attachment to clip.

required
**kwargs Any

Runtime options, typically process_config as GstVideoClipProcessConfig or dict.

{}
Notes
  1. Delegates shared mimetype validation and dispatch to MediaToolkit.process.

Returns:

Type Description
Attachment | list[Attachment]

Attachment | list[Attachment]: Single clipped attachment when exactly

Attachment | list[Attachment]

one window is used, otherwise a list of clipped attachments in window order.

Raises:

Type Description
MediaInputInvalidError

If the input is not an attachment, or the attachment has no MIME type, a malformed one, or no media bytes.

MediaInputUnsupportedError

If the attachment is not a video.

MediaOperationConfigError

If process_config is invalid or no windows are available.

MediaProfilingError

If the media format cannot be identified from the bytes.

MediaTrackMissingError

If the media has no video track.

MediaResultOperationError

If the pipeline completes without producing a video file.

MediaOperationTimeoutError

If the pipeline does not finish before config.timeout.

MediaBackendUnavailableError

If a required GStreamer element is unavailable.

MediaOperationFailedError

If clipping fails with an error outside the GLLM taxonomy.

Example
processor = GstVideoClipProcessor(
    config={"default_windows": [(0.0, 10.0)]},
)
clips = await processor.process(
    attachment=video_attachment,
    process_config={
        "windows": [(5.0, 12.5), (30.0, 45.0)],
    },
)

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type Description
type[GstVideoClipProcessConfig]

type[GstVideoClipProcessConfig]: The per-invocation configuration model for this processor.

set_windows(windows)

Update the constructor-level default windows.

Mutates config so process falls back to windows when no process_config is supplied. Prefer passing GstVideoClipProcessConfig to process for per-call overrides.

Parameters:

Name Type Description Default
windows list[tuple[float, float]]

The windows to set.

required

Returns:

Name Type Description
None None

self.config is replaced with a copy whose

None

default_windows is the validated windows.

Raises:

Type Description
ComponentConfigError

If windows is empty or contains a window that is not finite, nonnegative, and well ordered. Validation reuses GstVideoClipProcessConfig's field_validator.

VideoClipProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for clipping video attachments to time windows.

This class serves as a unified entry point for video clipping operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

  • Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
  • Simplicity: No need to handle fallback logic or conditional imports yourself.
  • Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = VideoClipProcessor.build()

# Set the target clipping window (start_time, end_time) in seconds
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)
from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = VideoClipProcessor.build(backend="ffmpeg")
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)
from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Explicitly force the moviepy backend
processor = VideoClipProcessor.build(backend="moviepy")
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)

set_windows(windows) abstractmethod

Configure one or more [start, end] clipping windows for the next call.

Parameters:

Name Type Description Default
windows list[tuple[float, float]]

List of (start, end) tuples in seconds.

required