Overview
Video clip processor family.
This package provides processors for extracting video clips from longer videos based on time ranges or keyframe boundaries.
Exported Classes
VideoClipProcessor-- Abstract base for video clip extraction.GstVideoClipProcessor-- GStreamer-based video clip processor.GstVideoClipConfig-- GStreamer video clip configuration.GstVideoClipProcessConfig-- GStreamer video clip process configuration.
GstVideoClipConfig(**data)
Bases: GstBaseConfig
Stable configuration for GstVideoClipProcessor.
Set once at construction. Controls encoder selection, timeout, and other
processor behaviour that does not change per attachment. Invalid options
raise ComponentConfigError when the model is constructed.
Attributes:
| Name | Type | Description |
|---|---|---|
default_windows |
list[tuple[float, float]] | None
|
Default clipping
windows used when |
Validate the options and report a failure as a component configuration error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**data
|
Any
|
The configuration options. |
{}
|
Raises:
| Type | Description |
|---|---|
ComponentConfigError
|
If an option is unknown or has an invalid value, such as empty
|
validate_default_windows(default_windows)
classmethod
Validate the constructor-level default windows, if any.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
default_windows
|
list[tuple[float, float]] | None
|
The candidate default windows to validate. |
required |
Returns:
| Type | Description |
|---|---|
list[tuple[float, float]] | None
|
list[tuple[float, float]] | None: |
list[tuple[float, float]] | None
|
when |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
GstVideoClipProcessConfig(**data)
Bases: ProcessorProcessConfig
Per-invocation configuration for [process][gllm_multimodal.media_toolkit.media_toolkit.MediaToolkit.process].
Invalid options raise MediaOperationConfigError when the model is constructed.
Attributes:
| Name | Type | Description |
|---|---|---|
windows |
list[tuple[float, float]]
|
Ordered |
Validate the options and report a failure as a media operation configuration error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**data
|
Any
|
The per-invocation options. |
{}
|
Raises:
| Type | Description |
|---|---|
MediaOperationConfigError
|
If an option is unknown or has an invalid value, such as empty
|
validate_windows(windows)
classmethod
Validate clipping window order and timestamp bounds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
windows
|
list[tuple[float, float]]
|
The windows to validate. |
required |
Returns:
| Type | Description |
|---|---|
list[tuple[float, float]]
|
list[tuple[float, float]]: The validated windows. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If windows are empty, contain nonfinite or negative
timestamps, exceed the signed GStreamer nanosecond range, or have
|
GstVideoClipProcessor(config=None)
Bases: BaseGstreamerProcessor[GstVideoClipConfig], VideoClipProcessor
Clips a video attachment to one or more [start_time, end_time] windows.
Stable settings belong in GstVideoClipConfig (constructor).
Per-call clip windows belong in GstVideoClipProcessConfig
(process(..., process_config=...)).
Attributes:
| Name | Type | Description |
|---|---|---|
config |
GstVideoClipConfig
|
Runtime configuration. |
Raises:
| Type | Description |
|---|---|
ComponentConfigError
|
If the constructor configuration is invalid. |
MediaOperationConfigError
|
If the per-call |
MediaInputInvalidError
|
If an attachment has no MIME type, a malformed one, or no media bytes. |
MediaInputUnsupportedError
|
If an attachment is not a video. |
MediaProfilingError
|
If the media format cannot be identified from the bytes. |
MediaTrackMissingError
|
If the media has no video track. |
MediaResultOperationError
|
If the pipeline completes without producing a video file. |
MediaOperationTimeoutError
|
If the pipeline does not finish before |
MediaOperationFailedError
|
If clipping fails with an error outside the GLLM taxonomy, for example when the pipeline fails, seeking fails, or a temporary file cannot be written. GLLM errors raised by the backend are re-raised unchanged. |
MediaBackendUnavailableError
|
If GStreamer or a required GStreamer element is unavailable. |
RuntimeError
|
If no suitable encoder is found. |
Initialise the clip processor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
dict[str, Any] | GstVideoClipConfig | None
|
Optional stable
configuration. |
None
|
Raises:
| Type | Description |
|---|---|
ComponentConfigError
|
If |
MediaBackendUnavailableError
|
If GStreamer is unavailable. |
RuntimeError
|
If no suitable encoder is found. |
config_model()
classmethod
Return the stable configuration model for this processor.
Returns:
| Type | Description |
|---|---|
type[GstVideoClipConfig]
|
type[GstVideoClipConfig]: The stable configuration model for this processor. |
process(attachment, **kwargs)
async
Clip one video attachment using resolved time windows.
This method is the canonical caller-facing API for clipping. It accepts
stable default windows set at construction time and also supports
per-call window overrides through process_config.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
attachment
|
Attachment
|
Input video attachment to clip. |
required |
**kwargs
|
Any
|
Runtime options, typically |
{}
|
Notes
- Delegates shared mimetype validation and dispatch to
MediaToolkit.process.
Returns:
| Type | Description |
|---|---|
Attachment | list[Attachment]
|
Attachment | list[Attachment]: Single clipped attachment when exactly |
Attachment | list[Attachment]
|
one window is used, otherwise a list of clipped attachments in window order. |
Raises:
| Type | Description |
|---|---|
MediaInputInvalidError
|
If the input is not an attachment, or the attachment has no MIME type, a malformed one, or no media bytes. |
MediaInputUnsupportedError
|
If the attachment is not a video. |
MediaOperationConfigError
|
If |
MediaProfilingError
|
If the media format cannot be identified from the bytes. |
MediaTrackMissingError
|
If the media has no video track. |
MediaResultOperationError
|
If the pipeline completes without producing a video file. |
MediaOperationTimeoutError
|
If the pipeline does not finish before |
MediaBackendUnavailableError
|
If a required GStreamer element is unavailable. |
MediaOperationFailedError
|
If clipping fails with an error outside the GLLM taxonomy. |
Example
processor = GstVideoClipProcessor(
config={"default_windows": [(0.0, 10.0)]},
)
clips = await processor.process(
attachment=video_attachment,
process_config={
"windows": [(5.0, 12.5), (30.0, 45.0)],
},
)
process_config_model()
classmethod
Return the per-invocation configuration model for this processor.
Returns:
| Type | Description |
|---|---|
type[GstVideoClipProcessConfig]
|
type[GstVideoClipProcessConfig]: The per-invocation configuration model for this processor. |
set_windows(windows)
Update the constructor-level default windows.
Mutates config so process falls back to windows when
no process_config is supplied. Prefer passing
GstVideoClipProcessConfig to process for per-call
overrides.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
windows
|
list[tuple[float, float]]
|
The windows to set. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
None |
None
|
|
None
|
|
Raises:
| Type | Description |
|---|---|
ComponentConfigError
|
If |
VideoClipProcessor()
Bases: BackendSelectableProcessor[Attachment, Attachment], ABC
Family base for clipping video attachments to time windows.
This class serves as a unified entry point for video clipping operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.
Why use this base class?
- Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
- Simplicity: No need to handle fallback logic or conditional imports yourself.
- Future-proofing: New backends can be added to the library without requiring changes to your application code.
Usage Example
from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment
# Instantiates the best available backend automatically
processor = VideoClipProcessor.build()
# Set the target clipping window (start_time, end_time) in seconds
processor.set_windows([(10.0, 20.5)])
attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)
from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment
# Explicitly force the ffmpeg backend
processor = VideoClipProcessor.build(backend="ffmpeg")
processor.set_windows([(10.0, 20.5)])
attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)
from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment
# Explicitly force the moviepy backend
processor = VideoClipProcessor.build(backend="moviepy")
processor.set_windows([(10.0, 20.5)])
attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = await processor.process(attachment)
set_windows(windows)
abstractmethod
Configure one or more [start, end] clipping windows for the next call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
windows
|
list[tuple[float, float]]
|
List of (start, end) tuples in seconds. |
required |