Overview

Media toolkit processor public API and loading entrypoint.

This package exposes processor families and shared backend base classes used by builder/factory flows. Importing from here ensures family classes and concrete backend implementations are loaded so subclass registration is populated in MediaToolkit.registry and each family's backend map.

Design overview

• Family package layout: <family>/base.py for the abstract family class and <family>/<backend>_backend.py for concrete implementations.
• Construction: MediaToolkit.build("AudioExtractionProcessor", backend="gstreamer").
• Composite components (segmenters/keyframe extractors): cache nested processors by class name + backend and can switch backend by changing component.backend before calling get_processor(...).

Contributor quick-start for a new processor family

1. Create processor/<new_family>/base.py subclassing BackendSelectableProcessor.
2. Add backend implementations declaring BACKEND (+ IS_DEFAULT if needed).
3. Re-export in processor/<new_family>/__init__.py and this module.
4. Add tests for family registration and factory creation.

AudioExtractionProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for extracting audio tracks from video attachments.

This class serves as a unified entry point for audio extraction operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

• Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
• Simplicity: No need to handle fallback logic or conditional imports yourself.
• Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

Auto (Recommended)

from gllm_multimodal.media_toolkit.processor.audio_extraction_processor import AudioExtractionProcessor
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = AudioExtractionProcessor.build()

attachment = Attachment(url="file:///path/to/video.mp4")
audio_attachment = processor(attachment)

FFmpeg Backend

from gllm_multimodal.media_toolkit.processor.audio_extraction_processor import AudioExtractionProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = AudioExtractionProcessor.build(backend="ffmpeg")

attachment = Attachment(url="file:///path/to/video.mp4")
audio_attachment = processor(attachment)

MoviePy Backend

from gllm_multimodal.media_toolkit.processor.audio_extraction_processor import AudioExtractionProcessor
from gllm_inference.schema import Attachment

# Explicitly force the moviepy backend
processor = AudioExtractionProcessor.build(backend="moviepy")

attachment = Attachment(url="file:///path/to/video.mp4")
audio_attachment = processor(attachment)

BackendSelectableProcessor()

Bases: MediaToolkit[T_in, T_out], ABC

Abstract base for a processor family with pluggable backends.

Each family owns its backend registry. Concrete backend classes are auto-registered via __init_subclass__ by declaring:

• BACKEND: backend key, e.g. gstreamer or ffmpeg
• IS_DEFAULT: whether this backend is the family default

Why this exists

It lets callers construct by stable family class name while deferring runtime selection of backend implementation.

Minimal contributor pattern

class ImageTilingProcessor(BackendSelectableProcessor): ... in base.py, then concrete backends such as class PilImageTilingProcessor(ImageTilingProcessor): BACKEND = "pil" and class Cv2ImageTilingProcessor(ImageTilingProcessor): BACKEND = "cv2".

Then callers can use

MediaToolkit.build("ImageTilingProcessor", backend="pil").

__init_subclass__(**kwargs)

Automatically register concrete backend implementations.

Family base classes reset their own registry; abstract intermediate classes are ignored; concrete classes must declare BACKEND.

build(backend=None, **kwargs) classmethod

Build a backend implementation for this processor family.

Parameters:

Name	Type	Description	Default
backend	str | None	Backend key. Uses the family default when omitted.	None
**kwargs	Any	Constructor kwargs forwarded to the backend class.	{}

Returns:

Name	Type	Description
BackendSelectableProcessor	BackendSelectableProcessor	Instantiated backend processor.

Raises:

Type	Description
ValueError	If the backend is unknown or no default is configured.

Example

For a family class AudioExtractionProcessor, AudioExtractionProcessor.build(backend="gstreamer") returns the registered GStreamer implementation class instance.

build_from_registry(backend=None, **kwargs) classmethod

Build a family backend or instantiate a concrete backend class.

Parameters:

Name	Type	Description	Default
backend	str | MediaBackend | None	Backend key for family resolution. Defaults to None.	None
**kwargs	Any	Constructor kwargs forwarded to the backend class.	{}

Returns:

Name	Type	Description
BackendSelectableProcessor	BackendSelectableProcessor	Instantiated processor.

is_family_base(processor_cls) classmethod

Return whether processor_cls is a family abstract base.

Parameters:

Name	Type	Description	Default
processor_cls	type[BackendSelectableProcessor]	Candidate processor class.	required

Returns:

Name	Type	Description
bool	bool	True when the class directly subclasses BackendSelectableProcessor.

list_available_backends() classmethod

Return registered backend keys for backend-selectable family bases.

Returns:

Type	Description
list[str]	list[str]: Backend keys available for build. Empty for concrete backend implementations and non-family classes.

list_backends() classmethod

Return registered backend keys for this processor family.

Returns:

Type	Description
list[str]	list[str]: Backend keys available for build.

BaseGstreamerProcessor(config=None, *, enable_video=True, enable_audio=True)

Bases: MediaToolkit[Attachment, Attachment]

Abstract base class for GStreamer-powered processors.

Subclasses must implement only _execute_pipeline. All common concerns — availability checks, Conda environment configuration, GStreamer initialisation, encoder/element selection, the standard EOS/error bus loop, and temporary-file cleanup — are handled here.

Typical subclass skeleton::

class MyGstProcessor(BaseGstreamerProcessor):
    def __init__(self, my_param, config=None):
        super().__init__(config=config)
        self.my_param = my_param

    async def _process(self, attachment: Attachment) -> Attachment:
        return await self._process_single(attachment)

    async def _execute_pipeline(self, input_path, output_path):
        # build & run YOUR GStreamer pipeline here
        ...

Attributes:

Name	Type	Description
logger		Logger bound to the concrete subclass name.
config	GstBaseConfig	Runtime configuration.
video_encoder_info	EncoderFormatInfo | None	Selected video encoder format info, or None when video is disabled.
audio_encoder_info	EncoderFormatInfo | None	Selected audio encoder format info, or None when audio is disabled.

Verify GStreamer availability, configure the environment, and select encoders.

Parameters:

Name	Type	Description	Default
config	dict[str, Any] | GstBaseConfig | None	Optional configuration. A plain dict is coerced into a GstBaseConfig. None uses all GstBaseConfig defaults.	None
enable_video	bool	When True, select a video encoder. Defaults to True.	True
enable_audio	bool	When True, select an audio encoder. Defaults to True.	True

Raises:

Type	Description
RuntimeError	If GStreamer is not installed or cannot be initialised.
RuntimeError	If no suitable video encoder is found in the registry.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type	Description
type[GstBaseConfig]	type[GstBaseConfig]: The configuration class used at construction time.

FrameSamplingProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for resampling video attachments to a target frame rate.

This class serves as a unified entry point for frame sampling operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

• Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
• Simplicity: No need to handle fallback logic or conditional imports yourself.
• Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

Auto (Recommended)

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = FrameSamplingProcessor.build(target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

FFmpeg Backend

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = FrameSamplingProcessor.build(backend="ffmpeg", target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

OpenCV Backend

from gllm_multimodal.media_toolkit.processor.frame_sampling_processor import FrameSamplingProcessor
from gllm_inference.schema import Attachment

# Explicitly force the cv2 backend
processor = FrameSamplingProcessor.build(backend="cv2", target_fps=2)

attachment = Attachment(url="file:///path/to/video.mp4")
sampled_video = processor(attachment)

GstAudioExtractionConfig

Bases: GstBaseConfig

Configuration for GstAudioExtractionProcessor.

Attributes:

Name	Type	Description
sample_rate	int | None	Force output sample rate in Hz (e.g. 16000). None preserves the source rate. Defaults to None.
channels	int | None	Force output channel count (e.g. 1 for mono). None preserves the source count. Defaults to None.
output_format	str | None	Preferred container/extension to extract to ("wav", "mp3", "m4a", or "ogg"). When set, only encoders for that format are considered. Defaults to None (auto-select from all candidates, preferring WAV when available).
audio_encoder	str | None	Pin a specific GStreamer audio encoder element (e.g. "lamemp3enc"). When both audio_encoder and output_format are set, the candidate list is first narrowed to the pinned encoder, then further filtered to the requested format; an incompatible combination raises ValueError. Defaults to None.

GstAudioExtractionProcessConfig

Bases: ProcessorProcessConfig

Per-invocation configuration for process.

Attributes:

Name	Type	Description
sample_rate	int | None	Force output sample rate in Hz. None is a legitimate per-call override meaning "do not force a sample rate" — it suppresses the constructor self.config.sample_rate so the source rate is preserved.
channels	int | None	Force output channel count. None is a legitimate per-call override meaning "do not force a channel count" — it suppresses the constructor self.config.channels.

GstAudioExtractionProcessor(config=None)

Bases: BaseGstreamerProcessor, AudioExtractionProcessor

Extracts the audio track from a video Attachment using GStreamer.

The processor demuxes the input video, re-encodes (or passes through) the audio into the best available format, and returns the result as an Attachment whose mime_type reflects the audio container.

If the video has no audio track the original Attachment is returned unchanged, so callers do not need to handle None.

Attributes:

Name	Type	Description
audio_encoder_info	EncoderFormatInfo | None	Selected audio encoder format info.
config	GstAudioExtractionConfig	Runtime configuration.

Raises:

Type	Description
RuntimeError	If GStreamer is unavailable or no suitable audio encoder is found.
ValueError	If output_format or audio_encoder is unsupported.

Example

processor = GstAudioExtractionProcessor(config={"output_format": "mp3"})
audio_attachment = await processor.process(video_attachment)
# audio_attachment.mime_type == "audio/mpeg"
# audio_attachment.filename  == "audio_my_video.mp3"

Initialise GStreamer and select the audio encoder / output format.

Parameters:

Name	Type	Description	Default
config	dict[str, Any] | GstAudioExtractionConfig | None	Optional configuration. A plain dict is coerced into GstAudioExtractionConfig. None uses defaults. Use output_format (e.g. "mp3") or audio_encoder (e.g. "lamemp3enc") to override auto-selection.	None

Raises:

Type	Description
RuntimeError	If GStreamer is unavailable or no audio encoder is found.
ValueError	If output_format or audio_encoder is unsupported.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type	Description
type[GstAudioExtractionConfig]	type[GstAudioExtractionConfig]: The configuration class used at
type[GstAudioExtractionConfig]	construction time for stable (per-instance) settings.

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type	Description
type[GstAudioExtractionProcessConfig]	type[GstAudioExtractionProcessConfig]: The configuration class
type[GstAudioExtractionProcessConfig]	accepted by process for per-call overrides.

GstBaseConfig

Bases: BaseModel

Minimal shared configuration for all GStreamer-based processors.

Attributes:

Name	Type	Description
timeout	int	Maximum seconds a GStreamer pipeline may run before being forcibly terminated. Defaults to 300 seconds.
audio_passthrough	bool	When True (the default), encoded audio streams are passed directly to the muxer without being decoded and re-encoded. Set to False to force re-encoding via the best available audio encoder.
video_encoder	str | None	Pin a specific GStreamer video encoder element name (e.g. "x264enc"). When set, the candidate-list scan is skipped entirely and this element is used directly. The element must exist in the GStreamer registry. Defaults to None (auto-select from _VIDEO_FORMAT_MAP).
audio_encoder	str | None	Pin a specific GStreamer audio encoder element name (e.g. "voaacenc"). When set, the candidate-list scan is skipped entirely and this element is used directly. The element must exist in the GStreamer registry. Defaults to None (auto-select from _AUDIO_FORMAT_MAP).

GstFrameSamplingConfig

Bases: GstBaseConfig

Stable configuration for GstFrameSamplingProcessor.

Attributes:

Name	Type	Description
min_size_mb	int	Minimum video size in MB to process. 0 disables the check. Defaults to 0.
default_target_fps	int	Default target FPS used when GstFrameSamplingProcessor.process is called without a process_config. Must be greater than 0. Defaults to 1.

validate_default_target_fps(default_target_fps) classmethod

Validate that default_target_fps is positive.

Parameters:

Name	Type	Description	Default
default_target_fps	int	The candidate default FPS to validate.	required

Returns:

Name	Type	Description
int	int	default_target_fps unchanged when valid.

Raises:

Type	Description
ValueError	If default_target_fps is not greater than 0.

GstFrameSamplingProcessConfig

Bases: ProcessorProcessConfig

Per-invocation configuration for process.

Attributes:

Name	Type	Description
target_fps	int	Target frames-per-second for the output video.

validate_target_fps(target_fps) classmethod

Validate that target_fps is positive.

Parameters:

Name	Type	Description	Default
target_fps	int	The candidate target FPS to validate.	required

Returns:

Name	Type	Description
int	int	target_fps unchanged when valid.

Raises:

Type	Description
ValueError	If target_fps is not greater than 0.

GstFrameSamplingProcessor(target_fps=None, config=None)

Bases: BaseGstreamerProcessor, FrameSamplingProcessor

Resamples a video attachment to a target frame-rate using GStreamer.

The processor is format-agnostic: it handles MP4, MKV, MOV, and similar containers and preserves any audio track (passthrough by default).

Attributes:

Name	Type	Description
target_fps	int	The target frames-per-second for the output video.
config	GstFrameSamplingConfig	Runtime configuration.

Raises:

Type	Description
RuntimeError	If GStreamer is not available or no suitable encoder is found.

Initialise and select encoders.

Deprecated

The target_fps parameter is deprecated. Pass config=GstFrameSamplingConfig(default_target_fps=target_fps) instead.

Parameters:

Name	Type	Description	Default
target_fps	int | None	Deprecated. Default target FPS applied at construction. Equivalent to passing config=GstFrameSamplingConfig(default_target_fps=target_fps). Defaults to None (no override; falls back to GstFrameSamplingConfig.default_target_fps).	None
config	dict[str, Any] | GstFrameSamplingConfig | None	Optional stable configuration.	None

Raises:

Type	Description
ValueError	If target_fps (or config.default_target_fps) is non-positive.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type	Description
type[GstFrameSamplingConfig]	type[GstFrameSamplingConfig]: The stable configuration model for this processor.

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type	Description
type[GstFrameSamplingProcessConfig]	type[GstFrameSamplingProcessConfig]: The configuration class
type[GstFrameSamplingProcessConfig]	accepted by process for per-call overrides.

GstVideoClipConfig

Bases: GstBaseConfig

Stable configuration for GstVideoClipProcessor.

Set once at construction. Controls encoder selection, timeout, and other processor behaviour that does not change per attachment.

Attributes:

Name	Type	Description
default_windows	list[tuple[float, float]] | None	Default clipping windows used when GstVideoClipProcessor.process is called without a process_config. None (the default) means windows must be supplied per-call via GstVideoClipProcessConfig. When set, the value is validated by GstVideoClipProcessConfig's field_validator.

validate_default_windows(default_windows) classmethod

Validate the constructor-level default windows, if any.

Parameters:

Name	Type	Description	Default
default_windows	list[tuple[float, float]] | None	The candidate default windows to validate.	required

Returns:

Type	Description
list[tuple[float, float]] | None	list[tuple[float, float]] | None: default_windows unchanged
list[tuple[float, float]] | None	when None or when every window is well-ordered.

Raises:

Type	Description
ValueError	If default_windows is empty or any window has end <= start. Validation mirrors GstVideoClipProcessConfig's field_validator.

GstVideoClipProcessConfig

Bases: ProcessorProcessConfig

Per-invocation configuration for process.

Attributes:

Name	Type	Description
windows	list[tuple[float, float]]	Ordered (start_time, end_time) windows in seconds. Each window must satisfy end > start.

validate_windows(windows) classmethod

Validate that every clipping window is non-empty and well ordered.

Parameters:

Name	Type	Description	Default
windows	list[tuple[float, float]]	The windows to validate.	required

Returns:

Type	Description
list[tuple[float, float]]	list[tuple[float, float]]: The validated windows.

GstVideoClipProcessor(windows=None, config=None)

Bases: BaseGstreamerProcessor, VideoClipProcessor

Clips a video attachment to one or more [start_time, end_time] windows.

Stable settings belong in GstVideoClipConfig (constructor). Per-call clip windows belong in GstVideoClipProcessConfig (process(..., process_config=...)).

Initialise the clip processor.

Deprecated

The windows parameter is deprecated. Pass config=GstVideoClipConfig(default_windows=windows) instead.

Parameters:

Name	Type	Description	Default
windows	list[tuple[float, float]] | None	Deprecated. Default windows applied at construction. Equivalent to passing config=GstVideoClipConfig(default_windows=windows). Prefer config.default_windows or passing GstVideoClipProcessConfig to process directly.	None
config	dict[str, Any] | GstVideoClipConfig | None	Optional stable configuration. None uses GstVideoClipConfig defaults.	None

Raises:

Type	Description
ValueError	If windows (or config.default_windows) is empty or any window has end <= start.

config_model() classmethod

Return the stable configuration model for this processor.

Returns:

Type	Description
type[GstVideoClipConfig]	type[GstVideoClipConfig]: The stable configuration model for this processor.

process_config_model() classmethod

Return the per-invocation configuration model for this processor.

Returns:

Type	Description
type[GstVideoClipProcessConfig]	type[GstVideoClipProcessConfig]: The per-invocation configuration model for this processor.

set_windows(windows)

Update the constructor-level default windows.

Mutates config so process falls back to windows when no process_config is supplied. Prefer passing GstVideoClipProcessConfig to process for per-call overrides.

Parameters:

Name	Type	Description	Default
windows	list[tuple[float, float]]	The windows to set.	required

Returns:

Name	Type	Description
None	None	self.config is replaced with a copy whose
	None	default_windows is the validated windows.

Raises:

Type	Description
ValueError	If windows is empty or any window has end <= start. Validation reuses GstVideoClipProcessConfig's field_validator.

MediaToolkit()

Bases: ABC, Generic[T_in, T_out]

Base abstraction for all media toolkit processing components.

This class provides the shared lifecycle and registry behavior used by both: - concrete leaf processors (e.g. backend-specific audio/video processors), and - composite components (e.g. segmenters, keyframe extractors) that orchestrate nested processors.

Key responsibilities: - auto-register subclasses by class name for class-name-based construction via build; - provide consistent input validation against supported_mimetypes; - define async processing contracts through process and process_batch.

Contributor guidance: - inherit this class directly for concrete processors with custom behavior; - inherit BackendSelectableProcessor when one logical processor family maps to multiple backend implementations; - inherit composite bases (e.g. BaseSegmenter) for orchestration-style components.

Example

Building a processor by class name

from gllm_multimodal.media_toolkit.media_toolkit import MediaToolkit

processor = MediaToolkit.build("AudioExtractionProcessor", backend="gstreamer")
result = await processor.process(video_attachment)

Checking mimetype support

if processor.is_supported(attachment):
    result = await processor.process(attachment)

Listing registered processors

print(list(MediaToolkit.registry.keys()))
# ['GstAudioExtractionProcessor', 'GstVideoClipProcessor', ...]

Initialize processor logging.

supported_mimetypes = ['*/*'] class-attribute

MIME types this processor accepts (supports wildcards, e.g. 'video/*').

Defaults to ['*/*'] (accept all). Override as a class attribute in subclasses.

__init_subclass__(**kwargs)

Register every subclass by class name automatically.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Extra class declaration kwargs.	{}

Raises:

Type	Description
TypeError	If a subclass with the same class name is already registered, preventing silent dispatch to the wrong implementation.

available_backends_for(class_name) classmethod

Return backend keys registered for a processor family class name.

Parameters:

Name	Type	Description	Default
class_name	str	Registered processor family class name.	required

Returns:

Type	Description
list[str]	list[str]: Available backend keys. Empty when the class is unknown or not a backend-selectable family base.

Raises:

Type	Description
ValueError	If the class name is unknown.

build(class_name, backend=None, **kwargs) classmethod

Build a processor by class name.

Family abstract classes (e.g. AudioExtractionProcessor) resolve a concrete backend implementation via backend. Composite components (segmenters, keyframe extractors) store backend on the instance for nested resolution.

Parameters:

Name	Type	Description	Default
class_name	str	Registered subclass name.	required
backend	str | MediaBackend | None	Backend key for family classes, or nested processor preference for composite instances. Defaults to None.	None
**kwargs	Any	Constructor kwargs passed to the processor class.	{}

Returns:

Name	Type	Description
MediaToolkit	MediaToolkit	Instantiated processor.

Raises:

Type	Description
ValueError	If the class name is unknown.

Example

MediaToolkit.build("AudioExtractionProcessor", backend="gstreamer") resolves and returns a concrete backend class (e.g. GstAudioExtractionProcessor).

MediaToolkit.build("FixedDurationSegmenter", backend="gstreamer", segment_durations=[2.0]) creates the segmenter and stores backend preference for nested processor resolution.

build_from_registry(backend=None, **kwargs) classmethod

Instantiate this registered class.

Subclasses override this hook to customize registry-based construction (e.g. backend-selectable families resolve a concrete backend; composites store backend for nested processor resolution).

Parameters:

Name	Type	Description	Default
backend	str | MediaBackend | None	Backend key forwarded to subclass overrides. Ignored by the base implementation.	None
**kwargs	Any	Constructor kwargs passed to the processor class.	{}

Returns:

Name	Type	Description
MediaToolkit	MediaToolkit	Instantiated processor.

is_supported(attachment)

Return whether the attachment's mimetype is accepted by this processor.

Callers can use this to check compatibility before calling process or process_batch, avoiding a ValueError.

Parameters:

Name	Type	Description	Default
attachment	Attachment	The attachment to check.	required

Returns:

Name	Type	Description
bool	bool	True if the attachment's mimetype matches any entry in supported_mimetypes (including wildcards). True is also returned when the attachment has no mimetype set.

list_available_backends() classmethod

Return backend keys when this class supports backend selection.

Returns:

Type	Description
list[str]	list[str]: Available backend keys. Empty for classes that are not backend-selectable family bases.

process(attachment, **kwargs) async

Process a single attachment (or perform an aggregation on a list) and return the result.

Parameters:

Name	Type	Description	Default
attachment	T_in	The attachment or list of attachments to process.	required
**kwargs	Any	Additional keyword arguments forwarded to _process.	{}

Returns:

Name	Type	Description
T_out	T_out	The result of the processing.

process_batch(attachments, **kwargs) async

Process a batch of attachments.

Parameters:

Name	Type	Description	Default
attachments	list[T_in]	The batch of attachments to process.	required
**kwargs	Any	Additional keyword arguments forwarded to process.	{}

Returns:

Type	Description
list[T_out]	list[T_out]: The result of the batch processing.

ProcessorProcessConfig

Bases: BaseModel

Marker base class for per-invocation processor configuration.

Concrete processors define subclasses with the parameters that may change on every process() call. Stable processor settings belong in each processor's *Config model passed at construction time.

VideoClipProcessor()

Bases: BackendSelectableProcessor[Attachment, Attachment], ABC

Family base for clipping video attachments to time windows.

This class serves as a unified entry point for video clipping operations. It automatically routes requests to the most appropriate, available backend implementation based on your system environment.

Why use this base class?

• Portability: Your code will run regardless of which underlying libraries are installed on the host machine.
• Simplicity: No need to handle fallback logic or conditional imports yourself.
• Future-proofing: New backends can be added to the library without requiring changes to your application code.

Usage Example

Auto (Recommended)

from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Instantiates the best available backend automatically
processor = VideoClipProcessor.build()

# Set the target clipping window (start_time, end_time) in seconds
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = processor(attachment)

FFmpeg Backend

from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Explicitly force the ffmpeg backend
processor = VideoClipProcessor.build(backend="ffmpeg")
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = processor(attachment)

MoviePy Backend

from gllm_multimodal.media_toolkit.processor.video_clip_processor import VideoClipProcessor
from gllm_inference.schema import Attachment

# Explicitly force the moviepy backend
processor = VideoClipProcessor.build(backend="moviepy")
processor.set_windows([(10.0, 20.5)])

attachment = Attachment(url="file:///path/to/video.mp4")
clipped_video = processor(attachment)

set_windows(windows) abstractmethod

Configure one or more [start, end] clipping windows for the next call.

Parameters:

Name	Type	Description	Default
windows	list[tuple[float, float]]	List of (start, end) tuples in seconds.	required