Skip to content

Base Segmenter

Defines the base segmenter schema.

Segmenters divide media attachments (video or audio) into semantically or temporally distinct segments. This module provides BaseSegmenter, the abstract base class that defines the segment-and-materialize lifecycle.

Lifecycle

  1. await segmenter.segment(attachment) — computes segment boundaries only.
  2. await segmenter.process(attachment) — computes boundaries AND materializes media segments into separate Attachment objects.

Architecture

BaseSegmenter (abstract)
├── uses CompositeMediaMixin for nested processor resolution
├── shared cuts_to_segments + VideoClipProcessor materialization
├── FixedDurationSegmenter (concrete implementation)
└── ShotBasedSegmenter (concrete, FIPS-friendly implementation)

BaseSegmenter(config=None)

Bases: CompositeMediaMixin, MediaToolkit[Attachment, list[Attachment]], ABC, Generic[TConfig]

Abstract base class for segmenting media attachments.

Segmenters are responsible for dividing a media attachment (usually video or audio) into semantically or temporally distinct segments. The resulting attachments should contain VideoSegment metadata.

Attributes:

Name Type Description
backend MediaBackend | str | None

Preferred backend for nested processor lookup. Set when built via the registry with an explicit or resolved backend.

Processing contract
  • await segment(attachment) computes boundaries only.
  • await process(attachment) computes + materializes media segments.
  • Materialization can call nested processors via get_processor(...) and uses this segmenter's backend preference.
Example

segmenter = build_media_toolkit( "FixedDurationSegmenter", backend=MediaBackend.GSTREAMER, ... ) then await segmenter.process(video_attachment) uses the GStreamer video clip processor during materialization.

Validate and store the concrete segmenter's configuration.

Parameters:

Name Type Description Default
config TConfig | BaseSegmenterConfig | dict[str, Any] | None

Config model, base config, mapping, or None for model defaults. Concrete config models decide whether defaults satisfy their required fields.

None

Raises:

Type Description
ComponentConfigError

If config has an unknown option or an invalid value, or is not a supported config type.

config_model() classmethod

Return the concrete Pydantic model used to validate this segmenter's config.

The base implementation provides BaseSegmenterConfig for simple subclasses that do not define additional configuration fields.

Returns:

Type Description
type[TConfig]

type[TConfig]: The config model associated with the concrete segmenter.

cuts_to_segments(cut_times, duration, *, min_shot_duration) staticmethod

Convert interior cut timestamps into contiguous shot windows.

Cut timestamps are sorted first, so the windows are contiguous whatever order the cuts arrive in.

Parameters:

Name Type Description Default
cut_times list[float]

Interior cut timestamps in seconds.

required
duration float

Total video duration in seconds.

required
min_shot_duration float

Windows shorter than this (after the first) are merged into the previous window.

required

Returns:

Type Description
list[VideoSegment]

list[VideoSegment]: Shot windows spanning [0, duration].

Raises:

Type Description
MediaOperationConfigError

If duration is not a finite number greater than 0, or cut_times is not a list of finite numbers.

materialize(attachment, segment, segment_index=0) async

Materialize one segment plan into a media attachment.

Guarantees mimetype validation before calling _materialize.

Parameters:

Name Type Description Default
attachment Attachment

Source attachment to clip or reference.

required
segment VideoSegment

Segment plan with boundary metadata.

required
segment_index int

Zero-based index within the segment plan list. Defaults to 0.

0

Returns:

Name Type Description
Attachment Attachment

Materialized segment attachment with

Attachment

[VideoSegment][gllm_core.schema.multimodal.video_caption.VideoSegment] metadata.

Raises:

Type Description
MediaInputInvalidError

If attachment is not an Attachment or its MIME type is malformed.

MediaInputUnsupportedError

If the attachment is audio or not a video.

MediaOperationConfigError

If segment is not a VideoSegment with finite times where 0 <= start_time < end_time.

MediaOperationFailedError

If clipping fails with an error outside the taxonomy.

output_validator(attachment)

Validate that each output attachment carries VideoSegment metadata with start and end times.

Parameters:

Name Type Description Default
attachment Attachment | list[Attachment]

The attachment(s) to validate.

required

Returns:

Type Description
Attachment | list[Attachment]

Attachment | list[Attachment]: The validated attachment(s).

Raises:

Type Description
MediaResultOperationError

If an output is not an Attachment, its metadata is not a dictionary, cannot be parsed as [VideoSegment][gllm_core.schema.multimodal.video_caption.VideoSegment], or has no start_time or end_time.

process(attachment, **kwargs) async

Plan segments and materialize each one, converting failures outside the taxonomy.

Segmenters take no per-call options: segment settings belong in the config. process_config=None is accepted for compatibility with the media-toolkit contract.

Parameters:

Name Type Description Default
attachment Attachment

The video attachment to segment.

required
**kwargs Any

Per-call options. Only process_config=None is accepted.

{}

Returns:

Type Description
list[Attachment]

list[Attachment]: Materialized segment attachments.

Raises:

Type Description
MediaOperationConfigError

If a per-call option is passed, or a planned segment is invalid.

MediaInputInvalidError

If attachment is not an Attachment or its MIME type is malformed.

MediaInputUnsupportedError

If the attachment is audio or not a video.

MediaResultOperationError

If a segment's metadata is missing or invalid.

MediaOperationFailedError

If segmenting fails with an error outside the taxonomy.

segment(attachment) async

Compute segment boundaries for one attachment without materializing media.

Guarantees mimetype validation before calling _segment.

Parameters:

Name Type Description Default
attachment Attachment

The attachment to segment.

required

Returns:

Type Description
list[VideoSegment]

list[VideoSegment]: Segment plans containing at least start_time and end_time.

Raises:

Type Description
MediaInputInvalidError

If attachment is not an Attachment or its MIME type is malformed.

MediaInputUnsupportedError

If the attachment is neither video nor audio.

MediaOperationFailedError

If planning fails with an error outside the taxonomy.

BaseSegmenterConfig(**data)

Bases: BaseModel

Empty base config used to carry values into a concrete segmenter config.

Concrete segmenter configs declare and validate only the settings their implementation consumes. Extra fields are retained here so callers can up-cast an exact base config with from_dict. A validation failure of any segmenter config is raised as ComponentConfigError naming component_name.

Validate the options and report a failure as a component configuration error.

Parameters:

Name Type Description Default
**data Any

The configuration options.

{}

Raises:

Type Description
ComponentConfigError

If an option is unknown or has an invalid value.

from_dict(data=None) classmethod

Build a validated config from a dict, base config, or existing instance.

Sibling subclass configs are rejected: re-validating an unrelated model would silently drop (or carry over) fields the caller never set. Pass a dict or an exact BaseSegmenterConfig to up-cast into the concrete subclass.

Parameters:

Name Type Description Default
data Self | BaseSegmenterConfig | dict[str, Any] | None

Raw config payload. None uses field defaults. Defaults to None.

None

Returns:

Name Type Description
Self Self

Validated config of the concrete subclass.

Raises:

Type Description
ComponentConfigError

If an option is unknown or has an invalid value, or data is a BaseModel that is neither an instance of cls nor an exact BaseSegmenterConfig, or is of any other unsupported type.

model_validate(obj, **kwargs) classmethod

Validate an object and report a failure as a component configuration error.

Parameters:

Name Type Description Default
obj Any

The object to validate.

required
**kwargs Any

Options forwarded to pydantic's model_validate.

{}

Returns:

Name Type Description
Self Self

The validated config.

Raises:

Type Description
ComponentConfigError

If an option is unknown or has an invalid value.

model_validate_json(json_data, **kwargs) classmethod

Validate a JSON payload and report a failure as a component configuration error.

Parameters:

Name Type Description Default
json_data str | bytes | bytearray

The JSON payload to validate.

required
**kwargs Any

Options forwarded to pydantic's model_validate_json.

{}

Returns:

Name Type Description
Self Self

The validated config.

Raises:

Type Description
ComponentConfigError

If the payload is malformed, or an option is unknown or has an invalid value.