Overview

Unified schema exports for gllm_multimodal.

This module re-exports all data models and result schemas used by modality converters and transformers. Importing from this module provides a convenient single entry point for all schema types.

Core multimodal types such as AudioTranscript, Keyframe, and VideoSegment live in [gllm_core.schema.multimodal][] and should be imported from there directly.

Exported schemas

Schema	Description
`TextResult`	Base result for text extraction operations.
`CaptionResult`	Result of a captioning operation.
`Caption`	Input schema for captioning operations.
`OcrResult`	Structured result of an OCR operation.
`DetectedObject`	A detected region located by a pixel polygon.
`Mermaid`	Additional metadata for mermaid generation.
`ModelWeight`	Immutable model artifact metadata.

Usage

from gllm_multimodal.schema import TextResult, CaptionResult, OcrResult

`Caption`

Bases: BaseModel

Result class for captioning operations (image, video, etc.).

This class provides a structured format for captioning results, supporting: - Multiple caption types (one-liner, detailed, domain-specific) - Caption count tracking - Metadata storage for processing details

Attributes:

Name	Type	Description
`text_one_liner`	`str`	Brief, single-sentence summary of the content. Defaults to empty string if not provided.
`text_context`	`str`	Detailed, multi-sentence description of the content. Defaults to empty string if not provided.
`domain_knowledge`	`str`	Domain-specific interpretation or context. Defaults to empty string if not provided.
`number_of_captions`	`int`	Total number of distinct captions generated. Defaults to 0 if no captions are generated.
`media_metadata`	`dict[str, Any]`	Additional information about the media such as location.
`multimodal_context`	`list[Attachment \| str]`	Optional list of external context objects (files, bytes, or pre-processed inputs) or raw strings that can enrich captioning results. Bytes are automatically converted into Attachment objects via `Attachment.from_bytes`.
`output_schema`	`str`	Output schema. Defaults to empty string if not provided.
`schema_description`	`str`	Schema description. Defaults to empty string if not provided.
`language`	`str`	Language of the captions. Defaults to "Indonesian" if not provided.

`handle_multimodal_context(multimodal_value)` `classmethod`

Normalize and validate multimodal_context.

This method ensures that the multimodal_context field is a list of Attachment objects or strings. It handles multiple input cases:

None -> returns an empty list
list[bytes] -> converts each item into an Attachment via Attachment.from_bytes
list[Attachment] -> keeps as-is
list[str] -> keeps as-is if it's not a valid image/binary source, otherwise converts to Attachment.
list[mixed] -> normalizes supported types

Parameters:

Name	Type	Description	Default
`multimodal_value`	`Any`	Input value provided to `multimodal_context`.	required

Returns:

Type	Description
`Any`	list[Attachment \| str]: A normalized list of `Attachment` objects or strings.

`handle_none_metadata(metadata_value)` `classmethod`

Handle None values for media_metadata by using empty dict.

`handle_none_number_of_captions(caption_value)` `classmethod`

Handle None values for number_of_captions by using default.

`handle_none_values(str_value)` `classmethod`

Handle None values by converting them to default values.

`CaptionResult`

Bases: Caption

Result of a captioning operation.

Attributes:

Name	Type	Description
`captions`	`str \| list[str] \| dict[str, Any]`	The caption result. May be a single string, a list of captions, or a structured dictionary depending on the output format.

`DetectedObject`

Bases: BaseModel

A region detected in an image, located by a polygon in pixel coordinates.

The polygon is the canonical geometry, so rotated or perspective-skewed regions keep their shape. Coordinates are pixels of the source image, whose size is carried by the producing result (such as OcrResult.image_width and OcrResult.image_height), with the origin at the top-left corner. Use from_bounding_box to build an object from an axis-aligned box and to_bounding_box to obtain the axis-aligned envelope.

Attributes:

Name	Type	Description
`class_name`	`str`	The kind of detected region, such as the values in `DetectedObjectClass`.
`polygon`	`list[Point2D]`	The region outline as `(x, y)` points in pixels, each coordinate at least 0.0, with at least three points. Four-point polygons are ordered top-left, top-right, bottom-right, bottom-left when built with `from_bounding_box`.
`confidence`	`float \| None`	Confidence score of the detection, between 0.0 and 1.0. None when the producer does not score the region. Defaults to None.
`content`	`str \| None`	The textual content of the region, such as the transcribed line text or a table rendered as HTML. Defaults to None.
`metadata`	`dict[str, Any]`	Producer-specific attributes, such as the keys in `DetectedObjectMetadataKey`. Defaults to an empty dictionary.

`from_bounding_box(bounding_box, class_name, content=None, confidence=None, metadata=None)` `classmethod`

Creates a detected object from an axis-aligned box in pixels.

Parameters:

Name	Type	Description	Default
`bounding_box`	`XYXY`	The `(x_min, y_min, x_max, y_max)` edges in pixels.	required
`class_name`	`str`	The kind of detected region.	required
`content`	`str \| None`	The textual content of the region. Defaults to None.	`None`
`confidence`	`float \| None`	Confidence score of the detection. Defaults to None.	`None`
`metadata`	`dict[str, Any] \| None`	Producer-specific attributes. Defaults to None.	`None`

Returns:

Name	Type	Description
`DetectedObject`	`DetectedObject`	The detected object whose polygon is the box corners ordered top-left, top-right, bottom-right, bottom-left.

Raises:

Type	Description
`ValidationError`	If an edge is negative or `confidence` is out of range.

`to_bounding_box()`

Returns the axis-aligned envelope of the polygon.

Returns:

Name	Type	Description
`XYXY`	`XYXY`	The `(x_min, y_min, x_max, y_max)` edges in pixels.

`Mermaid`

Bases: BaseModel

Additional metadata for mermaid diagram generation.

Attributes:

Name	Type	Description
`diagram_type`	`str \| None`	Type of the diagram to be generated (e.g. `"flowchart"`, `"sequence"`). Defaults to `None`.
`context`	`str \| None`	Additional context or prompt used to generate the mermaid diagram. Defaults to `None`.

`ModelWeight`

Bases: BaseModel

Describe immutable model metadata accepted by the weight resolver.

`validate_integrity_metadata()`

Reject incomplete or unsafe artifact metadata.

Returns:

Name	Type	Description
`ModelWeight`	`'ModelWeight'`	This validated immutable specification.

Raises:

Type	Description
`ValueError`	If the filename, digest, URL, or size limit is invalid.

`OcrResult`

Bases: BaseModel

Structured result of an OCR operation.

Attributes:

Name	Type	Description
`text`	`str`	Full concatenated text extracted from the document. This value is mirrored in TextResult.result for API consistency.
`lines`	`list[str]`	Engine-native line units when available (e.g. from specialized OCR backends). LM-based implementations populate this only when bounding boxes are requested, with one entry per text region; otherwise they leave it empty. Defaults to an empty list.
`page_count`	`int`	Number of pages processed. Populated by engines that support multi-page documents (e.g., Azure Document Intelligence). Defaults to 1.
`detected_objects`	`list[DetectedObject]`	Localized regions, such as text lines and tables, populated by engines that return layout information. Text lines come first in engine reading order, followed by tables. Filter by `DetectedObject.class_name` to select a region kind. Defaults to an empty list.
`image_width`	`int \| None`	Width in pixels of the source image that the detected object polygons are measured against. Use it with `image_height` to rescale the polygons for a resized image. None when no detected objects are returned. Defaults to None.
`image_height`	`int \| None`	Height in pixels of the source image that the detected object polygons are measured against. None when no detected objects are returned. Defaults to None.

`TextResult`

Bases: BaseModel

Base class for all modality-to-text operation results.

This class provides the foundation for structured results from any modality conversion operation, including: - Image Captioning - OCR / Scene Text Detection - Audio Transcription - Video Captioning

Attributes:

Name	Type	Description
`result`	`str`	The extracted or generated text from the source. This is the primary output of any modality conversion operation. May be empty if the operation fails or no text is found.
`tag`	`str`	A label identifying the type of conversion that produced this result (e.g. `"captions"`, `"ocr"`).
`metadata`	`dict[str, Any] \| BaseModel \| None`	Additional metadata from the conversion process.

Overview

Exported schemas

Usage

Caption

handle_multimodal_context(multimodal_value) classmethod

handle_none_metadata(metadata_value) classmethod

handle_none_number_of_captions(caption_value) classmethod

handle_none_values(str_value) classmethod

CaptionResult

DetectedObject

from_bounding_box(bounding_box, class_name, content=None, confidence=None, metadata=None) classmethod

to_bounding_box()

Mermaid

ModelWeight

validate_integrity_metadata()

OcrResult

TextResult

`Caption`

`handle_multimodal_context(multimodal_value)` `classmethod`

`handle_none_metadata(metadata_value)` `classmethod`

`handle_none_number_of_captions(caption_value)` `classmethod`

`handle_none_values(str_value)` `classmethod`

`CaptionResult`

`DetectedObject`

`from_bounding_box(bounding_box, class_name, content=None, confidence=None, metadata=None)` `classmethod`

`to_bounding_box()`

`Mermaid`

`ModelWeight`

`validate_integrity_metadata()`

`OcrResult`

`TextResult`