Schema

Backward-compatible modality converter schema exports.

AudioTranscript

Bases: BaseModel

A timed transcript of spoken audio.

Attributes:

Name	Type	Description
text	str	The transcribed text.
start_time	float	The transcript start time in seconds.
end_time	float	The transcript end time in seconds.
lang_id	str | None	The transcript language identifier. Defaults to None.

Caption

Bases: BaseModel

Result class for captioning operations (image, video, etc.).

This class provides a structured format for captioning results, supporting: - Multiple caption types (one-liner, detailed, domain-specific) - Caption count tracking - Metadata storage for processing details

Attributes:

Name	Type	Description
text_one_liner	str	Brief, single-sentence summary of the content. Defaults to empty string if not provided.
text_context	str	Detailed, multi-sentence description of the content. Defaults to empty string if not provided.
domain_knowledge	str	Domain-specific interpretation or context. Defaults to empty string if not provided.
number_of_captions	int	Total number of distinct captions generated. Defaults to 0 if no captions are generated.
media_metadata	dict[str, Any]	Additional information about the media such as location.
multimodal_context	list[Attachment | str]	Optional list of external context objects (files, bytes, or pre-processed inputs) or raw strings that can enrich captioning results. Bytes are automatically converted into Attachment objects via Attachment.from_bytes.
output_schema	str	Output schema. Defaults to empty string if not provided.
schema_description	str	Schema description. Defaults to empty string if not provided.
language	str	Language of the captions. Defaults to "Indonesian" if not provided.

Deprecated Fields

• image_one_liner: Use text_one_liner instead. Will be removed in 0.4.0.
• image_description: Use text_context instead. Will be removed in 0.4.0.
• image_metadata: Use media_metadata instead. Will be removed in 0.4.0.

handle_deprecated_fields(values) classmethod

Map deprecated field names to their replacements and emit warnings.

Deprecated Fields

• image_one_liner: Use text_one_liner instead. Will be removed in 0.4.0.
• image_description: Use text_context instead. Will be removed in 0.4.0.
• image_metadata: Use media_metadata instead. Will be removed in 0.4.0.
• attachments_context: Use multimodal_context instead. Will be removed in 0.4.0.

handle_multimodal_context(multimodal_value) classmethod

Normalize and validate multimodal_context.

This method ensures that the multimodal_context field is a list of Attachment objects or strings. It handles multiple input cases:

• None -> returns an empty list
• list[bytes] -> converts each item into an Attachment via Attachment.from_bytes
• list[Attachment] -> keeps as-is
• list[str] -> keeps as-is if it's not a valid image/binary source, otherwise converts to Attachment.
• list[mixed] -> normalizes supported types

Parameters:

Name	Type	Description	Default
multimodal_value	Any	Input value provided to multimodal_context.	required

Returns:

Type	Description
Any	list[Attachment | str]: A normalized list of Attachment objects or strings.

handle_none_metadata(metadata_value) classmethod

Handle None values for media_metadata by using empty dict.

handle_none_number_of_captions(caption_value) classmethod

Handle None values for number_of_captions by using default.

handle_none_values(str_value) classmethod

Handle None values by converting them to default values.

Keyframe

Bases: BaseModel

A keyframe extracted from a video segment.

Attributes:

Name	Type	Description
time_offset	float	Time within the segment where the keyframe occurs.
caption	str | None	Text description of the keyframe.

TextResult

Bases: BaseModel

Base class for all modality-to-text operation results.

This class provides the foundation for structured results from any modality conversion operation, including: - Image Captioning - OCR / Scene Text Detection - Audio Transcription - Video Captioning

Attributes:

Name	Type	Description
result	str	The extracted or generated text from the source. This is the primary output of any modality conversion operation. May be empty if the operation fails or no text is found.
tag	str	A label identifying the type of conversion that produced this result (e.g. "captions", "ocr").
metadata	dict[str, Any] | BaseModel | None	Additional metadata from the conversion process.

VideoCaptionMetadata

Bases: BaseModel

Caption metadata for an entire video.

Attributes:

Name	Type	Description
video_summary	str	A high-level video summary.
segments	list[VideoSegment]	Captioned video segments.

ensure_segment_end_time_greater_than_start_time()

Normalize segments whose end time is not after their start time.

Returns:

Name	Type	Description
VideoCaptionMetadata	VideoCaptionMetadata	The video caption metadata with normalized segment end times.