Overview

Image-to-text conversion module providing OCR and captioning capabilities.

Submodules

• image_to_text -- Abstract base class for image-to-text converters.
• image_to_caption -- Image captioning implementations.
• image_to_mermaid -- Image-to-mermaid diagram generation.

LMBasedImageToCaption(lm_request_processor, formatter=DEFAULT_FORMATTER)

Bases: BaseImageToCaption, UsesLM

Image captioning implementation using Language Models.

This class implements the ImageToCaption interface using LMs for generating natural language captions.

Usage

Use from_preset to easily instantiate with predefined configuration:

from gllm_multimodal.modality_converter.image_to_text.image_to_caption.lm_based_image_to_caption import LMBasedImageToCaption

converter = LMBasedImageToCaption.from_preset()

Initialize the LM based image captioning component.

Parameters:

Name	Type	Description	Default
lm_request_processor	LMRequestProcessor	Language model request processor instance that supports multimodal inputs.	required
formatter	BaseCaptionOutputFormatter	The formatter to use for formatting the output.	DEFAULT_FORMATTER

from_preset(preset_name=ImageCaptionPreset.DEFAULT, lm_invoker_kwargs=None, prompt_builder_kwargs=None, **kwargs) classmethod

Initialize the LM based image captioning component using preset model configurations.

Parameters:

Name	Type	Description	Default
preset_name	str	Name of the preset to use.	DEFAULT
lm_invoker_kwargs	dict | None	Keyword arguments to pass to the LM invoker.	None
prompt_builder_kwargs	dict | None	Keyword arguments to pass to the prompt builder.	None
**kwargs	Any	Additional keyword arguments to pass to from_lm_components().	{}

Returns:

Name	Type	Description
LMBasedImageToCaption	LMBasedImageToCaption	Initialized image captioning component using preset model.

LMBasedImageToMermaid(lm_request_processor)

Bases: BaseImageToMermaid, UsesLM

LM-based implementation for converting an image into Mermaid diagram syntax.

This class leverages a language model (LM) pipeline to generate structured Mermaid syntax from image inputs and optional metadata. It uses prompt builders, LM invokers, and output parsers defined via a preset system to streamline model usage.

Inherits

BaseImageToMermaid: Base class defining the image-to-mermaid interface. UsesLM: Mixin providing shared logic for components using language models.

Attributes:

Name	Type	Description
lm_request_processor	LMRequestProcessor	Handles prompt creation, LM invocation, and output parsing. Core component that orchestrates the image-to-mermaid pipeline.

Initializes the LMBasedImageToMermaid instance with a language model request processor.

Parameters:

Name	Type	Description	Default
lm_request_processor	LMRequestProcessor	The processor handling prompt creation, LM invocation, and output parsing.	required

from_preset(preset_name='default', lm_invoker_kwargs=None, prompt_builder_kwargs=None, **kwargs) classmethod

Constructs an LMBasedImageToMermaid instance using a named preset configuration.

Parameters:

Name	Type	Description	Default
preset_name	str | None	Name of the predefined preset configuration to use. Defaults to "default".	'default'
lm_invoker_kwargs	dict | None	dict | None: Additional arguments for lm invoker.	None
prompt_builder_kwargs	dict | None	dict | None: Additional arguments for prompt builder.	None
**kwargs	Any	Additional keyword arguments passed to the constructor.	{}

Returns:

Name	Type	Description
LMBasedImageToMermaid	LMBasedImageToMermaid	An instance initialized with the preset's components.