Overview

Image-to-caption modules for generating descriptive captions from images.

This package provides concrete implementations for captioning images using language models or preset configurations.

Submodules

• output_formatter -- Caption output formatting (free-text and structured).

Exported Classes

• LMBasedImageToCaption -- LM-based image captioning.
• ImageCaptionPreset -- Preset-based image captioning.

ImageCaptionPreset

Bases: StrEnum

Available presets for image captioning.

Attributes:

Name	Type	Description
DEFAULT		Standard image captioning.
STRUCTURED		Structured data extraction from images.
DJARUM_ENG_SVLM		Specialized maintenance document analysis.
KEYFRAME_CAPTIONING		Video keyframe captioning with chronological context.

LMBasedImageToCaption(lm_request_processor, formatter=DEFAULT_FORMATTER)

Bases: BaseImageToCaption, UsesLM

Image captioning implementation using Language Models.

This class implements the ImageToCaption interface using LMs for generating natural language captions.

Usage

Use from_preset to easily instantiate with predefined configuration:

from gllm_multimodal.modality_converter.image_to_text.image_to_caption.lm_based_image_to_caption import LMBasedImageToCaption

converter = LMBasedImageToCaption.from_preset()

Initialize the LM based image captioning component.

Parameters:

Name	Type	Description	Default
lm_request_processor	LMRequestProcessor	Language model request processor instance that supports multimodal inputs.	required
formatter	BaseCaptionOutputFormatter	The formatter to use for formatting the output.	DEFAULT_FORMATTER

from_preset(preset_name=ImageCaptionPreset.DEFAULT, lm_invoker_kwargs=None, prompt_builder_kwargs=None, **kwargs) classmethod

Initialize the LM based image captioning component using preset model configurations.

Parameters:

Name	Type	Description	Default
preset_name	str	Name of the preset to use.	DEFAULT
lm_invoker_kwargs	dict | None	Keyword arguments to pass to the LM invoker.	None
prompt_builder_kwargs	dict | None	Keyword arguments to pass to the prompt builder.	None
**kwargs	Any	Additional keyword arguments to pass to from_lm_components().	{}

Returns:

Name	Type	Description
LMBasedImageToCaption	LMBasedImageToCaption	Initialized image captioning component using preset model.