Modality Transformer Builder

Defines a convenience function to build a modality transformer.

`build_modality_transformer(source_modality=Modality.IMAGE, target_modality=Modality.TEXT, transformer_type=ModalityTransformerType.STANDARD, router_config=None, converter_config=None, **kwargs)`

Build and initialize a modality transformer instance for a given configuration.

The factory looks up the converter class based on the combination of

source_modality: input modality (e.g., Modality.IMAGE, Modality.AUDIO)
target_modality: output modality (e.g., Modality.TEXT)
transformer_type: transformer type (e.g., ModalityTransformerType.STANDARD)

The factory then delegates construction to each class's from_config() classmethod.

Supported combinations of (source_modality, target_modality, transformer_type):

(Modality.IMAGE, Modality.TEXT, ModalityTransformerType.STANDARD): Builds a StandardImageModalityTransformer.
(Modality.IMAGE, Modality.TEXT, ModalityTransformerType.GENERIC): Builds a GenericImageModalityTransformer.

Parameters:

Name	Type	Description	Default
`source_modality`	`Modality`	The source modality. Defaults to Modality.IMAGE.	`IMAGE`
`target_modality`	`Modality`	The output modality. Defaults to Modality.TEXT.	`TEXT`
`transformer_type`	`ModalityTransformerType`	The transformer type. Defaults to ModalityTransformerType.STANDARD.	`STANDARD`
`router_config`	`RouterConfig \| None`	Additional router configuration. Defaults to None.	`None`
`converter_config`	`dict[str, ConverterConfig] \| None`	Additional converter configuration. Defaults to None.	`None`
`**kwargs`	`Any`	Additional keyword arguments passed to transformer.	`{}`

Returns:

Name	Type	Description
`BaseModalityTransformer`	`BaseModalityTransformer`	An instance of the matching transformer class.

Raises:

Type	Description
`ValueError`	If the configuration is invalid or not registered, including: - (source_modality, target_modality, transformer_type) not registered.

Examples:

Default routers and converters

from gllm_multimodal.constants import Modality, ModalityTransformerType
from gllm_multimodal.builder.modality_transformer_builder import build_modality_transformer

transformer = build_modality_transformer(
    source_modality=Modality.IMAGE,
    target_modality=Modality.TEXT,
    transformer_type=ModalityTransformerType.STANDARD,
)

Custom router and default converters

from gllm_multimodal.constants import Modality, ModalityTransformerType
from gllm_multimodal.modality_transformer.schema.router_config import RouterConfig
from gllm_multimodal.builder.modality_transformer_builder import build_modality_transformer

transformer = build_modality_transformer(
    source_modality=Modality.IMAGE,
    target_modality=Modality.TEXT,
    transformer_type=ModalityTransformerType.STANDARD,
    router_config=RouterConfig(
        modality=Modality.IMAGE,
        preset="multimodal",
        model_id="openai/text-embedding-3-small",
        es_url="http://localhost:9200",
        es_index_name="gllm_multimodal",
        route_mapping={"chart": "captioning", "diagram": "mermaid"},
    ),
)

Custom converters with preset

from gllm_multimodal.constants import (
    Modality, ModalityConverterTask, ModalityConverterApproach, ModalityTransformerType,
)
from gllm_multimodal.modality_transformer.schema.converter_config import ConverterConfig
from gllm_multimodal.builder.modality_transformer_builder import build_modality_transformer

transformer = build_modality_transformer(
    source_modality=Modality.IMAGE,
    target_modality=Modality.TEXT,
    transformer_type=ModalityTransformerType.STANDARD,
    converter_config={
        "captioning": ConverterConfig(
            source_modality=Modality.IMAGE,
            target_modality=Modality.TEXT,
            task_type=ModalityConverterTask.CAPTIONING,
            approach_type=ModalityConverterApproach.LM_BASED,
            preset="default",
        ),
        "mermaid": ConverterConfig(
            source_modality=Modality.IMAGE,
            target_modality=Modality.TEXT,
            task_type=ModalityConverterTask.MERMAID,
            approach_type=ModalityConverterApproach.LM_BASED,
            preset="default",
        ),
    },
)

Custom converters with custom LMRP

from gllm_multimodal.constants import (
    Modality, ModalityConverterTask, ModalityConverterApproach, ModalityTransformerType,
)
from gllm_multimodal.modality_transformer.schema.converter_config import ConverterConfig
from gllm_multimodal.builder.modality_transformer_builder import build_modality_transformer

transformer = build_modality_transformer(
    source_modality=Modality.IMAGE,
    target_modality=Modality.TEXT,
    transformer_type=ModalityTransformerType.STANDARD,
    converter_config={
        "captioning": ConverterConfig(
            source_modality=Modality.IMAGE,
            target_modality=Modality.TEXT,
            task_type=ModalityConverterTask.CAPTIONING,
            approach_type=ModalityConverterApproach.LM_BASED,
            lmrp_config={
                "model_id": "google/gemini-3-flash-preview",
                "system_template": "Describe this image...",
                "user_template": "What is in this image?",
                "output_parser_type": "json",
            },
        ),
        "mermaid": ConverterConfig(
            source_modality=Modality.IMAGE,
            target_modality=Modality.TEXT,
            task_type=ModalityConverterTask.MERMAID,
            approach_type=ModalityConverterApproach.LM_BASED,
            lmrp_config={
                "model_id": "openai/gpt-5.2-latest",
                "system_template": "Generate mermaid...",
                "user_template": "Convert to mermaid...",
            },
        ),
    },
)