Model

Document Processing Orchestrator Model Package.

Modules:

Name	Description
Element	An Element model.
ElementMetadata	An ElementMetadata model.
LoaderType	An LoaderType model containing the loader type enum.

ChunkerType

Bases: StrEnum

Chunker Type Enum.

This enum defines the different chunker types.

DownloaderType

Bases: StrEnum

Downloader Type Enum.

This enum defines input source categories used by DownloaderRouter. Values represent what the source IS, not which concrete downloader handles it. The pipeline maps each type to a concrete downloader via its own downloader_mapping.

Element

Bases: BaseModel

An Element model.

This class serves as the Element model for storing element text, structure, and metadata.

Attributes:

Name	Type	Description
text	str	The element text.
structure	str	The element structure.
metadata	dict	The element metadata.

from_chunk(chunk) staticmethod

Convert a Chunk object to an Element object.

Restores structure from the chunk's metadata (see Element.to_chunk), falling back to UNCATEGORIZED_TEXT when the key is not present.

Parameters:

Name	Type	Description	Default
chunk	Chunk	The Chunk object to convert.	required

Returns:

Name	Type	Description
Element	Element	The converted Element object.

from_dict(element) staticmethod

Convert a dictionary to an Element object.

Parameters:

Name	Type	Description	Default
element	dict[str, Any]	The dictionary to convert.	required

Returns:

Name	Type	Description
Element	Element	The Element object.

from_list_chunk(chunks) staticmethod

Convert a list of Chunk objects to a list of Element objects.

Parameters:

Name	Type	Description	Default
chunks	list[Chunk]	The list of Chunk objects to convert.	required

Returns:

Type	Description
list[Element]	list[Element]: The list of converted Element objects.

from_list_dict(elements) staticmethod

Convert a list of dictionaries to a list of Element objects.

Parameters:

Name	Type	Description	Default
elements	list[dict[str, Any]]	The list of dictionaries to convert.	required

Returns:

Type	Description
list[Element]	list[Element]: The list of Element objects.

to_chunk()

Convert this Element to a Chunk object.

The element's structure is embedded into the chunk's metadata under the STRUCTURE_METADATA_KEY key so it can be restored by Element.from_chunk.

Returns:

Name	Type	Description
Chunk	Chunk	The converted Chunk object.

to_list_chunk(elements) staticmethod

Convert a list of Element objects to a list of Chunk objects.

Parameters:

Name	Type	Description	Default
elements	list[Element]	The list of Element objects to convert.	required

Returns:

Type	Description
list[Chunk]	list[Chunk]: The list of converted Chunk objects.

to_list_dict(elements) staticmethod

Convert a list of Element objects to a list of dictionaries.

Parameters:

Name	Type	Description	Default
elements	list[Element]	The list of Element objects to convert.	required

Returns:

Type	Description
list[dict[str, Any]]	list[dict[str, Any]]: The list of dictionaries.

ElementMetadata

Bases: BaseModel

Element metadata model.

This class serves as the Element metadata model for storing element metadata.

Mandatory Attributes

source (str): The source of the element. source_type (str): The source type of the element. loaded_datetime (str): The datetime when the element is loaded.

Config

Pydantic model configuration.

This class defines the Pydantic model configuration for the ElementMetadata model.

Attributes:

Name	Type	Description
extra	str	Allow extra fields.

LoaderType

Bases: StrEnum

Loader Type Enum.

This enum defines the different loader types.

ParserType

Bases: StrEnum

Parser Type Enum.

This enum defines the different parser types.