Langchain Wrapper

GLLM Invoker wrapper for compatibility with the OpenEvals framework.

This module provides a bridge between the GLLM Invoker and the OpenEvals evaluation library. The OpenEvals framework requires specific client interfaces for interacting with language models, which differ from the GLLM Invoker's API.

To close this gap, this module implements two main wrapper classes: - GLLMModelClient: Implements the openevals.types.ModelClient interface. - LangChainLLMWrapper: Implements the openevals.types.ChatCompletionsClient interface with automatic fallback chain support via LMComponent.

These wrappers handle the conversion of request and response schemas between the two systems, allowing GLLM Invoker-compatible models to be seamlessly used as a backend for running evaluations with OpenEvals.

For example usage, these clients can be instantiated with a GLLM model and then passed to an OpenEvals evaluator.

Example:

from openevals import create_async_llm_as_judge llm = GLLMModelClient(model=lm_invoker) evaluator = create_async_llm_as_judge(judge=llm, ...) ...

GLLMModelClient(model, fallback_lms=None)

Bases: ModelClient

A model client that uses a GLLM Invoker.

This class implements the openevals.types.ModelClient interface, allowing a GLLM Invoker to be used within the OpenEvals framework.

Attributes:

Name	Type	Description
model	BaseLMInvoker	The GLLM Invoker to use.
chat_completion	LangChainLLMWrapper	The chat completions client.

Initialize the GLLMModelClient.

Parameters:

Name	Type	Description	Default
model	BaseLMInvoker	The model invoker to use.	required
fallback_lms	list[BaseLMInvoker] | None	Ordered fallback invokers forwarded to LangChainLLMWrapper. Defaults to None.	None

chat property

Get the chat completions client.

Returns:

Name	Type	Description
ChatCompletionsClient	ChatCompletionsClient	The chat completions client.

LangChainLLMWrapper(model, fallback_lms=None)

Bases: ChatCompletionsClient, LMComponent

A chat completions client that uses a GLLM Invoker with fallback chain support.

This class implements the openevals.types.ChatCompletionsClient interface, allowing a GLLM Invoker to be used within the OpenEvals framework with automatic fallback to alternative invokers on failure.

Attributes:

Name	Type	Description
lm_invoker	BaseLMInvoker	Primary GLLM Invoker.
model	BaseLMInvoker	Alias for lm_invoker (backward compatibility).
fallback_lms	list[BaseLMInvoker]	Ordered fallback invokers.

Initialize the LangChainLLMWrapper.

Parameters:

Name	Type	Description	Default
model	BaseLMInvoker	The primary GLLM Invoker to use.	required
fallback_lms	list[BaseLMInvoker] | None	Ordered fallback invokers. Defaults to None.	None

completions property

Get the completions client.

Returns:

Name	Type	Description
LangChainLLMWrapper	LangChainLLMWrapper	The completions client (self).

winning_invoker property

The invoker that handled the most recent successful request.

__getstate__()

Exclude non-picklable asyncio lock from serialization.

asyncio.Lock accumulates _contextvars.Context after use, making copy.deepcopy fail. We drop the lock here and recreate it fresh in __setstate__.

__setstate__(state)

Restore state and recreate a fresh lock for this wrapper.

create(messages, response_format, **kwargs) async

Create a chat completion with automatic fallback chain.

Sets the response schema on all invokers (primary and fallbacks) before invoking, then restores all schemas in the finally block.

Parameters:

Name	Type	Description	Default
messages	list[dict[str, str]]	The messages to send to the model.	required
response_format	dict[str, Any]	The response format to use.	required
**kwargs		Additional arguments to pass to the model.	{}

Returns:

Name	Type	Description
ChatCompletion	ChatCompletion	The chat completion.