Client manager

An explicit, caller-owned tracker for HTTP clients created within one asyncio run.

ClientManager is for a caller building several invokers together in one asyncio run (e.g. one process handling multiple models/providers). It gives that caller a single place to guarantee every client it created gets closed. Two invokers never share a client unless the caller explicitly opts both into it, by passing them the same manager-carrying ClientConfig.

ClientManager()

Explicit, caller-owned tracker for clients created within one asyncio run.

Examples:

async with ClientManager() as manager:
    client_config = ClientConfig(http_client=http_client, manager=manager)
    invoker = AnthropicLMInvoker(..., client_config=client_config)
    ...

get_or_create() also works without the async with block -- construct a ClientManager() directly and call release_resources() explicitly when done:

manager = ClientManager()
client_config = ClientConfig(http_client=http_client, manager=manager)
invoker = AnthropicLMInvoker(..., client_config=client_config)
...
await manager.release_resources()

Every client returned by get_or_create() is closed by release_resources() regardless of whether the caller keeps its ClientConfig alive -- the manager tracks each client independent of that config's lifetime.

Initializes a new, empty ClientManager.

__aenter__() async

Enters the async context manager.

Returns:

Name	Type	Description
ClientManager	ClientManager	This manager instance.

__aexit__(*exc_info) async

Closes every tracked client.

Parameters:

Name	Type	Description	Default
*exc_info	object	Exception info passed by the async with protocol (unused).	()

get_or_create(config, factory, identity, model_id_log)

Return the tracked client for config's identity if one exists, else build and track one.

Parameters:

Name	Type	Description	Default
config	ClientConfig	The caller's client configuration.	required
factory	Callable[[], ManagedClient]	Builds a fresh client from config.http_client; called only on a cache miss.	required
identity	dict[str, Any]	The calling invoker's own connection parameters (api_key, base_url, organization, headers, timeout, ...).	required
model_id_log	str	Log identifier for the calling invoker.	required

Returns:

Type	Description
ManagedClient	tuple[ManagedClient, bool]: The live client and whether it is manager-tracked
bool	(False on the closed-manager fallback path).

Raises:

Type	Description
ValueError	If config was already used with a different identity.

release_resources() async

Closes every tracked client and marks this manager closed.

This includes clients built around a caller-supplied config.http_client: opting a ClientConfig into a manager transfers close-ownership of its client to that manager, even for an injected transport.

Concurrent calls are serialized so each tracked client is closed exactly once.

ManagedClient

Bases: Protocol

Structural shape a client tracker needs: something it can close and check.

close() async

Closes the client and releases its connections.

is_closed()

Returns whether the client is closed.