Realtime Session

[BETA] Modules concerning the realtime session modules to interact with realtime session models.

`GoogleRealtimeSession(model_name, api_key=None, credentials_path=None, credentials_info=None, project_id=None, location='us-central1', tools=None, config=None, retry_config=None)`

Bases: BaseRealtimeSession

[BETA] A realtime session module to interact with Gemini Live models.

Warning

The 'GoogleRealtimeSession' class is currently in beta and may be subject to changes in the future. It is intended only for quick prototyping in local environments. Please avoid using it in production environments.

Attributes:

Name	Type	Description
`model_name`	`str`	The name of the language model.
`client_params`	`dict[str, Any]`	The Google client instance init parameters.
`config`	`dict[str, Any]`	The configuration for the realtime session.
`retry_config`	`RetryConfig`	The retry configuration for the realtime session.
`tool_dict`	`dict[str, Tool]`	A dictionary of tools provided to the model.

Authentication

The GoogleRealtimeSession can use either Google Gen AI or Google Vertex AI.

Google Gen AI is recommended for quick prototyping and development. It requires a Gemini API key for authentication.

Usage example:

realtime_session = GoogleRealtimeSession(
    model_name="gemini-2.5-flash-native-audio-preview-12-2025",
    api_key="your_api_key"
)

Google Vertex AI is recommended to build production-ready applications. It requires service account credentials for authentication.

Usage example:

realtime_session = GoogleRealtimeSession(
    model_name="gemini-2.5-flash-native-audio-preview-12-2025",
    credentials_path="path/to/service_account.json"
)

The service account JSON contents may also be passed directly through credentials_info.

If neither api_key, credentials_path, nor credentials_info is provided, Google Gen AI will be used by default. The GOOGLE_API_KEY environment variable will be used for authentication.

Examples:

Basic usage: The GoogleRealtimeSession can be used as started as follows:

realtime_session = GoogleRealtimeSession(model_name="gemini-live-2.5-flash-preview")
await realtime_session.invoke()

Tool calling: The GoogleRealtimeSession can call provided tools to perform certain tasks. This feature can be enabled by providing a list of Tool objects to the tools parameter.

Usage example:
```python
tools = [get_weather, get_temperature]
realtime_session = GoogleRealtimeSession(model_name="gemini-live-2.5-flash-preview", tools=tools)
await realtime_session.start()
```

Custom IO streamers: The GoogleRealtimeSession can be used with custom IO streamers.

input_streamers = [KeyboardInputStreamer(), LinuxMicInputStreamer()]
output_streamers = [ConsoleOutputStreamer(), LinuxSpeakerOutputStreamer()]
realtime_session = GoogleRealtimeSession(model_name="gemini-live-2.5-flash-preview")
await realtime_session.start(input_streamers=input_streamers, output_streamers=output_streamers)

In the above example, we added a capability to use a Linux system microphone and speaker,
allowing realtime audio input and output to the model.

Initializes a new instance of the GoogleRealtimeSession class.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	The name of the model to use.	required
`api_key`	`str \| None`	Required for Google Gen AI authentication. Cannot be used together with `credentials_path` or `credentials_info`. Defaults to None.	`None`
`credentials_path`	`str \| None`	For Google Vertex AI authentication. Path to the service account credentials JSON file. Cannot be used together with `api_key` or `credentials_info`. Defaults to None.	`None`
`credentials_info`	`dict[str, Any] \| None`	For Google Vertex AI authentication. Service account credentials JSON contents. Cannot be used with `api_key` or `credentials_path`. Defaults to None.	`None`
`project_id`	`str \| None`	The Google Cloud project ID for Vertex AI. Only used when authenticating with service account credentials. Defaults to None, in which case it will be loaded from the credentials.	`None`
`location`	`str`	The location of the Google Cloud project for Vertex AI. Only used when authenticating with service account credentials. Defaults to "us-central1".	`'us-central1'`
`tools`	`list[Tool] \| None`	Tools provided to the model to enable tool calling. Defaults to None.	`None`
`config`	`dict[str, Any] \| None`	Additional configuration for the realtime session. Defaults to None.	`None`
`retry_config`	`RetryConfig \| None`	The retry configuration for the realtime session. Defaults to None, in which case a default config with no retry and 60 minutes timeout will be used.	`None`

Note

If neither api_key, credentials_path, nor credentials_info is provided, Google Gen AI will be used by default. The GOOGLE_API_KEY environment variable will be used for authentication.

`LiveKitRealtimeSession(url, api_key=None, api_secret=None, room_name=None, identity=None, retry_config=None)`

Bases: BaseRealtimeSession

[BETA] A realtime session module to interact with LiveKit services.

Warning

The 'LiveKitRealtimeSession' class is currently in beta and may be subject to changes in the future. It is intended only for quick prototyping in local environments. Please avoid using it in production environments.

Attributes:

Name	Type	Description
`url`	`str`	The URL of the LiveKit server.
`retry_config`	`RetryConfig`	The retry configuration for the realtime session.

Examples:

Basic usage: The LiveKitRealtimeSession can be used as started as follows:

realtime_session = LiveKitRealtimeSession(url="ws://127.0.0.1:7880", token="...")
await realtime_session.start()

Custom IO streamers: The LiveKitRealtimeSession can be used with custom IO streamers.

input_streamers = [KeyboardInputStreamer(), LinuxMicInputStreamer()]
output_streamers = [ConsoleOutputStreamer(), LinuxSpeakerOutputStreamer()]
realtime_session = LiveKitRealtimeSession(url="ws://127.0.0.1:7880", token="...")
await realtime_session.start(input_streamers=input_streamers, output_streamers=output_streamers)

In the above example, we added a capability to use a Linux system microphone and speaker,
allowing realtime audio input and output to the model.

Initializes a new instance of the LiveKitRealtimeSession class.

Parameters:

Name	Type	Description	Default
`url`	`str`	The URL of the LiveKit server.	required
`api_key`	`str \| None`	The API key for authentication. Defaults to None, in which case the `LIVEKIT_API_KEY` environment variable will be used.	`None`
`api_secret`	`str \| None`	The API secret for authentication. Defaults to None, in which case the `LIVEKIT_API_SECRET` environment variable will be used.	`None`
`room_name`	`str \| None`	The name of the room. Defaults to None, in which case a random UUID will be used.	`None`
`identity`	`str \| None`	The identity of the participant. Defaults to None, in which case a random UUID will be used.	`None`
`retry_config`	`RetryConfig \| None`	The retry configuration for the realtime session. Defaults to None, in which case a default config with no retry and 60 minutes timeout will be used.	`None`

`OpenAIRealtimeSession(model_name, api_key=None, config=None, retry_config=None)`

Bases: BaseRealtimeSession

[BETA] A realtime session module to interact with OpenAI Realtime API.

Warning

The 'OpenAIRealtimeSession' class is currently in beta and may be subject to changes in the future. It is intended only for quick prototyping in local environments. Please avoid using it in production environments.

Attributes:

Name	Type	Description
`model_name`	`str`	The model name to use for realtime inference.
`api_key`	`str`	OpenAI API key used for authentication.
`config`	`dict[str, Any]`	Session configuration passed in `session.update`.
`retry_config`	`RetryConfig`	Retry configuration for the realtime session.

Examples:

Basic usage: The OpenAIRealtimeSession can be used as started as follows:

realtime_session = OpenAIRealtimeSession(model_name="gpt-realtime-2")
await realtime_session.invoke()

Custom IO streamers: The OpenAIRealtimeSession can be used with custom IO streamers.

input_streamers = [KeyboardInputStreamer(), LinuxMicInputStreamer()]
output_streamers = [ConsoleOutputStreamer(), LinuxSpeakerOutputStreamer()]
realtime_session = OpenAIRealtimeSession(model_name="gpt-realtime-2")
await realtime_session.start(input_streamers=input_streamers, output_streamers=output_streamers)

In the above example, we added a capability to use a Linux system microphone and speaker,
allowing realtime audio input and output to the model.

Initializes a new instance of the OpenAIRealtimeSession class.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	The name of the model to use.	required
`api_key`	`str \| None`	The API key to use.	`None`
`config`	`dict[str, Any] \| None`	The configuration to use.	`None`
`retry_config`	`RetryConfig \| None`	The retry configuration to use.	`None`

`PipecatRealtimeSession(websocket_url, retry_config=None)`

Bases: BaseRealtimeSession

[BETA] A realtime session module to interact with Pipecat agents.

Warning

The 'PipecatRealtimeSession' class is currently in beta and may be subject to changes in the future. It is intended only for quick prototyping in local environments. Please avoid using it in production environments.

Attributes:

Name	Type	Description
`websocket_url`	`str`	The WebSocket URL of the Pipecat agent.
`retry_config`	`RetryConfig`	The retry configuration for the realtime session.

Examples:

Basic usage: The PipecatRealtimeSession can be used as started as follows:

realtime_session = PipecatRealtimeSession(websocket_url="ws://127.0.0.1:8765/ws")
await realtime_session.start()

Custom IO streamers: The PipecatRealtimeSession can be used with custom IO streamers.

input_streamers = [KeyboardInputStreamer(), LinuxMicInputStreamer()]
output_streamers = [ConsoleOutputStreamer(), LinuxSpeakerOutputStreamer()]
realtime_session = PipecatRealtimeSession(websocket_url="ws://127.0.0.1:8765/ws")
await realtime_session.start(input_streamers=input_streamers, output_streamers=output_streamers)

In the above example, we added a capability to use a Linux system microphone and speaker,
allowing realtime audio input and output to the Pipecat agent.

Initializes a new instance of the PipecatRealtimeSession class.

Parameters:

Name	Type	Description	Default
`websocket_url`	`str`	The WebSocket URL of the Pipecat agent.	required
`retry_config`	`RetryConfig \| None`	The retry configuration for the realtime session. Defaults to None, in which case a default config with no retry and 60 minutes timeout will be used.	`None`

Raises:

Type	Description
`ImportError`	If the `websockets` or `pydub` package is not installed.

`XAIRealtimeSession(model_name, api_key=None, tools=None, config=None, retry_config=None)`

Bases: BaseRealtimeSession

[BETA] A realtime session module to interact with xAI Grok Voice models.

Warning

The 'XAIRealtimeSession' class is currently in beta and may be subject to changes in the future. It is intended only for quick prototyping in local environments. Please avoid using it in production environments.

Attributes:

Name	Type	Description
`model_name`	`str`	The name of the Grok voice model.
`api_key`	`str`	The xAI API key for authentication.
`config`	`dict[str, Any]`	The session configuration.
`retry_config`	`RetryConfig`	The retry configuration.
`tool_dict`	`dict[str, Tool]`	The dictionary of tools provided to the model.

Initializes a new instance of the XAIRealtimeSession class.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	The name of the model to use (e.g., "grok-voice-think-fast-1.0").	required
`api_key`	`str \| None`	The xAI API key for authentication. Falls back to XAI_API_KEY env var. Defaults to None.	`None`
`tools`	`list[Tool] \| None`	Tools provided to the model. Defaults to None.	`None`
`config`	`dict[str, Any] \| None`	Additional session configuration. Merged with defaults. Defaults to None.	`None`
`retry_config`	`RetryConfig \| None`	The retry configuration. Defaults to None.	`None`

Raises:

Type	Description
`ValueError`	If `api_key` is not provided and XAI_API_KEY env var is not set.
`ImportError`	If required optional packages are missing.