> ## Documentation Index
> Fetch the complete documentation index at: https://daily-hush-pcc-session-recordings-docs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Baseten

> LLM service implementation using Baseten's OpenAI-compatible API

## Overview

`BasetenLLMService` provides access to Baseten's Model APIs and dedicated deployments through an OpenAI-compatible interface. It inherits from `OpenAILLMService` and supports streaming responses, function calling, and context management.

Baseten's Model APIs offer serverless access to open-weights models including GLM, Kimi, DeepSeek, Nemotron, and gpt-oss. For dedicated deployments running on your own GPUs, point `base_url` to your deployment's endpoint.

<CardGroup cols={2}>
  <Card title="Baseten LLM API Reference" icon="code" href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.baseten.llm.html">
    Pipecat's API methods for Baseten integration
  </Card>

  <Card title="Example Implementation" icon="play" href="https://github.com/pipecat-ai/pipecat/blob/main/examples/function-calling/function-calling-baseten.py">
    Complete example with function calling
  </Card>

  <Card title="Baseten Documentation" icon="book" href="https://docs.baseten.co/">
    Official Baseten API documentation
  </Card>

  <Card title="Baseten Console" icon="microphone" href="https://app.baseten.co/">
    Access models and manage API keys
  </Card>
</CardGroup>

## Installation

To use Baseten services, install the required dependency:

```bash theme={null}
uv add "pipecat-ai[baseten]"
```

## Prerequisites

### Baseten Account Setup

Before using Baseten LLM services, you need:

1. **Baseten Account**: Sign up at [Baseten](https://www.baseten.co/)
2. **API Key**: Generate an API key from your account dashboard
3. **Model Selection**: Choose from available Model APIs or deploy your own model

### Required Environment Variables

* `BASETEN_API_KEY`: Your Baseten API key for authentication

## Configuration

<ParamField path="api_key" type="str" required>
  Baseten API key for authentication.
</ParamField>

<ParamField path="base_url" type="str" default="https://inference.baseten.co/v1">
  Base URL for Baseten API endpoint. Use the default for serverless Model APIs,
  or set to your dedicated deployment's `/sync/v1` URL to use a model running on
  your own GPUs.
</ParamField>

<ParamField path="settings" type="BasetenLLMService.Settings" default="None">
  Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `BasetenLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

This service uses the same settings as `OpenAILLMService`. See [OpenAI LLM Settings](/api-reference/server/services/llm/openai#settings) for the full parameter reference.

The default model is `moonshotai/Kimi-K2.5`, which streams visible content immediately for low time-to-first-word.

## Usage

### Basic Setup

```python theme={null}
import os
from pipecat.services.baseten import BasetenLLMService

llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    settings=BasetenLLMService.Settings(
        model="moonshotai/Kimi-K2.5",
    ),
)
```

### With Custom Settings

```python theme={null}
from pipecat.services.baseten import BasetenLLMService

llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    settings=BasetenLLMService.Settings(
        model="moonshotai/Kimi-K2.5",
        temperature=0.7,
        max_completion_tokens=1024,
        system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
    ),
)
```

### Using a Dedicated Deployment

To use a model running on your own dedicated GPUs, point `base_url` to your deployment's endpoint:

```python theme={null}
llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
    settings=BasetenLLMService.Settings(
        model="Qwen/Qwen2.5-3B-Instruct",
    ),
)
```

## Notes

* **Model APIs vs. Dedicated Deployments**: The default `base_url` connects to Baseten's serverless Model APIs. For dedicated deployments on your own GPUs, set `base_url` to your deployment's `/sync/v1` endpoint URL.
* **Token Usage Reporting**: Baseten reports cumulative token usage on every streamed chunk. The service automatically accumulates these values and reports the final total once at the end of processing.
* **Message Roles**: Baseten supports `system`, `user`, `assistant`, and `tool` message roles. The `developer` role is not supported.
* **OpenAI Compatibility**: Baseten fully supports the OpenAI-compatible parameter set inherited from `OpenAILLMService`, including function calling and streaming responses.

<Tip>
  The `InputParams` / `params=` pattern is deprecated as of v0.0.105. Use
  `Settings` / `settings=` instead. See the [Service Settings
  guide](/pipecat/fundamentals/service-settings) for migration details.
</Tip>