Baseten

​

Overview

​

Installation

uv add "pipecat-ai[baseten]"

​

Prerequisites

​

Baseten Account Setup

1. Baseten Account: Sign up at Baseten
2. API Key: Generate an API key from your account dashboard
3. Model Selection: Choose from available Model APIs or deploy your own model

​

Required Environment Variables

• BASETEN_API_KEY: Your Baseten API key for authentication

​

Configuration

​

Settings

​

Usage

​

Basic Setup

import os
from pipecat.services.baseten import BasetenLLMService

llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    settings=BasetenLLMService.Settings(
        model="moonshotai/Kimi-K2.5",
    ),
)

​

With Custom Settings

from pipecat.services.baseten import BasetenLLMService

llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    settings=BasetenLLMService.Settings(
        model="moonshotai/Kimi-K2.5",
        temperature=0.7,
        max_completion_tokens=1024,
        system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
    ),
)

​

Using a Dedicated Deployment

llm = BasetenLLMService(
    api_key=os.getenv("BASETEN_API_KEY"),
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
    settings=BasetenLLMService.Settings(
        model="Qwen/Qwen2.5-3B-Instruct",
    ),
)

​

Notes

• Model APIs vs. Dedicated Deployments: The default base_url connects to Baseten’s serverless Model APIs. For dedicated deployments on your own GPUs, set base_url to your deployment’s /sync/v1 endpoint URL.
• Token Usage Reporting: Baseten reports cumulative token usage on every streamed chunk. The service automatically accumulates these values and reports the final total once at the end of processing.
• Message Roles: Baseten supports system, user, assistant, and tool message roles. The developer role is not supported.
• OpenAI Compatibility: Baseten fully supports the OpenAI-compatible parameter set inherited from OpenAILLMService, including function calling and streaming responses.