# Using LiteLLM with curator

This guide demonstrates how to use LiteLLM as a backend for curator to generate synthetic data using various LLM providers. We'll walk through an example of generating synthetic recipes, but this approach can be adapted for any synthetic data generation task.

## Prerequisites

* Python 3.10+
* Curator (`pip install bespokelabs-curator`)
* Access to an LLM provider (e.g., Gemini API key)

## Steps

### 1. Create a curator.LLM Subclass

First, create a class that inherits from `curator.LLM`. You'll need to implement two key methods:

* `prompt()`: Generates the prompt for the LLM
* `parse()`: Processes the LLM's response into your desired format

```python
"""Generate synthetic recipes for different cuisines using curator."""

from datasets import Dataset

from bespokelabs import curator


class RecipeGenerator(curator.LLM):
    """A recipe generator that generates recipes for different cuisines."""

    def prompt(self, input: dict) -> str:
        """Generate a prompt using the template and cuisine."""
        return f"Generate a random {input['cuisine']} recipe. Be creative but keep it realistic."

    def parse(self, input: dict, response: str) -> dict:
        """Parse the model response along with the input to the model into the desired output format.."""
        return {
            "recipe": response,
            "cuisine": input["cuisine"],
        }
```

### 2. Set Up Your Seed Dataset

Create a dataset of inputs using the HuggingFace `Dataset` class:

```python
# List of cuisines to generate recipes for
cuisines = [
    {"cuisine": cuisine}
    for cuisine in [
        "Chinese",
        "Italian",
        "Mexican",
        "French",
        "Japanese",
        "Indian",
        "Thai",
        "Korean",
        "Vietnamese",
        "Brazilian",
    ]
]
cuisines = Dataset.from_list(cuisines)
```

### 3. Configure LiteLLM Backend

Initialise your generator with LiteLLM configuration:

```python
recipe_generator = RecipeGenerator(
    model_name="gemini/gemini-1.5-flash",  # LiteLLM model identifier
    backend="litellm",                      # Specify LiteLLM backend
    backend_params={
        "max_requests_per_minute": 2_000,   # Rate limit for requests
        "max_tokens_per_minute": 4_000_000  # Token usage limit
    },
)
```

### 4. Generate Data

Generate your synthetic data:

```python
recipes = recipe_generator(cuisines)
print(recipes.dataset.to_pandas())
```

## LiteLLM Configuration

### API Keys and Environment Variables

For Gemini:

```bash
export GEMINI_API_KEY='your-api-key-here'  # Get from https://aistudio.google.com/app/apikey
```

## Curator Configuration

### Rate Limits

Configure rate limit with backend parameters:

```python
# Custom RPM/TPM configuration
# By default, this is set to:
# - max_requests_per_minute: 10
# - max_tokens_per_minute: 100_000
backend_params={
    "max_requests_per_minute": 2_000,     # 2K requests/minute
    "max_tokens_per_minute": 4_000_000    # 4M tokens/minute
}
```

## Providers and Models

Here are a list of providers. This is not an exhaustive list. Please refer to the [litellm provider documentation](https://docs.litellm.ai/docs/providers).

### Together

```
export TOGETHER_API_KEY='your-api-key-here'
```

```python
recipe_generator = RecipeGenerator(
    model_name="together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo",
    backend="litellm",
)
```

Other common models ([more info](< https://api.together.ai/models?filter=serverless.>)):

```python
together_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
together_ai/Qwen/Qwen2.5-7B-Instruct-Turbo # doesn't support structured outputs
```

### DeepInfra

```
export DEEPINFRA_API_KEY='your-api-key-here'
```

```python
recipe_generator = RecipeGenerator(
    model_name="deepinfra/deepseek-ai/DeepSeek-R1-Turbo",
    backend="litellm",
)
```

Other common models ([all models](https://deepinfra.com/models) — use prefix deepinfra):

```
deepinfra/meta-llama/Llama-3.3-70B-Instruct
deepinfra/meta-llama/Llama-3.3-70B-Instruct-Turbo
deepinfra/Qwen/Qwen2.5-72B-Instruct
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-litellm-with-curator.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.