# Online Processing

This guide demonstrates how to use **Curator** for **online processing** by generating diverse topics for poems and composing poems based on these topics. We'll focus on key parameters to manage rate limits effectively, ensuring smooth operation with LLMs.

## Prerequisites

* Python 3.10+
* Curator: Install via `pip install bespokelabs-curator`
* Access to an LLM provider (e.g., OpenAI or an equivalent API)

## Steps

### 1. Define the Response Formats

**Code Example:**

```python
from typing import List
from pydantic import BaseModel, Field

class Topics(BaseModel):
    """A list of topics."""
    topics_list: List[str] = Field(description="A list of topics.")

class Poems(BaseModel):
    """A list of poems."""
    poems_list: List[str] = Field(description="A list of poems.")
```

***

### 2. Create a Muse

Define a subclass of `curator.LLM` to generate a list of topics for our poems.

**Code Example:**

```python
from bespokelabs import curator

class Muse(curator.LLM):
    response_format = Topics

    def prompt(self, input: dict) -> str:
        return "Generate 10 diverse topics that are suitable for writing poems about."

    def parse(self, input: dict, response: Topics) -> dict:
        return [{"topic": t} for t in response.topics_list]

# Instantiate the Topic Generator
topic_generator = TopicGenerator(model_name="gpt-4o-mini", backend_params={"max_requests_per_minute": 100})

# Generate topics
from datasets import Dataset
topics: Dataset = topic_generator()
print(topics["topic"])  # View the generated topics
```

***

### 3. Create a Poet

Define another subclass of `curator.LLM` to create poems based on the generated topics.

**Code Example:**

```python
class Poet(curator.LLM):
    """A poet that generates poems about given topics."""
    response_format = Poems

    def prompt(self, input: dict) -> str:
        return f"Write two poems about {input['topic']}."

    def parse(self, input: dict, response: Poems) -> dict:
        return [{"topic": input["topic"], "poem": p} for p in response.poems_list]

# Instantiate the Poet
poet = Poet(model_name="gpt-4o-mini", backend_params={"max_requests_per_minute": 100})

# Generate poems based on topics
poems = poet(topics)
print(poems.to_pandas())  # View the poems in a tabular format
```

***

### Example Output

```plaintext
                                           topic                                               poem
0                            Dreams vs. reality  In the realm where dreams take flight,\nWhere ...
1                            Dreams vs. reality  Reality stands with open eyes,\nA weighty thro...
2           Urban loneliness in a bustling city  In the city's heart where shadows blend,\nAmon...
3           Urban loneliness in a bustling city  Among the crowds, I walk alone,\nA sea of face...
```

***

## Online Processing Configuration

Curator provides several configuration parameters to better control your  API usage policies and improve efficiency. 

#### 1. **`max_requests_per_minute`**

**Definition**: The maximum number of API requests allowed per minute.

* **Use Case**: Controls the frequency of requests to prevent exceeding API quotas.
* **Example**: Setting this to `60` ensures only 60 requests are sent per minute.

**Example**:

```python
backend_params={
    "max_requests_per_minute": 60  # Sets maximum requests in a minute.
}
```

**Note**: Retries will also count under the limit set.

#### 2. **`max_tokens_per_minute`**

**Definition**: The maximum number of tokens allowed to be processed per minute.

* **Use Case**: Defines contraint on maximum usable tokens  in a minute.
* **Example**: Setting this to `30,000` ensures token limits are respected for responses.

**Example**:

```python
backend_params={
    "max_tokens_per_minute": 1_000_000  # Sets maximum tokens in a minute.
}
```

**Note**: Token limit will be counted including input and output tokens.

#### 3. **`seconds_to_pause_on_rate_limit`**

**Definition**: The duration (in seconds) to pause when a rate limit is hit.

* **Use Case**: Automatically waits before retrying, ensuring compliance with API limits.
* **Example**: Setting this to `60` pauses the processing for one minute if a rate limit error occurs.

**Example**:

```python
backend_params={
    "seconds_to_pause_on_rate_limit": 60  # Sets seconds to wait before new request on rate limit.
}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/online-processing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.