Reasoning traces

This recipe asks Claude questions with thinking turned on, and saves both the thinking and the final answer for each question.

Prerequisites

Python 3.10 or later.
Curator, installed with pip install bespokelabs-curator.
An Anthropic API key.

Set the API key

export ANTHROPIC_API_KEY=<your-api-key>

Create an LLM subclass

Set return_completions_object = True so that parse gets the full API response instead of only the text. The response has a list of content blocks. A thinking block holds the thinking and a text block holds the answer.

from bespokelabs import curator


class Reasoner(curator.LLM):
    return_completions_object = True

    def prompt(self, input: dict) -> str:
        return input["question"]

    def parse(self, input: dict, response: dict) -> dict:
        thinking = ""
        text = ""
        for block in response["content"]:
            if block["type"] == "thinking":
                thinking = block["thinking"]
            elif block["type"] == "text":
                text = block["text"]

        input["claude_thinking"] = thinking
        input["claude_answer"] = text
        return input

Configure the model

Curator passes generation_params to the Anthropic Messages API as they are.

llm = Reasoner(
    model_name="claude-sonnet-5-5",
    backend="anthropic",
    generation_params={
        "max_tokens": 16000,
        "thinking": {"type": "adaptive", "display": "summarized"},
    },
    backend_params={"require_all_responses": False},
)

With adaptive thinking, Claude decides how much to think for each question. Set display to "summarized" to get a summary of Claude’s reasoning in each response.

Generate the data

result = llm([
    {"question": "How to solve for world peace?"},
    {"question": "What is the fifteenth prime number?"},
])
print(result.dataset.to_pandas())

The output looks like this.

question	claude_thinking	claude_answer
How to solve for world peace?	This is a question about solving for world pea…	The Path to World Peace\n\nWorld peace is on…
What is the fifteenth prime number?	Let me list out the prime numbers in order to …	The fifteenth prime number is 47.\n\nThe seque…

Thinking settings differ between Claude models. Older models such as Claude Haiku 4.5 use {"type": "enabled", "budget_tokens": 8000} instead of adaptive thinking, and newer models reject budget_tokens. Check the Anthropic extended thinking docs for the model you use.

Use batch mode

For large datasets, set batch=True to use the Anthropic batch API at a lower price. The rest of the code stays the same. See Batch inference.

llm = Reasoner(
    model_name="claude-sonnet-5-5",
    backend="anthropic",
    batch=True,
    generation_params={
        "max_tokens": 16000,
        "thinking": {"type": "adaptive", "display": "summarized"},
    },
    backend_params={"require_all_responses": False},
)

For more settings, see the API reference.

Get started

Core features

Guides

Recipes

Fine-tuning

Reference

Prerequisites

Use batch mode

​Prerequisites

​Use batch mode

Prerequisites

Use batch mode