Key concepts

• prompt takes one input row and returns the prompt for the LLM.
• parse takes the same input row and the LLM response, and returns one or more output rows.

from bespokelabs import curator


class Translator(curator.LLM):
    def prompt(self, input: dict) -> str:
        return f"Translate this sentence to French: {input['sentence']}"

    def parse(self, input: dict, response: str) -> dict:
        return {"sentence": input["sentence"], "french": response}


translator = Translator(model_name="gpt-4o-mini")
result = translator([
    {"sentence": "The cat is on the table."},
    {"sentence": "I like to read in the morning."},
])
print(result.dataset.to_pandas())

​

prompt

• A string. Curator sends it as a single user message.
• A list of messages, e.g., [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}].
• A tuple of a string and an image or file, for multimodal prompts.

​

parse

• The input row that went into prompt.
• The LLM response. This is a string by default. If you set a response format, it is an instance of your Pydantic model.

​

Inputs

• A single string or a single list of messages.
• A list of strings or a list of dictionaries.
• A Hugging Face Dataset.
• The CuratorResponse from an earlier call. Curator uses its dataset.
• No input. Curator then sends one request, which is useful for a first step that generates seed data.

​

Data flow

Input dataset:
  Row A
  Row B

Processing:
  Row A -> prompt(A) -> Response R1 -> parse(A, R1) -> [C, D]
  Row B -> prompt(B) -> Response R2 -> parse(B, R2) -> [E, F]

Output dataset:
  Row C
  Row D
  Row E
  Row F

​

CuratorResponse