Caching and recovery

• Recovery. A long run can fail or stop part way. When you start it again, Curator keeps the responses it already has and only sends the requests that are missing.
• Reuse. In a pipeline with several steps, you often change a later step and run the whole pipeline again. Curator reads the earlier steps from the cache, so they cost no time or money.

from bespokelabs import curator

llm = curator.LLM(model_name="gpt-4o-mini")
poem = llm("Write a poem about the importance of data in AI.")
print(poem.dataset.to_pandas())

​

Turn off caching

export CURATOR_DISABLE_CACHE=1

​

Change the cache directory

• Set the CURATOR_CACHE_DIR environment variable to the directory you want.
• Pass working_dir when you call the LLM object, e.g., llm("Write a poem.", working_dir="/path/to/my/poems").

​

What decides a cache hit

• The input dataset.
• The prompt method.
• The model name.
• The response format.
• Whether the run uses batch mode.
• The generation parameters, e.g., temperature.

​

What is in the cache directory

• metadata.db, a SQLite database with details about each run.
• One directory per run, named after the run’s fingerprint.

ls ~/.cache/curator
# 032bc5ead2892f8f        6d1f31229726231d
# 137851647e75f9a7        91f4ad23d5821c9f
# 24b1d8917f7ef6f1        a2a3c8e5a58e3fc3
# metadata.db

​

Troubleshooting

rm -rf ~/.cache/curator