Hugging Face Inference Providers

Hugging Face Inference Providers give you one API for models hosted by many inference partners. The API is compatible with the OpenAI API, so you can use it with the openai backend in Curator. This guide generates vegan recipes.

Setup

Create a Hugging Face account if you do not have one.
Create an access token. You use it to authenticate your requests.
Turn on at least one provider on the Inference Providers settings page.

Install the packages.

pip install bespokelabs-curator huggingface_hub

Create an LLM subclass

from datasets import Dataset

from bespokelabs import curator


class RecipeGenerator(curator.LLM):
    """Generate vegan recipes for different cuisines."""

    def prompt(self, input: dict) -> str:
        return f"Generate a random {input['cuisine']} recipe. Be creative but keep it realistic and make the recipe vegan."

    def parse(self, input: dict, response: str) -> dict:
        return {"recipe": response, "cuisine": input["cuisine"]}

Create the input dataset

cuisines = Dataset.from_list([
    {"cuisine": cuisine}
    for cuisine in [
        "Chinese", "Italian", "Mexican", "French", "Japanese",
        "Indian", "Thai", "Korean", "Vietnamese", "Brazilian",
    ]
])

Point the OpenAI backend at Hugging Face

Set these three values.

base_url is the Hugging Face router, https://router.huggingface.co/v1.
api_key is your Hugging Face token.
model_name is the model ID on the Hub. Add :<provider> to the end to choose a provider, e.g., :together. Without it, Hugging Face picks a provider for you.

import os

HF_TOKEN = os.environ["HF_TOKEN"]

recipe_generator = RecipeGenerator(
    model_name="meta-llama/Llama-3.3-70B-Instruct:together",
    backend="openai",
    backend_params={
        "base_url": "https://router.huggingface.co/v1",
        "api_key": HF_TOKEN,
    },
)

results = recipe_generator(cuisines).dataset
print(results.to_pandas())

The model page on the Hub shows which providers serve a model.

Share the results

To look through the results in the Curator Viewer, upload them with push_to_viewer.

url = curator.push_to_viewer(results)

The results are a Hugging Face Dataset, so you can also push them to the Hub.

results.push_to_hub("<hf-username>/vegan-recipes", private=False, token=HF_TOKEN)

This dataset on the Hub was made in the same way.

Switch models

You can change the model or the provider by changing model_name. To find the models that one provider serves, filter the Hub by that provider, e.g., the models served by Together.

Get started

Core features

Guides

Recipes

Fine-tuning

Reference

Hugging Face Inference Providers

Setup

Switch models

​Setup

​Switch models

Setup

Switch models