Bespoke Labs
  • Welcome
  • BESPOKE CURATOR
    • Getting Started
      • Quick Tour
      • Key Concepts
      • Visualize your dataset with the Bespoke Curator Viewer
      • Automatic recovery and caching
      • Structured Output
    • Save $$$ on LLM inference
      • Using OpenAI for batch inference
      • Using Anthropic for batch inference
      • Using Gemini for batch inference
      • Using Mistral for batch inference
      • Using kluster.ai for batch inference
    • How-to Guides
      • Using vLLM with Curator
      • Using Ollama with Curator
      • Using LiteLLM with curator
      • Handling Multimodal Data in Curator
      • Executing LLM-generated code
      • Using HuggingFace inference providers with Curator
    • Data Curation Recipes
      • Generating a diverse QA dataset
      • Using SimpleStrat block for generating diverse data
      • Curate Reasoning data with Claude-3.7 Sonnet
      • Synthetic Data for function calling
    • Finetuning Examples
      • Aspect based sentiment analysis
      • Finetuning a model to identify features of a product
    • API Reference
  • Models
    • Bespoke MiniCheck
      • Self-Hosting
      • Integrations
      • API Service
    • Bespoke MiniChart
    • OpenThinker
Powered by GitBook
On this page
  1. Models
  2. Bespoke MiniCheck

Self-Hosting

PreviousBespoke MiniCheckNextIntegrations

Last updated 8 months ago

You can access the model here:

Feel free to use this which uses the MiniCheck library that supports automated chunking of long documents.

Or, you can host the model directly on vLLM with docker as follows:

```shellscript
sudo docker run \
  --runtime=nvidia \
  --gpus=all \
  -v ~/.cache/huggingface:/root/.cache/huggingface     \
  --env "HUGGING_FACE_HUB_TOKEN=hf_xyz" \
  --ipc=host \
  -p 8000:8000 \
  vllm/vllm-openai:latest \
  --model bespokelabs/Bespoke-MiniCheck-7B --trust_remote_code --api-key your_api_key --disable-log-requests \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --tensor-parallel-size 1 &
```

Please contact us for commercial licensing.

https://huggingface.co/bespokelabs/Bespoke-Minicheck-7B
colab