Skip to main content
Nimble is a family of models from Bespoke Labs. You send Nimble a piece of text or JSON, which is called the state, and a list of questions about it. Nimble answers each question with probabilities, so you can set your own thresholds. You call every Nimble model at the same endpoint, POST https://api.bespokelabs.ai/v1/systemone. You pick the model with the model field of the request. You can send requests with plain HTTP or with TypeSafe’s Python SDK.

​
nimble-latest

nimble-latest answers up to 64 questions about the state in one request. There are three types of question:
  • A noul question asks whether a statement is true. The answer is the probability that it is true.
  • A choice question asks which of 2 to 255 options fits best. The answer names the most likely option and gives the probability of every option.
  • A score question asks where the state falls on a scale of 2 to 255 ordered levels. The answer is the expected level.
nimble-latest is the newest general Nimble model, which is nimble-v3 now. Use nimble-v3 to stay on this model when a newer one comes out. The question types guide shows how to write each type of question.

​
nimble-factcheck

You send the fact check models a document and a list of claims. For each claim, the answer is the probability that the document supports the claim. There are three fact check models. nimble-factcheck-lite is the fastest and the cheapest. nimble-factcheck-max is the most accurate. nimble-factcheck is in between. We compared these models with Jev 1.13.0 from TypeSafe on the test split of LLM-AggreFact, a public benchmark of 11 datasets with 29,320 claims. By default, the fact check models score each sentence of a claim on its own. Jev scores each claim whole, so in this comparison we scored whole claims with every model. Each number in the table is the balanced accuracy, averaged over the 11 datasets. Balanced accuracy is the average of two shares. One is the share of supported claims that a model calls supported. The other is the share of unsupported claims that it calls unsupported.
  • nimble-factcheck-lite is 1.3 points behind Jev.
  • nimble-factcheck is 0.5 points behind Jev. The 95% interval of this difference includes zero, so the two are about even.
  • nimble-factcheck-max is 0.8 points ahead of Jev. The whole 95% interval of this difference is above zero.
The fact check guide has the full results and notes on how we measured them. For a fact check model that you can download and run yourself, see Bespoke-MiniCheck-7B on Hugging Face.

​
nimble-code

You send nimble-code one unit of code, e.g., a function, a file or a diff, and questions about its quality. It answers the same three types of question as nimble-latest. The code quality guide shows how to describe the code and read the answers.

​
Get an API key

Create a key in the Bespoke Labs console. The Nimble quickstart shows how to use it.