Fact check

Nimble fact check takes a document and a list of claims. For each claim, it returns the probability that the document supports the claim. Send the request to POST /v1/systemone with one of the fact check models:

Put the document in state, as a string.
Add one noul question for each claim, with the claim in instructions.
The question IDs are your own. The answers use the same IDs.

Make a request

curl https://api.bespokelabs.ai/v1/systemone \
  -H "Authorization: Bearer $BESPOKE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nimble-factcheck",
    "state": "The board moved the vote to Tuesday because the CFO was travelling.",
    "questions": {
      "moved": { "type": "noul", "instructions": "The vote was moved to Tuesday." },
      "ceo": { "type": "noul", "instructions": "The CEO was travelling." }
    }
  }'

from bespokelabs import BespokeLabs

client = BespokeLabs()  # reads BESPOKE_API_KEY

response = client.nimble.system_one(
    model="nimble-factcheck",
    state="The board moved the vote to Tuesday because the CFO was travelling.",
    questions={
        "moved": {"type": "noul", "instructions": "The vote was moved to Tuesday."},
        "ceo": {"type": "noul", "instructions": "The CEO was travelling."},
    },
)
for claim_id, answer in response.answers.items():
    print(claim_id, answer.noul)

Read the answer

{
  "model": "nimble-factcheck",
  "answers": {
    "moved": { "type": "noul", "noul": 0.99 },
    "ceo": { "type": "noul", "noul": 0.002 }
  },
  "usage": { "input_tokens": 25, "output_tokens": 0 },
  "request_id": "6b57f82e-6905-48a1-bc44-fe0af138d8c5"
}

noul is the probability that the document supports every part of the claim.
A claim counts as supported when noul is above 0.5. You can use a higher threshold, e.g., 0.8, when a wrong “supported” costs you more than a wrong “not supported”.
A claim that the document contradicts and a claim that the document does not mention are both not supported.

Models

Model	How it answers
`nimble-factcheck-lite`	A small model scores every claim. This is the fastest and the cheapest.
`nimble-factcheck`	The small model scores every claim. Then a larger model scores again the claims that the small model is unsure about. This is about one claim in five.
`nimble-factcheck-max`	The small model scores every claim. Then two of the largest models score again the claims that the small model is unsure about, and Nimble uses the average of their scores. This is the most accurate.

The largest models of nimble-factcheck-max are running on weekdays from 07:30 to 19:00 Pacific time. At other times, a request that needs those models gets 503 with a Retry-After header while the models start. Starting takes from about 5 to 20 minutes. You are not charged for a 503.

Options

split_claims is true by default. Nimble then scores each sentence of a claim on its own and keeps the lowest score. Set it to false to score each claim whole. Put it at the top level of the request, next to model.

{
  "model": "nimble-factcheck",
  "split_claims": false,
  "state": "The board moved the vote to Tuesday because the CFO was travelling.",
  "questions": {
    "moved": { "type": "noul", "instructions": "The vote was moved to Tuesday because the CFO was travelling." }
  }
}

Limits

Limit	Value
Claims in a request	1 to 64
Characters in one claim	4,000
Characters in the document	400,000

The questions must be noul questions with no criteria. Any other question gets 422. Nimble splits a long document into parts. A sentence counts as supported if any part supports it.

Get started

Guides

Make a request

Read the answer

Models

Options

Limits

​Make a request

​Read the answer

​Models

​Options

​Limits

Make a request

Read the answer

Models

Options

Limits