Limits and errors - Bespoke Labs

Limits

These limits apply to every model.

Limit	Value
Request body	1 MiB
Requests that one organization can run at the same time	8

nimble-latest and nimble-v3 also have these limits.

Limit	Value
Questions in a request	1 to 64
Options in a `choice` question, or levels in a `score` question	2 to 255
Prompt tokens for one question, including the state	32,768

The previous model, bespokelabs/Bespoke-Nimble-9B, allows 8,192 prompt tokens for one question. Nimble never cuts a prompt. A request above a limit gets an error. The fact check and code search guides list the limits of those models.

Errors

Status	Meaning	What to do
401	The API key is missing or not valid.	Check the key and the `Authorization` header.
402	Your organization does not have enough credit for the request.	Buy credit in the console.
413	The request body is larger than 1 MiB.	Send a smaller state.
422	The request is not valid, or it is above a limit. The `detail` field says why.	Fix the request. Do not retry it unchanged.
429	Your organization already has 8 requests running.	Retry after the number of seconds in the `Retry-After` header.
502	A model returned an error.	Retry with backoff.
503	A model is starting, or Nimble is not available for a moment.	Retry after the number of seconds in the `Retry-After` header. Without the header, retry with backoff.
504	The request took too long.	Send fewer questions or a shorter state.
529	A general model, e.g., `nimble-latest`, is busy.	Retry after about one second, with backoff.

You are not charged for a request that gets an error. A 422 with the detail “Nimble request failed” comes from a general model itself. One cause is a prompt that is longer than the model allows. Another cause is a model name that Nimble does not know. These are the model names:

nimble-latest, which is nimble-v3 now
nimble-v3 (or bespokelabs/Bespoke-Nimble-9B-v3)
bespokelabs/Bespoke-Nimble-9B, the previous model
nimble-factcheck-lite, nimble-factcheck, nimble-factcheck-max
nimble-codegrep-lite, nimble-codegrep, nimble-codegrep-max

Request IDs

Every response has an x-request-id header, including an error response. The x-typesafe-request-id header has the same ID. Error responses also have it in the request_id field, and so do fact check and code search responses. Send this ID to Bespoke Labs when you report a problem with a request.