# Latency

> How input tokens, images, questions and options add to the response time.

Canonical: https://platform.uprelic.com/docs/latency

---

A response takes the model time plus your network round trip. Strom answers each question with a single token, so the model time depends on how much it has to read: the input tokens of your request, the images and the number of questions. The numbers below are measured against the production API, one request at a time, with fresh content in every request so no cache helps. They show the median, with the 90th percentile in brackets.

## Text: model time by input tokens

One yes/no question about a text of the given length.

| Input tokens | Model time |
|---|---|
| 550 | 91 ms (102 ms) |
| 1,000 | 122 ms (135 ms) |
| 2,000 | 177 ms (200 ms) |
| 4,000 | 183 ms (198 ms) |
| 8,000 | 228 ms (233 ms) |
| 16,000 | 427 ms (437 ms) |
| 24,000 | 673 ms (682 ms) |

Every response reports `usage.input_tokens`, so you can see where your requests sit in this table.

## Images: tokens and model time by size

An image costs one input token per 32 × 32 pixels. Images larger than 1 megapixel are scaled down to 1 megapixel and very small ones are scaled up, so one image costs between 64 and 1,024 tokens. Measured with one new image and one yes/no question per request:

| Image size | Image tokens | Model time |
|---|---|---|
| 256 × 256 | 64 | 58 ms (67 ms) |
| 512 × 512 | 256 | 65 ms (70 ms) |
| 768 × 768 | 576 | 126 ms (133 ms) |
| 1024 × 1024 or larger | 1,024 | 168 ms (178 ms) |

Send images only as large as the detail you need: 512 pixels is often enough, at a quarter of the tokens and less than half the time of a full-size image.

## Questions per request

All questions of a request are answered on the same input, which is read once. Each extra question still adds time, so ask what you need, not every question you can think of.

| Questions | With one 512 px image | With a 2,000-token text |
|---|---|---|
| 1 | 66 ms (70 ms) | 181 ms (187 ms) |
| 2 | 117 ms (126 ms) | 185 ms (203 ms) |
| 4 | 157 ms (204 ms) | 201 ms (204 ms) |
| 8 | 180 ms (233 ms) | 281 ms (294 ms) |
| 16 | 239 ms (266 ms) | 438 ms (452 ms) |

## Options per question

Every option of a choice or score question, with its description, is part of the input. One choice question with short option descriptions:

| Options | With one 512 px image | With a 500-token text |
|---|---|---|
| 5 | 66 ms (71 ms) | 92 ms (98 ms) |
| 25 | 115 ms (129 ms) | 124 ms (128 ms) |
| 50 | 152 ms (158 ms) | 180 ms (187 ms) |
| 100 | 154 ms (157 ms) | 178 ms (188 ms) |
| 250 | 217 ms (235 ms) | 176 ms (183 ms) |

## API and network

On top of the model time, the API itself adds 5 to 20 ms (key check, billing, image preparation), more for very large inputs. Then comes your network round trip, which depends on where you are. Requests go straight to our servers in Gravelines, France.

## Keeping requests fast

- Put only what the questions need into the state: the relevant part of a document, not the whole file.
- Ask the questions you need about an item in one request, rather than one request per question.
- Scale images to the detail you need before sending them.
- Keep option lists and their descriptions short.
- Send independent requests in parallel, within the account's concurrency limit.