# Video detection

> Find the object you describe in every frame of a video.

Canonical: https://platform.uprelic.com/docs/video-detection

---

The video detection API, `POST /v1/harness/video-detection`, finds the object you describe in every frame of a video. You describe the object in plain words, for example “Player with the ball” or “the red car that runs the light”, and get its bounding box in each frame.

## How it works

The video is read at 5 frames a second. A tracking model follows every object that could match your description, for example every player. Once per second of video, Strom watches the last 4 seconds with those objects numbered and picks the one your whole description fits. Each frame gets that object’s box. When the description fits a different object later in the video, the box moves to it: in a football match, when the ball is passed, the box moves to the player who now has it.

## Starting a job

A video takes a while to process, so the API works in two steps. First, start a job with the video and the description:

```bash Request
curl https://api.uprelic.com/v1/harness/video-detection \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "video": {"url": "https://example.com/match.mp4"},
    "description": "Player with the ball"
  }'
```

`video.url` must be a public http(s) URL. The video can be MP4, MOV, WebM or another common format, at most 500 MB and 10 minutes long. The response is a `202` with the job’s `id`:

```json Response
{
  "id": "5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12",
  "status": "queued",
  "progress": 0.0,
  "description": "Player with the ball",
  "frames": null,
  "usage": null,
  "error": null,
  "created_at": "2026-11-01T09:30:00+00:00"
}
```

## Getting the result

Then get the job with its `id` every few seconds, until `status` is `done` or `failed`. A minute of video takes about 30 seconds. While the job runs, `progress` goes from 0 to 1.

```bash Request
curl https://api.uprelic.com/v1/harness/video-detection/5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12 \
  -H "Authorization: Bearer YOUR_API_KEY"
```

```json Response · illustrative
{
  "id": "5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12",
  "status": "done",
  "progress": 1.0,
  "description": "Player with the ball",
  "frames": [
    {
      "t": 0.0,
      "box": {"left": 0.4103, "top": 0.2981, "right": 0.4702, "bottom": 0.5215},
      "confidence": 0.81
    },
    {
      "t": 0.2,
      "box": {"left": 0.4155, "top": 0.3012, "right": 0.4761, "bottom": 0.5254},
      "confidence": 0.81
    },
    ...
    {"t": 9.0, "box": null, "confidence": null},
    ...
  ],
  "usage": {"video_seconds": 20.0, "cost": 0.0293, "currency": "eur"},
  "error": null,
  "created_at": "2026-11-01T09:30:00+00:00"
}
```

When the job is done, `frames` has one entry for every 0.2 seconds of video, from the start:

- `t`: the frame’s time in seconds.
- `box`: where the object is in the frame. `left` and `right` are fractions of the frame’s width, `top` and `bottom` fractions of its height, measured from the top left corner. `null` when the object isn’t in the frame.
- `confidence`: how sure Strom is that the box is around the object you described, from 0 to 1. `null` when `box` is.

You can get a job’s frames for as long as your [plan](https://platform.uprelic.com/docs/pricing#plans) keeps request contents (7 days on Free). After that, `frames` and `description` are `null`.

The Python SDK starts the job and waits for it in one call:

```python Python
from uprelic import Uprelic

client = Uprelic()
job = client.video_detection(
    description="Player with the ball",
    video="https://example.com/match.mp4",
)

for frame in job.frames:
    if frame.box:
        print(frame.t, frame.box.left, frame.box.top, frame.box.right, frame.box.bottom)
```

## Cost and limits

- 0.10 USD per minute of video, billed by the second and charged in your account’s currency. A 20-second video costs about 0.033 USD. The cost is in `usage.cost` once the video is downloaded.
- A job that fails costs nothing.
- At most 10 jobs per account can be queued or running at once.

## Errors

- `400`: `video.url` isn’t an http(s) URL.
- `402`: your balance is too low. A job also fails if the balance doesn’t cover the whole video once it’s downloaded.
- `404`: there is no job with this `id` on your account.
- `429`: you already have 10 jobs queued or running.
- A job that can’t be finished has `status` `failed` and the reason in `error`, for example when the video can’t be downloaded or read, or is longer than 10 minutes.