# Building a video-gen pipeline that doesn't fall over: fallback across providers

> Disclosure: I work on [Revana](https://revana.si/?utm_source=hashnode&utm_medium=article&utm_campaign=crosspost-3), a router for image and video models. The code below targets its API (spec: https://revana.si/openapi.json), but the failure modes and the fixes apply to any generation provider.

A demo calls a video model once and shows the result. A production pipeline calls it thousands of times, and then you see what the demo hid. Generation endpoints fail far more often than ordinary APIs, in more ways, and every failure mode has a cost:

- **Capacity:** the provider is at capacity (`429`/`503`) for minutes at a time, usually right when you launch a campaign.
- **Slow jobs:** a 10-second 1080p clip can take anywhere from 40 seconds to 15 minutes, so your HTTP client times out long before the job does.
- **Refusals:** the model refuses the prompt (content policy) or returns text instead of a video.
- **Network:** a connection drops after you've submitted, and you don't know whether the job exists.
- **Billing:** some providers charge for failed or timed-out jobs.

Any one of these is easy to handle. The trouble is that a naive fix for one usually makes another worse. Retrying blindly turns a dropped connection into two paid videos, and falling back to another model turns a $1 clip into a $4 one. This post builds a small pipeline that handles all five, in about 120 lines of Python.

## The request shape

Every model goes through the same endpoint:

```http
POST https://api.revana.si/v1/generations
Authorization: Bearer sr_live_...
Idempotency-Key: campaign-42/shot-07/seedance-2.5/a0
Content-Type: application/json

{
  "model": "seedance-2.5",
  "input": {
    "prompt": "close-up of hands pouring cold brew over ice, morning light",
    "resolution": "720p", "duration": 8, "aspect_ratio": "9:16", "audio": true,
    "image_url": "https://example.com/start-frame.png"
  },
  "max_cost_usd": 2.00
}
```

The response is `202` with a generation object (`id`, `status`, `quoted_usd`, `reserved_usd`, `cost_usd`, `error`). You poll `GET /v1/generations/{id}` until `status` is `succeeded` or `failed`, then download `GET /v1/generations/{id}/content`.

Two fields do most of the safety work:

- **`max_cost_usd`** is a hard ceiling per generation. If the only available route costs more, the request fails with `max_cost_exceeded` and nothing is charged.
- **`Idempotency-Key`**: sending the same key with the same body returns the original generation (`200`) instead of starting a new one. The same key with a *different* body is `409 idempotency_conflict`.

## Layer 1: routing inside one model

The same model is often sold by several upstream providers. Seedance 2.5, for example, is available from ByteDance directly and from a handful of resellers. A router can try the cheapest healthy route first and move to the next one if it's down, without the caller ever seeing the failure.

Revana does this server-side, and the generation object tells you what it may cost you:

- `quoted_usd` is the price on the first route tried.
- `reserved_usd` is the most it can cost: the dearest backup route allowed under your `max_cost_usd`. That amount is held from your balance while the job runs.
- `cost_usd` is what you were actually charged: the price of the route that delivered, `0` if it failed, `null` while it's running.

If you want "this model, this price, or nothing", set `max_cost_usd` equal to the quote and dearer backups are skipped. If you'd rather get the clip, leave headroom. To check the quote before submitting, read the live price list (no key needed):

```python
import requests

MODELS = {m["id"]: m for m in requests.get("https://api.revana.si/v1/models", timeout=30).json()["data"]}

def quote(model: str, inp: dict) -> float:
    """Price of one generation from the live list: per_second x duration, or per_image."""
    price = MODELS[model]["price"]
    usd = price["usd"]
    if isinstance(usd, dict):
        res = inp.get("resolution")
        key = f"{res}+audio" if inp.get("audio") and f"{res}+audio" in usd else res
        usd = usd[key]
    return round(usd * inp.get("duration", 1), 4) if price["unit"] == "per_second" else usd
```

As of 2026-11-05, `quote("seedance-2.5", {"resolution": "720p", "duration": 8})` returns `1.68` ($0.21/s x 8 s).

There's also `max_wait_s`. Routes expected to take longer than that are skipped, and once that time has passed, no new route is tried. That's useful for interactive features where a clip that arrives in 20 minutes is worthless.

## Layer 2: fallback across models

Route-level fallback keeps *one model* up. Sometimes the model itself is the problem: it refuses a prompt the others accept, or every route is saturated. At that point you want a ranked list of acceptable models, each with its own price cap:

```python
FALLBACK = [
    # (model, input overrides, max_cost_usd)
    ("seedance-2.5", {}, 1.68),
    ("kling-3.0",    {"resolution": "1080p"}, 1.36),
    ("wan-3.0",      {"resolution": "720p"}, 0.56),
]
```

Two rules make this safe:

1. **Only fall back on errors that another model could fix.** `model_busy`, `model_unavailable`, `timeout`, `all_routes_failed` and `model_declined` are good reasons to try the next model. `invalid_input` (a bad reference URL) and `max_cost_exceeded` will fail everywhere, so stop and fix the request. Treat `content_policy` as a product decision: sometimes another model accepts the prompt, but often you *want* the refusal to stand.
2. **Each fallback is a new intent, so it gets a new idempotency key.** The body changes (different model), so reusing the key would be a `409` anyway.

## Layer 3: idempotency keys that mean something

The most expensive bug in generation pipelines is the double submit. A worker submits a job, the response is lost (a timeout, a deploy, an OOM kill), the queue redelivers the message, and the worker submits again. Now you've paid twice.

The fix is to derive the key from **what you're trying to make**, not from the HTTP call:

```
{job_id}/{shot_id}/{model}/a{attempt}
```

- After a crash, a redelivered message rebuilds the same key, so the resubmit returns the existing generation and you resume polling it.
- `attempt` increases **only after a generation has reached `failed`**. Failed generations are never charged, so a fresh attempt can't double-bill. Never mint a new key while the previous attempt's status is unknown. Resubmit with the old key and let the server tell you.
- Persist the generation `id` next to the key as soon as you get it, so a restart can skip straight to polling.

The HTTP layer retries with the **same** key: `429` (honour `Retry-After`), `503 busy`/`model_unavailable`, `500`, and connection errors. All of those are safe to replay because of the key.

## Putting it together

```python
import os, time, requests

API = "https://api.revana.si/v1"
S = requests.Session()
S.headers["Authorization"] = f"Bearer {os.environ['REVANA_API_KEY']}"

RETRY_HTTP = {429, 500, 502, 503, 504}
NEXT_MODEL = {"model_busy", "model_unavailable", "timeout", "all_routes_failed", "model_declined"}

def submit(model, inp, max_cost, key, tries=6):
    body = {"model": model, "input": inp, "max_cost_usd": max_cost}
    for i in range(tries):
        try:
            r = S.post(f"{API}/generations", json=body, headers={"Idempotency-Key": key}, timeout=60)
        except requests.ConnectionError:
            time.sleep(2 ** i); continue                      # same key: safe to replay
        if r.status_code in (200, 202):
            return r.json()
        if r.status_code in RETRY_HTTP:
            time.sleep(float(r.headers.get("Retry-After", 2 ** i))); continue
        err = r.json()["error"]                               # 4xx: retrying won't help
        raise RuntimeError(f"{r.status_code} {err['code']}: {err['message']}")
    raise RuntimeError("gave up submitting")

def wait(gen_id, deadline_s=1800):
    t0, delay = time.time(), 5
    while time.time() - t0 < deadline_s:
        r = S.get(f"{API}/generations/{gen_id}", timeout=30)
        if r.status_code == 200:
            g = r.json()
            if g["status"] in ("succeeded", "failed"):
                return g
        time.sleep(delay); delay = min(delay * 1.5, 30)       # back off on long jobs
    raise TimeoutError(gen_id)

def download(gen_id, path):
    with S.get(f"{API}/generations/{gen_id}/content", stream=True, timeout=300) as r:   # follows the 302
        r.raise_for_status()
        with open(path, "wb") as f:
            for chunk in r.iter_content(1 << 20):
                f.write(chunk)

def make_shot(job_id, shot_id, base_input, fallback, out_path, max_attempts=2):
    for model, overrides, max_cost in fallback:
        inp = {**base_input, **overrides}
        for attempt in range(max_attempts):
            key = f"{job_id}/{shot_id}/{model}/a{attempt}"
            g = wait(submit(model, inp, max_cost, key)["id"])
            if g["status"] == "succeeded":
                download(g["id"], out_path)
                return {"model": model, "id": g["id"], "cost_usd": g["cost_usd"]}
            err = g["error"]
            print(f"{key}: {err['code']} (retryable={err['retryable']}) {err['message']}")
            if err["code"] not in NEXT_MODEL:
                raise RuntimeError(err["message"])            # fix the request, don't burn fallbacks
            if not err["retryable"]:
                break                                         # this model won't do it: next model
            # retryable on this model: one more attempt with a new key (the failed one cost $0)
    raise RuntimeError(f"{shot_id}: every model in the fallback list failed")

if __name__ == "__main__":
    shot = {"prompt": "close-up of hands pouring cold brew over ice, morning light, 9:16 vertical",
            "resolution": "720p", "duration": 8, "aspect_ratio": "9:16", "audio": True}
    print(make_shot("campaign-42", "shot-07", shot, FALLBACK, "shot-07.mp4"))
```

(`FALLBACK` is the list from Layer 2. Paste it above `__main__`.)

Notes on the choices:

- **Back off on polling.** Generation takes minutes, and polling every second just burns rate limit. Start at 5 s and cap at 30 s.
- **4xx on submit is final.** `402 insufficient_credits`, `409 idempotency_conflict` and `422 invalid_input` all need a human or a code change, not a loop.
- **`cost_usd` goes in your ledger, not `quoted_usd`.** If a backup route delivered, the charge can differ from the first quote, though never above `max_cost_usd`.
- **Concurrency.** `429 concurrency_limit` means too many jobs are in flight. Run the shots through a bounded worker pool (8-16 is plenty) instead of firing 100 at once.

## Only paying for what succeeds

The billing model shapes the retry strategy. If a provider charges for failed or timed-out generations, every retry is a gamble, and you end up writing elaborate pre-validation to avoid them. When failures cost $0, as with Revana and ByteDance's own API (which documents charging only successful videos), retrying a `model_busy` is free, and the only real cost control you need is `max_cost_usd` per attempt.

The worst case is easy to bound: `attempts x max_cost_usd`, but only for attempts that succeed, and `make_shot` stops at the first success. For the fallback list above, a shot costs at most $1.68 (the highest cap in the list), even if all three models are tried.

Before a big batch, check that the account can cover it. `GET /v1/balance` returns `available_usd` (what new jobs can reserve) and `held_usd` (what running jobs are holding):

```python
bal = S.get(f"{API}/balance", timeout=30).json()
need = 100 * FALLBACK[0][2]          # 100 shots at the first model's cap
assert bal["available_usd"] >= need, f"top up: {bal['available_usd']} < {need}"
```

## Checklist

- [ ] Price from the live list, cap every request with `max_cost_usd`.
- [ ] Idempotency key = job + shot + model + attempt. Persist the generation id as soon as you have it.
- [ ] Replay transport errors, `429`, `5xx` with the *same* key. Honour `Retry-After`.
- [ ] New attempt key only after a confirmed `failed`.
- [ ] Fall back to the next model on capacity/timeout errors, not on `invalid_input`.
- [ ] Poll with backoff, download from `/content`, record `cost_usd`.
- [ ] Bound concurrency, and check `available_usd` before a batch.

The OpenAPI spec with every error code is at https://revana.si/openapi.json, and live model prices are at `GET https://api.revana.si/v1/models`. Prices in this post are as of 2026-11-05.

---

*Try it: [revana.si](https://revana.si/?utm_source=hashnode&utm_medium=article&utm_campaign=crosspost-3) has one API key for 200+ video and image models, with live prices on [revana.si/pricing](https://revana.si/pricing?utm_source=hashnode&utm_medium=article&utm_campaign=crosspost-3).*