Calling /predict

30-second tour

A deployed runtime is just an HTTPS endpoint. Hit it with the API key and the raw bytes of your input.

Where the URL + key come from

Both are returned exactly once when you submit:

• SDK: job.endpoint and job.api_key after rt_submit_inference().
• Wizard: the one-time panel shown at the end of step 7 — copy both before closing it (the key is unrecoverable from there).

The key isn't retrievable after the one-time panel — save it the first time. Key rotation will ship as a separate dashboard flow.

Just HTTP — call it from anywhere

The endpoint is a plain HTTPS POST. Anything that can speak HTTP works — curl, Postman, a Go/Node/Rust service, a Bash one-liner in a CI job. No SDK required.

The wire format is exactly two things:

• Header X-API-Key: <your key> (omit for PUBLIC runtimes).
• Body: the raw bytes your predict(data: bytes) expects. Content type is whatever you set — the platform doesn't inspect it, it just forwards the body.

So why use InferenceClient at all?

For a one-off shell call or a non-Python service, raw HTTP is the right answer — it's simpler than installing the SDK. The Python client is convenience over the same wire, and it earns its place when you want:

• Typed methods for common shapes — predict_file() reads the file + guesses the content type; predict_url() lets the server fetch a remote URL; predict_json() sidesteps theapplication/json footgun.
• Batched concurrency — predict_many(items, concurrency=8) runs N requests in flight without you wiring up a thread pool.
• Typed exceptions — AuthError / ValidationError / PredictError / ServerError map cleanly onto try/except instead of "did the request succeed? let me read the status code".
• Auto-retry on transient 5xx — 502/503/504 get retried with backoff; 4xx don't (your input is the problem, not the server).

Python client methods

If you've picked the Python client, here's the menu — pick the method that matches where your input lives.

I have a file on disk

Reads the file, sends the bytes with a content type guessed from the extension. Same wire format as predict() below — just convenience.

I have bytes in memory

The data lives at a URL

The server-side fetches the URL and feeds the bytes to your predictor. Saves your local bandwidth when the input is already in the cloud.

I want to send parameters alongside my data

Use predict_json() — your dict is sent with Content-Type: application/octet-stream and your predict() receives the raw bytes, which it can json.loads().

I have a lot of inputs and I want them fast

Parallel client-side; the server sees N independent requests at concurrency in flight. Tune concurrency against your runtime's cluster.max_ongoing_requests × replica count — going higher than that just stacks in the queue.

When things go wrong

The client raises a typed exception per status code. The error message includes the request ID (also in the response header) — quote it if you need someone to look at logs.

What's next

• Worked examples end-to-end: every folder in the predictor examples gallery ships a predict.pydemonstrating that job's exact request shape.
• Common failures: Troubleshooting has an Inference section that maps the errors above to "what to do about it".