The deal
Custom mode serves any model you can express as a Python class: segmentation, image generation, TTS, classifiers, your own research architecture. You write one class with a predict(data: bytes) → dict method. The platform wraps it in an HTTP server, guards it with your API key, runs it on the workers your yaml asks for, and exposes it at POST /predict.
No base class to inherit, no SDK import inside your model code, no FastAPI to write. Bytes in, JSON-serializable dict out. Everything else is the platform's problem.
Three terms
| Term | Meaning |
|---|---|
| runtime | one deployed job: your bundle running behind one endpoint URL with one predict key |
| worker | a GPU machine the scheduler reserved for your job |
| replica | one loaded instance of your class. One replica per worker; more replicas = more parallel throughput |
Your class is constructed once per replica at boot, then serves requests for the life of the runtime. Heavy setup (loading weights, building preprocessors) belongs in __init__, where it runs once, not in predict, where it would run per request.
What the runtime exposes
| Route | Auth | What |
|---|---|---|
| POST /predict | X-API-Key | the model call: request body in, your dict out as JSON |
| GET /health | none | readiness; your class can customize the answer with an optional health() method |
| POST /warmup | X-API-Key | optional pre-warming hook so the first real request skips JIT cost |
The platform owns HTTP, auth at the edge, request ids, structured logs, webhooks, replica scaling, and GPU reservation. Re-implementing any of those inside your class fights the wrapper. Keep it pure and it composes.
Next: the class contract, the yaml, or a complete example. Client-side calling patterns: Calling /predict.