The deal
Custom mode serves any model you can express as a Python class: segmentation, image generation, TTS, classifiers, your own research architecture. You write one class with a predict(data: bytes) → dict method. The platform wraps it in an HTTP server, guards it with your API key, runs it on the workers your yaml asks for, and exposes it at POST /predict.
Your bundle
- inference.yaml
- scripts/my_serve.py
The running job
GPU workers running your class
- POST /predict
- GET /health
No base class to inherit, no SDK import inside your model code, no FastAPI to write. Bytes in, JSON-serializable dict out. Everything else is the platform's problem.
Three terms
Your class is constructed once per replica at boot, then serves requests for the life of the runtime. Heavy setup (loading weights, building preprocessors) belongs in __init__, where it runs once, not in predict, where it would run per request.
What the runtime exposes
The platform owns HTTP, auth at the edge, request ids, structured logs, webhooks, replica scaling, and GPU reservation. Re-implementing any of those inside your class fights the wrapper. Keep it pure and it composes.
Next: the class contract, the yaml, or a complete example. Client-side calling patterns: Calling your model.