The examples repository
examples/inference_jobs/predict holds complete predictor-mode jobs across modalities. Like the vLLM set, the models are general-purpose tools with no sector attached; each folder is a working reference for one shape of the predict(data) → dict contract.
| Folder | What it serves | Min VRAM |
|---|---|---|
| sam2.1_segmentation | point/box-prompted image segmentation — masks out | 6 GB |
| FLUX.1-schnell_imagegen | text-to-image generation | 16 GB |
| Orpheus-TTS-3B_tts | text-to-speech | 8 GB |
| image_classification | a compact CV classifier — the smallest real-model reference | — |
| minimal_echo | the five-line Echo predictor — the contract with nothing else in the way | none (CPU) |
Every folder has the same layout:
inference.yaml
module + class_name + init_args + cluster sizing
scripts/<name>_serve.py
the predictor class
submit.py
deploys via the SDK
predict.py
a working client for that job's request shape
Run any of them
i
Or paste the serve script and yaml into the dashboard's inference wizard. Jobs that pull weights from HuggingFace Hub take a few minutes on first boot; the submit script waits for RUNNING.