Sign inStart now ›

Pick a model. Get an endpoint

OpenAI-compatible serving. Any model from Hugging Face — or any Python you write will become a callable endpoint

Deploy your first endpoint ›API docs

How an endpoint comes up

Name a model.

Pick a Hugging Face model right in code, or a Python class with a predict method. Set any vLLM engine options right for your needs and submit a job.

endpointhttps://divine-safari.predict.beta.reson.techmodelQwen2.5-Coder-7Bhugging facereplica 15070 Ti16 GB · vLLM—waiting for a modelreplica 25070 Ti16 GB · vLLM—not startedreqreqreqreq177 answers a minute behind one URL$serve Qwen2.5-Coder-7B · vllm · openai api▉
endpointdivine-safari.predict.beta.reson.techmodelQwen2.5-Coder-7Breplica 15070 TivLLMwaiting for a modelreplica 25070 TivLLMnot startedreqreqreqreq$serve Qwen2.5-Coder-7B · vllm · openai api▉

One SDK. One console.

The same SDK to serve your model.
A console for all of it. Monitor every job, endpoint calls, cluster health and adjust the nodes as you need.
Try different vLLM params and see how the model changes its behaviour.
Read the SDK docs
deploy.pypython sdk
job = sdk.rt_submit_inference(    name="Chat Assistant",    inference_yaml="inference.yaml",    scripts_dir="scripts/",)

Serve something this evening.

Billed per GPU-minute. No token, request or traffic fees.

Deploy your first endpoint ›Read more