Sign inStart now ›

Your loop running on many GPUs

Wrap the training loop you already have in one function and run it across sites. Iterate faster, ship it on time

Run your first job ›How it works

Inside one training round

Your data is split into shards, one per GPU.

Each worker gets its own slice of the dataset. Nothing else about your data changes.

serveraggregationFedAvg · round 1✓ checkpoint saved · round 1 → 2gpu 0A4048 GBgpu 1A4048 GBgpu 2A4048 GBshard 0shard 1shard 2w↑w↑w↑w↓w↓w↓$split the dataset · 3 shards → 3 gpus▉
serveraggregationFedAvg✓ checkpoint · round 2gpu 0A40 · 48 GBgpu 1A40 · 48 GBgpu 2A40 · 48 GBshard 0shard 1shard 2w↑w↑w↑w↓w↓w↓$split the dataset · 3 shards → 3 gpus▉

One SDK. One console.

The same SDK for training and serving. Submit a job, promote a checkpoint.
A console for all of it. Every job, progress, files, logs and cluster telemetry.
Start from a checkpoint. Progress is saved every round, so if some part falls off you resume from the last checkpoint.
Read the SDK docs

Your loop, moved once.

The body of the loop stays. It moves into one function that returns the weights and the sample count.

train.py, today1 GPU
model = TinyCNN(num_classes=10) for _ in range(EPOCHS):    train_one_epoch(model, DATA_DIR) torch.save(model.state_dict(), "out.pt")
model_def.pyN GPUs
def fl_train(model, env, out_dir) -> dict:    # your optimizer, your loss, your data    for _ in range(env["EPOCHS"]):        train_one_epoch(model, env["DATA_ROOT"])    return {"weights": model.state_dict(),            "samples": n}

Run your first job.

Billed per GPU-minute.

Run your first job ›Browse the recipes