Choosing hardware: Training · Docs

What you choose

A training job runs on clients you pick and one FL server the platform picks.

	What it is	GPU	Who picks it
Client (worker)	One machine with one GPU that trains the whole model on its own shard	yes	You, through the job’s requirements
FL server	Hands out the model each round, collects every result, averages them. Trains nothing	no	The platform. See Central servers

Term	Meaning
Shard	One zip of data per client. Shards = clients
Round	Every client trains for local_epochs, the server averages, the next round starts from the average. The server waits for every client
Full fine-tune	Updates every weight
LoRA / QLoRA	Freezes the model and trains small adapters; QLoRA on a 4-bit base. The method sets memory and traffic more than model size does
One GPU per client	A multi-GPU client trains on one card today. Bigger models need a bigger card, not more cards

1. VRAM per client

Training holds weights, gradients, optimizer state and activations. Optimizer state dominates a full fine-tune; base weights dominate LoRA.

Method	Bytes / param	1B	3B	8B	27B	70B
Full fine-tune (Adam, mixed precision)	~16	16 GB	48 GB	128 GB	432 GB	1.1 TB
LoRA (bf16 base)	~2.2	2 GB	7 GB	18 GB	60 GB	154 GB
QLoRA (4-bit base)	~0.6	1 GB	2 GB	5 GB	16 GB	42 GB

Add 10–30% for activations with gradient checkpointing on (the examples turn it on). Activations grow with batch × sequence length, or batch × image size.

One client with…	QLoRA	LoRA	Full fine-tune
8–12 GB (T4, 3060, 4070)	≤ 8B	≤ 3B	Small CNNs, ViT-B, DenseNet
16–24 GB (5070 Ti, 3090, 4090, L4)	≤ 14B; 30B tight	≤ 8B	≤ 1B
48 GB (A6000, L40S)	≤ 32B; 70B tight	≤ 14B	≤ 2B
80–96 GB (H100, RTX PRO 6000)	70B	≤ 32B	≤ 4–5B
141 GB (H200)	70B with room	~50B	≤ 7B

Measured on the example jobs

Example job	Model, method	Min GPU
job_csgo	YOLOv5 detection	4 GB
job_classify	ViT-B/16, full weights, batch 32 fp16	8 GB
job_medical	DenseNet121 at 224 px, batch 32	8 GB
job_embed	Embedding model fine-tune	8 GB
job_llm	Phi-3.5-mini (3.8B) LoRA	12 GB
job_speech	Whisper-small LoRA, 30 s audio	12 GB
job_yolo11	YOLO11m at 640 px, batch 16	12 GB
job_diffusion	SDXL LoRA	24 GB
job_agent	Llama-3.1-8B 4-bit + LoRA r16, 2,048 tokens	24 GB
job_lm_scratch	Pythia-1B from scratch, bf16	80 GB

Set the GPU memory minimum from this table plus activation headroom; the platform checks it on one card. Some families advise against 4-bit training: for Qwen3.5 use bf16 LoRA, which puts a 27B at ~56 GB.

2. How many clients

Force	Rule
Data	Each round should give every client ≥ ~50 optimizer steps: `shard_size ÷ batch_size ≥ 50`. Dataset and batch size cap the useful client count
Time	Rounds wait for the slowest client. More clients, smaller shards, shorter rounds. Measured: DeepLab 63M took 3 days on 3 × A40 vs 9 days on 1 × RTX 5090, 47% cheaper
Where the data is	In a real federation each site that owns data is a client; the count is given

Mixing GPU models works: four runs across two GPU generations landed within 0.09 percentage points. The smallest card sets the model size (every client holds the whole model); the slowest sets the round time, so size shards by card speed. T4 and RTX 2080 Ti have no bf16 and train in fp16.

3. Disk, RAM and CPU per client

Resource	Rule of thumb	Submit form
Disk	Base weights + 2.5 × shard zip + checkpoints + 25 GB. An 8B base needs its 16 GB on every client	Disk
Host RAM	2 × VRAM (data loaders, decoding)	RAM
CPU	4 cores for text; 8+ for images and audio, where decoding is often the bottleneck
Driver	R570 or newer

4. The FL server

The training job’s central server. The platform picks it at submit. Each round it sends the current model to every client, waits for all of their results, averages them weighted by sample count, and starts the next round. It also reports the job’s status. It trains nothing and needs no GPU.

What it needs depends on what travels each round:

Method	Server RAM	Server disk	Network per round, N clients
LoRA / QLoRA, any base	< 1 GB	MBs	N × 10–500 MB
Full weights	~12 bytes per parameter	2 copies of the model	2 × N × 4 bytes per parameter

worked example · 8B full fine-tune, 8 clients

per round  = 2 × 8 × 4 bytes × 8B params = 512 GB
at 10 Gbps ≈ 7 minutes      at 1 Gbps ≈ over an hour
server RAM ≈ 12 bytes × 8B  ≈ 96 GB

Setting	Default	Where you set it
Free RAM	≥ 1 GB	Filters → Central server RAM
Free disk	No minimum	Filters → Central server disk
A whole server for this job	Shared	Filters → Dedicated central server
A server only your jobs use	None	Order a central server (workers step or Support)

LoRA and QLoRA at any model size: the defaults work.
Full weights above ~1B: set RAM to ~12 bytes per parameter, disk to two copies of the model, turn on dedicated, and raise the result timeout from its 300 s default.
Many clients or long runs: a reserved server keeps other jobs off it.

A server reserved for you is used first; otherwise the platform picks a shared one that meets your minimums. If none does, the launch stops and the wizard offers to order one.

warn

The FL server has no failover. If it goes down, the round is lost and the job fails; resubmit it.

5. Rounds, time and cost

Total optimizer steps decide quality. Match the centralized recipe:

formula

steps_per_round = shard_size ÷ batch_size
num_rounds      ≈ centralized_total_steps ÷ (steps_per_round × local_epochs)

wall-clock      ≈ rounds × (slowest client's local time + exchange)
cost            ≈ clients × GPU-hour price × wall-clock

Fewer local epochs and more rounds beat the reverse. Measured: job_classify at 4 clients × 5 rounds × 148 steps matched the centralized step budget and its accuracy (89.4% vs 89.1%).

Example jobs on their minimum GPU	Typical wall-clock
Classification, small CNNs	30–90 min
Speech, 3B-LLM LoRA	3–6 h
YOLO11 on COCO	6–10 h
SDXL LoRA	8–12 h
8B QLoRA on 113k examples	12–18 h

When a client can’t keep up

Rounds wait for every client, so one client’s failure fails the job.

What goes wrong	What happens	Prevent it with
A client runs out of VRAM	Its process dies; after the 600 s heartbeat timeout the job fails	GPU memory minimum with headroom; gradient checkpointing
One client is much slower	Every round waits; the others idle and bill	Same VRAM tier; shards sized by card speed
A client’s disk fills	Unpacking or a checkpoint write fails	Disk minimum (step 3)
A full-weight result misses the upload window	The round result is rejected after 300 s	Raise the result timeout; LoRA above ~1B
The FL server runs out of RAM	Aggregation dies	~12 bytes per parameter of server RAM; dedicated

note

A one-client, one-round run of the exact recipe shows the real VRAM, step time and unpacked shard size for a few GPU-minutes.

By job type

Type	Model	Method	Per client	Per-round transfer
LLM / agent tuning	Llama-3.1-8B, Phi-3.5-mini	QLoRA / LoRA	24 GB / 12 GB	10–25 MB
LLM, larger base	Qwen3.5-27B	bf16 LoRA	80–96 GB	100–500 MB
Image classification	ViT-B/16, DenseNet121	Full weights	8 GB	~350 MB
Detection	YOLO11m, YOLOv5	Full weights	12 GB / 4 GB	~80 MB
Segmentation	DeepLabV3+ 63M	Full weights	8–12 GB	~250 MB
Embeddings	Sentence-embedding model	Full weights	8 GB	Small
Speech	Whisper-small	LoRA	12 GB	5–10 MB
Image generation	SDXL	LoRA	24 GB	Tens of MB
Pretraining from scratch	Pythia-1B, ImageNet ResNet	Full weights	80 GB	4 GB each way

Full-weight training is practical up to ~1B parameters, since weights travel in fp32 every round. Above that, use LoRA or QLoRA: the adapter is what you get back and what you serve.

Before you submit

	The platform checks at submit	You check
VRAM on one card	yes	That you added activation headroom (step 1)
Disk and host RAM per client	yes	The numbers (step 3)
GPU generation and driver	yes	Compute 8.0+ for bf16 recipes
FL server RAM, disk, dedicated	yes	Full-weight models (step 4)
Steps per round ≥ ~50	no	shard_size ÷ batch_size

Worked examples

Goal	Sizing
Fine-tune an 8B chat model on support tickets, 4 sites	QLoRA on 24 GB cards (the job_agent recipe). Four clients, one per site. LoRA traffic is MBs, so any server works. 12–18 h on 113k examples.
Classify 50k product photos across 3 warehouses	ViT-B full weights fit 8 GB; three clients on any 12 GB card; ~350 MB per client per round. 5 rounds of 1 epoch, under 2 hours.
Train a 27B model without 4-bit	bf16 LoRA at ~56 GB: one 80 or 96 GB card per client (H100, RTX PRO 6000).
Full fine-tune a 7B model	~112 GB of state per client doesn’t fit one card today. Use LoRA.

The shard count sets the client count: Dataset format & sharding.
Measured time and cost of a real run: Training benchmarks.

PreviousSubmit via UI wizard Next Dataset format & sharding