Solution

The Kernel Concept

ResonTech is structured around one central metaphor: a kernel. Like an operating system kernel, it sits between your code and the raw hardware, managing resources, isolating workloads, and handling failures transparently.

The kernel serves both primary ML workload types from a single control plane:

Both runtimes share a common infrastructure layer: Coordinator, Dispatcher, Message Broker, Worker Registry, Job Scheduler, and Billing Metering. You submit a training job and an inference deployment through the same API, manage them on the same dashboard, and pay on the same pay-as-you-go invoice.

The Two-Sided Platform

ResonTech operates as a two-sided marketplace with the kernel at the center. GPU providers supply raw compute; ML teams consume it. The kernel routes, orchestrates, and guarantees delivery.

Core Service Components

Main Backend (NestJS API)

The central control plane and client-facing API. Built with NestJS (TypeScript), it is the primary entry point for ML clients, the web dashboard, and the Python SDK.

Coordinator (Intelligent Job Router)

The scheduling brain of the platform. When a job is submitted, the Coordinator:

• Parses GPU requirements (VRAM hint, GPU type, shard count)
• Filters eligible workers (free VRAM ≥ job hint, availability = true)
• Ranks candidates by score: uptime × hardware tier × geo affinity × current load
• Selects the top N workers (N = shard count for training, N = replica count for inference)
• Assigns jobs via the async Message Broker
• Monitors heartbeats and triggers replacement on failure

Dispatcher (Kubernetes Cluster Manager)

Bridges high-level job intent into running Kubernetes infrastructure via gRPC. Responsible for the full cluster lifecycle: creating K8s resources for each training job, managing inference deployments with HPA and load balancers, and tearing down clusters after completion.

Worker Agent (Node-Side Runtime)

A Docker container installed on each GPU provider's machine. It registers with the Coordinator, sends heartbeat telemetry, and executes job commands: pull the FLARE image, initialize the FL client, begin training on the assigned shard, write checkpoints, serve inference.

S3 Workspace & Storage Layer

Every user account gets a private S3-compatible bucket automatically provisioned at account creation. The bucket is backed by Garage — a self-hosted, geographically distributed S3-compatible object store. Your data stays under ResonTech infrastructure, never in a third-party public cloud unless you configure a Private Cluster with your own storage.

Workspace folder layout

How uploads work

The API generates presigned PUT URLs (valid 15 minutes) for uploads. Your browser or SDK sends data directly to Garage — zero bytes flow through the API server.

For large files (≥ 10 MB), the platform automatically initiates multipart upload: files are split into 50 MB chunks, up to 5 parts uploaded in parallel, with automatic abort on failure. Maximum supported upload size: 500 GB.

How workers access data

At job dispatch, each worker receives a presigned 1-hour GET URL for its assigned shard ZIP. The worker downloads and unpacks the ZIP into /var/tmp/nvflare/data. Your training code accesses this path viapayload["dataset"]["data_root"]. The presigned URL expires after the download — workers never have persistent access to your bucket.

How results are returned

After each FL round, partial metrics are written to files_out/. After the final round, the aggregated model checkpoint is written to model_out/. Downloads use presigned 1-hour GET URLs served directly from Garage — no proxy bottleneck. Large checkpoints download at full network speed.

Training Pipeline — End to End

Fault recovery during training

When a worker node loses connectivity:

• Coordinator detects the missing heartbeat within seconds
• A replacement worker is selected from the available pool
• FLARE server manages partial aggregation (partial round is handled gracefully)
• Replacement node loads the last successful checkpoint
• Training resumes — zero compute lost from the last checkpoint

Inference Pipeline — End to End

Inference autoscaling architecture

Incoming traffic hits a Load Balancer which round-robins across healthy replicas. An HPA Controller monitors both RPS and GPU utilization:

• RPS > threshold → scale out (add replicas)
• RPS < threshold → scale in (remove replicas)
• RPS = 0 for N minutes → scale to zero (billing stops)

Framework Compatibility

• Training frameworks: PyTorch, TensorFlow, JAX, HuggingFace Trainer, Fastai
• Distributed strategies: DDP, FSDP, DeepSpeed ZeRO-2/ZeRO-3, model parallelism, NVIDIA FLARE federated learning
• Model hubs: HuggingFace Hub, local workspace checkpoints, custom containers
• Observability: Weights & Biases, TensorBoard, MLflow (bring your own tracker)
• Inference APIs: OpenAI-compatible REST endpoints for LLM serving, raw tensor outputs for custom models
• Runtime: CUDA 11.8+, NVIDIA GPUs from GTX 1080 Ti to H100 SXM5