MoreGPU

Turn the GPUs — or CPUs — you already own into one pooled, sealed, verified, adaptively-throttled compute pool. One-liner install, any OS.

WebGPU · Metal / Vulkan / D3D12 AES-256-GCM sealed Ed25519-signed results adaptive per-user throttle Apache-2.0

GitHub repo Run it Can it run what I run?

submit → shard → seal → compute on each worker's GPU/CPU → pool → verify

What it is

An admin runs a coordinator. Worker machines join with a one-liner over an outbound WebSocket and contribute GPU or CPU compute. Jobs are sharded across the fleet, each work unit is AES-256-GCM sealed on the wire, computed on each worker (on a GPU worker every kernel — tiled matmul, elementwise, softmax/layernorm — runs on the GPU via WebGPU → Metal/Vulkan/D3D12; CPU otherwise), then pooled, verified against a CPU reference, and returned with an Ed25519 signature from the worker that produced it. The whole fleet is presented as one virtual GPU with a Slurm-like job queue. It is an honest, verified fp32 linear-algebra service — not a CUDA replacement or a black-box model runner.

Run it — three steps

1. Start the admin server. Its first run prints a wizard with this pool's tokens and the exact worker command. Add --worker (plus --allow-run --allow-sys) and your own machine joins as admin-slot, so you can submit jobs immediately with no separate worker — just like a local GPU.

deno run --allow-net --allow-env --allow-read --allow-write \
  https://raw.githubusercontent.com/ArioMoniri/moregpu/main/apps/coordinator/server.ts

2. Join any machine as a worker with the join token (installs Deno if needed; GPU if present, else CPU).

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/ArioMoniri/moregpu/main/scripts/install.sh \
  | MOREGPU_SERVER=wss://ADMIN:8787/ws MOREGPU_TOKEN=<join-token> sh

# Windows (PowerShell)
$env:MOREGPU_SERVER="wss://ADMIN:8787/ws"; $env:MOREGPU_TOKEN="<join-token>"
irm https://raw.githubusercontent.com/ArioMoniri/moregpu/main/scripts/install.ps1 | iex

Add MOREGPU_SERVICE=1 to install a reboot-surviving, self-healing service.

3. Submit a job (admin token) — benchmark mode, or send your own tensors (data mode) and get results back.

curl -X POST http://ADMIN:8787/submit -H "authorization: Bearer <admin-token>" \
  -H "content-type: application/json" -d '{"kernel":"matmul","size":1024}'

What is a token?

Every pool mints its own secrets on first run and writes them to .moregpu-server.json — no one shares another pool's credentials. Two are yours to hand out; one never leaves the server.

👑 Admin token

Full control: submit jobs, read every job's input and output, read /workers, /jobs, /metrics. Send only over TLS, in the Authorization header.

🖥️ Join token

Enrolls a machine as a worker. A valid join token earns the AES-256-GCM tenant key — treat it like a password; send only over wss://.

🔒 Tenant key

32 random bytes; never displayed. Seals / unseals shard payloads on the wire. Stays in .moregpu-server.json.

MoreGPU is single-trust-domain: enrolled workers hold the tenant key, so sealing protects data on the wire, not from the workers. Full model + hardening/TEE roadmap in SECURITY.md.

Adaptive per-user throttle

Each worker samples its own machine's load and lowers its pool duty as the machine's user gets busier, keeping total utilization under a cap (default 85%). Use the PC harder → the pool quietly steps back, and power draw stays low. CPU-only machines contribute too.

Isolated, sealed, signed

Every server's wizard mints its own admin token, worker join token, and encryption key. Work units are AES-256-GCM sealed on the wire; each result is Ed25519-signed by the worker and verified by the coordinator. Optional native TLS (wss://).

Compute kernels

matmulvector_addvector_mul saxpyrelugeluscale softmaxlayernorm

Enough to compose a real attention block and small MLPs (all fp32, verified). Extensible — add a WGSL kernel plus a CPU reference to support a new op. See AI usage.

Can it run what I run?

An honest, verified fp32 linear-algebra service — ✅ native · 🧩 compose from primitives · 🟡 partial · ❌ not supported.

Workload		How / why
Dense fp32 matmul / GEMM	✅	Workgroup-tiled WGSL GEMM on the GPU (shared-memory tiling), verified. fp32 only, no tensor cores → correct but slower than cuBLAS.
Elementwise + activations	✅	add / mul / scale / saxpy / relu / gelu — WGSL kernels that run on the GPU on a GPU worker (CPU otherwise). Memory-bound, so the GPU mainly relieves the CPU.
Softmax / LayerNorm	✅	Row-wise WGSL reductions on the GPU on a GPU worker (CPU otherwise), match a CPU reference to ~1e-8.
Scaled dot-product attention (1 head)	🧩	`matmul(Q,Kᵀ)→scale→softmax→matmul(·,V)` — verified. Not flash-attention, but the GPT-2 demo adds a client-side KV cache.
Transformer block / small MLP inference	🧩	Compose LN → matmuls → attention → FFN from the SDK; split a model across workers with weight residency.
Small LLM (real GPT-2 124M)	🟡→✅	A real GPT-2 runs on the pool (`examples/llm_infer.py`) matching Hugging Face token-for-token, now with a client-side KV cache. On the native torch worker the resident-model path serves at up to ~66 tok/s warm on an unloaded machine (~6–23 tok/s under load), exact match (`examples/llm_serve.py`).
Fast small-LLM serving (low latency)	🟡	The new native torch worker (`apps/worker/worker_torch.py`, Apple MPS / CUDA) holds a model resident on-device and runs the whole forward per call — one round-trip per token, exact match. Opt-in native tier, not the zero-install WGSL worker.
Training / fine-tuning (LoRA)	🟡	LoRA fine-tuning on the torch worker — single-worker (`examples/lora_finetune.py`) and distributed across N workers via DiLoCo (`examples/lora_distributed.py`): each worker trains its own shard, the coordinator averages adapters + an outer Nesterov step. Verified out-of-band: single-worker bit-for-bit vs a seeded reference, DiLoCo to fp tolerance (~2e-5); run live across a heterogeneous Colab T4 (CUDA) + Mac (MPS) fleet (loss 4.45→1.64/3 rounds). Synchronous DiLoCo; async + secure aggregation are next.
Modern architectures (RMSNorm/RoPE/SwiGLU/GQA)	🟡	Qwen3-0.6B runs on the pool matching Hugging Face token-for-token (`examples/generic_infer.py`). Forward-only, ~minutes/token — a portability proof, not a speed win.
Large / out-of-core matmul	🟡	Sharded across workers + pooled — bounded by WGSL cores + WAN, not NVLink.
Big-model inference across machines (pipeline sharding)	🟡	Pipeline-parallel sharding (`examples/llm_shard.py`): split a model's transformer layers into contiguous stages, one per torch worker, piping only activations (never weights) — the low-bandwidth path. Works for GPT-2 and Llama-family (RMSNorm/RoPE) — exact match for GPT-2 (6+6) and SmolLM-135M (15+15) split across 2 workers; each worker holds only its stage. No int8/tensor cores.
Custom CUDA/PTX kernels · tensor-core/int8 GEMM · Stable Diffusion · NVENC	❌	The portable WGSL path is compute-only fp32/fp16. The opt-in torch worker does run on CUDA via PyTorch on NVIDIA — verified live (GPT-2 served on a Colab T4 through the pool, ~44 tok/s, exact match). cuBLAS uses tensor cores automatically for large fp16 GEMMs, but small-model decode is memory-bound (measured fp16≈fp32 on a T4). No custom CUDA/PTX kernels, no int8 GEMM, no graphics/codec paths.

It runs the small, correct primitive set it ships on your real GPU with verified results, and lets you compose them into attention, transformer blocks, and small classifiers. An opt-in native torch worker adds device-resident model serving (fast GPT-2, exact match) and LoRA fine-tuning — single-worker and distributed (DiLoCo). It is still not a drop-in for big-model (7B+) inference, tensor-core speed, CUDA kernels, or graphics.

Admin dashboard

Observability (Prometheus + Grafana)

A Prometheus /metrics endpoint exposes fleet size, queue depth, throughput and per-worker contribution. A turnkey Prometheus + Grafana bundle with a pre-provisioned dashboard ships in config/observability — docker compose up, then Grafana on :3000.

MoreGPU Grafana dashboard — fleet size, queue depth, total units pooled, jobs done/failed, average user utilization and pool duty, and per-worker share and units — The pre-provisioned Grafana dashboard, live: fleet size, queue depth, units pooled, jobs done/failed, avg user-util & pool-duty, and per-worker share + units.

Use boundaries

⚠️ Run workers only on machines you own or are explicitly authorized to use. Running on free third-party platforms — Google Colab free, GitHub Actions and other CI runners, free/trial VPS — is usually prohibited by their Terms of Service. MoreGPU ships a generic authorized-runtime worker joiner and no automation for evading any platform's Terms of Service, quotas, or bot-detection. All legal and compliance responsibility is the operator's. See ACCEPTABLE_USE.md.