Gentoo Build Swarm (Archived)

Status: Archived / Decommissioned. Depends on Argo OS (Gentoo), which is no longer in use.

The Build Swarm was a distributed package compilation system for Gentoo Linux. A coordinator node distributed emerge build jobs across multiple Proxmox VMs based on available RAM, CPU, and build queue depth.

Architecture

Coordinator (scheduler + binhost)
  ├── Worker VM 1 (8 cores, 16GB RAM)
  ├── Worker VM 2 (4 cores, 8GB RAM)
  └── Worker VM 3 (4 cores, 8GB RAM)

• Coordinator — Accepted build requests, split dependency trees into parallelizable units, assigned to workers, collected binary packages
• Workers — Lightweight VMs with build agent, ccache, tmpfs for portage temp files, distcc for distributed compilation
• Binhost — HTTP server hosting compiled binary packages (.tbz2) for quick installs on other machines

Key Features

• Dependency-aware scheduling — Understood emerge dependency trees, parallelized independent builds
• Resource-based assignment — Jobs assigned based on worker's available RAM (large packages like Chromium routed to high-RAM workers)
• Incremental binhost — Only rebuilt packages with changed USE flags or version bumps
• ccache integration — Compilation cache shared across rebuilds
• Webhook triggers — Gitea push to overlay repo triggered rebuilds of affected packages

Why It Was Archived

Argo OS (Gentoo) was replaced by openSUSE Tumbleweed, which uses pre-compiled binary packages. The build swarm concept had no purpose without a source-based distribution.

Versions

Version	Status	Notes
v3	Last active	Python, improved scheduling
v4	Prototype	Never completed
gentoo-build-swarm	Original	Basic job distribution

Related

• Argo OS — The OS this was built for
• APKG — Custom package format (also archived)