Self-hosting Loam with GitOps

Proposal The deployment design in this post is under review. Today the engine runs as a single binary (operon dev, operon standalone, operon cluster) with TiKV from tiup playground in development. The open-source boundary at the end is approved.

A platform that lives in your bucket should also be deployable in your cluster, by you, without calling us. The goal is one deployment path: the same Git repository and charts deploy a laptop k3d cluster, a self-hosted cluster on your own hardware, and the managed Loam Cloud.

The piecesCopy Anchor Link

The Loam operatorCopy Anchor Link

The proposal forks Clever Cloud's Kubernetes operator (MIT, Rust, kube-rs) as the skeleton of a Loam operator, keeping its MIT notice. What carries over is modest and we say so: a controller registry, finalizers, event recording, a metrics server and a hardened Helm chart, about 1,300 lines. Its own custom resources provision Clever Cloud add-ons through Clever's API, so they are dropped. The operator adds four resources:

Object stores sit behind a provider trait that reports capabilities, and Loam refuses a provider without conditional writes, because the write-ahead log's commit protocol depends on them.

We also evaluated Clever Cloud's edge proxy, Sōzu. It is well engineered and reconfigures without restarts, but it is AGPL-3.0 and, as far as its documentation shows, has neither HTTP/3 nor gRPC routes, both of which the runtime needs. Envoy stays the edge. The same evaluation proposes Biscuit (Apache-2.0, now at the Eclipse Foundation, started at Clever Cloud) for sandbox tokens inside the runtime (part 5).

Sync wavesCopy Anchor Link

Argo CD would apply an app-of-apps in waves. Each wave waits for the previous one to be healthy, which needs custom health checks for the child Application resources (Argo CD has none built in), the TiDB Operator's resources and Loam's own.

deploy/
  helm/loam-stack/          # umbrella chart; every component can be switched off
  gitops/
    bootstrap/argocd/       # Argo CD and its health checks
    root.yaml               # the app-of-apps
    apps/                   # one Application per wave
    envs/{dev-k3d,selfhosted,cloud}/values.yaml

The costs, plainly: Argo CD is heavier than Flux (an application controller, a repo server, Redis and a server), which matters on a laptop and in small clusters, and TiDB Operator v2's resources are still v1alpha1, so each Loam release pins an operator version. A local k3d profile runs one replica of everything; PD, TiKV and TiDB together peaked at about 3.2 GB of memory in our development playground.

What stays true without KubernetesCopy Anchor Link

The engine is still one binary. operon dev runs every role in one process with the openraft metastore and a local directory or any S3 bucket, and needs no TiKV. TiKV becomes necessary for Loam Live, for jobs and for clusters that use the TiKV metastore.

Open source and the managed cloudCopy Anchor Link

The line is one rule: anything needed to self-host Loam as a single organisation is open source, under Apache-2.0. Anything needed only to run Loam as a multi-tenant paid cloud is part of the managed Loam Cloud. The open repository must never depend on the managed cloud; the cloud consumes the same hooks and APIs any self-hoster can use.

Quotas show the split well: the engine enforces whatever limits it is given; the managed cloud decides what those limits are for each plan. And we do not hold back reliability or performance features from open source. The hot tier is in the open engine today, and the planned replicated write-ahead log will be too, not behind a license key. A few calls are still open, such as enterprise SSO and SCIM.

Where this standsCopy Anchor Link

That is the end of the platform series. The other series on this blog, What Loam is built on, takes each open-source project under Loam in turn: what it is, how it works inside, why we chose it, exactly how we use it, its license and its current limits.