Skip to content

Aether is a Kubernetes service mesh data plane that runs one Envoy per node instead of one per pod, capturing traffic transparently and mTLS-ing every hop with SPIFFE identities.

Written in Go. Routed with the Gateway API. HTTP/3 between proxies. Each node receives only the configuration its own pods actually use.

Get started → Architecture

Apache-2.0 · images and charts on quay.io/aethermesh

The shape of it

graph TD
    subgraph node["Node (DaemonSet)"]
        Pod["Workload Pod"]
        CNI["CNI Plugin<br/><i>TPROXY capture (TCP + UDP) · DNS DNAT · endpoint registration</i>"]
        Agent["Agent<br/><i>xDS · CNI server · SPIRE bridge</i>"]
        Proxy["aether-proxy<br/><i>custom Envoy · PID 1: proxy-supervisor (hot restart)</i>"]
        MeshDNS["mesh-dns<br/><i>own DaemonSet · snapshot-fed resolver</i>"]
        SPIRE["SPIRE Agent<br/><i>workload identity</i>"]
        UdsCsi["uds-csi<br/><i>csi.aether.io · per-pod socket tmpfs</i>"]

        Pod == "captured TCP + UDP<br/>(original destination kept)" ==> Proxy
        Pod -. "DNS :53 (CNI DNAT)" .-> MeshDNS
        Agent -. "record snapshot (file)" .-> MeshDNS
        CNI -. "register (gRPC/UDS)" .-> Agent
        UdsCsi -. "socket volume" .-> Pod
        Agent -. "xDS, demand-scoped<br/>LDS·CDS·EDS·RDS·SDS·ODCDS" .-> Proxy
        Agent -. "SPIFFE Broker API" .-> SPIRE
        SPIRE -. "X.509 SVIDs (via SDS)" .-> Proxy
    end

    Peer["Peer node<br/><i>aether-proxy → workload pod</i>"]
    Proxy == "mTLS (SPIFFE): HTTP/3 over QUIC · HTTP/2 · TCP<br/>UDP in plaintext" ==> Peer

    Registrar["Registrar<br/><i>in-cluster Deployment, active/active</i>"]
    Agent -. "register · watch · list" .-> Registrar

    Registry[("External Registry<br/>etcd · Kubernetes")]
    Registrar -. "sync + persist" .-> Registry

    OTel["OTel Collector<br/><i>metrics · traces</i>"]
    Agent -. "OTLP push" .-> OTel
    Proxy -. "stats sink + aether_stats" .-> OTel

Solid arrows are the workload data path; dashed arrows are control plane and telemetry.

Four decisions

One proxy per node

A single aether-proxy DaemonSet carries the traffic for every managed pod on the node, entering each pod's network namespace rather than living inside it. A chained CNI plugin diverts each pod's outbound TCP and UDP to it, TPROXY-style, so the original destination survives and workloads are unmodified: there is no sidecar to inject, size, or restart. Proposal 038 →

Demand-scoped configuration

Each node receives only the clusters, registry watches, and endpoints its local pods actually depend on, declared with the config.aether.io/upstreams annotation, with on-demand CDS for the cold path. Config size tracks the node's footprint, not the size of the mesh. Proposal 004 →

Hitless proxy rollouts

A small supervisor, PID 1 of the proxy container, takes Envoy through a cross-pod hot restart with two-phase connection draining. In the most recent eight-hour soak written up, 9,180,000 of 9,180,000 requests were answered 2xx across 35 rollout steps of the proxy, the agent, the edge, mesh DNS, the CSI plugin and the workloads; the external prober's liveness tier recorded no failure, and its mesh-DNS tier recorded 37 timeouts on one node. Proposal 001 → · Proposal 042 →

The Gateway API, natively

No bespoke routing CRDs. East-west traffic uses GAMMA — HTTPRoute, GRPCRoute, TCPRoute, TLSRoute and UDPRoute parented to a Service — and north-south uses the same API against the optional edge gateway. The GATEWAY-HTTP profile passes Core 36/36 and Extended 10/10 and the MESH-HTTP profile Core 7/7, and conformance is gated in CI. Proposal 018 → · Proposal 024 →

What it does today

  • HTTP/3 between proxies. HTTP requests cross the mesh over QUIC with the caller's own SPIFFE identity, with nothing to enable: a per-source cluster is fetched on first use. A weighted GAMMA split and a destination behind the east/west waypoint stay on HTTP/2. How it works →
  • TCP and UDP capture. Raw TCP rides the mesh as an mTLS passthrough and UDPRoute services are reached transparently. UDP is carried in plaintext: there is no DTLS. Proposal 038 →
  • Identity from SPIRE, by pod reference. The agent brokers every pod's X.509-SVID over the SPIFFE Broker API (SPIRE 1.15.2 or later), and an init container holds a mesh pod's application until its identity exists. Proposal 036 →
  • Retries that know what was delivered. A request that never reached the application is retried on another endpoint whatever its method; one the application had received is replayed only when the method is idempotent. What the mesh retries →
  • Workloads that serve on a Unix socket. The socket lives in a per-pod volume mounted by the mesh's own CSI node plugin, csi.aether.io; callers still dial the pod over mTLS. Proposal 039 →
  • Agent rolls with a sub-second gap, opt-in. A rolled node agent's successor can start beside it as a standby and take the node over when a lock is released. Proposal 041 →
  • An edge gateway and more than one cluster, both optional. North-south ingress with HTTP/3 as an opt-in; endpoints, routes and a per-node tunnel across clusters on the etcd backend. Edge → · Multi-cluster →
  • Signed releases. Images and charts on quay.io/aethermesh, signed keyless with cosign. Verifying a release →

The design record

Every decision above was argued in writing before it was built, and the argument is published as it was written — status, dead ends and all. These are the five most recent.

044Meshed pods reach non-mesh Services without kube-proxy

043A mesh service is addressed by its real ports, and nothing else

042sortie as the soak's load driver — k6 retired

041Surge roll for the node agent — a standby agent takes the node over

040Publish to Quay (quay.io/aethermesh)

All proposals →

Install

Two charts: the CRDs first, so they can be upgraded independently, then the system. SPIRE 1.15.2 or later, with its Broker Endpoint enabled, comes first: see the prerequisites.

# Pick the published version: <X.Y.Z>-<full git sha of the release>.
VERSION=<X.Y.Z>-<commit>

# 1) CRDs (MeshConfig, HTTPFilter, EdgeConfig, EndpointPolicy) — install/upgrade first.
helm upgrade --install aether-crds \
  oci://quay.io/aethermesh/chart-crds \
  --version "$VERSION"

# 2) First install only: the namespace, labelled for Pod Security admission (the
#    agent, proxy, mesh-dns and uds-csi pods need hostNetwork, hostPath volumes
#    and NET_ADMIN).
kubectl create namespace aether-system
kubectl label namespace aether-system \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/audit=privileged \
  pod-security.kubernetes.io/warn=privileged

# 3) The system: agent + proxy + mesh-dns + registrar + controller.
helm upgrade --install aether \
  oci://quay.io/aethermesh/chart-aether \
  --version "$VERSION" \
  --namespace aether-system --create-namespace \
  --set clusterName=my-cluster \
  --set meshDomain=aether.internal

Aether is built by one person working with AI coding agents, which write most of the code, tests and documentation under that person's direction. It is soak-tested on a real cluster and gated on Gateway API conformance in CI, but it has no support commitment and no stability guarantee: breaking changes still land between chart versions. Read the proposals and the conformance baselines before you run it.