Configuration Reference¶
A structured reference for configuring Aether: the aether Helm chart values,
the CRDs, the CLI flags of each binary, and the labels/annotations
workloads use. Values and defaults are sourced from
charts/aether/values.yaml,
common/constants/, and each binary's Cobra flags — those
files are authoritative if anything here drifts.
Configuration has two layers:
- Deploy-time system config —
aetherchart values, set once and inherited by every component (agent, proxy, registrar, controller, edge). - Runtime proxy observability — the namespaced
MeshConfigCR, which overrides only the proxy data plane's access-log / tracing / stats policy without a redeploy.
1. Chart values (charts/aether)¶
Top-level¶
| Key | Default | Purpose |
|---|---|---|
nameOverride / fullnameOverride |
"" |
Override the chart name / fully-qualified resource name. |
namespace.create |
true |
Create the release namespace with privileged pod-security labels (the agent needs hostNetwork + NET_ADMIN). |
namespace.name |
"" |
Namespace all resources deploy into (defaults to the release namespace). |
clusterName |
talos-main |
Cluster name passed to agent + registrar (--cluster-name); used in registry keys. |
controlCluster |
"" |
Cross-cluster config authority (026 EM3). Set to a cluster name → only that cluster's registrar exports GAMMA config and everyone imports only from it. Empty = federated (any peer, highest-version wins). |
debug |
true |
Verbose logging on all components (--debug). |
meshDomain |
aether.internal |
DNS-style domain services are addressed under (<service>.<meshDomain>); also the ODCDS catch-all suffix. |
otel — system-wide telemetry¶
Enable once; every component inherits it. The proxy may override its own access-log/tracing policy via the MeshConfig CR.
| Key | Default | Purpose |
|---|---|---|
otel.enabled |
false |
Enable the OTel MeterProvider + push telemetry everywhere. |
otel.endpoint |
"" |
OTLP gRPC collector host:port (insecure). Empty disables OTLP + the proxy/CNI stat sink. Deploy-time value baked into the CNI plugin and Envoy bootstrap (never read from a runtime ConfigMap). |
otel.logs |
false |
Export component logs over OTLP (also tee'd to stderr). |
otel.traceSampleRate |
0.1 |
Head-sampling ratio (0.0–1.0); bounds exported spans only. |
otel.traceExport |
false |
Export spans over OTLP (needs a collector traces pipeline). |
spire — system-wide mTLS¶
enabled is the mesh-wide mTLS switch; the rest is per-component plumbing.
| Key | Default | Purpose |
|---|---|---|
spire.enabled |
true |
Mesh-wide mTLS switch (agent, registrar, controller webhook cert). |
spire.workloadSocketPath |
/run/secrets/workload-spiffe-uds/socket |
Workload API socket (the csi.spiffe.io mount). |
spire.adminSocket.hostPath |
/run/spire/agent/sockets/csi.spiffe.io/admin |
SPIRE agent admin (Delegated Identity) socket the agent uses to mint proxy SVIDs. |
spire.adminSocket.mountPath |
/run/spire/admin-sockets |
Where the admin socket is mounted. |
spire.adminSocket.socketName |
admin.sock |
Admin socket filename. |
meshConfig — proxy MeshConfig seeding¶
| Key | Default | Purpose |
|---|---|---|
meshConfig.createDefault |
true |
Seed the singleton MeshConfig (default) on first install only — never overwritten on upgrade (operators own it via kubectl). |
meshConfig.proxy |
{} |
The spec.proxy seeded into that CR (protojson field names). Empty = proxy inherits everything from system config. |
agent¶
| Key | Default | Purpose |
|---|---|---|
agent.gamma |
true |
GAMMA east-west L7 routing (018): watch HTTPRoutes and GRPCRoutes parented to a Service (plus ReferenceGrants and HTTPFilter attachments) and apply them on both the explicit outbound path and the transparent-capture path. MESH-HTTP Core conformance-green. Safe without the Gateway API CRDs (CRD-detected, degrades with a warning); false is a kill switch. |
agent.cniConflistReassert |
true |
Keep aether chained in the node's active CNI conflist (#645): the agent watches /etc/cni/net.d (read-write mount) and re-appends the aether-cni entry whenever a competing writer strips it — kube-flannel cp -fs its ConfigMap template over 10-flannel.conflist on every flannel pod recreation, which a Talos bootstrap-manifest re-sync triggers, silently unmeshing every pod started afterwards. Never creates a conflist of its own; false is a kill switch. |
agent.importConfig |
false |
Cross-cluster config import (026): poll the registrar for peer-exported GAMMA projections and materialize them (merged with local; local wins). Pairs with registrar.registryBackend=etcd. |
agent.eastWestWaypoint |
false |
East/west waypoint (019): dial cross-cluster endpoints at their node's routable IP + the fixed tunnel port 18009 instead of the (unroutable) pod IP; this node's host-network proxy SNI-forwards inbound tunnel traffic to local pods. Intra-cluster stays direct pod-to-pod. Needs cross-cluster endpoint visibility (shared or replicated etcd) + a shared SPIRE trust domain. |
agent.captureRedirectAllDefault |
true |
Redirect-all as the DEFAULT for managed pods (022 Step 4); opt out per-pod with capture.aether.io/redirect-all="false". false = per-pod opt-in via the same annotation set to "true". (Transparent capture itself and the passthrough chain are unconditional since proposal 031.) |
agent.meshDns |
true |
Per-pod mesh DNS (018). Gates BOTH halves: the agent's in-process resolver (which writes the record snapshot) and the separate aether-mesh-dns DaemonSet that serves pods from it. |
agent.meshDnsUpstream |
[] |
Upstream resolver(s) for non-mesh queries, passed to the mesh-dns daemon (--mesh-dns-upstream), not the agent. Empty = the daemon's own resolv.conf (kube-dns). |
agent.meshDnsDaemon.image.* / .resources |
repo+digest placeholders | The slim mesh-dns image, pinned separately from the agent's (#583) — the DaemonSet ships only the /mesh-dns binary. Rendered when agent.meshDns is true. |
agent.image.* |
repo+digest placeholders, pullPolicy: Always |
Digest-pinned image; mirror by overriding repository alone. |
agent.resources.{requests,limits} |
cpu 200m, mem 64Mi |
The agent reaches the registry only through the registrar (gRPC) — it carries no AWS credentials. AWS is used solely by the registrar's dynamodb backend.
proxy — per-node Envoy DaemonSet¶
| Key | Default | Purpose |
|---|---|---|
proxy.enabled |
true |
Deploy the per-node Envoy. Disable to run only the agent. |
proxy.image.repository |
ghcr.io/bpalermo/aether/aether-proxy |
External image built by the //proxy workspace, tag-pinned. |
proxy.image.tag |
(commit SHA) | The publishing commit. |
proxy.logLevel |
info |
Envoy log level. |
proxy.jsonLogs |
true |
Envoy application logs as one JSON object per line. |
proxy.hotRestart.baseId |
0 |
Envoy hot-restart tunables (mechanism is not optional; see proposal 001). |
proxy.hotRestart.drainTime |
10s |
Graceful connection-close window for the draining epoch. |
proxy.hotRestart.parentShutdownTime |
15s |
When the previous epoch is terminated (must exceed drainTime). Also the supervisor's admin re-verify budget: the epoch-identity probe re-confirms on a fresh connection every parentShutdownTime/3 (floor 2s, ceiling 15s), so a cross-pod takeover is diagnosed while the draining parent still lives. Below ~6s the floor takes over and the supervisor logs the lost margin at startup; raising it also delays successor-pod readiness by the same amount. |
proxy.hotRestart.handoffDeadline / adminUnresponsiveDeadline |
0 |
Supervisor watchdogs (0 = built-in defaults). |
proxy.hotRestart.shmHostPath |
/run/aether/shm |
Shared-memory hostPath for cross-pod hot restart. |
proxy.udsWorkloads.enabled |
true |
UDS delivery (034). Gates the proxy's /var/lib/kubelet/pods hostPath mount, the agent's --kubelet-pods-dir, and the agent's read access to the EndpointPolicy CRD. Inert until a workload asks for it; turning it off later silently degrades annotated pods to TCP (nothing listens, so their endpoints stay unpromoted). |
proxy.overload.enabled |
true |
Envoy overload-manager graceful-degradation ladder. |
proxy.overload.maxHeapSizeBytes |
402653184 (384Mi) |
Keep at ~75% of resources.limits.memory. |
proxy.resources.{requests,limits} |
cpu 500m, mem 512Mi |
proxy.authzSidecar — external authorization (proposal 027)¶
| Key | Default | Purpose |
|---|---|---|
proxy.authzSidecar.enabled |
false |
Add a node-local authz gRPC sidecar (UDS) + a DISABLED ext_authz filter entry; zero effect until an HTTPFilter (extAuthz) opts a route/service in. |
proxy.authzSidecar.opa.enabled |
false |
Built-in OPA preset (opt-in). |
proxy.authzSidecar.opa.image |
openpolicyagent/opa:1.18.2-envoy-static |
OPA image. |
proxy.authzSidecar.opa.policy |
"" |
Rego policy (ConfigMap-mounted); required when opa.enabled. |
proxy.authzSidecar.image.{repository,tag,args} |
"" / [] |
Bring-your-own authz container (serves envoy.service.auth.v3.Authorization on unix:///run/aether/authz/authz.sock). |
proxy.authzSidecar.timeout |
200ms |
Per-check gRPC timeout. |
proxy.authzSidecar.failureMode |
DENY |
DENY (fail-closed, 403 when unreachable) or ALLOW (fail-open). |
cniInstall — CNI installer init container¶
| Key | Default | Purpose |
|---|---|---|
cniInstall.image.* |
repo+digest placeholders, pullPolicy: Always |
Digest-pinned image. |
cniInstall.resources.{requests,limits} |
cpu 100m, mem 32Mi |
registrar¶
| Key | Default | Purpose |
|---|---|---|
registrar.registryBackend |
kubernetes |
Backend (--registry-backend): kubernetes, dynamodb, or etcd. |
registrar.replicaCount |
2 |
Always 2 (exercises the multi-replica write-behind topology). |
registrar.enableMCS |
false |
Multi-Cluster Services phase 1 (018 + 006): export ServiceExports and materialize ServiceImports + clusterset VIPs. Requires the etcd backend + the MCS-API CRDs. |
registrar.region |
local |
Region owning this registrar's etcd partition (006); keys are /aether/v1/regions/<region>/clusters/<clusterName>/…. One region = one etcd. |
registrar.etcd.endpoints |
[] |
etcd client endpoints (etcd backend). |
registrar.peerEtcd |
[] |
Cross-region replication (006 Phase 2), one entry per peer region: "<region>=<endpoint>[,<endpoint>...]". The leader registrar mirrors this region's own registry subtree verbatim into each peer's etcd under an origin-heartbeat lease (TTL ~30s): if this region dies, its mirror expires on the peers — whole-region failover cleanup with no peer-side GC. Requires the etcd backend + a non-default region. |
registrar.aws.region |
us-east-1 |
AWS region for the dynamodb backend (IRSA; no static keys). |
registrar.aws.roleArn |
"" |
Role ARN annotated onto the registrar ServiceAccount. Empty = no AWS access. |
registrar.service.{port,targetPort} |
443 / 8443 |
gRPC service ports. |
registrar.image.* / registrar.resources.* |
placeholders / cpu 100m, mem 64Mi |
controller¶
| Key | Default | Purpose |
|---|---|---|
controller.replicaCount |
1 |
Leader election is on; extra replicas are warm standbys. |
controller.injectPodNdots |
true |
Pod-mutating webhook injects dnsConfig ndots into managed pods so mesh FQDNs resolve absolute-first (musl/Alpine safety). Pairs with mesh DNS. |
controller.namespaceInjection |
true |
Namespace auto-injection: a pod in a namespace labeled aether.io/managed=true is given the pod label automatically (opt out with aether.io/managed=false). |
controller.webhook.spire |
false |
Webhook serving cert source — decoupled from mesh SPIRE. false = Helm self-signed cert (works out of the box). true = serve with the controller's SPIRE SVID + inject the trust bundle. |
controller.webhook.clusterSpiffeID.create |
true |
When spire=true, create the controller's ClusterSPIFFEID with the webhook Service DNS SANs. |
controller.webhook.clusterSpiffeID.className |
"" |
spire-controller-manager class name; REQUIRED when create=true. |
controller.image.* / controller.resources.* |
placeholders / cpu 50m, mem 64Mi |
edge — north-south ingress gateway (proposals 003/018/021/028)¶
An unprivileged Deployment (Envoy + agent edge) that dials mesh pods directly
over mTLS and routes external traffic via the Gateway API. Disabled by default.
| Key | Default | Purpose |
|---|---|---|
edge.enabled |
false |
Deploy the edge. |
edge.namespace |
aether-ingress |
The edge runs in its own namespace, isolated from the control plane. |
edge.namespaceCreate |
true |
Let the chart create it (baseline PSA). |
edge.replicaCount |
2 |
Gateway replicas (standard RollingUpdate + readiness gate; no hot-restart supervisor). |
edge.gatewayClassName |
aether |
The GatewayClass whose Gateways this edge serves (controller gateway.aether.io/edge). Requires the Gateway API CRDs. |
edge.gateway.create |
true |
Chart-manage a Gateway of that class (HTTP + optional HTTPS listeners). |
edge.gateway.tlsSecretName / tlsSecretNamespace |
"" |
The kubernetes.io/tls Secret for the downstream cert; REQUIRED when tls.enabled + gateway.create. |
edge.gateway.address |
"" |
Pin the Gateway's LoadBalancer IP (021 Phase 2, via MetalLB). Empty = auto-assign. |
edge.gateway.hostname |
"" |
Constrain the chart-managed Gateway's listeners (e.g. "*.example.com"). |
edge.gateway.httpRoutes |
[] |
Declaratively managed HTTPRoutes parented to the chart Gateway (the supported replacement for hand-applied manifests). |
edge.tls.enabled |
false |
Downstream TLS: HTTPS listener (certs per Gateway listener via SDS) + HTTP→HTTPS redirect. The edge→pod hop stays mTLS. |
edge.geoip.enabled |
false |
Emit x-geo-* request headers from a MaxMind DB (028). The x-geo-* namespace is always stripped from client requests. |
edge.geoip.headers |
[country] |
Which headers to emit: country, city. |
edge.geoip.database.secretName / fileName |
"" / GeoLite2-City.mmdb |
The bring-your-own mmdb Secret + key. |
edge.xffNumTrustedHops |
0 |
Trusted proxies in front of the edge (feeds HCM client-address + geoip XFF). |
edge.httpPort / httpsPort |
80 / 443 |
Public listener ports (privileged ports via NET_BIND_SERVICE; pod stays unprivileged). |
edge.routeNamespace |
"" |
Namespace the edge watches Gateways/HTTPRoutes in. Empty = its own namespace. |
edge.service.{type,port,httpsPort,annotations,extraPorts} |
LoadBalancer / 80 / 443 / {} / [] |
The edge's Service; extraPorts exposes TCP/TLS listener ports. |
edge.drain.preStopSeconds |
10 |
preStop sleep holding off SIGTERM during drain (matches proxy.hotRestart.drainTime). 0 disables. |
edge.drain.terminationGracePeriodSeconds |
30 |
Must exceed preStop + Envoy drain. |
edge.admin.{enabled,port} |
false / 9901 |
Envoy admin on loopback only; off by default. |
edge.overload.{enabled,maxHeapSizeBytes} |
true / 201326592 (192Mi) |
Overload monitor (works here; the pod is unprivileged). |
edge.spire.clusterSpiffeID.{create,className} |
true / "" |
Create the edge's ClusterSPIFFEID (when spire.enabled); className required when create=true. |
edge.resources.{requests,limits} |
cpu 200m, mem 128Mi/256Mi |
2. CRDs (charts/crds)¶
All are config.aether.io/v1, Namespaced, structural-but-permissive
(x-kubernetes-preserve-unknown-fields); authoritative validation is the
controller's protovalidate webhook, not OpenAPI. All carry
helm.sh/resource-policy: keep.
| CRD | Kind (short) | Purpose |
|---|---|---|
meshconfigs.config.aether.io |
MeshConfig (mc) |
Per-namespace proxy observability overrides (access logs, tracing, per-pod stats). A namespace inherits the control-plane namespace's MeshConfig field-by-field unless it sets its own (proposal 015). |
httpfilters.config.aether.io |
HTTPFilter (htf) |
The proxy-extension escape hatch (proposal 025): attach a supported Envoy HTTP filter (ext_authz, RBAC, header-to-metadata) at a chosen scope. |
edgeconfigs.config.aether.io |
EdgeConfig |
Edge Envoy tuning (proposal 029): best-practices hardening defaults, HTTP/3 (QUIC, ALPN h3), timeouts/limits. Attached natively via Gateway API parametersRef on the GatewayClass (fleet default) or per-Gateway (override-wins merge). |
endpointpolicies.config.aether.io |
EndpointPolicy |
Service-scoped UDS delivery (proposal 034 Phase 1b): spec.targetRef (kind=Service, same namespace) + spec.udsSocket (<volume>/<file>) declares socket delivery for every pod of a service. The per-pod endpoint.aether.io/uds-socket annotation wins; one policy per Service (lexicographically smallest name wins). Read by the agent only when proxy.udsWorkloads.enabled. |
HTTPFilter scopes (spec.scope, plus the target_refs attachment):
| Scope | Attachment | Applies to |
|---|---|---|
SCOPE_ROUTE (default) |
Gateway API ExtensionRef on an HTTPRoute/GRPCRoute rule | that single route |
| (targetRef) | spec.targetRefs (kind=Service) |
every route of the Service |
SCOPE_CHAIN |
Service targetRef, always-on | the service's capture vhost (one per service; rides the 026 cross-cluster channel) |
SCOPE_INBOUND |
Service targetRef, destination-side | the target service's own pods' inbound listeners (not propagated cross-cluster) |
3. CLI flags¶
Every binary also gets the shared manager flags
(common/manager/flags.go): --debug, --metrics-enabled,
--metrics-bind-address, --otel-enabled, --otlp-endpoint, --logs-enabled,
--trace-sample-rate, --trace-export. (The chart sets these from the values
above.)
agent (node agent — root command)¶
Identity/registrar/SPIRE: --mesh-config (/etc/aether/mesh-config.yaml),
--mesh-domain (aether.internal), --spire-enabled (true), --node-name
(required; doubles as the xDS node identity — the old --proxy-id was retired),
--cluster-name (required),
--registrar-address (aether-registrar.aether-system.svc:443),
--spire-workload-socket.
Node-agent-specific:
| Flag | Default | Purpose |
|---|---|---|
--mounted-registry-dir |
/host/var/lib/aether/registry |
Local pod-data dir for the CNI plugin. |
--kubelet-pods-dir |
/var/lib/kubelet/pods |
Kubelet's pod-volumes dir, mounted into the proxy at the identical host path, through which the proxy reaches a workload's Unix socket (034). Empty disables UDS delivery: pods annotated endpoint.aether.io/uds-socket fall back to TCP loopback. Gated by the chart's proxy.udsWorkloads.enabled. |
--spire-admin-socket |
/tmp/spire-agent/private/admin.sock |
SPIRE admin socket for proxy SVID delegation. |
--gamma |
true |
GAMMA east-west routing (018); default-on kill switch (031). CRD-detected. |
--cni-conflist-reassert |
true |
Re-assert the chained aether-cni entry in the node's active CNI conflist whenever a competing writer strips it (#645). Watches --mounted-cni-net-dir (fsnotify) plus a 60s re-check; only ever appends to an existing, valid conflist that still carries a primary CNI plugin. |
--mounted-cni-net-dir |
/host/etc/cni/net.d |
Host CNI config dir as mounted into the agent (read-write) for the re-assert loop. |
--import-config |
false |
Enable cross-cluster config import (026). |
--control-cluster |
"" |
Trust imported config ONLY from this origin (026 EM3). Empty = federated. |
--east-west-waypoint |
false |
Per-node east/west waypoint for cross-cluster traffic (019); tunnel port is the fixed constant 18009. |
--mesh-dns |
false |
Per-pod mesh DNS (018): answer <svc>.<ns>.<mesh-domain> from the generated mesh Services. Upstream forwarding belongs to the mesh-dns daemon, not the agent. |
--mesh-dns-snapshot-path |
/host/var/lib/aether/registry/mesh-dns/records.json |
Host-persistent record table the in-process resolver writes and warm-loads at boot (and the mesh-dns daemon watches). Under the CNI registry hostPath so it survives a rolling restart; empty disables persistence. |
--authz-sidecar |
false |
Node-local ext_authz sidecar entry (027). |
--authz-sidecar-timeout |
200ms |
Per-check gRPC timeout. |
--authz-sidecar-failure-mode-allow |
false |
Fail-open (default: fail-closed). |
The chart's booleans (
agent.gamma,agent.meshDns,agent.captureRedirectAllDefault, …) map to these flags. Transparent capture, the redirect-all passthrough chain, L4 route types, and the startup-taint removal are unconditional since proposal 031 (no flags).
agent edge (subcommand)¶
Inherits the manager + identity flags (but --node-name is relaxed, derived
from POD_NAME). Adds: --edge-http-port (80), --edge-https-port
(443), --edge-tls (false), --gateway-class (aether),
--route-namespace ("" — the default namespace for Gateway TLS Secrets;
watching is cluster-wide), --edge-service-name (""), --geoip-city-db
(""), --geoip-headers ([country]), --xff-num-trusted-hops (0).
The readiness listener is the fixed port 18021 (030 constant), per-Gateway
addressing is unconditional (021 Phase 2), and the empty local store lives at
the fixed pod-local path.
agent proxy-supervisor (subcommand — standalone flag set)¶
The Envoy hot-restart supervisor (proposal 001): --envoy-path
(/usr/local/bin/envoy), --config (/etc/envoy/envoy.yaml), --base-id (0),
--drain-time (45s), --parent-shutdown-time (60s), --watch-config
(true), --state-dir (/run/aether/hotrestart), --ready-marker, --envoy-arg
(repeatable), --handoff-deadline/--admin-unresponsive-deadline (0 = defaults),
--admin-address (127.0.0.1:9901), --install-path,
--install-readiness-path, --readiness-check (deprecated), --otlp-endpoint.
--install-path and --install-readiness-path are the initContainer's staging
mode: the first self-copies this binary to the shared volume as the supervisor,
the second copies the bundled /proxy-ready prober out of the agent image. A
requested --install-readiness-path against an agent image predating #673 is a
hard failure, so chart/image skew surfaces in the initContainer rather than as a
pod that can never become Ready.
--readiness-check is the pre-#673 exec probe and is deprecated: re-execing this
67MB binary every 2s per pod spent >=31% of the supervisor container's CPU on Go
package init alone (which runs before main(), so no argv check can avoid it).
The chart now execs the standalone proxy-ready binary below instead. The flag
still works, so a chart predating #673 keeps a probe against a newer image.
proxy-ready (standalone binary — bundled in the agent image, not run from it)¶
The aether-proxy pod's exec readiness probe (#673). One flag: --ready-marker
(/var/run/aether-proxy/ready); exit 0 iff that path stats. It is deliberately
stdlib-only (~1.7MB vs the agent's 67MB) — it imports nothing but
common/readymarker, and //agent/cmd/proxy-ready:deps_test fails the build if
that ever changes. It ships as an extra layer in the agent image (no second pull:
the install-supervisor initContainer, which already runs that image, copies it
onto the proxy pod's shared volume at /opt/aether/proxy-ready).
The probe stays an exec probe on the pod-local marker rather than an
httpGet/tcpSocket: the proxy DaemonSet is hostNetwork: true with
maxSurge: 1, so predecessor and successor share the host netns for the whole
handoff and no port-based check is provably pod-local (the reason #582 was closed
for mesh-dns). Envoy's admin endpoint is not an equivalent target either — a
draining hot-restart parent answers LIVE at its old epoch for the entire
--parent-shutdown-time-s window (proposal 001, lesson 6), and the supervisor
deliberately holds readiness while it is still the serving parent.
mesh-dns (standalone daemon — its own binary and image)¶
The aether-mesh-dns DaemonSet (#578, #583). It answers <svc>.<ns>.<mesh-domain>
from the snapshot file the agent writes and forwards everything else upstream, so
agent rolls never gap pod DNS. It does not share the agent's flag set:
--snapshot-path (/host/var/lib/aether/registry/mesh-dns/records.json),
--mesh-domain (aether.internal), --mesh-dns-upstream (repeatable,
host[:port]; empty = /etc/resolv.conf), --ready-marker
(/run/aether/mesh-dns.ready), --readiness-check (deprecated),
--forward-pool-size (8), --otlp-endpoint, --debug.
It binds UDP+TCP on the host at port 18054, which the CNI DNATs each managed
pod's :53 to.
--readiness-check is the pre-#683 exec probe and is deprecated: re-execing this
16.9MB daemon every 15s per pod spent ~10 core-seconds per 25 minutes fleet-wide
(~3-4% of the container's CPU) on container exec and Go package init alone (which
runs before main(), so no argv check can avoid it). The chart execs the
standalone mesh-dns-ready binary below instead. The flag still works, so a
chart predating #683 keeps a probe against a newer image.
--debug only raises the log level (Info to Trace); it gates no feature and no
data-path behaviour. The mesh-DNS forward path logs nothing per query at any
level — the resolver's only Debug-level call site is the snapshot reload. The
chart nonetheless defaults it off for this daemon (#684) via its own
agent.meshDnsDaemon.debug key: the global debug: true deliberately no longer
reaches mesh-dns, because every record it emits is also fanned out to the OTLP
log exporter and this is the one component on every managed pod's :53 path.
Turn it on for a diagnosis with --set agent.meshDnsDaemon.debug=true.
mesh-dns-ready (standalone binary — bundled in the mesh-dns image)¶
The aether-mesh-dns pod's exec readiness probe (#683). One flag:
--ready-marker (/run/aether/mesh-dns.ready); exit 0 iff that path stats. It
is deliberately stdlib-only (~1.7MB vs the daemon's 16.9MB) — it imports nothing
but common/readymarker, and //agent/cmd/mesh-dns-ready:deps_test fails the
build if that ever changes. It ships as an extra layer in the mesh-dns image, the
image the DaemonSet already runs, so there is no second pull and no chart/image
skew is possible: the prober and the daemon that writes the marker are the same
artifact.
Like proxy-ready, it stays an exec probe on the pod-local marker rather
than an httpGet/tcpSocket: this DaemonSet is hostNetwork: true with
maxSurge: 1, so predecessor and successor share the host netns for the whole
handoff and a port-based check could be answered by the peer pod's SO_REUSEPORT
socket. That is precisely what #582 proposed and why it was closed abandoned.
registrar¶
--cluster-name (required), --mesh-domain (aether.internal),
--control-cluster (""), --region (""), --registry-backend
(kubernetes), --etcd-endpoints ([localhost:2379]), --peer-etcd
(repeatable, <region>=<endpoint>[,<endpoint>...] — cross-region replication,
006 Phase 2; requires the etcd backend + an explicit --region),
--sync-interval (5s), --enable-mcs
(false), --grpc-address (:8443), --spire-enabled (true),
--spire-workload-socket. The mesh-Service generator is unconditional (031
round 2), and the mTLS peer trust domain is resolved from the registrar's own
SVID (no --spire-trust-domain).
controller¶
--mesh-config-configmap (aether-mesh-config), --spire-enabled (false),
--spire-workload-socket, --webhook-config-name (""),
--mutating-webhook-config-name (""), --mesh-domain (aether.internal —
the pod-mutating webhook derives its injected ndots from the domain's label
count; the old --pod-ndots was retired).
cni-install (init container)¶
--cni-bin-dir, --cni-bin-target-dir, --mounted-cni-net-dir,
--otlp-endpoint, --capture-redirect-all-default,
--mesh-dns, --host-ip, --debug. The per-pod capture redirect is
unconditional (no --transparent-capture; per-pod capture.aether.io/*
annotations opt out). (The cni plugin binary itself is configured via
CNI-spec stdin, not flags.)
4. Labels & annotations¶
Defined in common/constants/. Prefixes:
config.aether.io/* = what a pod consumes (client config);
endpoint.aether.io/* = endpoint registration facts (what a pod serves);
capture.aether.io/* = transparent-capture behavior; metadata.endpoint.aether.io/*
= free-form endpoint metadata usable as routing subsets.
Pod / namespace labels¶
| Label | Value | Meaning |
|---|---|---|
aether.io/managed |
"true" |
Opt a pod (or, with controller.namespaceInjection, a namespace) into the mesh. |
aether.io/agent-not-ready |
(taint) | Startup taint keeping pods off a node until the agent's CNI serves. |
Endpoint annotations (endpoint.aether.io/*)¶
| Annotation | Default | Meaning |
|---|---|---|
endpoint.aether.io/port |
8080 |
Primary/default service port. |
endpoint.aether.io/ports |
— | All served ports, comma-separated (multi-port, 005). |
endpoint.aether.io/weight |
1024 |
Load-balancing weight. |
endpoint.aether.io/health-path |
/ |
Path the agent active-health-checks. |
endpoint.aether.io/health-check-mode |
eds |
eds (agent vets + publishes over EDS) or active (each client proxy probes). |
endpoint.aether.io/protocol |
http |
Wire protocol served: http or tcp. |
endpoint.aether.io/uds-socket |
— | Deliver inbound to a Unix socket (<volume>/<file>, emptyDir only, no subPath) instead of the TCP port (034). Wins over an EndpointPolicy on the service; needs proxy.udsWorkloads.enabled. |
metadata.endpoint.aether.io/<key> |
— | Free-form metadata → selectable routing subset (e.g. …/version=v2). |
Config annotations (config.aether.io/*)¶
| Annotation | Meaning |
|---|---|
config.aether.io/upstreams |
Comma-separated upstream services this pod calls; drives demand-scoped distribution + ODCDS (004). |
Capture annotations (capture.aether.io/*, proposal 022)¶
| Annotation | Meaning |
|---|---|
capture.aether.io/redirect-all |
"true" force redirect-all, "false" opt out, else node default. |
capture.aether.io/exclude-outbound-ports |
Comma-separated outbound TCP ports to carve out of capture. |
capture.aether.io/exclude-outbound-ip-ranges |
Comma-separated IPv4 CIDRs to carve out (TCP+UDP). |
Gateway / other¶
| Constant | Value | Meaning |
|---|---|---|
| edge GatewayClass controller | gateway.aether.io/edge |
controllerName of the edge GatewayClass. |
| mesh (GAMMA) controller | gateway.aether.io/mesh |
controllerName for Service-parented route status. |
| Gateway HTTP redirect | gateway.aether.io/http-redirect: "true" |
Opt a Gateway's plain-HTTP listener into HTTP→HTTPS 301. |
| workload SPIFFE ID | aether.io/spiffe-id |
Rejected and ignored (#669). A pod's mesh identity is always spiffe://<trust-domain>/ns/<namespace>/sa/<service-account>, derived from the API server. A pod carrying this annotation is logged at WARN and counted by aether.agent.identity.spiffe_id_override_rejected. |
Always-ignored namespaces¶
Never intercepted regardless of labels (so the control plane + SPIRE never depend
on the mesh): kube-system, aether-system, spire-mgmt, spire-server,
spire-system.
See also¶
getting-started.md— install + workload onboarding.runbook.md— build/test/e2e developer loop.workload-requirements.md— the full workload contract.charts/README.md— chart layout, image mirroring, versioning.../charts/aether/values.yaml— the authoritative values with full inline comments.