Skip to content

ADR-0012: Cost-cap suspension acts operator-direct (recorded VI deviation)

Status: Accepted — 003-governance-depth (2026-07) Principles served: II, V; recorded deviation from VI Source: specs/003-governance-depth/research.md G-5, G-8; specs/003-governance-depth/plan.md Complexity Tracking

Context

Cost caps must be enforcement, not suggestion: suspend at breach within 15 minutes (SC-005), even if the git path is degraded. But Principle VI says all changes flow through git and GitOps — and the sandbox Argo CD Applications sync with automated + selfHeal, so a bare scale-down would be healed straight back.

Decision

An operator cost reconciler polls OpenCost's allocation API on a fixed interval (default 5 minutes), writes status.costStatus, and drives the state machine normal → warning-75 → warning-90 → suspended. Each threshold fires once per crossing (the persisted warning75At/warning90At timestamps prevent repeat fire on oscillation), surfaced as Kubernetes Events, a status condition, and estate-view columns. Suspension acts operator-direct, in order: (1) patch the app's Argo Application to drop syncPolicy.automated FIRST — selfHeal would otherwise revert the scale-down; (2) scale the namespace's Deployments to zero; (3) record state, condition, Event, and evidence entry. Missing telemetry NEVER enforces: OpenCost unreachable or returning no allocation sets status.costStatus.state=unknown with no transitions and no enforcement — the state is unknown, not breached. Resume (owner raises cap or explicitly resumes) restores sync and lets GitOps restore replicas.

Alternatives considered

  • Platform-raised "suspend PR" — rejected: puts an approval loop in front of an enforcement action, contradicting "enforcement, not suggestion"; state and evidence are still recorded via CR status, Events, and the evidence entry.
  • Resource-request-derived pseudo-cost — rejected: not attribution; breaks the honest "what does it cost" answer.
  • Kubecost — rejected: a superset that is not needed; OpenCost is the constitution's named component.

Consequences

This is the recorded Principle VI deviation in the plan's Complexity Tracking: suspension is a platform lifecycle enforcement (Principle V), time-sensitive, and must work when git is degraded. The audit trail is preserved through CR status, Events, and evidence rather than a commit. Because resume does not hand-scale workloads, replicas return only where GitOps runs — on the GitOps-less local path they stay at zero, which is contract behavior. Restated OpenCost windows never re-punish already evaluated periods, since transitions key off the accrued total versus cap.