App lifecycle¶
Principle V in one line: no app may be immortal and no
app may be unowned. Internal app estates rot when apps outlive their owners
and their purpose; Tend makes lifecycle a platform-enforced state machine on
the App CRD rather than a convention. frozen and retired were reserved
states from 001; 005-lifecycle-estate made them live.
Environment states¶
The authoritative state lives in the App CR (status.state). One state
authority only — other concerns (cost, conditions, freeze causes) are
separate fields and never add environment states.
stateDiagram-v2
[*] --> sandbox : App created\n(defaulted at admission)
sandbox --> promotion_pending : owner requests promotion\n(platform raises PR + evidence)
promotion_pending --> production : approver merges PR
promotion_pending --> sandbox : PR rejected / withdrawn
production --> frozen : expiry without renewal,\nor owner deactivated past grace
sandbox --> frozen : expiry without renewal,\nor owner deactivated past grace
frozen --> production : every freeze cause cleared\n(renewal by a validated active owner)
frozen --> sandbox : every freeze cause cleared,\napp was frozen from sandbox
frozen --> retired : retention period elapsed,\nor tendtech.io/retire
retired --> frozen : platform-engineer restore\n(cause: restored)
note right of promotion_pending
Strictness gate: promotion is refused (422)
unless governance fields are complete.
A frozen app cannot be promoted; a pending
promotion is held, its disposition recorded.
end note
note right of frozen
priorState (sandbox|production) is preserved
and returned to on unfreeze
end note
priorState is what makes the frozen → production/frozen → sandbox split
correct: freeze records which environment the app came from, and unfreeze
returns it there — never "up" to production or "down" to sandbox regardless
of history.
Freeze: causes compose¶
Freezing is one mechanism serving three distinct triggers. status.lifecycle.freeze
holds a set of causes, each with its own onset timestamp; the app stays frozen
until every cause clears, and clearing is cause-specific:
| Cause | Set when | Cleared by |
|---|---|---|
expired |
spec.expiry passes without renewal |
the tendtech.io/renew annotation (or a direct future spec.expiry edit), only while the owner is validated active |
ownerless |
the owner stays deactivated/unresolved past GRACE_PERIOD |
a valid, active owner reassigned or the original owner reactivated |
restored |
a retired app is restored from its archive | a renewal performed while a validated active owner is in place — restore never auto-resumes, even against a still-future archived expiry |
An app can carry more than one cause at once — expired and ownerless, for
example — and renewing alone does not unfreeze it while it is still
ownerless: ownership must be reassigned first, and a renewal attempted on an
ownerless app is refused (recorded as AppRenewalRefused, not silently
dropped). Freezing stops the workload (the same suspend mechanics as cost
enforcement below — pause Argo CD sync, then scale to zero) while definition,
evidence, and status are all preserved; nothing is deleted until retirement.
Cost suspension (003) is a fully independent, orthogonal posture: a cost-suspended app can also freeze, the two mechanisms never mask each other, and resume (manual or cost) only ever lifts a suspension — it never clears a freeze. Freezes clear exclusively by clearing their causes.
Retirement and restore¶
Retention is measured as the duration of the current uninterrupted
freeze — any unfreeze (a renewal, a reassignment) resets the clock. An app
frozen past RETENTION_PERIOD is retired automatically at the next lifecycle
evaluation, or immediately via the tendtech.io/retire annotation:
- Work in flight concludes first — running builds (PipelineRuns) are cancelled and a pending promotion's disposition is recorded in history, so retirement never leaves an orphaned half-state.
- A complete archive is assembled and committed to
archive/in the app's own Gitea repo: the full App CR snapshot (including its complete lifecycle history) asapp.yaml, a full evidence-estate-api assembly asevidence.json, and amanifest.yamlrecording the trigger, actor, prior state, and the evidence assembler's per-category completeness — a partial assembly is explicit, never silent. - The repo is flipped to archived (read-only), and the App's namespace, Argo CD Application, Keycloak per-app client, and model-gateway budget are released. The per-app Tekton triggers and the Gitea push webhook are not — they live outside the namespace that cascades, and cleaning them up is deferred work rather than part of retirement today.
- The App CR itself survives as a tombstone (
state: retired,archiveRefpointing at the repo + commit) — retirement never deletes the App record, so it stays queryable and restorable.
A platform engineer restores a retired app via tendtech.io/restore (or the
estate view). Restore un-archives the repo, re-runs provisioning from the
tombstone, and lands the app frozen with the explicit hold-cause
restored — restore never starts a workload and never auto-resumes, by
design (2026-07-13 analyze-remediation clarification), regardless of whether
the archived expiry is still in the future. The restored cause clears only
through a renewal performed with a validated active owner in place, and the
app's original promotion approval is retained in its evidence — a restored
production app does not need to be re-approved. If a piece of the archive is
no longer recoverable (e.g. images already garbage-collected), that surfaces
in the restore event message rather than silently.
Owner sync and the ownerless flow¶
OWNER_SYNC_INTERVAL drives a periodic check of spec.owner against
Keycloak. The result feeds status.lifecycle.ownerState
(active | ownerless | unknown) under positive-confirmation semantics —
the same fail-safe posture 003's cost telemetry uses:
ownerlessis set only on a definite signal: the user is disabled (enabled: false) or definitively not found. Flagging fires within one sync interval, targets the owning organisational unit (resolved from the owner's Keycloak realm group; an owner in no group falls back to/platform-engineersas the owner-of-last-resort target — a notification target always exists), and starts the grace clock (ownerlessSince).unknownis set on any error — Keycloak unreachable, a 5xx, a transport failure.unknownnever flags or freezes an app, and it never advances or lapses an existing grace clock: an app already flagged keeps its flag and its clock, but the grace-period freeze itself only fires on an evaluation that positively re-confirms ownerlessness. Sync health (OwnerSyncDegraded) is surfaced to platform engineers so a stale IdP is visible, but it enforces nothing.- Reactivating the owner (or reassigning a valid, active one) within the grace period clears the flag automatically on the next sync — no freeze ever fires for that cycle.
- Past
GRACE_PERIOD, the next positively-confirmed evaluation freezes the app with causeownerless.
This is the same shape as the cost-telemetry-unknown pattern below: absence of good data is observed, never punished.
Durable history vs. transient Events¶
Every lifecycle transition — flagged ownerless, frozen, renewed, retired,
restored, and more — is appended to status.lifecycle.history in the same
status patch that performs the transition: an append-only, FIFO-capped
(100 entries) list carrying type, cause, actor (where a human acted), and
timestamp. This is the durable record behind FR-007/FR-025. Kubernetes
Events are still emitted alongside each transition (AppFrozen,
AppRenewed, AppRetired, and so on) and remain the notification surface,
but Events are TTL-bound (roughly one hour by default) and were never meant
to be queried after the fact — a lifecycle audit that only checked
kubectl get events would silently lose history within the hour. Query the
durable record instead: GET /estate/{app}/lifecycle on the evidence &
estate API, or the estate portal's
lifecycle-history view, both of which read status.lifecycle.history
directly and survive component restarts and Event expiry alike.
Configuration¶
All periods and intervals are platform-configurable environment variables on
the operator, with sane defaults — every clock-driven scenario in this page
is demonstrable with short values and no code changes (hack/demo/
ships a minutes-scale kustomize patch for exactly this).
| Variable | Default | Meaning |
|---|---|---|
LIFECYCLE_POLL_INTERVAL |
5m |
Lifecycle evaluation cadence (expiry, retention, freeze/unfreeze). |
OWNER_SYNC_INTERVAL |
5m |
IdP owner verification cadence. |
GRACE_PERIOD |
720h (30 days) |
Ownerless → freeze. |
RETENTION_PERIOD |
2160h (90 days) |
Continuous freeze → retire. |
RENEWAL_EXTENSION |
2160h (90 days) |
Default spec.expiry extension applied by an empty tendtech.io/renew annotation. |
A non-positive or unparsable value falls back to the default with a logged warning — a config typo never crash-loops the operator.
The cost axis (orthogonal)¶
Cost enforcement deliberately does not add environment states. It is a
separate status.costStatus.state plus a Suspended condition:
stateDiagram-v2
normal --> warning75 : ≥75% of cap
warning75 --> warning90 : ≥90% of cap
warning90 --> suspended : ≥100% of cap\n(pause Argo sync, scale to zero)
suspended --> normal : cap raised or spend drops
warning90 --> normal : spend drops / cap raised
warning75 --> normal : spend drops / cap raised
normal --> unknown : cost telemetry unreachable
unknown --> normal : telemetry restored
note right of unknown
Missing telemetry NEVER enforces —
unknown is observed, not punished
end note
Each crossing fires exactly one event and leaves one timestamp on
status.costStatus — the durable trace the estate reads. Suspension pauses the
app's Argo CD sync before scaling every Deployment in the namespace to zero,
so selfHeal cannot undo the enforcement. In the sandbox track that pause does
not hold today: the ApplicationSet that owns the Application reasserts
syncPolicy.automated, so enforcement wins by re-scaling rather than by the
intended pause, and a suspended app sits OutOfSync with brief pod churn.
Making the pause stick is open work, and resume restores replicas via
GitOps rather than hand-scaling — see
ADR-0012 and
Respond to a cost-cap suspension.
Freeze reuses this same pause-sync-then-scale-to-zero mechanic, generalized
behind composable causes rather than a single cost trigger.
Why enforcement, not suggestion¶
Dashboards of stale apps get ignored; expiring credentials do not. Every lifecycle rule here exists because the alternative — asking teams nicely — is how estates decay into unowned graveyards touching data nobody mapped. The platform freezes and retires so that the answer to "can we turn this off?" is always already known, and the estate portal is where that answer is a single view.