Skip to content

App lifecycle

Principle V in one line: no app may be immortal and no app may be unowned. Internal app estates rot when apps outlive their owners and their purpose; Tend makes lifecycle a platform-enforced state machine on the App CRD rather than a convention. frozen and retired were reserved states from 001; 005-lifecycle-estate made them live.

Environment states

The authoritative state lives in the App CR (status.state). One state authority only — other concerns (cost, conditions, freeze causes) are separate fields and never add environment states.

stateDiagram-v2
    [*] --> sandbox : App created\n(defaulted at admission)
    sandbox --> promotion_pending : owner requests promotion\n(platform raises PR + evidence)
    promotion_pending --> production : approver merges PR
    promotion_pending --> sandbox : PR rejected / withdrawn
    production --> frozen : expiry without renewal,\nor owner deactivated past grace
    sandbox --> frozen : expiry without renewal,\nor owner deactivated past grace
    frozen --> production : every freeze cause cleared\n(renewal by a validated active owner)
    frozen --> sandbox : every freeze cause cleared,\napp was frozen from sandbox
    frozen --> retired : retention period elapsed,\nor tendtech.io/retire
    retired --> frozen : platform-engineer restore\n(cause: restored)

    note right of promotion_pending
        Strictness gate: promotion is refused (422)
        unless governance fields are complete.
        A frozen app cannot be promoted; a pending
        promotion is held, its disposition recorded.
    end note

    note right of frozen
        priorState (sandbox|production) is preserved
        and returned to on unfreeze
    end note

priorState is what makes the frozen → production/frozen → sandbox split correct: freeze records which environment the app came from, and unfreeze returns it there — never "up" to production or "down" to sandbox regardless of history.

Freeze: causes compose

Freezing is one mechanism serving three distinct triggers. status.lifecycle.freeze holds a set of causes, each with its own onset timestamp; the app stays frozen until every cause clears, and clearing is cause-specific:

Cause Set when Cleared by
expired spec.expiry passes without renewal the tendtech.io/renew annotation (or a direct future spec.expiry edit), only while the owner is validated active
ownerless the owner stays deactivated/unresolved past GRACE_PERIOD a valid, active owner reassigned or the original owner reactivated
restored a retired app is restored from its archive a renewal performed while a validated active owner is in place — restore never auto-resumes, even against a still-future archived expiry

An app can carry more than one cause at once — expired and ownerless, for example — and renewing alone does not unfreeze it while it is still ownerless: ownership must be reassigned first, and a renewal attempted on an ownerless app is refused (recorded as AppRenewalRefused, not silently dropped). Freezing stops the workload (the same suspend mechanics as cost enforcement below — pause Argo CD sync, then scale to zero) while definition, evidence, and status are all preserved; nothing is deleted until retirement.

Cost suspension (003) is a fully independent, orthogonal posture: a cost-suspended app can also freeze, the two mechanisms never mask each other, and resume (manual or cost) only ever lifts a suspension — it never clears a freeze. Freezes clear exclusively by clearing their causes.

Retirement and restore

Retention is measured as the duration of the current uninterrupted freeze — any unfreeze (a renewal, a reassignment) resets the clock. An app frozen past RETENTION_PERIOD is retired automatically at the next lifecycle evaluation, or immediately via the tendtech.io/retire annotation:

  1. Work in flight concludes first — running builds (PipelineRuns) are cancelled and a pending promotion's disposition is recorded in history, so retirement never leaves an orphaned half-state.
  2. A complete archive is assembled and committed to archive/ in the app's own Gitea repo: the full App CR snapshot (including its complete lifecycle history) as app.yaml, a full evidence-estate-api assembly as evidence.json, and a manifest.yaml recording the trigger, actor, prior state, and the evidence assembler's per-category completeness — a partial assembly is explicit, never silent.
  3. The repo is flipped to archived (read-only), and the App's namespace, Argo CD Application, Keycloak per-app client, and model-gateway budget are released. The per-app Tekton triggers and the Gitea push webhook are not — they live outside the namespace that cascades, and cleaning them up is deferred work rather than part of retirement today.
  4. The App CR itself survives as a tombstone (state: retired, archiveRef pointing at the repo + commit) — retirement never deletes the App record, so it stays queryable and restorable.

A platform engineer restores a retired app via tendtech.io/restore (or the estate view). Restore un-archives the repo, re-runs provisioning from the tombstone, and lands the app frozen with the explicit hold-cause restored — restore never starts a workload and never auto-resumes, by design (2026-07-13 analyze-remediation clarification), regardless of whether the archived expiry is still in the future. The restored cause clears only through a renewal performed with a validated active owner in place, and the app's original promotion approval is retained in its evidence — a restored production app does not need to be re-approved. If a piece of the archive is no longer recoverable (e.g. images already garbage-collected), that surfaces in the restore event message rather than silently.

Owner sync and the ownerless flow

OWNER_SYNC_INTERVAL drives a periodic check of spec.owner against Keycloak. The result feeds status.lifecycle.ownerState (active | ownerless | unknown) under positive-confirmation semantics — the same fail-safe posture 003's cost telemetry uses:

  • ownerless is set only on a definite signal: the user is disabled (enabled: false) or definitively not found. Flagging fires within one sync interval, targets the owning organisational unit (resolved from the owner's Keycloak realm group; an owner in no group falls back to /platform-engineers as the owner-of-last-resort target — a notification target always exists), and starts the grace clock (ownerlessSince).
  • unknown is set on any error — Keycloak unreachable, a 5xx, a transport failure. unknown never flags or freezes an app, and it never advances or lapses an existing grace clock: an app already flagged keeps its flag and its clock, but the grace-period freeze itself only fires on an evaluation that positively re-confirms ownerlessness. Sync health (OwnerSyncDegraded) is surfaced to platform engineers so a stale IdP is visible, but it enforces nothing.
  • Reactivating the owner (or reassigning a valid, active one) within the grace period clears the flag automatically on the next sync — no freeze ever fires for that cycle.
  • Past GRACE_PERIOD, the next positively-confirmed evaluation freezes the app with cause ownerless.

This is the same shape as the cost-telemetry-unknown pattern below: absence of good data is observed, never punished.

Durable history vs. transient Events

Every lifecycle transition — flagged ownerless, frozen, renewed, retired, restored, and more — is appended to status.lifecycle.history in the same status patch that performs the transition: an append-only, FIFO-capped (100 entries) list carrying type, cause, actor (where a human acted), and timestamp. This is the durable record behind FR-007/FR-025. Kubernetes Events are still emitted alongside each transition (AppFrozen, AppRenewed, AppRetired, and so on) and remain the notification surface, but Events are TTL-bound (roughly one hour by default) and were never meant to be queried after the fact — a lifecycle audit that only checked kubectl get events would silently lose history within the hour. Query the durable record instead: GET /estate/{app}/lifecycle on the evidence & estate API, or the estate portal's lifecycle-history view, both of which read status.lifecycle.history directly and survive component restarts and Event expiry alike.

Configuration

All periods and intervals are platform-configurable environment variables on the operator, with sane defaults — every clock-driven scenario in this page is demonstrable with short values and no code changes (hack/demo/ ships a minutes-scale kustomize patch for exactly this).

Variable Default Meaning
LIFECYCLE_POLL_INTERVAL 5m Lifecycle evaluation cadence (expiry, retention, freeze/unfreeze).
OWNER_SYNC_INTERVAL 5m IdP owner verification cadence.
GRACE_PERIOD 720h (30 days) Ownerless → freeze.
RETENTION_PERIOD 2160h (90 days) Continuous freeze → retire.
RENEWAL_EXTENSION 2160h (90 days) Default spec.expiry extension applied by an empty tendtech.io/renew annotation.

A non-positive or unparsable value falls back to the default with a logged warning — a config typo never crash-loops the operator.

The cost axis (orthogonal)

Cost enforcement deliberately does not add environment states. It is a separate status.costStatus.state plus a Suspended condition:

stateDiagram-v2
    normal --> warning75 : ≥75% of cap
    warning75 --> warning90 : ≥90% of cap
    warning90 --> suspended : ≥100% of cap\n(pause Argo sync, scale to zero)
    suspended --> normal : cap raised or spend drops
    warning90 --> normal : spend drops / cap raised
    warning75 --> normal : spend drops / cap raised
    normal --> unknown : cost telemetry unreachable
    unknown --> normal : telemetry restored

    note right of unknown
        Missing telemetry NEVER enforces —
        unknown is observed, not punished
    end note

Each crossing fires exactly one event and leaves one timestamp on status.costStatus — the durable trace the estate reads. Suspension pauses the app's Argo CD sync before scaling every Deployment in the namespace to zero, so selfHeal cannot undo the enforcement. In the sandbox track that pause does not hold today: the ApplicationSet that owns the Application reasserts syncPolicy.automated, so enforcement wins by re-scaling rather than by the intended pause, and a suspended app sits OutOfSync with brief pod churn. Making the pause stick is open work, and resume restores replicas via GitOps rather than hand-scaling — see ADR-0012 and Respond to a cost-cap suspension. Freeze reuses this same pause-sync-then-scale-to-zero mechanic, generalized behind composable causes rather than a single cost trigger.

Why enforcement, not suggestion

Dashboards of stale apps get ignored; expiring credentials do not. Every lifecycle rule here exists because the alternative — asking teams nicely — is how estates decay into unowned graveyards touching data nobody mapped. The platform freezes and retires so that the answer to "can we turn this off?" is always already known, and the estate portal is where that answer is a single view.