Data gateway components (C4 L3)

Inside the sole data path. The data gateway (services/data-gateway, Go stdlib net/http) is where contract-not-code meets actual bytes: every app request is identified, evaluated against the App CR's declared scopes, logged whichever way the decision goes, and only then served by a connector.

C4Component
    title Data gateway — components

    Container_Boundary(dgw, "Data gateway (tend-system, :8080)") {
        Component(handler, "HTTP handler", "internal/handler", "Routes /v1/sources/{name}/… (apps) and /v1/logs… (platform); /healthz, /readyz")
        Component(auth, "App auth", "internal/auth", "TokenReview of projected SA tokens (audience tend-data-gateway), verdict cache ≤2 min")
        Component(pauth, "Platform auth", "internal/platformauth", "Keycloak OIDC for the log-query API; app SA tokens structurally rejected")
        Component(reader, "App reader", "internal/appreader", "namespace → App CR lookup via cached controller-runtime client (informer)")
        Component(core, "Decision core", "internal/gateway", "scope evaluation: declared? sufficient access? — fail closed on any internal error")
        Component(reg, "Connector registry", "internal/registry, internal/sources", "instances from sources.yaml: synthetic | postgres | s3")
        ComponentDb(log, "Log store", "internal/logstore, SQLite (modernc.org/sqlite) on PVC", "append-only AccessLogRecord, WAL, lifetime retention; readiness-gating")
    }

    Component_Ext(app, "App pod", "projected SA token")
    Component_Ext(platform, "Evidence API / auditors", "OIDC client-credentials")
    Component_Ext(sources, "Data sources", "in-memory synthetic, Postgres, S3")

    Rel(app, handler, "GET/POST/PUT /v1/sources/{name}/…")
    Rel(handler, auth, "authenticate")
    Rel(auth, reader, "SA subject → App CR")
    Rel(handler, core, "evaluate(scope, access)")
    Rel(core, log, "append decision (allow AND deny)")
    Rel(core, reg, "serve via connector (allow only)")
    Rel(reg, sources, "connector call")
    Rel(platform, handler, "GET /v1/logs, /v1/logs/summary")
    Rel(handler, pauth, "validate platform token")

    UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="1")

The life of a request

sequenceDiagram
    autonumber
    participant A as App pod
    participant H as Handler
    participant T as TokenReview (API server)
    participant R as App reader
    participant D as Decision core
    participant L as Log store (PVC)
    participant C as Connector

    A->>H: GET /v1/sources/orders-db/rows<br/>Authorization: Bearer <projected SA token>
    H->>T: TokenReview (audience tend-data-gateway)
    T-->>H: system:serviceaccount:tend-app-x:default
    H->>R: namespace → App CR
    R-->>H: spec.dataScopes
    H->>D: evaluate(source=orders-db, access=read)
    alt scope declared, access sufficient
        D->>L: append AccessLogRecord (allow)
        D->>C: serve
        C-->>A: 200 + rows
    else undeclared / insufficient / any internal error
        D->>L: append AccessLogRecord (deny, reason)
        D-->>A: 403 undeclared-source · 403 insufficient-access · 503 evaluator-error
    end

Three properties of this flow carry the design's weight:

Identity without secrets. The app proves who it is with a projected ServiceAccount token scoped to audience tend-data-gateway — mounted by the platform, rotated by Kubernetes, revoked with the namespace. No per-app credential is ever minted, so there is nothing to embed, leak, or rotate (ADR-0009). TokenReview (rather than local JWKS validation) keeps the API server the authority on expiry and revocation, identically on kind, vcluster, and GKE.

No unlogged path. The append to the log store happens before the response, for allows and denies alike — a deny is evidence, often the most interesting kind (Principle III). The readiness probe requires a writable log store: can't log → don't serve. A gateway that could serve data without recording it would be a governance hole shaped like a disk failure.

Fail closed. TokenReview failure, App CR lookup miss, connector error, log-write failure — every internal fault denies (503 evaluator-error) and is itself logged if at all possible. The only fail-open behavior in Tend's governance is cost telemetry, and that is a deliberate, opposite choice.

Connectors — one contract, many sources

There is one small interface per source type — Synthetic, Postgres, S3 — and the registry hands out the right one for each declared scope. A REST or warehouse connector adds an interface of its own; what it inherits is the governance, not the signature, because scope enforcement and logging run in the core ahead of any connector call. The implementations: synthetic (the 001 in-memory store, absorbed here when services/synthetic-data was retired), postgres (statement queries, writes require readwrite, DDL always refused), and s3 (object list/get/put). REST and warehouse connectors are pre-v1 follow-ups under the same contract — scope enforcement and logging live in the core, so a new connector inherits governance instead of reimplementing it (ADR-0008).

The log store is a component, not a database

Access records live in embedded SQLite on a PVC — append-only, WAL mode, queryable only through the gateway's own /v1/logs API. That API is the seam: if evidence-index scale ever demands a real database, it slots in behind the same queries (ADR-0010). At internal-app volume (tens of apps), operating a platform database for a single-writer log would be complexity without a customer.