Operations Runbook

Operator-facing notes for running the broker in production: admin access, health, databases, password write-back, and where the quality baselines live.

Admin access & MFA

• Admin MFA is mandatory. Every administrator (including the configured bootstrap and break-glass accounts) must enroll a second factor before reaching /admin/**. On first sign-in an admin is routed to enrollment automatically; this is enforced both at login and on every admin request.
• Scoped admin roles are enforced. A role like Help Desk Administrator or Read-Only Administrator grants only its permissions — it is not equivalent to Super Administrator. Grant least privilege.
• Break-glass. Keep the break-glass account credentials sealed and audited; it is a real DB-backed admin and is also subject to MFA enrollment.

Health & readiness

Actuator health groups are exposed for orchestration:

Endpoint	Use
/actuator/health/live	liveness probe
/actuator/health/ready	readiness probe (includes DB)
/actuator/health/detailed	full component health (admin-only)

Databases

• PostgreSQL (default): schema is managed by Flyway migrations (db/migration). This is the recommended production database.
• MySQL: the Flyway migrations are Postgres-specific, so on the mysql profile the broker disables Flyway and builds the schema from the JPA entities (ddl-auto=update, with identifier quoting for reserved words). Functional, but Postgres remains the reference. See Deployment → Database Configuration.
• Set the database with DB_TYPE=postgresql|mysql.

Backup & restore

Two complementary levels:

• Full database backup (operational DR). The complete state lives in the broker DB, so a pg_dump / managed point-in-time backup is the most complete snapshot. Restores are only usable on an instance with the same SECRET_ENCRYPTION_KEY — secrets (RP/IDP client secrets, integration credentials) are encrypted with it, and signing_keys come back in the dump. Back up that key alongside the database, and restore the Flyway history table to avoid migration replay.

• Configuration backup & restore (portable). Settings → Backup & Restore (/admin/config-backup) exports the admin-configured surface — IDP configs, relying-party clients, integrations, and domain/attribute mappings — as one versioned JSON bundle sealed with a passphrase (PBKDF2-SHA256 → AES-256-GCM). Use it to promote config between environments or keep it under version control (the file is encrypted, so it's safe to commit). Restore previews the changes (dry-run), then applies idempotently (upsert by id) in a single transaction with a Replace existing or Skip existing strategy. The bundle records a fingerprint of the source secret.encryption.key; importing onto a different key warns that the carried secrets won't decrypt there — re-enter those values after restoring. POST /admin/api/config/{export,import/preview,import} (admin only).

Password write-back (hybrid AD/Entra)

If configured, a password change in the portal (or an admin reset / forgot-password reset) is written back to on-prem Active Directory over LDAPS, which Azure AD Connect then syncs up to Entra.

• Prerequisites: a DC reachable on LDAPS 636, a service account with delegated Reset Password rights, and CA trust for the DC certificate.
• The write-back is asynchronous and best-effort with retry — the user's local change always succeeds even if the directory is briefly unavailable; outcomes are audited.
• New passwords must satisfy the directory's complexity/history policy (rejections are surfaced).

Quality & compliance baselines

These living artifacts are regenerated from test runs (see deliverables/PBS-SOW-001/):

• Validation report — NX-05_Validation_Report (capability/persona coverage, findings ledger).
• Accessibility baseline — NX-06_Accessibility_Baseline (WCAG 2.1 AA / VPAT input).
• Security baseline — NX-07_Security_Baseline (dependency/CVE scan + recommended CI scanners).

Engineering dashboards: the Playwright HTML report + screenshot gallery under e2e/.