Custody Downgrade Drill
A custody downgrade — moving a project from HYOK (Hold Your Own Key) to BYOK (Bring Your Own Key), from BYOK to SKR (Secure Key Release), from SKR to Platform-Managed, or any other step that reduces custody — surrenders custody. The platform's IRREVERSIBLE_ACTION_GATE policy family ensures the downgrade requires:
- A signed handover token proving control of the current custody credential
- Multi-party approval from named roles, with maker ≠ checker enforced
- Fresh MFA from every approver
Drilling this chain periodically — against a test project, in a non-production custody mode — validates that the configured roles still exist, the routing still works, and the audit ledger captures the full chain. Discovering that your downgrade gate is broken during an actual incident is the worst possible time to discover it.
This tutorial walks through a drill against a test project.
Prerequisites
- A test project provisioned in
BYOKcustody with an AWS KMS-backed CEK - Two distinct Warden-role accounts with MFA enabled (you may need a colleague for the second approval)
- Access to the AWS KMS CMK's
Signpermission (for producing the handover token)
Step 1 — Verify the gate configuration
In the Console, navigate to Security → Policy → IRREVERSIBLE_ACTION_GATE configurations. Confirm a gate is defined for the custody-downgrade action with the structural elements that make a drill meaningful:
- The required approver count (
chainConfig.minApprovals) and the eligible approver roles - A
not_originatorconstraint (the initiator cannot also approve — maker ≠ checker), enforced as a non-downgradable server floor for custody-downgrade - Approver constraints requiring fresh MFA and a recent session
- A handover-token requirement signed by the current custody credential
The gate is a policy of type irreversible-action-gate (pol_<base62> identifier) in the project's _sys.authorization_policies table, and it is in force because the project's currently-published policy set includes it. The Console review surface shows the policy identifier that governs the gate, so the forensic record is unambiguous about which policy version applied.
Canonical configuration syntax is in API → Control. If the gate is missing or misconfigured, fix it (in draft, then publish a new policy version that supersedes the current one) before drilling. The drill against a misconfigured gate validates nothing.
Step 2 — Initiate the downgrade
From the Console, navigate to Settings → Custody mode → Downgrade to SKR. The action triggers the gate; the platform creates a gate-token capturing the action, the initiator, the required approval count and eligible roles, and the handover-token requirement. The token opens in a pending state — the downgrade will not execute until the gate is approved and explicitly redeemed.
Step 3 — Produce the handover token
The handover token proves that the entity initiating the downgrade controls the current custody credential. Under BYOK, this means signing the gate-token's challenge payload with the AWS KMS CMK. The exact challenge schema is published per the gate's configuration in the Console; copy or download it from the gate-token review surface.
Sign the challenge payload with AWS KMS (the CMK that currently wraps the project's DEKs):
CHALLENGE_B64=$(echo -n '<the JSON challenge above>' | base64)
aws kms sign \
--key-id <CMK_ARN> \
--message "fileb://<(echo -n "$CHALLENGE_B64" | base64 -d)" \
--message-type RAW \
--signing-algorithm RSASSA_PSS_SHA_256 \
--output text \
--query Signature
The output is the base64-encoded signature. Submit it back to Kibo through the Console's gate-token review surface (or the platform's CLI / SDK per the canonical CLI reference). Kibo verifies the signature against the CMK's public key (which it fetches from AWS KMS); on verification, the gate-token is marked as handover-verified and the approval chain can proceed.
Step 4 — Collect the first approval
The drill needs two distinct approvals. The first is from the initiator's colleague (a Warden-role account). They access the gate-token review surface:
Console → Pending approvals → gate_aB3cD9eF2gH7
The review surface shows:
- The action and parameters
- The initiator's identity
- The handover-token verification status
- An Approve button (gated by an MFA re-authentication prompt)
The reviewer clicks Approve, re-authenticates with MFA, and approves. The gate-token's approval count increments to 1.
Step 5 — Collect the second approval
A second, distinct subject performs the same review-and-approve flow — the not_originator floor guarantees this must be a different person from the initiator. Approval count → 2. The gate-token's status transitions to approved: it is now redeemable.
Step 6 — Redeem, or (for the drill) cancel
An approved gate does not execute on its own. To complete a real downgrade, the initiator redeems the gate by re-issuing the custody-downgrade action with the approved token in the X-K9O-Gate-Token header; the platform re-checks the gate at redemption — the approval floor must still be met, the handover verified, and the action parameters must match exactly what was approved — before it executes.
For the drill, you do not want to downgrade the test project — you wanted to exercise the chain, so you cancel the gate instead of redeeming it:
Pending approvals → gate_aB3cD9eF2gH7 → Cancel
The cancellation is recorded against the gate-token, and the token is sealed in the cancelled state. No custody change occurs.
Step 7 — Verify the chain in the audit ledger
Open Kibo → Audit ledger and filter to the gate-token's identifier. Custody-mode transitions are custody-lifecycle events, so the gate's lifecycle is recorded in the signed hash-chained ledger: the initiation event, the handover-token verification, each approver's approval with the corresponding MFA-verified claim, and the cancellation event (since this is a drill). Each entry carries the policy identifier (pol_<base62>) that authorised the gate at the time — the forensic record is unambiguous about which policy governed the gate.
Run Verify chain segment on the gate-token's range. All entries should verify against the project's published signing key, and the chain hash should link cleanly to the surrounding ledger entries. If the gate's constraint evaluation had failed during the drill (a single subject attempting both approvals, for example), the redemption would be refused with K9O_GATE_CONSTRAINT_VIOLATION naming the failing constraint — the same surface a customer's runtime would receive.
What just happened
You exercised the full custody-downgrade chain without actually downgrading. The drill validated:
- The gate configuration is correctly applied
- The handover-token signing path works against your KMS
- The configured approver roles still resolve to actual subjects
- The maker ≠ checker (
not_originator) floor holds - The MFA gating fires correctly
- The audit ledger captures the full transaction chain with intact signatures
Run the drill quarterly at minimum. The cost is one hour of two people's time and zero production impact. The cost of not drilling — discovering a broken gate during an actual incident — is unbounded.
Drill schedule recommendations
| Frequency | Drill |
|---|---|
| Quarterly | Custody-downgrade chain (this drill) |
| Quarterly | Account-deletion chain |
| Annually | Key-revocation chain |
| Per role-personnel change | Re-validate the approver-role membership and re-run the drill |
Next
- Out-of-Band Ledger Verification — verify the gate-token's chain segment with the offline tool.
- Exporting Compliance Attestation Bundle — include drill records in your compliance evidence package.
- Guides → Maker-Checker — the broader pattern this drill exercises.