Incident simulations
Exit

INC-3507 · P1

Merchant payments failing: the API can no longer reach its database since last night's cutover

Casablanca · AWS eu-west-3 · 3 accounts (network, payments, data) · Wafira Pay · Monday 06:52

Wafira Pay (fictional company, Casablanca) processes card payments for 3,200 merchants (api.wafirapay.ma). The AWS organization is split into accounts: network (302918476150, the Transit Gateway and its route tables), payments (518204736912, the API) and data (714390265881, the PostgreSQL databases on EC2).

Last night, the data team migrated the payments database to a new VPC donnees-v2 (10.44.0.0/16), created by its Terraform module. The network team attached this VPC to the Transit Gateway (ticket NET-418). At 06:30, the record db.donnees.wafira.internal was switched to the new instance pg-donnees-02 (10.44.12.30); the old one pg-donnees-01 (10.40.12.30) was set to read only. Since 06:31, every transaction fails. The incident channel is already blaming “the new database's security group”. The supermarkets' payment terminals open at 08:00.

You are a level-3 cloud engineer, on poste-cloud with the AWS CLI v2. Profiles: reseau (default, delegated Reachability Analyzer administrator for the organization), paiements, donnees. SSM access to the instances: app-paiements-01 = i-0c41a7e9d2b58f306 (paiements profile), pg-donnees-02 = i-09b2e5c7a4f13d860 (donnees profile). A Reachability Analyzer path nip-0a8d3f61c2e47b915 (app-paiements-01 → 10.44.12.30:5432) has been prepared by the network team.

This simulation is reserved for students enrolled in a course.

Already enrolled? Log in.