Incident simulations
Exit

INC-3509 · P1

Payment impossible when confirming the basket, since the new payment module went live

Casablanca · Google Cloud europe-southwest1 · Shared VPC · Bladi Fresh · Monday 07:45

Bladi Fresh (fictional company, Casablanca) delivers groceries to homes in six cities. Its Google Cloud organization follows the Shared VPC model: the network team administers the host project bf-reseau-hote and its network vpc-partage-prod; the product teams deploy their VMs in service projects (bf-commandes-prod, bf-paiement-prod) attached to that network.

The new payment service (VMs vm-paiement-01/02 of the managed group mig-paiement-api, port 8443, behind the internal load balancer fr-paiement-ilb 10.30.32.50) was switched on at 07:00 by a feature flag in the orders API. Since 07:02, every basket confirmation fails: “Paiement indisponible, réessayez” (payment unavailable, try again). The payment team says it “opened port 8443 last night” and suspects IAM or Cloud NAT. The flag cannot be switched off: the former payment provider's contract was terminated on Sunday. Lunchtime order peak at 11:00.

You are a level-3 cloud engineer, on poste-cloud with gcloud (active configuration inherited from the ticket: default project bf-paiement-prod, region europe-southwest1, zone europe-southwest1-a), read rights on the three projects and compute.securityAdmin on the host project. SSH to the VMs through IAP.

This simulation is reserved for students enrolled in a course.

Already enrolled? Log in.