INC-3511 · P1
Sahel Assurances: since last night, policyholders see yesterday's contracts, claim declarations fail and paid contracts are suspended
Rabat · AWS eu-west-3 (Paris) + eu-central-1 (Frankfurt) · Sahel Assurances · 07:05
Sahel Assurances, a car and home mutual insurer from Rabat (310,000 policyholders), serves its
mobile app and its web portal from api.sahel-assurances.ma:
- primary region
eu-west-3(Paris): ALB protected by AWS WAFwaf-sahel-public, instancesapp-paris-*, RDS PostgreSQL databasesahel-prod; - recovery region
eu-central-1(Frankfurt): instancesapp-francfort-*in “secours-lecture” (read-only recovery) mode, connected to the cross-region replicasahel-dr-replica; - Route 53 with failover routing: PRIMARY = Paris (health check
hc-parison/health), SECONDARY = Frankfurt; - the payment gateway PayMaghreb notifies each payment by webhook; at 06:00, the job
relances-impayessuspends the contracts whose instalment has not been paid.
Yesterday at 18:30, after a bot attack, the SOC added a geo-restriction WAF rule (Morocco and France only) in Count mode; it was to switch to Block “after one night of observation”.
Since dawn: policyholders see stale data, claim declarations fail (500),
agents report customers suspended for non-payment although they have paid. The night team noticed
an automatic failover to Frankfurt at 03:08 and suggests promoting the replica to “get writes
back”. You are on call (AWS CLI v2, the partner's paymaghreb CLI, SSH access to the machines).
The night's logbook is in ~/INC-3511/main-courante.txt.
This simulation is reserved for students enrolled in a course.
Already enrolled? Log in.