Incident simulations
Exit

INC-3519 · P1

Assafa Distribution: nobody can get into Azure, the AWS console or the internal tools any more, and CI no longer deploys

Tangier · head office + Entra ID tenant + AWS organization · Assafa Distribution · 08:05

Assafa Distribution (82 supermarkets, 3,000 employees, head office in Tangier) has an on-premises Active Directory and an AD FS farm (sts.assafa-distribution.ma). Everything starts there:

  • the assafa-distribution.ma domain is federated in Entra ID: every sign-in to Azure, Microsoft 365 and the enterprise applications goes through AD FS;
  • IAM Identity Center (AWS organization, 14 accounts) uses Entra ID as its identity source;
  • the internal tools on AKS (Grafana, Argo CD) authenticate through Entra ID;
  • the CI (runner runner-ci-01) deploys to production with saml2aws: AD FS → IAM SAML provider ADFS-Assafa → role ci-deploy-prod.

Since 02:00, no federated sign-in goes through any more. The tills work (offline), but the price update for this week's promotion — which must be deployed before the stores open at 09:00 — is blocked, and the teams cannot even open the console to act by hand.

You are on the on-call workstation: the AWS CLI uses the emergency (break-glass) account and az a certificate-based service principal — both outside the federation. SSH access to adfs-01 (OpenSSH, PowerShell, Microsoft Graph module), runner-ci-01 and bastion-aks.

This simulation is reserved for students enrolled in a course.

Already enrolled? Log in.