Incident simulations
Exit

INC-3514 · P1

Sahel Fret: the ERP in Azure refuses every login and, at headquarters, the customs EDI can no longer reach the storage or the database

Casablanca · Aïn Sebaâ headquarters + Azure subscription “sahel-prod” francecentral · Sahel Fret · Monday 06:40

Sahel Fret is a freight forwarder and customs broker based in Casablanca (Aïn Sebaâ), 380 employees, several hundred customs clearances a day. Its IT is hybrid:

  • at headquarters: the Active Directory corp.sahelfret.ma (controllers dc01 and dc02, which are also the headquarters DNS servers) and the customs EDI server srv-edi-01, which drops the declarations into the storage account stsahelfretdocs and records them in the Azure SQL database sql-sahelfret;
  • in Azure (francecentral): a hub VNet vnet-hub-frc (VPN to headquarters, DNS Private Resolver dnspr-hub-frc, private endpoints for the storage and the database, public access disabled) and a VNet vnet-erp-frc where the ERP runs (vm-erp-01), which authenticates users against AD.

DNS is crossed: the Azure VMs query the resolver's inbound endpoint (address set in the VNet), which forwards corp.sahelfret.ma to the DCs; the DCs forward blob.core.windows.net and database.windows.net to that same inbound endpoint to get the private addresses.

On Sunday evening, the platform team shipped version 3 of the “hub network” Terraform module (CHG-2207, “DNS resolver modernisation”). This morning, the ERP refuses every login and the EDI keeps piling up failures: 140 trucks are waiting for their release note at the port. The network manager, Hicham Ouazzani, suggests “re-enabling public access on the storage and the database to unblock customs”; the ERP project manager wants to “recreate the DNS resolver cleanly”.

You are Ilyas, systems and cloud engineer, on poste-cloud (Azure CLI logged in), with SSH access to vm-erp-01, srv-edi-01 and dc01 (OpenSSH, PowerShell).

This simulation is reserved for students enrolled in a course.

Already enrolled? Log in.