Incident simulations
Exit

INC-3260 · P1

The whole online shop went down at once: site, API, database and jobs, with no deployment at all

Marrakesh · GCP project masterlinux-boutique-prod (europe-southwest1) · MasterLinux shop · Monday 07:05

The MasterLinux shop (books, kits and training vouchers) runs entirely in the Google Cloud project masterlinux-boutique-prod: the site and the API on Cloud Run (boutique-web, boutique-api), the Cloud SQL database sql-boutique and a Compute Engine VM ml-worker-01 that prepares shipments.

Since 06:12, nothing responds: the site shows an error, so does the API, and monitoring is red everywhere at the same time. No release has been made since Friday. Customer service is already getting calls and this week's email campaign must go out at 09:00.

The cloud administrator is on leave. You are the on-call engineer, on your workstation ml-poste-01 with the gcloud CLI signed in as ops@masterlinux.lab (project owner, user of MasterLinux's billing accounts).

This simulation is part of a production simulation pack.

Unlock every production simulation

AWS, Azure, Google Cloud, on-premise and multi-cloud, from Beginner to Extreme: the real incidents of IT service companies and large IT organisations, available for a year.

Try the free demo simulation first

Already enrolled? Log in.