Parcel tracking frozen since 06:41: millions of events waiting in Kafka

Souss Colis Express (fictional company, Agadir) delivers 90,000 parcels a day in the South. Every agency scan produces an event in the Kafka topic colis.evenements (24 partitions). The consumer group suivi-indexeur (6 instances on conso-sc-01 and conso-sc-02) enriches each event (geocoding of the agency) and updates the tracking page suivi.soussexpress.ma.

Since 06:41, tracking no longer moves. Grafana shows a lag of 2.4 million messages and growing. At 06:10, the “kafka-sc-02: ISR shrink” alert woke up the on-call engineer, who suspects the broker. Last night, the data team pushed a “performance tuning” of the consumers.

You are a level-3 platform engineer, logged in to bastion-sc (Ansible, configuration repository, root SSH). Mission: get consumption going again without losing a single event (customer claims depend on it).