The PDF transcript generator pods keep dying and grade reports no longer arrive

The MasterLinux school produces its transcripts and certificates as PDFs with the generateur-rapports service (namespace rapports, 3 replicas) on the GKE cluster ml-gke-prod. The final files are written to a persistent disk mounted on /data/sortie, then sent to the students.

Since Monday, the support team has been receiving complaints: late transcripts, sometimes never received. In the console, dozens of generateur-rapports-… pods show up as Evicted. This morning, a pod of the grades API was evicted at the same time on the same node. Version 3.8.0 (“new PDF rendering engine, twice as fast”) was released last Thursday.

The developer suspects a memory leak in the new engine; the platform team suggests “using bigger nodes”. The Marrakech exam boards expect their transcripts by 14:00.

You are the platform engineer, on ml-poste-01, with kubectl connected to the cluster, in the ~/rapports repository that holds the manifests.