At 2:14 a.m. on a Tuesday, a payment-processing service started returning HTTP 500s for roughly one in every 900 requests. The release that had gone out six hours earlier had passed every staging test. Load tests were green. Contract tests were green. The staging environment had been running the exact Continue Reading