[Live] Degraded performance on the Patina Project app and patinaproject.com

Monitoring - Oct 1, 15:23 UTC

Most services have recovered as of 14:59 UTC. Photos, including those submitted for the Thunderdome, are loading again. Your photos and submissions are safe, and no data has been lost. We are monitoring closely and will post another update once we are sure everything is stable.

Investigating - Oct 1, 14:24 UTC

We are investigating elevated activity on our production database. Database reported 97.5% CPU usage on a primary database node, which may cause delays when loading some photos. We are reviewing database queries and system capacity. Your photos and submissions remain safe, and we will provide another update when we know more.

Update - Oct 1, 14:17 UTC

We have shipped a fix and are checking the remaining affected photos. No submitted photos are stuck in processing. Some images still need review because certain JPEG uploads cannot be read, several large TIFF uploads may exceed processing limits, and we are investigating the remaining failures. Your photos and submissions remain safe.

Identified - Oct 1, 12:36 UTC

Since 02:45 UTC, some photos, including new contest submissions, may not load. Your photos and submissions are safe and no data has been lost. We found the cause and are working on a fix.