Wednesday 21 April, 2021

Outage

Master Database Failure

Generic placeholder image
A postmortem analysis has been posted for this incident.
The application has returned to it's normal performance profile. We will be following up with a postmortem about future plans to guard against additional master database failure.
The slave database has been successfully promoted, but is running slow due to a cold query cache. The application is open and available for requests, but should will be performing in a degraded state for the next few hours. We will continue to monitor the situation.
The slave database is 60% through it's recovery process. We will provide another update once the application is back up.
The master database server could not boot due to a corrupted EBS volume. We are in the process of failing over to the slave database. ETA for the application recovering is 5 minutes.
We're investigating an outage with our master database server.
Services affected
Login/SSO
Status
Resolved
Notice

Data Layer Migration

We will be performing a data layer migration from our existing Postgres system over to our new, multi-region, distributed Riak cluster. The application will be taken offline during the entirety of this migration. We apologize in advance for the inconvenience
Services affected
Connections
Status
Scheduled for Wednesday 4 August, 2021 01:00 AM
Outage

SSL Certificate Failure

The slave database is 60% through it's recovery process. We will provide another update once the application is back up.
We're investigating an outage with our master database server.
Services affected
Webstores
Status
Identified
Outage

404 Issue

We're investigating aconnection issue.
Services affected
Webstores
Notifications
Status
Investigating