Yes Josh, apologies again. The root cause of the issue is impacted disk I/O where the SAN has experienced multiple disk failures at the same time greatly reducing I/O capacity. We'll announce tomorrow a maintenance window for the window where we will transition to a SSD-backed SAN which will greatly increase the performance of our cluster.
Update: 5:14PM - We're down again. We've decided to switch to the new storage now.
Update: 5:50PM - We're 20% done with the move to the new storage, we anticipate another 2 hours before we can restore service.
Update: 6:24PM - 69% done with the move.
Update: 6:58PM - Move completed successfully, databases are starting up.
-Paul
