Staff Victoria - Pulseway Team Posted November 2, 2022 Staff Share Posted November 2, 2022 Please be advised that we're experiencing storage issues with our hosting provider in the US hosting center. It is causing our database server to continuously crash. We apologise for any inconvenience and are actively working on a resolution. The billing system may be temporarily unavailable as well. Link to comment Share on other sites More sharing options...
Martin Astley Posted November 2, 2022 Share Posted November 2, 2022 Hope everyone is ok? Link to comment Share on other sites More sharing options...
augment-tech Posted November 2, 2022 Share Posted November 2, 2022 Please keep us up to date with an ETA for resolution. Thanks! Victoria - Pulseway Team 1 Link to comment Share on other sites More sharing options...
Administrators Paul Posted November 2, 2022 Administrators Share Posted November 2, 2022 Hey everyone, Apologies for the post-Halloween scare. We've worked with the hosting provider and have restored the service. We will be announcing later this week a maintenance window when we will be switching to a more modern storage solution. -Paul Link to comment Share on other sites More sharing options...
Martin Astley Posted November 2, 2022 Share Posted November 2, 2022 7 minutes ago, Paul said: Hey everyone, Apologies for the post-Halloween scare. We've worked with the hosting provider and have restored the service. We will be announcing later this week a maintenance window when we will be switching to a more modern storage solution. -Paul Thank you for the update Paul! - Very happy to be back in control of our infrastructure again 😎 Link to comment Share on other sites More sharing options...
DarienAL Posted November 2, 2022 Share Posted November 2, 2022 What were the symptoms for us as users? I received a some offline notifications for machines that weren't offline, was that related to this? Link to comment Share on other sites More sharing options...
Josh Zets Posted November 2, 2022 Share Posted November 2, 2022 We are currently seeing an issue logging into web app as well as remote control. Is this issue still ongoing? The error is "Database Query Exception" so I assume is related to this issue. Link to comment Share on other sites More sharing options...
Administrators Paul Posted November 2, 2022 Administrators Share Posted November 2, 2022 (edited) Yes Josh, apologies again. The root cause of the issue is impacted disk I/O where the SAN has experienced multiple disk failures at the same time greatly reducing I/O capacity. We'll announce tomorrow a maintenance window for the window where we will transition to a SSD-backed SAN which will greatly increase the performance of our cluster. Update: 5:14PM - We're down again. We've decided to switch to the new storage now. Update: 5:50PM - We're 20% done with the move to the new storage, we anticipate another 2 hours before we can restore service. Update: 6:24PM - 69% done with the move. Update: 6:58PM - Move completed successfully, databases are starting up. -Paul Edited November 2, 2022 by Paul Josh Zets and Alec Hackbarth 2 Link to comment Share on other sites More sharing options...
DarienAL Posted November 2, 2022 Share Posted November 2, 2022 I continue to receive multiple offline/online notifications from pulseway on many of our machines across multiple environments (onprem and cloud) is this related to the issues you are having? Link to comment Share on other sites More sharing options...
Administrators Paul Posted November 2, 2022 Administrators Share Posted November 2, 2022 (edited) Hi Darien, Yes. That would be the cause of the database server finally starting up. Apologies for the inconvenience. Databases are still starting, will post another update when all services are completely back up. Update: 7:10PM: All services are now fully operational. -Paul Edited November 2, 2022 by Paul Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now