Five Star Tax and Business Solutions has been an Umbrella client for many years. They offer a full spectrum of tax, business consulting, payroll, financial planning, and accounting services for a wide range of individuals, small businesses, corporations, and other organizations. And we enjoy working with them!
On Monday morning, February 13th, Five Star Tax’s only server went offline.
The server was what we consider “older” and had some issues coming back online. When it finally did, the Active Directory services were no longer communicating on the network. This means that users couldn’t log in to do their work.
Because we were dealing with an older server, Windows Server Backup was the solution in place for backups because the older operating system would not support Veeam, our recommended backup solution. The daily backups were completing successfully so our first action was to do a restore of the last good backup as soon as possible.
As a result of the operating system being corrupted, we had some serious hurdles to overcome to perform a recovery from backup. The server wouldn’t operate properly, so it couldn’t recognize and restore the Windows Backup files. We had to do a repair and recovery outside of the operating system. These things take time… it had been 24 hours since the server had gone down.
Tuesday, February 14th – Valentine’s Day
We decided to expedite setting up a temporary server for Five Star Tax. We rebuilt their domain from scratch and created a virtual machine on our temporary server. We restored and copied all of their files and installed and configured their software applications.
We worked all day and continued into the night to make sure we got them back up and running quickly. (Umbrella was kind enough to send flowers to the wives and girlfriends of the engineers working the problem… Valentine’s plans had to be rescheduled.)
Wednesday, February 15th
Members of Team Gotham went on-site to assist the employees at Five Star Tax gain access to their files, emails and tax programs. By mid-day, Five Star Tax was able to resume business as usual. Some additional configuration was required over the next couple of days but no data was lost.
With the new virtual machine, we installed a newer operating system so we could use Veeam Endpoint for backups and we are currently working with the client to get them a new virtualized infrastructure with Veeam Backup & Replication. (Had Veeam Backup and Recovery been in place, that 48 hours of downtime would most likely have been reduced to 2 hours.)
Most businesses can’t afford to be completely down for a day or two. Do you know what your current disaster recovery plan is and how long it will take to get back up and running in the event of a disaster? Umbrella is constantly checking backups to make sure they are working properly. We are also migrating clients to our Veeam solution so that “Disaster Recovery” is no longer a four letter word.
A personal thank you to Marquita Miller and the staff at Five Star Tax who remained calm and showed a great deal of restraint during this very difficult time. In the end, they trusted in Umbrella and we were with them every step of the way.
Click here to read a technical extension of this story. (PDF)
If you aren’t a client, and you’re not confident in your current IT support, please give us a call to talk about how easy it is to have Umbrella protecting your company.