tidBiTS
Informs and engages the UNB community on IT developments and news

management Briefing - A silver Lining in Post-Tropical Storm Arthur

Author: ITS

Posted on Jul 16, 2014

Category: Management Briefings

Background In recent years ITS has invested heavily in equipment and facilities to ensure continuity during disruptions due to severe weather and other events. An industrial‐scale power generator, fueled by natural gas, was installed at Head Hall in 2012, and has been kept in a state of readiness ever since. When in operation, the generator keeps large batteries charged, which in turn power the networks, servers, applications, and communications infrastructure housed in the main data center. Last year, UNB’s IT disaster recovery facility was commissioned at the Wu Centre; it is on a different power grid from Head Hall, and ensures the university’s IT needs can continue to be met in the event of loss of the main data center. Furthermore, each of the 80+ buildings on campus has at least one IT networking and data room or closet; every single one has back‐up batteries that either run until exhausted or are connected to building generators, in which case they can run indefinitely. Flawless performance Post‐tropical storm Arthur barreled down on Fredericton unexpectedly and with a punch that left the city and surrounding areas reeling (post‐tropical means that a storm is no longer strong enough to be considered a hurricane, but by no means does that imply it is not dangerous!). Widespread power outages lasted up to 9 or 10 days for many. The Fredericton campus was up and running normally within a couple of days‐‐it could have been a lot worse. One of the key successes contributing to the rapid recovery was the performance of UNB’s IT services and systems, which were completely operational throughout the storm and the aftermath. When power to the campus went out around 8:00 am Saturday, July 5, the Head Hall generator started automatically, feeding power to the batteries that power the data center. The switchover was instantaneous; no loss of service or data occurred. Though power to the Wu Centre also went out, the disaster recovery facility there also has an array of back‐up batteries to ensure continuity, and they took over the power load immediately. Unlike Head Hall, the Wu facility does not have a generator to keep the batteries going‐‐in this event it didn’t matter though, since power was restored to the Wu section of campus by noon. Lessons learned Continuity of IT services during an event like Arthur is essential. For example, websites, email, and data processing capabilities were unimpaired, meaning it was straightforward for university staff to post updates and communicate effectively. VOIP phone services performed as designed, and in fact exceeded expectations, but as back‐up batteries in some buildings depleted, phone service was lost. Of course the university was closed at the time, but the limits of batteries became abundantly clear. In addition, when power was fully restored to campus on Monday, almost all building networks and VOIP phones restarted automatically, but a few didn’t, posing questions for ITS to research and resolve. There were also some unforeseeable events—for example a generator at one building failed to function properly, but didn’t impact the network and VOIP phones there. Network components that didn’t restart automatically were manually restarted, and by noon on Tuesday IT services on campus were 100% operational, a remarkable outcome after a serious event whose effects persist throughout the city and the region.

July 16, 2014, Terry Nikkel, AVP, ITS