Friday 24 February 2012
At 10.30pm, evening storms and wild weather ravaged the Sunshine Coast region which caused some carrier comms services to fail, and unfortunately, further carrier failures occured at 11.20pm that cascaded and prevented diverse/failover routing to the COY Data Centre.
No other Mach sites or Data Centres were similarly affected. Government clients using GWIP based services were not affected. Electricity outages did not affect the Data Centre as its redundant power systems worked flawlessly.
Mach declared a P1 incident for the COY DC and had 4x staff working on the issue and escalated immediately to upstream Carriers (as the fault is in their, not Mach’s network).
Sufficient redundant links were restored at 1.17am so end-end network traffic could flow again normally.