Thursday 20 October 2011 A high-end Sun server within a hosting cluster had an isolated hardware level fault associated with its storage subsystem and access to network storage services, which due to the nature of the failure, prevented services being immediately/live migrated/restored; a P1 incident was declared and Mach was able to successfully restore services […]
Tuesday 20 September 2011 The front-end to Mach’s Hosted Exchange platform was running much slower than normal but did not go offline (P2 SLA Incident). This affected the normal speed of email send/receive delivery from Email Clients such as Exchange or iPhone (root cause was a VMware software issue, which was resolved as quickly as […]
Wednesday 7 September 2011 Backup software issue caused extended timeframe and load on one Hosted Exchange node early this morning, which degraded “sync” performance for some users for 1hr 49mins (P2). Nature of Incident This morning the daily backup routine for one of the nodes within the Hosted Exchange clustered platform ran much longer than […]
Monday 1 August 2011 One of Mach’s Data Centre sites (Cooroy) suffered an unplanned outage for just over 1hr with one SAN hardware subsystem server as a result of hardware component failure, it was isolated and only affected a small number of servers/applications. Background This afternoon, one core storage subsystem server went offline unexpectedly and […]
Friday 20 May 2011 One of Mach’s Data Centre sites (Cooroy) temporarily lost some network (Public Internet) communications for 2hrs due to an upstream Service provider failure, until they resolved their failed maintenance change. Background Early this morning, one of Mach’s Data Centre sites (Cooroy) temporarily lost network (Public Internet) communications due to an upstream […]
Tuesday 23 November 2010 Upstream SYD-BNE carrier peering/interconnect services degraded, which impacted network routing for some hosting services between 1215 and 1222 today (issue presented as multiple/intermittent 2-3min outages as BGP cycled/reconverged several times).
Saturday 13 November 2010 Electricity failures in a major Brisbane telecommunications hub (Riparian Tower) affects multiple carriers as UPS/Genset/Mains supplies are reconfigured – unplanned outage of services experienced (non-HA services offline during process). Background Early this morning, electricity failures in a major Brisbane telecommunications hub (Riparian Tower) affected multiple carriers as UPS/Genset/Mains supplies were reconfigured […]
Saturday 4 September 2010 Hardware failure in a low level storage system affected some isolated, non-HA services for the 3.5hrs it took to install replacement. Following 24hrs during Sunday, the team migrated services to alternative platforms/hardware as a precaution (individual non-HA services intermittently offline during process). Background This afternoon, one particular server had an isolated […]
