Historical status reports for 2016


09-Dec-2016
07:19	 Some systems demonstrating degraded performance.
	 Within a few minutes, virtually all systems were non-responsive.
	 Half-million dollar SAN has crashed, and vendor-provided
	 configuration contains an error preventing failover to the
	 secondary unit! Rebuild under way, but it's taking forever.
	 Finally recovered and all systems back online at 15:39

30-Sep-2016
10:47	 Packet loss on all external links. Under investigation.
	 (Partially Restored 11:02 but some paths still showing loss)
	 (Fully restored 12:00, massive DDoS taking down all links)

01-Aug-2016
03:02	 syd-1 server became unresponsive. Engineers called, services
	 restored 03:28. Investigation into cause to commence later today.
	 Some websites, most authentication, mail affected.

04-Jun-2016
10:45	 All services to both datacentres are isolated. Engineers are
	 enroute to sites now to see what has happened. No ETR.
	 (Restored 12:46, waiting on word as to the cause)

20-May-2016
18:30	 We're declaring an urgent maintenance window of 30 minutes
	 to facilitate urgent maintenance on one of the blade servers.
	 Outage expected to be 10 minutes. Affected services will
	 include email, authentication and some web services.
	 Shutdown at 18:23, back online 18:43.

19-May-2016
06:37	 Loss of connectivity to all servers in both datacentres in
	 Sydney. Engineers investigating.
	 Resolved and all services back 09:27. Awaiting full report.

25-Feb-2016
06:30	 Due to a developing fault in a fibre-optic service, engineers
	 will be replacing a link at around 6:30am. All services except
	 for our 3G mobile services, will be unavailable during this
	 time. Duration is expected to be less than 1 minute.
	 Work was completed ahead of time. Outage commenced 05:49:50
	 and full connectivity restored 05:50:30  (40 seconds outage)

13-Jan-2016
16:00	 It is with some sadness, I report the decommissioning of our
	 longest-running server. "Starone" was built on 1-April-1997
	 and brought into service shortly after, where it has run 24/7
	 ever since. After starting to throw hard disk errors a while
	 ago, a replacement platform has been brought online to take
	 over all of its tasks. 18 years 9 months and 12 days.
	
02-Jan-2016
10:54	 Works completed. New servers running, all tested services are
	 back online, no mail missed. Will continue to check everything
	 has migrated properly and monitor for any problems. In the event
	 users do encounter issues, please call 0409 578 660.

02-Jan-2016
10:30	 A 30-minute maintenance window declared for migration of our
	 remaining servers from Global Switch datacentre at Ultimo to
	 our new facilities at S1.