Historical status reports for 2012

18:22	 Localised routing issue. Upstream BGP flapping, causing loss
	 of some routes to certain IP ranges. Affected sites vary.
	 Upstream BGP server rebooted, problem cleared 19:04.

08:15	 Outage identified, engineers rebuilding affected networks.
	 Customers comming back on-line from 08:15.

06:00	 Widespread major telstra outage affecting all services in and
	 around Albury. They are working on it but no ETR yet.

14:34	 Main mail and authentication server became non-responsive
	 at 14:34. System restarted at 14:47 and back online 14:50
	 Rack temperatures are excessive, talking with datacentre.

17:03	 Last of the links have been restored. BGP issues have
	 been affecting users all over the country from lots of

15:45	 Loss of connectivity again. Affecting different users
	 in different ways.

11:25	 Links returned. Seems to be BGP issue, possibly as a
	 result of connectivity issues to charter in the USA.

11:10	 Intermittent loss of connectivity to multiple networks,
	 particularly international.

22:04	 Our main mail and authentication server became non-responsive
	 at 22:04. System administrator restarted the box at 22:26.
	 Investigating the cause.

06:42	 New system built, tested, all data migrated from old system.
	 Old system mail system stopped, all mail moved to new system.
	 Old system taken out of service, new system brought online.
	 System back up 06:49am. 8 minutes downtime sunday morning.

16:00    Estimated time only. Hoping to have hardware ready to replace
         failing drive subsystem. Hoping for less than 1hr downtime.
         Will advise more when details available/confirmed.

18:00    Authentication and mail server stopped responding again.
         Cause is a failing drive array. Replacement parts being prepared.

06:18    Our authentication and mail server stopped responding.
         Remote restart has the system back online. Cause being investigated.

22:00	 We're moving the last 3 servers from the old rack at GS, to the
	 new racks from 10pm, 23-Jan-2012. Downtime is not expected to be
	 more than an hour for each server, and should be considerably
	 less. During this time, various functions will be unavailable.
	 Authentication, mail, websites, DNS etc. Most operations should
	 failover to other servers (like DNS) or only be down for fairly
	 short times (hopefully less than 30 minutes).
	 syd3: down 21:38   up 22:25  (47 mins)
	 syd2: down 22:29   up 22:52  (23 mins)
	 syd1: down 23:22   up 23:39  (17 mins)