Blog


Archive for January, 2011

Scheduled Maintenance – Chicago LaSalle Datacenter: Upgrading Power Boards (January 31st – Time 4-7am)

Thursday, January 27th, 2011

Maintenance Notice – Chicago (LaSalle St.), USA Datacenter
Our Chicago (LaSalle St.) datacenter will be upgrading power boards. This will affect all services from our Chicago LaSalle datacenter, including ns3 (our 3rd nameserver).

Maintenance Window : Monday 31st January from 4-7am (Chicago CDT time).

There is a three hour maintenance window but we expect the impact on server connectivity to be absolutely minimal lasting only 5-10 minutes during the maintenance window. We apologize for any inconvenience.

If you have any questions or concerns about this scheduled maintenance please contact us.

Chicago datacenter affected by DDoS

Thursday, January 27th, 2011

Wednesday January 26th, 2011
8:30pm
Our datacenter in Chicago is currently receiving a large inbound DDOS (Distributed Denial of Service) attack which can affect network speed and latency, which means some sites may load slower than normal or time out. Our datacenter technicians are working to mitigate this attack.

9:20pm This attack is on-going. Our datacenter are working to filter out all the offending traffic. There is connectivity to our servers but there is still a lot of lag on the network and websites will be slower than usual or possibly time out.

10:50pm We’ve been advised that while this attack is on-going, the nature and target of this attack are changing. They are targetting webservers and nameservers belonging to another company in our datacenter using different methods of attack. Our technicians are constantly working to block all the offending traffic as the attack changes.

11.30pm We believe this problem is now resolved. We are very sorry for any inconvenience this attack has caused.

Problem with MySQL on ‘eagle’ server

Monday, January 17th, 2011

17 January 2011
12.19 CST
We are aware of a problem with MySQL on the eagle server. Our technicians are working urgently to fix this.

12.40 CST We are rebuilding MySQL on this server. This takes a bit of time but it generally resolves most problems.

1.20pm CST Unfortunately we are still experiencing a problem with the MySQL server. We are working with cPanel technicians to get it fixed as soon as possible.

3.20pm CST cPanel have advised us that unfortunately the database engine (InnoDB) is corrupted. We are currently attempting to repair this. It’s a very tricky task to repair InnoDB so it is taking a lot longer than we would like.

3.35pm CST The InnoDB engine has now been repaired. We believe this issue is now resolved and all MySQL databases should be working. We apologise for the inconvenience this has caused and we thank you for your patience. We will be re-checking every database manually over the next few hours to make sure there are no problems, however this should not affect server performance.

Problem with connectivity provider in Chicago datacenter

Monday, January 17th, 2011

17 January 2011
11.02am CST
We are aware of a connectivity problem in our Chicago datacenter that is currently affecting customers on our shared and dedicated server clusters. We are awaiting more information from datacenter engineers.

11.20am CST Our datacenter has informed us: “We became aware of a power issue at the datacenter.  We had a technician on site that reported a facility technician arrived at our location immediately when this occurred to restore the power to the affected portions of our datacenter space.  We have multiple technicians at the datacenter now working to ensure that all affected servers complete booting up without issue.”

11.33am CST Servers are now coming back online and should all be back within 10-15 minutes.

11.49am CST All servers are now back online. We apologise for any inconvenience this downtime has caused. We have requested a full report from our Chicago datacenter which we will make available to customers in due course.

Emergency Maintenance on Eagle Server

Wednesday, January 5th, 2011

Emergency Maintenance – Eagle Server, Chicago Datacenter
8:50 PM We have detected that one of the hard drives in this server’s RAID array is degraded. Unfortunately we must take the server offline while we try to rebuild a stable RAID array and possible replace hard drives and cables. This should take 5-10 minutes. We will begin momentarily.

08:57 PM This server is now back online. We are very sorry for any inconvenience caused by this maintenance.

Emergency Maintenance on Eagle Server

Tuesday, January 4th, 2011

Emergency Maintenance – Eagle Server, Chicago Datacenter
3:25 PM We have been advised by our datacenter that the eagle server must be taken offline immediately and moved to a new location in the datacenter due to problems with their power systems. We will update this again shortly.

3:32 PM The move has been completed and the server is currently booting up. All services should be back within a few minutes. We apologise for any inconvenience caused.