Maintenance Notice – Dublin Datacentre
Technicians will be taking down our main Cisco router tonight for roughly 10 minutes to do an upgrade. All services in our Dublin datacentre will be affected.
Maintenance Window : 09/02/2010 from 00:30 to 00:40.
Reason: We must upgrade the IOS and install two new ethernet cards in this router.
If you have any questions about this please let us know.
09:20 One of our fiber connectivity providers is currently experiencing technical problems and customers on our shared and dedicated server clusters in our Dublin datacente may be unable to connect to servers at present. Engineers are working urgently to resolve this problem.
10:35 Engineers are still working on this problem and expect it to be resolved shortly.
11:20 This problem is now resolved. We apologies for any inconvenience caused.
12:19 A number of our server control panels are currently showing error messages “(Cannot read license file)”. We are working to fix this as soon as possible.
12:48 This problem is now resolved. We apologies for any inconvenience caused.
11:40 One of our fiber connectivity providers (NTL) is currently experiencing technical problems and customers on our shared and dedicated server clusters in our Dublin datacente may be unable to connect to servers at present. Engineers are working to resolve this problem.
11:44 Our NTL fiber connection is still down. Data traffic is being routed via our other providers but customers may experience a slow connection or time-outs.
12:16 This problem is now resolved. We apologies for any inconvenience caused.
Maintenance Notice – Domain Registration Systems
One of our domain registrars OpenSRS will be doing maintenance on their network switches.
Maintenance Window : 29/11/2009 from 04:00 to 07:00.
There is a three hour maintenance window, but service is expected to be unavailable for less than an hour. During this period we will be unable to process some domain name orders, renewals and transfers, and other management changes such as nameservers updates, whois updates etc.. You may encounter an error ‘Could not connect to Server – Please try again later‘ when trying to make changes to some domains through our Client Area.
If you have any questions about this please let us know.
13:25 One of our domain registrars eNom are experiencing technical problems, and as such we are currently unable to process nameservers updates, whois updates etc. on some domain names. You may encounter an error ‘Could not connect to Server – Please try again later‘ when trying to make changes to some domains through our Client Area.
Domain renewals requests are queued and will be processed once the problem is fixed. New domain registrations are not affected.
13:45 This issue is now resolved.
16:00 One of our domain registrars eNom are doing some unplanned maintenance on their API, and as such we are currently unable to process nameservers updates, whois updates etc. on some domain names. You may encounter a Timeout Error when trying to make changes to your domain through our Client Area.
Domain renewals requests are queued and will be processed once the problem is fixed. New domain registrations are not affected.
18:30 This issue is now resolved.
13:41 We are aware of problems with some websites on the ‘megalinux’ server showing blank pages. We are working to fix this urgently.
13:55 There are some compatibility issues between RPM packages installed during a operating system update last weekend and the HTTP build on the server. We are going to re-build the Apache HTTP server and this should fix any issues. This normally take 40-50 minutes to build.
14:33 This issue is now resolved. Sorry for any inconvenience this problem caused.
30/09/09 Wednesday
10:10 The ’supertuxy’ server is currently receiving a relatively large inbound DDOS (Distributed Denial of Service) attack which can affect network speed and latency, which means some sites may load slower than normal or time out.
The attackers have hijacked insecure computer systems and are using these systems to attack our network. We have blacklisted all the offending IP addresses and we will continue to monitor the network to ensure that we are blocking them all.
14:20 The attack is on-going but all the offending IPs are being blocked by our network firewall. There is still the occassional spike in traffic between new offending IPs appearing and us blocking them, but the server has been operating within normal loads since our post this morning and customers shouldn’t notice any interruptions to service.
19:30 This attack is still on-going. We are in the process of blocking more offending IP addresses. Network speed and latency may currently be affected.
20:50 The attack this evening over the past 80 minutes was much larger than this morning. We apologise for any inconvenience this is causing. We have now blocked all the offending traffic from our network, and these will also be blocked by our data carriers. We are continuing to monitor the network traffic and block any offending IP addresses as soon as they are identified. At present the server is operating at normal speed.
01/10/09 Thursday
11:50 The attack has been continuing over night, using several different DDOS methods, and targetting our mail and web services. The vast majority of these attacks are being blocked by our hardware firewall. A small amount is still getting through to the server and we are analysing log files to figure out why this is. All data traffic is being monitored and we are blocking any offensive IP addresses.
Since our last update, the server load has been within normal levels and we don’t believe the attack is currently affecting any customers. If you are experiencing problems, please contact our support helpdesk.
21:00 These attacks are continuing, but 96/97% are being blocked by the DDoS defense box on the network before they get anywhere near the server.
As a temporary measure, and on the ’supertuxy’ server only, we have enabled HTTP connection throttling. This means that users who are visiting a website excessively will have their connection blocked for 5 minutes at a time. This shouldn’t affect any normal users unless they are running 50+ copies of the same website simultaneously, but it is helping identify and block out some HTTP flooding. We hope to be able to remove this shortly.
02/10/09 Friday
12:00 These attacks are continuing but we are blocking 99.9% of them at the network level. The server has been running smoothly largely un-affected by the attacks since Wednesday evening. The server load remained within normal levels throughout Thursday and this morning.
10:57 Server monitoring indicates the ‘supertuxy’ server has stopped responding. We are working to fix this urgently.
11:00 Reboot under way.
11:10 Technicians are currently working on this server.
11:32 Server back online. We apologise for any inconvenience caused. Previous uptime was 60 days. Update to follow.
Update: Technicians tested the server on the workbench and found the network card to be faulty. A new card was fitted and the server was put back online.
