Email & Website Services Unavailable

Major incident Websites Email (IMAP)
2022-01-16 07:43 GMT · 2 days, 3 minutes

Updates

Resolved

We’re delighted to report that Enom have fixed the email issues they created over the weekend so email and web services are back to normal.

Throughout today we’ll keep an eye on email delivery and make sure that everything is running smoothly, however tests over night and this morning show everything working normally again.

Over the last two days we’ve manually moved hundreds of clients from Enom DNS to CloudFlare DNS, and although Enom DNS is now working, we’re aiming to continue to migrate our clients in the coming weeks to mitigate any future Enom DNS failures in the future.

If you have have any questions or would like any help please email or call us and we’ll be delighted to help, as always.

Hopefully we can now all enjoy being glued to our email again!

Thank you again for your patience and understanding over these last few days.

Marcus Green

January 18, 2022 · 07:47 GMT
Monitoring

We are continuing to move clients over to the new system as fast as we can.

If you have been moved already then emails from the last couple of days may continue arriving if they were queued, otherwise the sender should receive a bounceback.

You may also find that some emails still take a bit of time as servers around the world catch up, but this should improve quickly.

Enom continue to try and unwind the problems they caused at the weekend and our recent tests show that email service through the Enom DNS is beginning to recover as well.

You can follow their progress here:
https://enomstatus.com/incidents/03q064h6rb7x

Thank you again for your patience and apologies for the disruption.

Marcus Green

January 17, 2022 · 21:30 GMT
Investigating

We’re making significant progress moving clients over to the new system and restoring email service.

There are many sites to move over though so thank you for your patience while we sort this out.

We know how important email is to the running of every business and we’re really sorry that this happened and we know that it will have caused a lot of headaches.

The underlying issue is that Enom, who are a wholesale reseller of over 5 million Domain names, overwrote existing DNS records with something called a ‘CNAME on the bare domain’ which is well known to break email service. They did this without telling any of their resellers and we only became aware yesterday when we had reports of emails not coming through (their scheduled maintenance did not mention this change).

Here’s some more info if you’d like to find out more about ‘CNAMEs on the bare domain’:
https://serverfault.com/questions/91712/dns-using-cnames-breaks-mx-records/91718
https://www.freecodecamp.org/news/why-cant-a-domain-s-root-be-a-cname-8cbab38e5f5c/

After nearly 48 hours with very few updates, Enom have finally realised their mistake and have updated their own status page at: https://enomstatus.com/incidents/03q064h6rb7x

“Some customers may experience email service interruptions due to missing DNS MX records “

We are hopeful (although not overly optimistic) that Enom will reverse their mistake quickly and email service will resume for everyone very soon. However since it has taken so long for Enom to get to this stage we won’t be holding our breath, and therefore we’ll continue working throughout the night, as last night, to get your emails back up and running.

If you have any questions please email support@toolkit.uk and we’ll get back to you as soon as possible. Our team will be on hand this evening to answer your queries.

Thanks again for your patience and sincere apologies.

Marcus Green

January 17, 2022 · 16:42 GMT
Investigating

Early this morning we were able to access Enom’s settings page again, which means that we have been able to start moving your website DNS settings to a new provider.

This process takes roughly 10 minutes for each client and once completed means that your website and email will be back up and running again.

We are working through all our clients as fast as possible today to get everyone back online.

In the meantime we are still waiting for Enom to fix the issues that came about after their server migration which have broken many websites. Their service has been incredibly poor and for this we can only apologise.

Please email us on support@toolkit.uk with your domain name if you would like us to prioritise your services and we’ll do our best.

We are doing everything we possibly can to fix this problem as soon as possible.

January 17, 2022 · 08:15 GMT
Investigating

Enom have confirmed their Engineers expect to have services resumed to normal in the next few hours.

See their latest post on their status page here:
https://enomstatus.com/

We’ll keep this page updated and carry out the required tests once affected Website and Email services are back online.

We’re so sorry once more for the inconvenience this outage is causing and appreciate your understanding and patience as we wait for a fix.

January 16, 2022 · 18:28 GMT
Investigating

We are really sorry for today’s outage that’s affecting some customers.

This is being caused by Enom who we use as a service provider for website domain names.

Enom manage over 5 million domain names globally. They began a server migration last night and it’s unintentionally caused the problems which we’re seeing today.

We will update this page as and when we know more, but you can keep up with Enom’s status page directly here:

https://enomstatus.com

January 16, 2022 · 13:50 GMT
Issue

During a routine & planned systems update last night, our domain & DNS provider Enom encountered some unexpected problems which have resulted in service outages across a small number of our clients web services.

Enom are working on a fix urgently and will inform us as soon as normal services are resumed.

We’re really sorry for the inconvenience and frustration this has caused and will keep this status page updated as soon as we know more.

January 16, 2022 · 07:54 GMT

← Back