The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My work

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,808 other followers

The IoT strikes back again: half a million IoT devices killed DYN DNS for hours, but fixing this will be hard

Posted by jpluimers on 2016/10/22

Less than a month after The IoT strikes back: 650 Gigabit/second and 1 Terabit/second attacks by IoT devices within a week the IoT struck back again: an estimated half a million IoT devices was used to perform multiple DDoS attacks against Dyn Managed DNS that took around 11 hours to resolve.

Google DNS appears to

Google DNS appears to “live” near me in Amsterdam

High availability usually involves a mix of DNS TTL and/or BGP routing. That’s typically how CDN providers like Cloudflare work (it’s one of the reasons that global DNS servers like Google’s 8.8.8.8 appear near to you and over time routes – some MPLS – to it change). Short DNS TTL can help CDN, requires a very stable DNS infrastructure and is similar to but different fromFast Flux network.

Last months attacks were on a security researcher and a single ISP. The Dyn DNS attack affected even more internet services (not just sites like Twitter, WhatsApp, AirBnB and Github). So I’m with Bruce Schneier that Someone Is Learning How to Take Down the Internet.

Handling these attacks is hard as the DDoS mitigation firms simply cannot handle the sudden increase of attack sizes yet. BCP38 should be part of mitigation, but the puzzle is big and fixing it won’t be easy though root-causes of bugs change as a lot of research is in progress.

I’m not alone in expecting it to get worse though before getting better.

On the client side, I learned that many users could cope by changing their DNS servers to either of these Public DNS Servers:

  • OpenDNS 208.67.222.222, 208.67.220.220, 208.67.222.220, 208.67.220.222
    • OpenDNS does a good job of handing “last known good” IPs when they can’t resolve.
  • Google Public DNS 8.8.8.8, 8.8.4.4
  • Level 3 DNS 4.2.2.1, 4.2.2.2, 4.2.2.3, 4.2.2.4, 4.2.2.5, 4.2.2.6

Some more interesting tidbits on the progress and mitigation on this particular attack are the over time heat-maps of affected regions and BGP routing changes below.

Saturday Level 3 outage map

Saturday Level 3 outage map

Even today the above Level 3 Outage Map is still read in quite a few places. The Level 3 status graph is interesting as well:

Level 3 outage chart

Level 3 outage chart

BGP routing changes of 208.78.70.0/24 the main routing prefix used by Dyn DNS on 20161021:

–jeroen (who will post more updates here later).

Did I mention it will get worse before getting better?

References:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: