The IoT strikes back again: half a million IoT devices killed DYN DNS for hours, but fixing this will be hard
Posted by jpluimers on 2016/10/22
Less than a month after The IoT strikes back: 650 Gigabit/second and 1 Terabit/second attacks by IoT devices within a week the IoT struck back again: an estimated half a million IoT devices was used to perform multiple DDoS attacks against Dyn Managed DNS that took around 11 hours to resolve.
Google DNS appears to “live” near me in Amsterdam
High availability usually involves a mix of DNS TTL and/or BGP routing. That’s typically how CDN providers like Cloudflare work (it’s one of the reasons that global DNS servers like Google’s 8.8.8.8 appear near to you and over time routes – some MPLS – to it change). Short DNS TTL can help CDN, requires a very stable DNS infrastructure and is similar to but different from a Fast Flux network.
Last months attacks were on a security researcher and a single ISP. The Dyn DNS attack affected even more internet services (not just sites like Twitter, WhatsApp, AirBnB and Github). So I’m with Bruce Schneier that Someone Is Learning How to Take Down the Internet.
Handling these attacks is hard as the DDoS mitigation firms simply cannot handle the sudden increase of attack sizes yet. BCP38 should be part of mitigation, but the puzzle is big and fixing it won’t be easy though root-causes of bugs change as a lot of research is in progress.
I’m not alone in expecting it to get worse though before getting better.
On the client side, I learned that many users could cope by changing their DNS servers to either of these Public DNS Servers:
- OpenDNS 208.67.222.222, 208.67.220.220, 208.67.222.220, 208.67.220.222
- OpenDNS does a good job of handing “last known good” IPs when they can’t resolve.
- Google Public DNS 8.8.8.8, 8.8.4.4
- Level 3 DNS 4.2.2.1, 4.2.2.2, 4.2.2.3, 4.2.2.4, 4.2.2.5, 4.2.2.6
Some more interesting tidbits on the progress and mitigation on this particular attack are the over time heat-maps of affected regions and BGP routing changes below.
Saturday Level 3 outage map
Even today the above Level 3 Outage Map is still read in quite a few places. The Level 3 status graph is interesting as well:
Level 3 outage chart
BGP routing changes of 208.78.70.0/24 the main routing prefix used by Dyn DNS on 20161021:
–jeroen (who will post more updates here later).
Did I mention it will get worse before getting better?
References:
- The IoT strikes back: 650 Gigabit/second and 1 Terabit/second attacks by IoT devices within a week [WayBack]
- 500,000 infected devices is enough to create a botnet capable of crippling the internet. There’s over 8 billion IoT devices. We’re not going to fix this… – Matthew Garrett – Google+ [WayBack]
- DDoS Attack Against Dyn Managed DNS Incident Report for Dyn, Inc. [WayBack]
- Status: Resolved. I hope it will be resolved for a while, but the evil use of IoT seems to have far bigger consequences than many anticipated, so this fight is far from over. [WayBack]
- Fast-Flux Botnet Detection Based on Weighted SVM [WayBack]
- Massive Dyn DNS outage | Hacker News [WayBack]
- Massive DDOS attack against Dyn DNS is causing havoc online [Resolved] [WayBack]
- In the light of today’s DNS attacks [WayBack]
- Someone Is Learning How to Take Down the Internet [WayBack]
- mjg59 | Fixing the IoT isn’t going to be easy [WayBack]
- Killing bug classes like use-after-free to increase the time and cost of exploit development… [WayBack]
- Where does the Cyber come from? Entire bug classes are being killed while you are reading this. Except in your fridge. – Kristian Köhntopp – Google+ [WayBack]
- Windows 10 Mitigation Improvements [WayBack]
- We are IoT. We are legions. We do not forgive. We do not forget. Expect us. [WayBack]
- Level 3 Outage Map [WayBack]
- Level3 outage? Current problems and outages | Down Detector [WayBack]
- RIPEstat — Internet Measurements and Analysis [WayBack]
- BGP 208.78.70.16 route change video [WayBack]
- Ant Stanley on Twitter: “Watch the BGP routes change as DynDNS deal with the DDos attack earlier today…” via Kristian Köhntopp – Google+ [WayBack]
- Public DNS Servers [WayBack]
- Jason on Twitter: “DNS not working? Try a different set: – OpenDNS 208.67.222.222, 208.67.220.220 – google: 8.8.8.8, 8.8.4.4 – Level 3: 4.2.2.1, 4.2.2.4” [WayBack]
- Pro-tip: OpenDNS users generally see the Internet as they should. We do a good job of handing “last known good” IPs when we can’t resolve. [WayBack]
- Some more interesting links on the Dyn attack… – Jeroen Wiert Pluimers – Google+ [WayBack]
Leave a Reply