From piotr.1234 at interia.pl Sat Oct 1 07:03:26 2016 From: piotr.1234 at interia.pl (Pedro) Date: Sat, 1 Oct 2016 09:03:26 +0200 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: Message-ID: We had situations, that we lost all our bgp sessions, not even only on ports where flood was coming. Just cpu overloaded. I don't care about support too much, there are cheap enough to have spare. Soft is mature with known bugs so i assume that this risk are accepted. Bigger problem for me is technical details about features, which i desribed in my first post. Most of this features i tested on trident2 chipset extreme 670, it works but with problems and some limits. Now i have to change vendor. Really wondering what can i get from N3K-C3064PQ, its also build on trident2 AFAIK thanks for answers, Pedro W dniu 2016-09-30 o 22:50, Matt Freitag pisze: > Pedro, > > Please also keep in mind that the Juniper EX4500 is an end of life > product. Soon you won't be able to get Juniper to support you. That's > why there are so many for so cheap on eBay. > > Matt Freitag > Network Engineer I > Information Technology > Michigan Technological University > (906) 487-3696 > https://www.mtu.edu/ > https://www.it.mtu.edu/ > > > On Fri, Sep 30, 2016 at 4:06 PM, Saku Ytti > wrote: > > On 30 September 2016 at 22:42, Pedro > wrote: > > Hey Pedro, > > > I have some idea to put switch before bgp router in order to terminate isp > > 10G uplinks on switch, not router. Main reason is that could be some kind of > > 1st level of defence against ddos, second reason, less important, save cost > > of router ports, do many port mirrors. > > I don't understand your rationale, unless your router is software box, > but as it has 10G interface, probably not. > Your router should be able to limit packets in HW, likely with better > counter and filtering options than cheap switch. > > -- > ++ytti > > --- Ta wiadomo?? zosta?a sprawdzona na obecno?? wirus?w przez oprogramowanie antywirusowe Avast. https://www.avast.com/antivirus From saku at ytti.fi Sat Oct 1 10:43:16 2016 From: saku at ytti.fi (Saku Ytti) Date: Sat, 1 Oct 2016 13:43:16 +0300 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: Message-ID: On 1 October 2016 at 10:03, Pedro wrote: > We had situations, that we lost all our bgp sessions, not even only on ports > where flood was coming. Just cpu overloaded. I don't care about support too > much, there are cheap enough to have spare. What is the device you're trying to protect? Perhaps it supports reasonable CoPP features so that you can protect it directly on itself. To do this CoPP on neighbouring switch, you'll need unique policer for each and every BGP session and ARP, your switch may not support this and it is provisioning nightmare. -- ++ytti From dwessels at verisign.com Sat Oct 1 13:36:13 2016 From: dwessels at verisign.com (Wessels, Duane) Date: Sat, 1 Oct 2016 13:36:13 +0000 Subject: Root Zone DNSSEC Operational Update -- ZSK length change In-Reply-To: <626768A0-5D22-4B97-8EFC-174894B7F297@verisign.com> References: <626768A0-5D22-4B97-8EFC-174894B7F297@verisign.com> Message-ID: <909E9B16-CA18-4738-827D-BDF5F08A464E@verisign.com> I'm pleased to announce that this change is now complete. As of 13:34 UTC on October 1, 2016 the root zone has been signed and published with a 2048-bit ZSK. Please contact myself of Verisign customer service (info at verisign-grs.com) if you observe any problems related to this change. Duane W. > On Sep 29, 2016, at 11:15 AM, Wessels, Duane wrote: > > A quick update on this change: A 2048-bit ZSK has been pre-published in the root zone as of September 20. We are not aware of any issues related to the appearance of the larger key. > > In less than 48 hours we will being publishing root zones signed with the 2048-bit ZSK. I will send another note once that has happened. If you observe any problems related to this change, please contact Verisign's customer service at info at verisign-grs.com. > > Duane W. > >> On Jul 28, 2016, at 3:37 PM, Wessels, Duane wrote: >> >> As you may know, Verisign, in its role as the Root Zone Maintainer >> is also the operator of the root zone Zone Signing Key (ZSK). Later >> this year, we will increase the size of the ZSK from 1024-bits to >> 2048-bits. >> >> The root zone ZSK is normally rolled every calendar quarter, as per >> our ?DNSSEC Practice Statement for the Root Zone ZSK operator.?[1] >> The ZSK public keys are signed at quarterly key signing ceremonies >> by ICANN in its role as the IANA Functions Operator. >> >> On September 20, 2016 the 2048-bit ZSK will be pre-published in the >> root zone, following the standard ZSK rollover procedure. We intend >> to begin publishing root zones signed with the first 2048-bit ZSK >> on October 1, 2016. >> >> Some details of the ZSK size transition have recently been presented >> at the DNS-OARC, NANOG, RIPE, ICANN, and IETF meetings.[2] If you >> have any questions or concerns, please feel free to contact us at >> zms at verisign.com. >> >> Please feel free to forward this message to anyone who might not have >> seen it here. >> >> [1] https://www.verisign.com/assets/dps-zsk-operator-1532.pdf >> [2] https://ripe72.ripe.net/wp-content/uploads/presentations/168-verisign-zsk-change.pdf >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From nanog at ics-il.net Sat Oct 1 14:22:32 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 1 Oct 2016 09:22:32 -0500 (CDT) Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: Message-ID: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> That sort of thing has never bothered me much. If the platform is so great, surely it'll last more than a few years. What's the MTBF on these things? Decades? Better power performance, newer features, higher capacities sure are all great reasons to get newer hardware. EOL isn't. Don't too many of you adopt that strategy, though. I still want my source of cheap EOL hardware. :-) ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Matt Freitag" To: "Saku Ytti" Cc: "nanog list" Sent: Friday, September 30, 2016 3:50:25 PM Subject: Re: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos Pedro, Please also keep in mind that the Juniper EX4500 is an end of life product. Soon you won't be able to get Juniper to support you. That's why there are so many for so cheap on eBay. Matt Freitag Network Engineer I Information Technology Michigan Technological University (906) 487-3696 <%28906%29%20487-3696> https://www.mtu.edu/ https://www.it.mtu.edu/ On Fri, Sep 30, 2016 at 4:06 PM, Saku Ytti wrote: > On 30 September 2016 at 22:42, Pedro wrote: > > Hey Pedro, > > > I have some idea to put switch before bgp router in order to terminate > isp > > 10G uplinks on switch, not router. Main reason is that could be some > kind of > > 1st level of defence against ddos, second reason, less important, save > cost > > of router ports, do many port mirrors. > > I don't understand your rationale, unless your router is software box, > but as it has 10G interface, probably not. > Your router should be able to limit packets in HW, likely with better > counter and filtering options than cheap switch. > > -- > ++ytti > From nanog at ics-il.net Sat Oct 1 14:24:22 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 1 Oct 2016 09:24:22 -0500 (CDT) Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: Message-ID: <752510708.190.1475331853802.JavaMail.mhammett@ThunderFuck> I like putting a switch in front so then I can run two routers behind and get a /29 from the upstream. I can then do router maintenance, upgrades, etc. without taking the circuit down. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Pedro" To: nanog at nanog.org Sent: Friday, September 30, 2016 2:42:37 PM Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos Hello, I have some idea to put switch before bgp router in order to terminate isp 10G uplinks on switch, not router. Main reason is that could be some kind of 1st level of defence against ddos, second reason, less important, save cost of router ports, do many port mirrors. I think about N3K-C3064PQ or Juniper ex4500 because there are quite cheap and a lot of on Ebay. I would like on nexus or juniper try use some feature: - limit udp, icmp, bum packets (bandwith,pps) at ingress tagged port or vlan - create counters: passed and dropped packets, best way to get this counters via snmp oid, sent snmp traps, syslog etc in order to monitor or even as a action shut down port - port mirror from many ports/vlans to multiple port (other anty ddos solutions) - limited bgp but with flowspec to comunicate with another anty ddos devices I'm also wondering how this feature above impact on cpu/whole switch. It can be some performance degradation ot all of this feature are done in hardware, with wirespeeed ? Which model will better to do this ? Thanks for any advice, Pedro --- Ta wiadomo?? zosta?a sprawdzona na obecno?? wirus?w przez oprogramowanie antywirusowe Avast. https://www.avast.com/antivirus From james.jun at towardex.com Sat Oct 1 15:12:49 2016 From: james.jun at towardex.com (James Jun) Date: Sat, 1 Oct 2016 11:12:49 -0400 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> References: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> Message-ID: <20161001151249.GA36333@mis10.towardex.com> On Sat, Oct 01, 2016 at 09:22:32AM -0500, Mike Hammett wrote: > Better power performance, newer features, higher capacities sure are all great reasons to get newer hardware. EOL isn't. Don't too many of you adopt that strategy, though. I still want my source of cheap EOL hardware. :-) We also want support contracts from our vendors. EOL boxes get removed from support availability within few years of the announcement. James From josh at kyneticwifi.com Sat Oct 1 15:14:42 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Sat, 1 Oct 2016 10:14:42 -0500 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: <20161001151249.GA36333@mis10.towardex.com> References: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> <20161001151249.GA36333@mis10.towardex.com> Message-ID: Again, keep doing that :P Be sure to eBay it for a reasonable price when you are done! On Oct 1, 2016 10:12 AM, "James Jun" wrote: > On Sat, Oct 01, 2016 at 09:22:32AM -0500, Mike Hammett wrote: > > Better power performance, newer features, higher capacities sure are all > great reasons to get newer hardware. EOL isn't. Don't too many of you adopt > that strategy, though. I still want my source of cheap EOL hardware. :-) > > We also want support contracts from our vendors. EOL boxes get removed > from support availability within few years of the announcement. > > James > From saku at ytti.fi Sat Oct 1 15:17:42 2016 From: saku at ytti.fi (Saku Ytti) Date: Sat, 1 Oct 2016 18:17:42 +0300 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: <20161001151249.GA36333@mis10.towardex.com> References: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> <20161001151249.GA36333@mis10.towardex.com> Message-ID: On 1 October 2016 at 18:12, James Jun wrote: > We also want support contracts from our vendors. EOL boxes get removed from support availability within few years of the announcement. Support, particularly software maintenance is indeed the key deadline, after that you're on your own. For EX this would be 2019 or 2021 depending on model, if that fits to your amortisation times, then it's fine. You may get more out of it, but you can't build business case on it. -- ++ytti From james.jun at towardex.com Sat Oct 1 15:24:08 2016 From: james.jun at towardex.com (James Jun) Date: Sat, 1 Oct 2016 11:24:08 -0400 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: <164094289.182.1475331745154.JavaMail.mhammett@ThunderFuck> <20161001151249.GA36333@mis10.towardex.com> Message-ID: <20161001152408.GA36649@mis10.towardex.com> On Sat, Oct 01, 2016 at 06:17:42PM +0300, Saku Ytti wrote: > On 1 October 2016 at 18:12, James Jun wrote: > > > We also want support contracts from our vendors. EOL boxes get removed from support availability within few years of the announcement. > > Support, particularly software maintenance is indeed the key deadline, > after that you're on your own. For EX this would be 2019 or 2021 > depending on model, if that fits to your amortisation times, then it's > fine. You may get more out of it, but you can't build business case on > it. Yup, exactly. There are things to keep around from used market for unimportant stuff (OOB etc), but software maintenance support on production box is key. James From mike-nanog at tiedyenetworks.com Sat Oct 1 15:29:11 2016 From: mike-nanog at tiedyenetworks.com (Mike) Date: Sat, 1 Oct 2016 08:29:11 -0700 Subject: Root Zone DNSSEC Operational Update -- ZSK length change In-Reply-To: <909E9B16-CA18-4738-827D-BDF5F08A464E@verisign.com> References: <626768A0-5D22-4B97-8EFC-174894B7F297@verisign.com> <909E9B16-CA18-4738-827D-BDF5F08A464E@verisign.com> Message-ID: On 10/01/2016 06:36 AM, Wessels, Duane wrote: > I'm pleased to announce that this change is now complete. As of 13:34 UTC on October 1, 2016 the root zone has been signed and published with a 2048-bit ZSK. Please contact myself of Verisign customer service (info at verisign-grs.com) if you observe any problems related to this change. > > Duane W. I emailed you but got a 'host not found' error. Does that count as a problem related to the change.....? Lol From jra at baylink.com Sun Oct 2 01:25:31 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sun, 2 Oct 2016 01:25:31 +0000 (UTC) Subject: Request for comment -- BCP38 In-Reply-To: References: <0c6509c7-6a67-0527-5fe4-3175a0553010@satchell.net> <78BDCC2C-0678-44F5-8367-0C89693F3F93@mykolab.com> <20160926144724.GD15891@sizone.org> <20160926151254.GH16464@bamboo.slabnet.com> Message-ID: <876300215.2242.1475371531180.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Laszlo Hanyecz" >> If you have links from both ISP A and ISP B and decide to send traffic >> out ISP A's link sourced from addresses ISP B allocated to you, ISP A >> *should* drop that traffic on the floor. There is no automated or >> scalable way for ISP A to distinguish this "legitimate" use from >> spoofing; unless you consider it scalable for ISP A to maintain >> thousands if not more "exception" ACLs to uRPF and BCP38 egress >> filters to cover all of the cases of customers X, Y, and Z sourcing >> traffic into ISP A's network using IPs allocated to them by other ISPs? > > This is a legitimate and interesting use case that is broken by BCP38. > The effectiveness of BCP38 at reducing abuse is dubious, but the > benefits of asymmetric routing are well understood. Why should everyone > have to go out of their way to break this.. it works fine if you just > don't mess with it. Let me see if I have your argument straight: In order to enable an "interesting" use case that is used by maybe well under 1% of end nodes not in PI address space, we should decide *not* to do something which makes it much easier to protect attack targets against well over 75% of incoming attacks of ridiculous (>OC-12) bandwidth? Is that what you're advocating? No. Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From jra at baylink.com Sun Oct 2 01:27:13 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sun, 2 Oct 2016 01:27:13 +0000 (UTC) Subject: Request for comment -- BCP38 In-Reply-To: <20160926160433.14119.qmail@ary.lan> References: <20160926160433.14119.qmail@ary.lan> Message-ID: <1996558649.2243.1475371633452.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "John Levine" >>If you have links from both ISP A and ISP B and decide to send traffic out >>ISP A's link sourced from addresses ISP B allocated to you, ISP A *should* >>drop that traffic on the floor. There is no automated or scalable way for >>ISP A to distinguish this "legitimate" use from spoofing; unless you >>consider it scalable for ISP A to maintain thousands if not more >>"exception" ACLs to uRPF and BCP38 egress filters to cover all of the cases >>of customers X, Y, and Z sourcing traffic into ISP A's network using IPs >>allocated to them by other ISPs? > > I gather the usual customer response to this is "if you don't want our > $50K/mo, I'm sure we can find another ISP who does." Come on, John. Anyone spending 50K a month belongs in PI space with BGP, and they're a big enough customer for the ISPs to both put exception rules in their ingress filters even if they're not. Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From jra at baylink.com Sun Oct 2 01:35:01 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sun, 2 Oct 2016 01:35:01 +0000 (UTC) Subject: Request for comment -- BCP38 In-Reply-To: <20160927043742.njtfx7xytjcbflfx@slab-wks-04.int.slabnet.com> References: <20160926180355.1229.qmail@ary.lan> <20160926210827.AD092551412D@rock.dv.isc.org> <20160927043742.njtfx7xytjcbflfx@slab-wks-04.int.slabnet.com> Message-ID: <1323564052.2270.1475372101381.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Hugo Slabbert" > This seems to have split into a few different sub-threads and had some > cross-talk on which network is being described where (thanks in no small > part to my flub on John's figures and target), but this seems to be exactly > the kind of info folks are looking for but missing at > http://www.bcp38.info. I heartily encourage people to add content to the wiki for network types that I'm insufficiently familiar with; cookbook entries are where I'd like to see it end up. If anyone wants to contribute please poke me or Alain for an account (keeping a MediaWiki despammed is a fulltime job these days, if you allow user created accounts, so we don't). The address to poke at is moderator (at) bcp38.info Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From jra at baylink.com Sun Oct 2 01:39:10 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sun, 2 Oct 2016 01:39:10 +0000 (UTC) Subject: Request for comment -- BCP38 In-Reply-To: <87shsly3p4.fsf@mid.deneb.enyo.de> References: <0c6509c7-6a67-0527-5fe4-3175a0553010@satchell.net> <1894316783.1650.1474905622635.JavaMail.mhammett@ThunderFuck> <871t06sby2.fsf@mid.deneb.enyo.de> <52cdb2fe-ef51-a15a-d6a3-4cb55b0b3040@gmail.com> <87y42dzpwz.fsf@mid.deneb.enyo.de> <87shsly3p4.fsf@mid.deneb.enyo.de> Message-ID: <972112034.2283.1475372350677.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Florian Weimer" > * Jason Iannone: >> Are urpf and bcp38 interchangeable terms in this discussion? It seems >> impractical and operationally risky to implement two unique ways to dos >> customers. What are the lessons learned by operators doing static output >> filters, strict urpf, or loose/feasible urpf? > > Historically (in 1998, when RFC 2267 was released), BCP 38 was an > egress filter applied at the AS boundary. You meant ingress, no? The control of the address space allocation resides with the upstream, as must control of the filtering. You *can* do BCP38 egress filtering on your network, but that filter would *be in control of the Bad Guys* whom we're trying to kill off. The filtering needs to be on the other side of the administrative span of control fence. Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From jra at baylink.com Sun Oct 2 01:42:49 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sun, 2 Oct 2016 01:42:49 +0000 (UTC) Subject: BCP38 adoption "incentives"? In-Reply-To: References: <95921eaf-e215-9454-046e-375a86f19c33@satchell.net> Message-ID: <1823457432.2295.1475372569169.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Joe Klein" > What would it take to test for BCP38 for a specific AS? There's a tester tool, which I believe I put a link to on the wiki. One of the Cali tech universities; Berkeley? Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From hugo at slabnet.com Sun Oct 2 03:37:50 2016 From: hugo at slabnet.com (Hugo Slabbert) Date: Sat, 1 Oct 2016 20:37:50 -0700 Subject: Kudos to Rogers Wireless on IPv6 deployment Message-ID: <20161002033750.GH3770@bamboo.slabnet.com> So frequently on this list we hear people asking/begging their providers for IPv6 roadmaps or chastising them for the lack of same, that I thought it might be nice to actually give props to a provider actually moving the needle. I was pleasantly surprised today to notice an IPv6 address on my Android smartphone on the Rogers Wireless LTE network. I had to do a double-take and poke through test-ipv6.com to make sure something wasn't amiss, but there it was: honest-to-$deity dual stack service on a Canadian mobile provider, with a dual-stack resolver and everything! ;) So, kudos, Rogers Wireless! So that's Rogers on the wireless side (with Telus Mobility at last check being in early stages but not yet fully rolled out), and basically Rogers, Telus and a bunch of smaller or regional ISPs that have deployed IPv6 on residential and/or business wired service. Shaw? Bell? (FYI Bell, your IPv6 Starter Kit linked from http://ipv6.bell.ca/ currently hits a 404. -- Hugo Slabbert | email, xmpp/jabber: hugo at slabnet.com pgp key: B178313E | also on Signal -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From lyndon at orthanc.ca Sun Oct 2 04:50:40 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Sat, 1 Oct 2016 21:50:40 -0700 Subject: Kudos to Rogers Wireless on IPv6 deployment In-Reply-To: <20161002033750.GH3770@bamboo.slabnet.com> References: <20161002033750.GH3770@bamboo.slabnet.com> Message-ID: <11C357B1-0852-43F6-B2DE-5D23A0D2FCEE@orthanc.ca> > On Oct 1, 2016, at 8:37 PM, Hugo Slabbert wrote: > > So, kudos, Rogers Wireless! This has also been live on Roger's Fido sub-brand for a while now, too. 2605:8d80:484:: is live in Vancouver. --lyndon From swmike at swm.pp.se Sun Oct 2 05:10:21 2016 From: swmike at swm.pp.se (Mikael Abrahamsson) Date: Sun, 2 Oct 2016 07:10:21 +0200 (CEST) Subject: Kudos to Rogers Wireless on IPv6 deployment In-Reply-To: <20161002033750.GH3770@bamboo.slabnet.com> References: <20161002033750.GH3770@bamboo.slabnet.com> Message-ID: On Sat, 1 Oct 2016, Hugo Slabbert wrote: > So, kudos, Rogers Wireless! http://labs.apnic.net/cgi-bin/ccpagev6?c=CA Sort on "samples". Seems Telus and Rogers are the only top10 with any double digit % IPv6 users. Telus is at 65-70%, that's a really good number. -- Mikael Abrahamsson email: swmike at swm.pp.se From theodore at ciscodude.net Sun Oct 2 07:20:37 2016 From: theodore at ciscodude.net (Theodore Baschak) Date: Sun, 2 Oct 2016 02:20:37 -0500 Subject: Kudos to Rogers Wireless on IPv6 deployment In-Reply-To: <20161002033750.GH3770@bamboo.slabnet.com> References: <20161002033750.GH3770@bamboo.slabnet.com> Message-ID: <99A5D04C-BEB3-4634-97B2-8003491481D7@ciscodude.net> I'm also seeing IPv6 on Rogers 4g/LTE on an Android in Winnipeg! Looks like I'm part of 2605:8d80:400::/38 Theodore Baschak - AS395089 - Hextet Systems https://ciscodude.net/ - https://hextet.systems/ http://mbix.ca/ > On Oct 1, 2016, at 10:37 PM, Hugo Slabbert wrote: > > So frequently on this list we hear people asking/begging their providers for IPv6 roadmaps or chastising them for the lack of same, that I thought it might be nice to actually give props to a provider actually moving the needle. > > I was pleasantly surprised today to notice an IPv6 address on my Android smartphone on the Rogers Wireless LTE network. I had to do a double-take and poke through test-ipv6.com to make sure something wasn't amiss, but there it was: honest-to-$deity dual stack service on a Canadian mobile provider, with a dual-stack resolver and everything! ;) > > So, kudos, Rogers Wireless! > > So that's Rogers on the wireless side (with Telus Mobility at last check being in early stages but not yet fully rolled out), and basically Rogers, Telus and a bunch of smaller or regional ISPs that have deployed IPv6 on residential and/or business wired service. Shaw? Bell? (FYI Bell, your IPv6 Starter Kit linked from http://ipv6.bell.ca/ currently hits a 404. > > -- > Hugo Slabbert | email, xmpp/jabber: hugo at slabnet.com > pgp key: B178313E | also on Signal > From octalnanog at alvarezp.org Sun Oct 2 08:34:31 2016 From: octalnanog at alvarezp.org (Octavio Alvarez) Date: Sun, 2 Oct 2016 01:34:31 -0700 Subject: Request for comment -- BCP38 In-Reply-To: References: <0c6509c7-6a67-0527-5fe4-3175a0553010@satchell.net> <78BDCC2C-0678-44F5-8367-0C89693F3F93@mykolab.com> <20160926144724.GD15891@sizone.org> <20160926151254.GH16464@bamboo.slabnet.com> Message-ID: <57F0C697.2070107@alvarezp.org> On 09/26/2016 08:47 AM, Laszlo Hanyecz wrote: >> If you have links from both ISP A and ISP B and decide to send traffic >> out ISP A's link sourced from addresses ISP B allocated to you, ISP A >> *should* drop that traffic on the floor. There is no automated or >> scalable way for ISP A to distinguish this "legitimate" use from >> spoofing; unless you consider it scalable for ISP A to maintain >> thousands if not more "exception" ACLs to uRPF and BCP38 egress >> filters to cover all of the cases of customers X, Y, and Z sourcing >> traffic into ISP A's network using IPs allocated to them by other ISPs? >> > > This is a legitimate and interesting use case that is broken by BCP38. > The effectiveness of BCP38 at reducing abuse is dubious, but the > benefits of asymmetric routing are well understood. Why should everyone > have to go out of their way to break this.. it works fine if you just > don't mess with it. This is really wrong. Any ISP reserves the right to drop irrelevant traffic, traffic that does not belong to its network (read: dropping traffic that is not required to fulfill or is preventing the fulfillment of its contractual and technical requirements). BCP38 does not get in the way of the above and provides some potential benefits like avoiding blacklists in some cases, detecting attacks and hacked computers and contributing to the greater good of the community (yes, some ISPs choose to be good netizens as much as possible, and this is good). This means that applying BCP38 is just natural for those that wish to adopt it. Furthermore, being against it means being against the right to drop irrelevant traffic. In turn, this means that whenever a technical problem comes in conflict with BCP38, it is just a sign that there is some underlying technical flaw in the global Internet structure that should be addressed. I see this as a feature of BCP38. BCP38 does not break anything that it is not already broken, like the PD-addressing multihoming problem. The problem is not BCP38. Octavio. From fw at deneb.enyo.de Sun Oct 2 12:59:16 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 02 Oct 2016 14:59:16 +0200 Subject: Request for comment -- BCP38 In-Reply-To: <972112034.2283.1475372350677.JavaMail.zimbra@baylink.com> (Jay R. Ashworth's message of "Sun, 2 Oct 2016 01:39:10 +0000 (UTC)") References: <0c6509c7-6a67-0527-5fe4-3175a0553010@satchell.net> <1894316783.1650.1474905622635.JavaMail.mhammett@ThunderFuck> <871t06sby2.fsf@mid.deneb.enyo.de> <52cdb2fe-ef51-a15a-d6a3-4cb55b0b3040@gmail.com> <87y42dzpwz.fsf@mid.deneb.enyo.de> <87shsly3p4.fsf@mid.deneb.enyo.de> <972112034.2283.1475372350677.JavaMail.zimbra@baylink.com> Message-ID: <8760pac3ff.fsf@mid.deneb.enyo.de> * Jay R. Ashworth: > ----- Original Message ----- >> From: "Florian Weimer" > >> * Jason Iannone: > >>> Are urpf and bcp38 interchangeable terms in this discussion? It seems >>> impractical and operationally risky to implement two unique ways to dos >>> customers. What are the lessons learned by operators doing static output >>> filters, strict urpf, or loose/feasible urpf? >> >> Historically (in 1998, when RFC 2267 was released), BCP 38 was an >> egress filter applied at the AS boundary. > > You meant ingress, no? It's a bit murky. Section 4 suggests that it's not possible to apply ingress filters on dialup access concnetrators. > The control of the address space allocation resides with the upstream, > as must control of the filtering. That's not really true for customers who maintain their own routes and IP assignments/allocations. > You *can* do BCP38 egress filtering on your network, but that filter > would *be in control of the Bad Guys* whom we're trying to kill off. If you can't do ingress filtering (e.g. you do not give customers dedicated physical ports, or the equipment does not allow tying ports to specific IP addresses), egress filtering is surely better than nothing at all. In theory, it would not matter because the other side should have a matching ingress filter. In practice, egress filtering would make a significant difference in traceability of attacks. Any additional filtering would do so. Again, the goal should not be to deploy specific techonology in a certain way, but to reduce spoofing and attacks which cannot be traced back to the packet sources. From list at satchell.net Sun Oct 2 16:25:50 2016 From: list at satchell.net (Stephen Satchell) Date: Sun, 2 Oct 2016 09:25:50 -0700 Subject: Request for comment -- BCP38 In-Reply-To: <972112034.2283.1475372350677.JavaMail.zimbra@baylink.com> References: <0c6509c7-6a67-0527-5fe4-3175a0553010@satchell.net> <1894316783.1650.1474905622635.JavaMail.mhammett@ThunderFuck> <871t06sby2.fsf@mid.deneb.enyo.de> <52cdb2fe-ef51-a15a-d6a3-4cb55b0b3040@gmail.com> <87y42dzpwz.fsf@mid.deneb.enyo.de> <87shsly3p4.fsf@mid.deneb.enyo.de> <972112034.2283.1475372350677.JavaMail.zimbra@baylink.com> Message-ID: On 10/01/2016 06:39 PM, Jay R. Ashworth wrote: > You *can* do BCP38 egress filtering on your network, but that filter > would *be in control of the Bad Guys* whom we're trying to kill off. I don't see how you arrive at this conclusion. For an aggregating router, the Bad Guys(tm) don't get anywhere near the control plane of the thing. Besides, my security training (such as it is) demands that one implement defence in depth. Specifically, if the Bad Guys(tm) find a way around my ingress filtering, the egress filtering will bump 'em off. Where egress filtering really makes sense is with tunnels over SSH. I haven't found where I can "hook into" a SSH tunnel with Linux. I can turn off shell (and do), but the inbound packets look like local origination to the NetFilter. And (at this early time) The Rules(sm) say that you always ACCEPT packets to and from "lo". I've learned from hard experience that violating that rule breaks a lot of stuff. Then there is the web server case. The Bad Guys(tm) have access to PHP, or Perl, or even a user-level shell, but again NO ACCESS TO THE CONTROL PLANE. Do we really want web-generated packets to get a bye? (I want to put BGP egress filters on my mail servers, my FTP servers, my time servers, my *anything* servers. It's easy, and it means the defence gets as close to the source as I can get it.) > The filtering needs to be on the other side of the administrative > span of control fence. No reason NOT to have filtering on BOTH sides of that fence... From jay at west.net Sun Oct 2 19:15:00 2016 From: jay at west.net (Jay Hennigan) Date: Sun, 2 Oct 2016 12:15:00 -0700 Subject: Request for comment -- BCP38 In-Reply-To: <20160926163705.GE3770@bamboo.slabnet.com> References: <20160926160433.14119.qmail@ary.lan> <303915647.1728.1474906508724.JavaMail.mhammett@ThunderFuck> <20160926162155.GI16464@bamboo.slabnet.com> <20160926163705.GE3770@bamboo.slabnet.com> Message-ID: On 9/26/16 9:37 AM, Hugo Slabbert wrote: > > On Mon 2016-Sep-26 12:25:58 -0400, John R. Levine wrote: > >>>>> I gather the usual customer response to this is "if you don't want our >>>>> $50K/mo, I'm sure we can find another ISP who does." >> >>> I myself am talking about the latter and included the option of PI >>> space to cover that (although I guess at some point this can be made >>> fly with PA space from another provider if both providers are willing >>> enough to play ball), though from the $50/mo figure John listed, I'm >>> assuming he's talking about the latter. >> >> Who said $50/mo? > > Apparently I need even more caffeine that I first imagined... > > If we're talking about networks with that kind of MRC, is it really that > far of a stretch to require PI space for this? Then again: If we're > talking about that kind of MRC, then I'm assuming ISP A can be coaxed to > allow explicit and well-defined exceptions on the customer's links. > > This discussion started wrt to COTS dual-ISP routers though, as > mentioned in Ken Chase's message, no? Where I'm assuming we're talking > about mom-and-pop operations rather than a $50K/mo business account. This is getting insanely silly, especially for this list. A $50K/mo customer should have PI space and announce via BGP to both (all) upstreams. He isn't going to do this with a COTS Belkin "router". A mom-and-pop with said Belkin router who ties to source its /29 or dynamic /32 from ISP A out ISP B isn't going to get very far with it. The COTS "router" isn't going to NAT the traffic out the wrong interface in the first place. In any case, ISP B will never see the return traffic. The rest of the Net isn't going to accept its /29 and /32 random advertisements, period. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV From jcurran at arin.net Sun Oct 2 20:56:25 2016 From: jcurran at arin.net (John Curran) Date: Sun, 2 Oct 2016 20:56:25 +0000 Subject: ARIN legacy block transfer process In-Reply-To: References: Message-ID: On 30 Sep 2016, at 12:49 PM, Bryan Fields wrote: > > I'm trying to find a place on ARIN's website where this is addressed, but > coming up short. I'm not the seller or buyer in this, but basically someone > has a legacy block allocated by Postel and wants to sell the block as it's an > owned asset. > > What's the process to get ARIN to move the admin/ownership of this? Do they > only need to see a valid asset purchase agreement? There is no legacy RSA for > this. Bryan - It is not required (but if they were presently under LRSA, then the transfer would indeed be faster since the organization and IP address block would not have to be revalidated?) > I'm thinking of referring both parties to an experienced broker as well. That certainly works - there is a list of several brokers available here and many of them are very experienced in facilitating transfers under a wide variety of circumstances. > Does anyone have current process experience with this? Yes, I seem to have some experience with these processes? The legacy resource holder has to be validated by ARIN. This can be accomplished by simply making sure that the seller is a point-of-contact on the organization which hold rights to the IP address block (and is smart thing to do in any case, since they can then update the contacts, correct reverse DNS servers, etc.) You can have the source (i.e. the legacy resource holder) contact ARIN?s hostmaster (Hostmaster Arin ), or have them call us Monday through Friday, 7:00 AM to 7:00 PM ET, Phone: +1.703.227.0660 and we?ll walk them through the organization recovery process. Once the legacy holder has control of the organization and IP address block, they can submit a transfer ticket at any time. ARIN will seek documentation behind the transfer, and will confirm that the recipient is valid. For a recipient in another region, we seek concurrence from the serving RIR that the recipient is valid. If you need any additional assistance, feel free to reach out to myself or the ARIN registration services helpdesk. Thanks, /John John Curran President and CEO ARIN p.s. Apologies for the delay in getting you this response - it has been a bit hectic these last 48 hours... From jcurran at istaff.org Sun Oct 2 21:30:58 2016 From: jcurran at istaff.org (John Curran) Date: Sun, 2 Oct 2016 17:30:58 -0400 Subject: ARIN legacy block transfer process In-Reply-To: <20160930174745.GF12280@ernw.de> References: <20c65b90-59d6-6e48-caae-8aa58985f0e7@rollernet.us> <20160930174745.GF12280@ernw.de> Message-ID: <191ED970-3D68-4D7A-A731-8082AE1134C1@istaff.org> On 30 Sep 2016, at 1:47 PM, Enno Rey wrote: > ... > Note also there's voices recommending not to sign an RSA for legacy space (in certain situations, at least), see http://ipv4marketgroup.com/dont-sign-an-rsa-during-your-82-ipv4-transfer/. Probably best to talk to a variety of IP brokers (including the one publishing the above blog) for more current information, as this has been a fairly rapidly changing area and the referenced blog contains both inaccuracies and statements that lack context. For example, there is a suggestion not to undertake a ?NRPM 8.2 Merger and Acquisition IPV4 transfer? due to the need to sign an RSA in the process. The advice may be applicable to some who are adverse to the RSA, but is specifically about only one type of transfer, i.e. transfer via merger and acquisition, not an specified IP address block sale as most people consider? Another example is the assertion that "ARIN says it will then not allow transfers from ARIN to RIPE, once inter-RIR transfers are allowed by RIPE, because there is no ?like? needs based policy.? While that might have been correct at one point in time, RIPE did adopt a rather generous needs-based policy that is indeed compatible with ARIN?s inter-RIR transfer policy, and we?ve done many transfers to the RIPE region since that time. The combination of inaccuracies and outdated information dilute the value of historical blog posts, so caveat emptor when revisiting such? FYI, /John John Curran President and CEO ARIN From jcurran at istaff.org Sun Oct 2 21:57:40 2016 From: jcurran at istaff.org (John Curran) Date: Sun, 2 Oct 2016 17:57:40 -0400 Subject: ARIN legacy block transfer process In-Reply-To: <20160930174203.GA22877@reptiles.org> References: <20160930174203.GA22877@reptiles.org> Message-ID: On 30 Sep 2016, at 1:42 PM, Jim Mercer wrote: > > On Fri, Sep 30, 2016 at 01:22:19PM -0400, William Herrin wrote: >> 1. Buyer gets approved by ARIN for a specified transfer in the amount >> to be purchased. Signs RSA. >> 2. Legacy registrant (seller) signs the LRSA, proves identity and >> authority to ARIN, places legacy block under the LRSA. > > i don't recall having to sign an LRSA, but they certainly do need proof that > you are the entity that holds the block. That?s correct Jim. >> 3. Buyer requests specified transfer from ARIN, seller authorizes it. >> 4. ARIN transfers registration to buyer, now under the RSA. > > if you are a legacy holder, and you want a dead simple way to sell off your > rights to blocks, register for the STLS, and get your blocks added to the > list. > https://www.arin.net/public/secure/downloads/index.xhtml > (at the bottom) > > this is also a good place to go if you want to buy rights to a block. > > you can also solicit offers to buy from anyone on the list. Also correct - ARIN runs the Specified Transfer Listing Service (STLS) so that parties that wish to be vetted by ARIN may do so in advance, and also optionally be listed as ?interested in offers? from others using the STLS service. Also, potential recipients may have their IP address need verified in advance, therefore making them a qualified recipient in the ARIN region. It?s completely optional, but available for those who wish to validate their half of a transfer (whether as a source or recipient) in advance. > works well, and less dodgy-ness, since everyone on the list has been > vetted (to some degree) by ARIN. I?m glad you like the STLS service; it does work well for straightforward transfers among parties. There are some cases (complex corporate changes, long absent or incorrect registration information, etc.) where parties may find the support and guidance of one of the brokers helpful; it doesn?t matter to ARIN either way nor does it change the criteria that we apply - all transfers follow the same policy. Thanks! /John John Curran President and CEO ARIN From jcurran at arin.net Sun Oct 2 22:18:24 2016 From: jcurran at arin.net (John Curran) Date: Sun, 2 Oct 2016 22:18:24 +0000 Subject: ARIN legacy block transfer process In-Reply-To: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> Message-ID: <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> On 30 Sep 2016, at 1:34 PM, Bryan Fields wrote: > > On 9/30/16 1:22 PM, William Herrin wrote: >> Note that you can't sell the block as an "owned asset" and have ARIN >> recognize the change. ARIN does not recognize ownership of IP address >> blocks, they only recognize registration and authorized agents. > > This would seem to be in violation of what the NSF has said about this space. > I thought ARIN was slapped hard once before about this very thing? NSF?s Chief Counsel provided his views on the matter, but had to subsequently clarify that his views were based on NSF?s historic role and noted that he did not speak for the USG on such matters (as they were now properly within the remit of the US Department of Commerce/NTIA? oops!) - The agency with actual authority in these matters (NTIA) subsequently issued a statement of the the US Government?s Internet Protocol Numbering Principles, which noted that ?the American Registry for Internet Numbers (ARIN) is the RIR for Canada, many Caribbean and North Atlantic islands, and the United States and furthermore that the USG ?participates in the development of and is supportive of the policies, processes, and procedures agreed upon by the Internet technical community through ARIN.? i.e. ARIN continues to enforce the community-developed policies on all resources in the registry, and including legal undertakings where necessary to that end. FYI, /John John Curran President and CEO ARIN p.s. Note that all organizations may use ARIN Online to update their ARIN IP number resource records. Organizations that received number resources directly from ARIN have ARIN Online access via their Registration Services Agreement. For organizations that received resources before ARIN?s formation in December 1997 (i.e. ?Legacy Resource Holders?), ARIN has been providing, without any fee or registration services agreement, access to ARIN Online as well as the basic IP registration services in place at the time of ARIN?s formation; this does include the ability to submit tickets to transfer an IP address block to another party. Legacy Resource Holder organizations that wish to receive these basic registration services under a formal agreement, or wish to utilize additional services (such as resource certification, i.e. RPKI), and/or desire a written statement of their rights to their IP number resources must enter into a Registration Services Agreement. From joelja at bogus.com Mon Oct 3 04:59:10 2016 From: joelja at bogus.com (joel jaeggli) Date: Sun, 2 Oct 2016 21:59:10 -0700 Subject: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos In-Reply-To: References: Message-ID: <5f248944-fc02-11f7-123d-62fad7719c07@bogus.com> On 9/30/16 12:42 PM, Pedro wrote: > > Hello, > > I have some idea to put switch before bgp router in order to terminate > isp 10G uplinks on switch, not router. Main reason is that could be some > kind of 1st level of defence against ddos, second reason, less > important, save cost of router ports, do many port mirrors. The distinction on cost of ports is somewhat germain when dealing with things like span ports. maybe strictly speaking if the router platform can handle line rate forwarding at minimum packet size it is just as performant as the switch at both forwarding and probably acl application (there are of course exceptions). in general these switches has substantially smaller port buffers then a router or high end l3 switch platform (qfx10k or ptx for example) would have when spanning ports or doing some statistical multiplexing. Which can be a liability. A number of us no doubt use layer2/3 switches as customer aggregation or indeed peering platforms. and suitability for such may depend on the mix of hardware and software features available as well as non-forwarding attributes such as the amount of memory available. i have boxes for example that have a full table rib but only default route for non-customer routes. the prospects for gettting away with that sort of thing with only 2GB of ram are growing increasingly dire. So i would say this sort thing does work, and with some familiarity with the platforms you become more comfortable with their limitations, but it's not automatically the best route, and the additional bump in the forwarding path is not free of costs or complexity. > I think about N3K-C3064PQ or Juniper ex4500 because there are quite > cheap and a lot of on Ebay. > > I would like on nexus or juniper try use some feature: > > - limit udp, icmp, bum packets (bandwith,pps) at ingress tagged port or > vlan > - create counters: passed and dropped packets, best way to get this > counters via snmp oid, sent snmp traps, syslog etc in order to monitor > or even as a action shut down port > - port mirror from many ports/vlans to multiple port (other anty ddos > solutions) > - limited bgp but with flowspec to comunicate with another anty ddos > devices > > I'm also wondering how this feature above impact on cpu/whole switch. It > can be some performance degradation ot all of this feature are done in > hardware, with wirespeeed ? Which model will better to do this ? > > Thanks for any advice, > Pedro > > --- > Ta wiadomo?? zosta?a sprawdzona na obecno?? wirus?w przez oprogramowanie > antywirusowe Avast. > https://www.avast.com/antivirus > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From hannigan at gmail.com Mon Oct 3 07:57:51 2016 From: hannigan at gmail.com (Martin Hannigan) Date: Mon, 3 Oct 2016 09:57:51 +0200 Subject: ARIN legacy block transfer process In-Reply-To: References: Message-ID: On Sun, Oct 2, 2016 at 10:56 PM, John Curran wrote: > On 30 Sep 2016, at 12:49 PM, Bryan Fields wrote: >> [ clip ] > >> I'm thinking of referring both parties to an experienced broker as well. > > That certainly works - there is a list of several brokers available here > > and many of them are very experienced in facilitating transfers under > a wide variety of circumstances. Morning NANOG'ers; Disclaimer: Get IPv6. Deploy it. Avoid cost. With that said... The theme of the thread is that brokers can be useful. I'd argue they are almost imperative for those wishing to be completely informed. I'm not criticizing, but I'd be careful to understand that along with using the STLS (as well as making transfers) comes additional sacrifice of rights and assumption of liabilities. https://www.arin.net/resources/transfer_listing/tos.pdf That list also missing at least one significant company: ADDREX http://www.addrex.net/ - My personal list of go-to brokers is for questions like asset disposition, acquisition, transfers and the conversation above (alphabetically): Addrex Hilco IP Trading Hope this is helpful. YMMV, and best, -M< From paul at paulstewart.org Mon Oct 3 08:04:38 2016 From: paul at paulstewart.org (Paul Stewart) Date: Mon, 3 Oct 2016 04:04:38 -0400 Subject: Any ISPs using AS852 for IP Transit? In-Reply-To: References: <97EE8152-6FCE-4650-8C26-0B24C4F67F4D@lixfeld.ca> <9B17C89B-5FCB-4030-A12A-48FECAF2D090@lixfeld.ca> <20160915191529.GE16464@bamboo.slabnet.com> <93F8A4A5-B3A3-47C0-A4AC-C630F1D52278@lixfeld.ca> Message-ID: <1C1B414C-82BF-4B82-B566-C3BD9DC7F84F@paulstewart.org> To confirm AS852 and AS577 don?t charge $dayjob for prefix changes ?. they both do them manually though which is a pain :( > On Sep 15, 2016, at 4:09 PM, Theodore Baschak wrote: > > I don't think this is standard across the board with Telus. > > I've also heard (rumours?) of a similar $250 prefix change free associated with Shaw/AS6327 changes before, and also a much larger $750 change prefix change fee with BELL-GT/AS6539, but the customers I know who use them definitely don't get charged these types of fees. > > > Theodore Baschak - AS395089 - Hextet Systems > https://ciscodude.net/ - https://hextet.systems/ > http://mbix.ca/ > >> On Sep 15, 2016, at 2:21 PM, Jason Lixfeld wrote: >> >> Sure. My question was whether every TELUS BGP customer was being charged for these too, or if I?m the only one. If I?m the only one, then I?m obviously caught in some administrative black hole there that I would like to get myself out of. This is something that has only started happening in the last 6 months or so. Prior to that, we were never charged by them for these requests. Unfortunately, my sales rep has been less than helpful in trying to understand what changed to make us susceptible to these new charges. >> >>> On Sep 15, 2016, at 3:15 PM, Hugo Slabbert wrote: >>> >>> So, to be blunt, I would cast this as their charging you NRC for manual work because of their failure to automate this. >>> >>> -- >>> Hugo Slabbert | email, xmpp/jabber: hugo at slabnet.com >>> pgp key: B178313E | also on Signal >>> >>> On Thu 2016-Sep-15 15:09:33 -0400, Jason Lixfeld wrote: >>> >>>> Last time I asked, that wasn?t something that they had implemented, and had no definite plans to do so within any timeframe that was on their radar. >>>> >>>>> On Sep 15, 2016, at 2:50 PM, Steven Schecter wrote: >>>>> >>>>> I question their motivation here and would follow up by asking if they support filtering by IRRdb and are merely trying to encourage the practice? >>>>> >>>>> >>>>> /Steve >>>>> >>>>> On Thu, Sep 15, 2016 at 2:07 PM, Jason Lixfeld wrote: >>>>> If there are any ISPs who use TELUS/AS852 for IP Transit over BGP, I?d be interested in hearing from you. >>>>> >>>>> I?d like to compare notes to see if you are also paying $250 for each BGP prefix filter updated request, or if we?re the only ones? >>>>> >>>>> Thanks in advance! >>>>> >>>>> >>>>> >>>>> -- >>>>> Steven J. Schecter >>>>> (m) 917.676.1646 >>>> >> > From jcurran at arin.net Mon Oct 3 12:09:01 2016 From: jcurran at arin.net (John Curran) Date: Mon, 3 Oct 2016 12:09:01 +0000 Subject: ARIN legacy block transfer process In-Reply-To: References: Message-ID: On 3 Oct 2016, at 3:57 AM, Martin Hannigan > wrote: ... Disclaimer: Get IPv6. Deploy it. Avoid cost. With that said? Indeed: we are likely to have a rather convoluted market supply curve for IPv4 blocks over the next decade, and those dependent on IPv4 as their primary means of network growth will have unpredictable costs as a result. The theme of the thread is that brokers can be useful. I'd argue they are almost imperative for those wishing to be completely informed. I'm not criticizing, but I'd be careful to understand that along with using the STLS (as well as making transfers) comes additional sacrifice of rights and assumption of liabilities. Marty - 100% agreement? while I am quite willing to explain to folks what I believe makes sense for them regarding transfers, I ultimately must represent the best options with respect to delivery of ARIN?s mission to the community. A broker (and/or informed legal counsel) can provide a view which is solely focused on the address holder?s interests ? while this may often lead to the same outcome, that is not assured in all cases. Thanks, /John John Curran President and CEO ARIN From dwessels at verisign.com Mon Oct 3 14:25:40 2016 From: dwessels at verisign.com (Wessels, Duane) Date: Mon, 3 Oct 2016 14:25:40 +0000 Subject: Root Zone DNSSEC Operational Update -- ZSK length change In-Reply-To: References: <626768A0-5D22-4B97-8EFC-174894B7F297@verisign.com> <909E9B16-CA18-4738-827D-BDF5F08A464E@verisign.com> Message-ID: > On Oct 1, 2016, at 11:29 AM, Mike wrote: > > On 10/01/2016 06:36 AM, Wessels, Duane wrote: >> I'm pleased to announce that this change is now complete. As of 13:34 UTC on October 1, 2016 the root zone has been signed and published with a 2048-bit ZSK. Please contact myself of Verisign customer service (info at verisign-grs.com) if you observe any problems related to this change. >> >> Duane W. > I emailed you but got a 'host not found' error. Does that count as a problem related to the change.....? > > Lol > Mike, I hope not but we should rule it out. I will follow up with you privately. Duane W. From randy at psg.com Mon Oct 3 15:50:22 2016 From: randy at psg.com (Randy Bush) Date: Mon, 03 Oct 2016 08:50:22 -0700 Subject: ARIN legacy block transfer process In-Reply-To: <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> Message-ID: > The agency with actual authority in these matters (NTIA) inappropriate use of present tense From selliott at getunwired.com Mon Oct 3 15:56:44 2016 From: selliott at getunwired.com (Shon Elliott) Date: Mon, 3 Oct 2016 15:56:44 +0000 Subject: Sling Media on list? Message-ID: <2FC66B6EEF733844895E94A7FEE4FA52073EEAC8@mbx032-e1-va-4.exch032.serverpod.net> Anyone from Sling Media on NANOG? If so, can you please contact me off-list? We have a net block that you guys have incorrect information for which is causing grief for our mutual customers. Kind Regards, Shon Elliott, KK6TOO selliott at getunwired.com From randy at psg.com Mon Oct 3 15:57:01 2016 From: randy at psg.com (Randy Bush) Date: Mon, 03 Oct 2016 08:57:01 -0700 Subject: ARIN legacy block transfer process In-Reply-To: References: Message-ID: fwiw, i did a transfer with the source in arin. i found arin's folk very very helpful, and the process navigable, even for someone who is administratively impaired such as i. ignore the political positioning, plan ahead, be sure of all your options at each fork in the road, and proceed with caution. if this space is strange to you, recommendations of using a broker or lawyer who has trod the path are apt. randy From jcurran at arin.net Mon Oct 3 16:00:32 2016 From: jcurran at arin.net (John Curran) Date: Mon, 3 Oct 2016 16:00:32 +0000 Subject: ARIN legacy block transfer process In-Reply-To: References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net>, Message-ID: <475E8C4B-9BB1-45E5-9FD1-76A28E8F7421@arin.net> On Oct 3, 2016, at 11:49 AM, Randy Bush wrote: >> The agency with actual authority in these matters (NTIA) > > inappropriate use of present tense Yes, Randy - excellent point. I should have included "at the time" in that sentence... Thanks! /John From jim at reptiles.org Mon Oct 3 16:11:08 2016 From: jim at reptiles.org (Jim Mercer) Date: Mon, 3 Oct 2016 12:11:08 -0400 Subject: ARIN legacy block transfer process In-Reply-To: References: Message-ID: <20161003161108.GA87394@reptiles.org> On Mon, Oct 03, 2016 at 08:57:01AM -0700, Randy Bush wrote: > fwiw, i did a transfer with the source in arin. i found arin's folk > very very helpful, and the process navigable, even for someone who is > administratively impaired such as i. yep, i agree. i recently was the source of a pile of transfers from /24's to a /16, both inside ARIN, as well as a couple out of ARIN into RIPE. also did some with brokers (iptrading.com), as well as direct deals. as long as all the documentation is straight, the process seems to move along cleanly. --jim -- Jim Mercer Reptilian Research jim at reptiles.org +1 416 410-5633 Life should not be a journey to the grave with the intention of arriving safely in a pretty and well preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming "Wow! What a Ride!" -- Hunter S. Thompson From owen at delong.com Mon Oct 3 16:21:51 2016 From: owen at delong.com (Owen DeLong) Date: Mon, 3 Oct 2016 09:21:51 -0700 Subject: ARIN legacy block transfer process In-Reply-To: <20160930174745.GF12280@ernw.de> References: <20c65b90-59d6-6e48-caae-8aa58985f0e7@rollernet.us> <20160930174745.GF12280@ernw.de> Message-ID: That page is rather out of date as the RIPE policy it refers to was passed and then modified to (somewhat) restore needs basis. Owen > On Sep 30, 2016, at 10:47 AM, Enno Rey wrote: > > Hi, > > On Fri, Sep 30, 2016 at 10:26:36AM -0700, Seth Mattinen wrote: >> On 9/30/16 09:49, Bryan Fields wrote: >>> I'm trying to find a place on ARIN's website where this is addressed, but >>> coming up short. I'm not the seller or buyer in this, but basically someone >>> has a legacy block allocated by Postel and wants to sell the block as it's an >>> owned asset. >>> >>> What's the process to get ARIN to move the admin/ownership of this? Do they >>> only need to see a valid asset purchase agreement? There is no legacy RSA for >>> this. >> >> >> It'll have to go under RSA to stay with ARIN. >> >> Or you can do a transfer to RIPE. > > carefully note the "or". > If you first move it under ARIN's jurisdiction (by signing an RSA) and *then* transfer it to RIPE, it won't be "legacy" any more in the course of the 2nd step and RIPE's 2-yr holding period comes into play (=> it can't be transferred during that time). > > Note also there's voices recommending not to sign an RSA for legacy space (in certain situations, at least), see http://ipv4marketgroup.com/dont-sign-an-rsa-during-your-82-ipv4-transfer/. > > best > > Enno > > > > > > > > >> >> ~Seth > > -- > Enno Rey > > ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de > Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 173 6745902 > > Handelsregister Mannheim: HRB 337135 > Geschaeftsfuehrer: Enno Rey > > ======================================================= > Blog: www.insinuator.net || Conference: www.troopers.de > Twitter: @Enno_Insinuator > ======================================================= From list at satchell.net Mon Oct 3 18:58:10 2016 From: list at satchell.net (Stephen Satchell) Date: Mon, 3 Oct 2016 11:58:10 -0700 Subject: Legislative proposal sent to my Congressman Message-ID: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> In thinking over the last DDos involving IoT devices, I think we don't have a good technical solution to the problem. Cutting off people with defective devices they they don't understand, and have little control over, is an action that makes sense, but hurts the innocent. "Hey, Grandma, did you know your TV set is hurting the Internet?" It's the people who foist bad stuff on the people who need to take the responsibility. Indeed, with enough moxie, we could avoid the net saturation problem in the first place. My proposal, as I sent it to my US House Representative: > Fixing the security of the Internet of Things: Now we have had > several distributed denial of service attacks ? generating > eye-popping amounts of network traffic to bury a web site or gamer ? > arguably traced to botnets-for-sale of "hacked" common devices with > Internet connectivity. It's time to look at the problem bad product > design can cause. Not being "computers", many of those devices ? > cameras, televisions, light bulbs, to name a few ? don't have > tough-enough security moxie baked in. And it's not enough to solve > today's attacks, they have to survive new attacks down the road. > > Some of these household items didn't conform to today's Best > Practices, taught in Security 101, with the rules learned (painfully) > over the last 30 years. And then there is the question of installing > security fixes: "Hey, Joe, you have to install an update to your > thermostat and washing machine." Right. > > This is nothing new. What is new is the tsunami of Internet-capable > devices hitting the market and the Internet...and doing it badly. By > sheer numbers, the situation rises to a whole new level of risk to the > nation's communications infrastructure. The magnitude of the problem? > Think how many light bulbs are in the typical house or apartment, and > you get the idea. > > This note comes a little late to the game, but I thought that one > wayto stem the flood of garbage from compromised household stuff is to > treat vulnerabilities that cause spew as design defects, defects as > serious as the exploding batteries in the Samsung Galaxy Note 7. So, > looking at the procedures already in place for dealing with merchandise > that can cause harm, this suggestion. > > Proposed: GIVEN > > * any Internet-connected device, > * "pwned" by cybercriminals, > * that cause significant harm, > * the manufacturer received notice of the defect, and > * did not, or cannot, provide a timely, zero-cost update > > THEREFORE the Consumer Product Safety Commission shall require that > the manufacturer provide a security update to the device within 30 day > of first notice; or failing that, to issue a complete recall of the > defective devices. > > I don't care if it's a television, camera, refrigerator, light bulb, > thermostat, washing machine, wireless access router, smart phone, > desktop computer, server, you-name-it...if it's broke, and can't (or > won't) be fixed, it gets recalled. > > That's the only way manufacturers will take Internet security > seriously. If they have to upgrade the stuff they sell, without > exception, the manufacturers will find a method that will keep their > expense for upgrades down. Upgrades should not be charged to the > customer ? the manufacturer screwed up, they should fix the problem, at > their expense. I further suggest that security testing should be > specifically permitted under law, not be considered part of "reverse > engineering", or other shrink-wrap or copyright restriction. > > The CSPC should develop guidelines for product with embedded computers > that connect to the Internet at large, either directly or indirectly. > > There are a number of things to consider, when building such a > regulation, that come into play that complicated things > > * orphaned devices, > * devices made by companies that have gone out of business, > * imported stuff, > * methods of notification, and > * enforcement > > This is an off-the-top-of-my-head idea. I think it's worth > consideringover other "solutions" I've seen proposed. There is precedent for this with radio and the FCC. According to current law, the owner/operator of the radio equipment is ultimately responsible for non-interference by any transmitter used in the United States. This includes so-called unlicensed transmitters. To help the people who are not radio gurus, the FCC also has a type acceptance program, in which radios have to meet certain requirements as built by the manufacturer. There is another possible wrinkle: if there were legal consequences with selling IoT equipment, businesses making the stuff would take out insurance against claims against them. The underwriters would then take notice, and require that policyholders meet some minimum standards. Remember, we are talking about the "underwriters" who form the first part of the name "Underwriters Laboratories". From UL's web page: > UL is a global independent safety science company with more than a > century of expertise innovating safety solutions from the public > adoption of electricity to new breakthroughs in sustainability, > renewable energy and nanotechnology. Dedicated to promoting safe > living and working environments, UL helps safeguard people, products > and places in important ways, facilitating trade and providing peace > of mind. We could build on these existing frameworks to the advantage of the Internet by mandating certain minimum requirements for equipment sold to the general public. I would suspect that the IETF would need to become involved in this effort, because the standards would have to come from SOMEWHERE. Which is why they are included in the header. There are other people on the Cc: list that might be interested...or might not. Why not nip the IoT problem in the bud? From lyndon at orthanc.ca Mon Oct 3 19:47:22 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 12:47:22 -0700 (PDT) Subject: Legislative proposal sent to my Congressman In-Reply-To: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> Message-ID: > In thinking over the last DDos involving IoT devices, I think we don't have a > good technical solution to the problem. Cutting off people with defective > devices they they don't understand, and have little control over, is an > action that makes sense, but hurts the innocent. "Hey, Grandma, did you know > your TV set is hurting the Internet?" The way this will get solved is for a couple of large ISPs and DDoS targets to sue a few of these IoT device manufacturers into oblivion. --lyndon From fw at deneb.enyo.de Mon Oct 3 19:54:38 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 03 Oct 2016 21:54:38 +0200 Subject: Legislative proposal sent to my Congressman In-Reply-To: (Lyndon Nerenberg's message of "Mon, 3 Oct 2016 12:47:22 -0700 (PDT)") References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> Message-ID: <87int9us1t.fsf@mid.deneb.enyo.de> * Lyndon Nerenberg: >> In thinking over the last DDos involving IoT devices, I think we >> don't have a good technical solution to the problem. Cutting off >> people with defective devices they they don't understand, and have >> little control over, is an action that makes sense, but hurts the >> innocent. "Hey, Grandma, did you know your TV set is hurting the >> Internet?" > > The way this will get solved is for a couple of large ISPs and DDoS > targets to sue a few of these IoT device manufacturers into oblivion. But that does not remove those devices from the network. From lyndon at orthanc.ca Mon Oct 3 19:58:01 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 12:58:01 -0700 (PDT) Subject: Legislative proposal sent to my Congressman In-Reply-To: <87int9us1t.fsf@mid.deneb.enyo.de> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> Message-ID: > But that does not remove those devices from the network. That ship has sailed. From bill at herrin.us Mon Oct 3 20:06:42 2016 From: bill at herrin.us (William Herrin) Date: Mon, 3 Oct 2016 16:06:42 -0400 Subject: ARIN legacy block transfer process In-Reply-To: <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> Message-ID: On Sun, Oct 2, 2016 at 6:18 PM, John Curran wrote: > The agency with actual authority in these matters (NTIA) > subsequently issued a > statement of the the US Government?s Internet Protocol > Numbering Principles, > which noted that ?the American Registry for Internet Numbers > (ARIN) is the RIR > for Canada, many Caribbean and North Atlantic islands, and > the United States > and furthermore that the USG ?participates in the development > of and is supportive > of the policies, processes, and procedures agreed upon by the > Internet technical community through ARIN.? > > > i.e. ARIN continues to enforce the community-developed policies > on all resources in the registry, and including legal undertakings > where necessary to that end. Howdy, In the interest of making the question just about as clear as mud, I would point out several things: 1. The NTIA does not claim to be a source of authority for the management of IP addresses. The things written above are essentially accurate but take care not to read more than was said. 2. No act of congress authorizes the NTIA nor any other branch of the U.S. government to act as a source of authority with respect to IP addresses. \ DARPA had clear authority to control what IP addresses were used on the ARPAnet. NSF had clear authority to control what IP addresses were used on the NSFnet backbone only. Neither claimed any authority over the use of IP addresses on any other network. 3. In it's 20-year history, ARIN has taken no actions whatsoever against legacy address holders inconsistent with legacy address blocks being common law property, save this: ARIN has not consistently updated the registry to reflect a new registrant for an address block unless the new registrant has signed a registration services agreement. Note that ARIN has updated the registrant for legacy registrations without service agreements, just not consistently. ARIN asserts they've taken no action because community developed policy instructs them not to. That is a half-truth. The whole truth is that ARIN has bent over backwards to avoid testing in court whether they have the lawful authority to enforce any policies against legacy address holders, even to the extent of bending or breaking their own policies when settling a court case. For example, Microsoft was permitted to register the Nortel addresses under the weaker Legacy Registration Services Agreement when the plain language of the then-extant policies required the use of the primary RSA. 4. No statute, regulation or judicial finding either confirms or refutes ARIN's ability to lawfully refuse such updates. All relevant cases have either been settled prior to a judicial finding, or resulted in the registration's termination for other reasons (generally fraud of some sort). 5. Regardless of the content's of ARIN's registry, ARIN claims no authority over whether and on whose behalf any specific IP addresses are routed on the Internet, save for those addresses ARIN uses directly for its own purposes. Pragmatically, if you want to buy an address block, the path of least resistance is: register with ARIN. >From a purist "what are my rights" standpoint: John Curran's comments notwithstanding, that's not so clear. Regards, Bill Herrin -- William Herrin ................ herrin at dirtside.com bill at herrin.us Owner, Dirtside Systems ......... Web: From johnl at iecc.com Mon Oct 3 20:39:58 2016 From: johnl at iecc.com (John Levine) Date: 3 Oct 2016 20:39:58 -0000 Subject: Legislative proposal sent to my Congressman In-Reply-To: Message-ID: <20161003203958.10869.qmail@ary.lan> In article you write: >> But that does not remove those devices from the network. > >That ship has sailed. This is where device profiles could help. If enough devices register profiles with the local router, at some point the router's default could be closed, so devices with no profile can't talk to the outside. For a lot of devices like lightbulbs, that would probably make no difference at all. It would mean you couldn't remotely monitor your five year old CCTV camera unless you take in the camera for an upgrade or replace it, but I can't get too upset about that. R's, John From ted.ietf at gmail.com Mon Oct 3 20:54:21 2016 From: ted.ietf at gmail.com (Ted Hardie) Date: Mon, 3 Oct 2016 13:54:21 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: <20161003203958.10869.qmail@ary.lan> References: <20161003203958.10869.qmail@ary.lan> Message-ID: On Mon, Oct 3, 2016 at 1:39 PM, John Levine wrote: > In article you write: > >> But that does not remove those devices from the network. > > > >That ship has sailed. > > This is where device profiles could help. If enough devices register > profiles with the local router, at some point the router's default > could be closed, so devices with no profile can't talk to the outside. > > Hi John, Are you thinking of MUD ( https://datatracker.ietf.org/doc/draft-ietf-opsawg-mud/) here, when you say "register profiles"? regards, Ted > For a lot of devices like lightbulbs, that would probably make no > difference at all. It would mean you couldn't remotely monitor your > five year old CCTV camera unless you take in the camera for an upgrade > or replace it, but I can't get too upset about that. > > R's, > John > > > > From lyndon at orthanc.ca Mon Oct 3 21:10:05 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 14:10:05 -0700 (PDT) Subject: Legislative proposal sent to my Congressman In-Reply-To: <20161003203958.10869.qmail@ary.lan> References: <20161003203958.10869.qmail@ary.lan> Message-ID: > This is where device profiles could help. If enough devices register > profiles with the local router, at some point the router's default > could be closed, so devices with no profile can't talk to the outside. That would be nice, but a manufacturer who can't be bothered to take even the most basic security precautions certainly isn't going to implement this, either. The only cure to this will be changing the law so that the directors of the companies that ship massively insecure devices like these are personally liable for all the financial loss attributed to their products. Bankrupt a few companies' board of directors and you'll start seeing things change in a hurry. --lyndon From jcurran at arin.net Mon Oct 3 21:09:00 2016 From: jcurran at arin.net (John Curran) Date: Mon, 3 Oct 2016 21:09:00 +0000 Subject: ARIN legacy block transfer process In-Reply-To: References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> Message-ID: <5071ADF5-6238-432F-B7F4-5EB113954F6C@arin.net> On 3 Oct 2016, at 4:06 PM, William Herrin wrote: > ... > 3. In it's 20-year history, ARIN has taken no actions whatsoever > against legacy address holders inconsistent with legacy address blocks > being common law property, save this: ARIN has not consistently > updated the registry to reflect a new registrant for an address block > unless the new registrant has signed a registration services > agreement. Note that ARIN has updated the registrant for legacy > registrations without service agreements, just not consistently. > > ARIN asserts they've taken no action because community developed > policy instructs them not to. That is a half-truth. The whole truth > is that ARIN has bent over backwards to avoid testing in court whether > they have the lawful authority to enforce any policies against legacy > address holders, even to the extent of bending or breaking their own > policies when settling a court case. For example, Microsoft was > permitted to register the Nortel addresses under the weaker Legacy > Registration Services Agreement when the plain language of the > then-extant policies required the use of the primary RSA. Bill - You?re not quite correct with these statements, as there have been many court orders that require parties to enter into an RSA and comply with the IP registry policy ? this is actually fairly common outcome of bankruptcy cases. (Refer to for just some examples.) It is true that parties consent to these agreements, as ARIN does not transfer the address block unless in compliance with policy. We have routinely argue that IP address blocks are not common law property and prevail - we have never been ordered to make registry updates contrary to the community policy. > 4. No statute, regulation or judicial finding either confirms or > refutes ARIN's ability to lawfully refuse such updates. All relevant > cases have either been settled prior to a judicial finding, or > resulted in the registration's termination for other reasons > (generally fraud of some sort). Half-correct, in that these bankruptcy cases do result in judicial orders; however, the fact that the parties consent to the terms in order to proceed with the transfer is correct. (Presumably they would do otherwise if they had a valid basis for argument, but that hasn?t happened.) > ... > Pragmatically, if you want to buy an address block, the path of least > resistance is: register with ARIN. > > From a purist "what are my rights" standpoint: John Curran's comments > notwithstanding, that's not so clear. Also correct - hence why I suggested that ?a broker (and/or informed legal counsel) can provide a view which is solely focused on the address holder?s interests? Thanks, /John John Curran President and CEO ARIN From johnl at iecc.com Mon Oct 3 21:11:59 2016 From: johnl at iecc.com (John R. Levine) Date: 3 Oct 2016 17:11:59 -0400 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <20161003203958.10869.qmail@ary.lan> Message-ID: >> This is where device profiles could help. If enough devices register >> profiles with the local router, at some point the router's default >> could be closed, so devices with no profile can't talk to the outside. > > Are you thinking of MUD ( > https://datatracker.ietf.org/doc/draft-ietf-opsawg-mud/) here, when you say > "register profiles"? Yes. Eliot Lear said they're working actively on it. Regards, John Levine, johnl at iecc.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly From johnl at iecc.com Mon Oct 3 21:14:07 2016 From: johnl at iecc.com (John R. Levine) Date: 3 Oct 2016 17:14:07 -0400 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <20161003203958.10869.qmail@ary.lan> Message-ID: >> This is where device profiles could help. If enough devices register >> profiles with the local router, at some point the router's default >> could be closed, so devices with no profile can't talk to the outside. > > That would be nice, but a manufacturer who can't be bothered to take even the > most basic security precautions certainly isn't going to implement this, > either. They will if the routers start rejecting their traffic. > The only cure to this will be changing the law so that the directors of the > companies that ship massively insecure devices like these are personally > liable for all the financial loss attributed to their products. Bankrupt a > few companies' board of directors and you'll start seeing things change in a > hurry. Good luck with that. R's, John From cb.list6 at gmail.com Mon Oct 3 22:24:18 2016 From: cb.list6 at gmail.com (Ca By) Date: Mon, 3 Oct 2016 15:24:18 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> Message-ID: On Monday, October 3, 2016, Lyndon Nerenberg wrote: > In thinking over the last DDos involving IoT devices, I think we don't >> have a good technical solution to the problem. Cutting off people with >> defective devices they they don't understand, and have little control over, >> is an action that makes sense, but hurts the innocent. "Hey, Grandma, did >> you know your TV set is hurting the Internet?" >> > > The way this will get solved is for a couple of large ISPs and DDoS > targets to sue a few of these IoT device manufacturers into oblivion. > > --lyndon > > FTC has a hand in this area https://www.ftc.gov/news-events/press-releases/2016/02/asus-settles-ftc-charges-insecure-home-routers-cloud-services-put From larrysheldon at cox.net Mon Oct 3 22:36:27 2016 From: larrysheldon at cox.net (Larry Sheldon) Date: Mon, 3 Oct 2016 17:36:27 -0500 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: Message-ID: <16951763-5f17-7d29-6e07-2c1dd49cf6af@cox.net> On 10/3/2016 13:58, Stephen Satchell wrote: > In thinking over the last DDos involving IoT devices, I think we don't > have a good technical solution to the problem. Cutting off people with > defective devices they they don't understand, and have little control > over, is an action that makes sense, but hurts the innocent. "Hey, > Grandma, did you know your TV set is hurting the Internet?" > > It's the people who foist bad stuff on the people who need to take the > responsibility. Indeed, with enough moxie, we could avoid the net > saturation problem in the first place. > > My proposal, as I sent it to my US House Representative: > [much snipping] > Why not nip the IoT problem in the bud? Why not, indeed? (Full disclosure: I am not and have not for some years been active in management of any networks, and I AM woefully behind the state of the arts.) Having said that, it occurs to me that Mr. Satchell's proposal (and most of the others I have read about here and elsewhere lately) are doomed to the same failure as Chicago's plan for reducing illegal deaths by firearm, and for much the same reason (discussion of which here I will spare you. Back in the day, I was fighting a problem that I summarized (then and now) as trying to stop the use and abuse of the University's (that employed me) 56kb Frame Relay link to the Internet. Then as now I defined "abuse" as the use of our facilities for purposes that no stretch of imagination or definition could be said to be to the University's benefit. Through some experimentation I concluded that there were several clearly identifiable sources of abuse. I disremember the ordering by severity but they included: Outright attacks on the University and others. Myriad "scans" for a variety of reasons. The first of these two I remember as being the worst (in terms of item-count AND in terms of packet-size. I also recall it being the easiest to fix, if anybody want to fix it. (The dominant reasons given where that it would cost money without a revenue stream, and it would reduce traffic that WAS in the revenue stream. The fix I proposed: Require (by law) that every service provider and every origination customer of a service provider would under penalty of law, block the transmission of a packet whose source address could not be reached via the link upon which it was found. The Myriad scans problem was a little harder (for among other reasons--the argument that they were good for us, even though they accounted for something like 60% of the traffic on that link). The solution I tried but ran out of dollars on was to detect somebody scanning and route them to the Loopback interface of the boundary router. -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From bill at herrin.us Mon Oct 3 23:03:22 2016 From: bill at herrin.us (William Herrin) Date: Mon, 3 Oct 2016 19:03:22 -0400 Subject: ARIN legacy block transfer process In-Reply-To: <5071ADF5-6238-432F-B7F4-5EB113954F6C@arin.net> References: <8c41e580-9d97-c919-41bb-c88ff2684a84@bryanfields.net> <0657B5CC-1E52-4B58-B75E-653979B5FCCD@corp.arin.net> <5071ADF5-6238-432F-B7F4-5EB113954F6C@arin.net> Message-ID: On Mon, Oct 3, 2016 at 5:09 PM, John Curran wrote: > On 3 Oct 2016, at 4:06 PM, William Herrin wrote: >> ARIN asserts they've taken no action because community developed >> policy instructs them not to. That is a half-truth. The whole truth >> is that ARIN has bent over backwards to avoid testing in court whether >> they have the lawful authority to enforce any policies against legacy >> address holders, even to the extent of bending or breaking their own >> policies when settling a court case. > > Bill - You?re not quite correct with these statements, as there > have been many court orders that require parties to enter > into an RSA and comply with the IP registry policy Hi John, I was a little sloppy with my language. When two parties in a court case reach an agreement and ask the judge to accept that agreement, the result is a court order. It is not, however, a precedent-setting judgement. > Half-correct, in that these bankruptcy cases do result in judicial orders; > however, the fact that the parties consent to the terms in order to proceed > with the transfer is correct. > (Presumably they would do otherwise if they > had a valid basis for argument, but that hasn?t happened.) That reasoning escapes me. How does years in court fighting ARIN advantage a buyer in a bankruptcy sale? Make the best deal you can without a fight and quickly close. Bankruptcies are all about pragmatism. > We have routinely argue that IP address blocks are not > common law property and prevail - we have never been > ordered to make registry updates contrary > to the community policy. You prevail through consent. In one sense, that speaks well of you. But no court has ruled either for you or against you. There is literally no precedent on the matter of addresses as property as a result of the cases ARIN has been involved in. And in my opinion, ARIN has cut a couple shifty deals to avoid having a judge rule in a way that would set a precedent. That probably comes across harsher than I intend. I think it's good that ARIN seeks agreement over conflict. Wise even - addresses handed out after ARIN's inception are governed under much clearer contract law. Enough delay settling the law with respect to legacy IPv4 addresses will let IPv6 render the issue moot. Anyway, my point is that ARIN's position on the matter aside, no one holding legacy IP addresses should believe that addresses as non-property is settled law. And with that I'll get off my soap box and bid you a good evening. Regards, Bill Herrin -- William Herrin ................ herrin at dirtside.com bill at herrin.us Owner, Dirtside Systems ......... Web: From jra at baylink.com Tue Oct 4 00:39:18 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Tue, 4 Oct 2016 00:39:18 +0000 (UTC) Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> Message-ID: <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Lyndon Nerenberg" >> But that does not remove those devices from the network. > > That ship has sailed. You're not familiar with CPSC mandatory recalls, are you? Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From beckman at angryox.com Tue Oct 4 01:04:06 2016 From: beckman at angryox.com (Peter Beckman) Date: Mon, 3 Oct 2016 21:04:06 -0400 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <20161003203958.10869.qmail@ary.lan> Message-ID: On Mon, 3 Oct 2016, Lyndon Nerenberg wrote: > The only cure to this will be changing the law so that the directors of the > companies that ship massively insecure devices like these are personally > liable for all the financial loss attributed to their products. Bankrupt a > few companies' board of directors and you'll start seeing things change in a > hurry. Manufacturers are global, and their distribution is global. Local, technical laws are difficult at best to get enacted, much less consistently and by 190+ countries. And even when technically-minded laws are implemented (see US Federal and State Do Not Call Lists) they are problematic and difficult to enforce when abuse may be coming from outside the US. And the tech usually is far ahead of the legislation. The common device through which all of these smart devices will pass is the router. Router manufacturers often build and sell larger big iron routers to ISPs, or ISPs are buying end-user routers from manufacturers and reselling to their customers. ISPs are motivated financially to avoid unwanted and "bad" traffic on their networks. The global ISP community is in the best position here to pressure their vendors to implement a standard on end-user routers which protects their networks from rogue and unsecured devices. The IoT manufacturers will need to follow standards that the router manufacturers implement to limit the negative impact of IoT devices if they want their devices on the network/Internet. When the standards are available to help protect the ISP networks at the end of the last mile from unwanted and fraudulently created traffic, and the ISPs pressure/demand the router manufacturers to implement the protections, IoT and other device manufacturers will fall in line. Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- From lyndon at orthanc.ca Tue Oct 4 01:15:09 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 18:15:09 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> Message-ID: <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> > On Oct 3, 2016, at 5:39 PM, Jay R. Ashworth wrote: > > You're not familiar with CPSC mandatory recalls, are you? I'm not sure how you could make the case that a compromised DVR, e.g., directly creates a risk of physical injury to a person. Without that, I don't see how the CPSA would apply. But even if a mandatory recall was made under some law, how many of those devices do you think would be returned/exchanged, realistically. And what percentage of those devices would fall under the jurisdiction of any one country's laws? The only way to stop this sort of thing once and for all is to make it punitively costly to the humans at the helm of the corporations selling this crap in the first place. Under corporate law, this almost always means the directors. Only when they start losing their homes/yachts/Jaguars, or start spending some quality time in jail, will this problem go away. Of course, this does require governments to grow some balls :-P --lyndon From Valdis.Kletnieks at vt.edu Tue Oct 4 01:31:35 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Mon, 03 Oct 2016 21:31:35 -0400 Subject: Legislative proposal sent to my Congressman In-Reply-To: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> Message-ID: <67604.1475544695@turing-police.cc.vt.edu> On Mon, 03 Oct 2016 11:58:10 -0700, Stephen Satchell said: > > THEREFORE the Consumer Product Safety Commission shall require that > > the manufacturer provide a security update to the device within 30 day > > of first notice; or failing that, to issue a complete recall of the > > defective devices. What percent of recalled devices are actually replaced/repaired? It's not too hard to (in principle) track down all owners of 2014 Ford Escapes. But how do you track down all purchasers of a light bulb? That's been sold in multiple continents with differing legal environments? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From mpetach at netflight.com Tue Oct 4 01:33:38 2016 From: mpetach at netflight.com (Matthew Petach) Date: Mon, 3 Oct 2016 18:33:38 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> Message-ID: On Mon, Oct 3, 2016 at 6:15 PM, Lyndon Nerenberg wrote: > [...] > > The only way to stop this sort of thing once and for all is to make it punitively costly to the humans at the helm of the corporations selling this crap in the first place. Under corporate law, this almost always means the directors. Only when they start losing their homes/yachts/Jaguars, or start spending some quality time in jail, will this problem go away. > > Of course, this does require governments to grow some balls :-P > > --lyndon Please, no. This will put a sword through the heart of open source. If you hold the executives of the hardware manufacturer responsible for the software running on their devices, then the next generation of hardware from every manufacturer is going to be hardware locked to ONLY run their software. No OpenWRT, no Tomato, no third party software that could be compromised and leave them holding the liability bag. If you want a world in which only a handful of companies make the hardware and software, with commensurately higher prices, and no freedom to select what software you'd like to load on it, I suspect this is a good path towards it. I think there's got to be solutions that don't drive us into a closed-software world. Before we start asking the government and the lawyers to solve this in ways we'll come to hate down the road, let's give it a few more tries ourselves, shall we? Thanks! Matt From lyndon at orthanc.ca Tue Oct 4 01:52:49 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 18:52:49 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> Message-ID: <46BEE32B-E962-4A5E-845A-B8682210CBBF@orthanc.ca> > On Oct 3, 2016, at 6:33 PM, Matthew Petach wrote: > > If you hold the executives of the hardware manufacturer > responsible for the software running on their devices, > then the next generation of hardware from every > manufacturer is going to be hardware locked to > ONLY run their software. No OpenWRT, no Tomato, > no third party software that could be compromised > and leave them holding the liability bag. It's the closed software that is fscking everything up right now. A little sunshine on the code base will go a long way towards those people not losing their Ferrari's after all. From lyndon at orthanc.ca Tue Oct 4 02:06:09 2016 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Mon, 3 Oct 2016 19:06:09 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: <46BEE32B-E962-4A5E-845A-B8682210CBBF@orthanc.ca> References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> <46BEE32B-E962-4A5E-845A-B8682210CBBF@orthanc.ca> Message-ID: <398E22D0-8D31-40A4-8206-9327E508FFF5@orthanc.ca> > On Oct 3, 2016, at 6:52 PM, Lyndon Nerenberg wrote: > > It's the closed software that is fscking everything up right now. A little sunshine on the code base will go a long way towards those people not losing their Ferrari's after all. Or coming from a more legalistic view, if they lock things down that hard, they cannot possibly blame anyone else for having "rooted" the gear, therefore no passing the buck. They would have to admit that it was their - and only their - code that was responsible for inflicting the damages. I've been in the tech biz for 30+ years, and have worked for a wide range of organizations over that time. The only common denominator across them all (small, large, and everything between - commercial and not) is that rapid response high level organizational change ONLY happen when the executives see the possibility of an imminent, significant, personal loss. That might be monetary loss, or loss of reputation. But it must be personally hurtful. When the reaper appears on the horizon, it's amazing how quickly they see the path to redemption. The sooner we all admit this is not a *technical* problem, the sooner we will eradicate it. --lyndon From Valdis.Kletnieks at vt.edu Tue Oct 4 02:58:18 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Mon, 03 Oct 2016 22:58:18 -0400 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: <3655d787-4642-976b-6c4d-843bc233a222@satchell.net> <87int9us1t.fsf@mid.deneb.enyo.de> <1546973942.8516.1475541558227.JavaMail.zimbra@baylink.com> <66C09BAE-09AB-497E-BE22-60D215A03FA1@orthanc.ca> Message-ID: <74778.1475549898@turing-police.cc.vt.edu> On Mon, 03 Oct 2016 18:33:38 -0700, Matthew Petach said: > If you hold the executives of the hardware manufacturer > responsible for the software running on their devices, > then the next generation of hardware from every > manufacturer is going to be hardware locked to > ONLY run their software. No OpenWRT, no Tomato, > no third party software that could be compromised > and leave them holding the liability bag. Turn it on its ear. Liability only attaches if the product is closed-source. Sure, that leaves us with lots of open-source light bulbs that are basically abandonware 5 years later, but at least at that point it's more possible to fix any remaining issues... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From bryan at shout.net Mon Oct 3 19:09:00 2016 From: bryan at shout.net (Bryan Holloway) Date: Mon, 3 Oct 2016 14:09:00 -0500 Subject: 10G tester recommendations? Message-ID: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> We're in the market for a hand-held 10G ethernet tester, and I was curious if the NANOG community had any recommendations or experiences they would be willing to share, negative or positive. Currently we're looking at the EXFO MAX-800 and NetScout's AT 10G, but we're open to other suggestions. Thank you! - bryan From Ida.Leung at rci.rogers.com Mon Oct 3 19:25:36 2016 From: Ida.Leung at rci.rogers.com (Ida Leung) Date: Mon, 3 Oct 2016 19:25:36 +0000 Subject: Kudos to Rogers Wireless on IPv6 deployment Message-ID: Yes, we have started the IPv6 enablement of our wireless network. West was completed dual stack on Sept 29, Ontario will come next then east region. Rogers Internet has completed all the IPv6 enablement. Fido Internet is coming! Please email me directly for your IPv6 experience with Rogers. ...Ida (IPv6Mom) ________________________________ This communication is confidential. We only send and receive email on the basis of the terms set out at www.rogers.com/web/content/emailnotice Ce message est confidentiel. Notre transmission et r?ception de courriels se fait strictement suivant les modalit?s ?nonc?es dans l'avis publi? ? www.rogers.com/aviscourriel ________________________________ From kenbreeman at gmail.com Mon Oct 3 21:43:10 2016 From: kenbreeman at gmail.com (Ken Breeman) Date: Mon, 3 Oct 2016 17:43:10 -0400 Subject: ATT contact? Message-ID: Could someone from AT&T please contact me off list? Your residential Internet DNS servers are redirecting all traffic destined for Akamai hosted sites to your homepage... Thanks, Ken From dustin at rseng.net Tue Oct 4 13:39:51 2016 From: dustin at rseng.net (Dustin Jurman) Date: Tue, 4 Oct 2016 13:39:51 +0000 Subject: 10G tester recommendations? In-Reply-To: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: We utilize the EXFO product line and it's a quality product. Dustin Dustin Jurman CEO Rapid Systems Corporation 1211 N. West Shore Blvd. Suite 711 Tampa, FL 33607 Ph: 813-232-4887 Cell: 813-892-7006 http://www.rapidsys.com ?Building Better Infrastructure?? -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Bryan Holloway Sent: Monday, October 3, 2016 3:09 PM To: nanog at nanog.org Subject: 10G tester recommendations? We're in the market for a hand-held 10G ethernet tester, and I was curious if the NANOG community had any recommendations or experiences they would be willing to share, negative or positive. Currently we're looking at the EXFO MAX-800 and NetScout's AT 10G, but we're open to other suggestions. Thank you! - bryan From bmengel at gmail.com Tue Oct 4 14:11:57 2016 From: bmengel at gmail.com (Brian Mengel) Date: Tue, 4 Oct 2016 10:11:57 -0400 Subject: 10G tester recommendations? In-Reply-To: References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: We've used the Veex TX300 (now discontinued and replaced with the TX230S I believe) and been satisfied. Our primary use is RFC 2544 testing, so I don't know much about it's capabilities beyond that. On Tue, Oct 4, 2016 at 9:39 AM, Dustin Jurman wrote: > We utilize the EXFO product line and it's a quality product. > > Dustin > > > Dustin Jurman > CEO > Rapid Systems Corporation > 1211 N. West Shore Blvd. Suite 711 > Tampa, FL 33607 > Ph: 813-232-4887 > Cell: 813-892-7006 > http://www.rapidsys.com > ?Building Better Infrastructure? > > > > > -----Original Message----- > From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Bryan Holloway > Sent: Monday, October 3, 2016 3:09 PM > To: nanog at nanog.org > Subject: 10G tester recommendations? > > We're in the market for a hand-held 10G ethernet tester, and I was curious > if the NANOG community had any recommendations or experiences they would be > willing to share, negative or positive. > > Currently we're looking at the EXFO MAX-800 and NetScout's AT 10G, but > we're open to other suggestions. > > Thank you! > > - bryan > > From manager at monmouth.com Tue Oct 4 15:01:57 2016 From: manager at monmouth.com (Mark Stevens) Date: Tue, 4 Oct 2016 11:01:57 -0400 Subject: Level 3 voice outage Message-ID: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Is anyone noticing issue with Level 3 voice? I can't even call their 800 number using one of my other carriers. Mark From bhatton at htva.net Tue Oct 4 15:04:16 2016 From: bhatton at htva.net (Benjamin Hatton) Date: Tue, 4 Oct 2016 11:04:16 -0400 Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: We just got a notification from one of our upstream Voice providers that they are seeing problems both inbound and outbound that seems to be Level 3 related *Ben Hatton* Network Engineer Haefele TV Inc. d:(607)589-8000 bhatton at htva.net www.htva.net On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other carriers. > > Mark > From kkadow at gmail.com Tue Oct 4 15:06:20 2016 From: kkadow at gmail.com (Kevin Kadow) Date: Tue, 4 Oct 2016 11:06:20 -0400 Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: Level 3 has issues nationwide today. We have voice outages at our offices in multiple cities, Chicago and out west. On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other carriers. > > Mark > From mlfreita at mtu.edu Tue Oct 4 15:19:40 2016 From: mlfreita at mtu.edu (Matt Freitag) Date: Tue, 4 Oct 2016 11:19:40 -0400 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: There's a long thread over at the outages list. Sign-up for the outages list is at https://puck.nether.net/mailman/listinfo/outages and the archive for the whole thread starts at https://puck.nether.net/pipermail/outages/2016-October/009609.html. Matt Freitag Network Engineer I Information Technology Michigan Technological University (906) 487-3696 <%28906%29%20487-3696> https://www.mtu.edu/ https://www.it.mtu.edu/ On Tue, Oct 4, 2016 at 11:06 AM, Kevin Kadow wrote: > Level 3 has issues nationwide today. We have voice outages at our offices > in multiple cities, Chicago and out west. > > On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens > wrote: > > > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > > number using one of my other carriers. > > > > Mark > > > From timrutherford at c4.net Tue Oct 4 15:20:01 2016 From: timrutherford at c4.net (timrutherford at c4.net) Date: Tue, 4 Oct 2016 11:20:01 -0400 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: <017a01d21e52$c6b88a20$54299e60$@c4.net> Our upstream Voice provider is also having inbound issues with numbers that were ported into their system, but the main trunk number is working. -Tim -----Original Message----- From: NANOG [mailto:nanog-bounces+timrutherford=c4.net at nanog.org] On Behalf Of Benjamin Hatton Sent: Tuesday, October 4, 2016 11:04 AM To: nanog at nanog.org Subject: Re: Level 3 voice outage We just got a notification from one of our upstream Voice providers that they are seeing problems both inbound and outbound that seems to be Level 3 related *Ben Hatton* Network Engineer Haefele TV Inc. d:(607)589-8000 bhatton at htva.net www.htva.net On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their > 800 number using one of my other carriers. > > Mark > From adairw at amarillowireless.net Tue Oct 4 15:21:03 2016 From: adairw at amarillowireless.net (Adair Winter) Date: Tue, 4 Oct 2016 10:21:03 -0500 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: Our provider is down as well. Saying Level 3 issues. Anyone know what's going on or and ETR? On Tue, Oct 4, 2016 at 10:06 AM, Kevin Kadow wrote: > Level 3 has issues nationwide today. We have voice outages at our offices > in multiple cities, Chicago and out west. > > On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens > wrote: > > > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > > number using one of my other carriers. > > > > Mark > > > -- Adair Winter VP, Network Operations / Co-Owner Amarillo Wireless | 806.316.5071 C: 806.231.7180 http://www.amarillowireless.net From sean76 at gmail.com Tue Oct 4 15:28:39 2016 From: sean76 at gmail.com (Sean) Date: Tue, 4 Oct 2016 10:28:39 -0500 Subject: Level 3 voice outage In-Reply-To: <017a01d21e52$c6b88a20$54299e60$@c4.net> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <017a01d21e52$c6b88a20$54299e60$@c4.net> Message-ID: Twitter is blowing up ( https://twitter.com/hashtag/outage ) and their website (www.level3.com) is down for me so yeah... I'd say they're having a frowny face day. On Tue, Oct 4, 2016 at 10:20 AM, wrote: > Our upstream Voice provider is also having inbound issues with numbers > that were ported into their system, but the main trunk number is working. > > -Tim > > -----Original Message----- > From: NANOG [mailto:nanog-bounces+timrutherford=c4.net at nanog.org] On > Behalf Of Benjamin Hatton > Sent: Tuesday, October 4, 2016 11:04 AM > To: nanog at nanog.org > Subject: Re: Level 3 voice outage > > We just got a notification from one of our upstream Voice providers that > they are seeing problems both inbound and outbound that seems to be Level 3 > related > > *Ben Hatton* > > Network Engineer > > Haefele TV Inc. > > d:(607)589-8000 > > bhatton at htva.net > > www.htva.net > > On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens > wrote: > > > Is anyone noticing issue with Level 3 voice? I can't even call their > > 800 number using one of my other carriers. > > > > Mark > > > > > From timrutherford at c4.net Tue Oct 4 15:57:20 2016 From: timrutherford at c4.net (timrutherford at c4.net) Date: Tue, 4 Oct 2016 11:57:20 -0400 Subject: Level 3 voice outage In-Reply-To: <017a01d21e52$c6b88a20$54299e60$@c4.net> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <017a01d21e52$c6b88a20$54299e60$@c4.net> Message-ID: <01f201d21e57$fd496780$f7dc3680$@c4.net> We are back in action here in MA. -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of timrutherford at c4.net Sent: Tuesday, October 4, 2016 11:20 AM To: nanog at nanog.org Subject: RE: Level 3 voice outage Our upstream Voice provider is also having inbound issues with numbers that were ported into their system, but the main trunk number is working. -Tim -----Original Message----- From: NANOG [mailto:nanog-bounces+timrutherford=c4.net at nanog.org] On Behalf Of Benjamin Hatton Sent: Tuesday, October 4, 2016 11:04 AM To: nanog at nanog.org Subject: Re: Level 3 voice outage We just got a notification from one of our upstream Voice providers that they are seeing problems both inbound and outbound that seems to be Level 3 related *Ben Hatton* Network Engineer Haefele TV Inc. d:(607)589-8000 bhatton at htva.net www.htva.net On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their > 800 number using one of my other carriers. > > Mark > From amitchell at isipp.com Tue Oct 4 16:07:01 2016 From: amitchell at isipp.com (Anne Mitchell) Date: Tue, 4 Oct 2016 10:07:01 -0600 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: Message-ID: (Interesting and inarguably well-intentioned, and possibly even sound, idea snipped, but noted.) There are a handful of reasons that this will never happen (well, I'm 98% certain it will never happen, nothing is every 100% sure when it comes to the law, and legislation)... among them the manufacturer's lobby is much more well-girded than is the 'home internet security' lobby; the cyber-security concerns of the Federal government are focussed on other things (whether they should be or not, they are); and for the most part legislators are still fairly unsavvy about tech in general, and these things make their eyes glaze over. That said, there are already tort (negligence, etc.) laws and precedents under which such manufacturers can be sued, along with things like breach of contract between the manufacturer and consumer, and breach of implied warranty of fitness for a particular purpose and breach of implied warranty of merchantability. A couple of winning lawsuits against manufacturers under these laws and theories - which judges *already understand* - is, I think, not only a more likely, but a much faster, route to industry reform. All that said, much of this faces the same issues that spam lawsuits faced - the people who care the most about it are not the ones who can afford to finance such lawsuits. Anne Anne P. Mitchell, Attorney at Law Legislative Consultant CEO/President, Institute for Social Internet Public Policy Member, Cal. Bar Cyberspace Law Committee Member, Colorado Cyber Committee Member, Asilomar Microcomputer Workshop Committee Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law) Ret. Professor of Law, Lincoln Law School of San Jose Ret. Chair, Asilomar Microcomputer Workshop From rjacobs at pslightwave.com Tue Oct 4 17:30:53 2016 From: rjacobs at pslightwave.com (Robert Jacobs) Date: Tue, 4 Oct 2016 17:30:53 +0000 Subject: 10G tester recommendations? In-Reply-To: References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: JDSU 5800 series .... very user friendly... nice reports... VNC remote connection so you can trigger test with out rolling a tech.. Robert Jacobs | Network Director/Architect Direct:? 832-615-7742 Main:?? 832-615-8000 Fax:??? 713-510-1650 5959 Corporate Dr. Suite 3300; Houston, TX 77036 A Certified Woman-Owned Business 24x7x365 Customer? Support: 832-615-8000 | support at pslightwave.com This electronic message contains information from Phonoscope Lightwave which may be privileged and confidential. The information is intended to be for the use of individual(s) or entity named above. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify me by telephone or e-mail immediately. -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Brian Mengel Sent: Tuesday, October 4, 2016 9:12 AM Cc: nanog at nanog.org Subject: Re: 10G tester recommendations? We've used the Veex TX300 (now discontinued and replaced with the TX230S I believe) and been satisfied. Our primary use is RFC 2544 testing, so I don't know much about it's capabilities beyond that. On Tue, Oct 4, 2016 at 9:39 AM, Dustin Jurman wrote: > We utilize the EXFO product line and it's a quality product. > > Dustin > > > Dustin Jurman > CEO > Rapid Systems Corporation > 1211 N. West Shore Blvd. Suite 711 > Tampa, FL 33607 > Ph: 813-232-4887 > Cell: 813-892-7006 > http://www.rapidsys.com > ?Building Better Infrastructure? > > > > > -----Original Message----- > From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Bryan > Holloway > Sent: Monday, October 3, 2016 3:09 PM > To: nanog at nanog.org > Subject: 10G tester recommendations? > > We're in the market for a hand-held 10G ethernet tester, and I was > curious if the NANOG community had any recommendations or experiences > they would be willing to share, negative or positive. > > Currently we're looking at the EXFO MAX-800 and NetScout's AT 10G, but > we're open to other suggestions. > > Thank you! > > - bryan > > From jlarsen at richweb.com Tue Oct 4 15:06:05 2016 From: jlarsen at richweb.com (C. Jon Larsen) Date: Tue, 4 Oct 2016 11:06:05 -0400 (EDT) Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: On Tue, 4 Oct 2016, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other carriers. Yes, major outage: http://downdetector.com/status/level3 Customers all over the mid-atlantic region are down. From mhorton at bluip.com Tue Oct 4 15:07:47 2016 From: mhorton at bluip.com (Mark Horton) Date: Tue, 4 Oct 2016 15:07:47 +0000 Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: <1848399D-20CB-4FDB-842C-4CB385291FB0@bluip.com> Major Level 3 outage; they are currently not taking support calls. On 10/4/16, 8:01 AM, "NANOG on behalf of Mark Stevens" wrote: >Is anyone noticing issue with Level 3 voice? I can't even call their 800 >number using one of my other carriers. > >Mark From sheupel at twlakes.coop Tue Oct 4 15:08:01 2016 From: sheupel at twlakes.coop (Shane Heupel) Date: Tue, 4 Oct 2016 15:08:01 +0000 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com>, Message-ID: <8191A478-4EDA-4312-AB46-FAA796C9747A@twlakes.coop> We can't make any LD calls through Level3 either. Thank you, Shane Heupel > On Oct 4, 2016, at 10:06 AM, Benjamin Hatton wrote: > > We just got a notification from one of our upstream Voice providers that > they are seeing problems both inbound and outbound that seems to be Level 3 > related > > *Ben Hatton* > > Network Engineer > > Haefele TV Inc. > > d:(607)589-8000 > > bhatton at htva.net > > www.htva.net > >> On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: >> >> Is anyone noticing issue with Level 3 voice? I can't even call their 800 >> number using one of my other carriers. >> >> Mark >> From Kenw at parkbankonline.com Tue Oct 4 15:12:39 2016 From: Kenw at parkbankonline.com (Ken Warnimont) Date: Tue, 4 Oct 2016 15:12:39 +0000 Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: There's all sorts of people online reporting problems with Level3 voice services - we're down for outbound calling, inbound is spotty. Their portal is unreachable, can't get to any of their #'s...seems like the world is on fire over there. We were a previous TW Telecom customer before Level3 bought them out, the service quality has definitely degraded since then. -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Mark Stevens Sent: Tuesday, October 04, 2016 10:02 AM To: nanog at nanog.org Subject: Level 3 voice outage Is anyone noticing issue with Level 3 voice? I can't even call their 800 number using one of my other carriers. Mark *********************************************************************************** This email message, including any attachments, is intended solely for the designated person or entity to which the message is addressed, and may contain confidential information. It is strictly prohibited to review, distribute, copy, or print this email, including attachments, by any person or entity other than the designated addressee. If you have erroneously received this message, please contact the sender immediately and remove the data from any computer on which the message resides. From ascher at slhs.org Tue Oct 4 15:11:03 2016 From: ascher at slhs.org (Asche, Ron) Date: Tue, 4 Oct 2016 15:11:03 +0000 Subject: [BULK] Re: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: <5378b0c3fce548adb2f2c5b029c0a010@EXMBX2.sl1.stlukes-int.org> Boise Idaho is affected....... -----Original Message----- From: NANOG [mailto:nanog-bounces+ascher=slhs.org at nanog.org] On Behalf Of Benjamin Hatton Sent: Tuesday, October 04, 2016 9:04 AM To: nanog at nanog.org Subject: [BULK] Re: Level 3 voice outage Importance: High WARNING: This email originated outside of St. Luke?s email system. DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. We just got a notification from one of our upstream Voice providers that they are seeing problems both inbound and outbound that seems to be Level 3 related *Ben Hatton* Network Engineer Haefele TV Inc. d:(607)589-8000 bhatton at htva.net www.htva.net On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their > 800 number using one of my other carriers. > > Mark > "This message is intended for the use of the person or entity to which it is addressed and may contain information that is confidential or privileged, the disclosure of which is governed by applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this information is strictly prohibited. If you have received this message by error, please notify us immediately and destroy the related message." From ivokatovski at gmail.com Tue Oct 4 15:32:06 2016 From: ivokatovski at gmail.com (Ivaylo Katovski) Date: Tue, 4 Oct 2016 11:32:06 -0400 Subject: Level 3 voice outage In-Reply-To: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: When will L3 notify their customers for the outage?!? According to l3 twitter account their are aware of the voice impact and working on it On Oct 4, 2016 11:03 AM, "Mark Stevens" wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other carriers. > > Mark > From jhellenthal at dataix.net Tue Oct 4 18:06:40 2016 From: jhellenthal at dataix.net (Jason Hellenthal) Date: Tue, 4 Oct 2016 13:06:40 -0500 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> Message-ID: <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> Patience Obi Wan ! They are investigating the root cause and like most root causes they don?t just hold out a flag and say here I am !!! > On Oct 4, 2016, at 10:32, Ivaylo Katovski wrote: > > When will L3 notify their customers for the outage?!? According to l3 > twitter account their are aware of the voice impact and working on it > > On Oct 4, 2016 11:03 AM, "Mark Stevens" wrote: > >> Is anyone noticing issue with Level 3 voice? I can't even call their 800 >> number using one of my other carriers. >> >> Mark >> -- Jason Hellenthal JJH48-ARIN From saku at ytti.fi Tue Oct 4 18:12:31 2016 From: saku at ytti.fi (Saku Ytti) Date: Tue, 4 Oct 2016 21:12:31 +0300 Subject: 10G tester recommendations? In-Reply-To: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: On 3 October 2016 at 22:09, Bryan Holloway wrote: > We're in the market for a hand-held 10G ethernet tester, and I was curious > if the NANOG community had any recommendations or experiences they would be > willing to share, negative or positive. What are you looking to test? Are you interested in having the kit emulate many devices? Do you want to be able to test QoS realistically? Do you care about one-way latency accurately? Or is this more about proving the physical media is fine? -- ++ytti From mel at beckman.org Tue Oct 4 18:14:54 2016 From: mel at beckman.org (Mel Beckman) Date: Tue, 4 Oct 2016 18:14:54 +0000 Subject: Level 3 voice outage In-Reply-To: <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> Message-ID: We are seeing high latencies and loss of transit through Level3 in Southern California. Level 3 has opened Global Ticket 115724: Network devices in multiple markets in North America are rebooting, impacting IP services.? This could be DoS attack. > On Oct 4, 2016, at 11:06 AM, Jason Hellenthal wrote: > > Patience Obi Wan ! They are investigating the root cause and like most root causes they don?t just hold out a flag and say here I am !!! > >> On Oct 4, 2016, at 10:32, Ivaylo Katovski wrote: >> >> When will L3 notify their customers for the outage?!? According to l3 >> twitter account their are aware of the voice impact and working on it >> >> On Oct 4, 2016 11:03 AM, "Mark Stevens" wrote: >> >>> Is anyone noticing issue with Level 3 voice? I can't even call their 800 >>> number using one of my other carriers. >>> >>> Mark >>> > > > -- > Jason Hellenthal > JJH48-ARIN > > > > From Valdis.Kletnieks at vt.edu Tue Oct 4 18:42:57 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Tue, 04 Oct 2016 14:42:57 -0400 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> Message-ID: <17035.1475606577@turing-police.cc.vt.edu> On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: > This could be DoS attack. Or a missing comma in a code update. Or a fumble-fingered NOC monkey. Or.... You have any reason to suspect a DoS attack rather than all the other possibilities? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From mel at beckman.org Tue Oct 4 18:52:42 2016 From: mel at beckman.org (Mel Beckman) Date: Tue, 4 Oct 2016 18:52:42 +0000 Subject: Level 3 voice outage In-Reply-To: <17035.1475606577@turing-police.cc.vt.edu> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> Message-ID: <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> Sure. The recent release of the IoT DDoS attack code in the wild. -mel > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: > > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: > >> This could be DoS attack. > > Or a missing comma in a code update. > > Or a fumble-fingered NOC monkey. > > Or.... > > You have any reason to suspect a DoS attack rather than all the other > possibilities? From admin at marcoteixeira.com Tue Oct 4 19:10:44 2016 From: admin at marcoteixeira.com (Marco Teixeira) Date: Tue, 4 Oct 2016 20:10:44 +0100 Subject: Level 3 voice outage In-Reply-To: <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> Message-ID: Multiple reboots across several markets... Does not seem something that full pipes would trigger. Had it been an approved chance it would have been rolled back i guess... On the other hand, a zero day could apply... Em 04/10/2016 19:54, "Mel Beckman" escreveu: > Sure. The recent release of the IoT DDoS attack code in the wild. > > -mel > > > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: > > > > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: > > > >> This could be DoS attack. > > > > Or a missing comma in a code update. > > > > Or a fumble-fingered NOC monkey. > > > > Or.... > > > > You have any reason to suspect a DoS attack rather than all the other > > possibilities? > > From mel at beckman.org Tue Oct 4 19:23:08 2016 From: mel at beckman.org (Mel Beckman) Date: Tue, 4 Oct 2016 19:23:08 +0000 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> Message-ID: <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> 765 Gbps per second directed at a router?s interface IP might give the router pause, so to speak :) -mel On Oct 4, 2016, at 12:10 PM, Marco Teixeira > wrote: Multiple reboots across several markets... Does not seem something that full pipes would trigger. Had it been an approved chance it would have been rolled back i guess... On the other hand, a zero day could apply... Em 04/10/2016 19:54, "Mel Beckman" > escreveu: Sure. The recent release of the IoT DDoS attack code in the wild. -mel > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: > > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: > >> This could be DoS attack. > > Or a missing comma in a code update. > > Or a fumble-fingered NOC monkey. > > Or.... > > You have any reason to suspect a DoS attack rather than all the other > possibilities? From Valdis.Kletnieks at vt.edu Tue Oct 4 19:33:26 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Tue, 04 Oct 2016 15:33:26 -0400 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> Message-ID: <21828.1475609606@turing-police.cc.vt.edu> On Tue, 04 Oct 2016 20:10:44 +0100, Marco Teixeira said: > Had it been an approved chance it would have been > rolled back i guess... See the 1990 ATT long-distance collapse for a worked example. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From admin at marcoteixeira.com Tue Oct 4 19:47:05 2016 From: admin at marcoteixeira.com (Marco Teixeira) Date: Tue, 4 Oct 2016 20:47:05 +0100 Subject: Level 3 voice outage In-Reply-To: <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> Message-ID: I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman wrote: > 765 Gbps per second directed at a router?s interface IP might give the > router pause, so to speak :) > > -mel > > On Oct 4, 2016, at 12:10 PM, Marco Teixeira > wrote: > > Multiple reboots across several markets... Does not seem something that > full pipes would trigger. Had it been an approved chance it would have been > rolled back i guess... On the other hand, a zero day could apply... > > Em 04/10/2016 19:54, "Mel Beckman" escreveu: > >> Sure. The recent release of the IoT DDoS attack code in the wild. >> >> -mel >> >> > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: >> > >> > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: >> > >> >> This could be DoS attack. >> > >> > Or a missing comma in a code update. >> > >> > Or a fumble-fingered NOC monkey. >> > >> > Or.... >> > >> > You have any reason to suspect a DoS attack rather than all the other >> > possibilities? >> >> > From admin at marcoteixeira.com Tue Oct 4 19:52:52 2016 From: admin at marcoteixeira.com (Marco Teixeira) Date: Tue, 4 Oct 2016 20:52:52 +0100 Subject: Level 3 voice outage In-Reply-To: <21828.1475609606@turing-police.cc.vt.edu> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <21828.1475609606@turing-police.cc.vt.edu> Message-ID: ... but back then they didn't have ITIL and the likes (at least not that i knew... but then again, i was just starting with networking back then), where a ChM process must produce risk, impact, etc, reports, and at least one peer review meeting for approval :) = Marco On Tue, Oct 4, 2016 at 8:33 PM, wrote: > On Tue, 04 Oct 2016 20:10:44 +0100, Marco Teixeira said: > > Had it been an approved chance it would have been > > rolled back i guess... > > See the 1990 ATT long-distance collapse for a worked example. > From mel at beckman.org Tue Oct 4 19:59:04 2016 From: mel at beckman.org (Mel Beckman) Date: Tue, 4 Oct 2016 19:59:04 +0000 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> , Message-ID: <6AD20E41-875E-4867-9FB5-B0E4F138EA40@beckman.org> Just so. We are just speculating, but in light of recent GRE attack developments, the ddos possibility is worth considering. -mel beckman On Oct 4, 2016, at 12:54 PM, Shawn Ritchie > wrote: Well, Level3 has by no means said that this was the result of a DDoS, that's just speculation on behalf of folks who do not work at Level3 so far. On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman > wrote: > 765 Gbps per second directed at a router's interface IP might give the > router pause, so to speak :) > > -mel > > On Oct 4, 2016, at 12:10 PM, Marco Teixeira > > wrote: > > Multiple reboots across several markets... Does not seem something that > full pipes would trigger. Had it been an approved chance it would have been > rolled back i guess... On the other hand, a zero day could apply... > > Em 04/10/2016 19:54, "Mel Beckman" > escreveu: > >> Sure. The recent release of the IoT DDoS attack code in the wild. >> >> -mel >> >> > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: >> > >> > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: >> > >> >> This could be DoS attack. >> > >> > Or a missing comma in a code update. >> > >> > Or a fumble-fingered NOC monkey. >> > >> > Or.... >> > >> > You have any reason to suspect a DoS attack rather than all the other >> > possibilities? >> >> > -- -- Shawn From admin at marcoteixeira.com Tue Oct 4 20:05:47 2016 From: admin at marcoteixeira.com (Marco Teixeira) Date: Tue, 4 Oct 2016 21:05:47 +0100 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> Message-ID: Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There are so many things that can go wrong on complex networks even with all the controls imposed on changes... On Tue, Oct 4, 2016 at 8:54 PM, Shawn Ritchie wrote: > Well, Level3 has by no means said that this was the result of a DDoS, > that's just speculation on behalf of folks who do not work at Level3 so > far. > > On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: > >> I won't believe a company like Level3 would not deploy backplane >> protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH >> network didn't pause or rebooted routers. And i guess both companies have >> had their share of (D)DoS in the past, so they had the time to get up to >> the challenge. Now... there where times where one malformed IP packet >> would >> cause a memory leak leading to a router reboot... :)? >> >> >> >> >> On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman wrote: >> >> > 765 Gbps per second directed at a router?s interface IP might give the >> > router pause, so to speak :) >> > >> > -mel >> > >> > On Oct 4, 2016, at 12:10 PM, Marco Teixeira >> > wrote: >> > >> > Multiple reboots across several markets... Does not seem something that >> > full pipes would trigger. Had it been an approved chance it would have >> been >> > rolled back i guess... On the other hand, a zero day could apply... >> > >> > Em 04/10/2016 19:54, "Mel Beckman" escreveu: >> > >> >> Sure. The recent release of the IoT DDoS attack code in the wild. >> >> >> >> -mel >> >> >> >> > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: >> >> > >> >> > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: >> >> > >> >> >> This could be DoS attack. >> >> > >> >> > Or a missing comma in a code update. >> >> > >> >> > Or a fumble-fingered NOC monkey. >> >> > >> >> > Or.... >> >> > >> >> > You have any reason to suspect a DoS attack rather than all the other >> >> > possibilities? >> >> >> >> >> > >> > -- > > -- > Shawn > From mel at beckman.org Tue Oct 4 20:09:22 2016 From: mel at beckman.org (Mel Beckman) Date: Tue, 4 Oct 2016 20:09:22 +0000 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> , Message-ID: Possibly somebody YANGed when they should have yinged :) -mel beckman On Oct 4, 2016, at 1:06 PM, Marco Teixeira > wrote: Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There are so many things that can go wrong on complex networks even with all the controls imposed on changes... On Tue, Oct 4, 2016 at 8:54 PM, Shawn Ritchie > wrote: Well, Level3 has by no means said that this was the result of a DDoS, that's just speculation on behalf of folks who do not work at Level3 so far. On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman > wrote: > 765 Gbps per second directed at a router's interface IP might give the > router pause, so to speak :) > > -mel > > On Oct 4, 2016, at 12:10 PM, Marco Teixeira > > wrote: > > Multiple reboots across several markets... Does not seem something that > full pipes would trigger. Had it been an approved chance it would have been > rolled back i guess... On the other hand, a zero day could apply... > > Em 04/10/2016 19:54, "Mel Beckman" > escreveu: > >> Sure. The recent release of the IoT DDoS attack code in the wild. >> >> -mel >> >> > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: >> > >> > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: >> > >> >> This could be DoS attack. >> > >> > Or a missing comma in a code update. >> > >> > Or a fumble-fingered NOC monkey. >> > >> > Or.... >> > >> > You have any reason to suspect a DoS attack rather than all the other >> > possibilities? >> >> > -- -- Shawn From m4rtntns at gmail.com Wed Oct 5 07:45:20 2016 From: m4rtntns at gmail.com (Martin T) Date: Wed, 5 Oct 2016 10:45:20 +0300 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> Message-ID: Florian: > Are the autonomous systems for the /19 and /24 connected directly? Yes they are. > (1) can be better from B's perspective because it prevents certain routing table optimizations (due to the lack of the covering prefix) What kind of routing table optimizations are possible if covering /19 prefix is also present in global routing table? > But (1) can also be worse for B and A's other customers if /24s (and slightly shorter prefixes) in this part of the IPv4 address space are commonly filtered. Based on my experience /24 is allowed in prefix-filters.. Longer IPv4 prefixes are not. Roy, Mel: Could you please elaborate on that option. What kind of advantages does this have compared to option 2? thanks, Martin On Tue, Sep 27, 2016 at 8:52 PM, Michael Hallgren wrote: > Hi Martin, > > What do you want to do? Move from A to B or add A to B? > > Cheers, > mh > > > > Le 27 sept. 2016 17:52, ? 17:52, Mel Beckman a ?crit: >>Precisely. This is how it's done by providers I've worked with. >> >> -mel beckman >> >>> On Sep 27, 2016, at 7:06 AM, Roy wrote: >>> >>> >>> >>> Option 3? >>> >>> ISP A announces the /19 and the /24 while ISP B does just the /24 >>> >>>> On 9/27/2016 4:20 AM, Martin T wrote: >>>> Hi, >>>> >>>> let's assume that there is an ISP "A" operating in Europe region who >>>> has /19 IPv4 allocation from RIPE. From this /19 they have leased >>/24 >>>> to ISP "B" who is multi-homed. This means that ISP "B" would like to >>>> announce this /24 prefix to ISP "A" and also to ISP "C". AFAIK this >>>> gives two possibilities: >>>> >>>> 1) Deaggregate /19 in ISP "A" network and create "inetnum" and >>"route" >>>> objects for all those networks to RIPE database. This means that ISP >>>> "A" announces around dozen IPv4 prefixes to Internet except this /24 >>>> and ISP "B" announces this specific /24 to Internet. >>>> >>>> 2) ISP "A" continues to announce this /19 to Internet and at the >>same >>>> time ISP "B" starts to announce /24 to Internet. As this /24 is >>>> more-specific than /19, then traffic to hosts in this /24 will end >>up >>>> in ISP "B" network. >>>> >>>> >>>> Which approach is better? To me the second one seems to be better >>>> because it keeps the IPv4 routing-table smaller and requires ISP "A" >>>> to make no deaggregation related configuration changes. Only bit >>weird >>>> behavior I can see with the second option is that if ISP "B" stops >>for >>>> some reason announcing this /24 network to Internet, then traffic to >>>> hosts in this /24 gets to ISP "A" network and is blackholed there. >>>> >>>> >>>> thanks, >>>> Martin >>> From ekgermann at semperen.com Wed Oct 5 03:15:49 2016 From: ekgermann at semperen.com (Eric Germann) Date: Tue, 4 Oct 2016 23:15:49 -0400 Subject: Questions re: VPN protocols globally Message-ID: I?ve been charged with building a global VPN as an overlay on top of a certain 3 letter company who also sells lots of stuff. We?re looking at US East US West US Central (eventually) Brazil Singapore Frankfurt Ireland Sydney Maybe Canada Maybe India (outsourcesrs) In the planning stages now and wondering if there are any protocols I need to stay away from ITAR wise with this list of countries. Contemplating Suite B with GCM, etc and AES acceleration. Any land mines? Thanks in advance EKG -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3705 bytes Desc: not available URL: From bryan at shout.net Wed Oct 5 12:54:00 2016 From: bryan at shout.net (Bryan Holloway) Date: Wed, 5 Oct 2016 07:54:00 -0500 Subject: 10G tester recommendations? In-Reply-To: References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: Mainly RFC2544 and physical media. QoS could be handy, but it's not as important as proving that the circuit is meeting our expectations from the carrier(s). On 10/4/16 1:12 PM, Saku Ytti wrote: > On 3 October 2016 at 22:09, Bryan Holloway wrote: >> We're in the market for a hand-held 10G ethernet tester, and I was curious >> if the NANOG community had any recommendations or experiences they would be >> willing to share, negative or positive. > > What are you looking to test? Are you interested in having the kit > emulate many devices? Do you want to be able to test QoS > realistically? Do you care about one-way latency accurately? > Or is this more about proving the physical media is fine? > From saku at ytti.fi Wed Oct 5 12:59:15 2016 From: saku at ytti.fi (Saku Ytti) Date: Wed, 5 Oct 2016 15:59:15 +0300 Subject: 10G tester recommendations? In-Reply-To: References: <2fccffad-0996-fade-6511-6772f7a695c6@shout.net> Message-ID: On 5 October 2016 at 15:54, Bryan Holloway wrote: > Mainly RFC2544 and physical media. QoS could be handy, but it's not as > important as proving that the circuit is meeting our expectations from the > carrier(s). Then then EXFO is fine and much more affordable than Spirent or IXIA. -- ++ytti From beckman at angryox.com Wed Oct 5 15:05:47 2016 From: beckman at angryox.com (Peter Beckman) Date: Wed, 5 Oct 2016 11:05:47 -0400 Subject: Questions re: VPN protocols globally In-Reply-To: References: Message-ID: There is a Mumbai, India three letter company region available as of June 27, 2016 https://aws.amazon.com/blogs/aws/now-open-aws-asia-pacific-mumbai-region/ On Tue, 4 Oct 2016, Eric Germann wrote: > I?ve been charged with building a global VPN as an overlay on top of a certain 3 letter company who also sells lots of stuff. > > We?re looking at > > US East > US West > US Central (eventually) > Brazil > Singapore > Frankfurt > Ireland > Sydney > Maybe Canada > Maybe India (outsourcesrs) > > In the planning stages now and wondering if there are any protocols I need to stay away from ITAR wise with this list of countries. > > Contemplating Suite B with GCM, etc and AES acceleration. > > Any land mines? > > Thanks in advance > > EKG > > --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- From morrowc.lists at gmail.com Wed Oct 5 16:01:54 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Wed, 5 Oct 2016 12:01:54 -0400 Subject: Questions re: VPN protocols globally In-Reply-To: References: Message-ID: On Tue, Oct 4, 2016 at 11:15 PM, Eric Germann wrote: > I?ve been charged with building a global VPN as an overlay on top of a > certain 3 letter company who also sells lots of stuff. > > you say 'vpn' do you mean 'mpls vpn' or 'ipsec vpn over intertubes' ? > We?re looking at > > US East > US West > US Central (eventually) > Brazil > Singapore > Frankfurt > Ireland > Sydney > Maybe Canada > Maybe India (outsourcesrs) > > In the planning stages now and wondering if there are any protocols I need > to stay away from ITAR wise with this list of countries. > > Contemplating Suite B with GCM, etc and AES acceleration. > > most places dont' really care about encryption if your use is 'for corporate use', not providing use by external parties (internet access sorts of things), I believe. From deleskie at gmail.com Wed Oct 5 16:46:05 2016 From: deleskie at gmail.com (jim deleskie) Date: Wed, 5 Oct 2016 13:46:05 -0300 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: Message-ID: Can we please not get the government ( who's gov ) involved. I fully agree that it will not only not help, but will make some things worse. This is why we can't have nice things. On Tuesday, October 4, 2016, Anne Mitchell wrote: > (Interesting and inarguably well-intentioned, and possibly even sound, > idea snipped, but noted.) > > There are a handful of reasons that this will never happen (well, I'm 98% > certain it will never happen, nothing is every 100% sure when it comes to > the law, and legislation)... among them the manufacturer's lobby is much > more well-girded than is the 'home internet security' lobby; the > cyber-security concerns of the Federal government are focussed on other > things (whether they should be or not, they are); and for the most part > legislators are still fairly unsavvy about tech in general, and these > things make their eyes glaze over. > > That said, there are already tort (negligence, etc.) laws and precedents > under which such manufacturers can be sued, along with things like breach > of contract between the manufacturer and consumer, and breach of implied > warranty of fitness for a particular purpose and breach of implied warranty > of merchantability. > > A couple of winning lawsuits against manufacturers under these laws and > theories - which judges *already understand* - is, I think, not only a more > likely, but a much faster, route to industry reform. > > All that said, much of this faces the same issues that spam lawsuits faced > - the people who care the most about it are not the ones who can afford to > finance such lawsuits. > > Anne > > Anne P. Mitchell, > Attorney at Law > Legislative Consultant > CEO/President, Institute for Social Internet Public Policy > Member, Cal. Bar Cyberspace Law Committee > Member, Colorado Cyber Committee > Member, Asilomar Microcomputer Workshop Committee > Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law) > Ret. Professor of Law, Lincoln Law School of San Jose > Ret. Chair, Asilomar Microcomputer Workshop From list at satchell.net Wed Oct 5 17:02:59 2016 From: list at satchell.net (Stephen Satchell) Date: Wed, 5 Oct 2016 10:02:59 -0700 Subject: Legislative proposal sent to my Congressman In-Reply-To: References: Message-ID: On 10/05/2016 09:46 AM, jim deleskie wrote: > Can we please not get the government ( who's gov ) involved. I fully agree > that it will not only not help, but will make some things worse. This is > why we can't have nice things. I would be in favor of your pleas if you would accompany it with your proposal for eliminating exploitable devices from accessing the Internet and being the source of damaging traffic. This IS the NANOG mailing list. So far, the "solutions" I've seen put foreward are like requiring government ID at polling places. From Gareth.Tupper at warnerpacific.com Wed Oct 5 17:10:25 2016 From: Gareth.Tupper at warnerpacific.com (Gareth Tupper) Date: Wed, 5 Oct 2016 17:10:25 +0000 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> , Message-ID: <6eafdfcea8b046e1ac8d3608f5dbebfc@Warner1202.warner.local> Looks like a fat finger event... >From Level 3: "On October 4, our voice network experienced a service disruption affecting some of our customers in North America due to a configuration error. We know how important these services are to our customers. As an organization, we're putting processes in place to prevent issues like this from recurring in the future. We were able to restore all services by 9:31am Mountain time." http://www.theregister.co.uk/2016/10/05/level3_voip_blackout_cause/ -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Mel Beckman Sent: Tuesday, October 4, 2016 1:09 PM To: Marco Teixeira Cc: Shawn Ritchie ; NANOG list Subject: Re: Level 3 voice outage Possibly somebody YANGed when they should have yinged :) -mel beckman On Oct 4, 2016, at 1:06 PM, Marco Teixeira > wrote: Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There are so many things that can go wrong on complex networks even with all the controls imposed on changes... On Tue, Oct 4, 2016 at 8:54 PM, Shawn Ritchie > wrote: Well, Level3 has by no means said that this was the result of a DDoS, that's just speculation on behalf of folks who do not work at Level3 so far. On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman > wrote: > 765 Gbps per second directed at a router's interface IP might give the > router pause, so to speak :) > > -mel > > On Oct 4, 2016, at 12:10 PM, Marco Teixeira > > > wrote: > > Multiple reboots across several markets... Does not seem something > that full pipes would trigger. Had it been an approved chance it would > have been rolled back i guess... On the other hand, a zero day could apply... > > Em 04/10/2016 19:54, "Mel Beckman" > escreveu: > >> Sure. The recent release of the IoT DDoS attack code in the wild. >> >> -mel >> >> > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: >> > >> > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: >> > >> >> This could be DoS attack. >> > >> > Or a missing comma in a code update. >> > >> > Or a fumble-fingered NOC monkey. >> > >> > Or.... >> > >> > You have any reason to suspect a DoS attack rather than all the >> > other possibilities? >> >> > -- -- Shawn This electronic mail transmission contains information from Warner Pacific Insurance Services that may be confidential or privileged. Such information is solely for the intended recipient, and use by any other party is not authorized. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this message, its contents or any attachments is prohibited. Any wrongful interception of this message is punishable as a Federal Crime. If you have received this message in error, please notify the sender immediately by telephone (800) 801-2300 or by electronic mail at postmaster at warnerpacific.com. From ekgermann at semperen.com Wed Oct 5 15:08:24 2016 From: ekgermann at semperen.com (Eric Germann) Date: Wed, 5 Oct 2016 11:08:24 -0400 Subject: Questions re: VPN protocols globally In-Reply-To: References: Message-ID: <815A3E3A-ECA1-4349-B760-9DD72856013C@semperen.com> I?m aware. We?re considering them down the line. So, back to the question, any ITAR gotchas with any of these companies? Thanks EKG > On Oct 5, 2016, at 11:05 AM, Peter Beckman wrote: > > There is a Mumbai, India three letter company region available as of June 27, 2016 > > https://aws.amazon.com/blogs/aws/now-open-aws-asia-pacific-mumbai-region/ > > On Tue, 4 Oct 2016, Eric Germann wrote: > >> I?ve been charged with building a global VPN as an overlay on top of a certain 3 letter company who also sells lots of stuff. >> >> We?re looking at >> >> US East >> US West >> US Central (eventually) >> Brazil >> Singapore >> Frankfurt >> Ireland >> Sydney >> Maybe Canada >> Maybe India (outsourcesrs) >> >> In the planning stages now and wondering if there are any protocols I need to stay away from ITAR wise with this list of countries. >> >> Contemplating Suite B with GCM, etc and AES acceleration. >> >> Any land mines? >> >> Thanks in advance >> >> EKG >> >> > > --------------------------------------------------------------------------- > Peter Beckman Internet Guy > beckman at angryox.com http://www.angryox.com/ > --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3705 bytes Desc: not available URL: From ekgermann at semperen.com Wed Oct 5 16:06:07 2016 From: ekgermann at semperen.com (Eric Germann) Date: Wed, 5 Oct 2016 12:06:07 -0400 Subject: Questions re: VPN protocols globally In-Reply-To: References: Message-ID: <6B61B192-64AF-463E-BA00-1DD12D3338D1@semperen.com> IPSec and corporate. Customers will connect to their respective regional sites separately. Any ITAR concerns there? > On Oct 5, 2016, at 12:01 PM, Christopher Morrow wrote: > > > > On Tue, Oct 4, 2016 at 11:15 PM, Eric Germann > wrote: > I?ve been charged with building a global VPN as an overlay on top of a certain 3 letter company who also sells lots of stuff. > > > you say 'vpn' do you mean 'mpls vpn' or 'ipsec vpn over intertubes' ? > > We?re looking at > > US East > US West > US Central (eventually) > Brazil > Singapore > Frankfurt > Ireland > Sydney > Maybe Canada > Maybe India (outsourcesrs) > > In the planning stages now and wondering if there are any protocols I need to stay away from ITAR wise with this list of countries. > > Contemplating Suite B with GCM, etc and AES acceleration. > > > most places dont' really care about encryption if your use is 'for corporate use', not providing use by external parties (internet access sorts of things), I believe. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3705 bytes Desc: not available URL: From hcrowder at empiricalnetworks.com Wed Oct 5 14:15:57 2016 From: hcrowder at empiricalnetworks.com (Harry Crowder) Date: Wed, 5 Oct 2016 09:15:57 -0500 Subject: Legislative proposal sent to my Congressman In-Reply-To: <16951763-5f17-7d29-6e07-2c1dd49cf6af@cox.net> References: <16951763-5f17-7d29-6e07-2c1dd49cf6af@cox.net> Message-ID: <038501d21f12$fe7a5c70$fb6f1550$@empiricalnetworks.com> The term you are referencing is unicast reverse path verify strict/hard mode Enforces that the packets source can be reached via the interface of the receiving traffic If this is generaly applied at all provider edge routers and dsl/dialup/vpc pop's would solve the spoofing issue as a whole -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Larry Sheldon Sent: Monday, October 3, 2016 5:36 PM To: Stephen Satchell ; nanog at nanog.org; ietf-action at ietf.org Cc: soc at us-cert.gov; action at eff.org Subject: Re: Legislative proposal sent to my Congressman On 10/3/2016 13:58, Stephen Satchell wrote: > In thinking over the last DDos involving IoT devices, I think we don't > have a good technical solution to the problem. Cutting off people > with defective devices they they don't understand, and have little > control over, is an action that makes sense, but hurts the innocent. > "Hey, Grandma, did you know your TV set is hurting the Internet?" > > It's the people who foist bad stuff on the people who need to take the > responsibility. Indeed, with enough moxie, we could avoid the net > saturation problem in the first place. > > My proposal, as I sent it to my US House Representative: > [much snipping] > Why not nip the IoT problem in the bud? Why not, indeed? (Full disclosure: I am not and have not for some years been active in management of any networks, and I AM woefully behind the state of the arts.) Having said that, it occurs to me that Mr. Satchell's proposal (and most of the others I have read about here and elsewhere lately) are doomed to the same failure as Chicago's plan for reducing illegal deaths by firearm, and for much the same reason (discussion of which here I will spare you. Back in the day, I was fighting a problem that I summarized (then and now) as trying to stop the use and abuse of the University's (that employed me) 56kb Frame Relay link to the Internet. Then as now I defined "abuse" as the use of our facilities for purposes that no stretch of imagination or definition could be said to be to the University's benefit. Through some experimentation I concluded that there were several clearly identifiable sources of abuse. I disremember the ordering by severity but they included: Outright attacks on the University and others. Myriad "scans" for a variety of reasons. The first of these two I remember as being the worst (in terms of item-count AND in terms of packet-size. I also recall it being the easiest to fix, if anybody want to fix it. (The dominant reasons given where that it would cost money without a revenue stream, and it would reduce traffic that WAS in the revenue stream. The fix I proposed: Require (by law) that every service provider and every origination customer of a service provider would under penalty of law, block the transmission of a packet whose source address could not be reached via the link upon which it was found. The Myriad scans problem was a little harder (for among other reasons--the argument that they were good for us, even though they accounted for something like 60% of the traffic on that link). The solution I tried but ran out of dollars on was to detect somebody scanning and route them to the Loopback interface of the boundary router. -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From mel at beckman.org Wed Oct 5 18:24:48 2016 From: mel at beckman.org (Mel Beckman) Date: Wed, 5 Oct 2016 18:24:48 +0000 Subject: Level 3 voice outage In-Reply-To: <6eafdfcea8b046e1ac8d3608f5dbebfc@Warner1202.warner.local> References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> <6eafdfcea8b046e1ac8d3608f5dbebfc@Warner1202.warner.local> Message-ID: It?s good to see them acknowledging this. -mel On Oct 5, 2016, at 10:10 AM, Gareth Tupper > wrote: Looks like a fat finger event... From Level 3: "On October 4, our voice network experienced a service disruption affecting some of our customers in North America due to a configuration error. We know how important these services are to our customers. As an organization, we're putting processes in place to prevent issues like this from recurring in the future. We were able to restore all services by 9:31am Mountain time." http://www.theregister.co.uk/2016/10/05/level3_voip_blackout_cause/ -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Mel Beckman Sent: Tuesday, October 4, 2016 1:09 PM To: Marco Teixeira > Cc: Shawn Ritchie >; NANOG list > Subject: Re: Level 3 voice outage Possibly somebody YANGed when they should have yinged :) -mel beckman On Oct 4, 2016, at 1:06 PM, Marco Teixeira > wrote: Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There are so many things that can go wrong on complex networks even with all the controls imposed on changes... On Tue, Oct 4, 2016 at 8:54 PM, Shawn Ritchie > wrote: Well, Level3 has by no means said that this was the result of a DDoS, that's just speculation on behalf of folks who do not work at Level3 so far. On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman > wrote: 765 Gbps per second directed at a router's interface IP might give the router pause, so to speak :) -mel On Oct 4, 2016, at 12:10 PM, Marco Teixeira > wrote: Multiple reboots across several markets... Does not seem something that full pipes would trigger. Had it been an approved chance it would have been rolled back i guess... On the other hand, a zero day could apply... Em 04/10/2016 19:54, "Mel Beckman" > escreveu: Sure. The recent release of the IoT DDoS attack code in the wild. -mel On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: This could be DoS attack. Or a missing comma in a code update. Or a fumble-fingered NOC monkey. Or.... You have any reason to suspect a DoS attack rather than all the other possibilities? -- -- Shawn This electronic mail transmission contains information from Warner Pacific Insurance Services that may be confidential or privileged. Such information is solely for the intended recipient, and use by any other party is not authorized. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this message, its contents or any attachments is prohibited. Any wrongful interception of this message is punishable as a Federal Crime. If you have received this message in error, please notify the sender immediately by telephone (800) 801-2300 or by electronic mail at postmaster at warnerpacific.com. From Valdis.Kletnieks at vt.edu Wed Oct 5 18:56:18 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Wed, 05 Oct 2016 14:56:18 -0400 Subject: Questions re: VPN protocols globally In-Reply-To: <6B61B192-64AF-463E-BA00-1DD12D3338D1@semperen.com> References: <6B61B192-64AF-463E-BA00-1DD12D3338D1@semperen.com> Message-ID: <20301.1475693778@turing-police.cc.vt.edu> On Wed, 05 Oct 2016 12:06:07 -0400, Eric Germann said: > Customers will connect to their respective regional sites separately. > Any ITAR concerns there? If there are serious concerns there, I recommend spending the coin for an actual ITAR expert. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From fw at deneb.enyo.de Wed Oct 5 20:50:25 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 05 Oct 2016 22:50:25 +0200 Subject: nested prefixes in Internet In-Reply-To: (Martin T.'s message of "Wed, 5 Oct 2016 10:45:20 +0300") References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> Message-ID: <8737ka4j1q.fsf@mid.deneb.enyo.de> * Martin T.: > Florian: > >> Are the autonomous systems for the /19 and /24 connected directly? > > Yes they are. Then deaggregation really isn't necessary at all. >> (1) can be better from B's perspective because it prevents certain >> routing table optimizations (due to the lack of the covering prefix) > > What kind of routing table optimizations are possible if covering /19 > prefix is also present in global routing table? The /24 prefix could arguably be dropped and ignored for routing decisions. From fw at deneb.enyo.de Wed Oct 5 21:41:36 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 05 Oct 2016 23:41:36 +0200 Subject: Questions re: VPN protocols globally In-Reply-To: <20301.1475693778@turing-police.cc.vt.edu> (Valdis Kletnieks's message of "Wed, 05 Oct 2016 14:56:18 -0400") References: <6B61B192-64AF-463E-BA00-1DD12D3338D1@semperen.com> <20301.1475693778@turing-police.cc.vt.edu> Message-ID: <87d1je323z.fsf@mid.deneb.enyo.de> * Valdis Kletnieks: > On Wed, 05 Oct 2016 12:06:07 -0400, Eric Germann said: > >> Customers will connect to their respective regional sites separately. >> Any ITAR concerns there? > > If there are serious concerns there, I recommend spending the coin for > an actual ITAR expert. Right. I *think* it is possible to pull this off, but I expect that you have to file some paperwork, and doing that without proper training and knowledge of the applicable guidelines seems way too risky. (There isn't just ITAR. Local regulations apply as well.) From rfg at tristatelogic.com Wed Oct 5 23:55:18 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 05 Oct 2016 16:55:18 -0700 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot Message-ID: <11161.1475711718@segfault.tristatelogic.com> My analysis: Serious and apparently long-lived bogosity, with a clear history of substantial spamming aactivity. But you be the judge. Looks to me like an unregistered RIPE AS announcing a route to a /20 worth of unregistered RIPE IPv4 space. And this didn't exactly crop up just yesterday. Looks like this has been ongoing for one hell of a long time: https://stat.ripe.net/widget/routing-history#w.resource=AS47860 Of course, it's not even nearly as much of an issue -now- as it was, say, about 1 year ago, in October of 2015, when the /20 was apparently populated by a huge boat load of snowshoe spammer domains. Sadly, Spamhaus has a bad habit of consistantly failing to ever put any helpful date information on any of its listings, otherwise I'd be able to see when -they- first noticed this absurd mess. https://www.spamhaus.org/pbl/query/PBL1626432 Anyway, it's rather annoying to me personally... and I hope I'm not the only one who feels that way... to know that this has gone mostly unnoticed for so long, that nobody within the RIPE region has ever bothered to -do- anything about it, and that the AS and the bogus route are still being announced, even as we speak. Assuming the thing remains in play, how long will be be before the spammers return to use and abuse it yet again? Maybe they were just waiting for a full year to go by so that they might have some hopes of this /20 being automatically aged off some blacklists. Regards, rfg P.S. This crap appears to be be brought to us courtesy of AS29632, NetAssist, LLC: http://new.netassist.ua/ So anyway, where are the grownups? From morrowc.lists at gmail.com Thu Oct 6 02:44:16 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Wed, 5 Oct 2016 22:44:16 -0400 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <11161.1475711718@segfault.tristatelogic.com> References: <11161.1475711718@segfault.tristatelogic.com> Message-ID: On Wed, Oct 5, 2016 at 7:55 PM, Ronald F. Guilmette wrote: > > > > P.S. This crap appears to be be brought to us courtesy of AS29632, > NetAssist, LLC: > > http://new.netassist.ua/ > > So anyway, where are the grownups? > clearly whomever provides transit to 29632... probably worth hunting them all down and asking questions. From martin at airwire.ie Thu Oct 6 08:34:13 2016 From: martin at airwire.ie (Martin List-Petersen) Date: Thu, 6 Oct 2016 09:34:13 +0100 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <11161.1475711718@segfault.tristatelogic.com> References: <11161.1475711718@segfault.tristatelogic.com> Message-ID: <88dd5d7b-db22-484f-1cec-1bf9fecd1154@airwire.ie> On 06/10/16 00:55, Ronald F. Guilmette wrote: > Anyway, it's rather annoying to me personally... and I hope I'm not the > only one who feels that way... to know that this has gone mostly unnoticed > for so long, that nobody within the RIPE region has ever bothered to -do- > anything about it, and that the AS and the bogus route are still being > announced, even as we speak. I had a look in my feeds, then a few global BGP LG's and well, it's not in the BGP table. In reality, it's the upstream, that feeds it in, that really needs to be penalised. Kind regards, Martin List-Petersen -- Airwire Ltd. - Ag Nascadh Pobail an Iarthair http://www.airwire.ie Phone: 091-865 968 Registered Office: Moy, Kinvara, Co. Galway, 091-865 968 - Registered in Ireland No. 508961 From rfg at tristatelogic.com Thu Oct 6 19:28:35 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 06 Oct 2016 12:28:35 -0700 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <20161006163137.uvcnzodrve6tom43@cisco.com> Message-ID: <15476.1475782115@segfault.tristatelogic.com> In message <20161006163137.uvcnzodrve6tom43 at cisco.com>, Joseph Karpenko wrote: >> >> P.S. This crap appears to be be brought to us courtesy of AS29632, >> NetAssist, LLC: >> >> http://new.netassist.ua/ >> > >assuming accuracy of records, etc... ;-) Right. An that doesn't seem to be RIPE's strong suit. >or courtesy of both AS43659 (who was peering with and announcing the prefix to>) >and AS29632 (who was then accepting and announcing to its upstreams)? seems to >be an interesting relationship between the two (2) of them; along with an even >more interesting relationship/affiliation between AS43659 and AS57166 - and the >upstream for both the ASNs is/was AS29632 (NetAssist LLC). ;-) Well, yes. I tried to untangle the relationships here just by looking at bgp.he.net, but as I looked at all of the relevant pages, nothing seemed to be adding up, or even remaining consistant among all of the info that bgp.he.net was showing me. So I just shrugged, gave up, and reported the few facts that I felt sure about here. Specifically, bgp.he.net is reporting the name associated with AS47860 as "Albino, LLC", but personally, I have no idea where they are getting that name from. (And it sure doesn't look like a European style of company name... rather more American, I think.) Then I looked at the bgp.he.net connectivity graph for AS47860: http://bgp.he.net/AS47860#_graph4 This suggests that AS47860 is connected to the Internet only via AS43659, D2 International Investment Ukraine Ltd. (That AS, it seems, is currently announcing -zero- routes of its own, which seems, well, odd.) The connectivity graph for AS43659 is here: http://bgp.he.net/AS43659#_graph4 This seems to indicate that AS43659 is only connected to the Internet via AS29632 and that AS29632 is itself -only- connected to the Internet via AS6939. But then when I looked at the connectivity graph for AS29632 it actually appears to have -five- different IPv4 peers: http://bgp.he.net/AS29632#_graph4 But then I looked at the actual -list- of IPv4 peers of AS29632 and I see it has 121 of them! http://bgp.he.net/AS29632#_peers So, anyay, bottom line, there are clearly things about how bgp.he.net draws connectivity graphs that I don't actually undetrstand. That's OK. I don't need to understand any of that in order to understand that AS47860 is a bogus unregistered AS which is, and which has been, apparently, for some long time, announcing a route (93.175.240.0/20) to unregistered RIPE IPv4 space. Sadly, announcing of bogons is not uncommon, so I wouldn't even have mentioned this if it hadn't been for the fact that historical passive DNS data indicate quite clearly that at least one snowshoe spammer was using that IPv4 space at about this time last year. Regards, rfg From jlmcgraw at gmail.com Thu Oct 6 20:26:48 2016 From: jlmcgraw at gmail.com (Jesse McGraw) Date: Thu, 6 Oct 2016 16:26:48 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension Message-ID: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Nanog, (This is me scratching an itch of my own and hoping that sharing it might be useful to others on this list. Apologies if it isn't) When I'm trying to comprehend a new or complicated Cisco router, switch or firewall configuration an old pet-peeve of mine is how needlessly difficult it is to follow deeply nested logic in route-maps, ACLs, QoS policy-maps etc etc To make this a bit simpler I?ve been working on a perl script to convert these text-based configuration files into HTML with links between the different elements (e.g. To an access-list from the interface where it?s applied, from policy-maps to class-maps etc), hopefully making it easier to to follow the chain of logic via clicking links and using the forward and back buttons in your browser to go back and forth between command and referenced list. I've put the script itself up here : https://github.com/jlmcgraw/network_configuration_navigator See here for output examples http://htmlpreview.github.com/?https://github.com/jlmcgraw/network_configuration_navigator/blob/master/examples/html_test_case_1.cfg.html Here's a quick web demo on Heroku https://hidden-waters-8218.herokuapp.com/ (This is just a simple web front-end to the script. I'm not a web-savvy guy so I'm sure it's poorly coded and terribly insecure. Please don't upload anything sensitive to this, it's just for testing!) I know there is a lot of stuff that could be done better so let me know if you think of anything new or notice something I?ve done wrong. One unexpected thing that has come out of this script is the ability to catch items that are defined but never actually used, whether it's due to a fat-finger or just being leftover cruft. This has proven very valuable in catching mistakes that are otherwise hard to spot. Unfortunately the script can't currently catch the inverse (things that are called but never defined) due to the way the regexes are constructed Surely this has all been done before but I couldn't find anything in a few brief moments of searching so here we are. -Jesse Notes: See the box on the right for a key and links to jump to the first line of the various types of sections or unused items There are some command-line options for reformatting (make some numbers that are hard to read into more human-readable ones, add colors to permits/denies, scrub sensitive info etc, remove some redundancy). Try and see what you like. If you run it against multiple configuration files at once it will also attempt to link between them when applicable (e.g. BGP neighbors, route next hops, interfaces on the same subnet etc). I regularly use it on a ~900 configuration files set with no problems Developed under Ubuntu Linux, somewhat tested on Windows but not at all on OS Based on configs that I work with so it doesn't cover all possible commands. Send patches! From ler762 at gmail.com Thu Oct 6 21:33:28 2016 From: ler762 at gmail.com (Lee) Date: Thu, 6 Oct 2016 17:33:28 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: On 10/6/16, Jesse McGraw wrote: > Nanog, > > (This is me scratching an itch of my own and hoping that sharing it > might be useful to others on this list. Apologies if it isn't) > > When I'm trying to comprehend a new or complicated Cisco router, > switch or firewall configuration an old pet-peeve of mine is how > needlessly difficult it is to follow deeply nested logic in route-maps, > ACLs, QoS policy-maps etc etc > > To make this a bit simpler I?ve been working on a perl script to convert > these text-based configuration files into HTML with links between the > different elements (e.g. To an access-list from the interface where it?s > applied, from policy-maps to class-maps etc), hopefully making it easier > to to follow the chain of logic via clicking links and using the forward > and back buttons in your browser to go back and forth between command > and referenced list. > > > I've put the script itself up here > : > https://github.com/jlmcgraw/network_configuration_navigator > > See here > > > for output examples > http://htmlpreview.github.com/?https://github.com/jlmcgraw/network_configuration_navigator/blob/master/examples/html_test_case_1.cfg.html > > Here's a quick web demo on > Heroku > https://hidden-waters-8218.herokuapp.com/ > (This is just a simple web front-end to the script. I'm not a > web-savvy guy so I'm sure it's poorly coded and terribly insecure. > Please don't upload anything sensitive to this, it's just for > testing!) > > I know there is a lot of stuff that could be done better so let me know > if you think of anything new or notice something I?ve done wrong. > > One unexpected thing that has come out of this script is the ability to > catch items that are defined but never actually used, whether it's due > to a fat-finger or just being leftover cruft. This has proven very > valuable in catching mistakes that are otherwise hard to spot. > Unfortunately the script can't currently catch the inverse (things that > are called but never defined) due to the way the regexes are constructed > > Surely this has all been done before but I couldn't find anything in a > few brief moments of searching so here we are. dunno about creating web pages, but https://www.nanog.org/meetings/abstract?id=785 has a section on showing filters that are defined but not referenced & referenced but not defined Regards, Lee From eyeronic.design at gmail.com Thu Oct 6 21:37:11 2016 From: eyeronic.design at gmail.com (Mike Hale) Date: Thu, 6 Oct 2016 14:37:11 -0700 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: Neat! On Thu, Oct 6, 2016 at 1:26 PM, Jesse McGraw wrote: > Nanog, > > (This is me scratching an itch of my own and hoping that sharing it > might be useful to others on this list. Apologies if it isn't) > > When I'm trying to comprehend a new or complicated Cisco router, switch or > firewall configuration an old pet-peeve of mine is how needlessly difficult > it is to follow deeply nested logic in route-maps, ACLs, QoS policy-maps etc > etc > > To make this a bit simpler I?ve been working on a perl script to convert > these text-based configuration files into HTML with links between the > different elements (e.g. To an access-list from the interface where it?s > applied, from policy-maps to class-maps etc), hopefully making it easier to > to follow the chain of logic via clicking links and using the forward and > back buttons in your browser to go back and forth between command and > referenced list. > > > I've put the script itself up here > : > https://github.com/jlmcgraw/network_configuration_navigator > > See here > > for output examples > http://htmlpreview.github.com/?https://github.com/jlmcgraw/network_configuration_navigator/blob/master/examples/html_test_case_1.cfg.html > > Here's a quick web demo on > Heroku > https://hidden-waters-8218.herokuapp.com/ > (This is just a simple web front-end to the script. I'm not a web-savvy > guy so I'm sure it's poorly coded and terribly insecure. > Please don't upload anything sensitive to this, it's just for testing!) > > I know there is a lot of stuff that could be done better so let me know if > you think of anything new or notice something I?ve done wrong. > > One unexpected thing that has come out of this script is the ability to > catch items that are defined but never actually used, whether it's due to a > fat-finger or just being leftover cruft. This has proven very valuable in > catching mistakes that are otherwise hard to spot. Unfortunately the script > can't currently catch the inverse (things that are called but never defined) > due to the way the regexes are constructed > > Surely this has all been done before but I couldn't find anything in a few > brief moments of searching so here we are. > > -Jesse > > > > Notes: > See the box on the right for a key and links to jump to the first line > of the various types of sections or unused items > > There are some command-line options for reformatting (make some numbers > that are hard to read into more human-readable ones, add colors to > permits/denies, scrub sensitive info etc, remove some redundancy). Try and > see what you like. > > If you run it against multiple configuration files at once it will also > attempt to link between them when applicable (e.g. BGP neighbors, route next > hops, interfaces on the same subnet etc). I regularly use it on a ~900 > configuration files set with no problems > > Developed under Ubuntu Linux, somewhat tested on Windows but not at all > on OS > > Based on configs that I work with so it doesn't cover all possible > commands. Send patches! -- 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 From larrysheldon at cox.net Fri Oct 7 00:42:34 2016 From: larrysheldon at cox.net (Larry Sheldon) Date: Thu, 6 Oct 2016 19:42:34 -0500 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: Message-ID: On 10/6/2016 15:26, Jesse McGraw wrote: > (This is me scratching an itch of my own and hoping that sharing it > might be useful to others on this list. Apologies if it isn't) > > When I'm trying to comprehend a new or complicated Cisco router, > switch or firewall configuration an old pet-peeve of mine is how > needlessly difficult it is to follow deeply nested logic in route-maps, > ACLs, QoS policy-maps etc etc A dim, weak voice from the past. Has advantages of the plan proposed here. > > To make this a bit simpler I?ve been working on a perl script to convert > these text-based configuration files into HTML with links between the > different elements (e.g. To an access-list from the interface where it?s > applied, from policy-maps to class-maps etc), hopefully making it easier > to to follow the chain of logic via clicking links and using the forward > and back buttons in your browser to go back and forth between command > and referenced list. We used to (using a HB lead in a draftsman' lead holder and a stack for Forms SN 457* (Blank Spread Sheet, 11 x 17) sorted all of the requests, demands and other requirements into logical packages. Then, using the blank back side of the spread sheet, we drew "flow diagrams depicting how we would code the requirements. If a section got a little complicated and tedious, we'd put a symbol on the diagram, a title that made sense and a page number. On a new sheet, we wrote that title and that page number and drew the flow diagram for that messy bit of business. Then we would "desk check" the flow diagrams and in the process, note on the requirements sheet (s) the diagram number (and entry point if there was more than one) where the requirement was satisfied. Then we would start with a new sheet working from the flow diagrams, write the code for the machine (noting on the flow diagram the page and line number in the code where the operation on the flow diagram occurred. There are several advantages to this approach--hard to leave important stuff out, hard to include code that is never exercised, hard to make changes to the code because you don't know how to make HTML depict it correctly. No need to lecture me on the folly of the old ways--it is why I got fired for being too old. -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From marka at isc.org Fri Oct 7 02:02:42 2016 From: marka at isc.org (Mark Andrews) Date: Fri, 07 Oct 2016 13:02:42 +1100 Subject: EDNS compliance and BIND 9.11.0 Message-ID: <20161007020242.1BDC755EB3D7@rock.dv.isc.org> BIND 9.11.0 was released this week. BIND 9.11.0 and BIND 9.10.4 implement EDNS COOKIES. They are on by default in BIND 9.11.0 and BIND 9.10.4 Windows. For the non Windows builds of BIND 9.10.4 they need to be enabled at configure time. If your nameservers are not EDNS compliant, especially for unknown EDNS options, you could be seeing issues ranging from additional traffic, slow DNS lookups to full blown lookup failures. If you having fixed your servers already you need to fix them now. You can test your servers at https://ednscomp.isc.org/ednscomp/ to see if they are EDNS compliant. You can see EDNS compliance trends and list of failing servers at https://ednscomp.isc.org. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From hank at efes.iucc.ac.il Fri Oct 7 05:34:49 2016 From: hank at efes.iucc.ac.il (Hank Nussbacher) Date: Fri, 7 Oct 2016 08:34:49 +0300 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: On 07/10/2016 00:33, Lee wrote: > dunno about creating web pages, but > https://www.nanog.org/meetings/abstract?id=785 > has a section on showing filters that are defined but not referenced & > referenced but not defined In IOS-XR it is one command "sho rpl unused ?" RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused ? as-path-set Display as-path-set objects community-set Display community-set objects extcommunity-set Display extended community objects prefix-set Display prefix-set objects rd-set Display rd-set objects route-policy Display route-policy objects tag-set Display tag-set objects RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused prefix Fri Oct 7 08:24:53.237 IDT ACTIVE -- Referenced by at least one policy which is attached INACTIVE -- Only referenced by policies which are not attached UNUSED -- Not attached (directly or indirectly) and not referenced -Hank > > Regards, > Lee > From karpenko at cisco.com Thu Oct 6 16:31:37 2016 From: karpenko at cisco.com (Joseph Karpenko) Date: Thu, 6 Oct 2016 11:31:37 -0500 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <11161.1475711718@segfault.tristatelogic.com> References: <11161.1475711718@segfault.tristatelogic.com> Message-ID: <20161006163137.uvcnzodrve6tom43@cisco.com> > > P.S. This crap appears to be be brought to us courtesy of AS29632, > NetAssist, LLC: > > http://new.netassist.ua/ > assuming accuracy of records, etc... ;-) or courtesy of both AS43659 (who was peering with and announcing the prefix to) and AS29632 (who was then accepting and announcing to its upstreams)? seems to be an interesting relationship between the two (2) of them; along with an even more interesting relationship/affiliation between AS43659 and AS57166 - and the upstream for both the ASNs is/was AS29632 (NetAssist LLC). ;-) - AS57166 UA-D2INVESTUKRAINE-AS, UA; D2 International Investment Ukraine Ltd. - AS43659 BUDREMYER-AS, UA; D2 International Investment Ukraine Ltd. SAME EMAIL/ABUSE CONTACTS (and address) for both ASNs (AS43659 and AS57166): - EMAIL CONTACTS: abuse at etthua.net; d2invest at meta.ua; support at etthua.net - ABUSE CONTACTS: abuse at etthua.net RELATED DOMAINS: - budremyer.su - etthua.net - meta.ua BOGUS ROUTES AND AS ANNOUNCEMENTS 93.175.240.0/20 AS47860 -Reserved AS-, ZZ 93.175.240.0 - 93.175.255.255 - 93.175.240.0/20: http://93.175.240.0.20.potaroo.net/ Origins: 47860 (7d 10h 47m 1s, 1 times) -- (AS47860: -Reserved AS-, ZZ) Next AS Hops: 43659 (7d 10h 47m 1s, 1 times) -- (AS43659: BUDREMYER-AS , UA) Paths: 4608 1221 4637 174 29632 43659 47860 (5d 13h 41m 44s, 1 times, avg 5d 13h 41m 44.0s) 4777 2497 6939 29632 29632 29632 29632 29632 43659 47860 (1d 21h 5m 17s, 1 times, avg 1d 21h 5m 17.0s) AS47860 -> AS43659 -> AS29632 - AS47860 (RIPE NCC ASN BLOCK); http://www.cidr-report.org/cgi-bin/as-report?as=AS47860&view=2.0 - AS43659 (BUDREMYER-AS, UKRAINE); http://www.cidr-report.org/cgi-bin/as-report?as=AS43659&view=2.0 - AS29632 (NASSIST-AS, UKRAINE); http://www.cidr-report.org/cgi-bin/as-report?as=AS29632&view=2.0 - UPSTREAM ADJACENT AS - AS20485 TRANSTELECOM Moscow, Russia, RU - AS29107 SYNAPSE-AS , UA - AS8359 MTS , RU - AS35320 ETT-AS , UA - AS6939 HURRICANE - Hurricane Electric, Inc., US regards, -- .karpenko On 2016-10-05T16:55:18-0700, Ronald F. Guilmette wrote: > > My analysis: Serious and apparently long-lived bogosity, with a clear > history of substantial spamming aactivity. > > But you be the judge. > > Looks to me like an unregistered RIPE AS announcing a route to a /20 worth of > unregistered RIPE IPv4 space. > > And this didn't exactly crop up just yesterday. Looks like this has been > ongoing for one hell of a long time: > > https://stat.ripe.net/widget/routing-history#w.resource=AS47860 > > Of course, it's not even nearly as much of an issue -now- as it was, say, > about 1 year ago, in October of 2015, when the /20 was apparently populated > by a huge boat load of snowshoe spammer domains. Sadly, Spamhaus has a bad > habit of consistantly failing to ever put any helpful date information on any > of its listings, otherwise I'd be able to see when -they- first noticed this > absurd mess. > > https://www.spamhaus.org/pbl/query/PBL1626432 > > Anyway, it's rather annoying to me personally... and I hope I'm not the only > one who feels that way... to know that this has gone mostly unnoticed for so > long, that nobody within the RIPE region has ever bothered to -do- anything > about it, and that the AS and the bogus route are still being announced, even > as we speak. > > Assuming the thing remains in play, how long will be be before the spammers > return to use and abuse it yet again? > > Maybe they were just waiting for a full year to go by so that they might have > some hopes of this /20 being automatically aged off some blacklists. > > > Regards, > rfg > > > P.S. This crap appears to be be brought to us courtesy of AS29632, > NetAssist, LLC: > > http://new.netassist.ua/ > > So anyway, where are the grownups? > > > [ -------------------- END OF INCLUDED MESSAGE -------------------- ] From martin at airwire.ie Fri Oct 7 09:28:10 2016 From: martin at airwire.ie (Martin List-Petersen) Date: Fri, 7 Oct 2016 10:28:10 +0100 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: References: <11161.1475711718@segfault.tristatelogic.com> <88dd5d7b-db22-484f-1cec-1bf9fecd1154@airwire.ie> Message-ID: <16d65e07-13b4-f59f-1c69-892861eb6967@airwire.ie> On 06/10/16 16:38, Sandra Murphy wrote: > Private reply: > > bgp.he.net sees it. For me. > > http://bgp.he.net/net/93.175.240.0/20 > > I don?t know why they do and you do not. > > ?Sandy That just means, they "have" seen it. Not that they're seeing it right now, actually. I checked our feed, which you also can at http://lg.as42227.net And various upstream looking glasses, for example HE.net's actually. https://lg.he.net/ NIKHEF Amsterdam Interxion Copenhagen he.net Freemont 2 None of them have the route in the table. Even the CIDR report reports, that it's withdrawn: http://www.cidr-report.org/cgi-bin/as-report?as=AS47860&view=2.0 But it has been seen in the last 7 days. Kind regards, Martin List-Petersen Airwire Ltd. > > >> On Oct 6, 2016, at 4:34 AM, Martin List-Petersen wrote: >> >> On 06/10/16 00:55, Ronald F. Guilmette wrote: >>> Anyway, it's rather annoying to me personally... and I hope I'm not the >>> only one who feels that way... to know that this has gone mostly unnoticed >>> for so long, that nobody within the RIPE region has ever bothered to -do- >>> anything about it, and that the AS and the bogus route are still being >>> announced, even as we speak. >> >> I had a look in my feeds, then a few global BGP LG's and well, it's not in the BGP table. >> >> In reality, it's the upstream, that feeds it in, that really needs to be penalised. >> >> Kind regards, >> Martin List-Petersen >> -- >> Airwire Ltd. - Ag Nascadh Pobail an Iarthair >> http://www.airwire.ie >> Phone: 091-865 968 >> Registered Office: Moy, Kinvara, Co. Galway, 091-865 968 - Registered in Ireland No. 508961 > -- Airwire Ltd. - Ag Nascadh Pobail an Iarthair http://www.airwire.ie Phone: 091-865 968 Registered Office: Moy, Kinvara, Co. Galway, 091-865 968 - Registered in Ireland No. 508961 From martin at airwire.ie Fri Oct 7 09:38:14 2016 From: martin at airwire.ie (Martin List-Petersen) Date: Fri, 7 Oct 2016 10:38:14 +0100 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <15476.1475782115@segfault.tristatelogic.com> References: <15476.1475782115@segfault.tristatelogic.com> Message-ID: On 06/10/16 20:28, Ronald F. Guilmette wrote: > In message <20161006163137.uvcnzodrve6tom43 at cisco.com>, > Joseph Karpenko wrote: > >>> >>> P.S. This crap appears to be be brought to us courtesy of AS29632, >>> NetAssist, LLC: >>> >>> http://new.netassist.ua/ >>> >> >> assuming accuracy of records, etc... ;-) > > Right. An that doesn't seem to be RIPE's strong suit. > It's not so much a questions on RIPE's strong suit, but more the LIRs, that don't keep their info updated. RIPE only updates the basic data, to match it the contract data, but they're quite adament about updated data, if you want further allocations, which now sort of again is ... void. > Specifically, bgp.he.net is reporting the name associated with AS47860 as > "Albino, LLC", but personally, I have no idea where they are getting that > name from. (And it sure doesn't look like a European style of company > name... rather more American, I think.) I reckon .. but this is a guestimate, that the AS and prefix probably was allocated to that company in the past, but either their contract never was finalised or their contract was cancelled by one of the parties. So that might have been the name that "used" to be in the whois database for that prefix and ASN, but now isn't anymore, if the entity has ceased to exist. That could also be the reason, why the prefix and ASN have been seen historically. Either way ... that's a guestimate, but a very plausable one. Only somebody inside RIPE would be able to shed more light into, what actually happened. If they're actually permitted (could be prevented by data protection). Kind regards, Martin List-Petersen -- Airwire Ltd. - Ag Nascadh Pobail an Iarthair http://www.airwire.ie Phone: 091-865 968 Registered Office: Moy, Kinvara, Co. Galway, 091-865 968 - Registered in Ireland No. 508961 From job at instituut.net Fri Oct 7 12:53:18 2016 From: job at instituut.net (Job Snijders) Date: Fri, 7 Oct 2016 14:53:18 +0200 Subject: AS47860 - 93.175.240.0/20 - Wiskey Tango Foxtrot In-Reply-To: <16d65e07-13b4-f59f-1c69-892861eb6967@airwire.ie> References: <11161.1475711718@segfault.tristatelogic.com> <88dd5d7b-db22-484f-1cec-1bf9fecd1154@airwire.ie> <16d65e07-13b4-f59f-1c69-892861eb6967@airwire.ie> Message-ID: <20161007125318.GJ89179@Vurt.local> On Fri, Oct 07, 2016 at 10:28:10AM +0100, Martin List-Petersen wrote: > On 06/10/16 16:38, Sandra Murphy wrote: > > Private reply: > > > > bgp.he.net sees it. For me. > > > > http://bgp.he.net/net/93.175.240.0/20 > > > > I don?t know why they do and you do not. > > That just means, they "have" seen it. Not that they're seeing it right > now, actually. > > I checked our feed, which you also can at http://lg.as42227.net > > And various upstream looking glasses, for example HE.net's actually. > > https://lg.he.net/ > > NIKHEF Amsterdam > Interxion Copenhagen > he.net Freemont 2 > > None of them have the route in the table. Some still have the route: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=93.175.240.0/20 Kind regards, Job From ler762 at gmail.com Fri Oct 7 14:59:09 2016 From: ler762 at gmail.com (Lee) Date: Fri, 7 Oct 2016 10:59:09 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: On 10/7/16, Hank Nussbacher wrote: > On 07/10/2016 00:33, Lee wrote: >> dunno about creating web pages, but >> https://www.nanog.org/meetings/abstract?id=785 >> has a section on showing filters that are defined but not referenced & >> referenced but not defined > > In IOS-XR it is one command "sho rpl unused ?" > RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused ? > as-path-set Display as-path-set objects > community-set Display community-set objects > extcommunity-set Display extended community objects > prefix-set Display prefix-set objects > rd-set Display rd-set objects > route-policy Display route-policy objects > tag-set Display tag-set objects > > RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused prefix > Fri Oct 7 08:24:53.237 IDT > > ACTIVE -- Referenced by at least one policy which is attached > INACTIVE -- Only referenced by policies which are not attached > UNUSED -- Not attached (directly or indirectly) and not referenced I'm actually starting to miss being out of the game. I'm retired, so don't have access to anything running IOS-XR. Just out of curiosity, how does the output of 'show rpl unused prefix' compare to the output of the script at http://pastebin.com/pem7tHAJ Thanks, Lee From cscora at apnic.net Fri Oct 7 18:01:24 2016 From: cscora at apnic.net (Routing Analysis Role Account) Date: Sat, 8 Oct 2016 04:01:24 +1000 (AEST) Subject: Weekly Routing Table Report Message-ID: <20161007180124.0A512C55A1@thyme.apnic.net> This is an automated weekly mailing describing the state of the Internet Routing Table as seen from APNIC's router in Japan. The posting is sent to APOPS, NANOG, AfNOG, AusNOG, SANOG, PacNOG, SAFNOG, SdNOG, BJNOG, CaribNOG and the RIPE Routing WG. Daily listings are sent to bgp-stats at lists.apnic.net For historical data, please see http://thyme.rand.apnic.net. If you have any comments please contact Philip Smith . Routing Table Report 04:00 +10GMT Sat 08 Oct, 2016 Report Website: http://thyme.rand.apnic.net Detailed Analysis: http://thyme.rand.apnic.net/current/ Analysis Summary ---------------- BGP routing table entries examined: 613098 Prefixes after maximum aggregation (per Origin AS): 220270 Deaggregation factor: 2.78 Unique aggregates announced (without unneeded subnets): 299031 Total ASes present in the Internet Routing Table: 54952 Prefixes per ASN: 11.16 Origin-only ASes present in the Internet Routing Table: 36365 Origin ASes announcing only one prefix: 15357 Transit ASes present in the Internet Routing Table: 6515 Transit-only ASes present in the Internet Routing Table: 168 Average AS path length visible in the Internet Routing Table: 4.3 Max AS path length visible: 54 Max AS path prepend of ASN ( 55644) 51 Prefixes from unregistered ASNs in the Routing Table: 59 Unregistered ASNs in the Routing Table: 16 Number of 32-bit ASNs allocated by the RIRs: 15672 Number of 32-bit ASNs visible in the Routing Table: 12072 Prefixes from 32-bit ASNs in the Routing Table: 48549 Number of bogon 32-bit ASNs visible in the Routing Table: 171 Special use prefixes present in the Routing Table: 0 Prefixes being announced from unallocated address space: 368 Number of addresses announced to Internet: 2831156388 Equivalent to 168 /8s, 192 /16s and 4 /24s Percentage of available address space announced: 76.5 Percentage of allocated address space announced: 76.5 Percentage of available address space allocated: 100.0 Percentage of address space in use by end-sites: 98.3 Total number of prefixes smaller than registry allocations: 199304 APNIC Region Analysis Summary ----------------------------- Prefixes being announced by APNIC Region ASes: 156388 Total APNIC prefixes after maximum aggregation: 42908 APNIC Deaggregation factor: 3.64 Prefixes being announced from the APNIC address blocks: 170156 Unique aggregates announced from the APNIC address blocks: 69373 APNIC Region origin ASes present in the Internet Routing Table: 5186 APNIC Prefixes per ASN: 32.81 APNIC Region origin ASes announcing only one prefix: 1148 APNIC Region transit ASes present in the Internet Routing Table: 939 Average APNIC Region AS path length visible: 4.4 Max APNIC Region AS path length visible: 54 Number of APNIC region 32-bit ASNs visible in the Routing Table: 2406 Number of APNIC addresses announced to Internet: 760131012 Equivalent to 45 /8s, 78 /16s and 173 /24s APNIC AS Blocks 4608-4864, 7467-7722, 9216-10239, 17408-18431 (pre-ERX allocations) 23552-24575, 37888-38911, 45056-46079, 55296-56319, 58368-59391, 63488-64098, 64297-64395, 131072-137529 APNIC Address Blocks 1/8, 14/8, 27/8, 36/8, 39/8, 42/8, 43/8, 49/8, 58/8, 59/8, 60/8, 61/8, 101/8, 103/8, 106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8, 116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8, 123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8, 163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8, 203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8, 222/8, 223/8, ARIN Region Analysis Summary ---------------------------- Prefixes being announced by ARIN Region ASes: 184574 Total ARIN prefixes after maximum aggregation: 89501 ARIN Deaggregation factor: 2.06 Prefixes being announced from the ARIN address blocks: 190411 Unique aggregates announced from the ARIN address blocks: 88508 ARIN Region origin ASes present in the Internet Routing Table: 16168 ARIN Prefixes per ASN: 11.78 ARIN Region origin ASes announcing only one prefix: 5673 ARIN Region transit ASes present in the Internet Routing Table: 1700 Average ARIN Region AS path length visible: 3.8 Max ARIN Region AS path length visible: 23 Number of ARIN region 32-bit ASNs visible in the Routing Table: 1499 Number of ARIN addresses announced to Internet: 1107151264 Equivalent to 65 /8s, 253 /16s and 201 /24s ARIN AS Blocks 1-1876, 1902-2042, 2044-2046, 2048-2106 (pre-ERX allocations) 2138-2584, 2615-2772, 2823-2829, 2880-3153 3354-4607, 4865-5119, 5632-6655, 6912-7466 7723-8191, 10240-12287, 13312-15359, 16384-17407 18432-20479, 21504-23551, 25600-26591, 26624-27647, 29696-30719, 31744-33791 35840-36863, 39936-40959, 46080-47103 53248-55295, 62464-63487, 64198-64296, 393216-397212 ARIN Address Blocks 3/8, 4/8, 6/8, 7/8, 8/8, 9/8, 11/8, 12/8, 13/8, 15/8, 16/8, 17/8, 18/8, 19/8, 20/8, 21/8, 22/8, 23/8, 24/8, 26/8, 28/8, 29/8, 30/8, 32/8, 33/8, 34/8, 35/8, 38/8, 40/8, 44/8, 45/8, 47/8, 48/8, 50/8, 52/8, 53/8, 54/8, 55/8, 56/8, 57/8, 63/8, 64/8, 65/8, 66/8, 67/8, 68/8, 69/8, 70/8, 71/8, 72/8, 73/8, 74/8, 75/8, 76/8, 96/8, 97/8, 98/8, 99/8, 100/8, 104/8, 107/8, 108/8, 128/8, 129/8, 130/8, 131/8, 132/8, 134/8, 135/8, 136/8, 137/8, 138/8, 139/8, 140/8, 142/8, 143/8, 144/8, 146/8, 147/8, 148/8, 149/8, 152/8, 155/8, 156/8, 157/8, 158/8, 159/8, 160/8, 161/8, 162/8, 164/8, 165/8, 166/8, 167/8, 168/8, 169/8, 170/8, 172/8, 173/8, 174/8, 184/8, 192/8, 198/8, 199/8, 204/8, 205/8, 206/8, 207/8, 208/8, 209/8, 214/8, 215/8, 216/8, RIPE Region Analysis Summary ---------------------------- Prefixes being announced by RIPE Region ASes: 146152 Total RIPE prefixes after maximum aggregation: 71972 RIPE Deaggregation factor: 2.03 Prefixes being announced from the RIPE address blocks: 156615 Unique aggregates announced from the RIPE address blocks: 96828 RIPE Region origin ASes present in the Internet Routing Table: 18129 RIPE Prefixes per ASN: 8.64 RIPE Region origin ASes announcing only one prefix: 7815 RIPE Region transit ASes present in the Internet Routing Table: 3040 Average RIPE Region AS path length visible: 4.3 Max RIPE Region AS path length visible: 27 Number of RIPE region 32-bit ASNs visible in the Routing Table: 5059 Number of RIPE addresses announced to Internet: 709023488 Equivalent to 42 /8s, 66 /16s and 215 /24s RIPE AS Blocks 1877-1901, 2043, 2047, 2107-2136, 2585-2614 (pre-ERX allocations) 2773-2822, 2830-2879, 3154-3353, 5377-5631 6656-6911, 8192-9215, 12288-13311, 15360-16383 20480-21503, 24576-25599, 28672-29695 30720-31743, 33792-35839, 38912-39935 40960-45055, 47104-52223, 56320-58367 59392-61439, 61952-62463, 64396-64495 196608-207259 RIPE Address Blocks 2/8, 5/8, 25/8, 31/8, 37/8, 46/8, 51/8, 62/8, 77/8, 78/8, 79/8, 80/8, 81/8, 82/8, 83/8, 84/8, 85/8, 86/8, 87/8, 88/8, 89/8, 90/8, 91/8, 92/8, 93/8, 94/8, 95/8, 109/8, 141/8, 145/8, 151/8, 176/8, 178/8, 185/8, 188/8, 193/8, 194/8, 195/8, 212/8, 213/8, 217/8, LACNIC Region Analysis Summary ------------------------------ Prefixes being announced by LACNIC Region ASes: 62520 Total LACNIC prefixes after maximum aggregation: 12578 LACNIC Deaggregation factor: 4.97 Prefixes being announced from the LACNIC address blocks: 78550 Unique aggregates announced from the LACNIC address blocks: 37624 LACNIC Region origin ASes present in the Internet Routing Table: 2482 LACNIC Prefixes per ASN: 31.65 LACNIC Region origin ASes announcing only one prefix: 544 LACNIC Region transit ASes present in the Internet Routing Table: 587 Average LACNIC Region AS path length visible: 4.9 Max LACNIC Region AS path length visible: 22 Number of LACNIC region 32-bit ASNs visible in the Routing Table: 2844 Number of LACNIC addresses announced to Internet: 170638912 Equivalent to 10 /8s, 43 /16s and 190 /24s LACNIC AS Blocks 26592-26623, 27648-28671, 52224-53247, 61440-61951, 64099-64197, 262144-266652 + ERX transfers LACNIC Address Blocks 177/8, 179/8, 181/8, 186/8, 187/8, 189/8, 190/8, 191/8, 200/8, 201/8, AfriNIC Region Analysis Summary ------------------------------- Prefixes being announced by AfriNIC Region ASes: 14855 Total AfriNIC prefixes after maximum aggregation: 3301 AfriNIC Deaggregation factor: 4.50 Prefixes being announced from the AfriNIC address blocks: 16998 Unique aggregates announced from the AfriNIC address blocks: 6371 AfriNIC Region origin ASes present in the Internet Routing Table: 740 AfriNIC Prefixes per ASN: 22.97 AfriNIC Region origin ASes announcing only one prefix: 177 AfriNIC Region transit ASes present in the Internet Routing Table: 184 Average AfriNIC Region AS path length visible: 4.7 Max AfriNIC Region AS path length visible: 18 Number of AfriNIC region 32-bit ASNs visible in the Routing Table: 264 Number of AfriNIC addresses announced to Internet: 83812096 Equivalent to 4 /8s, 254 /16s and 223 /24s AfriNIC AS Blocks 36864-37887, 327680-328703 & ERX transfers AfriNIC Address Blocks 41/8, 102/8, 105/8, 154/8, 196/8, 197/8, APNIC Region per AS prefix count summary ---------------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5559 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3532 382 249 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2953 904 78 TELKOMNET-AS2-AP PT Telekomunikasi Indo 4766 2815 11131 853 KIXS-AS-KR Korea Telecom, KR 9829 2699 1497 532 BSNL-NIB National Internet Backbone, IN 9808 2185 8781 42 CMNET-GD Guangdong Mobile Communication 4755 2056 429 219 TATACOMM-AS TATA Communications formerl 4808 1769 2289 522 CHINA169-BJ China Unicom Beijing Provin 45899 1563 1235 90 VNPT-AS-VN VNPT Corp, VN 24560 1549 516 220 AIRTELBROADBAND-AS-AP Bharti Airtel Ltd Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-APNIC ARIN Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 22773 3526 2964 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 6389 2218 3671 41 BELLSOUTH-NET-BLK - BellSouth.net Inc., 18566 2196 405 110 MEGAPATH5-US - MegaPath Corporation, US 20115 1941 1985 406 CHARTER-NET-HKY-NC - Charter Communicat 30036 1746 336 294 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 209 1725 5070 658 CENTURYLINK-US-LEGACY-QWEST - Qwest Com 6983 1677 848 227 ITCDELTA - Earthlink, Inc., US 701 1612 10674 674 UUNET - MCI Communications Services, In 16509 1424 2537 474 AMAZON-02 - Amazon.com, Inc., US 7018 1366 20092 1006 ATT-INTERNET4 - AT&T Services, Inc., US Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-ARIN RIPE Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 39891 3329 169 15 ALJAWWALSTC-AS , SA 20940 2805 1054 2007 AKAMAI-ASN1 , US 34984 1977 326 354 TELLCOM-AS , TR 12479 1360 1018 46 UNI2-AS , ES 13188 1267 99 57 BANKINFORM-AS , UA 8551 1206 377 46 BEZEQ-INTERNATIONAL-AS Bezeqint Interne 6849 1142 355 21 UKRTELNET , UA 8402 1096 544 15 CORBINA-AS Russia, RU 12389 923 1120 406 ROSTELECOM-AS , RU 6830 892 2755 467 LGI-UPC formerly known as UPC Broadband Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-RIPE LACNIC Region per AS prefix count summary ----------------------------------------- ASN No of nets /20 equiv MaxAgg Description 10620 3497 540 183 Telmex Colombia S.A., CO 8151 2265 3353 556 Uninet S.A. de C.V., MX 7303 1547 958 246 Telecom Argentina S.A., AR 6503 1419 437 54 Axtel, S.A.B. de C.V., MX 11830 1322 367 57 Instituto Costarricense de Electricidad 6147 1230 377 27 Telefonica del Peru S.A.A., PE 28573 1000 2221 197 CLARO S.A., BR 7738 994 1882 40 Telemar Norte Leste S.A., BR 3816 974 471 195 COLOMBIA TELECOMUNICACIONES S.A. ESP, C 11172 906 125 76 Alestra, S. de R.L. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-LACNIC AfriNIC Region per AS prefix count summary ------------------------------------------ ASN No of nets /20 equiv MaxAgg Description 24863 1184 402 50 LINKdotNET-AS, EG 36903 682 343 125 MT-MPLS, MA 37611 665 67 6 Afrihost, ZA 36992 565 1373 25 ETISALAT-MISR, EG 8452 518 1471 15 TE-AS TE-AS, EG 37492 384 250 69 ORANGE-TN, TN 24835 350 611 17 RAYA-AS, EG 29571 332 37 12 CITelecom-AS, CI 15399 302 35 7 WANANCHI-KE, KE 2018 269 327 74 TENET-1, ZA Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-AFRINIC Global Per AS prefix count summary ---------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5559 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3532 382 249 TPG-INTERNET-AP TPG Telecom Limited, AU 22773 3526 2964 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3497 540 183 Telmex Colombia S.A., CO 39891 3329 169 15 ALJAWWALSTC-AS , SA 17974 2953 904 78 TELKOMNET-AS2-AP PT Telekomunikasi Indo 4766 2815 11131 853 KIXS-AS-KR Korea Telecom, KR 20940 2805 1054 2007 AKAMAI-ASN1 , US 9829 2699 1497 532 BSNL-NIB National Internet Backbone, IN 8151 2265 3353 556 Uninet S.A. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet Global Per AS Maximum Aggr summary ---------------------------------- ASN No of nets Net Savings Description 22773 3526 3380 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3497 3314 Telmex Colombia S.A., CO 39891 3329 3314 ALJAWWALSTC-AS , SA 7545 3532 3283 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2953 2875 TELKOMNET-AS2-AP PT Telekomunikasi Indo 6389 2218 2177 BELLSOUTH-NET-BLK - BellSouth.net Inc., 9829 2699 2167 BSNL-NIB National Internet Backbone, IN 9808 2185 2143 CMNET-GD Guangdong Mobile Communication 18566 2196 2086 MEGAPATH5-US - MegaPath Corporation, US 4766 2815 1962 KIXS-AS-KR Korea Telecom, KR Complete listing at http://thyme.rand.apnic.net/current/data-CIDRnet List of Unregistered Origin ASNs (Global) ----------------------------------------- Bad AS Designation Network Transit AS Description 65001 PRIVATE 5.143.176.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.200.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.208.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.216.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65000 PRIVATE 31.219.177.0/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65000 PRIVATE 31.219.177.128/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65512 PRIVATE 45.252.244.0/24 45899 VNPT-AS-VN VNPT Corp, VN 65512 PRIVATE 45.252.245.0/24 45899 VNPT-AS-VN VNPT Corp, VN Complete listing at http://thyme.rand.apnic.net/current/data-badAS Advertised Unallocated Addresses -------------------------------- Network Origin AS Description 23.249.144.0/20 40430 COLO4JAX-AS - colo4jax, LLC, US 27.100.7.0/24 56096 UNKNOWN 41.73.1.0/24 37004 -Reserved AS-, ZZ 41.73.2.0/24 37004 -Reserved AS-, ZZ 41.73.3.0/24 37004 -Reserved AS-, ZZ 41.73.4.0/24 37004 -Reserved AS-, ZZ 41.73.5.0/24 37004 -Reserved AS-, ZZ 41.73.6.0/24 37004 -Reserved AS-, ZZ 41.73.7.0/24 37004 -Reserved AS-, ZZ 41.73.8.0/24 37004 -Reserved AS-, ZZ Complete listing at http://thyme.rand.apnic.net/current/data-add-IANA Number of prefixes announced per prefix length (Global) ------------------------------------------------------- /1:0 /2:0 /3:0 /4:0 /5:0 /6:0 /7:0 /8:16 /9:13 /10:36 /11:103 /12:273 /13:523 /14:1052 /15:1775 /16:13173 /17:7840 /18:13061 /19:25397 /20:38800 /21:40321 /22:68217 /23:59789 /24:341400 /25:473 /26:397 /27:303 /28:57 /29:32 /30:13 /31:1 /32:33 Advertised prefixes smaller than registry allocations ----------------------------------------------------- ASN No of nets Total ann. Description 39891 2896 3329 ALJAWWALSTC-AS , SA 22773 2747 3526 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2088 2196 MEGAPATH5-US - MegaPath Corporation, US 30036 1561 1746 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 10620 1422 3497 Telmex Colombia S.A., CO 6389 1416 2218 BELLSOUTH-NET-BLK - BellSouth.net Inc., 6983 1325 1677 ITCDELTA - Earthlink, Inc., US 34984 1262 1977 TELLCOM-AS , TR 11492 1211 1303 CABLEONE - CABLE ONE, INC., US 13188 1042 1267 BANKINFORM-AS , UA Complete listing at http://thyme.rand.apnic.net/current/data-sXXas-nos Number of /24s announced per /8 block (Global) ---------------------------------------------- 1:1670 2:786 4:23 5:2196 6:31 8:992 12:1800 13:44 14:1743 15:46 16:2 17:99 18:126 20:50 22:1 23:1869 24:1809 27:2322 31:1790 32:83 33:4 35:4 36:326 37:2321 38:1260 39:36 40:102 41:2970 42:453 43:1862 44:45 45:2150 46:2544 47:98 49:1178 50:911 51:13 52:553 53:2 54:350 55:6 56:4 57:41 58:1568 59:1018 60:376 61:1826 62:1510 63:1917 64:4581 65:2175 66:4271 67:2216 68:1131 69:3267 70:1284 71:557 72:2080 74:2583 75:402 76:415 77:1404 78:1217 79:925 80:1307 81:1406 82:988 83:705 84:872 85:1575 86:482 87:1134 88:553 89:2095 90:222 91:6104 92:918 93:2369 94:2418 95:2496 96:578 97:344 98:958 99:89 100:147 101:1157 103:12506 104:2566 105:126 106:441 107:1435 108:780 109:2277 110:1278 111:1638 112:1069 113:1145 114:1084 115:1688 116:1678 117:1569 118:2050 119:1578 120:937 121:1122 122:2268 123:2035 124:1569 125:1827 128:693 129:438 130:419 131:1379 132:607 133:175 134:488 135:198 136:386 137:397 138:1781 139:431 140:601 141:447 142:668 143:957 144:726 145:165 146:956 147:653 148:1392 149:540 150:659 151:899 152:673 153:301 154:693 155:950 156:529 157:533 158:389 159:1207 160:517 161:735 162:2408 163:570 164:775 165:1120 166:336 167:1188 168:2253 169:658 170:1992 171:252 172:748 173:1738 174:767 175:671 176:1727 177:4038 178:2244 179:1174 180:2148 181:1918 182:2080 183:1006 184:835 185:7591 186:3201 187:2149 188:2208 189:1757 190:7832 191:1302 192:9040 193:5745 194:4461 195:3849 196:1784 197:1214 198:5589 199:5757 200:7176 201:3705 202:10120 203:9838 204:4513 205:2735 206:2999 207:3075 208:4069 209:3872 210:3887 211:2045 212:2731 213:2347 214:861 215:68 216:5704 217:1989 218:804 219:611 220:1634 221:880 222:698 223:1311 End of report From sean at donelan.com Fri Oct 7 20:10:36 2016 From: sean at donelan.com (Sean Donelan) Date: Fri, 7 Oct 2016 16:10:36 -0400 (EDT) Subject: FCC: DIRS activated for providers in Florida, Georgia, South Carolina Message-ID: The U.S. Federal Communications Commission has activated its Disaster Information Reporting System (DIRS) in response to Hurricane Matthew. DIRS is a voluntary, web-based system that communications providers, including wireless, wireline, broadcast, cable and Voice over Internet Protocol providers, can use to report communications infrastructure status and situational awareness information during times of crisis. http://transition.fcc.gov/Daily_Releases/Daily_Business/2016/db1007/DA-16-1160A1.pdf From hank at efes.iucc.ac.il Sat Oct 8 17:15:42 2016 From: hank at efes.iucc.ac.il (Hank Nussbacher) Date: Sat, 8 Oct 2016 20:15:42 +0300 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: <8f16b55b-8be1-020f-aaa8-78c6726a6934@efes.iucc.ac.il> On 07/10/2016 17:59, Lee wrote: > On 10/7/16, Hank Nussbacher wrote: >> On 07/10/2016 00:33, Lee wrote: >>> dunno about creating web pages, but >>> https://www.nanog.org/meetings/abstract?id=785 >>> has a section on showing filters that are defined but not referenced & >>> referenced but not defined >> In IOS-XR it is one command "sho rpl unused ?" >> RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused ? >> as-path-set Display as-path-set objects >> community-set Display community-set objects >> extcommunity-set Display extended community objects >> prefix-set Display prefix-set objects >> rd-set Display rd-set objects >> route-policy Display route-policy objects >> tag-set Display tag-set objects >> >> RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused prefix >> Fri Oct 7 08:24:53.237 IDT >> >> ACTIVE -- Referenced by at least one policy which is attached >> INACTIVE -- Only referenced by policies which are not attached >> UNUSED -- Not attached (directly or indirectly) and not referenced > I'm actually starting to miss being out of the game. I'm retired, so > don't have access to anything running IOS-XR. Just out of curiosity, > how does the output of 'show rpl unused prefix' compare to the output > of the script at http://pastebin.com/pem7tHAJ > > Thanks, > Lee > Samples: RP/0/RSP0/CPU0:petach-tikva-gp#sho rpl unused as-path Sat Oct 8 20:03:22.975 IDT ACTIVE -- Referenced by at least one policy which is attached INACTIVE -- Only referenced by policies which are not attached UNUSED -- Not attached (directly or indirectly) and not referenced The following as-path-sets are UNUSED ------------------------------------------ aspath_191_p1_permit P/0/RSP0/CPU0:petach-tikva-gp#sho rpl unused prefix Sat Oct 8 20:03:56.826 IDT ACTIVE -- Referenced by at least one policy which is attached INACTIVE -- Only referenced by policies which are not attached UNUSED -- Not attached (directly or indirectly) and not referenced The following prefix-sets are UNUSED ------------------------------------------ aspath_191_permit RP/0/RSP0/CPU0:petach-tikva-gp#sho rpl unused comm Sat Oct 8 20:04:20.953 IDT ACTIVE -- Referenced by at least one policy which is attached INACTIVE -- Only referenced by policies which are not attached UNUSED -- Not attached (directly or indirectly) and not referenced The following community-sets are UNUSED ------------------------------------------ 378:3300 378:65379 P/0/RSP0/CPU0:petach-tikva-gp#sho rpl unused rout Sat Oct 8 20:05:22.857 IDT ACTIVE -- Referenced by at least one policy which is attached INACTIVE -- Only referenced by policies which are not attached UNUSED -- Not attached (directly or indirectly) and not referenced The following policies are (UNUSED) ------------------------------------------ GEANT-QoS tagIIXroutes Note the sloppy code - sometimes they state UNUSED and sometimes (UNUSED). Or "the following policies are"... rather than "the following routing policies are". Just plain sloppy Cisco coding and poor QA. And once you delete these unreferenced objects, "show rpl unused" will still show them since there is a bug in Cisco code (CSCuy07932/CSCug9153). See: http://www.gossamer-threads.com/lists/cisco/nsp/192481 for details. -Hank From fw at deneb.enyo.de Sun Oct 9 11:20:00 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 09 Oct 2016 13:20:00 +0200 Subject: Krebs on Security booted off Akamai network after DDoS attack proves pricey In-Reply-To: (Eliot Lear's message of "Tue, 27 Sep 2016 13:52:06 +0200") References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> Message-ID: <87r37phiqn.fsf@mid.deneb.enyo.de> * Eliot Lear: > Not my end goal. My end goal is that consumers have a means to limit > risk in their home environments, and service providers have a means to > deliver that to them. They already have, with today's technology. It's just not a mass-market business. Consumers either have to educate themselves (which is not that hard), and service providers need to provide actual service, instead charging a fee for access to a computer system. There is little interest in this, however. There's a comparable business case for providing managed PCs to consumers, and I'm not sure if any such companies are still left. >> I'm not convinced that expected traffic profiles are the right answer. >> We already have that in the server hosting market, and it does >> constraint the types of services you can run on hosted servers (for >> the hosting providers who does this). I'm wary of the network putting >> severe constraints on application architecture, way beyond what is >> dictated by current technology. NAT more or less killed servers on >> consumer networks, and this kind of traffic profiling has begun to >> kill clients on server networks. > > The whole point of MUD is to leave control in the hands of those who > have developed and have to support Things. It is not simply for the SP > to decide what traffic is ok, or to charge more for it, but to respect > the wishes of the developers. That may be sufficient to stop a lot of > bad things from happening to a lot of Things. Nobody respects what developers want, otherwise we wouldn't have any shipping products at all. What I'm trying to say: Cutting corners is more often a non-development decision. If you can ship today without any security, or at some unknowable date in the future, with additional security features whose impact may not matter, things usually head for the earlier shipping date. I used to be frustrated by such decisions, but over the past few years, I've come to realize that most of us have so little data on the effectiveness of security features that mandates for them are essentially arbitrary. > And again, this is the wrong way to look at it. The consumer should > always get final say. They're the customer. This is a chance for the > manufacturer of the device they're using to explain how the device is > supposed to behave on the network. If we want to make consumers to make informed decisions, they need to learn how things work up to a certain level. And then current technology already works. (Sorry that I'm not inclined to read upon the specs?I do wonder how this an improvement over UPnP.) From johnl at iecc.com Sun Oct 9 14:02:50 2016 From: johnl at iecc.com (John R. Levine) Date: 9 Oct 2016 10:02:50 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <87r37phiqn.fsf@mid.deneb.enyo.de> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> Message-ID: On Sun, 9 Oct 2016, Florian Weimer wrote: > If we want to make consumers to make informed decisions, they need to > learn how things work up to a certain level. And then current > technology already works. I think it's fair to say that security through consumer education has been a failure every time anyone has tried it. Why do you think this would be any different? > There is little interest in this, however. There's a comparable > business case for providing managed PCs to consumers, and I'm not sure > if any such companies are still left. There's at least two large ones: Microsoft and Apple. Try installing Windows 10 without letting Microsoft update and reconfigure the software any time they want, any way they want. Expecting consumers to evaluate the security behavior of their lightbulbs and their refrigerator is absurd. We need to figure out how to have the devices and routers configure themselves so the devices can do what they need to do without doing what we really don't want them to do. Regards, John Levine, johnl at iecc.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly From mel at beckman.org Sun Oct 9 14:31:54 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 14:31:54 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de>, Message-ID: <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> I just bought a $20 Lacrosse remote RF temperature sensor hub for home, the GW-1000U. It does the usual IoT things: after you plug it in, it gets a DHCP address and phones home, then you register it using a smartphone on the same LAN, which I'm guessing finds the device via a broadcast and then configures the hub with my Lacrosse account info. All communication is thereafter through the cloud. It set itself up quite conveniently and efficiently, and now will start charging me $12/year for alerts and texts. An acceptable business model. Except the thing is a teaming mass of security vulnerabilities. How much authentication went on in this process? None. I captured the thing's packets in my firewall's onboard sniffer from the get go. All data is exchanged as plaintext on port 80. The protocol is completely undocumented, but I've since discovered that at least one enterprising tinkerer has reverse engineered it so people can bypass the manufacturer's monetization model. The device accepts TCP connections on 22, 80, and 443. Theoretically I can't see why it ever needs ongoing inbound connections, so this seems to be a security concession made by the maker. Also, it appears to support SSL, but uses plaintext. Why? Because it's easier to debug in the early deployments, I'll wager. But the thing has been out for years and they're still not using encryption, even though the device apparently has the ability. As a knowledgable consumer (and security researcher) I'll overcome these shortcomings by putting this device on its own VLAN with extensive firewalling. Still, I can't be sure it won't be malicious, or get exploited through the cloud. And VLANs have their own security weaknesses, despite my using pricey enterprise hardware at home. My point is that if an expert has to expend several hours of highly technical labor to "responsibly" use a $20 IoT sensor, and use enterprise-grade IT gear and methods to gain even a modicum of safety, then what hope do Ma and Pa Kettle have? This is not a consumer education problem, unless we think consumers should also learn thermodynamics in order to drive, the Bernoulli principle in order to be airline passengers, and biochemistry to cook their own food. It's clearly a giant screw-up by manufacturers who could easily spread the cost of best-practice security measures across a large customer base. That they don't shows lack of moral character, and nothing else. -mel beckman > On Oct 9, 2016, at 7:03 AM, John R. Levine wrote: > >> On Sun, 9 Oct 2016, Florian Weimer wrote: >> >> If we want to make consumers to make informed decisions, they need to >> learn how things work up to a certain level. And then current >> technology already works. > > I think it's fair to say that security through consumer education has been a failure every time anyone has tried it. Why do you think this would be any different? > >> There is little interest in this, however. There's a comparable >> business case for providing managed PCs to consumers, and I'm not sure >> if any such companies are still left. > > There's at least two large ones: Microsoft and Apple. Try installing Windows 10 without letting Microsoft update and reconfigure the software any time they want, any way they want. > > Expecting consumers to evaluate the security behavior of their lightbulbs and their refrigerator is absurd. We need to figure out how to have the devices and routers configure themselves so the devices can do what they need to do without doing what we really don't want them to do. > > Regards, > John Levine, johnl at iecc.com, Primary Perpetrator of "The Internet for Dummies", > Please consider the environment before reading this e-mail. https://jl.ly From list at satchell.net Sun Oct 9 15:33:19 2016 From: list at satchell.net (Stephen Satchell) Date: Sun, 9 Oct 2016 08:33:19 -0700 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> Message-ID: <98b356a2-5406-b264-73e5-ecfe5c7697f3@satchell.net> On 10/09/2016 07:31 AM, Mel Beckman wrote: > remote RF temperature sensor hub for home, the GW-1000U. > ... > The device accepts TCP connections on 22, 80, and 443. Theoretically > I can't see why it ever needs ongoing inbound connections, so this > seems to be a security concession made by the maker. Also, it appears > to support SSL, but uses plaintext. Why? Because it's easier to debug > in the early deployments, I'll wager. But the thing has been out for > years and they're still not using encryption, even though the device > apparently has the ability. I could see one reason, and one reason only: to allow the customer to use a "control panel" with a local computer, smartphone app, or tablet app to set capabilities, options, and preferences. That said, the manufacturer probably thought that the sensor would be shielded from the Internet by a Wireless Access Point with NAT, so that there would be no direct exposure (in theory) to inbound connections from the outside world. For IPv4, this is barely tolerable. For IPv6, not so much. As a developer, I can tell you that "easier to debug in the early deployments" means that the later deployments won't be locked down until the manufacturer gets a fine, judgement, or other monetary hit. Would you put this thing on a DMZ? I thought not... :) From mel at beckman.org Sun Oct 9 15:46:50 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 15:46:50 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <98b356a2-5406-b264-73e5-ecfe5c7697f3@satchell.net> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> <98b356a2-5406-b264-73e5-ecfe5c7697f3@satchell.net> Message-ID: <436B6095-ED50-4DD1-AF27-DBB9168FBFA4@beckman.org> Stephen, But they don?t, in fact, allow such a console. And I don?t think such a thing is even a good idea on IoT devices, because permitting inbound connections is a pathway to exploitation. As I noted in my post, I?ve put it on its own VLAN, which is better than a DMZ: no inbound access at all, and no access to any other network or devices. I only permit port 80 outbound to the Lacrosse cloud server, and will get notified of any other traffic. But this is a wired device, which made it easier to sequester. If it were WiFi my task would have been much harder, and most IoT devices do seem to be WiFi. -mel > On Oct 9, 2016, at 8:33 AM, Stephen Satchell wrote: > > On 10/09/2016 07:31 AM, Mel Beckman wrote: >> remote RF temperature sensor hub for home, the GW-1000U. >> ... >> The device accepts TCP connections on 22, 80, and 443. Theoretically >> I can't see why it ever needs ongoing inbound connections, so this >> seems to be a security concession made by the maker. Also, it appears >> to support SSL, but uses plaintext. Why? Because it's easier to debug >> in the early deployments, I'll wager. But the thing has been out for >> years and they're still not using encryption, even though the device >> apparently has the ability. > > I could see one reason, and one reason only: to allow the customer to > use a "control panel" with a local computer, smartphone app, or tablet > app to set capabilities, options, and preferences. That said, the > manufacturer probably thought that the sensor would be shielded from the > Internet by a Wireless Access Point with NAT, so that there would be no > direct exposure (in theory) to inbound connections from the outside world. > > For IPv4, this is barely tolerable. For IPv6, not so much. > > As a developer, I can tell you that "easier to debug in the early > deployments" means that the later deployments won't be locked down until > the manufacturer gets a fine, judgement, or other monetary hit. > > Would you put this thing on a DMZ? I thought not... :) From Valdis.Kletnieks at vt.edu Sun Oct 9 17:28:40 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Sun, 09 Oct 2016 13:28:40 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de>, <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> Message-ID: <218440.1476034120@turing-police.cc.vt.edu> On Sun, 09 Oct 2016 14:31:54 -0000, Mel Beckman said: > I just bought a $20 Lacrosse remote RF temperature sensor hub for home, the > GW-1000U. It does the usual IoT things: after you plug it in, it gets a DHCP > address and phones home, then you register it using a smartphone on the same > LAN, which I'm guessing finds the device via a broadcast and then configures > the hub with my Lacrosse account info. All communication is thereafter through > the cloud. That last sentence is sub-optimal. And as you note, things go downhill from there.... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From mel at beckman.org Sun Oct 9 18:05:20 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 18:05:20 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <218440.1476034120@turing-police.cc.vt.edu> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de>, <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org>, <218440.1476034120@turing-police.cc.vt.edu> Message-ID: I don't know why it's "sub optimal" to use the cloud from an isolated network. Can you elaborate? -mel beckman > On Oct 9, 2016, at 10:28 AM, "Valdis.Kletnieks at vt.edu" wrote: > > On Sun, 09 Oct 2016 14:31:54 -0000, Mel Beckman said: > >> I just bought a $20 Lacrosse remote RF temperature sensor hub for home, the >> GW-1000U. It does the usual IoT things: after you plug it in, it gets a DHCP >> address and phones home, then you register it using a smartphone on the same >> LAN, which I'm guessing finds the device via a broadcast and then configures >> the hub with my Lacrosse account info. All communication is thereafter through >> the cloud. > > That last sentence is sub-optimal. And as you note, things go downhill from > there.... From Valdis.Kletnieks at vt.edu Sun Oct 9 18:30:37 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Sun, 09 Oct 2016 14:30:37 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de>, <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org>, <218440.1476034120@turing-police.cc.vt.edu> Message-ID: <227147.1476037837@turing-police.cc.vt.edu> On Sun, 09 Oct 2016 18:05:20 -0000, Mel Beckman said: > I don't know why it's "sub optimal" to use the cloud from an isolated network. Can you elaborate? Why should something out in the cloud have any part of the communication, other than perhaps telling your cellphone the current address of your widget? (And *that* should probably have a standardized protocol/service rather than every vendor rolling their own. Hello, IETF?) And even *that* can be bypassed if you cellphone is able to talk to your home network directly. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From nanog at shankland.org Sun Oct 9 18:44:58 2016 From: nanog at shankland.org (Jim Shankland) Date: Sun, 9 Oct 2016 11:44:58 -0700 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <227147.1476037837@turing-police.cc.vt.edu> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> <218440.1476034120@turing-police.cc.vt.edu> <227147.1476037837@turing-police.cc.vt.edu> Message-ID: <3849634b-6bd1-6e5a-2c38-6511544662c7@shankland.org> On 10/9/16 11:30 AM, Valdis.Kletnieks at vt.edu wrote: > On Sun, 09 Oct 2016 18:05:20 -0000, Mel Beckman said: >> I don't know why it's "sub optimal" to use the cloud from an isolated network. Can you elaborate? > Why should something out in the cloud have any part of the communication, > other than perhaps telling your cellphone the current address of your widget? > > (And *that* should probably have a standardized protocol/service rather than > every vendor rolling their own. Hello, IETF?) > > And even *that* can be bypassed if you cellphone is able to talk to your > home network directly. A fair question, if it's intended rhetorically. If not, then the short answer is: because monetizing your personal information is a large (possibly dominant) part of the IoT business model, today. Rampant security holes don't necessarily perturb that model excessively -- though goodness knows they should, and maybe eventually they will. In the meantime, we have what Zeynep Tufekci has called the "Internet of Hacked Things" (anybody want to help get the acronym IoHT into general currency?). Jim From mel at beckman.org Sun Oct 9 18:48:17 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 18:48:17 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <227147.1476037837@turing-police.cc.vt.edu> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de>, <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org>, <218440.1476034120@turing-police.cc.vt.edu> , <227147.1476037837@turing-police.cc.vt.edu> Message-ID: The idea behind IoT is that devices collect data, but the power to process that data, and archive it, is in the cloud. -mel beckman > On Oct 9, 2016, at 11:30 AM, "Valdis.Kletnieks at vt.edu" wrote: > > On Sun, 09 Oct 2016 18:05:20 -0000, Mel Beckman said: >> I don't know why it's "sub optimal" to use the cloud from an isolated network. Can you elaborate? > > Why should something out in the cloud have any part of the communication, > other than perhaps telling your cellphone the current address of your widget? > > (And *that* should probably have a standardized protocol/service rather than > every vendor rolling their own. Hello, IETF?) > > And even *that* can be bypassed if you cellphone is able to talk to your > home network directly. From fw at deneb.enyo.de Sun Oct 9 19:01:47 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 09 Oct 2016 21:01:47 +0200 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: (John R. Levine's message of "9 Oct 2016 10:02:50 -0400") References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> Message-ID: <87fuo5e484.fsf@mid.deneb.enyo.de> * John R. Levine: > On Sun, 9 Oct 2016, Florian Weimer wrote: > >> If we want to make consumers to make informed decisions, they need to >> learn how things work up to a certain level. And then current >> technology already works. > > I think it's fair to say that security through consumer education has > been a failure every time anyone has tried it. Why do you think this > would be any different? People start to care once they have to. Currently, there is not much reason to worry about which devices you connect to your home network. Even the interaction with Internet banking appears to be benign these days. If your Internet connection goes down because something starts spewing packets, you can probably find it by disconnecting everything until you have found the culprit. It's probably not much different from how you find which device triggers the breaker. Anything that's more proactive requires some level of knowledge, and if we assume that it cannot be dispersed to consumers, then it means someone else gets to manage their home networks. And I'm not sure if the ISPs should be doing this (or if they want any part in this, maybe except if it enables them to charge per connected device and function). >> There is little interest in this, however. There's a comparable >> business case for providing managed PCs to consumers, and I'm not sure >> if any such companies are still left. > > There's at least two large ones: Microsoft and Apple. Try installing > Windows 10 without letting Microsoft update and reconfigure the > software any time they want, any way they want. I don't think I can phone Microsoft if something goes wrong. In most countries, they even disclaim responsiblity for breakage introduced by updates and point to the PC makers instead (from whom most consumers baught their OEM version). Apple may be different. > Expecting consumers to evaluate the security behavior of their > lightbulbs and their refrigerator is absurd. We need to figure out > how to have the devices and routers configure themselves so the > devices can do what they need to do without doing what we really don't > want them to do. We already have UPnP. Clearly, it does not work, but it's not obvious to me why any different solution would end up as being just as ineffective. From large.hadron.collider at gmx.com Sun Oct 9 19:50:21 2016 From: large.hadron.collider at gmx.com (Large Hadron Collider) Date: Sun, 9 Oct 2016 12:50:21 -0700 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <98b356a2-5406-b264-73e5-ecfe5c7697f3@satchell.net> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <3D9FCEDE-806F-40D4-BA39-4EC5CA211E17@beckman.org> <98b356a2-5406-b264-73e5-ecfe5c7697f3@satchell.net> Message-ID: On 2016-10-09 08:33 AM, Stephen Satchell wrote: > On 10/09/2016 07:31 AM, Mel Beckman wrote: >> remote RF temperature sensor hub for home, the GW-1000U. >> ... >> The device accepts TCP connections on 22, 80, and 443. Theoretically >> I can't see why it ever needs ongoing inbound connections, so this >> seems to be a security concession made by the maker. Also, it appears >> to support SSL, but uses plaintext. Why? Because it's easier to debug >> in the early deployments, I'll wager. But the thing has been out for >> years and they're still not using encryption, even though the device >> apparently has the ability. > I could see one reason, and one reason only: to allow the customer to > use a "control panel" with a local computer, smartphone app, or tablet > app to set capabilities, options, and preferences. That said, the > manufacturer probably thought that the sensor would be shielded from the > Internet by a Wireless Access Point with NAT, so that there would be no > direct exposure (in theory) to inbound connections from the outside world. > > For IPv4, this is barely tolerable. For IPv6, not so much. For v6, what I'd do is firewall all but the safest (SIP, RTP basically) of out-of-local-network(s) inbounds to the device unless you visit an intranet webpage from the device that allows you to open all inbound. The page would be a one time deal (would survive across reinstalls as long as the router remembers you) and would record your MAC address. It would ask "You hereby agree that your device's connection security is your responsibility and only your responsibility. You hereby indemnify and hold harmless the owner of the network infrastructure for [bla de bla legal jargon basically don't sue if yer hakt]. Would you like to open blocked inbound connections? [Yes / Oui / ??] [No / Non / ???]" > > As a developer, I can tell you that "easier to debug in the early > deployments" means that the later deployments won't be locked down until > the manufacturer gets a fine, judgement, or other monetary hit. > > Would you put this thing on a DMZ? I thought not... :) I wouldn't even put a well-secured desktop running all the best firewalling in a TNZ (trusted network zone, term I think is less misleading than DMZ, referring to a state of being unfirewalled) From large.hadron.collider at gmx.com Sun Oct 9 19:50:49 2016 From: large.hadron.collider at gmx.com (Large Hadron Collider) Date: Sun, 9 Oct 2016 12:50:49 -0700 Subject: Krebs on Security booted off Akamai network after DDoS attack proves pricey In-Reply-To: <28934fb6-83e6-ae03-fa5b-80c48ab4177f@gmx.com> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <28934fb6-83e6-ae03-fa5b-80c48ab4177f@gmx.com> Message-ID: Sorry florian. Meant to put it to list. On 2016-10-09 12:25 PM, Large Hadron Collider wrote: > > > On 2016-10-09 04:20 AM, Florian Weimer wrote: >> * Eliot Lear: >> >>> Not my end goal. My end goal is that consumers have a means to limit >>> risk in their home environments, and service providers have a means to >>> deliver that to them. >> They already have, with today's technology. It's just not a >> mass-market business. Consumers either have to educate themselves >> (which is not that hard), and service providers need to provide actual >> service, instead charging a fee for access to a computer system. >> >> There is little interest in this, however. There's a comparable >> business case for providing managed PCs to consumers, and I'm not sure >> if any such companies are still left. > I'd wager that after the Indian tech support fucks, they've went like > "too risky" > > But yeah there's a good case. If I had it in me I'd hire a bunch of > people to manage consumers' managed PCs. >> >>>> I'm not convinced that expected traffic profiles are the right answer. >>>> We already have that in the server hosting market, and it does >>>> constraint the types of services you can run on hosted servers (for >>>> the hosting providers who does this). I'm wary of the network putting >>>> severe constraints on application architecture, way beyond what is >>>> dictated by current technology. NAT more or less killed servers on >>>> consumer networks, and this kind of traffic profiling has begun to >>>> kill clients on server networks. >>> The whole point of MUD is to leave control in the hands of those who >>> have developed and have to support Things. It is not simply for the SP >>> to decide what traffic is ok, or to charge more for it, but to respect >>> the wishes of the developers. That may be sufficient to stop a lot of >>> bad things from happening to a lot of Things. >> Nobody respects what developers want, otherwise we wouldn't have any >> shipping products at all. >> >> What I'm trying to say: Cutting corners is more often a >> non-development decision. If you can ship today without any security, >> or at some unknowable date in the future, with additional security >> features whose impact may not matter, things usually head for the >> earlier shipping date. >> >> I used to be frustrated by such decisions, but over the past few >> years, I've come to realize that most of us have so little data on the >> effectiveness of security features that mandates for them are >> essentially arbitrary. >> >>> And again, this is the wrong way to look at it. The consumer should >>> always get final say. They're the customer. This is a chance for the >>> manufacturer of the device they're using to explain how the device is >>> supposed to behave on the network. >> If we want to make consumers to make informed decisions, they need to >> learn how things work up to a certain level. And then current >> technology already works. >> >> (Sorry that I'm not inclined to read upon the specs?I do wonder how >> this an improvement over UPnP.) > From bzs at TheWorld.com Sun Oct 9 20:01:10 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 9 Oct 2016 16:01:10 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> Message-ID: <22522.41478.59506.907676@gargle.gargle.HOWL> Elsewhere, for decades, I've bemoaned the fact that keyboards (etc) don't have credit card swipes (perhaps today "and chip readers") so with some care on the part of the software someone could prove they likely have physical access to the card. But it would be very useful in this IoT problem. You power up a new device, it won't enable until you run some web (e.g.) interface. At that point you swipe a card which generates a hash which secures the IoT device from further config until it's presented again. The device can have the usual reset to factory config button for the case of lost cards. It needn't even be an active credit card. It could be an old spent gift card. It could even be a free card that comes right in the box tho that might invite predictability, but maybe a basket of cards to use at the checkout counter "take one you'll need it for setup". The software just has to be able to read the magstripe or chip and use the info to generate a reasonably secure hash which is stored (preferably in the device.) Need to reconfig, open the window, swipe the same card. Hotel safes often use this approach as an alternative to PIN entry. The device doesn't store any info about the card directly, only the hash. And as I said it could be most anything that looks like a credit card and has a readable mag stripe. The user doesn't have to come up with a password and can't use the device until a hash is stored. But, alas, no swipes... -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From mel at beckman.org Sun Oct 9 20:07:48 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 20:07:48 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <22522.41478.59506.907676@gargle.gargle.HOWL> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> , <22522.41478.59506.907676@gargle.gargle.HOWL> Message-ID: <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> Barry, The problem isn't authentication during initial installation, since that can be done using SSL and a web login to the cloud service. The problem is that vendors aren't even using minimal security protections, such as SSL, and then leaving devices open to inbound connections, which is bad even behind a firewall (because viruses typically scan LANs for these vulnerable devices). These are the devices exploited by hackers to become DDoS attack vectors. -mel beckman > On Oct 9, 2016, at 1:02 PM, "bzs at TheWorld.com" wrote: > > > Elsewhere, for decades, I've bemoaned the fact that keyboards (etc) > don't have credit card swipes (perhaps today "and chip readers") so > with some care on the part of the software someone could prove they > likely have physical access to the card. > > But it would be very useful in this IoT problem. > > You power up a new device, it won't enable until you run some web > (e.g.) interface. > > At that point you swipe a card which generates a hash which secures > the IoT device from further config until it's presented again. The > device can have the usual reset to factory config button for the case > of lost cards. > > It needn't even be an active credit card. It could be an old spent > gift card. It could even be a free card that comes right in the box > tho that might invite predictability, but maybe a basket of cards to > use at the checkout counter "take one you'll need it for setup". > > The software just has to be able to read the magstripe or chip and use > the info to generate a reasonably secure hash which is stored > (preferably in the device.) > > Need to reconfig, open the window, swipe the same card. > > Hotel safes often use this approach as an alternative to PIN entry. > > The device doesn't store any info about the card directly, only the > hash. And as I said it could be most anything that looks like a credit > card and has a readable mag stripe. > > The user doesn't have to come up with a password and can't use the > device until a hash is stored. > > But, alas, no swipes... > > -- > -Barry Shein > > Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com > Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD > The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Sun Oct 9 20:19:07 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 9 Oct 2016 16:19:07 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <22522.41478.59506.907676@gargle.gargle.HOWL> <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> Message-ID: <22522.42555.121770.918579@gargle.gargle.HOWL> On October 9, 2016 at 20:07 mel at beckman.org (Mel Beckman) wrote: > Barry, > > The problem isn't authentication during initial installation, since that can be done using SSL and a web login to the cloud service. The problem is that vendors aren't even using minimal security protections, such as SSL, and then leaving devices open to inbound connections, which is bad even behind a firewall (because viruses typically scan LANs for these vulnerable devices). These are the devices exploited by hackers to become DDoS attack vectors. It helps solve the bad (including manufacturer's default) password problem which was one of the attack vectors. The proposal only forces this to be used during initial installation and configuration (and any reconfig) arguing that it so lowers the threshold, just swipe a magstripe, there's really no excuse. And eliminates the owner choosing a password for the device, bad or otherwise. But, again, alas no swipe hardware. Big historical error I think but rectifying is feasible. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From mel at beckman.org Sun Oct 9 20:24:11 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 9 Oct 2016 20:24:11 +0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <22522.42555.121770.918579@gargle.gargle.HOWL> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <22522.41478.59506.907676@gargle.gargle.HOWL> <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org>, <22522.42555.121770.918579@gargle.gargle.HOWL> Message-ID: <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> You might as well wish for fingerprint readers. It's not going to happen, and thus can't be remedied. But there are already acceptable COTS solutions that need no special hardware. IoT vendors simply have to use them. -mel beckman > On Oct 9, 2016, at 1:20 PM, "bzs at TheWorld.com" wrote: > > >> On October 9, 2016 at 20:07 mel at beckman.org (Mel Beckman) wrote: >> Barry, >> >> The problem isn't authentication during initial installation, since that can be done using SSL and a web login to the cloud service. The problem is that vendors aren't even using minimal security protections, such as SSL, and then leaving devices open to inbound connections, which is bad even behind a firewall (because viruses typically scan LANs for these vulnerable devices). These are the devices exploited by hackers to become DDoS attack vectors. > > It helps solve the bad (including manufacturer's default) password > problem which was one of the attack vectors. > > The proposal only forces this to be used during initial installation > and configuration (and any reconfig) arguing that it so lowers the > threshold, just swipe a magstripe, there's really no excuse. And > eliminates the owner choosing a password for the device, bad or > otherwise. > > But, again, alas no swipe hardware. Big historical error I think but > rectifying is feasible. > > -- > -Barry Shein > > Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com > Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD > The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Sun Oct 9 20:47:30 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 9 Oct 2016 16:47:30 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> References: <20160926155649.14061.qmail@ary.lan> <20160926230946.685605514EDF@rock.dv.isc.org> <03DC1038-024A-4D9F-AC5B-3E88CDF56246@cable.comcast.com> <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <22522.41478.59506.907676@gargle.gargle.HOWL> <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> <22522.42555.121770.918579@gargle.gargle.HOWL> <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> Message-ID: <22522.44258.206039.834117@gargle.gargle.HOWL> On October 9, 2016 at 20:24 mel at beckman.org (Mel Beckman) wrote: > You might as well wish for fingerprint readers. It's not going to happen, and thus can't be remedied. But there are already acceptable COTS solutions that need no special hardware. IoT vendors simply have to use them. Ok, let them go for that, or not. But I well remember proposed spam mitigations back in 2000 being just as forcefully shot down because IT WOULD TAKE A DECADE TO IMPLEMENT THAT!!! That was 16 years ago. It's a dumb sort of thing to waste people's time posting, whether it's attractive or not will stand on its own merits and not someone arguing that in their opinion (based on who knows what) market penetration is insurmountable. And many smart phones do have fingerprint readers and have for a few years now, so do quite a few keyboards. My trackpad on my 4 year old laptop has one (Aspire 8930G.) Your point is several years too late already. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From marka at isc.org Sun Oct 9 22:53:20 2016 From: marka at isc.org (Mark Andrews) Date: Mon, 10 Oct 2016 09:53:20 +1100 Subject: Anyone from GoDaddy here? Message-ID: <20161009225320.1832E5603D39@rock.dv.isc.org> Can you please explain why you do not acknowledge reports of your nameservers being broken. Can you please explain why you are are blocking EDNS 1 queries over IPv4 when your servers are clearly capable of handling them over IPv6? Can you please explain why your servers mishandle EDNS flags? What part of the following is unclear. You *send* replies. The flag bits should be empty in the replies even if they are set in the request. You obviously are not ignore them as you echo them back (IPv6) or drop the query (IPv4). This will make the flag bits less useful when they are finally defined as no one will be able to depend on the bits not being echoed. We already have a flag bit where we can't depend on its return state (AD) because too many server just echo it. We do not need this to happen to any other bits. Z Set to zero by senders and ignored by receivers, unless modified in a subsequent specification. Mark Checking: 'utahtrust.gov' as at 2016-10-09T22:37:30Z utahtrust.gov @216.69.185.50 (pdns01.domaincontrol.com.): edns=ok edns1=timeout edns at 512=ok ednsopt=ok edns1opt=timeout do=ok ednsflags=timeout docookie=ok edns at 512tcp=ok optlist=nsid,subnet utahtrust.gov @2607:f208:207::32 (pdns01.domaincontrol.com.): edns=ok edns1=ok edns at 512=ok ednsopt=ok edns1opt=ok do=ok ednsflags=mbz docookie=ok edns at 512tcp=ok optlist=nsid,subnet utahtrust.gov @208.109.255.50 (pdns02.domaincontrol.com.): edns=ok edns1=timeout edns at 512=ok ednsopt=ok edns1opt=timeout do=ok ednsflags=timeout docookie=ok edns at 512tcp=ok optlist=nsid,subnet utahtrust.gov @2607:f208:303::32 (pdns02.domaincontrol.com.): edns=ok edns1=ok edns at 512=ok ednsopt=ok edns1opt=ok do=ok ednsflags=mbz docookie=ok edns at 512tcp=ok optlist=nsid,subnet The Following Tests Failed Warning: test failures may indicate that some DNS clients cannot resolve the zone or will get a unintended answer or resolution will be slower than necessary. Warning: failure to address issues identified here may make future DNS extensions that you want to use ineffective. In particular echoing back unknown EDNS options and unknown EDNS flags will break future signaling between DNS client and DNS server. We already have examples of this were you cannot depend on the AD flag bit meaning anything in replies because too many DNS servers just echo it back. Similarly the EDNS Client Subnet (ECS) option cannot just be sent to everyone in part because of servers just echoing it back. EDNS - Unknown Version Handling (edns1) dig +nocookie +norec +noad +edns=1 +noednsneg soa zone @server expect: BADVERS expect: OPT record with version set to 0 expect: not to see SOA See RFC6891, 6.1.3. OPT Record TTL Field Use EDNS - Unknown Version with Unknown Option Handling (edns1opt) dig +nocookie +norec +noad +edns=1 +noednsneg +ednsopt=100 soa zone @server expect: BADVERS expect: OPT record with version set to 0 expect: not to see SOA expect: that the option will not be present in response See RFC6891 EDNS - Unknown Flag Handling (ednsflags) dig +nocookie +norec +noad +ednsflags=0x80 soa zone @server expect: SOA expect: NOERROR expect: OPT record with version set to 0 expect: Z bits to be clear in response See RFC6891, 6.1.4 Flags Codes ok - test passed. nsid - NSID supported [RFC5001]. subnet - EDNS Client Subnet supported [RFC7871]. mbz - EDNS flags echoed back. timeout - lookup timed out. To retrieve this report in the future: https://ednscomp.isc.org/ednscomp/79f338bd47 -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From m4rtntns at gmail.com Mon Oct 10 06:50:39 2016 From: m4rtntns at gmail.com (Martin T) Date: Mon, 10 Oct 2016 09:50:39 +0300 Subject: nested prefixes in Internet In-Reply-To: <8737ka4j1q.fsf@mid.deneb.enyo.de> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> Message-ID: Florian: as I told in my initial e-mail, ISP-B is multi-homed, i.e connected to ISP-A(who leases the /24 to ISP-B from their /19 block) and also to ISP-C. ISP-B wants to announce this /24 both to ISP-A and ISP-C. That's the reason why either solution 1 or 2 in my initial e-mail is needed. However, I would like to hear from Roy and Mel why do they prefer a third option where ISP A announces the /19 and the /24 while ISP B does just the /24. thanks, Martin On Wed, Oct 5, 2016 at 11:50 PM, Florian Weimer wrote: > * Martin T.: > >> Florian: >> >>> Are the autonomous systems for the /19 and /24 connected directly? >> >> Yes they are. > > Then deaggregation really isn't necessary at all. > >>> (1) can be better from B's perspective because it prevents certain >>> routing table optimizations (due to the lack of the covering prefix) >> >> What kind of routing table optimizations are possible if covering /19 >> prefix is also present in global routing table? > > The /24 prefix could arguably be dropped and ignored for routing > decisions. From jra at baylink.com Mon Oct 10 13:55:20 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Mon, 10 Oct 2016 13:55:20 +0000 (UTC) Subject: Kudos to Rogers Wireless on IPv6 deployment In-Reply-To: References: Message-ID: <187313483.728.1476107720848.JavaMail.zimbra@baylink.com> ----- Original Message ----- > From: "Ida Leung" > [ ... ... ] Fido Internet is > coming! Cool! My gateway is 1:3603/150. Who do I poll? Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From contact at winterei.se Mon Oct 10 14:12:19 2016 From: contact at winterei.se (Paul S.) Date: Mon, 10 Oct 2016 23:12:19 +0900 Subject: After a Korea Telecom / KT sales rep Message-ID: <1bdbf513-f156-966a-7b71-295bda7e41cd@winterei.se> Hi folks, Looking for a Korea Telecom / KT sales rep for access into Korea *from* Hong Kong. Leads so far have turned up empty over normal channels, anyone mind sharing their contacts? Thanks! From jcurran at arin.net Mon Oct 10 15:17:33 2016 From: jcurran at arin.net (John Curran) Date: Mon, 10 Oct 2016 15:17:33 +0000 Subject: Important - Register for NANOG 68 by 13 October to Vote in the NRO NC Election Message-ID: <41663069-F5C3-4AEB-8B9C-49EF5D4809E3@corp.arin.net> NANOGers - Meeting attendees who register for NANOG 68 by close of business on Thursday, 13 October will be eligible to vote online in the Number Resource Organization Number Council (NRO NC) election for the ARIN region. (If you are also an ARIN eligible voting contact, please note that you will be able to vote in this same election beginning 20 October when voting opens for the ARIN Board of Trustees and ARIN Advisory Council elections.) The 2016 candidates for the one open seat on the NRO NC are: Robert J. Kenney Jason Schiller You can view candidate questionnaires and see/make Statements of Support at: https://www.bigpulse.com/p42725/ You can learn more about the role of the NRO NC at: https://www.arin.net/about_us/nronc.html Note that all eligible NANOG 68 registered attendees will receive an email at 11:00 AM EDT on Monday 17 October with instructions on how to cast your ballot. The election closes at 3:00 PM EDT Friday 28 October and results will be announced the following week. Please visit the ARIN Elections Helpdesk onsite in Dallas if you have issues voting or contact us at "members at arin.net". Thanks for participating! /John John Curran President and CEO American Registry for Internet Numbers From r.engehausen at gmail.com Mon Oct 10 16:04:32 2016 From: r.engehausen at gmail.com (Roy) Date: Mon, 10 Oct 2016 09:04:32 -0700 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> Message-ID: <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> The solution proposed allows ISP-B to use both paths at the same time, needs ISP-C to minimal changes, and has low impact on the global routing tables.. I have successfully used it in the past and my old company is still using it today. .On 10/9/2016 11:50 PM, Martin T wrote: > Florian: > > as I told in my initial e-mail, ISP-B is multi-homed, i.e connected to > ISP-A(who leases the /24 to ISP-B from their /19 block) and also to > ISP-C. ISP-B wants to announce this /24 both to ISP-A and ISP-C. > That's the reason why either solution 1 or 2 in my initial e-mail is > needed. > > However, I would like to hear from Roy and Mel why do they prefer a > third option where ISP A announces the /19 and the /24 while ISP B > does just the /24. > > > thanks, > Martin > > On Wed, Oct 5, 2016 at 11:50 PM, Florian Weimer wrote: >> * Martin T.: >> >>> Florian: >>> >>>> Are the autonomous systems for the /19 and /24 connected directly? >>> Yes they are. >> Then deaggregation really isn't necessary at all. >> >>>> (1) can be better from B's perspective because it prevents certain >>>> routing table optimizations (due to the lack of the covering prefix) >>> What kind of routing table optimizations are possible if covering /19 >>> prefix is also present in global routing table? >> The /24 prefix could arguably be dropped and ignored for routing >> decisions. From joelja at bogus.com Mon Oct 10 16:16:56 2016 From: joelja at bogus.com (joel jaeggli) Date: Mon, 10 Oct 2016 09:16:56 -0700 Subject: nested prefixes in Internet In-Reply-To: <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> Message-ID: On 10/10/16 9:04 AM, Roy wrote: > > > The solution proposed allows ISP-B to use both paths at the same time, > needs ISP-C to minimal changes, and has low impact on the global > routing tables.. I have successfully used it in the past and my old > company is still using it today. Having two parties in control of a prefix announcement is a bit of a disaster. ISP A becomes partitioned from isp B isp B does not withdraw the covering aggregate and black-holes the of ISP A that lands on it's edge. bummer. > > .On 10/9/2016 11:50 PM, Martin T wrote: >> Florian: >> >> as I told in my initial e-mail, ISP-B is multi-homed, i.e connected to >> ISP-A(who leases the /24 to ISP-B from their /19 block) and also to >> ISP-C. ISP-B wants to announce this /24 both to ISP-A and ISP-C. >> That's the reason why either solution 1 or 2 in my initial e-mail is >> needed. >> >> However, I would like to hear from Roy and Mel why do they prefer a >> third option where ISP A announces the /19 and the /24 while ISP B >> does just the /24. >> >> >> thanks, >> Martin >> >> On Wed, Oct 5, 2016 at 11:50 PM, Florian Weimer >> wrote: >>> * Martin T.: >>> >>>> Florian: >>>> >>>>> Are the autonomous systems for the /19 and /24 connected directly? >>>> Yes they are. >>> Then deaggregation really isn't necessary at all. >>> >>>>> (1) can be better from B's perspective because it prevents certain >>>>> routing table optimizations (due to the lack of the covering prefix) >>>> What kind of routing table optimizations are possible if covering /19 >>>> prefix is also present in global routing table? >>> The /24 prefix could arguably be dropped and ignored for routing >>> decisions. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From rsk at gsp.org Mon Oct 10 16:48:11 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Mon, 10 Oct 2016 12:48:11 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <22522.44258.206039.834117@gargle.gargle.HOWL> References: <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <22522.41478.59506.907676@gargle.gargle.HOWL> <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> <22522.42555.121770.918579@gargle.gargle.HOWL> <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> <22522.44258.206039.834117@gargle.gargle.HOWL> Message-ID: <20161010164811.GA22239@gsp.org> On Sun, Oct 09, 2016 at 04:47:30PM -0400, bzs at TheWorld.com wrote: > But I well remember proposed spam mitigations back in 2000 being just > as forcefully shot down because IT WOULD TAKE A DECADE TO IMPLEMENT > THAT!!! I remember that. I also remember the dire predictions that it would take a decade...which it wouldn't have. The problems we face today, including spam, DoS attacks, spoofing, IoT-sourced attacks, etc., have the same easy-to-implement fixes: it's just there exists no collective will to implement those fixes. Consider: everyone who is paying attention to their logs knows that AWS is a systemic/chronic source of spam, SSH brute-force attacks, etc. I don't think Amazon is actively hostile, I just think that they're incompetent, lazy, and cheap -- too incompetent, lazy, and cheap to even cover basics like having a fully-functional abuse@ address, which is something everyone learns in the first hour of the first day in Network Administration 101. This has gone on for *years*. But if everyone on this list simultaneously decided to stop accepting packets from AWS, I guarantee you that it would receive attention within hours. It might not be completely fixed by close-of-business that day, but it would not be the same operation doing the same things. And by the end of that day, we would all be better off - including Amazon, although they may not realize it or want to admit it. The same is true for many other kinds of attacks/abuses from many other sources. Either their hostile behavior is the result of deliberate intent (in which case of *course* they should be blocked) or it's the result of negligence (in which case their attention will be pointedly drawn to it). If you want someone to take action, stop letting it be your problem and make it THEIR problem. Or we can all continue to gripe about it for another decade and spend another $500M on equipment, software, services, and personnel as we try to solve other peoples' problems at our own expense. ---rsk From r.engehausen at gmail.com Mon Oct 10 17:18:33 2016 From: r.engehausen at gmail.com (Roy) Date: Mon, 10 Oct 2016 10:18:33 -0700 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> Message-ID: <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> I don't think I ever said that ISP-B would announce the /19. That would only be announced by ISP-A. ISP-B would only announce the /24 that has been delegated to it. If the ISP-A/ISP-B link goes down then the /24 would be seen only via ISP-C which is the desired result. On 10/10/2016 9:16 AM, joel jaeggli wrote: > On 10/10/16 9:04 AM, Roy wrote: >> >> The solution proposed allows ISP-B to use both paths at the same time, >> needs ISP-C to minimal changes, and has low impact on the global >> routing tables.. I have successfully used it in the past and my old >> company is still using it today. > Having two parties in control of a prefix announcement is a bit of a > disaster. ISP A becomes partitioned from isp B isp B does not withdraw > the covering aggregate and black-holes the of ISP A that lands on it's > edge. bummer. > > From johnl at iecc.com Mon Oct 10 17:18:41 2016 From: johnl at iecc.com (John Levine) Date: 10 Oct 2016 17:18:41 -0000 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> Message-ID: <20161010171841.38389.qmail@ary.lan> >> It helps solve the bad (including manufacturer's default) password >> problem which was one of the attack vectors. That problem has been adddressed pretty well by giving each device a random password and printing the password on the device. Another hack that works pretty well is a button you push that allows TOFU authentication for 30 seconds or so. Neither is perfect, but they both largely solve the problem of scanning for open ports unless the scanner happens to scan at exactly the right time. From sean at donelan.com Mon Oct 10 17:23:58 2016 From: sean at donelan.com (Sean Donelan) Date: Mon, 10 Oct 2016 13:23:58 -0400 (EDT) Subject: FCC: DIRS activated for providers in Florida, Georgia, South Carolina In-Reply-To: References: Message-ID: On Fri, 7 Oct 2016, Sean Donelan wrote: > The U.S. Federal Communications Commission has activated its Disaster > Information Reporting System (DIRS) in response to Hurricane Matthew. The FCC is now requesting daily reports. The FCC has also expanded the area it requests reports, including North Carolina, South Carolina and Virginia. Of course, DIRS is "voluntary." There doesn't seem to be any benefit for communication providers. DIRS doesn't appear to improve or speed up any disaster recovery efforts. https://www.fcc.gov/document/fcc-expands-dirs-coverage-area-hurricane-matthew From niels=nanog at bakker.net Mon Oct 10 17:24:45 2016 From: niels=nanog at bakker.net (Niels Bakker) Date: Mon, 10 Oct 2016 19:24:45 +0200 Subject: nested prefixes in Internet In-Reply-To: <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> Message-ID: <20161010172445.GD45065@excession.tpb.net> * r.engehausen at gmail.com (Roy) [Mon 10 Oct 2016, 19:19 CEST]: >I don't think I ever said that ISP-B would announce the /19. That >would only be announced by ISP-A. ISP-B would only announce the /24 >that has been delegated to it. > >If the ISP-A/ISP-B link goes down then the /24 would be seen only >via ISP-C which is the desired result. What if ISP-A then receives traffic inside its /19 destined for ISP-B's /24? It will have to send it over transit and won't bill ISP-B for that traffic. You cannot expect 100% of the rest of the Internet to honour the more specific all the time. -- Niels. From niels=nanog at bakker.net Mon Oct 10 17:26:47 2016 From: niels=nanog at bakker.net (Niels Bakker) Date: Mon, 10 Oct 2016 19:26:47 +0200 Subject: FCC: DIRS activated for providers in Florida, Georgia, South Carolina In-Reply-To: References: Message-ID: <20161010172647.GE45065@excession.tpb.net> * sean at donelan.com (Sean Donelan) [Mon 10 Oct 2016, 19:25 CEST]: >On Fri, 7 Oct 2016, Sean Donelan wrote: >>The U.S. Federal Communications Commission has activated its Disaster >>Information Reporting System (DIRS) in response to Hurricane Matthew. > >The FCC is now requesting daily reports. The FCC has also expanded >the area it requests reports, including North Carolina, South >Carolina and Virginia. > >Of course, DIRS is "voluntary." There doesn't seem to be any benefit >for communication providers. DIRS doesn't appear to improve or speed >up any disaster recovery efforts. Mandatory timed status reports rarely do -- Niels. From sean at donelan.com Mon Oct 10 17:41:33 2016 From: sean at donelan.com (Sean Donelan) Date: Mon, 10 Oct 2016 13:41:33 -0400 (EDT) Subject: FCC: DIRS activated for providers in Florida, Georgia, South Carolina In-Reply-To: References: Message-ID: Based on voluntary reporting to the DIRS, status as of October 9, 2016. 9% of the cellular sites are out of service in the disaster area (ranges from 0% in some counties to 85% in Marion, SC) 8% of the video subscribers are out of service in the disaster area. 16% of the VoIP and telephone subscribers are out of service in the disaster area. 14% of the broadband access subscribers are out of service. Most outages are due to loss of commercial power. From bzs at TheWorld.com Mon Oct 10 18:39:49 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Mon, 10 Oct 2016 14:39:49 -0400 Subject: IoT security, was Krebs on Security booted off Akamai network In-Reply-To: <20161010164811.GA22239@gsp.org> References: <68205034-2027-e1ed-13ec-fb22eaa1a47e@cisco.com> <87wphxy4h1.fsf@mid.deneb.enyo.de> <87r37phiqn.fsf@mid.deneb.enyo.de> <22522.41478.59506.907676@gargle.gargle.HOWL> <2B9DC3F8-4F03-499A-B83C-F6916A1B334B@beckman.org> <22522.42555.121770.918579@gargle.gargle.HOWL> <74206B0A-F16C-4EF4-884D-C4245DBB2C55@beckman.org> <22522.44258.206039.834117@gargle.gargle.HOWL> <20161010164811.GA22239@gsp.org> Message-ID: <22523.57461.996455.461290@gargle.gargle.HOWL> Rich, Thanks for the nice confirmation. My dabbling in internet governance topics has taught me (I guess) that the real challenge is to eschew easy approaches such as shutting off sites as a remedy. The hard work is trying to come up with effective measures which are anything but take downs / blocking -- those should be an absolute last resort at the end of some well-defined and transparent process. Obviously at some extreme point a site has gone so rogue it's just an act of self-defense. But that's the extreme case and still needs a process even if an emergency, short-circuited process. But for sites which imagine themselves to be responsibly managed but fall down on that job sufficiently to merit a response -- my favorite saying in life: EVERYONE forgives themselves! -- there's a need to structure proportionate and effective responses to failings ranging from warnings to actions. And to define clearly what those failings are. For example everyone might not agree that letting 1% of their traffic be spam or otherwise malicious traffic without opposition is even a problem worth exerting effort over. Ok, is it 2%? 0.1%? What is the threshold we can all live with? Or is a percentage just a bad idea and it's the effect which needs to be measured and judged? I suspect a contractual approach might be more productive, as one example. There are other possibilities. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From baldur.norddahl at gmail.com Mon Oct 10 19:44:55 2016 From: baldur.norddahl at gmail.com (Baldur Norddahl) Date: Mon, 10 Oct 2016 21:44:55 +0200 Subject: nested prefixes in Internet In-Reply-To: <20161010172445.GD45065@excession.tpb.net> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: Den 10/10/2016 kl. 19.24 skrev Niels Bakker: > * r.engehausen at gmail.com (Roy) [Mon 10 Oct 2016, 19:19 CEST]: >> I don't think I ever said that ISP-B would announce the /19. That >> would only be announced by ISP-A. ISP-B would only announce the /24 >> that has been delegated to it. >> >> If the ISP-A/ISP-B link goes down then the /24 would be seen only via >> ISP-C which is the desired result. > > What if ISP-A then receives traffic inside its /19 destined for > ISP-B's /24? It will have to send it over transit and won't bill > ISP-B for that traffic. You cannot expect 100% of the rest of the > Internet to honour the more specific all the time. Is that a real problem? In my experience a /24 is honoured almost universially. If we assume the big tier 1 transit providers honour the /24 announcement, the only possible way for ISP-A to receive traffic via the /19 is if ISP-A is directly peered with someone that ignores the /24. Even if some small amount of traffic does go that route, it might not be viewed as a problem as the volume is likely to be very low. Regards, Baldur From owen at delong.com Mon Oct 10 20:27:38 2016 From: owen at delong.com (Owen DeLong) Date: Mon, 10 Oct 2016 13:27:38 -0700 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> > On Oct 10, 2016, at 12:44 PM, Baldur Norddahl wrote: > > > > Den 10/10/2016 kl. 19.24 skrev Niels Bakker: >> * r.engehausen at gmail.com (Roy) [Mon 10 Oct 2016, 19:19 CEST]: >>> I don't think I ever said that ISP-B would announce the /19. That would only be announced by ISP-A. ISP-B would only announce the /24 that has been delegated to it. >>> >>> If the ISP-A/ISP-B link goes down then the /24 would be seen only via ISP-C which is the desired result. >> >> What if ISP-A then receives traffic inside its /19 destined for ISP-B's /24? It will have to send it over transit and won't bill ISP-B for that traffic. You cannot expect 100% of the rest of the Internet to honour the more specific all the time. > > Is that a real problem? In my experience a /24 is honoured almost universally. In my experience, with notable exceptions, ISPs don?t like to provide transit to people who aren?t paying them, so if it becomes enough traffic to get noticed, it?s not at all unlikely that ISP-A would start dropping it, even if they didn?t ignore the prefix. > If we assume the big tier 1 transit providers honour the /24 announcement, the only possible way for ISP-A to receive traffic via the /19 is if ISP-A is directly peered with someone that ignores the /24. Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. > Even if some small amount of traffic does go that route, it might not be viewed as a problem as the volume is likely to be very low. Until some clever miscreant notices the situation and decides to exploit it for a dDOS. Owen From niels=nanog at bakker.net Mon Oct 10 20:39:18 2016 From: niels=nanog at bakker.net (Niels Bakker) Date: Mon, 10 Oct 2016 22:39:18 +0200 Subject: nested prefixes in Internet In-Reply-To: References: <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: <20161010203918.GF45065@excession.tpb.net> * baldur.norddahl at gmail.com (Baldur Norddahl) [Mon 10 Oct 2016, 21:45 CEST]: >Den 10/10/2016 kl. 19.24 skrev Niels Bakker: >>* r.engehausen at gmail.com (Roy) [Mon 10 Oct 2016, 19:19 CEST]: >>>I don't think I ever said that ISP-B would announce the /19. >>>That would only be announced by ISP-A. ISP-B would only >>>announce the /24 that has been delegated to it. >>> >>>If the ISP-A/ISP-B link goes down then the /24 would be seen >>>only via ISP-C which is the desired result. >> >>What if ISP-A then receives traffic inside its /19 destined for >>ISP-B's /24? It will have to send it over transit and won't bill >>ISP-B for that traffic. You cannot expect 100% of the rest of the >>Internet to honour the more specific all the time. > >Is that a real problem? In my experience a /24 is honoured almost >universially. > >If we assume the big tier 1 transit providers honour the /24 >announcement, the only possible way for ISP-A to receive traffic via >the /19 is if ISP-A is directly peered with someone that ignores the >/24. > >Even if some small amount of traffic does go that route, it might >not be viewed as a problem as the volume is likely to be very low. Sure, continue believing that, until you run into a customer who takes a /24 and then announces it no-export just to your upstream, and mysteriously has lots of outages on their link to you. Aside from that, this does happen and apparently much more often than you think. Think CDN nodes with partial views. There are plenty networks that filter on or close to RIR boundaries due to hardware limitations as well. -- Niels. From baldur.norddahl at gmail.com Mon Oct 10 21:59:20 2016 From: baldur.norddahl at gmail.com (Baldur Norddahl) Date: Mon, 10 Oct 2016 23:59:20 +0200 Subject: nested prefixes in Internet In-Reply-To: <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> Message-ID: Den 10/10/2016 kl. 22.27 skrev Owen DeLong: > Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. > But ISP-A is in fact being paid to carry the traffic. Supposedly ISP-B has a paid transit relation to ISP-A. In the case the transit link is down ISP-A might have to transport the traffic through a less profitable link however. I know that if ISP-A was my network I would be making money even with the transit link down. Yes I might have to transport something out of my network through one of my transits, but outbound traffic is in fact free for us because we are heavy inbound loaded. Regards, Baldur From jay at west.net Mon Oct 10 23:33:02 2016 From: jay at west.net (Jay Hennigan) Date: Mon, 10 Oct 2016 16:33:02 -0700 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> Message-ID: <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> On 10/6/16 1:26 PM, Jesse McGraw wrote: > Nanog, > > (This is me scratching an itch of my own and hoping that sharing it > might be useful to others on this list. Apologies if it isn't) > > When I'm trying to comprehend a new or complicated Cisco router, > switch or firewall configuration an old pet-peeve of mine is how > needlessly difficult it is to follow deeply nested logic in route-maps, > ACLs, QoS policy-maps etc etc > > To make this a bit simpler I?ve been working on a perl script to convert > these text-based configuration files into HTML with links between the > different elements (e.g. To an access-list from the interface where it?s > applied, from policy-maps to class-maps etc), hopefully making it easier > to to follow the chain of logic via clicking links and using the forward > and back buttons in your browser to go back and forth between command > and referenced list. Way cool. Now to hook it into RANCID.... -- Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV From mysidia at gmail.com Tue Oct 11 09:09:00 2016 From: mysidia at gmail.com (Jimmy Hess) Date: Tue, 11 Oct 2016 04:09:00 -0500 Subject: nested prefixes in Internet In-Reply-To: <20161010172445.GD45065@excession.tpb.net> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: On Mon, Oct 10, 2016 at 12:24 PM, Niels Bakker wrote: > * r.engehausen at gmail.com (Roy) [Mon 10 Oct 2016, 19:19 CEST]: [snip] >> If the ISP-A/ISP-B link goes down then the /24 would be seen only via >> ISP-C which is the desired result. > > What if ISP-A then receives traffic inside its /19 destined for ISP-B's /24? > It will have to send it over transit and won't bill ISP-B for that traffic. This is still the favored approach over ISP-A de-aggregating their address space to support some multi-homed customers. It's all something that ISP-A can put in their contract with ISP-B. A can bill ISP-B per packet-byte routed with destination of B's /24, if they wanted, or reach other mutually agreeable conditions that make sure the /24 carve out does not commit A to giving service to B involving more traffic than B pays for. Or they can list Rate #1 for incoming traffic to B's /24 sent to a peering link with ISP-B, and apply pricing Rate #2 for packets destined for B's /24 between two transit providers. ...However, ISP-A and ISP-B should have a link; Either a physical connection or a virtual one (such as a tunnel), with BGP peering between A and B, for ISP-B to be using a /24 from A's IP space with other providers. > You cannot expect 100% of the rest of the Internet to honour the more > specific all the time. That is true, but not really notable / isn't really a significant problem. Probably close enough to 100% of the internet to honor it 99% of the time, and the 1% that don't probably will likely not be one of ISP A or B's adjacent providers. (If they are, then A or B will open a ticket, and the /24 will be fixed to work as desired) Heck.... on some occasion when ISP A has an outage, the /19 won't be there, so less than 100% of the internet might 'honor' that one at times, as well. -- -JH From ler762 at gmail.com Tue Oct 11 13:23:18 2016 From: ler762 at gmail.com (Lee) Date: Tue, 11 Oct 2016 09:23:18 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <8f16b55b-8be1-020f-aaa8-78c6726a6934@efes.iucc.ac.il> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <8f16b55b-8be1-020f-aaa8-78c6726a6934@efes.iucc.ac.il> Message-ID: On 10/8/16, Hank Nussbacher wrote: > On 07/10/2016 17:59, Lee wrote: >> On 10/7/16, Hank Nussbacher wrote: >>> On 07/10/2016 00:33, Lee wrote: >>>> dunno about creating web pages, but >>>> https://www.nanog.org/meetings/abstract?id=785 >>>> has a section on showing filters that are defined but not referenced & >>>> referenced but not defined >>> In IOS-XR it is one command "sho rpl unused ?" >>> RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused ? >>> as-path-set Display as-path-set objects >>> community-set Display community-set objects >>> extcommunity-set Display extended community objects >>> prefix-set Display prefix-set objects >>> rd-set Display rd-set objects >>> route-policy Display route-policy objects >>> tag-set Display tag-set objects >>> >>> RP/0/RSP0/CPU0:petach-tikva-gp#show rpl unused prefix >>> Fri Oct 7 08:24:53.237 IDT >>> >>> ACTIVE -- Referenced by at least one policy which is attached >>> INACTIVE -- Only referenced by policies which are not attached >>> UNUSED -- Not attached (directly or indirectly) and not referenced >> I'm actually starting to miss being out of the game. I'm retired, so >> don't have access to anything running IOS-XR. Just out of curiosity, >> how does the output of 'show rpl unused prefix' compare to the output >> of the script at http://pastebin.com/pem7tHAJ >> >> Thanks, >> Lee >> > Samples: > <.. snip samples ..> interesting.. thanks! > Note the sloppy code - sometimes they state UNUSED and sometimes > (UNUSED). Or "the following policies are"... rather than "the following > routing policies are". Just plain sloppy Cisco coding and poor QA. And > once you delete these unreferenced objects, "show rpl unused" will still > show them since there is a bug in Cisco code (CSCuy07932/CSCug9153). See: > http://www.gossamer-threads.com/lists/cisco/nsp/192481 > for details. Which is why I like having the source code -- there's the possibility of fixing whatever myself instead of having to wait for the vendor to fix it :) Thanks, Lee From ler762 at gmail.com Tue Oct 11 13:48:17 2016 From: ler762 at gmail.com (Lee) Date: Tue, 11 Oct 2016 09:48:17 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> Message-ID: On 10/10/16, Jay Hennigan wrote: > On 10/6/16 1:26 PM, Jesse McGraw wrote: >> Nanog, >> >> (This is me scratching an itch of my own and hoping that sharing it >> might be useful to others on this list. Apologies if it isn't) >> >> When I'm trying to comprehend a new or complicated Cisco router, >> switch or firewall configuration an old pet-peeve of mine is how >> needlessly difficult it is to follow deeply nested logic in route-maps, >> ACLs, QoS policy-maps etc etc >> >> To make this a bit simpler I?ve been working on a perl script to convert >> these text-based configuration files into HTML with links between the >> different elements (e.g. To an access-list from the interface where it?s >> applied, from policy-maps to class-maps etc), hopefully making it easier >> to to follow the chain of logic via clicking links and using the forward >> and back buttons in your browser to go back and forth between command >> and referenced list. > > Way cool. Now to hook it into RANCID.... It looks like what I did in 2.3.8 should still work - control_rancid puts the diff output into $TMP.diff so add this bit: grep "^Index: " $TMP.diff | awk '/^Index: configs/{ if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } printf("%s ", $2) } END{ printf("\n") } ' >$TMP.doit /bin/sh $TMP.doit >$TMP.out if [ -s $TMP.out ] ; then .. send mail / whatever rm $TMP.doit $TMP.out fi Regards, Lee From kamtha at ak-labs.net Tue Oct 11 14:23:19 2016 From: kamtha at ak-labs.net (Carlos Kamtha) Date: Tue, 11 Oct 2016 10:23:19 -0400 Subject: List of US server providers? Message-ID: <20161011142319.GA93954@ak-labs.net> Hello Everyone, Was wondering if anyone can point me to a current list of dedicated/VPS providers in the US. That is, if such a list exists... Any help would be greatly appreciated. Cheers. -C From job at ntt.net Tue Oct 11 15:01:56 2016 From: job at ntt.net (Job Snijders) Date: Tue, 11 Oct 2016 17:01:56 +0200 Subject: Large BGP Communities beacon in the wild Message-ID: <20161011150156.GF4457@hanna.meerval.net> Dear all, Large BGP Communities are a novel way to signal information between networks. An example of a Large BGP Communities is: 2914:4056024901:80. Large BGP Communities are composed of three 4-octet integers, separated by something like a colon. This is easy to remember and accommodates advanced routing policies in relation to 4-Byte ASNs. It is the tool that has been missing since 4-octet ASNs were introduced. IANA has made an Early Allocation of the value 30 (LARGE_COMMUNITY) in the "BGP Path Attributes" registry under the "Border Gateway Protocol (BGP) Parameters" group. The draft can be read here: https://tools.ietf.org/html/draft-ietf-idr-large-community Additional information about Large BGP Communities can be found here: http://largebgpcommunities.net/ Starting today (2016.10.11), the following two BGP beacons are available to the general public, with AS_PATH 2914_15562$ Both these prefixes have a Large BGP Community attached: 2001:67c:208c::/48 192.147.168.0/24 Large BGP Community - 15562:1:1 The NLNOG RING BGP Looking Glass is running the latest version of BIRD which understands the Large BGP Community Path Attribute. IPv4 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=192.147.168.0/24 IPv6 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2001:67c:208c::/48 In theory, since this is an optional transitive BGP Path Attribute, all the Looking Glass' peers should boomerang the Large Community back to the LG. However we currently observe that 50 out of 75 peers propagate the Large BGP Community to the LG. Relevant Router commands to see if you receive the attribute, or whether one of intermediate networks has stripped the attribute from the route: IOS: show ip bgp path-attribute unknown shows all prefixes with unknown path attributes. IOS #2 - like on route views: route-views>sh ip bgp 192.147.168.0 BGP routing table entry for 192.147.168.0/24, version 98399100 Paths: (39 available, best #30, table default) Not advertised to any peer Refresh Epoch 1 701 2914 15562 137.39.3.55 from 137.39.3.55 (137.39.3.55) Origin IGP, localpref 100, valid, external unknown transitive attribute: flag 0xE0 type 0x1E length 0xC value 0000 3CCA 0000 0001 0000 0001 rx pathid: 0, tx pathid: 0 IOS-XR: (you must look at specific prefixes) RP/0/RSP0/CPU0:Router#show bgp ipv6 unicast 2001:67c:208c::/48 unknown-attributes BGP routing table entry for 2001:67c:208c::/48 Community: 2914:370 2914:1206 2914:2203 2914:3200 Unknown attributes have size 15 Raw value: e0 1e 0c 00 00 3c ca 00 00 00 01 00 00 00 01 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ JunOS: user at JunOS-re6> show route 2001:67c:208c::/48 detail 2001:67c:208c::/48 (1 entry, 1 announced) AS path: 15562 I Unrecognized Attributes: 15 bytes Attr flags e0 code 1e: 00 00 3c ca 00 00 00 01 00 00 00 01 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A note about router Configurations: Ensure you are not fitlering the path attributes, eg: JunOS: [edit protocols bgp] user at junos# delete drop-path-attributes 30 XR: configure router bgp YourASN attribute-filter group ReallyBadIdea ! avoid creating bogons no attribute 30 ! ! Contact persons: myself or Jared Mauch or NTT NOC. BGP Session identifier 83.231.213.230 / 2001:728:0:5000::a92 AS 15562. Kind regards, Job From jtk at depaul.edu Tue Oct 11 16:25:50 2016 From: jtk at depaul.edu (John Kristoff) Date: Tue, 11 Oct 2016 11:25:50 -0500 Subject: List of US server providers? In-Reply-To: References: Message-ID: <20161011112550.43029e02@p50.localdomain> On Tue, 11 Oct 2016 14:23:19 +0000 Carlos Kamtha wrote: > Was wondering if anyone can point me to a current list of > dedicated/VPS providers in the US. That is, if such a list exists... I'm not sure such a comprehensive and regularly maintained list is available, and I'm not sure it could be. It would be very large and difficult to keep current. You can find many providers advertised or discussed in web-based forums such as Web Hosting Talk or Low End Talk. I maintain a small subset and woefully incomplete list of providers, most of which have some U.S. presence, but not necessarily all, that I've had some recent experience with here: John From baldur.norddahl at gmail.com Tue Oct 11 17:31:13 2016 From: baldur.norddahl at gmail.com (Baldur Norddahl) Date: Tue, 11 Oct 2016 19:31:13 +0200 Subject: Large BGP Communities beacon in the wild In-Reply-To: <20161011150156.GF4457@hanna.meerval.net> References: <20161011150156.GF4457@hanna.meerval.net> Message-ID: <7471d953-001e-f149-5050-4849afa7b007@gmail.com> Hi This looks like this on the ZTE M6000 platform: ballerup-edge1#show bgp vpnv4 unicast vrf internet detail 192.147.168.0 255.255.255.0 BGP routing table entry for 192.147.168.0/24 25w4d received from 149.6.136.169 (154.26.32.23), path-id 0 Origin i, nexthop 149.6.136.169, metric 100, localpref 100,weight 0, rtpref 200, best, block best, selected, Community 174:21100 174:22010 60876:174 *Unknown attribute type 30 flag e0 len 12* As path [174 2914 15562] As4 path Received label notag 25w4d received from 216.66.83.101 (216.218.252.202), path-id 0 Origin i, nexthop 216.66.83.101, metric 25, localpref 100,weight 0, rtpref 200, Community 60876:6939 *Unknown attribute type 30 flag e0 len 12* As path [6939 1299 2914 15562] As4 path Received label notag Regards, Baldur Den 11/10/2016 kl. 17.01 skrev Job Snijders: > Dear all, > > Large BGP Communities are a novel way to signal information between > networks. An example of a Large BGP Communities is: 2914:4056024901:80. > > Large BGP Communities are composed of three 4-octet integers, separated > by something like a colon. This is easy to remember and accommodates > advanced routing policies in relation to 4-Byte ASNs. It is the tool that has > been missing since 4-octet ASNs were introduced. > > IANA has made an Early Allocation of the value 30 (LARGE_COMMUNITY) in > the "BGP Path Attributes" registry under the "Border Gateway Protocol > (BGP) Parameters" group. > > The draft can be read here: https://tools.ietf.org/html/draft-ietf-idr-large-community > > Additional information about Large BGP Communities can be found here: > http://largebgpcommunities.net/ > > Starting today (2016.10.11), the following two BGP beacons are available > to the general public, with AS_PATH 2914_15562$ > > Both these prefixes have a Large BGP Community attached: > > 2001:67c:208c::/48 > 192.147.168.0/24 > > Large BGP Community - 15562:1:1 > > The NLNOG RING BGP Looking Glass is running the latest version of BIRD > which understands the Large BGP Community Path Attribute. > > IPv4 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=192.147.168.0/24 > IPv6 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2001:67c:208c::/48 > > In theory, since this is an optional transitive BGP Path Attribute, all > the Looking Glass' peers should boomerang the Large Community back to > the LG. However we currently observe that 50 out of 75 peers propagate > the Large BGP Community to the LG. > > Relevant Router commands to see if you receive the attribute, or whether > one of intermediate networks has stripped the attribute from the route: > > IOS: show ip bgp path-attribute unknown > shows all prefixes with unknown path attributes. > > IOS #2 - like on route views: > route-views>sh ip bgp 192.147.168.0 > BGP routing table entry for 192.147.168.0/24, version 98399100 > Paths: (39 available, best #30, table default) > Not advertised to any peer > Refresh Epoch 1 > 701 2914 15562 > 137.39.3.55 from 137.39.3.55 (137.39.3.55) > Origin IGP, localpref 100, valid, external > unknown transitive attribute: flag 0xE0 type 0x1E length 0xC > value 0000 3CCA 0000 0001 0000 0001 > rx pathid: 0, tx pathid: 0 > > IOS-XR: (you must look at specific prefixes) > RP/0/RSP0/CPU0:Router#show bgp ipv6 unicast 2001:67c:208c::/48 unknown-attributes > BGP routing table entry for 2001:67c:208c::/48 > Community: 2914:370 2914:1206 2914:2203 2914:3200 > Unknown attributes have size 15 > Raw value: > e0 1e 0c 00 00 3c ca 00 00 00 01 00 00 00 01 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > JunOS: > user at JunOS-re6> show route 2001:67c:208c::/48 detail > 2001:67c:208c::/48 (1 entry, 1 announced) > AS path: 15562 I > Unrecognized Attributes: 15 bytes > Attr flags e0 code 1e: 00 00 3c ca 00 00 00 01 00 00 00 01 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > A note about router Configurations: > > Ensure you are not fitlering the path attributes, eg: > > JunOS: > [edit protocols bgp] > user at junos# delete drop-path-attributes 30 > > XR: > configure > router bgp YourASN > attribute-filter group ReallyBadIdea ! avoid creating bogons > no attribute 30 > ! > ! > > Contact persons: myself or Jared Mauch or NTT NOC. BGP Session > identifier 83.231.213.230 / 2001:728:0:5000::a92 AS 15562. > > Kind regards, > > Job From kamtha at ak-labs.net Tue Oct 11 16:30:46 2016 From: kamtha at ak-labs.net (Carlos Kamtha) Date: Tue, 11 Oct 2016 12:30:46 -0400 Subject: List of US server providers? In-Reply-To: <20161011112550.43029e02@p50.localdomain> References: <20161011112550.43029e02@p50.localdomain> Message-ID: <20161011163046.GC93954@ak-labs.net> thanks! -C On Tue, Oct 11, 2016 at 11:25:50AM -0500, John Kristoff wrote: > On Tue, 11 Oct 2016 14:23:19 +0000 > Carlos Kamtha wrote: > > > Was wondering if anyone can point me to a current list of > > dedicated/VPS providers in the US. That is, if such a list exists... > > I'm not sure such a comprehensive and regularly maintained list is > available, and I'm not sure it could be. It would be very large and > difficult to keep current. You can find many providers advertised or > discussed in web-based forums such as Web Hosting Talk or Low End Talk. > > I maintain a small subset and woefully incomplete list of providers, > most of which have some U.S. presence, but not necessarily all, that > I've had some recent experience with here: > > > > John From marka at isc.org Tue Oct 11 22:09:35 2016 From: marka at isc.org (Mark Andrews) Date: Wed, 12 Oct 2016 09:09:35 +1100 Subject: Anyone from Facebook here? Message-ID: <20161011220935.2989156610D1@rock.dv.isc.org> You may want to follow up on this email thread. IPv6 vs IPv4 performance to m.facebook.com. https://www.mail-archive.com/bind-users at lists.isc.org/msg23649.html -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From owen at delong.com Tue Oct 11 22:58:09 2016 From: owen at delong.com (Owen DeLong) Date: Tue, 11 Oct 2016 15:58:09 -0700 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> Message-ID: <49C83B6E-C0D5-44C0-A13C-7B55F5AF5254@delong.com> > On Oct 10, 2016, at 14:59 , Baldur Norddahl wrote: > > > > Den 10/10/2016 kl. 22.27 skrev Owen DeLong: >> Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. >> > > But ISP-A is in fact being paid to carry the traffic. Supposedly ISP-B has a paid transit relation to ISP-A. In the case the transit link is down ISP-A might have to transport the traffic through a less profitable link however. Which isn?t really in the agreement between ISP-B and ISP-A unless it was specifically (and unusually) negotiated. Also, you?re assuming that the leased space came with a transit agreement. In many cases, address leases don?t, so consider the additional scenario where ISP-B leases addresses from ISP-A, but has transit contracts with ISP-C and ISP-D but no connection at all to ISP-A. > I know that if ISP-A was my network I would be making money even with the transit link down. Yes I might have to transport something out of my network through one of my transits, but outbound traffic is in fact free for us because we are heavy inbound loaded. Yes, but it doesn?t help if it also came in on a transit link. Any traffic you both receive and transmit on transit costs you money pretty much no matter who you are. Owen From joly at punkcast.com Tue Oct 11 23:07:27 2016 From: joly at punkcast.com (Joly MacFie) Date: Tue, 11 Oct 2016 19:07:27 -0400 Subject: Anyone from Facebook here? In-Reply-To: <20161011220935.2989156610D1@rock.dv.isc.org> References: <20161011220935.2989156610D1@rock.dv.isc.org> Message-ID: Robert Pepper is now working for fb. ?Don't have an email. ? On Tue, Oct 11, 2016 at 6:09 PM, Mark Andrews wrote: > > You may want to follow up on this email thread. IPv6 vs IPv4 performance > to m.facebook.com. > > https://www.mail-archive.com/bind-users at lists.isc.org/msg23649.html > > -- > Mark Andrews, ISC > 1 Seymour St., Dundas Valley, NSW 2117, Australia > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org > -- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast -------------------------------------------------------------- - From joly at punkcast.com Wed Oct 12 08:51:15 2016 From: joly at punkcast.com (Joly MacFie) Date: Wed, 12 Oct 2016 04:51:15 -0400 Subject: Ronog livestream today Message-ID: https://youtu.be/fp3YU6hRlpc Some in English. Agenda http://ronog.ro/ ION Bucharest at 14:00 EEST | 11:00 UTC | 07:00 EDT http://www.internetsociety.org/deploy360/ion/bucharest2016/ -- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast -------------------------------------------------------------- - From jhellenthal at dataix.net Wed Oct 12 13:28:44 2016 From: jhellenthal at dataix.net (Jason Hellenthal) Date: Wed, 12 Oct 2016 08:28:44 -0500 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> Message-ID: <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> Give these a shot. https://github.com/jlmcgraw/networkUtilities I know J could use a little feedback on those as well but all in all they are pretty solid. > On Oct 11, 2016, at 08:48, Lee wrote: > > On 10/10/16, Jay Hennigan wrote: >> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>> Nanog, >>> >>> (This is me scratching an itch of my own and hoping that sharing it >>> might be useful to others on this list. Apologies if it isn't) >>> >>> When I'm trying to comprehend a new or complicated Cisco router, >>> switch or firewall configuration an old pet-peeve of mine is how >>> needlessly difficult it is to follow deeply nested logic in route-maps, >>> ACLs, QoS policy-maps etc etc >>> >>> To make this a bit simpler I?ve been working on a perl script to convert >>> these text-based configuration files into HTML with links between the >>> different elements (e.g. To an access-list from the interface where it?s >>> applied, from policy-maps to class-maps etc), hopefully making it easier >>> to to follow the chain of logic via clicking links and using the forward >>> and back buttons in your browser to go back and forth between command >>> and referenced list. >> >> Way cool. Now to hook it into RANCID.... > > It looks like what I did in 2.3.8 should still work - control_rancid > puts the diff output into $TMP.diff so add this bit: > grep "^Index: " $TMP.diff | awk '/^Index: configs/{ > if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } > printf("%s ", $2) > } > END{ printf("\n") } > ' >$TMP.doit > /bin/sh $TMP.doit >$TMP.out > if [ -s $TMP.out ] ; then > .. send mail / whatever > rm $TMP.doit $TMP.out > fi > > Regards, > Lee -- Jason Hellenthal JJH48-ARIN From Jason_Livingood at comcast.com Wed Oct 12 20:38:12 2016 From: Jason_Livingood at comcast.com (Livingood, Jason) Date: Wed, 12 Oct 2016 20:38:12 +0000 Subject: INFORM: Web-Based Speed Test Hackathon - Princeton, NJ in November Message-ID: May be of interest to folks on this list? Comcast is developing a new open source web-based speed test ? see http://labs.comcast.com/beta-testing-a-new-open-source-speed-test. ISPs, academic researchers, and others will likely be interested in using this rather than older tools. One goal is to make it easy for any organization to customize it for their use and UI, and we?ll soon find a neutral non-Comcast home for the code. To kick this off we are sponsoring a hackathon at Princeton University in November (the 3rd and 4th). Anyone can participate ? though participants should preferably be developers or have development skills (the code is written in Node.JS with some Angular components for the UI). All details at https://citp.princeton.edu/event/speedtest-hackathon/ and we?re asking folks to click the RSVP link at the top to register ASAP. Regards, Jason Livingood Comcast From eric-list at truenet.com Wed Oct 12 22:43:18 2016 From: eric-list at truenet.com (Eric Tykwinski) Date: Wed, 12 Oct 2016 18:43:18 -0400 Subject: Just a quick question... Message-ID: <5425FF07-FFF6-4C45-9972-3284A9B1D8E5@truenet.com> IPv4 routes did a quick bounce to 600,949 around 9:30AM EST, than went back down to 599,241 shortly after. Seemed like a big jump so I setup an alert, just wondering if anyone else noticed anything, I?m not overly concerned, but seemed like a route leak possibly and I didn?t really see anything on bgpstream. Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 From job at instituut.net Wed Oct 12 22:50:56 2016 From: job at instituut.net (Job Snijders) Date: Thu, 13 Oct 2016 00:50:56 +0200 Subject: Just a quick question... In-Reply-To: <5425FF07-FFF6-4C45-9972-3284A9B1D8E5@truenet.com> References: <5425FF07-FFF6-4C45-9972-3284A9B1D8E5@truenet.com> Message-ID: <20161012225056.GD22805@Vurt.local> Hi Eric, On Wed, Oct 12, 2016 at 06:43:18PM -0400, Eric Tykwinski wrote: > IPv4 routes did a quick bounce to 600,949 around 9:30AM EST, than went > back down to 599,241 shortly after. Seemed like a big jump so I setup > an alert, just wondering if anyone else noticed anything, I?m not > overly concerned, but seemed like a route leak possibly and I didn?t > really see anything on bgpstream. Those numbers are entirely reasonable. In fact, the aggregate of unique IPv4 routes on the NLNOG RING Looking Glass is 685,926. Hanging of AS 2914 it is 601,186 in Amsterdam. Similar numbers are observed by AS 131072 at http://www.cidr-report.org/as2.0/ Kind regards, Job From ler762 at gmail.com Wed Oct 12 23:59:54 2016 From: ler762 at gmail.com (Lee) Date: Wed, 12 Oct 2016 19:59:54 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> Message-ID: On 10/12/16, Jason Hellenthal wrote: > Give these a shot. https://github.com/jlmcgraw/networkUtilities > > I know J could use a little feedback on those as well but all in all they > are pretty solid. Where does one get Modern/Perl.pm ? Can't locate Modern/Perl.pm in @INC (you may need to install the Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int /usr/lib/perl5/site_perl/5.22 /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int /usr/lib/perl5/vendor_perl/5.22 /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) at /tmp/iosToHtml.pl line 87. BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. Lee > >> On Oct 11, 2016, at 08:48, Lee wrote: >> >> On 10/10/16, Jay Hennigan wrote: >>> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>>> Nanog, >>>> >>>> (This is me scratching an itch of my own and hoping that sharing it >>>> might be useful to others on this list. Apologies if it isn't) >>>> >>>> When I'm trying to comprehend a new or complicated Cisco router, >>>> switch or firewall configuration an old pet-peeve of mine is how >>>> needlessly difficult it is to follow deeply nested logic in route-maps, >>>> ACLs, QoS policy-maps etc etc >>>> >>>> To make this a bit simpler I?ve been working on a perl script to >>>> convert >>>> these text-based configuration files into HTML with links between the >>>> different elements (e.g. To an access-list from the interface where >>>> it?s >>>> applied, from policy-maps to class-maps etc), hopefully making it >>>> easier >>>> to to follow the chain of logic via clicking links and using the >>>> forward >>>> and back buttons in your browser to go back and forth between command >>>> and referenced list. >>> >>> Way cool. Now to hook it into RANCID.... >> >> It looks like what I did in 2.3.8 should still work - control_rancid >> puts the diff output into $TMP.diff so add this bit: >> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ >> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } >> printf("%s ", $2) >> } >> END{ printf("\n") } >> ' >$TMP.doit >> /bin/sh $TMP.doit >$TMP.out >> if [ -s $TMP.out ] ; then >> .. send mail / whatever >> rm $TMP.doit $TMP.out >> fi >> >> Regards, >> Lee > > > -- > Jason Hellenthal > JJH48-ARIN From ag4ve.us at gmail.com Thu Oct 13 01:23:01 2016 From: ag4ve.us at gmail.com (shawn wilson) Date: Wed, 12 Oct 2016 21:23:01 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> Message-ID: Cpan? Cpan minus? Or just download [1] and there's probably a Make::Maker or similar Build.PL to build a makefile or just install it for you - there's a #perl channel on freenode if you need more and Google doesn't get you set. 1. http://search.cpan.org/~chromatic/Modern-Perl-1.20161005/lib/Modern/Perl.pm On Oct 12, 2016 8:02 PM, "Lee" wrote: > On 10/12/16, Jason Hellenthal wrote: > > Give these a shot. https://github.com/jlmcgraw/networkUtilities > > > > I know J could use a little feedback on those as well but all in all they > > are pretty solid. > > Where does one get Modern/Perl.pm ? > > Can't locate Modern/Perl.pm in @INC (you may need to install the > Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 > /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int > /usr/lib/perl5/site_perl/5.22 > /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int > /usr/lib/perl5/vendor_perl/5.22 > /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) > at /tmp/iosToHtml.pl line 87. > BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. > > Lee > > > > > > >> On Oct 11, 2016, at 08:48, Lee wrote: > >> > >> On 10/10/16, Jay Hennigan wrote: > >>> On 10/6/16 1:26 PM, Jesse McGraw wrote: > >>>> Nanog, > >>>> > >>>> (This is me scratching an itch of my own and hoping that sharing it > >>>> might be useful to others on this list. Apologies if it isn't) > >>>> > >>>> When I'm trying to comprehend a new or complicated Cisco router, > >>>> switch or firewall configuration an old pet-peeve of mine is how > >>>> needlessly difficult it is to follow deeply nested logic in > route-maps, > >>>> ACLs, QoS policy-maps etc etc > >>>> > >>>> To make this a bit simpler I?ve been working on a perl script to > >>>> convert > >>>> these text-based configuration files into HTML with links between the > >>>> different elements (e.g. To an access-list from the interface where > >>>> it?s > >>>> applied, from policy-maps to class-maps etc), hopefully making it > >>>> easier > >>>> to to follow the chain of logic via clicking links and using the > >>>> forward > >>>> and back buttons in your browser to go back and forth between command > >>>> and referenced list. > >>> > >>> Way cool. Now to hook it into RANCID.... > >> > >> It looks like what I did in 2.3.8 should still work - control_rancid > >> puts the diff output into $TMP.diff so add this bit: > >> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ > >> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } > >> printf("%s ", $2) > >> } > >> END{ printf("\n") } > >> ' >$TMP.doit > >> /bin/sh $TMP.doit >$TMP.out > >> if [ -s $TMP.out ] ; then > >> .. send mail / whatever > >> rm $TMP.doit $TMP.out > >> fi > >> > >> Regards, > >> Lee > > > > > > -- > > Jason Hellenthal > > JJH48-ARIN > From andree+nanog at toonk.nl Thu Oct 13 06:55:39 2016 From: andree+nanog at toonk.nl (Andree Toonk) Date: Wed, 12 Oct 2016 23:55:39 -0700 Subject: Just a quick question... In-Reply-To: <5425FF07-FFF6-4C45-9972-3284A9B1D8E5@truenet.com> References: <5425FF07-FFF6-4C45-9972-3284A9B1D8E5@truenet.com> Message-ID: <57FF2FEB.30500@toonk.nl> Hi Eric, My secret spy satellite informs me that Eric Tykwinski wrote On 2016-10-12, 3:43 PM: > IPv4 routes did a quick bounce to 600,949 around 9:30AM EST, than went back down to 599,241 shortly after. > Seemed like a big jump so I setup an alert, just wondering if anyone else noticed anything, I?m not overly concerned, but seemed like a route leak possibly and I didn?t really see anything on bgpstream. Looks like AS45899 (Vietnam Posts and Telecommunications Group) de-aggregated a bunch of their prefixes into 2,090 new more specifics at around the time you mentioned. BGPmon.net data observed the two thousand new prefixes or so originated by AS45899 at around 2016-10-12 13:26 (UTC). Most peers lost the prefix a few minutes later at 13:29 UTC you can find an example here: https://stat.ripe.net/widget/bgplay#w.resource=14.191.200.0%2F22 some other example prefixes include: 14.191.200.0/22 14.191.228.0/22 14.191.24.0/21 14.191.24.0/21 14.191.140.0/22 Looking at the data it appears the same de-aggregation event involving AS45899 happened on 2016-08-03 as well. You didn't see this on BGPstream.com because we currently don't publish de-aggregation event (ie. more specifics (by the same AS). Cheers Andree From jlmcgraw at gmail.com Thu Oct 13 13:08:07 2016 From: jlmcgraw at gmail.com (Jesse McGraw) Date: Thu, 13 Oct 2016 09:08:07 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> Message-ID: <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> Lee, Check out the setup.sh script, hopefully it does everything necessary to get the script working on a Debian-derived Linux system I've attempted to make the only globally-installed dependencies be cpanm and carton. Once those are installed it uses carton to install the dependencies locally On 10/12/2016 07:59 PM, Lee wrote: > On 10/12/16, Jason Hellenthal wrote: >> Give these a shot. https://github.com/jlmcgraw/networkUtilities >> >> I know J could use a little feedback on those as well but all in all they >> are pretty solid. > Where does one get Modern/Perl.pm ? > > Can't locate Modern/Perl.pm in @INC (you may need to install the > Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 > /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int > /usr/lib/perl5/site_perl/5.22 > /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int > /usr/lib/perl5/vendor_perl/5.22 > /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) > at /tmp/iosToHtml.pl line 87. > BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. > > Lee > > > >>> On Oct 11, 2016, at 08:48, Lee wrote: >>> >>> On 10/10/16, Jay Hennigan wrote: >>>> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>>>> Nanog, >>>>> >>>>> (This is me scratching an itch of my own and hoping that sharing it >>>>> might be useful to others on this list. Apologies if it isn't) >>>>> >>>>> When I'm trying to comprehend a new or complicated Cisco router, >>>>> switch or firewall configuration an old pet-peeve of mine is how >>>>> needlessly difficult it is to follow deeply nested logic in route-maps, >>>>> ACLs, QoS policy-maps etc etc >>>>> >>>>> To make this a bit simpler I?ve been working on a perl script to >>>>> convert >>>>> these text-based configuration files into HTML with links between the >>>>> different elements (e.g. To an access-list from the interface where >>>>> it?s >>>>> applied, from policy-maps to class-maps etc), hopefully making it >>>>> easier >>>>> to to follow the chain of logic via clicking links and using the >>>>> forward >>>>> and back buttons in your browser to go back and forth between command >>>>> and referenced list. >>>> Way cool. Now to hook it into RANCID.... >>> It looks like what I did in 2.3.8 should still work - control_rancid >>> puts the diff output into $TMP.diff so add this bit: >>> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ >>> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } >>> printf("%s ", $2) >>> } >>> END{ printf("\n") } >>> ' >$TMP.doit >>> /bin/sh $TMP.doit >$TMP.out >>> if [ -s $TMP.out ] ; then >>> .. send mail / whatever >>> rm $TMP.doit $TMP.out >>> fi >>> >>> Regards, >>> Lee >> >> -- >> Jason Hellenthal >> JJH48-ARIN > . > From jhellenthal at dataix.net Thu Oct 13 13:32:40 2016 From: jhellenthal at dataix.net (Jason Hellenthal) Date: Thu, 13 Oct 2016 08:32:40 -0500 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> Message-ID: <58F7DD53-68F4-434F-BEFA-4C55303E8EF8@dataix.net> Thanks for chiming in Jesse. > On Oct 13, 2016, at 08:08, Jesse McGraw wrote: > > Lee, > > Check out the setup.sh script, hopefully it does everything necessary to get the script working on a Debian-derived Linux system > > I've attempted to make the only globally-installed dependencies be cpanm and carton. Once those are installed it uses carton to install the dependencies locally > > > On 10/12/2016 07:59 PM, Lee wrote: >> On 10/12/16, Jason Hellenthal wrote: >>> Give these a shot. https://github.com/jlmcgraw/networkUtilities >>> >>> I know J could use a little feedback on those as well but all in all they >>> are pretty solid. >> Where does one get Modern/Perl.pm ? >> >> Can't locate Modern/Perl.pm in @INC (you may need to install the >> Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 >> /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int >> /usr/lib/perl5/site_perl/5.22 >> /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int >> /usr/lib/perl5/vendor_perl/5.22 >> /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) >> at /tmp/iosToHtml.pl line 87. >> BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. >> >> Lee >> >> >> >>>> On Oct 11, 2016, at 08:48, Lee wrote: >>>> >>>> On 10/10/16, Jay Hennigan wrote: >>>>> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>>>>> Nanog, >>>>>> >>>>>> (This is me scratching an itch of my own and hoping that sharing it >>>>>> might be useful to others on this list. Apologies if it isn't) >>>>>> >>>>>> When I'm trying to comprehend a new or complicated Cisco router, >>>>>> switch or firewall configuration an old pet-peeve of mine is how >>>>>> needlessly difficult it is to follow deeply nested logic in route-maps, >>>>>> ACLs, QoS policy-maps etc etc >>>>>> >>>>>> To make this a bit simpler I?ve been working on a perl script to >>>>>> convert >>>>>> these text-based configuration files into HTML with links between the >>>>>> different elements (e.g. To an access-list from the interface where >>>>>> it?s >>>>>> applied, from policy-maps to class-maps etc), hopefully making it >>>>>> easier >>>>>> to to follow the chain of logic via clicking links and using the >>>>>> forward >>>>>> and back buttons in your browser to go back and forth between command >>>>>> and referenced list. >>>>> Way cool. Now to hook it into RANCID.... >>>> It looks like what I did in 2.3.8 should still work - control_rancid >>>> puts the diff output into $TMP.diff so add this bit: >>>> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ >>>> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } >>>> printf("%s ", $2) >>>> } >>>> END{ printf("\n") } >>>> ' >$TMP.doit >>>> /bin/sh $TMP.doit >$TMP.out >>>> if [ -s $TMP.out ] ; then >>>> .. send mail / whatever >>>> rm $TMP.doit $TMP.out >>>> fi >>>> >>>> Regards, >>>> Lee >>> >>> -- >>> Jason Hellenthal >>> JJH48-ARIN >> . >> > -- Jason Hellenthal JJH48-ARIN From joly at punkcast.com Thu Oct 13 14:12:19 2016 From: joly at punkcast.com (Joly MacFie) Date: Thu, 13 Oct 2016 10:12:19 -0400 Subject: DEC-IX Summit New York livestream Message-ID: https://livestream.com/internetsociety/de-cix -- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast -------------------------------------------------------------- - From ler762 at gmail.com Thu Oct 13 16:38:55 2016 From: ler762 at gmail.com (Lee) Date: Thu, 13 Oct 2016 12:38:55 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> Message-ID: On 10/13/16, Jesse McGraw wrote: > Lee, > > Check out the setup.sh script, hopefully it does everything necessary > to get the script working on a Debian-derived Linux system I'm using Windows + Cygwin; maybe it's just that I don't have them installed, but there is no sudo or apt so setup.sh isn't going to work for me. So while I was interested in seeing what this bit looked like > If you run it against multiple configuration files at once it will also attempt to link > between them when applicable (e.g. BGP neighbors, route next hops, interfaces > on the same subnet etc). I'm not willing to take any more time on this. I appreciate all the people who've tried to help but at least for now, I'm done. Thanks, Lee > > I've attempted to make the only globally-installed dependencies be cpanm > and carton. Once those are installed it uses carton to install the > dependencies locally > > > On 10/12/2016 07:59 PM, Lee wrote: >> On 10/12/16, Jason Hellenthal wrote: >>> Give these a shot. https://github.com/jlmcgraw/networkUtilities >>> >>> I know J could use a little feedback on those as well but all in all >>> they >>> are pretty solid. >> Where does one get Modern/Perl.pm ? >> >> Can't locate Modern/Perl.pm in @INC (you may need to install the >> Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 >> /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int >> /usr/lib/perl5/site_perl/5.22 >> /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int >> /usr/lib/perl5/vendor_perl/5.22 >> /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) >> at /tmp/iosToHtml.pl line 87. >> BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. >> >> Lee >> >> >> >>>> On Oct 11, 2016, at 08:48, Lee wrote: >>>> >>>> On 10/10/16, Jay Hennigan wrote: >>>>> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>>>>> Nanog, >>>>>> >>>>>> (This is me scratching an itch of my own and hoping that sharing >>>>>> it >>>>>> might be useful to others on this list. Apologies if it isn't) >>>>>> >>>>>> When I'm trying to comprehend a new or complicated Cisco router, >>>>>> switch or firewall configuration an old pet-peeve of mine is how >>>>>> needlessly difficult it is to follow deeply nested logic in >>>>>> route-maps, >>>>>> ACLs, QoS policy-maps etc etc >>>>>> >>>>>> To make this a bit simpler I?ve been working on a perl script to >>>>>> convert >>>>>> these text-based configuration files into HTML with links between the >>>>>> different elements (e.g. To an access-list from the interface where >>>>>> it?s >>>>>> applied, from policy-maps to class-maps etc), hopefully making it >>>>>> easier >>>>>> to to follow the chain of logic via clicking links and using the >>>>>> forward >>>>>> and back buttons in your browser to go back and forth between command >>>>>> and referenced list. >>>>> Way cool. Now to hook it into RANCID.... >>>> It looks like what I did in 2.3.8 should still work - control_rancid >>>> puts the diff output into $TMP.diff so add this bit: >>>> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ >>>> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } >>>> printf("%s ", $2) >>>> } >>>> END{ printf("\n") } >>>> ' >$TMP.doit >>>> /bin/sh $TMP.doit >$TMP.out >>>> if [ -s $TMP.out ] ; then >>>> .. send mail / whatever >>>> rm $TMP.doit $TMP.out >>>> fi >>>> >>>> Regards, >>>> Lee >>> >>> -- >>> Jason Hellenthal >>> JJH48-ARIN >> . >> > > From voytek at trustdarkness.com Thu Oct 13 16:57:47 2016 From: voytek at trustdarkness.com (voytek) Date: Thu, 13 Oct 2016 11:57:47 -0500 Subject: Level 3 voice outage In-Reply-To: References: <068976b1-7367-4bc5-0d44-8f78f9605f4d@monmouth.com> <6F635755-3A2C-4AEE-A61E-FE9D423F9694@dataix.net> <17035.1475606577@turing-police.cc.vt.edu> <65582D28-14AE-40CB-A4C0-95DFE2366A60@beckman.org> <9FE75C96-C78E-4DA5-BD06-A2126E6F45A9@beckman.org> <6eafdfcea8b046e1ac8d3608f5dbebfc@Warner1202.warner.local> Message-ID: Can anyone who was affected by last week's outage confirm that 911 services were impacted (I assume they were)? Anyone know if the current outage is in any way related to this one from last week? http://downdetector.com/status/level3/map/ On 10/05/2016 01:24 PM, Mel Beckman wrote: > It?s good to see them acknowledging this. > > -mel > > On Oct 5, 2016, at 10:10 AM, Gareth Tupper > wrote: > > Looks like a fat finger event... > > From Level 3: > "On October 4, our voice network experienced a service disruption affecting some of our customers in North America due to a configuration error. We know how important these services are to our customers. As an organization, we're putting processes in place to prevent issues like this from recurring in the future. We were able to restore all services by 9:31am Mountain time." > > > http://www.theregister.co.uk/2016/10/05/level3_voip_blackout_cause/ > > > > -----Original Message----- > From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Mel Beckman > Sent: Tuesday, October 4, 2016 1:09 PM > To: Marco Teixeira > > Cc: Shawn Ritchie >; NANOG list > > Subject: Re: Level 3 voice outage > > Possibly somebody YANGed when they should have yinged :) > > -mel beckman > > On Oct 4, 2016, at 1:06 PM, Marco Teixeira > wrote: > > Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There are so many things that can go wrong on complex networks even with all the controls imposed on changes... > > > > > On Tue, Oct 4, 2016 at 8:54 PM, Shawn Ritchie > wrote: > Well, Level3 has by no means said that this was the result of a DDoS, that's just speculation on behalf of folks who do not work at Level3 so far. > > On Tue, Oct 4, 2016 at 2:49 PM Marco Teixeira > wrote: > I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge. Now... there where times where one malformed IP packet would cause a memory leak leading to a router reboot... :)? > > > > > On Tue, Oct 4, 2016 at 8:23 PM, Mel Beckman > wrote: > > 765 Gbps per second directed at a router's interface IP might give the > router pause, so to speak :) > > -mel > > On Oct 4, 2016, at 12:10 PM, Marco Teixeira > > > wrote: > > Multiple reboots across several markets... Does not seem something > that full pipes would trigger. Had it been an approved chance it would > have been rolled back i guess... On the other hand, a zero day could apply... > > Em 04/10/2016 19:54, "Mel Beckman" > escreveu: > > Sure. The recent release of the IoT DDoS attack code in the wild. > > -mel > > On Oct 4, 2016, at 11:42 AM, Valdis.Kletnieks at vt.edu wrote: > > On Tue, 04 Oct 2016 18:14:54 -0000, Mel Beckman said: > > This could be DoS attack. > > Or a missing comma in a code update. > > Or a fumble-fingered NOC monkey. > > Or.... > > You have any reason to suspect a DoS attack rather than all the > other possibilities? > > > > -- > > -- > Shawn > > > > This electronic mail transmission contains information from Warner Pacific Insurance Services that may be confidential or privileged. Such information is solely for the intended recipient, and use by any other party is not authorized. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this message, its contents or any attachments is prohibited. Any wrongful interception of this message is punishable as a Federal Crime. If you have received this message in error, please notify the sender immediately by telephone (800) 801-2300 or by electronic mail at postmaster at warnerpacific.com. > From hank at efes.iucc.ac.il Thu Oct 13 17:00:01 2016 From: hank at efes.iucc.ac.il (Hank Nussbacher) Date: Thu, 13 Oct 2016 20:00:01 +0300 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> Message-ID: <225bf92f-3c0e-4c9e-6f39-9bd6f16037c6@efes.iucc.ac.il> On 13/10/2016 19:38, Lee wrote: > On 10/13/16, Jesse McGraw wrote: >> Lee, >> >> Check out the setup.sh script, hopefully it does everything necessary >> to get the script working on a Debian-derived Linux system > I'm using Windows + Cygwin; maybe it's just that I don't have them > installed, but there is no sudo or apt so setup.sh isn't going to work > for me. So while I was interested in seeing what this bit looked like Have you tried Bash on Windows 10: http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/ http://www.pcworld.com/article/3106463/windows/how-to-get-bash-on-windows-10-with-the-anniversary-update.html -Hank >> If you run it against multiple configuration files at once it will also attempt to link >> between them when applicable (e.g. BGP neighbors, route next hops, interfaces >> on the same subnet etc). > I'm not willing to take any more time on this. > > I appreciate all the people who've tried to help but at least for now, I'm done. > > Thanks, > Lee > From jlmcgraw at gmail.com Thu Oct 13 17:04:33 2016 From: jlmcgraw at gmail.com (Jesse McGraw) Date: Thu, 13 Oct 2016 13:04:33 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension In-Reply-To: References: <878c4ccb-4d1b-dbb4-097f-a879e241ac01@gmail.com> <1520bc48-0580-7611-bbb2-f770489d7b87@west.net> <1105C1FE-4686-41F0-B484-325D5C714203@dataix.net> <590522cc-6a52-f152-50eb-e13d7e2442f7@gmail.com> Message-ID: Lee, FWIW, the script will work under straight Windows and I use it there frequently. I think Strawberry perl comes with cpanm (cpanminus) pre-installed so you can do: "cpanm Carton" and then cd to wherever you've got the script saved and do: "carton install" to install the dependencies Or, if you've got a set of configs with nothing sensitive/private left in them, try the simple web version I set up: https://hidden-waters-8218.herokuapp.com/ If I had a Windows VM setup I'd come up with a setup.bat On 10/13/2016 12:38 PM, Lee wrote: > On 10/13/16, Jesse McGraw wrote: >> Lee, >> >> Check out the setup.sh script, hopefully it does everything necessary >> to get the script working on a Debian-derived Linux system > I'm using Windows + Cygwin; maybe it's just that I don't have them > installed, but there is no sudo or apt so setup.sh isn't going to work > for me. So while I was interested in seeing what this bit looked like >> If you run it against multiple configuration files at once it will also attempt to link >> between them when applicable (e.g. BGP neighbors, route next hops, interfaces >> on the same subnet etc). > I'm not willing to take any more time on this. > > I appreciate all the people who've tried to help but at least for now, I'm done. > > Thanks, > Lee > > >> I've attempted to make the only globally-installed dependencies be cpanm >> and carton. Once those are installed it uses carton to install the >> dependencies locally >> >> >> On 10/12/2016 07:59 PM, Lee wrote: >>> On 10/12/16, Jason Hellenthal wrote: >>>> Give these a shot. https://github.com/jlmcgraw/networkUtilities >>>> >>>> I know J could use a little feedback on those as well but all in all >>>> they >>>> are pretty solid. >>> Where does one get Modern/Perl.pm ? >>> >>> Can't locate Modern/Perl.pm in @INC (you may need to install the >>> Modern::Perl module) (@INC contains: /tmp/local/lib/perl5 >>> /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int >>> /usr/lib/perl5/site_perl/5.22 >>> /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int >>> /usr/lib/perl5/vendor_perl/5.22 >>> /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) >>> at /tmp/iosToHtml.pl line 87. >>> BEGIN failed--compilation aborted at /tmp/iosToHtml.pl line 87. >>> >>> Lee >>> >>> >>> >>>>> On Oct 11, 2016, at 08:48, Lee wrote: >>>>> >>>>> On 10/10/16, Jay Hennigan wrote: >>>>>> On 10/6/16 1:26 PM, Jesse McGraw wrote: >>>>>>> Nanog, >>>>>>> >>>>>>> (This is me scratching an itch of my own and hoping that sharing >>>>>>> it >>>>>>> might be useful to others on this list. Apologies if it isn't) >>>>>>> >>>>>>> When I'm trying to comprehend a new or complicated Cisco router, >>>>>>> switch or firewall configuration an old pet-peeve of mine is how >>>>>>> needlessly difficult it is to follow deeply nested logic in >>>>>>> route-maps, >>>>>>> ACLs, QoS policy-maps etc etc >>>>>>> >>>>>>> To make this a bit simpler I?ve been working on a perl script to >>>>>>> convert >>>>>>> these text-based configuration files into HTML with links between the >>>>>>> different elements (e.g. To an access-list from the interface where >>>>>>> it?s >>>>>>> applied, from policy-maps to class-maps etc), hopefully making it >>>>>>> easier >>>>>>> to to follow the chain of logic via clicking links and using the >>>>>>> forward >>>>>>> and back buttons in your browser to go back and forth between command >>>>>>> and referenced list. >>>>>> Way cool. Now to hook it into RANCID.... >>>>> It looks like what I did in 2.3.8 should still work - control_rancid >>>>> puts the diff output into $TMP.diff so add this bit: >>>>> grep "^Index: " $TMP.diff | awk '/^Index: configs/{ >>>>> if ( ! got1 ) { printf("/usr/local/bin/myscript.sh "); got1=1; } >>>>> printf("%s ", $2) >>>>> } >>>>> END{ printf("\n") } >>>>> ' >$TMP.doit >>>>> /bin/sh $TMP.doit >$TMP.out >>>>> if [ -s $TMP.out ] ; then >>>>> .. send mail / whatever >>>>> rm $TMP.doit $TMP.out >>>>> fi >>>>> >>>>> Regards, >>>>> Lee >>>> -- >>>> Jason Hellenthal >>>> JJH48-ARIN >>> . >>> >> From eamon at eamonbauman.com Thu Oct 13 14:26:57 2016 From: eamon at eamonbauman.com (Eamon Bauman) Date: Thu, 13 Oct 2016 09:26:57 -0500 Subject: Excessive Netflix DNS Traffic? Message-ID: Hi all, Is anyone seeing excessive DNS traffic from game consoles (Xbox One, PS4) running Netflix? Starting 9/29 we have been seeing significant volume of DNS traffic from game consoles on our campus to our caching recursive boxes. Logs show repeated requests for api-global.netflix.com and nrdp.nccp.netflix.com. Anyone else experiencing this? Eamon From rar at syssrc.com Thu Oct 13 17:48:18 2016 From: rar at syssrc.com (rar) Date: Thu, 13 Oct 2016 17:48:18 +0000 Subject: Two BGP peering sessions on single Comcast Fiber Connection? Message-ID: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> After a many month wait, we were ready to turn up our BGP peering sessions on a new Comcast fiber connection. With our other providers (Level 3 and Verizon) we have edge routers that directly connect between the provider's on premise connection and our primary and a backup core routers. Each core router has a multihop BGP session with the provider's BGP router. The goal is to keep the single BGP router from being a single point of failure. Comcast said they could not support two separate BGP peering sessions on the same circuit. Does anyone have any counter examples? We used to have this setup with Comcast 5+ years ago, but now they say they can't support it. Bob Roswell broswell at syssrc.com 410-771-5544 ext 4336 Computer Museum Highlights From sryan at arbor.net Thu Oct 13 18:01:30 2016 From: sryan at arbor.net (Ryan, Spencer) Date: Thu, 13 Oct 2016 18:01:30 +0000 Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: I was going to point you to the reddit thread about it, but it looks to be your thread :) Spencer Ryan | Senior Systems Administrator | sryan at arbor.net Arbor Networks +1.734.794.5033 (d) | +1.734.846.2053 (m) www.arbornetworks.com ________________________________ From: NANOG on behalf of Eamon Bauman Sent: Thursday, October 13, 2016 10:26:57 AM To: nanog at nanog.org Subject: Excessive Netflix DNS Traffic? Hi all, Is anyone seeing excessive DNS traffic from game consoles (Xbox One, PS4) running Netflix? Starting 9/29 we have been seeing significant volume of DNS traffic from game consoles on our campus to our caching recursive boxes. Logs show repeated requests for api-global.netflix.com and nrdp.nccp.netflix.com. Anyone else experiencing this? Eamon From mpoublon at secantnet.net Thu Oct 13 19:04:29 2016 From: mpoublon at secantnet.net (Mike Poublon) Date: Thu, 13 Oct 2016 15:04:29 -0400 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> Message-ID: <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> I started a thread around the same topic back on 10/16 of 2014. A Comcast engineer (who ultimately spoke to the national product manager) came back after discussing and said the same thing "We don't support that". I got a slightly longer explanation of: -------------------------------------------- In a nutshell, when we design a product we do it to accommodate the most typical customer cases. Given that the design includes a single fiber path and thus the fiber path and device that terminates on either end each are a single point of failure, adding extra BGP sessions doesn?t seem to add value in the typical failure scenarios. In order to achieve the simplest and most scalable solution to address the market, we rely on narrowing the possible combinations of parameters. -------------------------------------------- I explained to them that their interpretation prevents me from being able to do concurrent maintenance on my side (single router reboot/upgrade, etc). Never got anywhere with it though. I'm still interested in having this set up, but have given up on it ever really coming to reality. Luckily ALL of my other providers were more than happy to set up an extra session. If anyone from Comcast is listening, there is customer demand for this. It's not about making it better for Comcast, it's about allowing customers to have more flexibility. Mike Poublon /Senior Datacenter Network Engineer/ *Secant Technologies* 6395 Technology Ave. Suite A Kalamazoo, MI 49009 On 10/13/2016 1:48 PM, rar wrote: > After a many month wait, we were ready to turn up our BGP peering sessions on a new Comcast fiber connection. > > With our other providers (Level 3 and Verizon) we have edge routers that directly connect between the provider's on premise connection and our primary and a backup core routers. Each core router has a multihop BGP session with the provider's BGP router. The goal is to keep the single BGP router from being a single point of failure. > > Comcast said they could not support two separate BGP peering sessions on the same circuit. Does anyone have any counter examples? We used to have this setup with Comcast 5+ years ago, but now they say they can't support it. > > > Bob Roswell > broswell at syssrc.com > 410-771-5544 ext 4336 > > Computer Museum Highlights > From dovid at telecurve.com Thu Oct 13 19:42:37 2016 From: dovid at telecurve.com (Dovid Bender) Date: Thu, 13 Oct 2016 15:42:37 -0400 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> Message-ID: Whenever we set up a bgp peer we do that to minimize downtime when doing maint. It's hit or miss. HE required a second physicall connection NTT was more than accommodating. On Oct 13, 2016 15:06, "Mike Poublon" wrote: > I started a thread around the same topic back on 10/16 of 2014. A Comcast > engineer (who ultimately spoke to the national product manager) came back > after discussing and said the same thing "We don't support that". I got a > slightly longer explanation of: > > -------------------------------------------- > > In a nutshell, when we design a product we do it to accommodate the most > typical customer cases. > Given that the design includes a single fiber path and thus the fiber path > and device that terminates on either end each are a single point of > failure, adding extra BGP sessions doesn?t seem to add value in the typical > failure scenarios. In order to achieve the simplest and most scalable > solution to address the market, we rely on narrowing the possible > combinations of parameters. > > -------------------------------------------- > > I explained to them that their interpretation prevents me from being able > to do concurrent maintenance on my side (single router reboot/upgrade, > etc). Never got anywhere with it though. > > I'm still interested in having this set up, but have given up on it ever > really coming to reality. Luckily ALL of my other providers were more than > happy to set up an extra session. > > If anyone from Comcast is listening, there is customer demand for this. > It's not about making it better for Comcast, it's about allowing customers > to have more flexibility. > > Mike Poublon > > /Senior Datacenter Network Engineer/ > > *Secant Technologies* > > 6395 Technology Ave. Suite A > > Kalamazoo, MI 49009 > > On 10/13/2016 1:48 PM, rar wrote: > >> After a many month wait, we were ready to turn up our BGP peering >> sessions on a new Comcast fiber connection. >> >> With our other providers (Level 3 and Verizon) we have edge routers that >> directly connect between the provider's on premise connection and our >> primary and a backup core routers. Each core router has a multihop BGP >> session with the provider's BGP router. The goal is to keep the single BGP >> router from being a single point of failure. >> >> Comcast said they could not support two separate BGP peering sessions on >> the same circuit. Does anyone have any counter examples? We used to have >> this setup with Comcast 5+ years ago, but now they say they can't support >> it. >> >> >> Bob Roswell >> broswell at syssrc.com >> 410-771-5544 ext 4336 >> >> Computer Museum Highlights >> >> > From jk at ip-clear.de Thu Oct 13 19:59:29 2016 From: jk at ip-clear.de (=?utf-8?q?J=C3=B6rg?= Kost) Date: Thu, 13 Oct 2016 21:59:29 +0200 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> Message-ID: <18DF9B15-9B64-445F-A40D-3A360045F87F@ip-clear.de> On 13 Oct 2016, at 19:48, rar wrote: > Comcast said they could not support two separate BGP peering sessions > on the same circuit. Does anyone have any counter examples? We used > to have this setup with Comcast 5+ years ago, but now they say they > can't support it. > So how do they connect ip6 sessions? ;-) J?rg From sryan at arbor.net Thu Oct 13 20:18:02 2016 From: sryan at arbor.net (Ryan, Spencer) Date: Thu, 13 Oct 2016 20:18:02 +0000 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <18DF9B15-9B64-445F-A40D-3A360045F87F@ip-clear.de> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com>, <18DF9B15-9B64-445F-A40D-3A360045F87F@ip-clear.de> Message-ID: Run your IPv4 peer to one router and IPv6 to another. Boom, redundancy! Spencer Ryan | Senior Systems Administrator | sryan at arbor.net Arbor Networks +1.734.794.5033 (d) | +1.734.846.2053 (m) www.arbornetworks.com ________________________________ From: NANOG on behalf of J?rg Kost Sent: Thursday, October 13, 2016 3:59:29 PM To: rar Cc: nanog at nanog.org Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? On 13 Oct 2016, at 19:48, rar wrote: > Comcast said they could not support two separate BGP peering sessions > on the same circuit. Does anyone have any counter examples? We used > to have this setup with Comcast 5+ years ago, but now they say they > can't support it. > So how do they connect ip6 sessions? ;-) J?rg From dsp at fb.com Thu Oct 13 23:22:08 2016 From: dsp at fb.com (Doug Porter) Date: Thu, 13 Oct 2016 23:22:08 +0000 Subject: Anyone from Facebook here? In-Reply-To: <20161011220935.2989156610D1@rock.dv.isc.org> References: <20161011220935.2989156610D1@rock.dv.isc.org> Message-ID: <0EF916E814DB5942B1E5360D60B23704D5C5538A@PRN-MBX01-2.TheFacebook.com> > You may want to follow up on this email thread. > IPv6 vs IPv4 performance to m.facebook.com. Mark: Thanks. We saw this on bind-users and are tracking in t13843732. -- dsp From josh at kyneticwifi.com Fri Oct 14 00:52:48 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Thu, 13 Oct 2016 19:52:48 -0500 Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: Same here :) On Oct 13, 2016 1:09 PM, "Ryan, Spencer" wrote: > I was going to point you to the reddit thread about it, but it looks to be > your thread :) > > > Spencer Ryan | Senior Systems Administrator | sryan at arbor.net sryan at arbor.net> > Arbor Networks > +1.734.794.5033 (d) | +1.734.846.2053 (m) > www.arbornetworks.com > > > ________________________________ > From: NANOG on behalf of Eamon Bauman < > eamon at eamonbauman.com> > Sent: Thursday, October 13, 2016 10:26:57 AM > To: nanog at nanog.org > Subject: Excessive Netflix DNS Traffic? > > Hi all, > > Is anyone seeing excessive DNS traffic from game consoles (Xbox One, PS4) > running Netflix? Starting 9/29 we have been seeing significant volume of > DNS traffic from game consoles on our campus to our caching recursive > boxes. Logs show repeated requests for api-global.netflix.com and > nrdp.nccp.netflix.com. > > Anyone else experiencing this? > > Eamon > From eamon at eamonbauman.com Fri Oct 14 15:43:03 2016 From: eamon at eamonbauman.com (Eamon Bauman) Date: Fri, 14 Oct 2016 10:43:03 -0500 Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: We're rate limiting it now, but it's definitely bad behavior. When I open the flood gates, over a 5-min sample from a single host I received well over 61,000 queries. The size of the records being requested cause this to be an (unintended) amplification attack, as a 30Mbps inbound sum is getting amplified to 150-200Mbps outbound. On Thu, Oct 13, 2016 at 7:52 PM, Josh Reynolds wrote: > Same here :) > > On Oct 13, 2016 1:09 PM, "Ryan, Spencer" wrote: > >> I was going to point you to the reddit thread about it, but it looks to >> be your thread :) >> >> >> Spencer Ryan | Senior Systems Administrator | sryan at arbor.net> sryan at arbor.net> >> Arbor Networks >> +1.734.794.5033 (d) | +1.734.846.2053 (m) >> www.arbornetworks.com >> >> >> ________________________________ >> From: NANOG on behalf of Eamon Bauman < >> eamon at eamonbauman.com> >> Sent: Thursday, October 13, 2016 10:26:57 AM >> To: nanog at nanog.org >> Subject: Excessive Netflix DNS Traffic? >> >> Hi all, >> >> Is anyone seeing excessive DNS traffic from game consoles (Xbox One, PS4) >> running Netflix? Starting 9/29 we have been seeing significant volume of >> DNS traffic from game consoles on our campus to our caching recursive >> boxes. Logs show repeated requests for api-global.netflix.com and >> nrdp.nccp.netflix.com. >> >> Anyone else experiencing this? >> >> Eamon >> > From bicknell at ufp.org Fri Oct 14 16:47:33 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Fri, 14 Oct 2016 09:47:33 -0700 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> Message-ID: <20161014164733.GA5892@ussenterprise.ufp.org> In a message written on Thu, Oct 13, 2016 at 05:48:18PM +0000, rar wrote: > The goal is to keep the single BGP router from being a single point of failure. I don't really understand the failure analysis / uptime calculation. There is one router on the Comcast side, which is a single point of failure. There is one circuit to your prem, which is a single point of failure. To connect two routers on your end you must terminate the circuit in a switch, which is a single point of failure. And yet, in the face of all that somehow running two routers with two BGP sessions on your end increases your uptime? The only way that would even remotely make sense is if the routers in question were horribly broken / mismanaged so (had to be?) reboot(ed) on a regular basis. However if uptime is so important using gear with that property makes no sense! I'm pretty sure without actually doing the math that you'll be more reliable with a single quality router (elminiation of complexity), and that if you really need maximum uptime that you had better get a second circuit, on a diverse path, into a different router probably from a different carrier. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From contact at winterei.se Fri Oct 14 17:34:38 2016 From: contact at winterei.se (Paul S.) Date: Sat, 15 Oct 2016 02:34:38 +0900 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <20161014164733.GA5892@ussenterprise.ufp.org> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <20161014164733.GA5892@ussenterprise.ufp.org> Message-ID: +1, could not have said it better. On 10/15/2016 01:47 AM, Leo Bicknell wrote: > In a message written on Thu, Oct 13, 2016 at 05:48:18PM +0000, rar wrote: >> The goal is to keep the single BGP router from being a single point of failure. > I don't really understand the failure analysis / uptime calculation. > > There is one router on the Comcast side, which is a single point of > failure. > > There is one circuit to your prem, which is a single point of failure. > > To connect two routers on your end you must terminate the circuit > in a switch, which is a single point of failure. > > And yet, in the face of all that somehow running two routers with > two BGP sessions on your end increases your uptime? > > The only way that would even remotely make sense is if the routers > in question were horribly broken / mismanaged so (had to be?) reboot(ed) > on a regular basis. However if uptime is so important using gear > with that property makes no sense! > > I'm pretty sure without actually doing the math that you'll be more > reliable with a single quality router (elminiation of complexity), > and that if you really need maximum uptime that you had better get > a second circuit, on a diverse path, into a different router probably > from a different carrier. > From bblackford at gmail.com Fri Oct 14 17:48:35 2016 From: bblackford at gmail.com (Bill Blackford) Date: Fri, 14 Oct 2016 10:48:35 -0700 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <20161014164733.GA5892@ussenterprise.ufp.org> Message-ID: It comes down to sizing your failure domain. Any single upstream Transit alone means the failure domain is the whole site (making assumptions about your topology). As mentioned earlier, any single point of failure doesn't reduce your failure footprint and gives little in terms of redundancy. Now if you point that second router to a second provider, now you've reduced the size of your failure domain to a single router/Transit, not the whole site. -b On Fri, Oct 14, 2016 at 10:34 AM, Paul S. wrote: > +1, could not have said it better. > > > On 10/15/2016 01:47 AM, Leo Bicknell wrote: > >> In a message written on Thu, Oct 13, 2016 at 05:48:18PM +0000, rar wrote: >> >>> The goal is to keep the single BGP router from being a single point of >>> failure. >>> >> I don't really understand the failure analysis / uptime calculation. >> >> There is one router on the Comcast side, which is a single point of >> failure. >> >> There is one circuit to your prem, which is a single point of failure. >> >> To connect two routers on your end you must terminate the circuit >> in a switch, which is a single point of failure. >> >> And yet, in the face of all that somehow running two routers with >> two BGP sessions on your end increases your uptime? >> >> The only way that would even remotely make sense is if the routers >> in question were horribly broken / mismanaged so (had to be?) reboot(ed) >> on a regular basis. However if uptime is so important using gear >> with that property makes no sense! >> >> I'm pretty sure without actually doing the math that you'll be more >> reliable with a single quality router (elminiation of complexity), >> and that if you really need maximum uptime that you had better get >> a second circuit, on a diverse path, into a different router probably >> from a different carrier. >> >> > -- Bill Blackford Logged into reality and abusing my sudo privileges..... From cscora at apnic.net Fri Oct 14 18:01:27 2016 From: cscora at apnic.net (Routing Analysis Role Account) Date: Sat, 15 Oct 2016 04:01:27 +1000 (AEST) Subject: Weekly Routing Table Report Message-ID: <20161014180127.BE69BC55A1@thyme.apnic.net> This is an automated weekly mailing describing the state of the Internet Routing Table as seen from APNIC's router in Japan. The posting is sent to APOPS, NANOG, AfNOG, AusNOG, SANOG, PacNOG, SAFNOG, SdNOG, BJNOG, CaribNOG and the RIPE Routing WG. Daily listings are sent to bgp-stats at lists.apnic.net For historical data, please see http://thyme.rand.apnic.net. If you have any comments please contact Philip Smith . Routing Table Report 04:00 +10GMT Sat 15 Oct, 2016 Report Website: http://thyme.rand.apnic.net Detailed Analysis: http://thyme.rand.apnic.net/current/ Analysis Summary ---------------- BGP routing table entries examined: 613784 Prefixes after maximum aggregation (per Origin AS): 220540 Deaggregation factor: 2.78 Unique aggregates announced (without unneeded subnets): 299320 Total ASes present in the Internet Routing Table: 55015 Prefixes per ASN: 11.16 Origin-only ASes present in the Internet Routing Table: 36378 Origin ASes announcing only one prefix: 15350 Transit ASes present in the Internet Routing Table: 6526 Transit-only ASes present in the Internet Routing Table: 169 Average AS path length visible in the Internet Routing Table: 4.3 Max AS path length visible: 54 Max AS path prepend of ASN ( 55644) 51 Prefixes from unregistered ASNs in the Routing Table: 55 Unregistered ASNs in the Routing Table: 14 Number of 32-bit ASNs allocated by the RIRs: 15751 Number of 32-bit ASNs visible in the Routing Table: 12111 Prefixes from 32-bit ASNs in the Routing Table: 48852 Number of bogon 32-bit ASNs visible in the Routing Table: 213 Special use prefixes present in the Routing Table: 0 Prefixes being announced from unallocated address space: 363 Number of addresses announced to Internet: 2834164004 Equivalent to 168 /8s, 237 /16s and 233 /24s Percentage of available address space announced: 76.6 Percentage of allocated address space announced: 76.6 Percentage of available address space allocated: 100.0 Percentage of address space in use by end-sites: 98.3 Total number of prefixes smaller than registry allocations: 199675 APNIC Region Analysis Summary ----------------------------- Prefixes being announced by APNIC Region ASes: 156199 Total APNIC prefixes after maximum aggregation: 42794 APNIC Deaggregation factor: 3.65 Prefixes being announced from the APNIC address blocks: 170023 Unique aggregates announced from the APNIC address blocks: 69559 APNIC Region origin ASes present in the Internet Routing Table: 5187 APNIC Prefixes per ASN: 32.78 APNIC Region origin ASes announcing only one prefix: 1148 APNIC Region transit ASes present in the Internet Routing Table: 948 Average APNIC Region AS path length visible: 4.4 Max APNIC Region AS path length visible: 54 Number of APNIC region 32-bit ASNs visible in the Routing Table: 2411 Number of APNIC addresses announced to Internet: 761008452 Equivalent to 45 /8s, 92 /16s and 17 /24s APNIC AS Blocks 4608-4864, 7467-7722, 9216-10239, 17408-18431 (pre-ERX allocations) 23552-24575, 37888-38911, 45056-46079, 55296-56319, 58368-59391, 63488-64098, 64297-64395, 131072-137529 APNIC Address Blocks 1/8, 14/8, 27/8, 36/8, 39/8, 42/8, 43/8, 49/8, 58/8, 59/8, 60/8, 61/8, 101/8, 103/8, 106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8, 116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8, 123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8, 163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8, 203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8, 222/8, 223/8, ARIN Region Analysis Summary ---------------------------- Prefixes being announced by ARIN Region ASes: 184767 Total ARIN prefixes after maximum aggregation: 89684 ARIN Deaggregation factor: 2.06 Prefixes being announced from the ARIN address blocks: 190563 Unique aggregates announced from the ARIN address blocks: 88633 ARIN Region origin ASes present in the Internet Routing Table: 16173 ARIN Prefixes per ASN: 11.78 ARIN Region origin ASes announcing only one prefix: 5668 ARIN Region transit ASes present in the Internet Routing Table: 1710 Average ARIN Region AS path length visible: 3.8 Max ARIN Region AS path length visible: 23 Number of ARIN region 32-bit ASNs visible in the Routing Table: 1515 Number of ARIN addresses announced to Internet: 1109002400 Equivalent to 66 /8s, 26 /16s and 8 /24s ARIN AS Blocks 1-1876, 1902-2042, 2044-2046, 2048-2106 (pre-ERX allocations) 2138-2584, 2615-2772, 2823-2829, 2880-3153 3354-4607, 4865-5119, 5632-6655, 6912-7466 7723-8191, 10240-12287, 13312-15359, 16384-17407 18432-20479, 21504-23551, 25600-26591, 26624-27647, 29696-30719, 31744-33791 35840-36863, 39936-40959, 46080-47103 53248-55295, 62464-63487, 64198-64296, 393216-397212 ARIN Address Blocks 3/8, 4/8, 6/8, 7/8, 8/8, 9/8, 11/8, 12/8, 13/8, 15/8, 16/8, 17/8, 18/8, 19/8, 20/8, 21/8, 22/8, 23/8, 24/8, 26/8, 28/8, 29/8, 30/8, 32/8, 33/8, 34/8, 35/8, 38/8, 40/8, 44/8, 45/8, 47/8, 48/8, 50/8, 52/8, 53/8, 54/8, 55/8, 56/8, 57/8, 63/8, 64/8, 65/8, 66/8, 67/8, 68/8, 69/8, 70/8, 71/8, 72/8, 73/8, 74/8, 75/8, 76/8, 96/8, 97/8, 98/8, 99/8, 100/8, 104/8, 107/8, 108/8, 128/8, 129/8, 130/8, 131/8, 132/8, 134/8, 135/8, 136/8, 137/8, 138/8, 139/8, 140/8, 142/8, 143/8, 144/8, 146/8, 147/8, 148/8, 149/8, 152/8, 155/8, 156/8, 157/8, 158/8, 159/8, 160/8, 161/8, 162/8, 164/8, 165/8, 166/8, 167/8, 168/8, 169/8, 170/8, 172/8, 173/8, 174/8, 184/8, 192/8, 198/8, 199/8, 204/8, 205/8, 206/8, 207/8, 208/8, 209/8, 214/8, 215/8, 216/8, RIPE Region Analysis Summary ---------------------------- Prefixes being announced by RIPE Region ASes: 146414 Total RIPE prefixes after maximum aggregation: 72087 RIPE Deaggregation factor: 2.03 Prefixes being announced from the RIPE address blocks: 157060 Unique aggregates announced from the RIPE address blocks: 97010 RIPE Region origin ASes present in the Internet Routing Table: 18147 RIPE Prefixes per ASN: 8.65 RIPE Region origin ASes announcing only one prefix: 7815 RIPE Region transit ASes present in the Internet Routing Table: 3036 Average RIPE Region AS path length visible: 4.4 Max RIPE Region AS path length visible: 27 Number of RIPE region 32-bit ASNs visible in the Routing Table: 5070 Number of RIPE addresses announced to Internet: 708682496 Equivalent to 42 /8s, 61 /16s and 163 /24s RIPE AS Blocks 1877-1901, 2043, 2047, 2107-2136, 2585-2614 (pre-ERX allocations) 2773-2822, 2830-2879, 3154-3353, 5377-5631 6656-6911, 8192-9215, 12288-13311, 15360-16383 20480-21503, 24576-25599, 28672-29695 30720-31743, 33792-35839, 38912-39935 40960-45055, 47104-52223, 56320-58367 59392-61439, 61952-62463, 64396-64495 196608-207259 RIPE Address Blocks 2/8, 5/8, 25/8, 31/8, 37/8, 46/8, 51/8, 62/8, 77/8, 78/8, 79/8, 80/8, 81/8, 82/8, 83/8, 84/8, 85/8, 86/8, 87/8, 88/8, 89/8, 90/8, 91/8, 92/8, 93/8, 94/8, 95/8, 109/8, 141/8, 145/8, 151/8, 176/8, 178/8, 185/8, 188/8, 193/8, 194/8, 195/8, 212/8, 213/8, 217/8, LACNIC Region Analysis Summary ------------------------------ Prefixes being announced by LACNIC Region ASes: 62690 Total LACNIC prefixes after maximum aggregation: 12636 LACNIC Deaggregation factor: 4.96 Prefixes being announced from the LACNIC address blocks: 78801 Unique aggregates announced from the LACNIC address blocks: 37403 LACNIC Region origin ASes present in the Internet Routing Table: 2482 LACNIC Prefixes per ASN: 31.75 LACNIC Region origin ASes announcing only one prefix: 546 LACNIC Region transit ASes present in the Internet Routing Table: 583 Average LACNIC Region AS path length visible: 4.9 Max LACNIC Region AS path length visible: 22 Number of LACNIC region 32-bit ASNs visible in the Routing Table: 2849 Number of LACNIC addresses announced to Internet: 170648128 Equivalent to 10 /8s, 43 /16s and 226 /24s LACNIC AS Blocks 26592-26623, 27648-28671, 52224-53247, 61440-61951, 64099-64197, 262144-266652 + ERX transfers LACNIC Address Blocks 177/8, 179/8, 181/8, 186/8, 187/8, 189/8, 190/8, 191/8, 200/8, 201/8, AfriNIC Region Analysis Summary ------------------------------- Prefixes being announced by AfriNIC Region ASes: 14806 Total AfriNIC prefixes after maximum aggregation: 3331 AfriNIC Deaggregation factor: 4.44 Prefixes being announced from the AfriNIC address blocks: 16974 Unique aggregates announced from the AfriNIC address blocks: 6385 AfriNIC Region origin ASes present in the Internet Routing Table: 740 AfriNIC Prefixes per ASN: 22.94 AfriNIC Region origin ASes announcing only one prefix: 173 AfriNIC Region transit ASes present in the Internet Routing Table: 181 Average AfriNIC Region AS path length visible: 4.7 Max AfriNIC Region AS path length visible: 18 Number of AfriNIC region 32-bit ASNs visible in the Routing Table: 266 Number of AfriNIC addresses announced to Internet: 84339968 Equivalent to 5 /8s, 6 /16s and 237 /24s AfriNIC AS Blocks 36864-37887, 327680-328703 & ERX transfers AfriNIC Address Blocks 41/8, 102/8, 105/8, 154/8, 196/8, 197/8, APNIC Region per AS prefix count summary ---------------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5542 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3535 383 252 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2950 904 78 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9829 2700 1497 529 BSNL-NIB National Internet Backbone, IN 4766 2662 11127 736 KIXS-AS-KR Korea Telecom, KR 9808 2201 8781 42 CMNET-GD Guangdong Mobile Communication 4755 2054 429 217 TATACOMM-AS TATA Communications formerl 4808 1768 2290 519 CHINA169-BJ China Unicom Beijing Provin 45899 1591 1235 89 VNPT-AS-VN VNPT Corp, VN 24560 1551 517 218 AIRTELBROADBAND-AS-AP Bharti Airtel Ltd Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-APNIC ARIN Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 22773 3534 2964 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2196 405 110 MEGAPATH5-US - MegaPath Corporation, US 6389 2098 3671 39 BELLSOUTH-NET-BLK - BellSouth.net Inc., 20115 1941 1988 405 CHARTER-NET-HKY-NC - Charter Communicat 30036 1732 334 332 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 209 1724 5068 657 CENTURYLINK-US-LEGACY-QWEST - Qwest Com 6983 1676 848 227 ITCDELTA - Earthlink, Inc., US 701 1608 10674 670 UUNET - MCI Communications Services, In 7018 1486 20492 1049 ATT-INTERNET4 - AT&T Services, Inc., US 16509 1427 2666 476 AMAZON-02 - Amazon.com, Inc., US Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-ARIN RIPE Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 39891 3329 169 15 ALJAWWALSTC-AS , SA 20940 2814 1053 2005 AKAMAI-ASN1 , US 34984 1980 326 355 TELLCOM-AS , TR 12479 1371 1018 46 UNI2-AS , ES 13188 1272 99 58 TRIOLAN , UA 8551 1205 377 46 BEZEQ-INTERNATIONAL-AS Bezeqint Interne 6849 1142 355 21 UKRTELNET , UA 8402 1092 544 15 CORBINA-AS Russia, RU 12389 921 1120 406 ROSTELECOM-AS , RU 6830 892 2755 467 LGI-UPC formerly known as UPC Broadband Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-RIPE LACNIC Region per AS prefix count summary ----------------------------------------- ASN No of nets /20 equiv MaxAgg Description 10620 3496 540 189 Telmex Colombia S.A., CO 8151 2319 3353 559 Uninet S.A. de C.V., MX 7303 1554 958 247 Telecom Argentina S.A., AR 6503 1420 437 54 Axtel, S.A.B. de C.V., MX 11830 1321 367 57 Instituto Costarricense de Electricidad 6147 1229 377 28 Telefonica del Peru S.A.A., PE 28573 1004 2259 165 CLARO S.A., BR 7738 994 1882 40 Telemar Norte Leste S.A., BR 3816 975 472 200 COLOMBIA TELECOMUNICACIONES S.A. ESP, C 11172 906 126 76 Alestra, S. de R.L. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-LACNIC AfriNIC Region per AS prefix count summary ------------------------------------------ ASN No of nets /20 equiv MaxAgg Description 24863 1181 402 50 LINKdotNET-AS, EG 36903 682 343 125 MT-MPLS, MA 37611 662 67 6 Afrihost, ZA 36992 565 1373 25 ETISALAT-MISR, EG 8452 516 1472 15 TE-AS TE-AS, EG 37492 386 250 69 ORANGE-TN, TN 24835 355 612 17 RAYA-AS, EG 29571 332 37 12 CITelecom-AS, CI 15399 308 35 7 WANANCHI-KE, KE 2018 269 327 74 TENET-1, ZA Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-AFRINIC Global Per AS prefix count summary ---------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5542 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3535 383 252 TPG-INTERNET-AP TPG Telecom Limited, AU 22773 3534 2964 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3496 540 189 Telmex Colombia S.A., CO 39891 3329 169 15 ALJAWWALSTC-AS , SA 17974 2950 904 78 TELKOMNET-AS2-AP PT Telekomunikasi Indo 20940 2814 1053 2005 AKAMAI-ASN1 , US 9829 2700 1497 529 BSNL-NIB National Internet Backbone, IN 4766 2662 11127 736 KIXS-AS-KR Korea Telecom, KR 8151 2319 3353 559 Uninet S.A. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet Global Per AS Maximum Aggr summary ---------------------------------- ASN No of nets Net Savings Description 22773 3534 3388 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 39891 3329 3314 ALJAWWALSTC-AS , SA 10620 3496 3307 Telmex Colombia S.A., CO 7545 3535 3283 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2950 2872 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9829 2700 2171 BSNL-NIB National Internet Backbone, IN 9808 2201 2159 CMNET-GD Guangdong Mobile Communication 18566 2196 2086 MEGAPATH5-US - MegaPath Corporation, US 6389 2098 2059 BELLSOUTH-NET-BLK - BellSouth.net Inc., 4766 2662 1926 KIXS-AS-KR Korea Telecom, KR Complete listing at http://thyme.rand.apnic.net/current/data-CIDRnet List of Unregistered Origin ASNs (Global) ----------------------------------------- Bad AS Designation Network Transit AS Description 65001 PRIVATE 5.143.176.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.200.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.208.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.216.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65000 PRIVATE 31.219.177.0/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65000 PRIVATE 31.219.177.128/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65512 PRIVATE 45.252.244.0/24 45899 VNPT-AS-VN VNPT Corp, VN 65512 PRIVATE 45.252.245.0/24 45899 VNPT-AS-VN VNPT Corp, VN Complete listing at http://thyme.rand.apnic.net/current/data-badAS Advertised Unallocated Addresses -------------------------------- Network Origin AS Description 23.249.144.0/20 40430 COLO4JAX-AS - colo4jax, LLC, US 27.100.7.0/24 56096 UNKNOWN 41.73.1.0/24 37004 -Reserved AS-, ZZ 41.73.2.0/24 37004 -Reserved AS-, ZZ 41.73.3.0/24 37004 -Reserved AS-, ZZ 41.73.4.0/24 37004 -Reserved AS-, ZZ 41.73.5.0/24 37004 -Reserved AS-, ZZ 41.73.6.0/24 37004 -Reserved AS-, ZZ 41.73.7.0/24 37004 -Reserved AS-, ZZ 41.73.8.0/24 37004 -Reserved AS-, ZZ Complete listing at http://thyme.rand.apnic.net/current/data-add-IANA Number of prefixes announced per prefix length (Global) ------------------------------------------------------- /1:0 /2:0 /3:0 /4:0 /5:0 /6:0 /7:0 /8:16 /9:13 /10:36 /11:103 /12:274 /13:523 /14:1052 /15:1780 /16:13190 /17:7833 /18:13031 /19:25447 /20:38848 /21:40360 /22:68377 /23:59883 /24:341808 /25:446 /26:342 /27:284 /28:58 /29:33 /30:13 /31:1 /32:33 Advertised prefixes smaller than registry allocations ----------------------------------------------------- ASN No of nets Total ann. Description 39891 2896 3329 ALJAWWALSTC-AS , SA 22773 2755 3534 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2088 2196 MEGAPATH5-US - MegaPath Corporation, US 30036 1546 1732 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 10620 1424 3496 Telmex Colombia S.A., CO 6389 1396 2098 BELLSOUTH-NET-BLK - BellSouth.net Inc., 6983 1325 1676 ITCDELTA - Earthlink, Inc., US 34984 1264 1980 TELLCOM-AS , TR 11492 1222 1314 CABLEONE - CABLE ONE, INC., US 13188 1046 1272 TRIOLAN , UA Complete listing at http://thyme.rand.apnic.net/current/data-sXXas-nos Number of /24s announced per /8 block (Global) ---------------------------------------------- 1:1539 2:804 4:23 5:2202 6:31 8:992 12:1794 13:47 14:1739 15:46 16:2 17:100 18:126 20:50 23:1849 24:1809 27:2355 31:1798 32:83 33:4 35:4 36:330 37:2331 38:1268 39:36 40:101 41:2981 42:461 43:1859 44:47 45:2175 46:2572 47:99 49:1211 50:909 51:13 52:558 53:2 54:350 55:7 56:4 57:40 58:1564 59:997 60:376 61:1821 62:1526 63:1931 64:4565 65:2180 66:4266 67:2254 68:1135 69:3281 70:1290 71:562 72:2081 74:2570 75:404 76:415 77:1421 78:1223 79:922 80:1308 81:1401 82:989 83:704 84:888 85:1577 86:483 87:1136 88:553 89:2090 90:221 91:6120 92:920 93:2345 94:2430 95:2490 96:566 97:345 98:894 99:89 100:147 101:1164 103:12588 104:2570 105:125 106:441 107:1435 108:783 109:2291 110:1281 111:1629 112:1076 113:1140 114:1087 115:1688 116:1673 117:1563 118:2052 119:1578 120:942 121:1075 122:2271 123:2021 124:1566 125:1817 128:678 129:442 130:420 131:1384 132:611 133:175 134:489 135:200 136:397 137:406 138:1780 139:428 140:602 141:451 142:672 143:922 144:737 145:167 146:950 147:655 148:1385 149:543 150:655 151:898 152:680 153:301 154:704 155:959 156:532 157:526 158:407 159:1210 160:527 161:736 162:2401 163:576 164:778 165:1121 166:337 167:1161 168:2268 169:664 170:2006 171:251 172:756 173:1749 174:768 175:671 176:1731 177:4121 178:2224 179:1203 180:2151 181:1916 182:2071 183:1006 184:833 185:7659 186:3202 187:2166 188:2211 189:1788 190:7835 191:1311 192:9056 193:5761 194:4472 195:3848 196:1790 197:1164 198:5581 199:5806 200:7157 201:3714 202:10135 203:9813 204:4510 205:2755 206:2997 207:3079 208:4060 209:3880 210:3887 211:2035 212:2724 213:2356 214:861 215:67 216:5727 217:1988 218:806 219:602 220:1634 221:878 222:693 223:1321 End of report From rjoffe at centergate.com Sat Oct 15 13:08:58 2016 From: rjoffe at centergate.com (Rodney Joffe) Date: Sat, 15 Oct 2016 09:08:58 -0400 Subject: 18 years ago today - rfc 2468 Message-ID: How time flies.... From rjoffe at centergate.com Sat Oct 15 13:19:06 2016 From: rjoffe at centergate.com (Rodney Joffe) Date: Sat, 15 Oct 2016 09:19:06 -0400 Subject: 18 years ago today - rfc 2468 In-Reply-To: References: Message-ID: To be clear - Oct 16. Which has just tolled in the APAC region. For most of you it will be tomorrow. But no matter. You get the point. > On Oct 15, 2016, at 9:08 AM, Rodney Joffe wrote: > > How time flies.... From patrick at ianai.net Sat Oct 15 13:21:01 2016 From: patrick at ianai.net (Patrick W. Gilmore) Date: Sat, 15 Oct 2016 09:21:01 -0400 Subject: 18 years ago today - rfc 2468 In-Reply-To: References: Message-ID: <2B6C190A-B731-4EE9-A767-68C050057D51@ianai.net> We do. Thank you for reminding us. And thanks to Dr. Postel for making what we do possible. -- TTFN, patrick > On Oct 15, 2016, at 9:19 AM, Rodney Joffe wrote: > > To be clear - Oct 16. Which has just tolled in the APAC region. For most of you it will be tomorrow. But no matter. You get the point. > >> On Oct 15, 2016, at 9:08 AM, Rodney Joffe wrote: >> >> How time flies.... From mianosm at gmail.com Sat Oct 15 13:23:08 2016 From: mianosm at gmail.com (Steven Miano) Date: Sat, 15 Oct 2016 09:23:08 -0400 Subject: 18 years ago today - rfc 2468 In-Reply-To: References: Message-ID: For those who are not sure: The significance of Jon Postel's contributions to building the Internet, both technical and personal, were such that a memorial recollection of his life forms part of the core technical literature sequence of the Internet in the form of RFC 2468 "I Remember IANA", written by Vinton Cerf. On Sat, Oct 15, 2016 at 9:08 AM, Rodney Joffe wrote: > How time flies.... > -- Miano, Steven M. http://stevenmiano.com From volists at staff.velocityonline.net Sat Oct 15 17:41:36 2016 From: volists at staff.velocityonline.net (Velocity Lists) Date: Sat, 15 Oct 2016 13:41:36 -0400 Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: We have seen it as well. In our cases it is all TCP DNS traffic as well. Velocity Online 850-205-4638 On Fri, Oct 14, 2016 at 11:43 AM, Eamon Bauman wrote: > We're rate limiting it now, but it's definitely bad behavior. When I open > the flood gates, over a 5-min sample from a single host I received well > over 61,000 queries. > The size of the records being requested cause this to be an (unintended) > amplification attack, as a 30Mbps inbound sum is getting amplified to > 150-200Mbps outbound. > > On Thu, Oct 13, 2016 at 7:52 PM, Josh Reynolds > wrote: > > > Same here :) > > > > On Oct 13, 2016 1:09 PM, "Ryan, Spencer" wrote: > > > >> I was going to point you to the reddit thread about it, but it looks to > >> be your thread :) > >> > >> > >> Spencer Ryan | Senior Systems Administrator | sryan at arbor.net >> sryan at arbor.net> > >> Arbor Networks > >> +1.734.794.5033 (d) | +1.734.846.2053 (m) > >> www.arbornetworks.com > >> > >> > >> ________________________________ > >> From: NANOG on behalf of Eamon Bauman < > >> eamon at eamonbauman.com> > >> Sent: Thursday, October 13, 2016 10:26:57 AM > >> To: nanog at nanog.org > >> Subject: Excessive Netflix DNS Traffic? > >> > >> Hi all, > >> > >> Is anyone seeing excessive DNS traffic from game consoles (Xbox One, > PS4) > >> running Netflix? Starting 9/29 we have been seeing significant volume of > >> DNS traffic from game consoles on our campus to our caching recursive > >> boxes. Logs show repeated requests for api-global.netflix.com and > >> nrdp.nccp.netflix.com. > >> > >> Anyone else experiencing this? > >> > >> Eamon > >> > > > From randy at psg.com Sun Oct 16 02:23:38 2016 From: randy at psg.com (Randy Bush) Date: Sun, 16 Oct 2016 11:23:38 +0900 Subject: 18 years ago today - rfc 2468 In-Reply-To: References: Message-ID: october, the month of deep sadness, jon, abha, itojun, ... From codygrosskopf at gmail.com Sun Oct 16 16:30:48 2016 From: codygrosskopf at gmail.com (Cody Grosskopf) Date: Sun, 16 Oct 2016 16:30:48 +0000 Subject: INFORM: Web-Based Speed Test Hackathon - Princeton, NJ in November In-Reply-To: References: Message-ID: It's completely possible I have overlooked but you call it an open source tool but provide no source? On Wed, Oct 12, 2016, 1:40 PM Livingood, Jason wrote: > May be of interest to folks on this list? > > Comcast is developing a new open source web-based speed test ? see > http://labs.comcast.com/beta-testing-a-new-open-source-speed-test. ISPs, > academic researchers, and others will likely be interested in using this > rather than older tools. One goal is to make it easy for any organization > to customize it for their use and UI, and we?ll soon find a neutral > non-Comcast home for the code. To kick this off we are sponsoring a > hackathon at Princeton University in November (the 3rd and 4th). > > Anyone can participate ? though participants should preferably be > developers or have development skills (the code is written in Node.JS with > some Angular components for the UI). > > All details at https://citp.princeton.edu/event/speedtest-hackathon/ and > we?re asking folks to click the RSVP link at the top to register ASAP. > > Regards, > > Jason Livingood > Comcast > From kraig at enguity.com Mon Oct 17 12:26:39 2016 From: kraig at enguity.com (Kraig Beahn) Date: Mon, 17 Oct 2016 12:26:39 +0000 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <20161014164733.GA5892@ussenterprise.ufp.org> Message-ID: Steering clear of the failure domain conversation, if its of any benefit - we can at least confirm that Comcast is willing to establish /29's for multiple BGP connections at 56 Marietta/ATL. These circuits are written on true wholesale/transit IP service contracts, which may be the difference. In our experience the Comcast Enterprise/Business groups have rather rigid circuit provisioning profiles, and even if you are able to talk an engineer into building a customer's configuration outside of their normal "scope", it usually comes back to haunt you at some point in the future, even if years later. Will send a link to the Comcast enterprise ip transit profiles separately, for reference, in the event you were not provided such previously...Or if Comcast wholesale is on the list, of course feel free to chime in too! On Fri, Oct 14, 2016, 1:49 PM Bill Blackford wrote: > It comes down to sizing your failure domain. Any single upstream Transit > alone means the failure domain is the whole site (making assumptions about > your topology). As mentioned earlier, any single point of failure doesn't > reduce your failure footprint and gives little in terms of redundancy. Now > if you point that second router to a second provider, now you've reduced > the size of your failure domain to a single router/Transit, not the whole > site. > > -b > > > On Fri, Oct 14, 2016 at 10:34 AM, Paul S. wrote: > > > +1, could not have said it better. > > > > > > On 10/15/2016 01:47 AM, Leo Bicknell wrote: > > > >> In a message written on Thu, Oct 13, 2016 at 05:48:18PM +0000, rar > wrote: > >> > >>> The goal is to keep the single BGP router from being a single point of > >>> failure. > >>> > >> I don't really understand the failure analysis / uptime calculation. > >> > >> There is one router on the Comcast side, which is a single point of > >> failure. > >> > >> There is one circuit to your prem, which is a single point of failure. > >> > >> To connect two routers on your end you must terminate the circuit > >> in a switch, which is a single point of failure. > >> > >> And yet, in the face of all that somehow running two routers with > >> two BGP sessions on your end increases your uptime? > >> > >> The only way that would even remotely make sense is if the routers > >> in question were horribly broken / mismanaged so (had to be?) reboot(ed) > >> on a regular basis. However if uptime is so important using gear > >> with that property makes no sense! > >> > >> I'm pretty sure without actually doing the math that you'll be more > >> reliable with a single quality router (elminiation of complexity), > >> and that if you really need maximum uptime that you had better get > >> a second circuit, on a diverse path, into a different router probably > >> from a different carrier. > >> > >> > > > > > -- > Bill Blackford > > Logged into reality and abusing my sudo privileges..... > -- From nanog at ics-il.net Mon Oct 17 12:30:38 2016 From: nanog at ics-il.net (Mike Hammett) Date: Mon, 17 Oct 2016 07:30:38 -0500 (CDT) Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> Message-ID: <213679615.3830.1476707434794.JavaMail.mhammett@ThunderFuck> It really seems like it's a grave oversight to *NOT* support multiple BGP sessions. I drop to two routers for that same reason, I can do maintenance on one, while the other carries traffic. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Mike Poublon" To: "rar" , nanog at nanog.org Sent: Thursday, October 13, 2016 2:04:29 PM Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? I started a thread around the same topic back on 10/16 of 2014. A Comcast engineer (who ultimately spoke to the national product manager) came back after discussing and said the same thing "We don't support that". I got a slightly longer explanation of: -------------------------------------------- In a nutshell, when we design a product we do it to accommodate the most typical customer cases. Given that the design includes a single fiber path and thus the fiber path and device that terminates on either end each are a single point of failure, adding extra BGP sessions doesn?t seem to add value in the typical failure scenarios. In order to achieve the simplest and most scalable solution to address the market, we rely on narrowing the possible combinations of parameters. -------------------------------------------- I explained to them that their interpretation prevents me from being able to do concurrent maintenance on my side (single router reboot/upgrade, etc). Never got anywhere with it though. I'm still interested in having this set up, but have given up on it ever really coming to reality. Luckily ALL of my other providers were more than happy to set up an extra session. If anyone from Comcast is listening, there is customer demand for this. It's not about making it better for Comcast, it's about allowing customers to have more flexibility. Mike Poublon /Senior Datacenter Network Engineer/ *Secant Technologies* 6395 Technology Ave. Suite A Kalamazoo, MI 49009 On 10/13/2016 1:48 PM, rar wrote: > After a many month wait, we were ready to turn up our BGP peering sessions on a new Comcast fiber connection. > > With our other providers (Level 3 and Verizon) we have edge routers that directly connect between the provider's on premise connection and our primary and a backup core routers. Each core router has a multihop BGP session with the provider's BGP router. The goal is to keep the single BGP router from being a single point of failure. > > Comcast said they could not support two separate BGP peering sessions on the same circuit. Does anyone have any counter examples? We used to have this setup with Comcast 5+ years ago, but now they say they can't support it. > > > Bob Roswell > broswell at syssrc.com > 410-771-5544 ext 4336 > > Computer Museum Highlights > From jason at unlimitednet.us Mon Oct 17 13:29:58 2016 From: jason at unlimitednet.us (Jason Canady) Date: Mon, 17 Oct 2016 09:29:58 -0400 Subject: Two BGP peering sessions on single Comcast Fiber Connection? In-Reply-To: <213679615.3830.1476707434794.JavaMail.mhammett@ThunderFuck> References: <22433E7377ECE14D9518DB3338A32EA30FABF3B6@ExchMB10.syssrcad.syssrc.com> <28667908-0d2a-13a9-1db3-c7a4ac27f443@secantnet.net> <213679615.3830.1476707434794.JavaMail.mhammett@ThunderFuck> Message-ID: <9DF7F144-2CEC-4DCA-89B9-6F2B5E44CF3E@unlimitednet.us> I completely concur. We spread our uplinks across separate boxes and we have /29 allocations. Get the best of all worlds. But if I only had one provider, I'd want to have multiple BGP sessions for this reason. > On Oct 17, 2016, at 08:30, Mike Hammett wrote: > > It really seems like it's a grave oversight to *NOT* support multiple BGP sessions. I drop to two routers for that same reason, I can do maintenance on one, while the other carries traffic. > > > > > ----- > Mike Hammett > Intelligent Computing Solutions > http://www.ics-il.com > > Midwest-IX > http://www.midwest-ix.com > > ----- Original Message ----- > > From: "Mike Poublon" > To: "rar" , nanog at nanog.org > Sent: Thursday, October 13, 2016 2:04:29 PM > Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? > > I started a thread around the same topic back on 10/16 of 2014. A > Comcast engineer (who ultimately spoke to the national product manager) > came back after discussing and said the same thing "We don't support > that". I got a slightly longer explanation of: > > -------------------------------------------- > > In a nutshell, when we design a product we do it to accommodate the most > typical customer cases. > Given that the design includes a single fiber path and thus the fiber > path and device that terminates on either end each are a single point of > failure, adding extra BGP sessions doesn?t seem to add value in the > typical failure scenarios. In order to achieve the simplest and most > scalable solution to address the market, we rely on narrowing the > possible combinations of parameters. > > -------------------------------------------- > > I explained to them that their interpretation prevents me from being > able to do concurrent maintenance on my side (single router > reboot/upgrade, etc). Never got anywhere with it though. > > I'm still interested in having this set up, but have given up on it ever > really coming to reality. Luckily ALL of my other providers were more > than happy to set up an extra session. > > If anyone from Comcast is listening, there is customer demand for this. > It's not about making it better for Comcast, it's about allowing > customers to have more flexibility. > > Mike Poublon > > /Senior Datacenter Network Engineer/ > > *Secant Technologies* > > 6395 Technology Ave. Suite A > > Kalamazoo, MI 49009 > >> On 10/13/2016 1:48 PM, rar wrote: >> After a many month wait, we were ready to turn up our BGP peering sessions on a new Comcast fiber connection. >> >> With our other providers (Level 3 and Verizon) we have edge routers that directly connect between the provider's on premise connection and our primary and a backup core routers. Each core router has a multihop BGP session with the provider's BGP router. The goal is to keep the single BGP router from being a single point of failure. >> >> Comcast said they could not support two separate BGP peering sessions on the same circuit. Does anyone have any counter examples? We used to have this setup with Comcast 5+ years ago, but now they say they can't support it. >> >> >> Bob Roswell >> broswell at syssrc.com >> 410-771-5544 ext 4336 >> >> Computer Museum Highlights > > From dave at temk.in Mon Oct 17 16:05:14 2016 From: dave at temk.in (Dave Temkin) Date: Mon, 17 Oct 2016 16:05:14 +0000 (UTC) Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: We (Netflix) are investigating this now. -Dave On Sat, Oct 15, 2016 at 12:44 PM -0500, "Velocity Lists" wrote: We have seen it as well. In our cases it is all TCP DNS traffic as well. Velocity Online 850-205-4638 On Fri, Oct 14, 2016 at 11:43 AM, Eamon Bauman wrote: > We're rate limiting it now, but it's definitely bad behavior. When I open > the flood gates, over a 5-min sample from a single host I received well > over 61,000 queries. > The size of the records being requested cause this to be an (unintended) > amplification attack, as a 30Mbps inbound sum is getting amplified to > 150-200Mbps outbound. > > On Thu, Oct 13, 2016 at 7:52 PM, Josh Reynolds > wrote: > > > Same here :) > > > > On Oct 13, 2016 1:09 PM, "Ryan, Spencer" wrote: > > > >> I was going to point you to the reddit thread about it, but it looks to > >> be your thread :) > >> > >> > >> Spencer Ryan | Senior Systems Administrator | sryan at arbor.net >> sryan at arbor.net> > >> Arbor Networks > >> +1.734.794.5033 (d) | +1.734.846.2053 (m) > >> www.arbornetworks.com > >> > >> > >> ________________________________ > >> From: NANOG on behalf of Eamon Bauman < > >> eamon at eamonbauman.com> > >> Sent: Thursday, October 13, 2016 10:26:57 AM > >> To: nanog at nanog.org > >> Subject: Excessive Netflix DNS Traffic? > >> > >> Hi all, > >> > >> Is anyone seeing excessive DNS traffic from game consoles (Xbox One, > PS4) > >> running Netflix? Starting 9/29 we have been seeing significant volume of > >> DNS traffic from game consoles on our campus to our caching recursive > >> boxes. Logs show repeated requests for api-global.netflix.com and > >> nrdp.nccp.netflix.com. > >> > >> Anyone else experiencing this? > >> > >> Eamon > >> > > > From rfg at tristatelogic.com Tue Oct 18 05:08:17 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Mon, 17 Oct 2016 22:08:17 -0700 Subject: Route It Or Lose It Message-ID: <27617.1476767297@segfault.tristatelogic.com> What a friendly, helpful place the modern Internet is! Like the forrest floor, its an ecosystem where things don't go to waste. If you happen to inadvertantly leave your shiny /18 IPv4 block lying around, don't worry. It won't be long before some helpful Bulgarian, Romania, Ukranian or Russian will happen by, notice that you failed to route it, and then fix that for you, at no charge, and without you even having to ask. Then, as a bonus, also at no charge, he'll fill it to the brim with snowshoe spammers for you! How helpful! 124.157.0.0/18 (VietNam) -> AS44814 (Bulgaria) From jhellenthal at dataix.net Tue Oct 18 14:27:30 2016 From: jhellenthal at dataix.net (Jason Hellenthal) Date: Tue, 18 Oct 2016 09:27:30 -0500 Subject: Route It Or Lose It In-Reply-To: <27617.1476767297@segfault.tristatelogic.com> References: <27617.1476767297@segfault.tristatelogic.com> Message-ID: Well what else would you expect from todays information age. It?s like leaving a $100.00 bill on the sidewalk and expecting it to be there the following day. > On Oct 18, 2016, at 00:08, Ronald F. Guilmette wrote: > > > What a friendly, helpful place the modern Internet is! > > Like the forrest floor, its an ecosystem where things don't go > to waste. > > If you happen to inadvertantly leave your shiny /18 IPv4 block > lying around, don't worry. It won't be long before some helpful > Bulgarian, Romania, Ukranian or Russian will happen by, notice > that you failed to route it, and then fix that for you, at no > charge, and without you even having to ask. Then, as a bonus, > also at no charge, he'll fill it to the brim with snowshoe spammers > for you! How helpful! > > 124.157.0.0/18 (VietNam) -> AS44814 (Bulgaria) > -- Jason Hellenthal JJH48-ARIN From Jason_Livingood at comcast.com Tue Oct 18 16:41:17 2016 From: Jason_Livingood at comcast.com (Livingood, Jason) Date: Tue, 18 Oct 2016 16:41:17 +0000 Subject: INFORM: Web-Based Speed Test Hackathon - Princeton, NJ in November In-Reply-To: References: Message-ID: Can I take that as a sign you are interested in participating in the project? In any case, I think I indicated it would be available soon, and prior to the hackathon, and eventually via a ?neutral? code repo. Perhaps you are interested in coming? I?ve copied Nick from Princeton if that is the case. Jason On 10/16/16, 12:30 PM, "Cody Grosskopf" > wrote: It's completely possible I have overlooked but you call it an open source tool but provide no source? On Wed, Oct 12, 2016, 1:40 PM Livingood, Jason > wrote: May be of interest to folks on this list? Comcast is developing a new open source web-based speed test ? see http://labs.comcast.com/beta-testing-a-new-open-source-speed-test. ISPs, academic researchers, and others will likely be interested in using this rather than older tools. One goal is to make it easy for any organization to customize it for their use and UI, and we?ll soon find a neutral non-Comcast home for the code. To kick this off we are sponsoring a hackathon at Princeton University in November (the 3rd and 4th). Anyone can participate ? though participants should preferably be developers or have development skills (the code is written in Node.JS with some Angular components for the UI). All details at https://citp.princeton.edu/event/speedtest-hackathon/ and we?re asking folks to click the RSVP link at the top to register ASAP. Regards, Jason Livingood Comcast From arnold at nipper.de Tue Oct 18 20:32:33 2016 From: arnold at nipper.de (Arnold Nipper) Date: Tue, 18 Oct 2016 22:32:33 +0200 Subject: Reminder: PeeringDB Product Committee Charter Survey / NANOG In-Reply-To: References: Message-ID: On 29.09.2016 21:52, Arnold Nipper wrote: > Hello PeeringDB users / hello NANOG, > > the PeeringDB Product Committee (PC, [0]) is charged with steering the > future product development and running the market outreach efforts to > continuously improve the value that PeeringDB delivers to the networks > registered with PeeringDB, and the broader community. > > We're looking for feedback and input from the community on our charter > proposal. Please take this short survey [1]. Your input and comments are > appreciated! > > > [0] http://docs.peeringdb.com/gov/ > [1] https://www.surveymonkey.com/r/JN36DT2 > The survey is still open until 2016-10-28 23:59 UTC. Make up your mind. Tell us where you would like PeeringDB to go. Arnold -- Arnold Nipper email: arnold at nipper.de phone: +49 6224 5593407 2 mobile: +49 172 2650958 fax: +49 6224 5593407 9 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From nanog at ics-il.net Tue Oct 18 20:49:19 2016 From: nanog at ics-il.net (Mike Hammett) Date: Tue, 18 Oct 2016 15:49:19 -0500 (CDT) Subject: Coherent CWDM 40G QSFP Message-ID: <1495599192.202.1476823756587.JavaMail.mhammett@ThunderFuck> Does anyone make a coherent CWDM 40G QSFP? I thought so, but the first couple places I checked, I struck out at. This would be for a passive mux\MROADM. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From ryanczak at gmail.com Tue Oct 18 22:19:59 2016 From: ryanczak at gmail.com (Matt Ryanczak) Date: Tue, 18 Oct 2016 17:19:59 -0500 Subject: Route It Or Lose It In-Reply-To: References: <27617.1476767297@segfault.tristatelogic.com> Message-ID: On Tue, Oct 18, 2016 at 9:27 AM, Jason Hellenthal wrote: > Well what else would you expect from todays information age. It?s like > leaving a $100.00 bill on the sidewalk and expecting it to be there the > following day. More like $2,621,440.00. Certainly too attractive to pass up! From tdurack at gmail.com Wed Oct 19 02:28:02 2016 From: tdurack at gmail.com (Tim Durack) Date: Wed, 19 Oct 2016 02:28:02 +0000 Subject: Coherent CWDM 40G QSFP In-Reply-To: <1495599192.202.1476823756587.JavaMail.mhammett@ThunderFuck> References: <1495599192.202.1476823756587.JavaMail.mhammett@ThunderFuck> Message-ID: Not aware of ACO/DCO in QSFP form factor. Inphi is doing 100G QSFP28 PAM4 DWDM for MS. Probably the best you will see for a while. On Tue, Oct 18, 2016 at 4:50 PM Mike Hammett wrote: > Does anyone make a coherent CWDM 40G QSFP? I thought so, but the first > couple places I checked, I struck out at. This would be for a passive > mux\MROADM. > > > > > ----- > Mike Hammett > Intelligent Computing Solutions > > Midwest Internet Exchange > > The Brothers WISP > > From savage at savage.za.org Wed Oct 19 07:31:33 2016 From: savage at savage.za.org (Chris Knipe) Date: Wed, 19 Oct 2016 09:31:33 +0200 Subject: NetRouting NOC Message-ID: Hi, Anyone from NetRouting NOC (AS47869) that can contact me off list please. Not getting anywhere resolving a routing issue with your support personnel. -- Regards, Chris Knipe From web at typo.org Wed Oct 19 07:44:31 2016 From: web at typo.org (Wayne Bouchard) Date: Wed, 19 Oct 2016 00:44:31 -0700 Subject: 18 years ago today - rfc 2468 In-Reply-To: <2B6C190A-B731-4EE9-A767-68C050057D51@ianai.net> References: <2B6C190A-B731-4EE9-A767-68C050057D51@ianai.net> Message-ID: <20161019074431.GA2455@spider.typo.org> And for those of you who you don't recognize his name, either you aren't old enough or you haven't read enough RFCs, though his contributions go wayyyyyyy beyond that. It is fair to say he is very much one of the cadre of personell who quite literally built the internet that so many of the rest now take for granted. On Sat, Oct 15, 2016 at 09:21:01AM -0400, Patrick W. Gilmore wrote: > We do. > > Thank you for reminding us. And thanks to Dr. Postel for making what we do possible. > > -- > TTFN, > patrick > > > On Oct 15, 2016, at 9:19 AM, Rodney Joffe wrote: > > > > To be clear - Oct 16. Which has just tolled in the APAC region. For most of you it will be tomorrow. But no matter. You get the point. > > > >> On Oct 15, 2016, at 9:08 AM, Rodney Joffe wrote: > >> > >> How time flies.... > --- Wayne Bouchard web at typo.org Network Dude http://www.typo.org/~web/ From nanog at ics-il.net Wed Oct 19 12:39:01 2016 From: nanog at ics-il.net (Mike Hammett) Date: Wed, 19 Oct 2016 07:39:01 -0500 (CDT) Subject: 18 years ago today - rfc 2468 In-Reply-To: <20161019074431.GA2455@spider.typo.org> References: <2B6C190A-B731-4EE9-A767-68C050057D51@ianai.net> <20161019074431.GA2455@spider.typo.org> Message-ID: <561894279.241.1476880738262.JavaMail.mhammett@ThunderFuck> "or you haven't read enough RFCs" so for those of us that aren't masochists.... ;-) I did get my summary last year at NANOG, though. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Wayne Bouchard" To: "Patrick W. Gilmore" Cc: "NANOG list" Sent: Wednesday, October 19, 2016 2:44:31 AM Subject: Re: 18 years ago today - rfc 2468 And for those of you who you don't recognize his name, either you aren't old enough or you haven't read enough RFCs, though his contributions go wayyyyyyy beyond that. It is fair to say he is very much one of the cadre of personell who quite literally built the internet that so many of the rest now take for granted. On Sat, Oct 15, 2016 at 09:21:01AM -0400, Patrick W. Gilmore wrote: > We do. > > Thank you for reminding us. And thanks to Dr. Postel for making what we do possible. > > -- > TTFN, > patrick > > > On Oct 15, 2016, at 9:19 AM, Rodney Joffe wrote: > > > > To be clear - Oct 16. Which has just tolled in the APAC region. For most of you it will be tomorrow. But no matter. You get the point. > > > >> On Oct 15, 2016, at 9:08 AM, Rodney Joffe wrote: > >> > >> How time flies.... > --- Wayne Bouchard web at typo.org Network Dude http://www.typo.org/~web/ From ekgermann at semperen.com Wed Oct 19 13:27:14 2016 From: ekgermann at semperen.com (Eric Germann) Date: Wed, 19 Oct 2016 08:27:14 -0500 Subject: Linux router guru sought for hairpulling issue Message-ID: <2011346D-EA12-4C06-80C0-626A487B71B3@semperen.com> Colleagues, I know we?re all usually running big gear, but I?ve been tasked with building some appliances to run in the cloud as VM?s. Looking for someone who has built on Centos 7 using IPSec and GRE tunnels. Having an issue with GRE tunnels and trace route. That?s pulling my hair out. If you?d like to discuss, reply off list. Thanks EKG -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3705 bytes Desc: not available URL: From ekgermann at semperen.com Wed Oct 19 15:38:04 2016 From: ekgermann at semperen.com (Eric Germann) Date: Wed, 19 Oct 2016 10:38:04 -0500 Subject: Linux router guru sought for hairpulling issue In-Reply-To: <2011346D-EA12-4C06-80C0-626A487B71B3@semperen.com> References: <2011346D-EA12-4C06-80C0-626A487B71B3@semperen.com> Message-ID: Thanks to Robert McKay for the answer that fixed it. His explanation was > Did you forget to add ttl 255 (or similar) to the tunnel setup? By default the gre packets will end up with the ttl set to the same as the inside payload ttl so when you traceroute they won't reach the other gateway.. that sounds like what you might be talking about? > > http://lartc.org/howto/lartc.tunnel.gre.html Added TTL=255 to the ifcfg-tun* config files and all is well. Thanks to the others for their ideas (too many to name). Great community EKG > On Oct 19, 2016, at 8:27 AM, Eric Germann wrote: > > Colleagues, > > I know we?re all usually running big gear, but I?ve been tasked with building some appliances to run in the cloud as VM?s. > > Looking for someone who has built on Centos 7 using IPSec and GRE tunnels. Having an issue with GRE tunnels and trace route. That?s pulling my hair out. > > If you?d like to discuss, reply off list. > > Thanks > > EKG > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3705 bytes Desc: not available URL: From m4rtntns at gmail.com Wed Oct 19 16:27:37 2016 From: m4rtntns at gmail.com (Martin T) Date: Wed, 19 Oct 2016 19:27:37 +0300 Subject: nested prefixes in Internet In-Reply-To: <49C83B6E-C0D5-44C0-A13C-7B55F5AF5254@delong.com> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> <49C83B6E-C0D5-44C0-A13C-7B55F5AF5254@delong.com> Message-ID: Hi, I made a drawing of those two best solutions: http://i.imgur.com/7NQVgUH.png As much as I understand, both solutions require no special changes from "ISP C". Only advantage of solution B over solution A, that I can see, is that at the time when link between "ISP C" and "ISP B" is up, the traffic from Internet towards "ISP B" prefers the "ISP C" connection. In case the link between "ISP A" and "ISP B" goes down, then traffic from "ISP A" addressed to this /24 will use a private peering link between "ISP A" and "ISP C" so the transit costs are not an issue. thanks, Martin On Wed, Oct 12, 2016 at 1:58 AM, Owen DeLong wrote: > >> On Oct 10, 2016, at 14:59 , Baldur Norddahl wrote: >> >> >> >> Den 10/10/2016 kl. 22.27 skrev Owen DeLong: >>> Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. >>> >> >> But ISP-A is in fact being paid to carry the traffic. Supposedly ISP-B has a paid transit relation to ISP-A. In the case the transit link is down ISP-A might have to transport the traffic through a less profitable link however. > > Which isn?t really in the agreement between ISP-B and ISP-A unless it was specifically (and unusually) negotiated. > > Also, you?re assuming that the leased space came with a transit agreement. In many cases, address leases don?t, so consider the additional scenario where ISP-B leases addresses from ISP-A, but has transit contracts with ISP-C and ISP-D but no connection at all to ISP-A. > >> I know that if ISP-A was my network I would be making money even with the transit link down. Yes I might have to transport something out of my network through one of my transits, but outbound traffic is in fact free for us because we are heavy inbound loaded. > > Yes, but it doesn?t help if it also came in on a transit link. Any traffic you both receive and transmit on transit costs you money pretty much no matter who you are. > > > Owen > From owen at delong.com Wed Oct 19 16:50:48 2016 From: owen at delong.com (Owen DeLong) Date: Wed, 19 Oct 2016 11:50:48 -0500 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> <49C83B6E-C0D5-44C0-A13C-7B55F5AF5254@delong.com> Message-ID: <67C8B898-EF98-4F0F-BD05-8BE36F83B9A7@delong.com> Assuming that there is a PNI A<->C assumes facts not in evidence. Owen > On Oct 19, 2016, at 11:27 AM, Martin T wrote: > > Hi, > > I made a drawing of those two best solutions: http://i.imgur.com/7NQVgUH.png > > As much as I understand, both solutions require no special changes > from "ISP C". Only advantage of solution B over solution A, that I can > see, is that at the time when link between "ISP C" and "ISP B" is up, > the traffic from Internet towards "ISP B" prefers the "ISP C" > connection. > > > In case the link between "ISP A" and "ISP B" goes down, then traffic > from "ISP A" addressed to this /24 will use a private peering link > between "ISP A" and "ISP C" so the transit costs are not an issue. > > > thanks, > Martin > > On Wed, Oct 12, 2016 at 1:58 AM, Owen DeLong wrote: >> >>> On Oct 10, 2016, at 14:59 , Baldur Norddahl wrote: >>> >>> >>> >>> Den 10/10/2016 kl. 22.27 skrev Owen DeLong: >>>> Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. >>>> >>> >>> But ISP-A is in fact being paid to carry the traffic. Supposedly ISP-B has a paid transit relation to ISP-A. In the case the transit link is down ISP-A might have to transport the traffic through a less profitable link however. >> >> Which isn?t really in the agreement between ISP-B and ISP-A unless it was specifically (and unusually) negotiated. >> >> Also, you?re assuming that the leased space came with a transit agreement. In many cases, address leases don?t, so consider the additional scenario where ISP-B leases addresses from ISP-A, but has transit contracts with ISP-C and ISP-D but no connection at all to ISP-A. >> >>> I know that if ISP-A was my network I would be making money even with the transit link down. Yes I might have to transport something out of my network through one of my transits, but outbound traffic is in fact free for us because we are heavy inbound loaded. >> >> Yes, but it doesn?t help if it also came in on a transit link. Any traffic you both receive and transmit on transit costs you money pretty much no matter who you are. >> >> >> Owen >> From volists at staff.velocityonline.net Wed Oct 19 17:16:42 2016 From: volists at staff.velocityonline.net (Velocity Lists) Date: Wed, 19 Oct 2016 13:16:42 -0400 Subject: Excessive Netflix DNS Traffic? In-Reply-To: References: Message-ID: Did (Netflix) find an issue? Velocity Online 850-205-4638 On Mon, Oct 17, 2016 at 12:05 PM, Dave Temkin wrote: > We (Netflix) are investigating this now. > > -Dave > > > > > > On Sat, Oct 15, 2016 at 12:44 PM -0500, "Velocity Lists" < > volists at staff.velocityonline.net> wrote: > > We have seen it as well. >> In our cases it is all TCP DNS traffic as well. >> >> Velocity Online850-205-4638 >> >> On Fri, Oct 14, 2016 at 11:43 AM, Eamon Bauman >> wrote: >> >> > We're rate limiting it now, but it's definitely bad behavior. When I open >> > the flood gates, over a 5-min sample from a single host I received well >> > over 61,000 queries. >> > The size of the records being requested cause this to be an (unintended) >> > amplification attack, as a 30Mbps inbound sum is getting amplified to >> > 150-200Mbps outbound. >> > >> > On Thu, Oct 13, 2016 at 7:52 PM, Josh Reynolds >> > wrote: >> > >> > > Same here :) >> > > >> > > On Oct 13, 2016 1:09 PM, "Ryan, Spencer" wrote: >> > > >> > >> I was going to point you to the reddit thread about it, but it looks to >> > >> be your thread :) >> > >> >> > >> >> > >> Spencer Ryan | Senior Systems Administrator | sryan at arbor.net >> sryan at arbor.net> >> > >> Arbor Networks >> > >> +1.734.794.5033 (d) | +1.734.846.2053 (m) >> > >> www.arbornetworks.com >> > >> >> > >> >> > >> ________________________________ >> > >> From: NANOG on behalf of Eamon Bauman < >> > >> eamon at eamonbauman.com> >> > >> Sent: Thursday, October 13, 2016 10:26:57 AM >> > >> To: nanog at nanog.org >> > >> Subject: Excessive Netflix DNS Traffic? >> > >> >> > >> Hi all, >> > >> >> > >> Is anyone seeing excessive DNS traffic from game consoles (Xbox One, >> > PS4) >> > >> running Netflix? Starting 9/29 we have been seeing significant volume of >> > >> DNS traffic from game consoles on our campus to our caching recursive >> > >> boxes. Logs show repeated requests for api-global.netflix.com and >> > >> nrdp.nccp.netflix.com. >> > >> >> > >> Anyone else experiencing this? >> > >> >> > >> Eamon >> > >> >> > > >> > >> >> From baldur.norddahl at gmail.com Wed Oct 19 20:29:29 2016 From: baldur.norddahl at gmail.com (Baldur Norddahl) Date: Wed, 19 Oct 2016 22:29:29 +0200 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <51EEB11C-C9EC-4F60-9C3D-53C83FF925BB@delong.com> <49C83B6E-C0D5-44C0-A13C-7B55F5AF5254@delong.com> Message-ID: <826aeaa1-811d-738c-5b6c-17c74f7f41e4@gmail.com> Hi Solution B is what happens by default and requires no changes by any party. A, B and C just do what they would do in any transit relation. The default BGP shortest AS path length first algorithm will make sure that traffic is delivered correctly. Solution A requires that ISP A actively filter the /24 announcement and that would have severe negative impact. It would mean that you would not receive any traffic at all through that link unless the link to ISP B is down. Not even traffic from A to B (that would go A -> C -> B because of the more specific). Maybe you meant that they would only filter the /24 announcement on the peering link between A and C, but that would have no effect and therefore makes no sense. Remember that ISP A is only originating their own /19. The /24 route announcement is received from ISP B and merely propagated (not originated) to ISP As uplink and peers. The moment that the link between A and B is down, the /24 route announcement will gone as well - instead A will be receiving the /24 route announcement via C. Regards, Baldur Den 19/10/2016 kl. 18.27 skrev Martin T: > Hi, > > I made a drawing of those two best solutions: http://i.imgur.com/7NQVgUH.png > > As much as I understand, both solutions require no special changes > from "ISP C". Only advantage of solution B over solution A, that I can > see, is that at the time when link between "ISP C" and "ISP B" is up, > the traffic from Internet towards "ISP B" prefers the "ISP C" > connection. > > > In case the link between "ISP A" and "ISP B" goes down, then traffic > from "ISP A" addressed to this /24 will use a private peering link > between "ISP A" and "ISP C" so the transit costs are not an issue. > > > thanks, > Martin > > On Wed, Oct 12, 2016 at 1:58 AM, Owen DeLong wrote: >>> On Oct 10, 2016, at 14:59 , Baldur Norddahl wrote: >>> >>> >>> >>> Den 10/10/2016 kl. 22.27 skrev Owen DeLong: >>>> Not true? There are myriad reasons that the /24 might not reach a network peered with ISP-A, including the possibility of being a downstream customer of a network peered with or buying transit from ISP-A. In the latter case, not an issue, since it?s paid transit, but in the former (peered, not transit), again, ISP-A is probably not super excited to carry traffic that someone isn?t paying them to carry. >>>> >>> But ISP-A is in fact being paid to carry the traffic. Supposedly ISP-B has a paid transit relation to ISP-A. In the case the transit link is down ISP-A might have to transport the traffic through a less profitable link however. >> Which isn?t really in the agreement between ISP-B and ISP-A unless it was specifically (and unusually) negotiated. >> >> Also, you?re assuming that the leased space came with a transit agreement. In many cases, address leases don?t, so consider the additional scenario where ISP-B leases addresses from ISP-A, but has transit contracts with ISP-C and ISP-D but no connection at all to ISP-A. >> >>> I know that if ISP-A was my network I would be making money even with the transit link down. Yes I might have to transport something out of my network through one of my transits, but outbound traffic is in fact free for us because we are heavy inbound loaded. >> Yes, but it doesn?t help if it also came in on a transit link. Any traffic you both receive and transmit on transit costs you money pretty much no matter who you are. >> >> >> Owen >> From nanog at ics-il.net Wed Oct 19 21:46:30 2016 From: nanog at ics-il.net (Mike Hammett) Date: Wed, 19 Oct 2016 16:46:30 -0500 (CDT) Subject: Coherent CWDM 40G QSFP In-Reply-To: References: <1495599192.202.1476823756587.JavaMail.mhammett@ThunderFuck> Message-ID: <1949363373.519.1476913587878.JavaMail.mhammett@ThunderFuck> Apparently I just remembered the big transport platforms using coherent 40G and 100G and assumed there was a cheap variant, but there isn't. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Tim Durack" To: "Mike Hammett" , "nanog list" Sent: Tuesday, October 18, 2016 9:28:02 PM Subject: Re: Coherent CWDM 40G QSFP Not aware of ACO/DCO in QSFP form factor. Inphi is doing 100G QSFP28 PAM4 DWDM for MS. Probably the best you will see for a while. On Tue, Oct 18, 2016 at 4:50 PM Mike Hammett < nanog at ics-il.net > wrote: Does anyone make a coherent CWDM 40G QSFP? I thought so, but the first couple places I checked, I struck out at. This would be for a passive mux\MROADM. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From rod.beck at unitedcablecompany.com Wed Oct 19 21:52:17 2016 From: rod.beck at unitedcablecompany.com (Rod Beck) Date: Wed, 19 Oct 2016 21:52:17 +0000 Subject: Coherent CWDM 40G QSFP In-Reply-To: <1949363373.519.1476913587878.JavaMail.mhammett@ThunderFuck> References: <1495599192.202.1476823756587.JavaMail.mhammett@ThunderFuck> , <1949363373.519.1476913587878.JavaMail.mhammett@ThunderFuck> Message-ID: True 40 gigs is rare period. Most of the time it is bandwidth limiting on a 100 gig wave. ________________________________ From: NANOG on behalf of Mike Hammett Sent: Wednesday, October 19, 2016 11:46 PM To: Tim Durack Cc: nanog list Subject: Re: Coherent CWDM 40G QSFP Apparently I just remembered the big transport platforms using coherent 40G and 100G and assumed there was a cheap variant, but there isn't. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Tim Durack" To: "Mike Hammett" , "nanog list" Sent: Tuesday, October 18, 2016 9:28:02 PM Subject: Re: Coherent CWDM 40G QSFP Not aware of ACO/DCO in QSFP form factor. Inphi is doing 100G QSFP28 PAM4 DWDM for MS. Probably the best you will see for a while. On Tue, Oct 18, 2016 at 4:50 PM Mike Hammett < nanog at ics-il.net > wrote: Does anyone make a coherent CWDM 40G QSFP? I thought so, but the first couple places I checked, I struck out at. This would be for a passive mux\MROADM. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP From matt at overloaded.net Wed Oct 19 22:16:35 2016 From: matt at overloaded.net (Matt Buford) Date: Wed, 19 Oct 2016 17:16:35 -0500 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: On Mon, Oct 10, 2016 at 2:44 PM, Baldur Norddahl wrote: > Is that a real problem? In my experience a /24 is honoured almost > universially. > Here's a real-world issue I ran into with this. In this case, it isn't that someone filtered /24s, but that they didn't have a full table (peering routes only plus a default). This resulted in them following the less specific route for traffic destined for the /24. A customer of mine was advertising only a /20 to me and then only a more specific /24 was advertised from a remote site of theirs to a different ISP. The customer did have a connection between their two sites, so in theory it was OK if the traffic arrived on their "wrong" link, except that the customer strongly didn't want this to happen, thus the inconsistent routes. This customer found that the remote /24 was unable to access a large CDN provider. This CDN told them that a traceroute to the /24 went to my network (we peered at an exchange) and was then dropped. This seemed odd at first, as I confirmed I was not advertising the /24 to them so why were they routing it to me? It turned out that the CDN provider was running a peer-routes-only network with a default to their transit. This meant that they saw the /20 from their peer (me) but never saw the /24, since they carried no transit routes. This resulted in them routing the entire /20 to me. My peering router was not willing to route traffic from a peering exchange towards transit I had to pay for, so it was dropped. The customer's split advertisements didn't seem particularly unreasonable or invalid, though perhaps they were not the preferred setup. It wasn't unreasonable for me to not route from a peering exchange to transit. It wasn't unreasonable of the CDN to have a peering-and-default routing table. But, those three things together were not compatible. I called the customer and presented them with my findings and some potential options to consider, and consistent advertisements was one of those options. The customer strongly wanted incoming traffic to arrive directly to the correct location so he didn't want to do that. I suggested a possible compromise was for him to advertise both the /24 and /20 to me, but use communities so that I never advertised his /24 to any upstreams or peering exchanges. That way, if traffic for the /24 accidentally hit my network like we were running into, I would route it to him and he could pass it to the correct site. It meant that a little traffic would arrive at the wrong site in his network and have to pass over his back-end link, but at least it would be fairly limited. He didn't like this either. He didn't want to split the /20 advertisement up to no longer cover the /24 either, I think just because "I shouldn't have to do that!" The option the customer chose in the end was to use a community on his advertisement to instruct my routers to no longer advertise his /20 to any peering exchanges at all. That ensured the CDN didn't directly send me anything for him (not the /24 or the /20), and the transit networks in-between took care of making sure traffic to the /24 didn't accidentally end up on my network. While I didn't find it very elegant to be shifting traffic from peering exchanges to transit, it wasn't a significant amount of traffic and it got him off my back. From ef14019 at gmail.com Thu Oct 20 13:43:26 2016 From: ef14019 at gmail.com (steven brock) Date: Thu, 20 Oct 2016 15:43:26 +0200 Subject: MPLS in the campus Network? Message-ID: Dear NANOG members, We operate a campus network reaching more than 100 buildings on 5 campuses. We also operate a regional backbone and the interconnexion to our NREN. The current architecture is made of a L2 backbone and a few routers. Most of the buildings are connected with a 1 Gb/s link using our own optical fiber (only a few building are connected at 10 Gb/s). In a smaller number of buildings (a few dozens), we also operate the internal network, made of ethernet switches (in a multi-vendor environment). In each building, we provide at least an edge switch, marking the boundary between us and the customer, where we deliver the different services on ethernet ports. The services we currently offer: - L2 interconnections (400 vlans are present in 2 buildings or more; only a few VLANs are present in more than 30 buildings) - IPv4 et IPv6 routing (hundreds of subnets) and Internet access, - specific interconnections (ex: terminating a VPN to the customer, say a national private infrastructure delivered by the NREN through MPLS L2/L3VPN and stitched to the customer network using a specific VLAN) - routing isolation using routing instances (~ VRF Lite) : only 5 instances, but we could have more, - routing and filtering using open source firewalls running on servers in our DCs (less than 15 platforms, as most customer operate their own firewall), - user authentication, - shared VPN platform allowing direct access for an identified user into the customer network (based on radius attribute) - this platform uses VLANs to interconnect to the rest of the network, - wireless LAN, also allowing direct access for an identified user into the customer network ; the platform is a centralized controller, and it uses VLAN to interconnect to the rest of the network. (those last two services could use just a VLAN or a dedicated subnet delivered on a port of the edge switch which is then connected to the customer firewall) We are not satisfied with the current backbone design ; we had our share of problems in the past: - high CPU load on the core switches due to multiple instances of spanning tree slowly converging when a topology change happens (somehow fixed with a few instances of MSTP) - spanning tree interoperability problems and spurious port blocking (fixed by BPDU filtering) - loops at the edge and broadcast/multicast storms (fixed with traffic limits and port blocking based on threshhold) - some small switches at the edge are overloaded with large numbers of MAC addresses (fixed with reducing broadcast domain size and subnetting) This architecture doesn't feel very solid. Even if the service provisionning seems easy from an operational point of view (create a VLAN and it is immediately available at any point of the L2 backbone), we feel the configuration is not always consistent. We have to rely on scripts pushing configuration elements and human discipline (and lots of duct-tape, especially for QoS and VRFs). We are re-designing our network architecture. We have enough fiber to imagine many ways to link the core network devices. We find MPLS has its merit as a platform, to bring all the network services we currently provide (L2, L3 VPN, VPLS, and soon EVPN) However, we also want to upgrade the infrastructure to allow future growth of the traffic. Some labs, especially in physics, could need more than 10 Gb/s in the coming years. Our cycles of evolution are long (we keep a backbone technology for 8 years). MPLS is definitely not cheap considering the price of a 10G or 100G router interface. Compared to MPLS, a L2 solution with 100 Gb/s interfaces between core switches and a 10G connection for each buildings looks so much cheaper. But we worry about future trouble using Trill, SPB, or other technologies, not only the "open" ones, but specifically the proprietary ones based on central controller and lots of magic (some colleagues feel the debug nightmare are garanteed). If you had to make such a choice recently, did you choose an MPLS design even at lower speed ? How would you convince your management that MPLS is the best solution for your campus network ? How would you justify the cost or speed difference ? Thanks for your insights! From bicknell at ufp.org Thu Oct 20 15:05:51 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Thu, 20 Oct 2016 08:05:51 -0700 Subject: MPLS in the campus Network? In-Reply-To: References: Message-ID: <20161020150551.GA93512@ussenterprise.ufp.org> From what you describe I do think you have many options, including more than just the ones you laid out. When you're under 10km and own your own fiber the possibilities are virtually limitless. First off, you don't want to be running spanning tree across a campus. While I don't think you need to elminate it completely as some in the industry are pressing, doing it at the scale you describe is probably a world of hurt. I would challenge your port cost assumption for "routers". For instance the Arista 7280 could deliver can be had with 48 10GE SFP+ ports with full Internet routing capabilities. If you're used to Cisco or Juniper, it is worth looking further afield these days. I would also challenge that there is one way to do the job. It may be easier to build a couple of networks. Perhaps a router based one to deliver IP services, and a separate "Metro Ethernet" network to deliver L2 VLAN transport. It may sound crazy that buying two boxes is chepaer than one, but it can be depending on the exact scale and port count. Heck, depending on your port count doing passive DWDM to interconnect switches in each office may be cheaper than encapsulating in MPLS. A lot of it also depends on your monitoring requirements, or lack of. In a message written on Thu, Oct 20, 2016 at 03:43:26PM +0200, steven brock wrote: > How would you convince your management that MPLS is the best solution for > your campus network ? How would you justify the cost or speed difference ? Well, cost and speed are two prime considerations, but there are other important considerations. Vendors support platforms and features based on the customer base. If you buy a box everyone does MPLS on, and then use it for TRILL, you'll be in a world of hurt. Particularly if you want long, stable life ride with the crowd. Use a platform many others are using for the same job. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From jason+nanog at lixfeld.ca Thu Oct 20 15:12:42 2016 From: jason+nanog at lixfeld.ca (Jason Lixfeld) Date: Thu, 20 Oct 2016 11:12:42 -0400 Subject: MPLS in the campus Network? In-Reply-To: References: Message-ID: <90797314-4CF2-4AFD-9E0D-8EF57EC6B080@lixfeld.ca> Hi, > On Oct 20, 2016, at 9:43 AM, steven brock wrote: > > Compared to MPLS, a L2 solution with 100 Gb/s interfaces between > core switches and a 10G connection for each buildings looks so much > cheaper. But we worry about future trouble using Trill, SPB, or other > technologies, not only the "open" ones, but specifically the proprietary > ones based on central controller and lots of magic (some colleagues feel > the debug nightmare are garanteed). From my perspective, in this day and age, no service provider or campus should really be using any sort of layer 2 protection mechanism in their backbone, if they can help it. > If you had to make such a choice recently, did you choose an MPLS design > even at lower speed ? Yup. 5 or so years ago, and never looked back. Actually, this was in conjunction with upgrading our 1G backbone to a 10G backbone, so it was an upgrade for us in all senses of the word. > How would you convince your management that MPLS is the best solution for > your campus network ? You already did: > We are not satisfied with the current backbone design ; we had our share > of problems in the past: > - high CPU load on the core switches due to multiple instances of spanning > tree slowly converging when a topology change happens (somehow fixed > with a few instances of MSTP) > - spanning tree interoperability problems and spurious port blocking > (fixed by BPDU filtering) > - loops at the edge and broadcast/multicast storms (fixed with traffic > limits and port blocking based on threshhold) > - some small switches at the edge are overloaded with large numbers of > MAC addresses (fixed with reducing broadcast domain size and subnetting) > > This architecture doesn't feel very solid. > Even if the service provisionning seems easy from an operational point > of view (create a VLAN and it is immediately available at any point of the > L2 backbone), we feel the configuration is not always consistent. > We have to rely on scripts pushing configuration elements and human > discipline (and lots of duct-tape, especially for QoS and VRFs). > How would you justify the cost or speed difference ? It?s only more expensive the more big vendor products you use. Sometimes you need to (i.e.: Boxes with big RIB/FIBs for DFZ, or deep buffers), but more and more, people are looking to OCP/White Box Switches [1][2]. For example, assuming a BCM Trident II based board with 48 SFP+ cages and 6 QSFP+ cages, you get a line-rate, MPLS capable 10G port for $65. Or, if you?re like me and hate the idea of breakout cables, you?re at about $100/SFP+ cage, at which points the QSPF+ cages are pretty much free. Software wise, there are lots of vendors. One that I like is IPInfusion?s OcNOS[3] codebase. They are putting a lot of resources into building a service provider feature set (full-blown MPLS/VPLS/EVPN, etc.) for OCP switches. There are others, but last time I looked a couple of years ago, they were less focused on MPLS and more focused on SDN: Cumulus Networks[4], PICA8[5], Big Switch Networks[6]. > Thanks for your insights! [1] https://www.linx.net/communications/press-releases/lon2-revolutionary-development [2] http://www.ipinfusion.com/about/press/london-internet-exchange-use-ip-infusion?s-ocnos?-network-operating-system-new-london-in [3] http://www.ipinfusion.com/products/ocnos [4] https://cumulusnetworks.com [5] http://www.pica8.com [6] http://www.bigswitch.com From stuarteclark at yahoo.com Thu Oct 20 15:56:03 2016 From: stuarteclark at yahoo.com (stuart clark) Date: Thu, 20 Oct 2016 15:56:03 +0000 (UTC) Subject: Incapsula | 19551 References: <847375422.1128050.1476978963307.ref@mail.yahoo.com> Message-ID: <847375422.1128050.1476978963307@mail.yahoo.com> Can someone ping me off list from Incapsula AS 19551 please? Thanks! From mark.tinka at seacom.mu Thu Oct 20 16:17:59 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:17:59 +0200 Subject: MPLS in the campus Network? In-Reply-To: References: Message-ID: On 20/Oct/16 15:43, steven brock wrote: > > If you had to make such a choice recently, did you choose an MPLS design > even at lower speed ? > How would you convince your management that MPLS is the best solution for > your campus network ? How would you justify the cost or speed difference ? IP/MPLS would be my recommendation. From an operational perspective, running it in your Core and Access backbone removes any need for STP (and all the associated headache). Not sure what gear you're using now, but you'll get full routing and MPLS features on the platforms such as the Cisco ASR920. I'd have recommended the Cisco ME3600X, but they just announced EoS/EoL last night, which means that while you can still order it until October 2017, the ASR920 will be cheaper and is the future of of Metro-E from Cisco in this segment. Brocade's CES 2000 platform would also be a good choice here. Juniper ACX5000 presents some challenges re: the use of that Broadcom chip, although I know a number of operators that have had the courage to deploy it for this use-case. Ultimately, whatever vendor you choose, the guaranteed way to not get that 3AM call is to run IP/MPLS for your Core and Access backbones, especially looking at how much Layer 2 traffic you're hauling around. Mark. From mark.tinka at seacom.mu Thu Oct 20 16:21:37 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:21:37 +0200 Subject: MPLS in the campus Network? In-Reply-To: <20161020150551.GA93512@ussenterprise.ufp.org> References: <20161020150551.GA93512@ussenterprise.ufp.org> Message-ID: On 20/Oct/16 17:05, Leo Bicknell wrote: > I would challenge your port cost assumption for "routers". For > instance the Arista 7280 could deliver can be had with 48 10GE SFP+ > ports with full Internet routing capabilities. If you're used > to Cisco or Juniper, it is worth looking further afield these days. We're currently looking at Arista's development in the IP/MPLS space. While we are not yet ready to spend money on them for that, we think the future is bright. So we are working very closely with them to get their IP and MPLS code to where it would make sense for us. I'm hopeful we shall be investing in Arista for routing in the very near future. But we love them for switching, and will be activating some of their platforms for that use-case very soon. Mark. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From mark.tinka at seacom.mu Thu Oct 20 16:23:44 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:23:44 +0200 Subject: MPLS in the campus Network? In-Reply-To: <90797314-4CF2-4AFD-9E0D-8EF57EC6B080@lixfeld.ca> References: <90797314-4CF2-4AFD-9E0D-8EF57EC6B080@lixfeld.ca> Message-ID: <99dc1aab-1682-9fb5-1e79-c36dd944603b@seacom.mu> On 20/Oct/16 17:12, Jason Lixfeld wrote: > > It?s only more expensive the more big vendor products you use. Sometimes you need to (i.e.: Boxes with big RIB/FIBs for DFZ, or deep buffers), but more and more, people are looking to OCP/White Box Switches [1][2]. It doesn't sound like the OP needs massive FIB space, so he could implement FIB filtering and run the smaller boxes that have all the features but lack the FIB real estate of the larger routers/switches. This is what we do for our Metro-E Access networks. Mark. From rdobbins at arbor.net Thu Oct 20 16:24:53 2016 From: rdobbins at arbor.net (Roland Dobbins) Date: Thu, 20 Oct 2016 23:24:53 +0700 Subject: MPLS in the campus Network? In-Reply-To: References: Message-ID: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> On 20 Oct 2016, at 23:17, Mark Tinka wrote: > especially looking at how much Layer 2 traffic you're hauling around. And I'd definitely recommend figuring out why that's being done so broadly today, and working to reduce its prevalence and scope, moving forward. ----------------------------------- Roland Dobbins From jason+nanog at lixfeld.ca Thu Oct 20 16:29:00 2016 From: jason+nanog at lixfeld.ca (Jason Lixfeld) Date: Thu, 20 Oct 2016 12:29:00 -0400 Subject: MPLS in the campus Network? In-Reply-To: <99dc1aab-1682-9fb5-1e79-c36dd944603b@seacom.mu> References: <90797314-4CF2-4AFD-9E0D-8EF57EC6B080@lixfeld.ca> <99dc1aab-1682-9fb5-1e79-c36dd944603b@seacom.mu> Message-ID: <08B0FFB0-E6C8-4DE6-BE40-3153F7EF47CD@lixfeld.ca> > On Oct 20, 2016, at 12:23 PM, Mark Tinka wrote: > > > > On 20/Oct/16 17:12, Jason Lixfeld wrote: > >> >> It?s only more expensive the more big vendor products you use. Sometimes you need to (i.e.: Boxes with big RIB/FIBs for DFZ, or deep buffers), but more and more, people are looking to OCP/White Box Switches [1][2]. > > It doesn't sound like the OP needs massive FIB space, so he could implement FIB filtering and run the smaller boxes that have all the features but lack the FIB real estate of the larger routers/switches. > > This is what we do for our Metro-E Access networks. > > Mark. Likely not at the PE, true, but he did say Internet access, so I err?d on the side of assuming DFZ, somewhere. If that assumption is true, FIB resources for the SP interconnect nodes and filtering towards the PEs, absolutely. From mark.tinka at seacom.mu Thu Oct 20 16:32:40 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:32:40 +0200 Subject: MPLS in the campus Network? In-Reply-To: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> References: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> Message-ID: <9783dd82-fcc2-1ee2-7a64-d1062e6f2d91@seacom.mu> On 20/Oct/16 18:24, Roland Dobbins wrote: > > And I'd definitely recommend figuring out why that's being done so > broadly today, and working to reduce its prevalence and scope, moving > forward. Some requirements call for Ethernet transport as opposed to IP. I don't know the details of the OP's user requirements, but from our side, we have as many customers asking for IP services as they do Ethernet. Mark. From mark.tinka at seacom.mu Thu Oct 20 16:37:51 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:37:51 +0200 Subject: MPLS in the campus Network? In-Reply-To: <08B0FFB0-E6C8-4DE6-BE40-3153F7EF47CD@lixfeld.ca> References: <90797314-4CF2-4AFD-9E0D-8EF57EC6B080@lixfeld.ca> <99dc1aab-1682-9fb5-1e79-c36dd944603b@seacom.mu> <08B0FFB0-E6C8-4DE6-BE40-3153F7EF47CD@lixfeld.ca> Message-ID: <1fd49e77-dd12-8e77-2ba7-a1ca3c6f76e1@seacom.mu> On 20/Oct/16 18:29, Jason Lixfeld wrote: > Likely not at the PE, true, but he did say Internet access, so I err?d on the side of assuming DFZ, somewhere. If that assumption is true, FIB resources for the SP interconnect nodes and filtering towards the PEs, absolutely.. I assumed 0/0 + ::/0 to upstreams, but if it had to be the DFZ, you can still hold a full table in RAM but install just default and some select entries into FIB. This would only be necessary if there are downstreams that require the full table. However, if it's for internal use, and a certain amount of DFZ-related traffic engineering is required, the OP could make the case for just having the border routers as the boxes that can handle a full feed. This could be just one or two routers. That would leave the majority of the network running cheaper routers with smaller FIB's. But without really knowing the OP's Internet routing design needs, it could go either way. Mark. From rdobbins at arbor.net Thu Oct 20 16:45:14 2016 From: rdobbins at arbor.net (Roland Dobbins) Date: Thu, 20 Oct 2016 23:45:14 +0700 Subject: MPLS in the campus Network? In-Reply-To: <9783dd82-fcc2-1ee2-7a64-d1062e6f2d91@seacom.mu> References: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> <9783dd82-fcc2-1ee2-7a64-d1062e6f2d91@seacom.mu> Message-ID: <9B25E653-391A-4195-ABDA-61FD0350607D@arbor.net> On 20 Oct 2016, at 23:32, Mark Tinka wrote: > Some requirements call for Ethernet transport as opposed to IP. Sure - but it's probably worth revisiting the origins of those requirements, and whether there are better alternatives. ----------------------------------- Roland Dobbins From mark.tinka at seacom.mu Thu Oct 20 16:58:51 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Thu, 20 Oct 2016 18:58:51 +0200 Subject: MPLS in the campus Network? In-Reply-To: <9B25E653-391A-4195-ABDA-61FD0350607D@arbor.net> References: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> <9783dd82-fcc2-1ee2-7a64-d1062e6f2d91@seacom.mu> <9B25E653-391A-4195-ABDA-61FD0350607D@arbor.net> Message-ID: <6a210ce9-6694-86c3-1690-499767d8a669@seacom.mu> On 20/Oct/16 18:45, Roland Dobbins wrote: > > Sure - but it's probably worth revisiting the origins of those > requirements, and whether there are better alternatives. Indeed. What we've seen is customers who prefer to manage their own IP layer, and just need transport. These types of customers tend to be split between EoDWDM and EoMPLS preferences. Whatever the case, their primary requirement is control of their IP domain. What we're not seeing anymore is l3vpn requirements, particularly on the back of on-premise IT infrastructure moving into the cloud. We see this driving a lot of regular IP growth. Mark. From nick at foobar.org Thu Oct 20 17:15:39 2016 From: nick at foobar.org (Nick Hilliard) Date: Thu, 20 Oct 2016 18:15:39 +0100 Subject: MPLS in the campus Network? In-Reply-To: References: Message-ID: <5808FBBB.5090403@foobar.org> Mark Tinka wrote: > Not sure what gear you're using now, but you'll get full routing and > MPLS features on the platforms such as the Cisco ASR920. I'd have > recommended the Cisco ME3600X, but they just announced EoS/EoL last > night, which means that while you can still order it until October 2017, > the ASR920 will be cheaper and is the future of of Metro-E from Cisco in > this segment. the ME3600x is mostly fine for this sort of thing, with the exception of its control plane policing mechanism which is badly limited in a number of important ways, not least the fact that its traffic categorisation buckets are non-configurable and not what you might necessarily want for customer edge service. Nick From math at sizone.org Thu Oct 20 18:59:49 2016 From: math at sizone.org (Ken Chase) Date: Thu, 20 Oct 2016 14:59:49 -0400 Subject: A perl script to convert Cisco IOS/Nexus/ASA configurations to HTML for easier comprehension Message-ID: <20161020185949.GK10000@sizone.org> re more general 'network utilities' and scripts: http://sizone.org/m/hacks/cidrmath.pl adds and removes subnets from networks giving list of remaining/aggregated (sub)nets. I couldnt find an online calculator that does this, most are just for 'translation' from subnet masks<>cidr or cisco inverse masks, etc. Wrote it years ago cuz I had an itch. The included perl module populates a hash entry per ip and I didnt want to write my own, so uses lots of ram+cpu on big ops (/8 - /9 for eg). But great for earthly operations like /23 - /27 + /28. Yes I should start my own git repo, but i've been lazy. No warranties provided. If anyone has a faster/better one, that'd be handy. /kc -- Ken Chase - ken at sizone.org Toronto & Guelph Canada From dave at temk.in Thu Oct 20 21:30:17 2016 From: dave at temk.in (Dave Temkin) Date: Thu, 20 Oct 2016 14:30:17 -0700 Subject: [NANOG-announce] 2016 NANOG Election Results Message-ID: Greetings NANOG Colleagues, The 2016 NANOG Board and Bylaw election process is now complete. The results were shared during NANOG 68, are posted on the NANOG website, and summarized here. In 2016, there were two regular open positions on the Board of Directors. The appointments are: - Will Charnock - 3 years - Patrick Gilmore - 3 years The officers elected and committee liaisons are: - Chair - Dave Temkin - Vice Chair - Ryan Donnelly - Treasurer - Will Charnock - Secretary - Betty Burke - Communications Committee Liaison - Jezzibell Gilmore - Program Committee Liaison - Patrick Gilmore The proposed amendments to the NANOG Bylaws were accepted. The updated Bylaws document will be posted to the NANOG website. Best Regards, -Dave Temkin Chair, NANOG Board of Directors -------------- next part -------------- _______________________________________________ NANOG-announce mailing list NANOG-announce at mailman.nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-announce From md at bts.sk Fri Oct 21 14:19:18 2016 From: md at bts.sk (Marian =?utf-8?B?xI51cmtvdmnEjQ==?=) Date: Fri, 21 Oct 2016 16:19:18 +0200 Subject: MPLS in the campus Network? Message-ID: <20161021141918.GA65694@bts.sk> > Compared to MPLS, a L2 solution with 100 Gb/s interfaces between > core switches and a 10G connection for each buildings looks so much > cheaper. But we worry about future trouble using Trill, SPB, or other > technologies, not only the "open" ones, but specifically the proprietary > ones based on central controller and lots of magic (some colleagues feel > the debug nightmare are garanteed). > > If you had to make such a choice recently, did you choose an MPLS design > even at lower speed ? A year ago we built NREN backbone using TRILL instead of MPLS. 40 POPs, no central controller, RFC standardized TRILL protocol i.e. L2 routing using IS-IS, no STP. See my recent presentation at http://md.bts.sk/sanet-100g-2.pdf for more details. Much easier to setup, operate & maintain than MPLS and obviously much lower cost. Based on 6-months production experience, my recommendation would be to stay away from MPLS in the campus. With kind regards, M. From cgrundemann at gmail.com Fri Oct 21 14:55:52 2016 From: cgrundemann at gmail.com (Chris Grundemann) Date: Fri, 21 Oct 2016 10:55:52 -0400 Subject: Dyn DDoS this AM? Message-ID: Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)... https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ -- @ChrisGrundemann http://chrisgrundemann.com From patrick at ianai.net Fri Oct 21 15:48:21 2016 From: patrick at ianai.net (Patrick W. Gilmore) Date: Fri, 21 Oct 2016 11:48:21 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> I cannot give additional info other than what?s been on ?public media?. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick > On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: > > Does anyone have any additional details? Seems to be over now, but I'm very > curious about the specifics of such a highly impactful attack (and it's > timing following NANOG 68)... > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ > > -- > @ChrisGrundemann > http://chrisgrundemann.com From rar at syssrc.com Fri Oct 21 15:57:47 2016 From: rar at syssrc.com (rar) Date: Fri, 21 Oct 2016 15:57:47 +0000 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <22433E7377ECE14D9518DB3338A32EA30FAFDA09@ExchMB10.syssrcad.syssrc.com> Anyone want a quick consulting gig helping us configure BCP38 and BCP84? Configurations is all cisco Edge routers connect to Verizon, Level 3 Fiber Each Edge router talks to two BGP routers. $150/hour, I'm guessing it is only an hour for somebody to explain, and guide us through the configuration, but OK if longer. Thanks. Bob Roswell broswell at syssrc.com 410-771-5544 ext 4336 Computer Museum Highlights -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Patrick W. Gilmore Sent: Friday, October 21, 2016 11:48 AM To: NANOG list Subject: Re: Dyn DDoS this AM? I cannot give additional info other than what?s been on ?public media?. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick > On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: > > Does anyone have any additional details? Seems to be over now, but I'm > very curious about the specifics of such a highly impactful attack > (and it's timing following NANOG 68)... > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotif > y-reddit/ > > -- > @ChrisGrundemann > http://chrisgrundemann.com From nanog at ics-il.net Fri Oct 21 16:01:27 2016 From: nanog at ics-il.net (Mike Hammett) Date: Fri, 21 Oct 2016 11:01:27 -0500 (CDT) Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> Are there sites that can test your BCP38\84 compliance? I'm okay, but interested in what I can share to raise awareness. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Patrick W. Gilmore" To: "NANOG list" Sent: Friday, October 21, 2016 10:48:21 AM Subject: Re: Dyn DDoS this AM? I cannot give additional info other than what?s been on ?public media?. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick > On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: > > Does anyone have any additional details? Seems to be over now, but I'm very > curious about the specifics of such a highly impactful attack (and it's > timing following NANOG 68)... > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ > > -- > @ChrisGrundemann > http://chrisgrundemann.com From outsider at scarynet.org Fri Oct 21 16:02:02 2016 From: outsider at scarynet.org (Alexander Maassen) Date: Fri, 21 Oct 2016 18:02:02 +0200 Subject: Dyn DDoS this AM? Message-ID: Feel free to feed me with attack sources. Once those companies notice their precious mail does not arrive at clients. They will attempt to fix things. Sad but true. Kind regards, Alexander Maassen - Technical Maintenance Engineer Parkstad Support BV- Maintainer DroneBL- Peplink Certified Engineer -------- Oorspronkelijk bericht --------Van: "Patrick W. Gilmore" Datum: 21-10-16 17:48 (GMT+01:00) Aan: NANOG list Onderwerp: Re: Dyn DDoS this AM? I cannot give additional info other than what?s been on ?public media?. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick > On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: > > Does anyone have any additional details? Seems to be over now, but I'm very > curious about the specifics of such a highly impactful attack (and it's > timing following NANOG 68)... > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ > > -- > @ChrisGrundemann > http://chrisgrundemann.com From Matthew.Black at csulb.edu Fri Oct 21 16:05:42 2016 From: Matthew.Black at csulb.edu (Matthew Black) Date: Fri, 21 Oct 2016 16:05:42 +0000 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: LA Times: Why sites like Twitter and Spotify were down for East Coast users this morning http://www.latimes.com/business/la-fi-tn-dyn-attack-20161021-snap-story.html -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Chris Grundemann Sent: Friday, October 21, 2016 7:56 AM To: nanog at nanog.org Subject: Dyn DDoS this AM? Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)... https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ -- @ChrisGrundemann http://chrisgrundemann.com From rdobbins at arbor.net Fri Oct 21 16:09:10 2016 From: rdobbins at arbor.net (Roland Dobbins) Date: Fri, 21 Oct 2016 23:09:10 +0700 Subject: Dyn DDoS this AM? In-Reply-To: <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> Message-ID: On 21 Oct 2016, at 23:01, Mike Hammett wrote: > Are there sites that can test your BCP38\84 compliance? ----------------------------------- Roland Dobbins From patrick at ianai.net Fri Oct 21 16:12:43 2016 From: patrick at ianai.net (Patrick W. Gilmore) Date: Fri, 21 Oct 2016 12:12:43 -0400 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: Attack has re-started. This is the time, folks. Rally the troops, offer help, watch your flow. STOP THIS NOW. -- TTFN, patrick > On Oct 21, 2016, at 11:48 AM, Patrick W. Gilmore wrote: > > I cannot give additional info other than what?s been on ?public media?. > > However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. > > To Dyn and everyone else being attacked: > The community is behind you. There are problems, but if we stick together, we can beat these miscreants. > > To the miscreants: > You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. > > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. > > But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. > > > OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. > > -- > TTFN, > patrick > >> On Oct 21, 2016, at 10:55 AM, Chris Grundemann > wrote: >> >> Does anyone have any additional details? Seems to be over now, but I'm very >> curious about the specifics of such a highly impactful attack (and it's >> timing following NANOG 68)... >> >> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ >> >> -- >> @ChrisGrundemann >> http://chrisgrundemann.com > From patrick at ianai.net Fri Oct 21 16:13:26 2016 From: patrick at ianai.net (Patrick W. Gilmore) Date: Fri, 21 Oct 2016 12:13:26 -0400 Subject: Dyn DDoS this AM? In-Reply-To: <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> Message-ID: <5102F1B6-0F5C-4F81-9C11-3DE0E92405F2@ianai.net> https://www.caida.org/projects/spoofer/ -- TTFN, patrick > On Oct 21, 2016, at 12:01 PM, Mike Hammett wrote: > > Are there sites that can test your BCP38\84 compliance? I'm okay, but interested in what I can share to raise awareness. > > > > > ----- > Mike Hammett > Intelligent Computing Solutions > http://www.ics-il.com > > Midwest-IX > http://www.midwest-ix.com > > ----- Original Message ----- > > From: "Patrick W. Gilmore" > To: "NANOG list" > Sent: Friday, October 21, 2016 10:48:21 AM > Subject: Re: Dyn DDoS this AM? > > I cannot give additional info other than what?s been on ?public media?. > > However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. > > To Dyn and everyone else being attacked: > The community is behind you. There are problems, but if we stick together, we can beat these miscreants. > > To the miscreants: > You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. > > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. > > But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. > > > OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. > > -- > TTFN, > patrick > >> On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: >> >> Does anyone have any additional details? Seems to be over now, but I'm very >> curious about the specifics of such a highly impactful attack (and it's >> timing following NANOG 68)... >> >> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ >> >> -- >> @ChrisGrundemann >> http://chrisgrundemann.com > From sethm at rollernet.us Fri Oct 21 16:15:04 2016 From: sethm at rollernet.us (Seth Mattinen) Date: Fri, 21 Oct 2016 09:15:04 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: On 10/21/16 09:05, Matthew Black wrote: > LA Times: Why sites like Twitter and Spotify were down for East Coast users this morning > http://www.latimes.com/business/la-fi-tn-dyn-attack-20161021-snap-story.html I actually can't resolve twitter.com this morning and I'm west coast. None of the four listed DNS servers are responding. twitter.com. 172800 IN NS ns1.p34.dynect.net. twitter.com. 172800 IN NS ns2.p34.dynect.net. twitter.com. 172800 IN NS ns3.p34.dynect.net. twitter.com. 172800 IN NS ns4.p34.dynect.net. Trace routes seem to point towards San Jose or Palo Alto or Los Angeles. ~Seth From ahebert at pubnix.net Fri Oct 21 16:30:44 2016 From: ahebert at pubnix.net (Alain Hebert) Date: Fri, 21 Oct 2016 12:30:44 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <55cd5727-381e-5f71-7a53-ad99cdd27e7d@pubnix.net> Rofl, Yeah good luck with that... 15+ years later and most of the actors that could fix that, for the planete, still refuses to do anything. Now you can start the usual circular discussion that goes nowhere after 3 days... PS: yeah usual BCP38 rant... but its friday. ----- Alain Hebert ahebert at pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/21/16 12:12, Patrick W. Gilmore wrote: > Attack has re-started. This is the time, folks. Rally the troops, offer help, watch your flow. > > STOP THIS NOW. > From bross at pobox.com Fri Oct 21 16:38:22 2016 From: bross at pobox.com (Brandon Ross) Date: Fri, 21 Oct 2016 12:38:22 -0400 (EDT) Subject: Dyn DDoS this AM? In-Reply-To: <22433E7377ECE14D9518DB3338A32EA30FAFDA09@ExchMB10.syssrcad.syssrc.com> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <22433E7377ECE14D9518DB3338A32EA30FAFDA09@ExchMB10.syssrcad.syssrc.com> Message-ID: On Fri, 21 Oct 2016, rar wrote: > Anyone want a quick consulting gig helping us configure BCP38 and BCP84? > > Configurations is all cisco > Edge routers connect to Verizon, Level 3 Fiber > Each Edge router talks to two BGP routers. > > $150/hour, I'm guessing it is only an hour for somebody to explain, and > guide us through the configuration, but OK if longer. Sure, we'll do it. That rate is quite a bit less than our normal retail rate, but in the spirit that Patrick posted about, Network Utility Force will be happy to provide you or any other operator resources at that rate to help configure BCP38 and BCP84. Anyone serious about that, email me privately at bross at netuf.net and we'll put paperwork together. -- Brandon Ross Yahoo & AIM: BrandonNRoss Voice: +1-404-635-6667 ICQ: 2269442 Signal Secure SMS: +1-404-644-9628 Skype: brandonross Schedule a meeting: http://www.doodle.com/bross From dhubbard at dino.hostasaurus.com Fri Oct 21 16:40:24 2016 From: dhubbard at dino.hostasaurus.com (David Hubbard) Date: Fri, 21 Oct 2016 16:40:24 +0000 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: Do we know the attack destinations so we can watch transit traffic destined for it to help sources that may be unaware? David From mark.tinka at seacom.mu Fri Oct 21 16:44:38 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Fri, 21 Oct 2016 18:44:38 +0200 Subject: MPLS in the campus Network? In-Reply-To: <20161021141918.GA65694@bts.sk> References: <20161021141918.GA65694@bts.sk> Message-ID: <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> On 21/Oct/16 16:19, Marian ?urkovi? wrote: > > Much easier to setup, operate & maintain than MPLS and obviously much > lower cost. Based on 6-months production experience, my recommendation > would be to stay away from MPLS in the campus. I'd be curious to hear what MPLS-specific issues you faced in the 6 months you had to operate such a network. Been running IP/MPLS Core, Edge and Access networks for over 15 years, and apart from bugs which affect any protocol or feature implementation, I can't say it has been a nightmare to operate to the point of not recommending it. I have far fewer words to say about STP, although - I'll admit - I've never run TRILL. Mark. From patrick at ianai.net Fri Oct 21 16:45:52 2016 From: patrick at ianai.net (Patrick W. Gilmore) Date: Fri, 21 Oct 2016 12:45:52 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: On Oct 21, 2016, at 12:40 PM, David Hubbard wrote: > > Do we know the attack destinations so we can watch transit traffic destined for it to help sources that may be unaware? My guess is you should track anything to as33517. -- TTFN, patrick From smeuse at mara.org Fri Oct 21 16:47:19 2016 From: smeuse at mara.org (Steve Meuse) Date: Fri, 21 Oct 2016 12:47:19 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <1122587289.1986.1477065684838.JavaMail.mhammett@ThunderFuck> Message-ID: On Fri, Oct 21, 2016 at 12:09 PM, Roland Dobbins wrote: > On 21 Oct 2016, at 23:01, Mike Hammett wrote: > > > Are there sites that can test your BCP38\84 compliance? > > Quick note: If anyone has this installed already on OSX, bring up the console and see if it's still running. I discovered (while watching the NANOG preso) that mine had an issue and was failing silently. Re-installing the new version fixed the issue. The funny part of the story, looking through the logs to see which networks I roamed on that were spoofable, the only positive hit was for the NANOG conference network in Chicago :) -Steve From jhazesnooty at gmail.com Fri Oct 21 17:02:24 2016 From: jhazesnooty at gmail.com (Javier Solis) Date: Fri, 21 Oct 2016 12:02:24 -0500 Subject: MPLS in the campus Network? In-Reply-To: <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> Message-ID: Our campus started off with L2 vlans spanning through the core, but we migrated to routing in the core and moved our many spanning tree/broadcast domains to the edge of buildings fronted by redundant routing with ecmp to a redundant core utilizing ospf. In a campus network the challenge becomes extending subnets across your core. You may have a college that started in one building with their own /24, but now have offices and labs in other buildings. They want to stay on the same network, but that's not feasible with the routed core setup without some other technology overlay. We end up not being able to extend the L2 like we did in the past and today we modify router ACL's to allow communications. If you already have hundreds of vlans spanned across the network, it's hard to get a campus to migrate to the routed core. I think this may be one of Marks challenge, correct me if I'm wrong please. With that said, what are the best options to be able to cost effectively scale without using vlans and maintaining a routed core? What technology would someone suggest (mpls, vxlan,etc) to be the best possible solution? Thank you to the participants in the discussion. I always enjoy reading comments posted. -Javier On Oct 21, 2016 11:46 AM, "Mark Tinka" wrote: > > > On 21/Oct/16 16:19, Marian ?urkovi? wrote: > > > > > Much easier to setup, operate & maintain than MPLS and obviously much > > lower cost. Based on 6-months production experience, my recommendation > > would be to stay away from MPLS in the campus. > > I'd be curious to hear what MPLS-specific issues you faced in the 6 > months you had to operate such a network. > > Been running IP/MPLS Core, Edge and Access networks for over 15 years, > and apart from bugs which affect any protocol or feature implementation, > I can't say it has been a nightmare to operate to the point of not > recommending it. > > I have far fewer words to say about STP, although - I'll admit - I've > never run TRILL. > > Mark. > From bicknell at ufp.org Fri Oct 21 17:45:15 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Fri, 21 Oct 2016 10:45:15 -0700 Subject: MPLS in the campus Network? In-Reply-To: References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> Message-ID: <20161021174514.GA48761@ussenterprise.ufp.org> In a message written on Fri, Oct 21, 2016 at 12:02:24PM -0500, Javier Solis wrote: > In a campus network the challenge becomes extending subnets across your > core. You may have a college that started in one building with their own > /24, but now have offices and labs in other buildings. They want to stay on > the same network, but that's not feasible with the routed core setup > without some other technology overlay. We end up not being able to extend > the L2 like we did in the past and today we modify router ACL's to allow > communications. If you already have hundreds of vlans spanned across the > network, it's hard to get a campus to migrate to the routed core. I think > this may be one of Marks challenge, correct me if I'm wrong please. FWIW, if I had to solve the "college across buildings with common access control" problem I would create MPLS L3 VPN's, one subnet per building (where it is a VLAN inside of a building), with a "firewall in the cloud" somewhere to get between VLAN's with all of the policy in one place. No risk of the L2 across buildings mess, including broadcast and multicast issues at L2. All tidy L3 routing. Can use a real firewall between L3 VPN instances to get real policy tools (AV, URL Filtering, Malware detection, etc) rather than router ACL's. Scales to huge sizes because it's all L3 based. Combine with 802.1x port authentication and NAC, and in theory every L3 VPN could be in every building, with each port dynamically assigning the VLAN based on the user's login! Imagine never manually configuring them again. Write a script that makes all the colleges (20? 40? 60?) appear in every building all attached to their own MPLS VPN's, and then the NAC handles port assignment. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From cscora at apnic.net Fri Oct 21 18:01:34 2016 From: cscora at apnic.net (Routing Analysis Role Account) Date: Sat, 22 Oct 2016 04:01:34 +1000 (AEST) Subject: Weekly Routing Table Report Message-ID: <20161021180135.04024C55A1@thyme.apnic.net> This is an automated weekly mailing describing the state of the Internet Routing Table as seen from APNIC's router in Japan. The posting is sent to APOPS, NANOG, AfNOG, AusNOG, SANOG, PacNOG, SAFNOG, SdNOG, BJNOG, CaribNOG and the RIPE Routing WG. Daily listings are sent to bgp-stats at lists.apnic.net For historical data, please see http://thyme.rand.apnic.net. If you have any comments please contact Philip Smith . Routing Table Report 04:00 +10GMT Sat 22 Oct, 2016 Report Website: http://thyme.rand.apnic.net Detailed Analysis: http://thyme.rand.apnic.net/current/ Analysis Summary ---------------- BGP routing table entries examined: 615166 Prefixes after maximum aggregation (per Origin AS): 220426 Deaggregation factor: 2.79 Unique aggregates announced (without unneeded subnets): 299813 Total ASes present in the Internet Routing Table: 55044 Prefixes per ASN: 11.18 Origin-only ASes present in the Internet Routing Table: 36336 Origin ASes announcing only one prefix: 15328 Transit ASes present in the Internet Routing Table: 6525 Transit-only ASes present in the Internet Routing Table: 166 Average AS path length visible in the Internet Routing Table: 4.3 Max AS path length visible: 39 Max AS path prepend of ASN ( 55644) 31 Prefixes from unregistered ASNs in the Routing Table: 56 Unregistered ASNs in the Routing Table: 15 Number of 32-bit ASNs allocated by the RIRs: 15849 Number of 32-bit ASNs visible in the Routing Table: 12183 Prefixes from 32-bit ASNs in the Routing Table: 49137 Number of bogon 32-bit ASNs visible in the Routing Table: 247 Special use prefixes present in the Routing Table: 0 Prefixes being announced from unallocated address space: 353 Number of addresses announced to Internet: 2829985572 Equivalent to 168 /8s, 174 /16s and 39 /24s Percentage of available address space announced: 76.4 Percentage of allocated address space announced: 76.4 Percentage of available address space allocated: 100.0 Percentage of address space in use by end-sites: 98.3 Total number of prefixes smaller than registry allocations: 200615 APNIC Region Analysis Summary ----------------------------- Prefixes being announced by APNIC Region ASes: 156347 Total APNIC prefixes after maximum aggregation: 42794 APNIC Deaggregation factor: 3.65 Prefixes being announced from the APNIC address blocks: 170253 Unique aggregates announced from the APNIC address blocks: 69829 APNIC Region origin ASes present in the Internet Routing Table: 5173 APNIC Prefixes per ASN: 32.91 APNIC Region origin ASes announcing only one prefix: 1143 APNIC Region transit ASes present in the Internet Routing Table: 946 Average APNIC Region AS path length visible: 4.3 Max APNIC Region AS path length visible: 37 Number of APNIC region 32-bit ASNs visible in the Routing Table: 2439 Number of APNIC addresses announced to Internet: 759730500 Equivalent to 45 /8s, 72 /16s and 145 /24s APNIC AS Blocks 4608-4864, 7467-7722, 9216-10239, 17408-18431 (pre-ERX allocations) 23552-24575, 37888-38911, 45056-46079, 55296-56319, 58368-59391, 63488-64098, 64297-64395, 131072-137529 APNIC Address Blocks 1/8, 14/8, 27/8, 36/8, 39/8, 42/8, 43/8, 49/8, 58/8, 59/8, 60/8, 61/8, 101/8, 103/8, 106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8, 116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8, 123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8, 163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8, 203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8, 222/8, 223/8, ARIN Region Analysis Summary ---------------------------- Prefixes being announced by ARIN Region ASes: 185253 Total ARIN prefixes after maximum aggregation: 89516 ARIN Deaggregation factor: 2.07 Prefixes being announced from the ARIN address blocks: 191091 Unique aggregates announced from the ARIN address blocks: 88732 ARIN Region origin ASes present in the Internet Routing Table: 16165 ARIN Prefixes per ASN: 11.82 ARIN Region origin ASes announcing only one prefix: 5671 ARIN Region transit ASes present in the Internet Routing Table: 1719 Average ARIN Region AS path length visible: 3.8 Max ARIN Region AS path length visible: 23 Number of ARIN region 32-bit ASNs visible in the Routing Table: 1527 Number of ARIN addresses announced to Internet: 1105174176 Equivalent to 65 /8s, 223 /16s and 158 /24s ARIN AS Blocks 1-1876, 1902-2042, 2044-2046, 2048-2106 (pre-ERX allocations) 2138-2584, 2615-2772, 2823-2829, 2880-3153 3354-4607, 4865-5119, 5632-6655, 6912-7466 7723-8191, 10240-12287, 13312-15359, 16384-17407 18432-20479, 21504-23551, 25600-26591, 26624-27647, 29696-30719, 31744-33791 35840-36863, 39936-40959, 46080-47103 53248-55295, 62464-63487, 64198-64296, 393216-397212 ARIN Address Blocks 3/8, 4/8, 6/8, 7/8, 8/8, 9/8, 11/8, 12/8, 13/8, 15/8, 16/8, 17/8, 18/8, 19/8, 20/8, 21/8, 22/8, 23/8, 24/8, 26/8, 28/8, 29/8, 30/8, 32/8, 33/8, 34/8, 35/8, 38/8, 40/8, 44/8, 45/8, 47/8, 48/8, 50/8, 52/8, 53/8, 54/8, 55/8, 56/8, 57/8, 63/8, 64/8, 65/8, 66/8, 67/8, 68/8, 69/8, 70/8, 71/8, 72/8, 73/8, 74/8, 75/8, 76/8, 96/8, 97/8, 98/8, 99/8, 100/8, 104/8, 107/8, 108/8, 128/8, 129/8, 130/8, 131/8, 132/8, 134/8, 135/8, 136/8, 137/8, 138/8, 139/8, 140/8, 142/8, 143/8, 144/8, 146/8, 147/8, 148/8, 149/8, 152/8, 155/8, 156/8, 157/8, 158/8, 159/8, 160/8, 161/8, 162/8, 164/8, 165/8, 166/8, 167/8, 168/8, 169/8, 170/8, 172/8, 173/8, 174/8, 184/8, 192/8, 198/8, 199/8, 204/8, 205/8, 206/8, 207/8, 208/8, 209/8, 214/8, 215/8, 216/8, RIPE Region Analysis Summary ---------------------------- Prefixes being announced by RIPE Region ASes: 146917 Total RIPE prefixes after maximum aggregation: 72154 RIPE Deaggregation factor: 2.04 Prefixes being announced from the RIPE address blocks: 157661 Unique aggregates announced from the RIPE address blocks: 97134 RIPE Region origin ASes present in the Internet Routing Table: 18137 RIPE Prefixes per ASN: 8.69 RIPE Region origin ASes announcing only one prefix: 7798 RIPE Region transit ASes present in the Internet Routing Table: 3032 Average RIPE Region AS path length visible: 4.4 Max RIPE Region AS path length visible: 29 Number of RIPE region 32-bit ASNs visible in the Routing Table: 5079 Number of RIPE addresses announced to Internet: 708755712 Equivalent to 42 /8s, 62 /16s and 193 /24s RIPE AS Blocks 1877-1901, 2043, 2047, 2107-2136, 2585-2614 (pre-ERX allocations) 2773-2822, 2830-2879, 3154-3353, 5377-5631 6656-6911, 8192-9215, 12288-13311, 15360-16383 20480-21503, 24576-25599, 28672-29695 30720-31743, 33792-35839, 38912-39935 40960-45055, 47104-52223, 56320-58367 59392-61439, 61952-62463, 64396-64495 196608-207259 RIPE Address Blocks 2/8, 5/8, 25/8, 31/8, 37/8, 46/8, 51/8, 62/8, 77/8, 78/8, 79/8, 80/8, 81/8, 82/8, 83/8, 84/8, 85/8, 86/8, 87/8, 88/8, 89/8, 90/8, 91/8, 92/8, 93/8, 94/8, 95/8, 109/8, 141/8, 145/8, 151/8, 176/8, 178/8, 185/8, 188/8, 193/8, 194/8, 195/8, 212/8, 213/8, 217/8, LACNIC Region Analysis Summary ------------------------------ Prefixes being announced by LACNIC Region ASes: 62556 Total LACNIC prefixes after maximum aggregation: 12624 LACNIC Deaggregation factor: 4.96 Prefixes being announced from the LACNIC address blocks: 78731 Unique aggregates announced from the LACNIC address blocks: 37419 LACNIC Region origin ASes present in the Internet Routing Table: 2480 LACNIC Prefixes per ASN: 31.75 LACNIC Region origin ASes announcing only one prefix: 547 LACNIC Region transit ASes present in the Internet Routing Table: 583 Average LACNIC Region AS path length visible: 4.9 Max LACNIC Region AS path length visible: 22 Number of LACNIC region 32-bit ASNs visible in the Routing Table: 2870 Number of LACNIC addresses announced to Internet: 170666048 Equivalent to 10 /8s, 44 /16s and 40 /24s LACNIC AS Blocks 26592-26623, 27648-28671, 52224-53247, 61440-61951, 64099-64197, 262144-266652 + ERX transfers LACNIC Address Blocks 177/8, 179/8, 181/8, 186/8, 187/8, 189/8, 190/8, 191/8, 200/8, 201/8, AfriNIC Region Analysis Summary ------------------------------- Prefixes being announced by AfriNIC Region ASes: 14899 Total AfriNIC prefixes after maximum aggregation: 3330 AfriNIC Deaggregation factor: 4.47 Prefixes being announced from the AfriNIC address blocks: 17077 Unique aggregates announced from the AfriNIC address blocks: 6379 AfriNIC Region origin ASes present in the Internet Routing Table: 734 AfriNIC Prefixes per ASN: 23.27 AfriNIC Region origin ASes announcing only one prefix: 169 AfriNIC Region transit ASes present in the Internet Routing Table: 180 Average AfriNIC Region AS path length visible: 4.7 Max AfriNIC Region AS path length visible: 28 Number of AfriNIC region 32-bit ASNs visible in the Routing Table: 268 Number of AfriNIC addresses announced to Internet: 85176832 Equivalent to 5 /8s, 19 /16s and 178 /24s AfriNIC AS Blocks 36864-37887, 327680-328703 & ERX transfers AfriNIC Address Blocks 41/8, 102/8, 105/8, 154/8, 196/8, 197/8, APNIC Region per AS prefix count summary ---------------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5528 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3587 382 248 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2954 905 71 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9829 2697 1497 530 BSNL-NIB National Internet Backbone, IN 4766 2658 11127 737 KIXS-AS-KR Korea Telecom, KR 9808 2214 8781 42 CMNET-GD Guangdong Mobile Communication 4755 2053 429 216 TATACOMM-AS TATA Communications formerl 4808 1768 2290 519 CHINA169-BJ China Unicom Beijing Provin 45899 1584 1239 90 VNPT-AS-VN VNPT Corp, VN 24560 1552 517 222 AIRTELBROADBAND-AS-AP Bharti Airtel Ltd Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-APNIC ARIN Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 22773 3542 2965 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2196 405 110 MEGAPATH5-US - MegaPath Corporation, US 6389 2095 3671 39 BELLSOUTH-NET-BLK - BellSouth.net Inc., 20115 1943 1992 407 CHARTER-NET-HKY-NC - Charter Communicat 30036 1744 335 322 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 209 1722 5067 656 CENTURYLINK-US-LEGACY-QWEST - Qwest Com 6983 1678 848 227 ITCDELTA - Earthlink, Inc., US 701 1610 10674 671 UUNET - MCI Communications Services, In 7018 1485 20492 1048 ATT-INTERNET4 - AT&T Services, Inc., US 16509 1430 2778 479 AMAZON-02 - Amazon.com, Inc., US Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-ARIN RIPE Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 39891 3329 169 15 ALJAWWALSTC-AS , SA 20940 2817 1048 2011 AKAMAI-ASN1 , US 34984 1988 326 359 TELLCOM-AS , TR 13188 1571 99 58 TRIOLAN , UA 12479 1370 1018 46 UNI2-AS , ES 8551 1203 377 46 BEZEQ-INTERNATIONAL-AS Bezeqint Interne 6849 1142 355 21 UKRTELNET , UA 8402 1109 544 15 CORBINA-AS Russia, RU 12389 949 1134 422 ROSTELECOM-AS , RU 6830 863 2756 464 LGI-UPC formerly known as UPC Broadband Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-RIPE LACNIC Region per AS prefix count summary ----------------------------------------- ASN No of nets /20 equiv MaxAgg Description 10620 3511 542 161 Telmex Colombia S.A., CO 8151 2285 3356 555 Uninet S.A. de C.V., MX 7303 1542 963 248 Telecom Argentina S.A., AR 6503 1420 437 54 Axtel, S.A.B. de C.V., MX 11830 1322 367 57 Instituto Costarricense de Electricidad 6147 1207 377 28 Telefonica del Peru S.A.A., PE 28573 1008 2263 166 CLARO S.A., BR 7738 994 1882 40 Telemar Norte Leste S.A., BR 3816 976 472 200 COLOMBIA TELECOMUNICACIONES S.A. ESP, C 11172 905 125 75 Alestra, S. de R.L. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-LACNIC AfriNIC Region per AS prefix count summary ------------------------------------------ ASN No of nets /20 equiv MaxAgg Description 24863 1184 402 50 LINKdotNET-AS, EG 36903 680 342 128 MT-MPLS, MA 37611 664 67 6 Afrihost, ZA 36992 566 1373 25 ETISALAT-MISR, EG 8452 514 1471 15 TE-AS TE-AS, EG 37492 386 250 69 ORANGE-TN, TN 24835 365 614 17 RAYA-AS, EG 29571 335 37 12 CITelecom-AS, CI 15399 307 35 6 WANANCHI-KE, KE 2018 269 327 74 TENET-1, ZA Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-AFRINIC Global Per AS prefix count summary ---------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5528 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3587 382 248 TPG-INTERNET-AP TPG Telecom Limited, AU 22773 3542 2965 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3511 542 161 Telmex Colombia S.A., CO 39891 3329 169 15 ALJAWWALSTC-AS , SA 17974 2954 905 71 TELKOMNET-AS2-AP PT Telekomunikasi Indo 20940 2817 1048 2011 AKAMAI-ASN1 , US 9829 2697 1497 530 BSNL-NIB National Internet Backbone, IN 4766 2658 11127 737 KIXS-AS-KR Korea Telecom, KR 8151 2285 3356 555 Uninet S.A. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet Global Per AS Maximum Aggr summary ---------------------------------- ASN No of nets Net Savings Description 22773 3542 3396 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3511 3350 Telmex Colombia S.A., CO 7545 3587 3339 TPG-INTERNET-AP TPG Telecom Limited, AU 39891 3329 3314 ALJAWWALSTC-AS , SA 17974 2954 2883 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9808 2214 2172 CMNET-GD Guangdong Mobile Communication 9829 2697 2167 BSNL-NIB National Internet Backbone, IN 18566 2196 2086 MEGAPATH5-US - MegaPath Corporation, US 6389 2095 2056 BELLSOUTH-NET-BLK - BellSouth.net Inc., 4766 2658 1921 KIXS-AS-KR Korea Telecom, KR Complete listing at http://thyme.rand.apnic.net/current/data-CIDRnet List of Unregistered Origin ASNs (Global) ----------------------------------------- Bad AS Designation Network Transit AS Description 65001 PRIVATE 5.143.176.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.200.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.208.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.216.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65000 PRIVATE 31.219.177.0/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65000 PRIVATE 31.219.177.128/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65512 PRIVATE 45.252.244.0/24 45899 VNPT-AS-VN VNPT Corp, VN 65512 PRIVATE 45.252.245.0/24 45899 VNPT-AS-VN VNPT Corp, VN Complete listing at http://thyme.rand.apnic.net/current/data-badAS Advertised Unallocated Addresses -------------------------------- Network Origin AS Description 27.100.7.0/24 56096 UNKNOWN 41.73.1.0/24 37004 -Reserved AS-, ZZ 41.73.2.0/24 37004 -Reserved AS-, ZZ 41.73.3.0/24 37004 -Reserved AS-, ZZ 41.73.4.0/24 37004 -Reserved AS-, ZZ 41.73.5.0/24 37004 -Reserved AS-, ZZ 41.73.6.0/24 37004 -Reserved AS-, ZZ 41.73.7.0/24 37004 -Reserved AS-, ZZ 41.73.8.0/24 37004 -Reserved AS-, ZZ 41.73.9.0/24 37004 -Reserved AS-, ZZ Complete listing at http://thyme.rand.apnic.net/current/data-add-IANA Number of prefixes announced per prefix length (Global) ------------------------------------------------------- /1:0 /2:0 /3:0 /4:0 /5:0 /6:0 /7:0 /8:16 /9:13 /10:36 /11:102 /12:274 /13:521 /14:1049 /15:1775 /16:13148 /17:7836 /18:13041 /19:25443 /20:38819 /21:40354 /22:69194 /23:59899 /24:342445 /25:447 /26:343 /27:275 /28:57 /29:33 /30:13 /31:1 /32:32 Advertised prefixes smaller than registry allocations ----------------------------------------------------- ASN No of nets Total ann. Description 39891 2896 3329 ALJAWWALSTC-AS , SA 22773 2764 3542 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2088 2196 MEGAPATH5-US - MegaPath Corporation, US 30036 1558 1744 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 10620 1430 3511 Telmex Colombia S.A., CO 6389 1393 2095 BELLSOUTH-NET-BLK - BellSouth.net Inc., 6983 1326 1678 ITCDELTA - Earthlink, Inc., US 13188 1300 1571 TRIOLAN , UA 34984 1272 1988 TELLCOM-AS , TR 11492 1232 1326 CABLEONE - CABLE ONE, INC., US Complete listing at http://thyme.rand.apnic.net/current/data-sXXas-nos Number of /24s announced per /8 block (Global) ---------------------------------------------- 1:1539 2:808 4:23 5:2172 6:32 8:993 12:1804 13:49 14:1739 15:46 16:2 17:100 18:126 20:48 23:1872 24:1811 27:2358 31:1802 32:83 33:4 35:3 36:336 37:2363 38:1279 39:33 40:100 41:2952 42:463 43:1874 44:47 45:2135 46:2581 47:101 49:1213 50:904 51:13 52:560 53:2 54:351 55:6 56:4 57:40 58:1572 59:993 60:378 61:1838 62:1531 63:1934 64:4571 65:2171 66:4298 67:2253 68:1138 69:3297 70:1290 71:562 72:2077 74:2580 75:403 76:416 77:1432 78:1236 79:918 80:1304 81:1402 82:986 83:705 84:887 85:1589 86:478 87:1133 88:555 89:2045 90:222 91:6114 92:930 93:2357 94:2415 95:2483 96:549 97:345 98:945 99:90 100:147 101:1167 103:12636 104:2523 105:125 106:440 107:1434 108:780 109:2436 110:1278 111:1638 112:1085 113:1141 114:1095 115:1686 116:1672 117:1566 118:2070 119:1575 120:938 121:1074 122:2274 123:2000 124:1572 125:1819 128:689 129:449 130:419 131:1335 132:610 133:175 134:512 135:200 136:399 137:407 138:1779 139:432 140:602 141:452 142:672 143:918 144:733 145:165 146:956 147:658 148:1382 149:542 150:661 151:888 152:684 153:300 154:703 155:934 156:531 157:530 158:410 159:1261 160:526 161:732 162:2407 163:578 164:780 165:1109 166:345 167:1187 168:2297 169:666 170:2018 171:254 172:773 173:1772 174:764 175:675 176:1737 177:4117 178:2333 179:1187 180:2151 181:1916 182:2094 183:1010 184:828 185:7753 186:3223 187:2153 188:2220 189:1760 190:7800 191:1316 192:9058 193:5745 194:4479 195:3842 196:1793 197:1241 198:5598 199:5810 200:7145 201:3710 202:10141 203:9818 204:4511 205:2741 206:2994 207:3088 208:3980 209:3891 210:3871 211:2035 212:2742 213:2373 214:876 215:68 216:5726 217:1982 218:812 219:598 220:1641 221:878 222:696 223:1329 End of report From bdavies at linkedin.com Fri Oct 21 15:52:25 2016 From: bdavies at linkedin.com (Brian Davies) Date: Fri, 21 Oct 2016 08:52:25 -0700 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: +1! Well said, Patrick. B On Friday, October 21, 2016, Patrick W. Gilmore wrote: > I cannot give additional info other than what?s been on ?public media?. > > However, I would very much like to say that this is a horrific trend on > the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can > Not Stand. See Krebs? on the Democratization of Censorship. See lots of > other things. > > To Dyn and everyone else being attacked: > The community is behind you. There are problems, but if we stick together, > we can beat these miscreants. > > To the miscreants: > You will not succeed. Search "churchill on the beaches?. It?s a bit > melodramatic, but it?s how I feel at this moment. > > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I > do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, > that doesn?t help Mirai, but it still helps. There are many other things > you can do as well. > > But a lot of it is just willingness to help. When someone asks you to help > trace an attack, do not let the request sit for a while. Damage is being > done. Help your neighbor. When someone?s house is burning, your current > project, your lunch break, whatever else you are doing is almost certainly > less important. If we stick together and help each other, we can - we WILL > - win this war. If we are apathetic, we have already lost. > > > OK, enough motivational speaking for today. But take this to heart. Our > biggest problem is people thinking they cannot or do not want to help. > > -- > TTFN, > patrick > > > On Oct 21, 2016, at 10:55 AM, Chris Grundemann > wrote: > > > > Does anyone have any additional details? Seems to be over now, but I'm > very > > curious about the specifics of such a highly impactful attack (and it's > > timing following NANOG 68)... > > > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- > twitter-spotify-reddit/ > > > > -- > > @ChrisGrundemann > > http://chrisgrundemann.com > > From youssef.ghorbal at gmail.com Fri Oct 21 20:18:07 2016 From: youssef.ghorbal at gmail.com (Youssef Ghorbal) Date: Fri, 21 Oct 2016 22:18:07 +0200 Subject: MPLS in the campus Network? In-Reply-To: <20161021174514.GA48761@ussenterprise.ufp.org> References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> <20161021174514.GA48761@ussenterprise.ufp.org> Message-ID: > FWIW, if I had to solve the "college across buildings with common > access control" problem I would create MPLS L3 VPN's, one subnet > per building (where it is a VLAN inside of a building), with a > "firewall in the cloud" somewhere to get between VLAN's with all > of the policy in one place. > > No risk of the L2 across buildings mess, including broadcast and > multicast issues at L2. All tidy L3 routing. Can use a real > firewall between L3 VPN instances to get real policy tools (AV, URL > Filtering, Malware detection, etc) rather than router ACL's. Scales > to huge sizes because it's all L3 based. Until people start complaining they can no more auto discover their Time Capsule left in the other building whereas their colleagues in the other building can etc etc. All fancy discover protocols breaks without L2 continuity ! Welcome to the campus network nightmare :) For now, there is no perfect solution ! either you cope with L2 hell or users inconvenience (and yes people tend to think that the campus network is expected to work as their home network) I've also stumbled upon some "Building Automation and Control Networks" (BACnet/IP for instance) where each building has some automats that all needs to be in the same network segment. > Combine with 802.1x port authentication and NAC, and in theory every > L3 VPN could be in every building, with each port dynamically assigning > the VLAN based on the user's login! Imagine never manually configuring > them again. Write a script that makes all the colleges (20? 40? 60?) > appear in every building all attached to their own MPLS VPN's, and > then the NAC handles port assignment. Here again, it's perfect until you start coping with old stuff, all fancy new ethernet capable "things" or scientific/industrial equipments. The "802.1x what ? it's plug'n play man !" attitude. (my experience is with research institutes/academy kind of campuses) Youssef Ghorbal From ahebert at pubnix.net Fri Oct 21 21:37:08 2016 From: ahebert at pubnix.net (Alain Hebert) Date: Fri, 21 Oct 2016 17:37:08 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: Just a FYI, That "horrific trend" has been happening since some techie got dissed on an IRC channel over 20 years ago. He used a bunch of hosted putters to ICMP flood the IRC server. Whatever the community is behind, until the carriers decide to wise up this will keep happening, that is without talking about the industries being developed around DDoSes events. Enjoy your weekend. ( I ain't on call anymore anyway =D ) ----- Alain Hebert ahebert at pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/21/16 11:52, Brian Davies via NANOG wrote: > +1! > > Well said, Patrick. > > B > > On Friday, October 21, 2016, Patrick W. Gilmore wrote: > >> I cannot give additional info other than what?s been on ?public media?. >> >> However, I would very much like to say that this is a horrific trend on >> the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can >> Not Stand. See Krebs? on the Democratization of Censorship. See lots of >> other things. >> >> To Dyn and everyone else being attacked: >> The community is behind you. There are problems, but if we stick together, >> we can beat these miscreants. >> >> To the miscreants: >> You will not succeed. Search "churchill on the beaches?. It?s a bit >> melodramatic, but it?s how I feel at this moment. >> >> To the rest of the community: >> If you can help, please do. I know a lot of you are thinking ?what can I >> do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, >> that doesn?t help Mirai, but it still helps. There are many other things >> you can do as well. >> >> But a lot of it is just willingness to help. When someone asks you to help >> trace an attack, do not let the request sit for a while. Damage is being >> done. Help your neighbor. When someone?s house is burning, your current >> project, your lunch break, whatever else you are doing is almost certainly >> less important. If we stick together and help each other, we can - we WILL >> - win this war. If we are apathetic, we have already lost. >> >> >> OK, enough motivational speaking for today. But take this to heart. Our >> biggest problem is people thinking they cannot or do not want to help. >> >> -- >> TTFN, >> patrick >> >>> On Oct 21, 2016, at 10:55 AM, Chris Grundemann > > wrote: >>> Does anyone have any additional details? Seems to be over now, but I'm >> very >>> curious about the specifics of such a highly impactful attack (and it's >>> timing following NANOG 68)... >>> >>> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- >> twitter-spotify-reddit/ >>> -- >>> @ChrisGrundemann >>> http://chrisgrundemann.com >> From james.cutler at consultant.com Fri Oct 21 21:40:33 2016 From: james.cutler at consultant.com (James R Cutler) Date: Fri, 21 Oct 2016 17:40:33 -0400 Subject: MPLS in the campus Network? In-Reply-To: References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> <20161021174514.GA48761@ussenterprise.ufp.org> Message-ID: On Oct 21, 2016, at 4:18 PM, Youssef Ghorbal wrote, in part: > > Until people start complaining they can no more auto discover their > Time Capsule left in the other building whereas their colleagues in > the other building can etc etc. All fancy discover protocols breaks > without L2 continuity ! Minor Correction: Correctly configured*, an Airport Extreme basestation (Time Capsule or not) does not require L2 connectivity to discover. In fact, Wide Area is used for discovery of many services not necessarily reachable by L2 connectivity. Apple?s Back to My Mac service is one example. *Apple?s "Back to My Mac? Wide Area Bonjour is enabled on an Airport basestation by entering appropriate Apple ID and password data in the Base Station tab as accessed by Airport Utility. James R. Cutler James.cutler at consultant.com PGP keys at http://pgp.mit.edu From davidbass570 at gmail.com Fri Oct 21 21:51:28 2016 From: davidbass570 at gmail.com (David Bass) Date: Fri, 21 Oct 2016 17:51:28 -0400 Subject: MPLS in the campus Network? In-Reply-To: <20161021174514.GA48761@ussenterprise.ufp.org> References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> <20161021174514.GA48761@ussenterprise.ufp.org> Message-ID: <61EF5503-B1A2-474B-A7C3-C029D6FF4D1A@gmail.com> This is exactly what we are recommending and building for our customers in that space. Most of the time the university network acts as a provider, so to me it only makes sense to use that type of tech. The biggest problem then is support, which could be something they are unwilling or unable to overcome. > On Oct 21, 2016, at 1:45 PM, Leo Bicknell wrote: > > In a message written on Fri, Oct 21, 2016 at 12:02:24PM -0500, Javier Solis wrote: >> In a campus network the challenge becomes extending subnets across your >> core. You may have a college that started in one building with their own >> /24, but now have offices and labs in other buildings. They want to stay on >> the same network, but that's not feasible with the routed core setup >> without some other technology overlay. We end up not being able to extend >> the L2 like we did in the past and today we modify router ACL's to allow >> communications. If you already have hundreds of vlans spanned across the >> network, it's hard to get a campus to migrate to the routed core. I think >> this may be one of Marks challenge, correct me if I'm wrong please. > > FWIW, if I had to solve the "college across buildings with common > access control" problem I would create MPLS L3 VPN's, one subnet > per building (where it is a VLAN inside of a building), with a > "firewall in the cloud" somewhere to get between VLAN's with all > of the policy in one place. > > No risk of the L2 across buildings mess, including broadcast and > multicast issues at L2. All tidy L3 routing. Can use a real > firewall between L3 VPN instances to get real policy tools (AV, URL > Filtering, Malware detection, etc) rather than router ACL's. Scales > to huge sizes because it's all L3 based. > > Combine with 802.1x port authentication and NAC, and in theory every > L3 VPN could be in every building, with each port dynamically assigning > the VLAN based on the user's login! Imagine never manually configuring > them again. Write a script that makes all the colleges (20? 40? 60?) > appear in every building all attached to their own MPLS VPN's, and > then the NAC handles port assignment. > > -- > Leo Bicknell - bicknell at ufp.org > PGP keys at http://www.ufp.org/~bicknell/ From randy at psg.com Fri Oct 21 21:58:05 2016 From: randy at psg.com (Randy Bush) Date: Sat, 22 Oct 2016 06:58:05 +0900 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: anyone who relies on a single dns provider is just asking for stuff such as this. randy From andrew.fried at gmail.com Fri Oct 21 22:09:30 2016 From: andrew.fried at gmail.com (Andrew Fried) Date: Fri, 21 Oct 2016 18:09:30 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: The brutal reality in todays world is that anyone that relies on the Internet is just asking for stuff like this. No service is safe. Andrew Andrew Fried andrew.fried at gmail.com On 10/21/16 5:58 PM, Randy Bush wrote: > anyone who relies on a single dns provider is just asking for stuff such > as this. > > randy > From mehmet at akcin.net Fri Oct 21 22:17:57 2016 From: mehmet at akcin.net (Mehmet Akcin) Date: Fri, 21 Oct 2016 15:17:57 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: amen. On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > anyone who relies on a single dns provider is just asking for stuff such > as this. > > randy > From randy at psg.com Fri Oct 21 22:21:33 2016 From: randy at psg.com (Randy Bush) Date: Sat, 22 Oct 2016 07:21:33 +0900 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: > amen. >> anyone who relies on a single dns provider is just asking for stuff >> such as this. part of the problem is that we think of it as attack surface when, in fact, it usually has more than two dimensions. randy From david at imgix.com Fri Oct 21 22:21:20 2016 From: david at imgix.com (David Birdsong) Date: Fri, 21 Oct 2016 15:21:20 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > anyone who relies on a single dns provider is just asking for stuff such > as this. > > randy > I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over. From randy at psg.com Fri Oct 21 22:25:21 2016 From: randy at psg.com (Randy Bush) Date: Sat, 22 Oct 2016 07:25:21 +0900 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: >> anyone who relies on a single dns provider is just asking for stuff such >> as this. > I'd love to hear how others are handling the overhead of managing two dns > providers. good question. staying in-band, hidden primary comes to mind. but i am sure clever minds can come up with more clever schemes. randy From nick at foobar.org Fri Oct 21 22:39:54 2016 From: nick at foobar.org (Nick Hilliard) Date: Fri, 21 Oct 2016 23:39:54 +0100 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <580A993A.8050501@foobar.org> Patrick W. Gilmore wrote: > Our biggest problem is people thinking they cannot or do not want to > help. Our biggest problem is that if the Internet community does not handle problems like this, governments and regulators may decide to intervene. If they do this in the wrong way, it will turn one major headache into two. Nick From drais at icantclick.org Fri Oct 21 22:45:16 2016 From: drais at icantclick.org (david raistrick) Date: Fri, 21 Oct 2016 18:45:16 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: On Fri, Oct 21, 2016 at 6:21 PM, David Birdsong wrote: > > I'd love to hear how others are handling the overhead of managing two dns > providers. Every time we brainstorm on it, we see it as blackhole of eng > effort WRT to keeping them in sync and and then waiting for TTLs to cut an > entire delegation over. > with the usual caveats - and I dont have any projects that currently need this but have in the past - pretty much every major dns provider allows you to ship them a full zone in some form or fashion. The effort to pull and ship a zone should be fairly minimal in and of itself. mixing your public zone providers in your authoritative NS records is also easy - and, depending on your registrar of choice, should be easy to manage changing those (including having non-public mirrors maintained that you can switch too..). setting TTLs that make sense for a design that supports change is also easy. the real developmental and architectural challenges are around what to do if the APIs you use to talk to your "primary" disappear and you need to consume them (creating new host entries, updating loadbalancer pools, whatever. we all have different and sometimes very diverse use cases for dns.). one approach - as randy suggested - is to switch to a purely hidden and self managed primary - which might mean running your own API stack in front of it to control whatever you need to control and change. this doesnt need to be a "real" dns server in todays world - the days of BIND style zone transfers are generally long gone anyway when you hit these scales and levels of intra complexity. then your zone-replication components that ship zone updates to your various external providers are shipping from the same place. at least in that case it's fully within your control - but dev time and complexity definitely comes into play. if your infra can survive internally without dns change control for the extent of an outage, that could be much easier to manage. anyway, random and incomplete thoughts - time ran out, work calls. ...david From joelja at bogus.com Fri Oct 21 23:04:41 2016 From: joelja at bogus.com (joel jaeggli) Date: Fri, 21 Oct 2016 16:04:41 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <0823e08b-77bc-6510-d611-62792fa45566@bogus.com> On 10/21/16 3:21 PM, David Birdsong wrote: > On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > >> anyone who relies on a single dns provider is just asking for stuff such >> as this. >> >> randy >> > I'd love to hear how others are handling the overhead of managing two dns > providers. Every time we brainstorm on it, we see it as blackhole of eng > effort WRT to keeping them in sync and and then waiting for TTLs to cut an > entire delegation over. Not all the ones you might choose based on scale support axfr... That's a bit of a problem for the most traditional approach to this., of those that do it's straight-forward to use one as the master for another, or use a hidden master. Your own master may have demonstrably lower availability then one or the other of your providers. getting two well considered choices to play nice with each other isn't that hard. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From niels=nanog at bakker.net Fri Oct 21 23:19:24 2016 From: niels=nanog at bakker.net (Niels Bakker) Date: Sat, 22 Oct 2016 01:19:24 +0200 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <20161021231924.GG45065@excession.tpb.net> >>>anyone who relies on a single dns provider is just asking for >>>stuff such as this. >>I'd love to hear how others are handling the overhead of managing >>two dns providers. * randy at psg.com (Randy Bush) [Sat 22 Oct 2016, 00:28 CEST]: >good question. staying in-band, hidden primary comes to mind. but >i am sure clever minds can come up with more clever schemes. The point of outsourcing DNS isn't just availability of static hostnames, it's the added services delivered, like returning different answers based on source of the question, even monitoring your infrastructure (or it reporting load into the DNS management system). That is very hard to replicate with two DNS providers. -- Niels. From mansaxel at besserwisser.org Fri Oct 21 23:19:57 2016 From: mansaxel at besserwisser.org (=?utf-8?B?TcOlbnM=?= Nilsson) Date: Sat, 22 Oct 2016 01:19:57 +0200 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <20161021231957.GF3307@besserwisser.org> Subject: Re: Dyn DDoS this AM? Date: Fri, Oct 21, 2016 at 03:21:20PM -0700 Quoting David Birdsong (david at imgix.com): > On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > > > anyone who relies on a single dns provider is just asking for stuff such > > as this. > > > > randy > > I'd love to hear how others are handling the overhead of managing two dns > providers. Every time we brainstorm on it, we see it as blackhole of eng > effort WRT to keeping them in sync and and then waiting for TTLs to cut an > entire delegation over. The fault is giving up the primary for an API connection. Sure, it is tempting. We do, however, need to push the "application-integrated" DNS vendors harder. They need to give their customers more choice in how the DNS is populated. They also very much need to let people with above-mentioned "application-integrated" needs add third party DNS providers in the mix. This diversity capability is what makes DNS resilient. Monocultures have suboptimal survivability in the long run. Adding DNS providers when you control the primary is completely painless. With EDNS0 there's lots of room for insanely large NS RRSETs. Also, do not fall in the "short TTL for service agility" trap. Besides, what Randy wrote. -- M?ns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Hold the MAYO & pass the COSMIC AWARENESS ... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: From mansaxel at besserwisser.org Fri Oct 21 23:22:55 2016 From: mansaxel at besserwisser.org (=?utf-8?B?TcOlbnM=?= Nilsson) Date: Sat, 22 Oct 2016 01:22:55 +0200 Subject: Dyn DDoS this AM? In-Reply-To: <20161021231924.GG45065@excession.tpb.net> References: <20161021231924.GG45065@excession.tpb.net> Message-ID: <20161021232255.GG3307@besserwisser.org> Subject: Re: Dyn DDoS this AM? Date: Sat, Oct 22, 2016 at 01:19:24AM +0200 Quoting Niels Bakker (niels=nanog at bakker.net): > The point of outsourcing DNS isn't just availability of static hostnames, > it's the added services delivered, like returning different answers based on > source of the question, even monitoring your infrastructure (or it reporting > load into the DNS management system). > > That is very hard to replicate with two DNS providers. Surely, it must be better to use a singular service that is provably easy to take out. The advantages are overwhelming. -- M?ns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Yow! Are we wet yet? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: From ktims at stargate.ca Fri Oct 21 23:34:28 2016 From: ktims at stargate.ca (Keenan Tims) Date: Fri, 21 Oct 2016 16:34:28 -0700 Subject: Dyn DDoS this AM? In-Reply-To: <20161021231924.GG45065@excession.tpb.net> References: <20161021231924.GG45065@excession.tpb.net> Message-ID: <821401bf-f6ac-ab3b-10c9-c2d897fe254c@stargate.ca> I don't have a horse in this race, and haven't used it in anger, but Netflix released denominator to attempt to deal with some of these issues: https://github.com/Netflix/denominator Their goal is to support the highest common denominator of features among the supported providers, Maybe that helps someone. Keenan On 2016-10-21 16:19, Niels Bakker wrote: > The point of outsourcing DNS isn't just availability of static > hostnames, it's the added services delivered, like returning different > answers based on source of the question, even monitoring your > infrastructure (or it reporting load into the DNS management system). > > That is very hard to replicate with two DNS providers. > > > -- Niels. From josh at kyneticwifi.com Fri Oct 21 23:40:58 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Fri, 21 Oct 2016 18:40:58 -0500 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: Ansible would be a decent start. On Oct 21, 2016 5:26 PM, "David Birdsong" wrote: > On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > > > anyone who relies on a single dns provider is just asking for stuff such > > as this. > > > > randy > > > > I'd love to hear how others are handling the overhead of managing two dns > providers. Every time we brainstorm on it, we see it as blackhole of eng > effort WRT to keeping them in sync and and then waiting for TTLs to cut an > entire delegation over. > From mansaxel at besserwisser.org Fri Oct 21 23:45:55 2016 From: mansaxel at besserwisser.org (=?utf-8?B?TcOlbnM=?= Nilsson) Date: Sat, 22 Oct 2016 01:45:55 +0200 Subject: Dyn DDoS this AM? In-Reply-To: <20161021233709.GH45065@excession.tpb.net> References: <20161021231957.GF3307@besserwisser.org> <20161021233709.GH45065@excession.tpb.net> Message-ID: <20161021234555.GJ3307@besserwisser.org> Subject: Re: Dyn DDoS this AM? Date: Sat, Oct 22, 2016 at 01:37:09AM +0200 Quoting Niels Bakker (niels at bakker.net): > * mansaxel at besserwisser.org (M?ns Nilsson) [Sat 22 Oct 2016, 01:27 CEST]: > >Also, do not fall in the "short TTL for service agility" trap. > > Several CDNs, Akamai among them, do use short TTLs for this exact reason. > Server load is constantly monitored and taken into account when crafting DNS > replies. But the problem is that this trashes caching, and DNS does not work without caches. At least not if you want it to survive when the going gets tough. If we're going to solve this we need to innovate beyond the pathetic CNAME chains that todays managed DNS services make us use, and get truly distributed load-balancing decision-making (which only will work if you give it sensible data; a single CNAME is not sensible data) all the way out in the client application. -- M?ns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Well, I'm INVISIBLE AGAIN ... I might as well pay a visit to the LADIES ROOM ... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: From josh at kyneticwifi.com Sat Oct 22 00:10:45 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Fri, 21 Oct 2016 19:10:45 -0500 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: Ah, disregard. I see what you're saying now. Yes, I can see how that would be problematic. On Oct 21, 2016 6:40 PM, "Josh Reynolds" wrote: > Ansible would be a decent start. > > On Oct 21, 2016 5:26 PM, "David Birdsong" wrote: > >> On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: >> >> > anyone who relies on a single dns provider is just asking for stuff such >> > as this. >> > >> > randy >> > >> >> I'd love to hear how others are handling the overhead of managing two dns >> providers. Every time we brainstorm on it, we see it as blackhole of eng >> effort WRT to keeping them in sync and and then waiting for TTLs to cut an >> entire delegation over. >> > From cjc+nanog at pumpky.net Sat Oct 22 00:11:34 2016 From: cjc+nanog at pumpky.net (Crist Clark) Date: Fri, 21 Oct 2016 17:11:34 -0700 Subject: Dyn DDoS this AM? In-Reply-To: <20161021234555.GJ3307@besserwisser.org> References: <20161021231957.GF3307@besserwisser.org> <20161021233709.GH45065@excession.tpb.net> <20161021234555.GJ3307@besserwisser.org> Message-ID: Given the scale of these attacks, whether having two providers does any good may be a crap shoot. That is, what if the target happens to share the same providers you do? Given the whole asymmetry of resources that make this a problem in the first place, the attackers probably have the resources to take out multiple providers. Having multiple providers may reduce your chance of being collateral damage (and I'd also still worry more about the more mundane risks of a single provider, maintenance or upgrade gone bad, business risks, etc., than these sensational ones), but multiple providers likely won't save you if you are the actual target of the attack. On Fri, Oct 21, 2016 at 4:45 PM, M?ns Nilsson wrote: > Subject: Re: Dyn DDoS this AM? Date: Sat, Oct 22, 2016 at 01:37:09AM +0200 > Quoting Niels Bakker (niels at bakker.net): > > * mansaxel at besserwisser.org (M?ns Nilsson) [Sat 22 Oct 2016, 01:27 > CEST]: > > >Also, do not fall in the "short TTL for service agility" trap. > > > > Several CDNs, Akamai among them, do use short TTLs for this exact reason. > > Server load is constantly monitored and taken into account when crafting > DNS > > replies. > > But the problem is that this trashes caching, and DNS does not work > without caches. At least not if you want it to survive when the going > gets tough. > > If we're going to solve this we need to innovate beyond the pathetic > CNAME chains that todays managed DNS services make us use, and get truly > distributed load-balancing decision-making (which only will work if you > give it sensible data; a single CNAME is not sensible data) all the way > out in the client application. > > -- > M?ns Nilsson primary/secondary/besserwisser/machina > MN-1334-RIPE +46 705 989668 > Well, I'm INVISIBLE AGAIN ... I might as well pay a visit to the LADIES > ROOM ... > From rfg at tristatelogic.com Sat Oct 22 00:39:47 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 21 Oct 2016 17:39:47 -0700 Subject: Death of the Internet, Film at 11 Message-ID: <84973.1477096787@segfault.tristatelogic.com> VICTOR LASZLO: If we stop fighing our enemies, the world will die. RICK BLAINE: Well, what of it? It will be out of its misery. -- From the movie "Casablanca" (1942) Sorry, but some days I just can't help thinking to myself "Oh well, as much fun as it has been, this whole lab experiment called The Internet was never really going last or stand the test of time anyway." The problem isn't the technology. It's the politics. It's fragility by design. Oh! And by the way, one news source that I was just reading a few minutes ago stated that all of the carnage at Dyn today was caused by something on the order of just 1/10th of the known CCTV bots out there. And I'm thinking, like, "Gee! I guess that we ought to count ourselves as lucky that whoever was running this thing, for whatever reason, just didn't much feel like firing up the whole entire bloody thing today. Otherwise, you know, we might have REALLY had a problem." :-) Regards, rfg P.S. To all of you Ayn Rand devotees out there who still vociferously argue that it's nobody else's business how you monitor or police your "private" networks, and who still refuse to take even minimalist steps (like BCP 38), congratulations. Via your inaction and self-centered intransigence you have today moved us all one step closer to the day when the relevant decisions will be taken out of your hands. You are succeding brilliantly at creating the exact thing that you most abhor, i.e. government control. Clemenceau said that war is too important to be left to the generals. Well, guess what? The Internet is too important to be left to the [[fill in the blank]]. It has already begun... https://krebsonsecurity.com/2016/10/europe-to-push-new-security-rules-amid-iot-mess/ This is just the first timid step. A few more days like today and more, much more, will follow. From laszlo at heliacal.net Sat Oct 22 00:52:42 2016 From: laszlo at heliacal.net (Laszlo Hanyecz) Date: Sat, 22 Oct 2016 00:52:42 +0000 Subject: Death of the Internet, Film at 11 In-Reply-To: <84973.1477096787@segfault.tristatelogic.com> References: <84973.1477096787@segfault.tristatelogic.com> Message-ID: <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> On 2016-10-22 00:39, Ronald F. Guilmette wrote: > P.S. To all of you Ayn Rand devotees out there who still vociferously > argue that it's nobody else's business how you monitor or police your > "private" networks, and who still refuse to take even minimalist steps > (like BCP 38), congratulations. What does BCP38 have to do with this? All that does is block one specific type of attack (and cause a lot of collateral damage). The IoT devices do not need to spoof addresses - they can just generate attack traffic directly. This is even better, because you can't cut those eyeball addresses off - those are the same addresses your target audience is using. If you cut off the eyeball networks there's not much point to running an internet business website anymore. -Laszlo From fergdawgster at mykolab.com Sat Oct 22 01:01:36 2016 From: fergdawgster at mykolab.com (Paul Ferguson) Date: Fri, 21 Oct 2016 18:01:36 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/21/2016 5:52 PM, Laszlo Hanyecz wrote: > > On 2016-10-22 00:39, Ronald F. Guilmette wrote: >> P.S. To all of you Ayn Rand devotees out there who still >> vociferously argue that it's nobody else's business how you >> monitor or police your "private" networks, and who still refuse >> to take even minimalist steps (like BCP 38), congratulations. > > What does BCP38 have to do with this? All that does is block one > specific type of attack (and cause a lot of collateral damage). > The IoT devices do not need to spoof addresses - they can just > generate attack traffic directly. This is even better, because you > can't cut those eyeball addresses off - those are the same > addresses your target audience is using. If you cut off the > eyeball networks there's not much point to running an internet > business website anymore. > Don't let the perfect be the enemy of the good. - - ferg (BCP38 instigator) - -- Paul Ferguson ICEBRG.io, Seattle USA -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlgKunAACgkQKJasdVTchbJJCQD+N6cosKffmfTqERBJ8q3pX+20 jY/FQvzUuKoy+iY3C4wA/2qKV01Z0e16BQ0/030euhCCmTUW0jut+Hp8xyWrVKkN =+oT7 -----END PGP SIGNATURE----- From egon at egon.cc Sat Oct 22 01:02:08 2016 From: egon at egon.cc (James Downs) Date: Fri, 21 Oct 2016 18:02:08 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <84973.1477096787@segfault.tristatelogic.com> References: <84973.1477096787@segfault.tristatelogic.com> Message-ID: <81D48463-9858-4CE4-B405-0CDEF31403FC@egon.cc> > On Oct 21, 2016, at 17:39, Ronald F. Guilmette wrote: > P.S. To all of you Ayn Rand devotees out there who still vociferously > argue that it's nobody else's business how you monitor or police your > "private" networks, and who still refuse to take even minimalist steps What does Ayn Rand have to do with it? She would hardly countenance incompetence. From rbf+nanog at panix.com Sat Oct 22 01:06:19 2016 From: rbf+nanog at panix.com (Brett Frankenberger) Date: Fri, 21 Oct 2016 20:06:19 -0500 Subject: Dyn DDoS this AM? In-Reply-To: References: <20161021231957.GF3307@besserwisser.org> <20161021233709.GH45065@excession.tpb.net> <20161021234555.GJ3307@besserwisser.org> Message-ID: <20161022010619.GA28585@panix.com> On Fri, Oct 21, 2016 at 05:11:34PM -0700, Crist Clark wrote: > > Given the scale of these attacks, whether having two providers does any > good may be a crap shoot. > > That is, what if the target happens to share the same providers you do? > Given the whole asymmetry of resources that make this a problem in the > first place, the attackers probably have the resources to take out multiple > providers. > > Having multiple providers may reduce your chance of being collateral damage > (and I'd also still worry more about the more mundane risks of a single > provider, maintenance or upgrade gone bad, business risks, etc., than these > sensational ones), but multiple providers likely won't save you if you are > the actual target of the attack. Good, perfect, enemy, etc. How many sites were down today? How many were the intended target? -- Brett From jfmezei_nanog at vaxination.ca Sat Oct 22 01:12:23 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Fri, 21 Oct 2016 21:12:23 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <580ABCF7.1030003@vaxination.ca> On 2016-10-21 18:45, david raistrick wrote: > switch too..). setting TTLs that make sense for a design that supports > change is also easy. Cuts both ways. Had Twitter had TTLs of say 7 days, vast majority wouldn't notice an outage of a few hours because their local cache wa still valid. It does prevent one from reacting quickly to emergencies. From lists at eitanadler.com Sat Oct 22 01:35:41 2016 From: lists at eitanadler.com (Eitan Adler) Date: Fri, 21 Oct 2016 18:35:41 -0700 Subject: Dyn DDoS this AM? In-Reply-To: <580ABCF7.1030003@vaxination.ca> References: <580ABCF7.1030003@vaxination.ca> Message-ID: On 21 October 2016 at 18:12, Jean-Francois Mezei wrote: > On 2016-10-21 18:45, david raistrick wrote: > >> switch too..). setting TTLs that make sense for a design that supports >> change is also easy. > > Cuts both ways. Had Twitter had TTLs of say 7 days, vast majority > wouldn't notice an outage of a few hours because their local cache wa > still valid. In practice TTLs tend to be ignored on the public internet. In past research I've been involved with browser[0] behavior was effectively random despite the TTL set. [0] more specifically, the chain of DNS resolution and caching down to the browser. -- Eitan Adler From george.herbert at gmail.com Sat Oct 22 01:43:58 2016 From: george.herbert at gmail.com (George William Herbert) Date: Fri, 21 Oct 2016 18:43:58 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: <580ABCF7.1030003@vaxination.ca> Message-ID: > On Oct 21, 2016, at 6:35 PM, Eitan Adler wrote: > > [...] > > In practice TTLs tend to be ignored on the public internet. In past > research I've been involved with browser[0] behavior was effectively > random despite the TTL set. > > [0] more specifically, the chain of DNS resolution and caching down to > the browser. Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations. If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault. Sent from my iPhone From randy at psg.com Sat Oct 22 03:08:55 2016 From: randy at psg.com (Randy Bush) Date: Sat, 22 Oct 2016 12:08:55 +0900 Subject: Death of the Internet, Film at 11 In-Reply-To: <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> Message-ID: > What does BCP38 have to do with this? nothing technical, as these iot attacks are not spoofed. think of it as a religion. From fergdawgster at mykolab.com Sat Oct 22 03:20:09 2016 From: fergdawgster at mykolab.com (Paul Ferguson) Date: Fri, 21 Oct 2016 20:20:09 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> Message-ID: <6c8d8ca3-faa4-2d9d-c970-8c539b45c718@mykolab.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/21/2016 8:08 PM, Randy Bush wrote: >> What does BCP38 have to do with this? > > nothing technical, as these iot attacks are not spoofed. > > think of it as a religion. > I'm going to save this e-mail forever! Cheers, - - ferg - -- Paul Ferguson ICEBRG.io, Seattle USA -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlgK2ukACgkQKJasdVTchbJDywD/frHeNpPnlwT1ddgh4kZyi5MJ YkH5lbx41an0WNpg3NAA/043VNnfKK5JQ7+dCsXyx8LEno8aIoIPvIvPGsWyjY50 =HMfV -----END PGP SIGNATURE----- From randy at psg.com Sat Oct 22 03:24:18 2016 From: randy at psg.com (Randy Bush) Date: Sat, 22 Oct 2016 12:24:18 +0900 Subject: Death of the Internet, Film at 11 In-Reply-To: <6c8d8ca3-faa4-2d9d-c970-8c539b45c718@mykolab.com> References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> <6c8d8ca3-faa4-2d9d-c970-8c539b45c718@mykolab.com> Message-ID: >>> What does BCP38 have to do with this? >> nothing technical, as these iot attacks are not spoofed. >> think of it as a religion. > I'm going to save this e-mail forever! no extra charge we deploy it more than most. we talk about it less than most. and every time something untoward happens on the internet, we do not tell everyone that they should deploy bcp38, iltering, origin validation, dnssec, ipv6, ... talk is cheap. From yang.yu.list at gmail.com Sat Oct 22 03:23:57 2016 From: yang.yu.list at gmail.com (Yang Yu) Date: Fri, 21 Oct 2016 22:23:57 -0500 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: On Fri, Oct 21, 2016 at 11:45 AM, Patrick W. Gilmore wrote: > My guess is you should track anything to as33517. And AS15135? From nanog at ics-il.net Sat Oct 22 03:25:42 2016 From: nanog at ics-il.net (Mike Hammett) Date: Fri, 21 Oct 2016 22:25:42 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> Message-ID: <2144358798.3535.1477106742397.JavaMail.mhammett@ThunderFuck> Block one type of attack enough times and you've accomplished something. Because script kiddies are taking advantage of published exploits doesn't mean we stop setting passwords on things. You have to protect from them all. No, no collateral damage. We discussed this a couple weeks ago and there was no credible evidence of collateral damage. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Laszlo Hanyecz" To: nanog at nanog.org Sent: Friday, October 21, 2016 7:52:42 PM Subject: Re: Death of the Internet, Film at 11 On 2016-10-22 00:39, Ronald F. Guilmette wrote: > P.S. To all of you Ayn Rand devotees out there who still vociferously > argue that it's nobody else's business how you monitor or police your > "private" networks, and who still refuse to take even minimalist steps > (like BCP 38), congratulations. What does BCP38 have to do with this? All that does is block one specific type of attack (and cause a lot of collateral damage). The IoT devices do not need to spoof addresses - they can just generate attack traffic directly. This is even better, because you can't cut those eyeball addresses off - those are the same addresses your target audience is using. If you cut off the eyeball networks there's not much point to running an internet business website anymore. -Laszlo From rekoil at semihuman.com Sat Oct 22 03:29:16 2016 From: rekoil at semihuman.com (Chris Woodfield) Date: Fri, 21 Oct 2016 22:29:16 -0500 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: As a Twitter network engineer (and the guy Patrick let camp out in your hotel room all day) - thank you for this. Whoever was behind this just poked a hornet?s nest. ?Govern yourselves accordingly?. -C (Obviously speaking for myself, not my employer?) > On Oct 21, 2016, at 10:48 AM, Patrick W. Gilmore wrote: > > I cannot give additional info other than what?s been on ?public media?. > > However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. > > To Dyn and everyone else being attacked: > The community is behind you. There are problems, but if we stick together, we can beat these miscreants. > > To the miscreants: > You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. > > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. > > But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. > > > OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. > > -- > TTFN, > patrick > >> On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: >> >> Does anyone have any additional details? Seems to be over now, but I'm very >> curious about the specifics of such a highly impactful attack (and it's >> timing following NANOG 68)... >> >> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ >> >> -- >> @ChrisGrundemann >> http://chrisgrundemann.com > From rfg at tristatelogic.com Sat Oct 22 05:53:42 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 21 Oct 2016 22:53:42 -0700 Subject: Death of the Internet, Film at 11 Message-ID: <85864.1477115622@segfault.tristatelogic.com> Laszlo Hanyecz wrote: >What does BCP38 have to do with this? Your're right. That's not specifically related to *this* attack. Nobody needs to spoof anything when you've got a zillion fire hoses just lying around where any 13 year old can command them from the TRS 80 in his mom's basement. (I've seen different estimates today. One said there's about a half million of these things, but I think I saw where Dyn itself put the number of unique IPs in the attack at something like ten million.) I just threw out BCP 38 as an example of something *very* minimal that the collective Internet, if it had any brains, would have made de rigueur for everyone ten+ years ago. BCP 38 is something that I personally view as a "no brainer", that is already widely accepted as being necessary, and yet is a critical security step that some (many?) are still resisting. So, it's like "Well, if the Internet-at-large can't even do *this* simple and relatively non-controversial thing, then we haven't got a prayer in hell of ever seeing a world-wide determined push to find and neutralize all of these bloody damn stupid CCTV things. And when the day comes when somebody figures out how to remotely pop a default config Windoze XP box... boy oh boy, will *that* be a fun day... NOT! Because we're not ready. Nobody's ready. Except maybe DoD, and I'm not even taking bets on that one." I didn't intend to focus on BCP 38. Everybody knows that's only one thing, designed to deal with just one part of the overall problem. The overall problem, in my view, is the whole mindset which says "Oh, we just connect the wires. Everything else is somebody else's problem." Ok, so this mailing list is a list of network operators. Swell. Every network operator who can do so, please raise your hand if you have *recently* scanned you own network and if you can -honestly- attest that you have taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified weeks or months ago as being fundamentally insecure can emit a single packet out onto the public Internet. And, cue the crickets... Recent events, like the Krebs DDoS and the even bigger OVH DDoS, and today's events make it perfectly clear to even the most blithering of blithering idiots that network operators, en mass, have to start scanning their own networks for insecurities. And you'd all better get on that, not next fiscal year or even next quarter, but right effing now, because the next major event is right around the corner. And remember, *you* may not be scanning your networks for easily pop'able boxes, but as we should all be crystal clear on by now, that *does not* mean that nobody else is doing so. Regards, rfg P.S. The old saying is that idle hands are the devil's playground. In the context of the various post-invasion insurgancies, etc., in Iraq, is is often mentioned that it was a somewhat less than a brilliant move for the U.S. to have disbanded the Iraq army, thereby leaving large numbers of trained young men on the streets with no jobs and nothing to do. To all of the network operators who think that (or argue that) it will be too expensive to hire professionals to come in an do the work to scan your networks for known vulnerabilities, I have a simple suggestion. Go down to your local high school, find the schmuck who teaches the kids about computers, and ask him for the name of his most clever student. Then hire that student and put him to work, scanning your network. As in Iraq, it will be *much* better to have capable young men inside the tent, pissing out, rather than the other way around. From nanogml at Mail.DDoS-Mitigator.net Sat Oct 22 06:16:17 2016 From: nanogml at Mail.DDoS-Mitigator.net (alvin nanog) Date: Fri, 21 Oct 2016 23:16:17 -0700 Subject: Dyn DDoS this AM? - dns In-Reply-To: References: Message-ID: <20161022061617.pkfaadk2sj2yfhzx@Mail.DDoS-Mitigator.net> On 10/21/16 at 03:21pm, David Birdsong wrote: > On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush wrote: > > anyone who relies on a single dns provider is just asking for stuff such > > as this. :-) > I'd love to hear how others are handling the overhead of managing two dns > providers. in my view of ( automated ) dns managment: Only on the one "master" dns server, make your DNS changes, update the serial number for example.com changes and reload the new update zone file ... notifications goes out to all known slave DNS servers .. For all the other authorized DNS servers, they should all automatically update itself ... magic all dns servers are in sync ... some folks don't like "master" DNS server vs slaves .. i donno why not .. but, you do have to configure your "master dns server" properly to only allow only authorized slaves access to their dns reccords similarly, slave DNS servers should only update from it's recognized master dns server there should be zero isues with managing 2 dns server or 100 dns servers before downloading new dns info, Man-in-the-Middle tests with OpenSSL certs should be done to confirm the other end is in fact who you think it is that you're going to be sending dns info to or receiving from c ya alvin http://DDoS-Mitigator.net From george.herbert at gmail.com Sat Oct 22 06:37:17 2016 From: george.herbert at gmail.com (George William Herbert) Date: Fri, 21 Oct 2016 23:37:17 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: Oh god, you invoked @popehat ... [dyndds and its customers sue XiongMai, the OEM integrators, and Does 1-10,000,000 who own the devices for neglegence?...] Sent from my iPhone > On Oct 21, 2016, at 8:29 PM, Chris Woodfield wrote: > > As a Twitter network engineer (and the guy Patrick let camp out in your hotel room all day) - thank you for this. Whoever was behind this just poked a hornet?s nest. > > ?Govern yourselves accordingly?. > > -C > > (Obviously speaking for myself, not my employer?) > >> On Oct 21, 2016, at 10:48 AM, Patrick W. Gilmore wrote: >> >> I cannot give additional info other than what?s been on ?public media?. >> >> However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can Not Stand. See Krebs? on the Democratization of Censorship. See lots of other things. >> >> To Dyn and everyone else being attacked: >> The community is behind you. There are problems, but if we stick together, we can beat these miscreants. >> >> To the miscreants: >> You will not succeed. Search "churchill on the beaches?. It?s a bit melodramatic, but it?s how I feel at this moment. >> >> To the rest of the community: >> If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. >> >> But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone?s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. >> >> >> OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. >> >> -- >> TTFN, >> patrick >> >>> On Oct 21, 2016, at 10:55 AM, Chris Grundemann wrote: >>> >>> Does anyone have any additional details? Seems to be over now, but I'm very >>> curious about the specifics of such a highly impactful attack (and it's >>> timing following NANOG 68)... >>> >>> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ >>> >>> -- >>> @ChrisGrundemann >>> http://chrisgrundemann.com > From rirving at antient.org Sat Oct 22 11:22:15 2016 From: rirving at antient.org (Richard Irving) Date: Sat, 22 Oct 2016 07:22:15 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <81D48463-9858-4CE4-B405-0CDEF31403FC@egon.cc> References: <84973.1477096787@segfault.tristatelogic.com> <81D48463-9858-4CE4-B405-0CDEF31403FC@egon.cc> Message-ID: <580B4BE7.80407@antient.org> Then, again, Ayn Rands idea of "sex" was to get slapped around first.. I am not sure I would acquire my "life philosophy" from her.... and, as *proudly* *independent* as she was, in the end, she relied upon American Social Security to get by.... talk is cheap. On 10/21/2016 09:02 PM, James Downs wrote: >> On Oct 21, 2016, at 17:39, Ronald F. Guilmette wrote: >> P.S. To all of you Ayn Rand devotees out there who still vociferously >> argue that it's nobody else's business how you monitor or police your >> "private" networks, and who still refuse to take even minimalist steps > What does Ayn Rand have to do with it? She would hardly countenance incompetence. > From outsider at scarynet.org Sat Oct 22 12:02:01 2016 From: outsider at scarynet.org (Alexander Maassen) Date: Sat, 22 Oct 2016 14:02:01 +0200 Subject: Dyn DDoS this AM? Message-ID: Remember ping packets containing +++ATH0 ? Kind regards, Alexander Maassen - Technical Maintenance Engineer Parkstad Support BV- Maintainer DroneBL- Peplink Certified Engineer -------- Oorspronkelijk bericht --------Van: Alain Hebert Datum: 21-10-16 23:37 (GMT+01:00) Aan: nanog at nanog.org Onderwerp: Re: Dyn DDoS this AM? ??? Just a FYI, ??? That "horrific trend" has been happening since some techie got dissed on an IRC channel over 20 years ago. ??? He used a bunch of hosted putters to ICMP flood the IRC server. ??? Whatever the community is behind, until the carriers decide to wise up this will keep happening, that is without talking about the industries being developed around DDoSes events. ??? Enjoy your weekend. ( I ain't on call anymore anyway =D ) ----- Alain Hebert??????????????????????????????? ahebert at pubnix.net?? PubNIX Inc.??????? 50 boul. St-Charles P.O. Box 26770???? Beaconsfield, Quebec???? H9W 6G7 Tel: 514-990-5911? http://www.pubnix.net??? Fax: 514-990-9443 On 10/21/16 11:52, Brian Davies via NANOG wrote: > +1! > > Well said, Patrick. > > B > > On Friday, October 21, 2016, Patrick W. Gilmore wrote: > >> I cannot give additional info other than what?s been on ?public media?. >> >> However, I would very much like to say that this is a horrific trend on >> the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can >> Not Stand. See Krebs? on the Democratization of Censorship. See lots of >> other things. >> >> To Dyn and everyone else being attacked: >> The community is behind you. There are problems, but if we stick together, >> we can beat these miscreants. >> >> To the miscreants: >> You will not succeed. Search "churchill on the beaches?. It?s a bit >> melodramatic, but it?s how I feel at this moment. >> >> To the rest of the community: >> If you can help, please do. I know a lot of you are thinking ?what can I >> do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, >> that doesn?t help Mirai, but it still helps. There are many other things >> you can do as well. >> >> But a lot of it is just willingness to help. When someone asks you to help >> trace an attack, do not let the request sit for a while. Damage is being >> done. Help your neighbor. When someone?s house is burning, your current >> project, your lunch break, whatever else you are doing is almost certainly >> less important. If we stick together and help each other, we can - we WILL >> - win this war. If we are apathetic, we have already lost. >> >> >> OK, enough motivational speaking for today. But take this to heart. Our >> biggest problem is people thinking they cannot or do not want to help. >> >> -- >> TTFN, >> patrick >> >>> On Oct 21, 2016, at 10:55 AM, Chris Grundemann > > wrote: >>> Does anyone have any additional details? Seems to be over now, but I'm >> very >>> curious about the specifics of such a highly impactful attack (and it's >>> timing following NANOG 68)... >>> >>> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- >> twitter-spotify-reddit/ >>> -- >>> @ChrisGrundemann >>> http://chrisgrundemann.com >> From nanog at ics-il.net Sat Oct 22 12:34:55 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 07:34:55 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <85864.1477115622@segfault.tristatelogic.com> References: <85864.1477115622@segfault.tristatelogic.com> Message-ID: <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" Serious question... how? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Ronald F. Guilmette" To: nanog at nanog.org Sent: Saturday, October 22, 2016 12:53:42 AM Subject: Re: Death of the Internet, Film at 11 Laszlo Hanyecz wrote: >What does BCP38 have to do with this? Your're right. That's not specifically related to *this* attack. Nobody needs to spoof anything when you've got a zillion fire hoses just lying around where any 13 year old can command them from the TRS 80 in his mom's basement. (I've seen different estimates today. One said there's about a half million of these things, but I think I saw where Dyn itself put the number of unique IPs in the attack at something like ten million.) I just threw out BCP 38 as an example of something *very* minimal that the collective Internet, if it had any brains, would have made de rigueur for everyone ten+ years ago. BCP 38 is something that I personally view as a "no brainer", that is already widely accepted as being necessary, and yet is a critical security step that some (many?) are still resisting. So, it's like "Well, if the Internet-at-large can't even do *this* simple and relatively non-controversial thing, then we haven't got a prayer in hell of ever seeing a world-wide determined push to find and neutralize all of these bloody damn stupid CCTV things. And when the day comes when somebody figures out how to remotely pop a default config Windoze XP box... boy oh boy, will *that* be a fun day... NOT! Because we're not ready. Nobody's ready. Except maybe DoD, and I'm not even taking bets on that one." I didn't intend to focus on BCP 38. Everybody knows that's only one thing, designed to deal with just one part of the overall problem. The overall problem, in my view, is the whole mindset which says "Oh, we just connect the wires. Everything else is somebody else's problem." Ok, so this mailing list is a list of network operators. Swell. Every network operator who can do so, please raise your hand if you have *recently* scanned you own network and if you can -honestly- attest that you have taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified weeks or months ago as being fundamentally insecure can emit a single packet out onto the public Internet. And, cue the crickets... Recent events, like the Krebs DDoS and the even bigger OVH DDoS, and today's events make it perfectly clear to even the most blithering of blithering idiots that network operators, en mass, have to start scanning their own networks for insecurities. And you'd all better get on that, not next fiscal year or even next quarter, but right effing now, because the next major event is right around the corner. And remember, *you* may not be scanning your networks for easily pop'able boxes, but as we should all be crystal clear on by now, that *does not* mean that nobody else is doing so. Regards, rfg P.S. The old saying is that idle hands are the devil's playground. In the context of the various post-invasion insurgancies, etc., in Iraq, is is often mentioned that it was a somewhat less than a brilliant move for the U.S. to have disbanded the Iraq army, thereby leaving large numbers of trained young men on the streets with no jobs and nothing to do. To all of the network operators who think that (or argue that) it will be too expensive to hire professionals to come in an do the work to scan your networks for known vulnerabilities, I have a simple suggestion. Go down to your local high school, find the schmuck who teaches the kids about computers, and ask him for the name of his most clever student. Then hire that student and put him to work, scanning your network. As in Iraq, it will be *much* better to have capable young men inside the tent, pissing out, rather than the other way around. From bicknell at ufp.org Sat Oct 22 12:53:35 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Sat, 22 Oct 2016 05:53:35 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: <20161022125335.GA84013@ussenterprise.ufp.org> In a message written on Sat, Oct 22, 2016 at 07:34:55AM -0500, Mike Hammett wrote: > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" From https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/#more-36754 The part that should outrage everyone on this list: That's because while many of these devices allow users to change the default usernames and passwords on a Web-based administration panel that ships with the products, those machines can still be reached via more obscure, less user-friendly communications services called "Telnet" and "SSH." "The issue with these particular devices is that a user cannot feasibly change this password," Flashpoints Zach Wikholm told KrebsOnSecurity. "The password is hardcoded into the firmware, and the tools necessary to disable it are not present. Even worse, the web interface is not aware that these credentials even exist." As much as I hate to say it, what is needed is regulation. It could be some form of self regulation, with retailers refusing to sell products that aren't "certified" by some group. It could be full blown government regulation. Perhaps a mix. It's not a problem for a network operator to "solve", any more than someone who builds roads can make an unsafe car safe. Yes, both the network operator and rood operator play a role in building safe infrastructure (BCP38, deformable barriers), but neither can do anything for a manufacturer who builds a device that is wholely deficient in the first place. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From list at satchell.net Sat Oct 22 13:41:08 2016 From: list at satchell.net (Stephen Satchell) Date: Sat, 22 Oct 2016 06:41:08 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: <7839cd8a-ea88-f46a-0e50-0062fad6a19a@satchell.net> On 10/22/2016 05:34 AM, Mike Hammett wrote: > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > > Serious question... how? > Network operators can only do so much. By the time traffic enters into an ISP's traffic aggregation point, any flow monitoring and throttling would have a minimal effect. Not saying that it shouldn't be considered. The correct answer includes throttling the traffic much closer to the source. The obvious answer is that the device that bridges IoT to the upstream link in the home or office have the capability of rate-limiting upstream traffic. Perhaps on a per-MAC basis. When does a thermostat, light bulb, or refrigerator need 1-megabyte/s uplink channels? For that matter, how many computers -- especially laptops -- need that kind of upstream capacity? (Yes, yes, YouTube publishers and VLAN links to the office, to name two, will need that kind of channel; see below. Gamers need small, low-latency channels, so the throttling can't be too aggressive. Public-access storage, web and mail servers, obviously. IP-connected Web cameras need some upstream capacity, but not a full-bore one. The uplink throttle can take into consideration "reasonable" upstream rates for cameras.) For wireless access points, the place to start would be with the OpenWRT package, to serve as a model for what *can* be done. Once we have a proof of concept, it would raise the bar for "commercial" implementations. THAT would then provide an opportunity for the three-letter Federal agencies to specify reasonable regulations, should Congress so decide this is necessary. It's much easier for regulatory bodies to say "this software does it, why can't yours?" instead of saying "you [manufacturer] go figure it out". The ripple effect throughout the world would go a long way to curbing the problem. Especially if other regulatory administrations follow suit, so that the enabling crap routers are weeded out. What about the exceptions? For those rare cases where one needs a high-rate upstream channel for a node on the wireless network (or wired network, for that matter), the firmware in the traffic aggregating device can allow for specific exceptions to the rate-limit rules. One method is to tie exceptions to the device MAC address, or range of MAC addresses. Another is to tie exceptions to ports, with WiFi being a single "port" in this context. Generators of high-speed upstream traffic would, for example, need a wired connection in order to do this. This would *not* affect most WiFi-connected peripherals, like printers, because the AP would limit upstream traffic, not downstream. The ISP would then have something to sell to the customer, to replace the local POS router/WAP that the customer is currently using. Hmmm...something to thing about as I build the Linux IPTABLES Firewall Rule Generator Mk III... From brandon at rd.bbc.co.uk Sat Oct 22 14:22:55 2016 From: brandon at rd.bbc.co.uk (Brandon Butterworth) Date: Sat, 22 Oct 2016 15:22:55 +0100 (BST) Subject: Death of the Internet, Film at 11 Message-ID: <201610221422.PAA11830@sunf10.rd.bbc.co.uk> > From: Mike Hammett > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > > Serious question... how? Well their addresses are now known so one way would be for each ISP to drop traffic from them. If people don't fix them why should these devices stay on the net? If say Comcast has a million of them it might be tricky to scale but not impossible It'd take a bit of effort and care to aggregate and disseminate the data to each responsible AS, there'd be risk of bad guys getting the data and false positives/people spoofing to attack others. They'd also be building a tool that some might try to hijack for other purposes. None of that is an excuse to do nothing as is usually the result with any suggested measure that involves doing work to fix a problem I know ISPs generaly don't want the support calls but they'll end up with them and a legislative burden with commerial liability if they don't sort it out themselves. brandon From nanog at ics-il.net Sat Oct 22 14:27:40 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 09:27:40 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <201610221422.PAA11830@sunf10.rd.bbc.co.uk> References: <201610221422.PAA11830@sunf10.rd.bbc.co.uk> Message-ID: <1777219032.4471.1477146458247.JavaMail.mhammett@ThunderFuck> "their" Whose addresses are known and who are they known to? I certainly don't know the addresses of anyone involved. Some work can produce Dyn allocations, I suppose. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Brandon Butterworth" To: nanog at ics-il.net Cc: nanog at nanog.org Sent: Saturday, October 22, 2016 9:22:55 AM Subject: Re: Death of the Internet, Film at 11 > From: Mike Hammett > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > > Serious question... how? Well their addresses are now known so one way would be for each ISP to drop traffic from them. If people don't fix them why should these devices stay on the net? If say Comcast has a million of them it might be tricky to scale but not impossible It'd take a bit of effort and care to aggregate and disseminate the data to each responsible AS, there'd be risk of bad guys getting the data and false positives/people spoofing to attack others. They'd also be building a tool that some might try to hijack for other purposes. None of that is an excuse to do nothing as is usually the result with any suggested measure that involves doing work to fix a problem I know ISPs generaly don't want the support calls but they'll end up with them and a legislative burden with commerial liability if they don't sort it out themselves. brandon From brandon at rd.bbc.co.uk Sat Oct 22 14:41:52 2016 From: brandon at rd.bbc.co.uk (Brandon Butterworth) Date: Sat, 22 Oct 2016 15:41:52 +0100 (BST) Subject: Death of the Internet, Film at 11 Message-ID: <201610221441.PAA13517@sunf10.rd.bbc.co.uk> > "their" Whose addresses are known The "CCVT thingies" you refer to. Unlike spoof attacks these are easy to trace > and who are they known to? Those who were attacked by them or worked on mitigation of the attack. If not this time then they should next time as there will be a next time. > Some work can produce Dyn allocations, I suppose. Indeed, that is what I was saying brandon From rsk at gsp.org Sat Oct 22 14:46:12 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Sat, 22 Oct 2016 10:46:12 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <201610221422.PAA11830@sunf10.rd.bbc.co.uk> References: <201610221422.PAA11830@sunf10.rd.bbc.co.uk> Message-ID: <20161022144612.GA25301@gsp.org> On Sat, Oct 22, 2016 at 03:22:55PM +0100, Brandon Butterworth wrote: > Well their addresses are now known so one way would be for each ISP to > drop traffic from them. If people don't fix them why should these > devices stay on the net? Bingo. The manufacturer of these decided to build them as cheaply as possible in order to maximize profit. They neglected even rudimentary security and maintenance/update measures. Because they could. Because they chose to. They thus shifted the burden, and thus the cost, of running them in a secure fashion onto us. Yesterday everyone paid that cost. It's time to shift the cost back. Drop all their traffic and when the support calls come, tell them that they bought a known-defective device which is an operational hazard to the network, and refer them to the manufacturer for replacement/repair/refund. Note: every other vendor out there who might be tempted to cut corners is no doubt watching this and trying to gauge whether they can do the same. ---rsk From nanog at ics-il.net Sat Oct 22 14:50:17 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 09:50:17 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <201610221441.PAA13517@sunf10.rd.bbc.co.uk> References: <201610221441.PAA13517@sunf10.rd.bbc.co.uk> Message-ID: <576980112.4541.1477147813554.JavaMail.mhammett@ThunderFuck> If they are easy to trace, then it should be easy for you to tell me how to find them on my network. The addresses being known to them doesn't help me at all clean up my network or help other networks clean up theirs. It would be rather difficult for me (and I'm sure many other operators) to distinguish normal Dyn traffic from DDoS Dyn traffic. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Brandon Butterworth" To: nanog at ics-il.net Cc: nanog at nanog.org Sent: Saturday, October 22, 2016 9:41:52 AM Subject: Re: Death of the Internet, Film at 11 > "their" Whose addresses are known The "CCVT thingies" you refer to. Unlike spoof attacks these are easy to trace > and who are they known to? Those who were attacked by them or worked on mitigation of the attack. If not this time then they should next time as there will be a next time. > Some work can produce Dyn allocations, I suppose. Indeed, that is what I was saying brandon From swmike at swm.pp.se Sat Oct 22 14:54:47 2016 From: swmike at swm.pp.se (Mikael Abrahamsson) Date: Sat, 22 Oct 2016 16:54:47 +0200 (CEST) Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: On Sat, 22 Oct 2016, Alexander Maassen wrote: > Remember ping packets containing +++ATH0 ? THat only worked because of patents: https://en.wikipedia.org/wiki/Time_Independent_Escape_Sequence Inband signaling is bad, mmmkay? -- Mikael Abrahamsson email: swmike at swm.pp.se From brandon at rd.bbc.co.uk Sat Oct 22 15:02:42 2016 From: brandon at rd.bbc.co.uk (Brandon Butterworth) Date: Sat, 22 Oct 2016 16:02:42 +0100 (BST) Subject: Death of the Internet, Film at 11 Message-ID: <201610221502.QAA15723@sunf10.rd.bbc.co.uk> > From nanog-bounces at nanog.org Sat Oct 22 15:51:34 2016 > If they are easy to trace, then it should be easy for you to > tell me how to find them on my network. Not sure if you're trolling now, apologies if what I wrote wasn't clear. If you did want to find them before they attack then you could scan for them, the miscreants already did and easily found them. For some attack vectors there are services that are doing it for you, see the excellent https://www.shadowserver.org/wiki/pmwiki.php/Involve/GetReportsOnYourNetwork > The addresses being known to them doesn't help me at all clean > up my network or help other networks clean up theirs. Did you read my whole mail? The suggestion is people who get attacked tell the ISPs of the devices doing the attacking > It would be rather difficult for me (and I'm sure many other operators) > to distinguish normal Dyn traffic from DDoS Dyn traffic. I was not suggesting you try and guess, I was suggesting you be given data from actual attacks. brandon From nanog at ics-il.net Sat Oct 22 15:08:48 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 10:08:48 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <201610221502.QAA15723@sunf10.rd.bbc.co.uk> References: <201610221502.QAA15723@sunf10.rd.bbc.co.uk> Message-ID: <241612512.4558.1477148926944.JavaMail.mhammett@ThunderFuck> Not trolling in the least. I'm genuinely trying my best to help the greater community. Agreed on ShadowServer. I get their reports and I recommend others do the same. Oh, okay, I responded to someone that said: ===== Every network operator who can do so, please raise your hand if you have *recently* scanned you own network and if you can -honestly- attest that you have taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified weeks or months ago as being fundamentally insecure can emit a single packet out onto the public Internet. ===== That's the direction I was heading. How can I as a network operator seek out and eliminate the sources of these attacks? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Brandon Butterworth" To: nanog at ics-il.net Cc: nanog at nanog.org Sent: Saturday, October 22, 2016 10:02:42 AM Subject: Re: Death of the Internet, Film at 11 > From nanog-bounces at nanog.org Sat Oct 22 15:51:34 2016 > If they are easy to trace, then it should be easy for you to > tell me how to find them on my network. Not sure if you're trolling now, apologies if what I wrote wasn't clear. If you did want to find them before they attack then you could scan for them, the miscreants already did and easily found them. For some attack vectors there are services that are doing it for you, see the excellent https://www.shadowserver.org/wiki/pmwiki.php/Involve/GetReportsOnYourNetwork > The addresses being known to them doesn't help me at all clean > up my network or help other networks clean up theirs. Did you read my whole mail? The suggestion is people who get attacked tell the ISPs of the devices doing the attacking > It would be rather difficult for me (and I'm sure many other operators) > to distinguish normal Dyn traffic from DDoS Dyn traffic. I was not suggesting you try and guess, I was suggesting you be given data from actual attacks. brandon From fw at deneb.enyo.de Sat Oct 22 15:10:12 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 22 Oct 2016 17:10:12 +0200 Subject: Dyn DDoS this AM? References: Message-ID: <87a8dwxvvv.fsf@mid.deneb.enyo.de> * Randy Bush: > anyone who relies on a single dns provider is just asking for stuff such > as this. Blaming the victim isn't helpful. And without end-user-visible changes, most of the victims would still depend on Verisign as a single provider for a critical part of their DNS service. From math at sizone.org Sat Oct 22 15:23:43 2016 From: math at sizone.org (Ken Chase) Date: Sat, 22 Oct 2016 11:23:43 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: Message-ID: <20161022152343.GC9489@sizone.org> (Inband signalling - bad except for BGP?) General comment: why are we blaming the client devices for the lack of security? This is like Microsoft villifying linux in the late 90s because "there's no restrictions on use or packet crafting on the client side" - of course there isn't, in Windows either -- cant trust the client side, ever. Check out online gaming, so many h4x 'n bots. Let's stop trying to fix the clients, there'll always be bad actors/crappy coding. Let's fix the networks. Pay-to-play? People are sensitive in the pocketbooks. NetCoin or something to purchase dataflows? I dont know. Also sounds terrible. ("That's an internet tax!!!111"). But Something Must Be Done[tm], by us, soon, or we'll be dealing with govt cures which will be worse than the disease. Regulating devices will never happen. Have you checked out world trade regulations? The US can't get Chinese firms to stop shipping deadly-to-the-touch chemwep/drug carfentanil, how we gonna enforce security standards on COTS electronics? (More govt soln's/approvals too. Fear.) We have control of the networks. Lets do something. (cant find the carfentanil story on nytimes anymore, pulled? http://www.nytimes.com/aponline/2016/10/07/world/asia/ap-as-china-chemical-weapons.html ) /kc On Sat, Oct 22, 2016 at 04:54:47PM +0200, Mikael Abrahamsson said: >On Sat, 22 Oct 2016, Alexander Maassen wrote: > >>Remember ping packets containing +++ATH0 ? > >THat only worked because of patents: > >https://en.wikipedia.org/wiki/Time_Independent_Escape_Sequence > >Inband signaling is bad, mmmkay? > >-- >Mikael Abrahamsson email: swmike at swm.pp.se -- Ken Chase - Guelph Canada From marcel.duregards at yahoo.fr Sat Oct 22 15:40:22 2016 From: marcel.duregards at yahoo.fr (marcel.duregards at yahoo.fr) Date: Sat, 22 Oct 2016 17:40:22 +0200 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <580B8866.2060200@yahoo.fr> Patrick, We are client of 3 tier1. On our netflow collector, we can observe that RFC1918 sources ip traffic is entering our AS via 2 of those tier-1. Yes, 2 bigs tier-1 allow private ip traffic coming from their networks, clients, peerings to reach others customers, via Internet link, on public ip.....Of course this traffic is dropped on our BGP borders as we are filtering. But it's still filling the pipe, and this is still INVALID/UNNAUTHORIZED traffic. We wrote to them to verify if customers are technically allowed to send RFC1918 traffic over their backbone, and if we are also allowed to do so. And the answer was really evasive like :"contractually you're are not allowed". So now tell me WTF BCP38 will provide you when tier1 does not care at all and does not maintain basic filtering to/from their customers. And then they try to sell you their anti ddos services, because you know DDOS it sucks. Big joke. What about BCP38+84 on 30 tier-1 instead of asking/hoping 55k others autonomous-system having good filters in place ? -- Marcel On 21.10.2016 17:48, Patrick W. Gilmore wrote: > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn?t help Mirai, but it still helps. There are many other things you can do as well. From cboyd at gizmopartners.com Sat Oct 22 16:42:05 2016 From: cboyd at gizmopartners.com (Chris Boyd) Date: Sat, 22 Oct 2016 11:42:05 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: > On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: > > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > > Serious question... how? Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user. ?Chris From deleskie at gmail.com Sat Oct 22 17:04:19 2016 From: deleskie at gmail.com (jim deleskie) Date: Sat, 22 Oct 2016 14:04:19 -0300 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site. -jim On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd wrote: > > > On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: > > > > "taken all necessary steps to insure that none of the numerous specific > types of CCVT thingies that Krebs and others identified" > > > > Serious question... how? > > Putting them behind a firewall without general Internet access seems to > work for us. We have a lot of cheap IP cameras in our facility and none of > them can reach the net. But this is probably a bit beyond the capabilities > of the general home user. > > ?Chris > > From nanog at ics-il.net Sat Oct 22 17:05:34 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 12:05:34 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> It's also generally counter to them being available outside of that network. (web and proprietary interfaces needed, SSH and telnet not). That's also not much I can do as a network operator. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Chris Boyd" To: "Elizabeth Zwicky via NANOG" Sent: Saturday, October 22, 2016 11:42:05 AM Subject: Re: Death of the Internet, Film at 11 > On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: > > "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > > Serious question... how? Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user. ?Chris From deleskie at gmail.com Sat Oct 22 17:21:45 2016 From: deleskie at gmail.com (jim deleskie) Date: Sat, 22 Oct 2016 14:21:45 -0300 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: Sure, but now we put it outside the skill level of 99.99% of the people that don't read and understand this list. -jim On Sat, Oct 22, 2016 at 2:09 PM, Luke Guillory wrote: > VPNs can accomplish this without opening ports directly to devices. > > Luke > > > *Sent from my iPhone* > > On Oct 22, 2016, at 12:06 PM, jim deleskie wrote: > > It is also likely the desired use case. In my office I like to be able to > login when needed when on the road, when the alarm company calls me at 2am > for a false alarm so I don't have to get someone else out of bed to have > them dispatched to check on the site. > > -jim > > On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd > wrote: > > > On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: > > > "taken all necessary steps to insure that none of the numerous specific > > types of CCVT thingies that Krebs and others identified" > > > Serious question... how? > > > Putting them behind a firewall without general Internet access seems to > > work for us. We have a lot of cheap IP cameras in our facility and none of > > them can reach the net. But this is probably a bit beyond the capabilities > > of the general home user. > > > ?Chris > > > > > > Luke Guillory > Network Operations Manager > > > > Tel: 985.536.1212 > Fax: 985.536.0300 > Email: lguillory at reservetele.com > Web: www.rtconline.com > Reserve Telecommunications > 100 RTC Dr > Reserve, LA 70084 > > > > > > > *Disclaimer:* > The information transmitted, including attachments, is intended only for > the person(s) or entity to which it is addressed and may contain > confidential and/or privileged material which should not disseminate, > distribute or be copied. Please notify Luke Guillory immediately by > e-mail if you have received this e-mail by mistake and delete this e-mail > from your system. E-mail transmission cannot be guaranteed to be secure or > error-free as information could be intercepted, corrupted, lost, destroyed, > arrive late or incomplete, or contain viruses. Luke Guillory therefore > does not accept liability for any errors or omissions in the contents of > this message, which arise as a result of e-mail transmission. > > From drc at virtualized.org Sat Oct 22 18:21:49 2016 From: drc at virtualized.org (David Conrad) Date: Sat, 22 Oct 2016 11:21:49 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <241612512.4558.1477148926944.JavaMail.mhammett@ThunderFuck> References: <201610221502.QAA15723@sunf10.rd.bbc.co.uk> <241612512.4558.1477148926944.JavaMail.mhammett@ThunderFuck> Message-ID: Mike, On October 22, 2016 at 8:09:34 AM, Mike Hammett (nanog at ics-il.net) wrote: How can I as a network operator seek out and eliminate the sources of these attacks?? Maybe (not sure) one way would be to examine your resolver query logs to look for queries for names that fit domain generation algorithm patterns, then tracking down the customers/devices that are issuing those queries and politely suggest they remove the malware on their systems?? Regards, -drc -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Message signed with OpenPGP using AMPGpg URL: From mark.tinka at seacom.mu Sat Oct 22 19:29:22 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Sat, 22 Oct 2016 21:29:22 +0200 Subject: MPLS in the campus Network? In-Reply-To: References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> Message-ID: <589792b3-f0d8-e4dd-bab2-733013dde2a2@seacom.mu> On 21/Oct/16 19:02, Javier Solis wrote: > With that said, what are the best options to be able to cost > effectively scale without using vlans and maintaining a routed core? > What technology would someone suggest (mpls, vxlan,etc) to be the best > possible solution? > IME, MPLS is a good use-case here. If you are going to use the same /24 (or whatever prefix applies to you) across multiple locations, you will need some kind of overlay. Be it IP-in-IP, GRE, MPLS (l2vpn's or l3vpn's) or plain old Ethernet, you will need something. MPLS makes a lot of sense to me. It's native in hardware, upper-layer agnostic, mature, and reasonably affordable even at low scale. While I would not say MPLS is simple from a "complexity in your network" standpoint, it does provide a notable amount of simplicity when you're looking for an overlay that is transparent to all manner of Layer 2 and Layer 3 applications that a packet-based network needs to transport. Mark. From jfmezei_nanog at vaxination.ca Sat Oct 22 20:47:34 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sat, 22 Oct 2016 16:47:34 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> Message-ID: <580BD066.8050407@vaxination.ca> Generic question: The media seems to have concluded it was an "internet of things" that caused this DDoS. I have not seen any evidence of this. Has this been published by an authoritative source or is it just assumed? Has the type of device involved been identified? I am curious on how some hacker in basement with his TRS80 or Commodore Pet would be able to reach "bilions" of these devices to reprogram them. Vast majority of homes are behind NAT, which means that an incoming packet has very little chance of reaching the IoT gizmo. I amn guessing/hoping such devices have been identified and some homweoners contacted ans asked to volunteer their device for forensic analysis of where the attack came from ? Is it more plausible that those devices were "hacked" in the OEM firmware and sold with the "virus" built-in ? That would explain the widespread attack. Also, in cases such as this one, while the target has managed to mitigate the attack, how long would such an attack typically continue and require blocking ? Since the attack seemed focused on eastern USA DNS servers, would it be fair to assume that the attacks came mostly from the same region (aka: devices installed in eastern USA) ? (since anycast would point them to that). OPr did the attack use actual IP addresses instead of the unicast ones to specifically target servers ? BTW, normally, if you change the "web" password on a "device", it would also change telnet/SSH/ftp passwords. From mel at beckman.org Sat Oct 22 21:21:53 2016 From: mel at beckman.org (Mel Beckman) Date: Sat, 22 Oct 2016 21:21:53 +0000 Subject: Death of the Internet, Film at 11 In-Reply-To: <580BD066.8050407@vaxination.ca> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck>, <580BD066.8050407@vaxination.ca> Message-ID: > Vast majority of homes are behind NAT, which means that an incoming > packet has very little chance of reaching the IoT gizmo. UPNP exposes many IoT devices to the Internet, plus they're always exposed on the LAN, where many viruses find them and use backdoors to conscript them. Several bad actors are currently selling access to their IoT minions for ddos purposes. This is not new. What's new is that minion control seems to have been aggregated into a small number of malicious twerps. -mel beckman > On Oct 22, 2016, at 1:48 PM, Jean-Francois Mezei wrote: > > Generic question: > > The media seems to have concluded it was an "internet of things" that > caused this DDoS. > > I have not seen any evidence of this. Has this been published by an > authoritative source or is it just assumed? > > Has the type of device involved been identified? > > I am curious on how some hacker in basement with his TRS80 or Commodore > Pet would be able to reach "bilions" of these devices to reprogram them. > Vast majority of homes are behind NAT, which means that an incoming > packet has very little chance of reaching the IoT gizmo. > > I amn guessing/hoping such devices have been identified and some > homweoners contacted ans asked to volunteer their device for forensic > analysis of where the attack came from ? > > Is it more plausible that those devices were "hacked" in the OEM > firmware and sold with the "virus" built-in ? That would explain the > widespread attack. > > Also, in cases such as this one, while the target has managed to > mitigate the attack, how long would such an attack typically continue > and require blocking ? > > Since the attack seemed focused on eastern USA DNS servers, would it be > fair to assume that the attacks came mostly from the same region (aka: > devices installed in eastern USA) ? (since anycast would point them to > that). > > OPr did the attack use actual IP addresses instead of the unicast ones > to specifically target servers ? > > > > BTW, normally, if you change the "web" password on a "device", it would > also change telnet/SSH/ftp passwords. From petebaldridge at gmail.com Sat Oct 22 21:45:13 2016 From: petebaldridge at gmail.com (Peter Baldridge) Date: Sat, 22 Oct 2016 14:45:13 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <580BD066.8050407@vaxination.ca> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> Message-ID: On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei wrote: > Generic question: > > The media seems to have concluded it was an "internet of things" that > caused this DDoS. > > I have not seen any evidence of this. Has this been published by an > authoritative source or is it just assumed? Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible looks like unless they release a packet but this is probably consensus. > Has the type of device involved been identified? routers and cameras with shitty firmware [3] > Is it more plausible that those devices were "hacked" in the OEM > firmware and sold with the "virus" built-in ? That would explain the > widespread attack. The source code has been released. krebs [4], code [5] > Also, in cases such as this one, while the target has managed to > mitigate the attack, how long would such an attack typically continue > and require blocking ? This is an actual question that hasn't been answered. > Since the attack seemed focused on eastern USA DNS servers, would it be > fair to assume that the attacks came mostly from the same region (aka: > devices installed in eastern USA) ? (since anycast would point them to > that). Aren't heat maps just population graphs? > BTW, normally, if you change the "web" password on a "device", it would > also change telnet/SSH/ftp passwords. Seems like no one is doing either. [0] https://www.flashpoint-intel.com/mirai-botnet-linked-dyn-dns-ddos-attacks/ [1] https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/ [2] http://arstechnica.com/security/2016/10/double-dip-internet-of-things-botnet-attack-felt-across-the-internet/ [3] https://blog.sucuri.net/2016/09/iot-home-router-botnet-leveraged-in-large-ddos-attack.html [4] https://krebsonsecurity.com/2016/10/source-code-for-iot-botnet-mirai-released/ [5] https://github.com/jgamblin/Mirai-Source-Code -- Pete Baldridge 206.992.2852 From nanog at ics-il.net Sat Oct 22 21:48:01 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 16:48:01 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> Message-ID: <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> Until Dyn says or someone says Dyn said, everything is assumed. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Peter Baldridge" To: "Jean-Francois Mezei" Cc: nanog at nanog.org Sent: Saturday, October 22, 2016 4:45:13 PM Subject: Re: Death of the Internet, Film at 11 On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei wrote: > Generic question: > > The media seems to have concluded it was an "internet of things" that > caused this DDoS. > > I have not seen any evidence of this. Has this been published by an > authoritative source or is it just assumed? Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible looks like unless they release a packet but this is probably consensus. > Has the type of device involved been identified? routers and cameras with shitty firmware [3] > Is it more plausible that those devices were "hacked" in the OEM > firmware and sold with the "virus" built-in ? That would explain the > widespread attack. The source code has been released. krebs [4], code [5] > Also, in cases such as this one, while the target has managed to > mitigate the attack, how long would such an attack typically continue > and require blocking ? This is an actual question that hasn't been answered. > Since the attack seemed focused on eastern USA DNS servers, would it be > fair to assume that the attacks came mostly from the same region (aka: > devices installed in eastern USA) ? (since anycast would point them to > that). Aren't heat maps just population graphs? > BTW, normally, if you change the "web" password on a "device", it would > also change telnet/SSH/ftp passwords. Seems like no one is doing either. [0] https://www.flashpoint-intel.com/mirai-botnet-linked-dyn-dns-ddos-attacks/ [1] https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/ [2] http://arstechnica.com/security/2016/10/double-dip-internet-of-things-botnet-attack-felt-across-the-internet/ [3] https://blog.sucuri.net/2016/09/iot-home-router-botnet-leveraged-in-large-ddos-attack.html [4] https://krebsonsecurity.com/2016/10/source-code-for-iot-botnet-mirai-released/ [5] https://github.com/jgamblin/Mirai-Source-Code -- Pete Baldridge 206.992.2852 From list at satchell.net Sat Oct 22 21:58:47 2016 From: list at satchell.net (Stephen Satchell) Date: Sat, 22 Oct 2016 14:58:47 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> Message-ID: That's what VPNs are for. On 10/22/2016 10:04 AM, jim deleskie wrote: > It is also likely the desired use case. In my office I like to be able to > login when needed when on the road, when the alarm company calls me at 2am > for a false alarm so I don't have to get someone else out of bed to have > them dispatched to check on the site. > > -jim > > On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd wrote: > >> >>> On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: >>> >>> "taken all necessary steps to insure that none of the numerous specific >> types of CCVT thingies that Krebs and others identified" >>> >>> Serious question... how? >> >> Putting them behind a firewall without general Internet access seems to >> work for us. We have a lot of cheap IP cameras in our facility and none of >> them can reach the net. But this is probably a bit beyond the capabilities >> of the general home user. >> >> ?Chris >> >> From md at bts.sk Sat Oct 22 21:59:09 2016 From: md at bts.sk (=?UTF-8?Q?Marian_=C4=8Eurkovi=C4=8D?=) Date: Sat, 22 Oct 2016 23:59:09 +0200 Subject: MPLS in the campus Network? In-Reply-To: <589792b3-f0d8-e4dd-bab2-733013dde2a2@seacom.mu> References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> <589792b3-f0d8-e4dd-bab2-733013dde2a2@seacom.mu> Message-ID: <20161022205543.M58704@bts.sk> On Sat, 22 Oct 2016 21:29:22 +0200, Mark Tinka wrote > On 21/Oct/16 19:02, Javier Solis wrote: > > With that said, what are the best options to be able to cost effectively > > scale without using vlans and maintaining a routed core? What technology > > would someone suggest (mpls, vxlan,etc) to be the best possible solution? > IME, MPLS is a good use-case here. If you are going to use the same /24 (or > whatever prefix applies to you) across multiple locations, you will need some > kind of overlay. Be it IP-in-IP, GRE, MPLS (l2vpn's or l3vpn's) or plain old > Ethernet, you will need something. > > MPLS makes a lot of sense to me. It's native in hardware, upper-layer > agnostic, mature, and reasonably affordable even at low scale. The question here is, whether MPLS is the *optimal* solution for campus needs. The same functionality could be obviously achived by multiple technologies, and while MPLS is well supported on high-end SP routers, various limitations appear when people try to use it on commodity ASICs which typically empower today's ethernet switches - one of them being e.g. limited ability to effectively load-balance traffic over multiple parallel links. Yes, in theory we could build all campus LANs using high-end SP routers, but when 100GE backbone is desired (which is often the case in EDU/NREN sector), the costs of such solution jump to unacceptable heights. Thus we looked for another technology, which doesn't have the usual L2 problems and is able to provide services we need (including L2 extensions to remote campuses) at reasonable costs and with enough simplicity. To avoid typical L2 problems, you clearly need a solution based on L3 routing. And TRILL is exactly that - although it maintains L2 interface to the outside world, internally it performs dynamic L3 routing by IS-IS protocol with all safety belts like TTL check, RPF check etc. IMHO, TRILL is much better fit for campus needs, since it was specifically designed for this networking space - and our 6-months production fully confirms that view (of course, YMMV). With kind regards, M. From marka at isc.org Sat Oct 22 22:09:26 2016 From: marka at isc.org (Mark Andrews) Date: Sun, 23 Oct 2016 09:09:26 +1100 Subject: Death of the Internet, Film at 11 In-Reply-To: Your message of "Sat, 22 Oct 2016 14:45:13 -0700." References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> Message-ID: <20161022220926.68FD3576BA95@rock.dv.isc.org> One way to deal with this would be for ISP's to purchase DoS attacks against their own servers (not necessarially hosted on your own network) then look at which connections from their network attacking these machines then quarantine these connections after a delay period so that attacks can't be corollated with quarantine actions easily. This doesn't require a ISP to attempt to break into a customers machine to identify them. It may take several runs to identify most of the connections associated with a DoS provider. -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From josh at kyneticwifi.com Sat Oct 22 22:22:49 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Sat, 22 Oct 2016 17:22:49 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> Message-ID: And then what? The labor to clean up this mess is not free. Who's responsibility is it? The grandma who got a webcam for Christmas to watch the squirrels? The ISP?... No... The vendor? What if the vendor had released a patch to fix the issue months back, and grandma hadn't installed it? Making grandma and auntie Em responsible for the IT things in their house is likely not going to go well. Making the vendor responsible might work for the reputable ones to a point, but won't work for the fly by night shops that will sell the same products under different company names and model names until they get sued or "one starred" into oblivion. Then they just change names and start all over. The ISPs won't do it because of the cost to fix... The labor and potential loss of customers. So once identified, how do you suggest this gets fixed? On Oct 22, 2016 5:11 PM, "Mark Andrews" wrote: One way to deal with this would be for ISP's to purchase DoS attacks against their own servers (not necessarially hosted on your own network) then look at which connections from their network attacking these machines then quarantine these connections after a delay period so that attacks can't be corollated with quarantine actions easily. This doesn't require a ISP to attempt to break into a customers machine to identify them. It may take several runs to identify most of the connections associated with a DoS provider. -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From blakjak at blakjak.net Sat Oct 22 22:30:04 2016 From: blakjak at blakjak.net (Mark Foster) Date: Sun, 23 Oct 2016 11:30:04 +1300 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> Message-ID: The person who owns the internet connection still has responsibility for what happens on it. So if the owners are educated to select reputable brands in order to prevent themselves from being implicated in a DDoS and liable for a fine or some other punitive thing, they 'vote with their feet' and the fly-by-nighters suddenly lose a chunk of marketshare, unless they up their game? I'm as sympathetic to Aunty Em and Grandma as the next I-started-on-a-helpdesk guys, but 'you get what you pay for' applies here as much as it does everywhere else...? On 23/10/2016 11:22 a.m., Josh Reynolds wrote: > And then what? The labor to clean up this mess is not free. Who's > responsibility is it? The grandma who got a webcam for Christmas to watch > the squirrels? The ISP?... No... The vendor? What if the vendor had > released a patch to fix the issue months back, and grandma hadn't installed > it? > > Making grandma and auntie Em responsible for the IT things in their house > is likely not going to go well. > > Making the vendor responsible might work for the reputable ones to a point, > but won't work for the fly by night shops that will sell the same products > under different company names and model names until they get sued or "one > starred" into oblivion. Then they just change names and start all over. > > The ISPs won't do it because of the cost to fix... The labor and potential > loss of customers. > > So once identified, how do you suggest this gets fixed? *snip* From rvandolson at esri.com Sat Oct 22 22:35:50 2016 From: rvandolson at esri.com (Ray Van Dolson) Date: Sat, 22 Oct 2016 15:35:50 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> Message-ID: <20161022223550.GA12610@esri.com> https://urldefense.proofpoint.com/v2/url?u=http-3A__hub.dyn.com_dyn-2Dblog_dyn-2Dstatement-2Don-2D10-2D21-2D2016-2Dddos-2Dattack&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=r4NBNYp4yEcJxC11Po5I-w&m=iGvkbfzRJPqKO1A6YGa-c1m0RBLNkRk03hCjvVGTH3k&s=bScBNFncB3kt_cG0L3iys0mfXBmwwUR7A8rIDmi94D4&e= On Sat, Oct 22, 2016 at 04:48:01PM -0500, Mike Hammett wrote: > Until Dyn says or someone says Dyn said, everything is assumed. > > ----- Original Message ----- > > From: "Peter Baldridge" > To: "Jean-Francois Mezei" > Cc: nanog at nanog.org > Sent: Saturday, October 22, 2016 4:45:13 PM > Subject: Re: Death of the Internet, Film at 11 > > On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei > wrote: > > Generic question: > > > > The media seems to have concluded it was an "internet of things" that > > caused this DDoS. > > > > I have not seen any evidence of this. Has this been published by an > > authoritative source or is it just assumed? > > Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible > looks like unless they release a packet but this is probably > consensus. > > > Has the type of device involved been identified? > > routers and cameras with shitty firmware [3] > > > Is it more plausible that those devices were "hacked" in the OEM > > firmware and sold with the "virus" built-in ? That would explain the > > widespread attack. > > The source code has been released. krebs [4], code [5] > > > Also, in cases such as this one, while the target has managed to > > mitigate the attack, how long would such an attack typically continue > > and require blocking ? > This is an actual question that hasn't been answered. > > > Since the attack seemed focused on eastern USA DNS servers, would it be > > fair to assume that the attacks came mostly from the same region (aka: > > devices installed in eastern USA) ? (since anycast would point them to > > that). > > Aren't heat maps just population graphs? > > > BTW, normally, if you change the "web" password on a "device", it would > > also change telnet/SSH/ftp passwords. > > Seems like no one is doing either. From josh at kyneticwifi.com Sat Oct 22 22:36:49 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Sat, 22 Oct 2016 17:36:49 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> Message-ID: I wish you luck with your plan, and please subscribe me to your newsletter in digest format. On Oct 22, 2016 5:32 PM, "Mark Foster" wrote: > The person who owns the internet connection still has responsibility for > what happens on it. > > So if the owners are educated to select reputable brands in order to > prevent themselves from being implicated in a DDoS and liable for a fine or > some other punitive thing, they 'vote with their feet' and the > fly-by-nighters suddenly lose a chunk of marketshare, unless they up their > game? > > I'm as sympathetic to Aunty Em and Grandma as the next > I-started-on-a-helpdesk guys, but 'you get what you pay for' applies here > as much as it does everywhere else...? > > > On 23/10/2016 11:22 a.m., Josh Reynolds wrote: > >> And then what? The labor to clean up this mess is not free. Who's >> responsibility is it? The grandma who got a webcam for Christmas to watch >> the squirrels? The ISP?... No... The vendor? What if the vendor had >> released a patch to fix the issue months back, and grandma hadn't >> installed >> it? >> >> Making grandma and auntie Em responsible for the IT things in their house >> is likely not going to go well. >> >> Making the vendor responsible might work for the reputable ones to a >> point, >> but won't work for the fly by night shops that will sell the same products >> under different company names and model names until they get sued or "one >> starred" into oblivion. Then they just change names and start all over. >> >> The ISPs won't do it because of the cost to fix... The labor and potential >> loss of customers. >> >> So once identified, how do you suggest this gets fixed? >> > > *snip* > From mark.tinka at seacom.mu Sat Oct 22 22:38:47 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Sun, 23 Oct 2016 00:38:47 +0200 Subject: MPLS in the campus Network? In-Reply-To: <20161022205543.M58704@bts.sk> References: <20161021141918.GA65694@bts.sk> <2bcbf431-f647-88e3-15ac-0cfd3b22991a@seacom.mu> <589792b3-f0d8-e4dd-bab2-733013dde2a2@seacom.mu> <20161022205543.M58704@bts.sk> Message-ID: On 22/Oct/16 23:59, Marian ?urkovi? wrote: > > The question here is, whether MPLS is the *optimal* solution for campus needs. > > The same functionality could be obviously achived by multiple technologies, > and while MPLS is well supported on high-end SP routers, various limitations > appear when people try to use it on commodity ASICs which typically empower > today's ethernet switches - one of them being e.g. limited ability to > effectively load-balance traffic over multiple parallel links. > > Yes, in theory we could build all campus LANs using high-end SP routers, but > when 100GE backbone is desired (which is often the case in EDU/NREN sector), > the costs of such solution jump to unacceptable heights. > > Thus we looked for another technology, which doesn't have the usual L2 problems > and is able to provide services we need (including L2 extensions to remote > campuses) at reasonable costs and with enough simplicity. > > To avoid typical L2 problems, you clearly need a solution based on L3 routing. > And TRILL is exactly that - although it maintains L2 interface to the outside > world, internally it performs dynamic L3 routing by IS-IS protocol with all > safety belts like TTL check, RPF check etc. > > IMHO, TRILL is much better fit for campus needs, since it was specifically > designed for this networking space - and our 6-months production fully confirms > that view (of course, YMMV). I don't consider the ASR920 or CES2000 to be particularly high-end routers, but YMMV. True, merchant silicon presents a number of data plane challenges that may, at first, seem non-trivial or completely go unnoticed. That is why we stay away from the ACX5000, for example. I expect improvements to come with newer-generation ASIC's/NP's, but that tests one's patience. But, like I said, I have not run TRILL myself, so I'm not going to tell you that it's not an ideal technology for this use-case. All I'll say is that MPLS is not limited to high-end platforms, even when custom silicon is involved. Mark. From nanog at ics-il.net Sat Oct 22 22:48:42 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sat, 22 Oct 2016 17:48:42 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: <20161022223550.GA12610@esri.com> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> <20161022223550.GA12610@esri.com> Message-ID: <460558240.4861.1477176517961.JavaMail.mhammett@ThunderFuck> Thanks for the link. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Ray Van Dolson" To: "Mike Hammett" Cc: nanog at nanog.org Sent: Saturday, October 22, 2016 5:35:50 PM Subject: Re: Death of the Internet, Film at 11 https://urldefense.proofpoint.com/v2/url?u=http-3A__hub.dyn.com_dyn-2Dblog_dyn-2Dstatement-2Don-2D10-2D21-2D2016-2Dddos-2Dattack&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=5PqhtPogDeswmEQMQZk1IQ&m=6rpDhHbntFiyuuA6uUxOIVfEwHY13H9SH6zBwx93OBE&s=QIsYvf_c8f_VWuMbYe7DbF58d1UqsbxJBEjf8CYotcc&e= On Sat, Oct 22, 2016 at 04:48:01PM -0500, Mike Hammett wrote: > Until Dyn says or someone says Dyn said, everything is assumed. > > ----- Original Message ----- > > From: "Peter Baldridge" > To: "Jean-Francois Mezei" > Cc: nanog at nanog.org > Sent: Saturday, October 22, 2016 4:45:13 PM > Subject: Re: Death of the Internet, Film at 11 > > On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei > wrote: > > Generic question: > > > > The media seems to have concluded it was an "internet of things" that > > caused this DDoS. > > > > I have not seen any evidence of this. Has this been published by an > > authoritative source or is it just assumed? > > Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible > looks like unless they release a packet but this is probably > consensus. > > > Has the type of device involved been identified? > > routers and cameras with shitty firmware [3] > > > Is it more plausible that those devices were "hacked" in the OEM > > firmware and sold with the "virus" built-in ? That would explain the > > widespread attack. > > The source code has been released. krebs [4], code [5] > > > Also, in cases such as this one, while the target has managed to > > mitigate the attack, how long would such an attack typically continue > > and require blocking ? > This is an actual question that hasn't been answered. > > > Since the attack seemed focused on eastern USA DNS servers, would it be > > fair to assume that the attacks came mostly from the same region (aka: > > devices installed in eastern USA) ? (since anycast would point them to > > that). > > Aren't heat maps just population graphs? > > > BTW, normally, if you change the "web" password on a "device", it would > > also change telnet/SSH/ftp passwords. > > Seems like no one is doing either. From marka at isc.org Sat Oct 22 22:56:02 2016 From: marka at isc.org (Mark Andrews) Date: Sun, 23 Oct 2016 09:56:02 +1100 Subject: Death of the Internet, Film at 11 In-Reply-To: Your message of "Sat, 22 Oct 2016 17:22:49 -0500." References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> Message-ID: <20161022225602.69E5D576BE82@rock.dv.isc.org> In message , Josh Reynolds writes: > > And then what? They get in someone to clean up their network. When they say it is clean you reconnect them. If this happens more often than once a year you charge them a months fees per additional incident. Have the year timer start when reconnect is requested. You give them what data you have to backup the claim. > The labor to clean up this mess is not free. Who's > responsibility is it? The grandma who got a webcam for Christmas to watch > the squirrels? The ISP?... No... The vendor? What if the vendor had > released a patch to fix the issue months back, and grandma hadn't installed > it? > > Making grandma and auntie Em responsible for the IT things in their house > is likely not going to go well. > > Making the vendor responsible might work for the reputable ones to a point, > but won't work for the fly by night shops that will sell the same products > under different company names and model names until they get sued or "one > starred" into oblivion. Then they just change names and start all over. > > The ISPs won't do it because of the cost to fix... The labor and potential > loss of customers. > > So once identified, how do you suggest this gets fixed? > > On Oct 22, 2016 5:11 PM, "Mark Andrews" wrote: > > > One way to deal with this would be for ISP's to purchase DoS attacks > against their own servers (not necessarially hosted on your own > network) then look at which connections from their network attacking > these machines then quarantine these connections after a delay > period so that attacks can't be corollated with quarantine actions > easily. > > This doesn't require a ISP to attempt to break into a customers > machine to identify them. It may take several runs to identify > most of the connections associated with a DoS provider. > > -- > Mark Andrews, ISC > 1 Seymour St., Dundas Valley, NSW 2117, Australia > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org > > --94eb2c030b6c594dc5053f7b994f > Content-Type: text/html; charset=UTF-8 > Content-Transfer-Encoding: quoted-printable > >

And then what? The labor to clean up this mess is not free. = > Who's responsibility is it? The grandma who got a webcam for Christmas = > to watch the squirrels? The ISP?... No... The vendor? What if the vendor ha= > d released a patch to fix the issue months back, and grandma hadn't ins= > talled it?

>

Making grandma and auntie Em responsible for the IT things i= > n their house is likely not going to go well.

>

Making the vendor responsible might work for the reputable o= > nes to a point, but won't work for the fly by night shops that will sel= > l the same products under different company names and model names until the= > y get sued or "one starred" into oblivion. Then they just change = > names and start all over.

>

The ISPs won't do it because of the cost to fix... The l= > abor and potential loss of customers.

>

So once identified, how do you suggest this gets fixed?

>

On Oct 22, 2016 5= > :11 PM, "Mark Andrews" <marka= > @isc.org> wrote:
style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> r> > One way to deal with this would be for ISP's to purchase DoS attacks > > against their own servers (not necessarially hosted on your own
> network) then look at which connections from their network attacking
> these machines then quarantine these connections after a delay
> period so that attacks can't be corollated with quarantine actions
> easily.
>
> This doesn't require a ISP to attempt to break into a customers
> machine to identify them.=C2=A0 It may take several runs to identify
> most of the connections associated with a DoS provider.
>
> --
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2= > 9871 4742=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= > =A0INTERNET: marka at isc.org
>

> > --94eb2c030b6c594dc5053f7b994f-- -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From josh at kyneticwifi.com Sat Oct 22 23:01:31 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Sat, 22 Oct 2016 18:01:31 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: <20161022225602.69E5D576BE82@rock.dv.isc.org> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <20161022225602.69E5D576BE82@rock.dv.isc.org> Message-ID: One sec, starting a relationship with $CPEvendor... I'll let you know how this goes. "Yes, every customer I went to had malware. That's okay, right?" ;) On Oct 22, 2016 5:56 PM, "Mark Andrews" wrote: > > In message mail.gmail.com> > , Josh Reynolds writes: > > > > And then what? > > They get in someone to clean up their network. When they say it > is clean you reconnect them. If this happens more often than once > a year you charge them a months fees per additional incident. Have > the year timer start when reconnect is requested. You give them > what data you have to backup the claim. > > > The labor to clean up this mess is not free. Who's > > responsibility is it? The grandma who got a webcam for Christmas to watch > > the squirrels? The ISP?... No... The vendor? What if the vendor had > > released a patch to fix the issue months back, and grandma hadn't > installed > > it? > > > > Making grandma and auntie Em responsible for the IT things in their house > > is likely not going to go well. > > > > > Making the vendor responsible might work for the reputable ones to a > point, > > but won't work for the fly by night shops that will sell the same > products > > under different company names and model names until they get sued or "one > > starred" into oblivion. Then they just change names and start all over. > > > > The ISPs won't do it because of the cost to fix... The labor and > potential > > loss of customers. > > > > So once identified, how do you suggest this gets fixed? > > > > On Oct 22, 2016 5:11 PM, "Mark Andrews" wrote: > > > > > > One way to deal with this would be for ISP's to purchase DoS attacks > > against their own servers (not necessarially hosted on your own > > network) then look at which connections from their network attacking > > these machines then quarantine these connections after a delay > > period so that attacks can't be corollated with quarantine actions > > easily. > > > > This doesn't require a ISP to attempt to break into a customers > > machine to identify them. It may take several runs to identify > > most of the connections associated with a DoS provider. > > > > -- > > Mark Andrews, ISC > > 1 Seymour St., Dundas Valley, NSW 2117, Australia > > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org > > > > --94eb2c030b6c594dc5053f7b994f > > Content-Type: text/html; charset=UTF-8 > > Content-Transfer-Encoding: quoted-printable > > > >

And then what? The labor to clean up this mess is not > free. = > > Who's responsibility is it? The grandma who got a webcam for > Christmas = > > to watch the squirrels? The ISP?... No... The vendor? What if the vendor > ha= > > d released a patch to fix the issue months back, and grandma hadn't > ins= > > talled it?

> >

Making grandma and auntie Em responsible for the IT > things i= > > n their house is likely not going to go well.

> >

Making the vendor responsible might work for the > reputable o= > > nes to a point, but won't work for the fly by night shops that will > sel= > > l the same products under different company names and model names until > the= > > y get sued or "one starred" into oblivion. Then they just > change = > > names and start all over.

> >

The ISPs won't do it because of the cost to fix... > The l= > > abor and potential loss of customers.

> >

So once identified, how do you suggest this gets > fixed?

> >

On Oct 22, > 2016 5= > > :11 PM, "Mark Andrews" < > marka= > > @isc.org> wrote:
class=3D"quote"= > > style=3D"margin:0 0 0 .8ex;border-left:1px #ccc > solid;padding-left:1ex"> > r> > > One way to deal with this would be for ISP's to purchase DoS > attacks > > > > against their own servers (not necessarially hosted on your own
> > network) then look at which connections from their network attacking
> > these machines then quarantine these connections after a delay
> > period so that attacks can't be corollated with quarantine > actions
> > easily.
> >
> > This doesn't require a ISP to attempt to break into a customers
> > machine to identify them.=C2=A0 It may take several runs to identify
> > most of the connections associated with a DoS provider.
> >
> > --
> > Mark Andrews, ISC
> > 1 Seymour St., Dundas Valley, NSW 2117, Australia
> > PHONE: value=3D"+61298714742">+61 2= > > 9871 4742=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > =C2= > > =A0INTERNET: marka at isc.org
> >

> > > > --94eb2c030b6c594dc5053f7b994f-- > -- > Mark Andrews, ISC > 1 Seymour St., Dundas Valley, NSW 2117, Australia > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org > From jra at baylink.com Sat Oct 22 16:05:18 2016 From: jra at baylink.com (Jay R. Ashworth) Date: Sat, 22 Oct 2016 16:05:18 +0000 (UTC) Subject: Honorary Unsubscribe: Leo Beranek Message-ID: <1228909557.6439.1477152318973.JavaMail.zimbra@baylink.com> How many people remember that Bolt Beranek and Newman was originally an acoustical consultancy, specializing in concert halls? http://www.honoraryunsubscribe.com/leo_beranek.html?awt_l=ACI.7&awt_m=JXfIgZRK.SAPkr Happy Landings, Leo! Cheers, -- jra -- Jay R. Ashworth Baylink jra at baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 From jfmezei_nanog at vaxination.ca Sat Oct 22 23:22:04 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sat, 22 Oct 2016 19:22:04 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <20161022223550.GA12610@esri.com> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> <20161022223550.GA12610@esri.com> Message-ID: <580BF49C.5090209@vaxination.ca> On 2016-10-22 18:35, Ray Van Dolson wrote: > https://urldefense.proofpoint.com/v2/url?u=http-3A__hub.dyn.com_dyn-2Dblog_dyn-2Dstatement-2Don-2D10-2D21-2D2016-2Dddos-2Dattack&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=r4NBNYp4yEcJxC11Po5I-w&m=iGvkbfzRJPqKO1A6YGa-c1m0RBLNkRk03hCjvVGTH3k&s=bScBNFncB3kt_cG0L3iys0mfXBmwwUR7A8rIDmi94D4&e= Thanks for the link. 10s of millons of IP addresses. Is it realistic to have 10s of millions of infected devices ? Or is that the dense smoke that points to IP spoofing ? re: newspaper reports: how did Flashpoint obtain enough details, while attack was ongoing to be able to draw conclusions told to the media ? Or was it educated speculation ? Obviously, Dyn had packet contents to look at and range of IPs being used etc. Would such a company typically release that info to a trusted investigator "as it happens" ? (would Flashpoint be such an outfit ?) Did the attack generate valid DNS queries (overwhelm the servers) or flood the links with long "random" UDP packets (overwhel the links). While I can understand that mitigation methods can be seen as "proprietary", releasing info on the specifics of the attack would help any/all neteowkrs and data centres better protect themselves. Assuming hackers don't talk to each others in the 21st century is silly. They already know how this was done, yet the victims typically remain silent for fear of educating the hackers for more attacks. From kmedcalf at dessus.com Sat Oct 22 23:03:44 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Sat, 22 Oct 2016 17:03:44 -0600 Subject: FW: Death of the Internet, Film at 11 In-Reply-To: <21505498d53f19429a65c2f1319ec4df@mail.dessus.com> Message-ID: <2795ecc96e8190449cd7e9987c7ba923@mail.dessus.com> > It's also generally counter to them being available outside of that > network. This does not follow and is not a natural consequence of sealing the little buggers up so that they cannot affect the Internet (or you private networks). Even if you lock you pet mouse in a cage, you can still feed it and clean up the shit in the cage. It just isn't free to wander out and eat the floral arrangements on the end-table. > (web and proprietary interfaces needed, SSH and telnet not). > That's also not much I can do as a network operator. > > > > > > > ----- > Mike Hammett > Intelligent Computing Solutions > http://www.ics-il.com > > Midwest-IX > http://www.midwest-ix.com > > ----- Original Message ----- > > From: "Chris Boyd" > To: "Elizabeth Zwicky via NANOG" > Sent: Saturday, October 22, 2016 11:42:05 AM > Subject: Re: Death of the Internet, Film at 11 > > > > On Oct 22, 2016, at 7:34 AM, Mike Hammett wrote: > > > > "taken all necessary steps to insure that none of the numerous specific > types of CCVT thingies that Krebs and others identified" > > > > Serious question... how? > > Putting them behind a firewall without general Internet access seems to > work for us. We have a lot of cheap IP cameras in our facility and none of > them can reach the net. But this is probably a bit beyond the capabilities > of the general home user. > > ?Chris > From md1clv at md1clv.com Sat Oct 22 16:20:38 2016 From: md1clv at md1clv.com (Daniel Ankers) Date: Sat, 22 Oct 2016 17:20:38 +0100 Subject: Dyn DDoS this AM? In-Reply-To: <580B8866.2060200@yahoo.fr> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <580B8866.2060200@yahoo.fr> Message-ID: On 22 October 2016 at 16:40, marcel.duregards--- via NANOG wrote: > What about BCP38+84 on 30 tier-1 instead of asking/hoping 55k others > autonomous-system having good filters in place ? The originating ISPs are in a far better position to check that traffic isn't from spoofed address ranges than transit networks are. The best thing to do is to ask EVERY network to do what they can, not just the few biggest ones. Any size ISP can be hit by and hurt by DDoS attacks, so every size ISP should be doing what they can to make sure they are not either the source or the victim of those attacks. Dan From masoodnt10 at gmail.com Sat Oct 22 02:22:45 2016 From: masoodnt10 at gmail.com (Masood Ahmad Shah) Date: Sat, 22 Oct 2016 13:22:45 +1100 Subject: Dyn DDoS this AM? In-Reply-To: References: <580ABCF7.1030003@vaxination.ca> Message-ID: > > > On Oct 21, 2016, at 6:35 PM, Eitan Adler wrote: > > > > [...] > > > > In practice TTLs tend to be ignored on the public internet. In past > > research I've been involved with browser[0] behavior was effectively > > random despite the TTL set. > > > > [0] more specifically, the chain of DNS resolution and caching down to > > the browser. > > > Yes, but that it can be both better and worse than your TTLs does not mean > that you can ignore properly working implementations. > > If the other end device chain breaks you that's their fault and out of > your control. If your own settings break you that's your fault. > +1 to what George wrote that we should make efforts to improve our part of the network. There are ISPs that ignore TTL settings and only update their cached records every two to three days or even more (particularly the smaller ones). OTOH, this results in your DNS data being inconsistent but it?s very common to cache DNS records at multiple levels. It's an effort that everyone needs to contribute to. > > Sent from my iPhone From szlists at szarka.org Sat Oct 22 03:23:56 2016 From: szlists at szarka.org (Rob Szarka) Date: Fri, 21 Oct 2016 23:23:56 -0400 Subject: Dyn DDoS this AM? In-Reply-To: <821401bf-f6ac-ab3b-10c9-c2d897fe254c@stargate.ca> References: <20161021231924.GG45065@excession.tpb.net> <821401bf-f6ac-ab3b-10c9-c2d897fe254c@stargate.ca> Message-ID: <4f091017-514c-a0ac-0777-275fd0dc3daa@szarka.org> On 10/21/2016 7:34 PM, Keenan Tims wrote: > I don't have a horse in this race, and haven't used it in anger, but > Netflix released denominator to attempt to deal with some of these > issues: > > https://github.com/Netflix/denominator > > Their goal is to support the highest common denominator of features > among the supported providers, > > Maybe that helps someone. Sadly, it looks like the project is stalled: . -- Rob Szarka http://szarka.org/ From lguillory at reservetele.com Sat Oct 22 17:09:21 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Sat, 22 Oct 2016 17:09:21 +0000 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> , Message-ID: VPNs can accomplish this without opening ports directly to devices. Luke Sent from my iPhone On Oct 22, 2016, at 12:06 PM, jim deleskie > wrote: It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site. -jim On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd > wrote: On Oct 22, 2016, at 7:34 AM, Mike Hammett > wrote: "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" Serious question... how? Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user. ?Chris Luke Guillory Network Operations Manager [cid:imagee03d14.JPG at 65e9954a.43918993] Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Web: www.rtconline.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. From lguillory at reservetele.com Sat Oct 22 17:28:09 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Sat, 22 Oct 2016 17:28:09 +0000 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> , Message-ID: <923596CB-CF53-4703-88EE-98EB953B0F29@reservetele.com> I was referring to your use case and it being a business, for residential I agree with you. Sent from my iPhone On Oct 22, 2016, at 12:21 PM, jim deleskie > wrote: Sure, but now we put it outside the skill level of 99.99% of the people that don't read and understand this list. -jim Luke Guillory Network Operations Manager [cid:image3d3347.JPG at 3f84cedc.4b99894e] Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Web: www.rtconline.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. On Sat, Oct 22, 2016 at 2:09 PM, Luke Guillory > wrote: VPNs can accomplish this without opening ports directly to devices. Luke Sent from my iPhone On Oct 22, 2016, at 12:06 PM, jim deleskie > wrote: It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site. -jim On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd > wrote: On Oct 22, 2016, at 7:34 AM, Mike Hammett > wrote: "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" Serious question... how? Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user. ?Chris Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Web: www.rtconline.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. From surfer at mauigateway.com Sat Oct 22 23:36:04 2016 From: surfer at mauigateway.com (Scott Weeks) Date: Sat, 22 Oct 2016 16:36:04 -0700 Subject: Death of the Internet, Film at 11 Message-ID: <20161022163604.5A213F1D@m0086238.ppops.net> > On Oct 22, 2016 5:11 PM, "Mark Andrews" wrote: > One way to deal with this would be for ISP's to purchase DoS attacks > against their own servers (not necessarially hosted on your own > network) then look at which connections from their network attacking > these machines then quarantine these connections after a delay > period so that attacks can't be corollated with quarantine actions > easily. > > This doesn't require a ISP to attempt to break into a customers > machine to identify them. It may take several runs to identify > most of the connections associated with a DoS provider. Josh Reynolds writes: > > And then what? --- marka at isc.org wrote: From: Mark Andrews They get in someone to clean up their network. When they say it is clean you reconnect them. If this happens more often than once a year you charge them a months fees per additional incident. Have the year timer start when reconnect is requested. You give them what data you have to backup the claim. -------------------------------------------------- I invoke randy's "i encourage my competitor's to do this". scott From jfmezei_nanog at vaxination.ca Sat Oct 22 23:41:17 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sat, 22 Oct 2016 19:41:17 -0400 Subject: FW: Death of the Internet, Film at 11 In-Reply-To: <2795ecc96e8190449cd7e9987c7ba923@mail.dessus.com> References: <2795ecc96e8190449cd7e9987c7ba923@mail.dessus.com> Message-ID: <580BF91D.9060902@vaxination.ca> On 2016-10-22 19:03, Keith Medcalf wrote: > This does not follow and is not a natural consequence of sealing the little buggers up so that they cannot affect the Internet Problem is that many of these gadgets want to be internet connected so mother at work can check on her kids at home, start the cooking, raise thermostat etc. The problem is that as a novelty, people are quick to adopt, but don't think about making their homes vulnerable to attack. (consider an internet connected door lock) From kmedcalf at dessus.com Sun Oct 23 00:14:13 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Sat, 22 Oct 2016 18:14:13 -0600 Subject: Death of the Internet, Film at 11 Message-ID: <83338d777e188642b655159c2c40b2b0@mail.dessus.com> On: Saturday, 22 October, 2016 17:41, Jean-Francois Mezei wrote: > On 2016-10-22 19:03, Keith Medcalf wrote: > > This does not follow and is not a natural consequence of sealing the > little buggers up so that they cannot affect the Internet > Problem is that many of these gadgets want to be internet connected so > mother at work can check on her kids at home, start the cooking, raise > thermostat etc. This does not require that the devices be open to the Internet, nor does it require that they are under the control of an Internet based controller. > The problem is that as a novelty, people are quick to adopt, but don't > think about making their homes vulnerable to attack. (consider an > internet connected door lock) There are many people who do not read this list who would have nothing whatsoever to do with such a scheme (earlier similar schemes are routers & etc that are programmed and controlled from the "web", and remote access crap which is proxied through a third-party web server -- another ill-conceived and brain-dead idea). This is a self-limiting issue. Darwin will take care of it. Unfortunately there will be collateral damage as those not fit to the continuation of the species are eliminated from the gene pool. We should do our duty and make sure that the pool cleaning proceeds with the maximum speed and efficiency possible. From josh at kyneticwifi.com Sun Oct 23 01:00:04 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Sat, 22 Oct 2016 20:00:04 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: <83338d777e188642b655159c2c40b2b0@mail.dessus.com> References: <83338d777e188642b655159c2c40b2b0@mail.dessus.com> Message-ID: Modern medicine, sanitation, and sedentary lifestyles for the developed world have effectively culled natural selection for most internet users. On Oct 22, 2016 7:16 PM, "Keith Medcalf" wrote: > > On: Saturday, 22 October, 2016 17:41, Jean-Francois Mezei < > jfmezei_nanog at vaxination.ca> wrote: > > > On 2016-10-22 19:03, Keith Medcalf wrote: > > > > This does not follow and is not a natural consequence of sealing the > > little buggers up so that they cannot affect the Internet > > > Problem is that many of these gadgets want to be internet connected so > > mother at work can check on her kids at home, start the cooking, raise > > thermostat etc. > > This does not require that the devices be open to the Internet, nor does > it require that they are under the control of an Internet based controller. > > > The problem is that as a novelty, people are quick to adopt, but don't > > think about making their homes vulnerable to attack. (consider an > > internet connected door lock) > > There are many people who do not read this list who would have nothing > whatsoever to do with such a scheme (earlier similar schemes are routers & > etc that are programmed and controlled from the "web", and remote access > crap which is proxied through a third-party web server -- another > ill-conceived and brain-dead idea). This is a self-limiting issue. Darwin > will take care of it. Unfortunately there will be collateral damage as > those not fit to the continuation of the species are eliminated from the > gene pool. > > We should do our duty and make sure that the pool cleaning proceeds with > the maximum speed and efficiency possible. > > > > > From fkittred at gwi.net Sun Oct 23 01:09:10 2016 From: fkittred at gwi.net (Fletcher Kittredge) Date: Sat, 22 Oct 2016 21:09:10 -0400 Subject: Honorary Unsubscribe: Leo Beranek In-Reply-To: <1228909557.6439.1477152318973.JavaMail.zimbra@baylink.com> References: <1228909557.6439.1477152318973.JavaMail.zimbra@baylink.com> Message-ID: Talk about a life well led. Leo Beranek had 102 years of sustained creativity. Any one of his three or four careers would have been remarkable. In the 1940s, 1950s, 1960s, he laid the foundation that young whippersnappers such as Cerf and Postel would build on. He was contributing into his 100s. He was always a kind and pleasant man who led by example and deference. On Sat, Oct 22, 2016 at 12:05 PM, Jay R. Ashworth wrote: > How many people remember that Bolt Beranek and Newman was originally an > acoustical consultancy, specializing in concert halls? > > http://www.honoraryunsubscribe.com/leo_beranek.html?awt_l=ACI.7&awt_ > m=JXfIgZRK.SAPkr > > Happy Landings, Leo! > > Cheers, > -- jra > -- > Jay R. Ashworth Baylink > jra at baylink.com > Designer The Things I Think RFC > 2100 > Ashworth & Associates http://www.bcp38.info 2000 Land > Rover DII > St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 > 1274 > -- Fletcher Kittredge GWI 207-602-1134 www.gwi.net From jw at nuclearfallout.net Sun Oct 23 05:17:40 2016 From: jw at nuclearfallout.net (John Weekes) Date: Sat, 22 Oct 2016 22:17:40 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <85864.1477115622@segfault.tristatelogic.com> References: <85864.1477115622@segfault.tristatelogic.com> Message-ID: <26b01962-9b09-11cb-0ac8-89cf3e0a5f96@nuclearfallout.net> > > Ok, so this mailing list is a list of network operators. Swell. Every > network operator who can do so, please raise your hand if you have > *recently* scanned you own network and if you can -honestly- attest > that you have taken all necessary steps to insure that none of the > numerous specific types of CCVT thingies that Krebs and others identified > weeks or months ago as being fundamentally insecure can emit a single > packet out onto the public Internet. Most of the time, scanning of your customers isn't strictly necessary, though it certainly won't hurt. That's because attackers will scan your network /for /you, compromise the hosts, and use them to attack. When they inevitably attack one of my customers, I'll send you an abuse email. Some other networks do the same. So if you want to help, the real keys are to make sure that you disallow spoofing, that the RIR has up-to-date contact information for your organization, and that you handle abuse notifications effectively. Large IoT botnets have been used extensively this year, launching frequent 100+ Gbps attacks (they were also used in prior years, but it wasn't to the degree that we've seen since January 2016). I've recorded about 2.4 million IP addresses involved in the last two months (a number that is higher than the number of actual devices, since most seem to have dynamic IP addresses). The ISPs behind those IP addresses have received notifications via email, so if you haven't heard anything, you're probably in good shape, assuming the RIR has the right abuse address on file for you. The bulk of the compromised devices are non-NA. In a relatively small 40 Gbps IoT attack a couple of days ago, we saw about 20k devices, for instance, and most were from a mix of China, Brazil, Russia, Korea, and Venezuela. -John From sthaug at nethelp.no Sun Oct 23 07:08:10 2016 From: sthaug at nethelp.no (sthaug at nethelp.no) Date: Sun, 23 Oct 2016 09:08:10 +0200 (CEST) Subject: Death of the Internet, Film at 11 In-Reply-To: <20161022223550.GA12610@esri.com> References: <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> <20161022223550.GA12610@esri.com> Message-ID: <20161023.090810.74670527.sthaug@nethelp.no> >From Dyn's statement, http://hub.dyn.com/static/hub.dyn.com/dyn-blog/dyn-statement-on-10-21-2016-ddos-attack.html we have "After restoring service, Dyn experienced a second wave of attacks just before noon ET. This second wave was more global in nature (i.e. not limited to our East Coast POPs), but was mitigated in just over an hour; service was restored at approximately 1:00 pm ET." I can't reconcile this what I saw on our recursive name servers here in Norway, looking at the period when Dyn's name servers didn't answer at all. We saw the second wave start at around 15:56 UTC (which corresponds well with "just before noon ET") and end at around 20:00 UTC - so from our point of view, service was unavailable for around *four* hours. Steinar Haug, Nethelp consulting, sthaug at nethelp.no From fw at deneb.enyo.de Sun Oct 23 10:08:10 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 23 Oct 2016 12:08:10 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: (David Conrad's message of "Sat, 22 Oct 2016 11:21:49 -0700") References: <201610221502.QAA15723@sunf10.rd.bbc.co.uk> <241612512.4558.1477148926944.JavaMail.mhammett@ThunderFuck> Message-ID: <87funnqsxh.fsf@mid.deneb.enyo.de> * David Conrad: > Maybe (not sure) one way would be to examine your resolver query logs > to look for queries for names that fit domain generation algorithm > patterns, then tracking down the customers/devices that are issuing > those queries and politely suggest they remove the malware on their > systems? Where would interested operators get that information? Would this include information how to identify those devices which participated in the CCTV-based botnet which allegedly took part in the recent attacks? From fw at deneb.enyo.de Sun Oct 23 10:11:30 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 23 Oct 2016 12:11:30 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: <83338d777e188642b655159c2c40b2b0@mail.dessus.com> (Keith Medcalf's message of "Sat, 22 Oct 2016 18:14:13 -0600") References: <83338d777e188642b655159c2c40b2b0@mail.dessus.com> Message-ID: <87bmybqsrx.fsf@mid.deneb.enyo.de> * Keith Medcalf: > On: Saturday, 22 October, 2016 17:41, Jean-Francois Mezei > wrote: > >> On 2016-10-22 19:03, Keith Medcalf wrote: > >> > This does not follow and is not a natural consequence of sealing the >> little buggers up so that they cannot affect the Internet > >> Problem is that many of these gadgets want to be internet connected so >> mother at work can check on her kids at home, start the cooking, raise >> thermostat etc. > > This does not require that the devices be open to the Internet, nor > does it require that they are under the control of an Internet based > controller. How would you know? It is perfectly reasonable to send a notification to a device by making a TCP connection to it. This is the way the Internet was built. You are not expected to sign a contract with the network operator for the target device before you can establish a connection to the device. The possibility of denial-of-service attacks is not a sufficient reason to change that model. From fw at deneb.enyo.de Sun Oct 23 10:16:13 2016 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 23 Oct 2016 12:16:13 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: (Randy Bush's message of "Sat, 22 Oct 2016 12:08:55 +0900") References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> Message-ID: <874m43qsk2.fsf@mid.deneb.enyo.de> * Randy Bush: >> What does BCP38 have to do with this? > > nothing technical, as these iot attacks are not spoofed. How do you know? Has anyone disclosed specifics? I can understand that keeping details under wraps is sometimes required for operational security, but if the attacks are clearly succeeding, I would have expected those who posted ?do something, now!? messages at least some pointer to technical details of what was going on. Not that the underlying threat will go away until we find a way to clean up almost all of the compromised devices (and without breaking the Internet along the way, forever). From clinton.mielke at gmail.com Sun Oct 23 12:12:49 2016 From: clinton.mielke at gmail.com (clinton mielke) Date: Sun, 23 Oct 2016 05:12:49 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <874m43qsk2.fsf@mid.deneb.enyo.de> References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> <874m43qsk2.fsf@mid.deneb.enyo.de> Message-ID: A number of people are asking for advice on how to detect this bug. Here are some thoughts. Im a mathematician, and not a network operator, so would love feedback. The source code of Mirai is here, and Ive had some fun taking it apart over the last week: https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/ Notable findings: * Primary infection vector is via telnet scanning. Port 23 is literally hardcoded in. 10% of the time, it scans for port 2323. Found that odd, but I suppose one of the devices its targeting uses that port. * The malware disables any services running on ports 22, 23, and 80, primarily to prevent other infection opportunities. This surprises me, for I figured that killing port 80 might attract attention from the device owner, but evidently the risk of reinfection is too high to not do it. See line 88: https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/killer.c * The malware uses a large set of signatures to kill other bots running in memory, like QBot. I find this interesting. A script kiddie wont, but a more sophisticated adversary could add Mirai itself to this list of signatures to out compete the released variant of the code. You can see the library of signatures here : https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/table.c Digging around, I found that several samples of Mirai related malware have been uploaded and processed by the Indian honeynet's Linux sandbox. Heres a sample: https://detux.org/report.php?sha256=0b28b39f25c748b69369c18f72e937950826f189cd43227431384d34a0dce6fa >From the host connectivity log, you can see all of the port 23 scanning activity. The scanning is completely random, and not sequential, hopping all over the place. From a detection standpoint, that is where I would start, but this assumes that the hosts on your network are actively scanning and not lying dormant. This file, starting on line 124, has all of the hard-coded passwords that the malware uses to login to telnet sessions: https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/scanner.c - Googling around, you can find the make and model number that each of those user/password combinations are associated with. Brian compiled a list actually: https://krebsonsecurity.com/wp-content/uploads/2016/10/IoTbadpass-Sheet1.csv My question for you guys, since Im a theoretician and not a seasoned operator: how feasible or legal is it to find telnet scanning activity or any of these passwords in high-bandwidth netflows? If its feasible, then this at least gets you the active scanning population of hosts, along with the IPs of all of their victims. Aside from the active scanning population, finding dormant hosts might only be feasible if we know the list of C&Cs used, which can very widely. For Mirai in particular, the actual bot itself is delivered via tftp or wget from another dropper host. Take a look at this other sample for this kind of behavior. It connects to a webserver in the netherlands and pulls down the payload binary: https://detux.org/report.php?sha256=996167e00f2aef787c432ca4ce4613edf39c5f83363b269137aff3a3e75af5a9 I think its unlikely that skilled users of this malware would keep using the default 'mirai.arm7' payload, but evidently some are in the wild! Finding these http drops might help you find recent successful infections. More importantly however, the payload delivered itself will have information about the C&C, which if we as a community gather and analyze, we can find more easily the total set of dormant devices waiting to attack. Ultimately if you know the C&C being used, you can much more easily find the bots. Im going to pull apart the server code next. About time I learn GO... Lastly, studying this malware long enough, some techniques jump to mind which could hypothetically infect and patch a large number of vulnerable hosts. Im sure someone brave enough might do this. Totally worked out for Robert Morris. On Sun, Oct 23, 2016 at 3:16 AM, Florian Weimer wrote: > * Randy Bush: > > >> What does BCP38 have to do with this? > > > > nothing technical, as these iot attacks are not spoofed. > > How do you know? Has anyone disclosed specifics? > > I can understand that keeping details under wraps is sometimes > required for operational security, but if the attacks are clearly > succeeding, I would have expected those who posted ?do something, > now!? messages at least some pointer to technical details of what was > going on. > > Not that the underlying threat will go away until we find a way to > clean up almost all of the compromised devices (and without breaking > the Internet along the way, forever). > From mel at beckman.org Sun Oct 23 14:22:20 2016 From: mel at beckman.org (Mel Beckman) Date: Sun, 23 Oct 2016 14:22:20 +0000 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> <874m43qsk2.fsf@mid.deneb.enyo.de>, Message-ID: <3A2766EC-BFD6-4C03-8079-FF7AE3C0CFDD@beckman.org> Clinton, This is excellent information. While it's not possible to see passwords in netflows (only headers are included, not packet contents), it's a sure thing that attacked victims could extract a list of infected machines from the IP address scan and then run verification scans against just those devices. Any confirmed infected devices could then be published on a blacklist, a la spam blockers. Providers then could either blackhole (at the source) or filter those addresses. -mel > On Oct 23, 2016, at 5:20 AM, clinton mielke wrote: > > A number of people are asking for advice on how to detect this bug. Here > are some thoughts. Im a mathematician, and not a network operator, so would > love feedback. > > The source code of Mirai is here, and Ive had some fun taking it apart over > the last week: > https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/ > > Notable findings: > > * Primary infection vector is via telnet scanning. Port 23 is literally > hardcoded in. 10% of the time, it scans for port 2323. Found that odd, but > I suppose one of the devices its targeting uses that port. > > * The malware disables any services running on ports 22, 23, and 80, > primarily to prevent other infection opportunities. This surprises me, for > I figured that killing port 80 might attract attention from the device > owner, but evidently the risk of reinfection is too high to not do it. See > line 88: > https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/killer.c > > * The malware uses a large set of signatures to kill other bots running in > memory, like QBot. I find this interesting. A script kiddie wont, but a > more sophisticated adversary could add Mirai itself to this list of > signatures to out compete the released variant of the code. You can see the > library of signatures here : > https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/table.c > > Digging around, I found that several samples of Mirai related malware have > been uploaded and processed by the Indian honeynet's Linux sandbox. Heres a > sample: > https://detux.org/report.php?sha256=0b28b39f25c748b69369c18f72e937950826f189cd43227431384d34a0dce6fa > > From the host connectivity log, you can see all of the port 23 scanning > activity. The scanning is completely random, and not sequential, hopping > all over the place. From a detection standpoint, that is where I would > start, but this assumes that the hosts on your network are actively > scanning and not lying dormant. > > This file, starting on line 124, has all of the hard-coded passwords that > the malware uses to login to telnet sessions: > https://github.com/jgamblin/Mirai-Source-Code/blob/master/mirai/bot/scanner.c > - Googling around, you can find the make and model number that each of > those user/password combinations are associated with. Brian compiled a list > actually: > https://krebsonsecurity.com/wp-content/uploads/2016/10/IoTbadpass-Sheet1.csv > > My question for you guys, since Im a theoretician and not a seasoned > operator: how feasible or legal is it to find telnet scanning activity or > any of these passwords in high-bandwidth netflows? If its feasible, then > this at least gets you the active scanning population of hosts, along with > the IPs of all of their victims. > > Aside from the active scanning population, finding dormant hosts might only > be feasible if we know the list of C&Cs used, which can very widely. For > Mirai in particular, the actual bot itself is delivered via tftp or wget > from another dropper host. Take a look at this other sample for this kind > of behavior. It connects to a webserver in the netherlands and pulls down > the payload binary: > https://detux.org/report.php?sha256=996167e00f2aef787c432ca4ce4613edf39c5f83363b269137aff3a3e75af5a9 > > I think its unlikely that skilled users of this malware would keep using > the default 'mirai.arm7' payload, but evidently some are in the wild! > Finding these http drops might help you find recent successful infections. > More importantly however, the payload delivered itself will have > information about the C&C, which if we as a community gather and analyze, > we can find more easily the total set of dormant devices waiting to attack. > Ultimately if you know the C&C being used, you can much more easily find > the bots. > > Im going to pull apart the server code next. About time I learn GO... > > Lastly, studying this malware long enough, some techniques jump to mind > which could hypothetically infect and patch a large number of vulnerable > hosts. Im sure someone brave enough might do this. Totally worked out for > Robert Morris. > >> On Sun, Oct 23, 2016 at 3:16 AM, Florian Weimer wrote: >> >> * Randy Bush: >> >>>> What does BCP38 have to do with this? >>> >>> nothing technical, as these iot attacks are not spoofed. >> >> How do you know? Has anyone disclosed specifics? >> >> I can understand that keeping details under wraps is sometimes >> required for operational security, but if the attacks are clearly >> succeeding, I would have expected those who posted ?do something, >> now!? messages at least some pointer to technical details of what was >> going on. >> >> Not that the underlying threat will go away until we find a way to >> clean up almost all of the compromised devices (and without breaking >> the Internet along the way, forever). >> From victor at jvknet.com Sun Oct 23 14:34:50 2016 From: victor at jvknet.com (Victor Kuarsingh) Date: Sun, 23 Oct 2016 10:34:50 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <84973.1477096787@segfault.tristatelogic.com> <011337e2-a402-4d0a-3888-6e74854693ad@heliacal.net> <874m43qsk2.fsf@mid.deneb.enyo.de> Message-ID: Clinton, On 10/23/2016 8:12 AM, clinton mielke wrote: > > My question for you guys, since Im a theoretician and not a seasoned > operator: how feasible or legal is it to find telnet scanning activity or > any of these passwords in high-bandwidth netflows? If its feasible, then > this at least gets you the active scanning population of hosts, along with > the IPs of all of their victims. If there is enough concentration of common flows from a certain set of IPs, it's quite possible to detect the scanning activity using sampled flow data if one were collecting such data. I say sampled as 1-for-1 flow data collection is not common. You would not see packet content just using flow data. regards, Victor K From bzs at TheWorld.com Sun Oct 23 18:47:50 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 23 Oct 2016 14:47:50 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <580BD066.8050407@vaxination.ca> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> Message-ID: <22541.1494.767673.991935@gargle.gargle.HOWL> I think you make a very good point with the TRS80 etc comment, at least implicitly: it's not just the vulnerable IoT devices, some sort of infrastructure is needed to get the attack going at the volume we've seen. And perhaps therein lies an answer. On October 22, 2016 at 16:47 jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) wrote: > Generic question: > > The media seems to have concluded it was an "internet of things" that > caused this DDoS. > > I have not seen any evidence of this. Has this been published by an > authoritative source or is it just assumed? > > Has the type of device involved been identified? > > I am curious on how some hacker in basement with his TRS80 or Commodore > Pet would be able to reach "bilions" of these devices to reprogram them. > Vast majority of homes are behind NAT, which means that an incoming > packet has very little chance of reaching the IoT gizmo. > > I amn guessing/hoping such devices have been identified and some > homweoners contacted ans asked to volunteer their device for forensic > analysis of where the attack came from ? > > Is it more plausible that those devices were "hacked" in the OEM > firmware and sold with the "virus" built-in ? That would explain the > widespread attack. > > Also, in cases such as this one, while the target has managed to > mitigate the attack, how long would such an attack typically continue > and require blocking ? > > Since the attack seemed focused on eastern USA DNS servers, would it be > fair to assume that the attacks came mostly from the same region (aka: > devices installed in eastern USA) ? (since anycast would point them to > that). > > OPr did the attack use actual IP addresses instead of the unicast ones > to specifically target servers ? > > > > BTW, normally, if you change the "web" password on a "device", it would > also change telnet/SSH/ftp passwords. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Sun Oct 23 19:41:49 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 23 Oct 2016 15:41:49 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> Message-ID: <22541.4733.784386.999943@gargle.gargle.HOWL> >So once identified, how do you suggest this gets fixed? Assuming these manufacturers who are culpable carry product liability insurance go to their insurance companies and explain the situation. Better would be someone launching a product liability lawsuit against one of them but it's not necessary, ins cos work on projections and probabilities as much as being reactive. The insurance companies will likely re-assess their risk on these policies and inform the manufacturers of any adjustment in premiums. If the premiums are adjusted up significantly the manufacturers will sit down with the ins cos and try to determine what needs to be improved in their product to bring premiums back down. Look at what Samsung just went thru with the Note 7. I'd imagine their product liability insurance premiums took a big hit. Even if they're self-insured they have to treat that as a cost center and make sure sufficient money to pay claims is going into that cost center. It's a button to push, so to speak, and has been successful many times in the past (cars, worker exposure to health hazards, etc.) -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From deleskie at gmail.com Sun Oct 23 19:46:48 2016 From: deleskie at gmail.com (jim deleskie) Date: Sun, 23 Oct 2016 16:46:48 -0300 Subject: Death of the Internet, Film at 11 In-Reply-To: <22541.4733.784386.999943@gargle.gargle.HOWL> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: Sure lets sue people because they put too many/bad packets/packets I don't like on the internet. Do you think this will really solve the porblem? Do you think we'll not just all end up with internet prices like US medical care prices? On Sun, Oct 23, 2016 at 4:41 PM, wrote: > > >So once identified, how do you suggest this gets fixed? > > Assuming these manufacturers who are culpable carry product liability > insurance go to their insurance companies and explain the situation. > > Better would be someone launching a product liability lawsuit against > one of them but it's not necessary, ins cos work on projections and > probabilities as much as being reactive. > > The insurance companies will likely re-assess their risk on these > policies and inform the manufacturers of any adjustment in premiums. > > If the premiums are adjusted up significantly the manufacturers will > sit down with the ins cos and try to determine what needs to be > improved in their product to bring premiums back down. > > Look at what Samsung just went thru with the Note 7. I'd imagine their > product liability insurance premiums took a big hit. Even if they're > self-insured they have to treat that as a cost center and make sure > sufficient money to pay claims is going into that cost center. > > It's a button to push, so to speak, and has been successful many times > in the past (cars, worker exposure to health hazards, etc.) > > -- > -Barry Shein > > Software Tool & Die | bzs at TheWorld.com | > http://www.TheWorld.com > Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD > The World: Since 1989 | A Public Information Utility | *oo* > From bzs at TheWorld.com Sun Oct 23 20:26:17 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 23 Oct 2016 16:26:17 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: <22541.7401.319526.253879@gargle.gargle.HOWL> I'm not sure who you mean when you say "people". My reference was to manufacturers of IoT devices only. But as I said in the note which you quoted lawsuits might be helpful but aren't necessary. One just has to get underwriters of the manufacturers' product liability insurance to acknowledge they have not fully assessed future financial risks of those policies. Since manufacturers probably won't like those new increased premiums, and it's within their control to improve the potential product liability and get their insurance premiums lowered, they would likely respond by improving their product (i.e.. security.) Put in simple terms if your auto insurance company told you your annual premiums are going up because your tires are bad (whatever) you might consider getting new tires particularly if the net cost (increased premiums year after year vs cost of tires) was positive to you. Please for the love of all that is sane and reasonable don't quibble about tires and insurance. If you have a pile of fire-prone materials near your house, no railings on steps, whatever, you're likely to prioritize fixing that if your premiums go up sufficiently. Since manufacturers have huge multipliers (number of devices, number of potential liability claims) this sort of approach can and has been effective. On October 23, 2016 at 16:46 deleskie at gmail.com (jim deleskie) wrote: > Sure lets sue people because they put too many/bad packets/packets I don't > like on the internet. Do you think this will really solve the porblem? Do > you think we'll not just all end up with internet prices like US medical > care prices? > > On Sun, Oct 23, 2016 at 4:41 PM, wrote: > > > > > >So once identified, how do you suggest this gets fixed? > > > > Assuming these manufacturers who are culpable carry product liability > > insurance go to their insurance companies and explain the situation. > > > > Better would be someone launching a product liability lawsuit against > > one of them but it's not necessary, ins cos work on projections and > > probabilities as much as being reactive. > > > > The insurance companies will likely re-assess their risk on these > > policies and inform the manufacturers of any adjustment in premiums. > > > > If the premiums are adjusted up significantly the manufacturers will > > sit down with the ins cos and try to determine what needs to be > > improved in their product to bring premiums back down. > > > > Look at what Samsung just went thru with the Note 7. I'd imagine their > > product liability insurance premiums took a big hit. Even if they're > > self-insured they have to treat that as a cost center and make sure > > sufficient money to pay claims is going into that cost center. > > > > It's a button to push, so to speak, and has been successful many times > > in the past (cars, worker exposure to health hazards, etc.) > > > > -- > > -Barry Shein > > > > Software Tool & Die | bzs at TheWorld.com | > > http://www.TheWorld.com > > Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD > > The World: Since 1989 | A Public Information Utility | *oo* > > -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From hannigan at gmail.com Sun Oct 23 21:14:38 2016 From: hannigan at gmail.com (Martin Hannigan) Date: Sun, 23 Oct 2016 17:14:38 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <22541.7401.319526.253879@gargle.gargle.HOWL> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> <22541.7401.319526.253879@gargle.gargle.HOWL> Message-ID: <374EB470-A1C2-4D0E-A069-91D006E2DF67@gmail.com> > On Oct 23, 2016, at 16:26, bzs at TheWorld.com wrote: > > > I'm not sure who you mean when you say "people". My reference was to > manufacturers of IoT devices only. The users are not going to be able to help. You're right, it's all about the manufacturers. If you can remove or reduce profits enough where it matters, it will help tremendously. I spent an hour looking through the IEEE standards RA pattern searching mac addrs thinking about mitigation techniques and doing random lookups of the registrants. These attacks are the canary in the coal mine in terms of what is probably coming. Best, -M< From jfmezei_nanog at vaxination.ca Sun Oct 23 21:30:11 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sun, 23 Oct 2016 17:30:11 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: <580D2BE3.3030204@vaxination.ca> On 2016-10-23 15:46, jim deleskie wrote: > Sure lets sue people because they put too many/bad packets/packets I don't > like on the internet. Do you think this will really solve the porblem? Do > you think we'll not just all end up with internet prices like US medical > care prices? If this were to get to a court of law, would there be proof that products Axis IP Camera Inc or Panasonic or even Xerox Printers were (partly) responsible for the attack ? Won't they deflect this to trying to find those who hacked their products ? Won't they deflect this to onwers who did not secure their networks from inbound telnet ? And do those units really declare their port 23 to the NAT router via UPnP ? that is really really stupid. One problem with consumer goods is lack of documentation and support. Could years back, I got a very early Smart RG DSL modem specially modified to work on Bell Canada's non standard VDSL dslams. No instruction manual, no documentation. I found a number of bugs in the software, and sent a lengthy email to document them. As an early adopter, I wanted to help the company fix those before wider deployment. (and yes, the units have a command line, and from the command line you can get into a linux shell). The response I got: Unless you sign a contract with one of our distributors, you cannot report bugs. Unfortunately, this appears to be widespread with consumer goods vendors who sell sophisticated devices without documentation or support. From rfg at tristatelogic.com Sun Oct 23 22:05:20 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sun, 23 Oct 2016 15:05:20 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: Message-ID: <97016.1477260320@segfault.tristatelogic.com> In message Josh Reynolds wrote: >And then what? The labor to clean up this mess is not free... >... >The ISPs won't do it because of the cost to fix... The labor and potential >loss of customers. Yes, and yes. Unfortunately, the economics of the current situation are rather clearly and rather sadly broken. And I feel sure that the same "cost" arguments were also advanced, in the 1970s, against the Clean Air Act and the Clean Water Act. More recently, I'm also fairly sure that banks have pushed back strongly against anti money-laundering regulations, based on similar or identical "cost and loss of customers" arguments. Nonetheless, government regulation in these areas has advanced, and has resulted in a salutary leveling of the playing field. All players in the affected industries must comply, and thus none can undercut the others by reducing their costs, in a relentless race to the bottom, by simply shirking their social responsibilities. And since all players across an industry must bear the same costs, all should find it equally possible to pass along these costs their respective customer bases. (This answers the question of who is going to pay to clean up this whole mess we call the Internet.) To those who would advance the argument that government regulation simply will not work and/or that such is not actually possible on a globally dispersed Internet, I would only note that essentially the same concerns, issues and arguments apply equally to the globally interconnected banking system, and that although there still remain major challanges, mostly now isolated to a few specific locales, the global fight against money laundering has made impressive advances in recent years, and continues to make steady progress. I cite this fact simply to point out that globally interconnected industries are not inherently immune to prudent cross-border regulation in the interests of the common good. Given the Internet industry's abject, long-standing, and ongoing near total abdication of any resposibility for even a modicum of self-regulation, I, for one, look forward to the Clean Internet Act, whenever that may arrive. Regards, Ronald F. Guilmette (DDoS'd off the Internet, to little or no public fanfare, 2003) From rfg at tristatelogic.com Sun Oct 23 22:23:48 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sun, 23 Oct 2016 15:23:48 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <580BF49C.5090209@vaxination.ca> Message-ID: <97128.1477261428@segfault.tristatelogic.com> In message <580BF49C.5090209 at vaxination.ca>, Jean-Francois Mezei wrote: >10s of millons of IP addresses. Is it realistic to have 10s of millions >of infected devices ? Or is that the dense smoke that points to IP >spoofing ? I haven't read the latest up-to-the-minute reports on this event, but I do suspect that Dyn knows the difference between UDP and TCP, and my understanding is that the latter is a wee bit difficult to spoof these days. Not impossible, perhaps, but quite tedious. I don't think that Dyn would have come out and said "10 million" if it was all easily spoofable UDP that they were getting. In that case, they would either have said "we got a ton of spoofed traffic" or else, if they felt like being publically lampooned, they would have estimated the number of attacking IPs at three billion. So, bottom line, if Dyn said "10 million" I suspect that they intended to imply also "via TCP". Regards, rfg From marka at isc.org Sun Oct 23 22:42:58 2016 From: marka at isc.org (Mark Andrews) Date: Mon, 24 Oct 2016 09:42:58 +1100 Subject: Dyn DDoS this AM? In-Reply-To: Your message of "Sat, 22 Oct 2016 13:22:45 +1100." References: <580ABCF7.1030003@vaxination.ca> Message-ID: <20161023224258.3569F576E7D7@rock.dv.isc.org> In message , Masood Ahmad Shah writes: > > > > > On Oct 21, 2016, at 6:35 PM, Eitan Adler wrote: > > > > > > [...] > > > > > > In practice TTLs tend to be ignored on the public internet. In past > > > research I've been involved with browser[0] behavior was effectively > > > random despite the TTL set. > > > > > > [0] more specifically, the chain of DNS resolution and caching down to > > > the browser. > > > > > > Yes, but that it can be both better and worse than your TTLs does not > > mean that you can ignore properly working implementations. > > > > If the other end device chain breaks you that's their fault and out of > > your control. If your own settings break you that's your fault. > > > > +1 to what George wrote that we should make efforts to improve our part of > the network. There are ISPs that ignore TTL settings and only update their > cached records every two to three days or even more (particularly the > smaller ones). OTOH, this results in your DNS data being inconsistent but > it???s very common to cache DNS records at multiple levels. It's an effort > that everyone needs to contribute to. For TTL there is a tension between being able to update with new data and resiliance when servers are unreachable. For zone transfers we have 3 timers refesh, retry and expire to deal with this tension. If we were doing DNS from scratch there would be at least two ttl values one for freshness and one for don't use past. Additionally a lot of the need for small TTL's is because clients don't fail over to second addresses in a reasonable amount of time. There is no reason for this other than poorly designed clients. A client can failover using sub-second timers. We do this for Happy Eyeballs. This strategy is viable for ALL connection attempts. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From rfg at tristatelogic.com Sun Oct 23 22:44:35 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sun, 23 Oct 2016 15:44:35 -0700 Subject: FW: Death of the Internet, Film at 11 In-Reply-To: <580BF91D.9060902@vaxination.ca> Message-ID: <97231.1477262675@segfault.tristatelogic.com> In message <580BF91D.9060902 at vaxination.ca>, Jean-Francois Mezei wrote: >Problem is that many of these gadgets want to be internet connected so >mother at work can check on her kids at home... Ah, technology! Just think what certain people could have accomplished if they had only lived long enough to enjoy the fruits of our marvelous technological advances. https://en.wikipedia.org/wiki/Richard_Hauptmann And the good news, of course, is that with IPv6 everybody will have their own set of 4 billion addresses, so all those babycam/kidcams won't even have to be sitting behind NAT SOHO routers anymore. Password? What password? Regards, rfg From rfg at tristatelogic.com Sun Oct 23 23:19:18 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sun, 23 Oct 2016 16:19:18 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <26b01962-9b09-11cb-0ac8-89cf3e0a5f96@nuclearfallout.net> Message-ID: <97353.1477264758@segfault.tristatelogic.com> In message <26b01962-9b09-11cb-0ac8-89cf3e0a5f96 at nuclearfallout.net>, John Weekes wrote: >... I've recorded >about 2.4 million IP addresses involved in the last two months (a number >that is higher than the number of actual devices, since most seem to >have dynamic IP addresses). The ISPs behind those IP addresses have >received notifications via email... Just curious... How well is that working out? I've tried this myself a few times in the past, when I've found things that appear to be seriously compromised, and for my extensive trouble I've mostly received back utter silence and no action. I remember that after properly notifying security@ some large end-luser cable network in the SouthEast (which shall remain nameless) I got back something along the lines of "Thank you. We'll look into it." and was disgusted to find, two months later, that the boxes in question were still utterly pwned and in the exact same state they were two months prior, when I had first reported them. I guess that's just an example of what somebody else already noted here, i.e. that providers don't care to spend the time and/or effort and/or money necessary to actually -do- anything about compromised boxes, and anyway, they don't want to lose a paying customer. So, you know, let's just say for the sake of argument that right now, today, I know about a botnet consiting of a quarter million popped boxes, and that I have in-hand all of the relevant IPs, and that I have no trouble finding contact email addresses for all of the relevant ASNs. So then what? The question is: Why should I waste my time informing all, or even any of these ASNs about the popped boxes on their networks when (a) I am not their customer... as many of them have been only too happy to gleefully inform me in the past... and when (b) the vast majority simply won't do anything with the information? And while we are on the subject, I just have to bring up one of my biggest pet peeves. Why is it that every time some public-spirited altrusitc well-meaning citizen such as myself reports any kind of a problem to any kind of a company on the Internet, the report itself gets immediately labeled and categorized as a "complaint". If I spend some of -my- valuable time to helpfully try to let somebody else know of a problem on their network, or with their web site, and if that report gets categorized as a "complaint" then what does that make me? A "complainer"?? I don't need this kind of abuse and denegration from people who I'm trying to help. Like most other people, if I am in need of some personal denegration and abuse... well... I have relatives for that. Regards, rfg From rfg at tristatelogic.com Sun Oct 23 23:34:28 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sun, 23 Oct 2016 16:34:28 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <874m43qsk2.fsf@mid.deneb.enyo.de> Message-ID: <97423.1477265668@segfault.tristatelogic.com> In message <874m43qsk2.fsf at mid.deneb.enyo.de>, Florian Weimer wrote: >Not that the underlying threat will go away until we find a way to >clean up almost all of the compromised devices (and without breaking >the Internet along the way, forever). The Internet *is* already broken. After the attack on Krebs, the terabit+ attack on OVH, and the events of Friday, if there are still some people who fail to grasp this fundamental point, then it can only be because some folks have become really adept at living in denial. Regards, rfg From bzs at TheWorld.com Mon Oct 24 00:52:35 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 23 Oct 2016 20:52:35 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <374EB470-A1C2-4D0E-A069-91D006E2DF67@gmail.com> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> <22541.7401.319526.253879@gargle.gargle.HOWL> <374EB470-A1C2-4D0E-A069-91D006E2DF67@gmail.com> Message-ID: <22541.23379.673429.73050@gargle.gargle.HOWL> On October 23, 2016 at 17:14 hannigan at gmail.com (Martin Hannigan) wrote: > > > >On Oct 23, 2016, at 16:26, bzs at TheWorld.com wrote: > > > > I'm not sure who you mean when you say "people". My reference was to > manufacturers of IoT devices only. > > >The users are not going to be able to help. You're right, it's all about the >manufacturers. If you can remove or reduce profits enough where it matters, it >will help tremendously. > >I spent an hour looking through the IEEE standards RA pattern searching mac >addrs thinking about mitigation techniques and doing random lookups of the >registrants. That's a good idea particularly in terms of not letting this stuff out. For example one could imagine a patch to DSL, cable, and similar last mile equipment to rate limit, perhaps flag etc, packets from known vulnerable MAC ID ranges if they can be identified. That'd be relatively cheap and easy. >These attacks are the canary in the coal mine in terms of what is probably >coming. Oh yeah...that code is out there. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From kmedcalf at dessus.com Mon Oct 24 01:39:52 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Sun, 23 Oct 2016 19:39:52 -0600 Subject: Death of the Internet, Film at 11 Message-ID: Why would the provider want to do anything? ?They suuport (make money from) their cudtomers. ?And the more traffic the send/receive, the more money the providers make. Wouldn't surprise me if the providers were selling access to their customers networks to the botherders so they could make money from both ends. --- Sent from Samsung Mobile
-------- Original message --------
From: "Ronald F. Guilmette"
Date:2016-10-23 17:20 (GMT-07:00)
To:
Cc: nanog at nanog.org
Subject: Re: Death of the Internet, Film at 11
From nanog at ics-il.net Mon Oct 24 01:43:19 2016 From: nanog at ics-il.net (Mike Hammett) Date: Sun, 23 Oct 2016 20:43:19 -0500 (CDT) Subject: Death of the Internet, Film at 11 In-Reply-To: References: Message-ID: <905165322.13.1477273394559.JavaMail.mhammett@ThunderFuck> A support call to an end-user serving ISP takes how long to ROI? That wouldn't make sense. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Keith Medcalf" To: "Ronald F. Guilmette" Cc: "NANOG" Sent: Sunday, October 23, 2016 8:39:52 PM Subject: Re: Death of the Internet, Film at 11 Why would the provider want to do anything? They suuport (make money from) their cudtomers. And the more traffic the send/receive, the more money the providers make. Wouldn't surprise me if the providers were selling access to their customers networks to the botherders so they could make money from both ends. --- Sent from Samsung Mobile
-------- Original message --------
From: "Ronald F. Guilmette"
Date:2016-10-23 17:20 (GMT-07:00)
To:
Cc: nanog at nanog.org
Subject: Re: Death of the Internet, Film at 11
From deleskie at gmail.com Mon Oct 24 01:47:15 2016 From: deleskie at gmail.com (jim deleskie) Date: Sun, 23 Oct 2016 22:47:15 -0300 Subject: Death of the Internet, Film at 11 In-Reply-To: References: Message-ID: I've heard this crap for 20+ years now. "attack traffic" is unplanned traffic. Build networks to support "random" bursts of garbage is much more expensive then you will ever get to bill for. You clearly have no understanding of the economics of networks. On Sun, Oct 23, 2016 at 10:39 PM, Keith Medcalf wrote: > Why would the provider want to do anything? They suuport (make money > from) their cudtomers. And the more traffic the send/receive, the more > money the providers make. > > Wouldn't surprise me if the providers were selling access to their > customers networks to the botherders so they could make money from both > ends. > > > > --- > Sent from Samsung Mobile > > > >
-------- Original message --------
From: "Ronald F. > Guilmette"
Date:2016-10-23 17:20 > (GMT-07:00)
To:
Cc: nanog at nanog.org >
Subject: Re: Death of the Internet, Film at 11
>
From list at satchell.net Mon Oct 24 01:51:20 2016 From: list at satchell.net (Stephen Satchell) Date: Sun, 23 Oct 2016 18:51:20 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <97353.1477264758@segfault.tristatelogic.com> References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: <68416db2-96f8-9d86-7b51-78fede6bfd07@satchell.net> On 10/23/2016 04:19 PM, Ronald F. Guilmette wrote: > I guess that's just an example of what somebody else already noted here, > i.e. that providers don't care to spend the time and/or effort and/or > money necessary to actually -do- anything about compromised boxes, and > anyway, they don't want to lose a paying customer. That was a lesson well-learned in the e-mail community, that the majority of providers don't care as long as the actions of "target" admins amount to nothing. Hell, they used to spam themselves! That's why the DNSBLs became so popular. So, bottom line, nothing is going to happen until the cost to those negligent provides rises so high as to affect profits. Period. Larger eyeball operators could help, by shutting down entire subnets infested with botted computers and things. From drc at virtualized.org Mon Oct 24 02:02:04 2016 From: drc at virtualized.org (David Conrad) Date: Sun, 23 Oct 2016 19:02:04 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <68416db2-96f8-9d86-7b51-78fede6bfd07@satchell.net> References: <97353.1477264758@segfault.tristatelogic.com> <68416db2-96f8-9d86-7b51-78fede6bfd07@satchell.net> Message-ID: On October 23, 2016 at 6:52:05 PM, Stephen Satchell (list at satchell.net) wrote: So, bottom line, nothing is going to happen until the cost to those? negligent provides rises so high as to affect profits. Period.? Yep. ?Or government intervention. Larger eyeball operators could help, by shutting down entire subnets? infested with botted computers and things.? Shut down subnets of your own customers?? Regards, -drc -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Message signed with OpenPGP using AMPGpg URL: From list at satchell.net Mon Oct 24 02:04:41 2016 From: list at satchell.net (Stephen Satchell) Date: Sun, 23 Oct 2016 19:04:41 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> <68416db2-96f8-9d86-7b51-78fede6bfd07@satchell.net> Message-ID: On 10/23/2016 07:02 PM, David Conrad wrote: > On October 23, 2016 at 6:52:05 PM, Stephen Satchell (list at satchell.net) wrote: > So, bottom line, nothing is going to happen until the cost to those > negligent provides rises so high as to affect profits. Period. > Yep. Or government intervention. > > Larger eyeball operators could help, by shutting down entire subnets > infested with botted computers and things. > > Shut down subnets of your own customers? I was thinking NetFlix, the search engines, Amazon, Facebook, and twitter, to name a few. From larrysheldon at cox.net Mon Oct 24 02:26:20 2016 From: larrysheldon at cox.net (Larry Sheldon) Date: Sun, 23 Oct 2016 21:26:20 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> <68416db2-96f8-9d86-7b51-78fede6bfd07@satchell.net> Message-ID: On 10/23/2016 21:02, David Conrad wrote: > Shut down subnets of your own customers? That was the problem I broke my pick on 20 years or more ago. ISPs absolute refusal to put in filters at no-revenue-expense since it would cost money to install and maintain, and worst of all MIGHT conceivably block revenue-producing-abuse traffic. No matter that paying customers were not able to use the service they were paying for. -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From aaron at heyaaron.com Mon Oct 24 04:04:03 2016 From: aaron at heyaaron.com (Aaron C. de Bruyn) Date: Sun, 23 Oct 2016 21:04:03 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <22541.4733.784386.999943@gargle.gargle.HOWL> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: On Sun, Oct 23, 2016 at 12:41 PM, wrote: > > Assuming these manufacturers who are culpable carry product liability > insurance go to their insurance companies and explain the situation. Cheaper solution: Start a company, build crappy firmware, carry product liability insurance, release the product, immediately sell millions of units to various vendors that 'rebrand' your product. Close your business / go out of business. Wait for lawsuits to roll in after the business has been shut down. -A From jfmezei_nanog at vaxination.ca Mon Oct 24 04:23:47 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Mon, 24 Oct 2016 00:23:47 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: <580D8CD3.90203@vaxination.ca> A bit tidbits of information from: > http://www.networkworld.com/article/3134035/chinese-firm-admits-its-hacked-products-were-behind-fridays-massive-ddos-attack.html Chinese firm admits its hacked products were behind Friday's massive DDOS attack Hangzhou Xiongmai Technology, a vendor behind DVRs and internet-connected cameras, said on Sunday that security vulnerabilities involving weak default passwords in its products were partly to blame. ... Because these devices have weak default passwords and are easy to infect, Mirai has been found spreading to at least 500,000 devices, according to internet backbone provider Level 3 Communications. ... Xiongmai says it patched the flaws with its products in September 2015 and its devices now ask the customer to change the default password when used for the first time. But products running older versions of the firmware are still vulnerable. To stop the Mirai malware, Xiongmai is advising that customers update their product?s firmware and change the default username and passwords to them. Customers can also disconnect the products from the internet. ## Note: the company's web site does not (yet) show a press release. Appears the information was sent to IDG via email. From esr at thyrsus.com Mon Oct 24 05:28:08 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Mon, 24 Oct 2016 01:28:08 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <20161022220926.68FD3576BA95@rock.dv.isc.org> <22541.4733.784386.999943@gargle.gargle.HOWL> Message-ID: <20161024052808.GA13925@thyrsus.com> Aaron C. de Bruyn via NANOG : > On Sun, Oct 23, 2016 at 12:41 PM, wrote: > > > > Assuming these manufacturers who are culpable carry product liability > > insurance go to their insurance companies and explain the situation. > > Cheaper solution: Start a company, build crappy firmware, carry > product liability insurance, release the product, immediately sell > millions of units to various vendors that 'rebrand' your product. > Close your business / go out of business. Wait for lawsuits to roll > in after the business has been shut down. > > -A For anyone who thinks this is a hypothetical, the market for consumer-grade GPSes already works this way -- though, not for liability reasons in quite the same sense. The issue in GPS-land is blocking patents and other IP. Fly-by-night GPS vendors with 60-to-90-day life cycles keep a lot of Shenzhen shops busy. -- Eric S. Raymond From jw at nuclearfallout.net Mon Oct 24 05:56:02 2016 From: jw at nuclearfallout.net (John Weekes) Date: Sun, 23 Oct 2016 22:56:02 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <97353.1477264758@segfault.tristatelogic.com> References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > >> ... I've recorded >> about 2.4 million IP addresses involved in the last two months (a number >> that is higher than the number of actual devices, since most seem to >> have dynamic IP addresses). The ISPs behind those IP addresses have >> received notifications via email... > Just curious... How well is that working out? For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better). For other botnets, such as those using compromised webservers running outdated phpMyAdmin installs at random hosts, harnessing spun-up services at reputable VPS providers (Amazon, Microsoft, Rackspace, etc.), or harnessing devices at large and small US and Canadian ISPs, we have had better luck. Usually, we don't hear a response back, but those emails are often forwarded to the end-user, who takes action (and may ask us for help, which is how we know they are being forwarded). The fixes can enough to reduce attack volumes to more manageable levels. Kudos go out to the large and small ISPs and NSPs who have started policing SSDP and other reflection traffic, which we also send out some notifications for. In some cases, it may be that our emails spurred them to notice how much damage those attacks were doing and how much it was costing them to carry the attack traffic. > I've tried this myself a few times in the past, when I've found things > that appear to be seriously compromised, and for my extensive trouble > I've mostly received back utter silence and no action. I remember that > after properly notifying security@ some large end-luser cable network > in the SouthEast (which shall remain nameless) I got back something > along the lines of "Thank you. We'll look into it." and was disgusted > to find, two months later, that the boxes in question were still utterly > pwned and in the exact same state they were two months prior, when I > had first reported them. We do get our share of that, as well, unfortunately, along with our share of people who send angry responses calling the notifications spam (I disagree with them that sending a legitimate abuse notification to a publicly-posted, designated abuse account should be considered spam) or who flame us for acting like "internet police". But, we persist. Some people change their minds after receiving multiple notifications or after we explain that DoS traffic costs them money and hurts their customers, who will be experiencing degraded service and may silently switch providers over it. > I guess that's just an example of what somebody else already noted here, > i.e. that providers don't care to spend the time and/or effort and/or > money necessary to actually -do- anything about compromised boxes, and > anyway, they don't want to lose a paying customer. > > So, you know, let's just say for the sake of argument that right now, > today, I know about a botnet consiting of a quarter million popped > boxes, and that I have in-hand all of the relevant IPs, and that I > have no trouble finding contact email addresses for all of the relevant > ASNs. So then what? I use scripts to send out an abuse notification to some percentage of the compromised hosts -- the ones sending some significant amount of the traffic. The notification includes a description of what we saw and timestamped example attack traffic, as interpreted by tcpdump. If further traffic is seen later from the same host, another notification will be sent, after a cool-off period. The emails are plain text and we don't try to use them as advertisement. We also don't force a link to be clicked to see more details or to respond back. I don't like to receive such emails myself and have found that those types are more likely to be ignored. > The question is: Why should I waste my time informing all, or even any > of these ASNs about the popped boxes on their networks when (a) I am > not their customer... as many of them have been only too happy to > gleefully inform me in the past... and when (b) the vast majority > simply won't do anything with the information? I'm not saying that everyone should send abuse notifications like we do, since it can be a big task. But, in response to someone wondering if their network is being used for attacks, or asking how they could help to police their own network, I am saying that making sure that inbound abuse notifications are arriving at the right place and being handled appropriately is important. > And while we are on the subject, I just have to bring up one of my > biggest pet peeves. Why is it that every time some public-spirited > altrusitc well-meaning citizen such as myself reports any kind of a > problem to any kind of a company on the Internet, the report itself > gets immediately labeled and categorized as a "complaint". If I spend > some of -my- valuable time to helpfully try to let somebody else know > of a problem on their network, or with their web site, and if that > report gets categorized as a "complaint" then what does that make me? > A "complainer"?? > > I don't need this kind of abuse and denegration from people who I'm > trying to help. Like most other people, if I am in need of some > personal denegration and abuse... well... I have relatives for that. There's a spectrum of people responding to these and some percentage are just jerks, as in real life. But, I like to think that the majority of at least NA providers are represented by professionals who just don't respond out of courtesy because they don't want to flood our inboxes with simple acknowledgements. Those of us experiencing these attacks appreciate the community support, both from people like you who also send notifications and those who handle the notifications on the receiving end. -John From holbor at sonss.net Mon Oct 24 06:23:00 2016 From: holbor at sonss.net (Richard Holbo) Date: Sun, 23 Oct 2016 23:23:00 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: I run/manage the networks for several smallish (in the thousands of customers) eyeball ISP's and I appreciate a nice "hey you've got a bot" or "someone is scanning" me notice to my abuse emails. They are useful in identifying crap that's going on, so for those of you who have the resources to do that... I appreciate it, we do read them at my networks and try to do something. That said... getting end users to actually fix the broken routers etc. etc. is NOT easy. Very often we'll notify customers, they will _take their stuff to the local computer repair guy_ ... or office depo.... and they will run whatever auto scan they have and say it's all fine. Customer puts it back in, it's still broke, and they call customer support and want us to pay for the trip because _their_ expert says it's fine... IMHO since the advent of Net Neutrality... I cannot simply block all of X, Y or Z at my edge and tell the customers it's for the best. I'd love to block some stuff in and outbound to customers, but then the customer just yells at us and files complaints with the PUC because _they have a right to it_.. So those of you calling for Government interference... we've already done that and it does not help. /rh On Sun, Oct 23, 2016 at 10:56 PM, John Weekes wrote: > On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > >> >> ... I've recorded >>> about 2.4 million IP addresses involved in the last two months (a number >>> that is higher than the number of actual devices, since most seem to >>> have dynamic IP addresses). The ISPs behind those IP addresses have >>> received notifications via email... >>> >> Just curious... How well is that working out? >> > > For the IoT botnets, most of the emails are ignored or rejected, because > most go to providers who either quietly bitbucket them or flat-out reject > all abuse emails. Most emails sent to mainland China, for instance, are in > that category (Hong Kong ISPs are somewhat better). > > For other botnets, such as those using compromised webservers running > outdated phpMyAdmin installs at random hosts, harnessing spun-up services > at reputable VPS providers (Amazon, Microsoft, Rackspace, etc.), or > harnessing devices at large and small US and Canadian ISPs, we have had > better luck. Usually, we don't hear a response back, but those emails are > often forwarded to the end-user, who takes action (and may ask us for help, > which is how we know they are being forwarded). The fixes can enough to > reduce attack volumes to more manageable levels. > > Kudos go out to the large and small ISPs and NSPs who have started > policing SSDP and other reflection traffic, which we also send out some > notifications for. In some cases, it may be that our emails spurred them to > notice how much damage those attacks were doing and how much it was costing > them to carry the attack traffic. > > I've tried this myself a few times in the past, when I've found things >> that appear to be seriously compromised, and for my extensive trouble >> I've mostly received back utter silence and no action. I remember that >> after properly notifying security@ some large end-luser cable network >> in the SouthEast (which shall remain nameless) I got back something >> along the lines of "Thank you. We'll look into it." and was disgusted >> to find, two months later, that the boxes in question were still utterly >> pwned and in the exact same state they were two months prior, when I >> had first reported them. >> > > We do get our share of that, as well, unfortunately, along with our share > of people who send angry responses calling the notifications spam (I > disagree with them that sending a legitimate abuse notification to a > publicly-posted, designated abuse account should be considered spam) or who > flame us for acting like "internet police". But, we persist. Some people > change their minds after receiving multiple notifications or after we > explain that DoS traffic costs them money and hurts their customers, who > will be experiencing degraded service and may silently switch providers > over it. > > I guess that's just an example of what somebody else already noted here, >> i.e. that providers don't care to spend the time and/or effort and/or >> money necessary to actually -do- anything about compromised boxes, and >> anyway, they don't want to lose a paying customer. >> >> So, you know, let's just say for the sake of argument that right now, >> today, I know about a botnet consiting of a quarter million popped >> boxes, and that I have in-hand all of the relevant IPs, and that I >> have no trouble finding contact email addresses for all of the relevant >> ASNs. So then what? >> > > I use scripts to send out an abuse notification to some percentage of the > compromised hosts -- the ones sending some significant amount of the > traffic. The notification includes a description of what we saw and > timestamped example attack traffic, as interpreted by tcpdump. If further > traffic is seen later from the same host, another notification will be > sent, after a cool-off period. > > The emails are plain text and we don't try to use them as advertisement. > We also don't force a link to be clicked to see more details or to respond > back. I don't like to receive such emails myself and have found that those > types are more likely to be ignored. > > The question is: Why should I waste my time informing all, or even any >> of these ASNs about the popped boxes on their networks when (a) I am >> not their customer... as many of them have been only too happy to >> gleefully inform me in the past... and when (b) the vast majority >> simply won't do anything with the information? >> > > I'm not saying that everyone should send abuse notifications like we do, > since it can be a big task. But, in response to someone wondering if their > network is being used for attacks, or asking how they could help to police > their own network, I am saying that making sure that inbound abuse > notifications are arriving at the right place and being handled > appropriately is important. > > And while we are on the subject, I just have to bring up one of my >> biggest pet peeves. Why is it that every time some public-spirited >> altrusitc well-meaning citizen such as myself reports any kind of a >> problem to any kind of a company on the Internet, the report itself >> gets immediately labeled and categorized as a "complaint". If I spend >> some of -my- valuable time to helpfully try to let somebody else know >> of a problem on their network, or with their web site, and if that >> report gets categorized as a "complaint" then what does that make me? >> A "complainer"?? >> >> I don't need this kind of abuse and denegration from people who I'm >> trying to help. Like most other people, if I am in need of some >> personal denegration and abuse... well... I have relatives for that. >> > > There's a spectrum of people responding to these and some percentage are > just jerks, as in real life. But, I like to think that the majority of at > least NA providers are represented by professionals who just don't respond > out of courtesy because they don't want to flood our inboxes with simple > acknowledgements. > > Those of us experiencing these attacks appreciate the community support, > both from people like you who also send notifications and those who handle > the notifications on the receiving end. > > -John > From Valdis.Kletnieks at vt.edu Mon Oct 24 06:29:02 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Mon, 24 Oct 2016 02:29:02 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <580BF49C.5090209@vaxination.ca> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> <20161022223550.GA12610@esri.com> <580BF49C.5090209@vaxination.ca> Message-ID: <103874.1477290542@turing-police.cc.vt.edu> On Sat, 22 Oct 2016 19:22:04 -0400, Jean-Francois Mezei said: > 10s of millons of IP addresses. Is it realistic to have 10s of millions > of infected devices ? Or is that the dense smoke that points to IP > spoofing ? A few years ago, Vint Cerf gave a keynote speech at a conference, where he claimed that there were 140 million pwned devices on the Internet - and this was before IoT was itself a thing. Not one person in the security industry called bullshit and said the number was too high. There were however a lot of people who thought Cerf had significantly lowballed the estimate. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From jfmezei_nanog at vaxination.ca Mon Oct 24 06:32:31 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Mon, 24 Oct 2016 02:32:31 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: <580DAAFF.6040407@vaxination.ca> Question: For something like Mirai and others, there appears to be a timer that starts the attack at a certain day/time (with unknown amount of time to distribute the software to any/all infectable devices prior to attack). Do these generally have a timer to also stop the attack and go dormant awaiting instructions from its master ? or do they continue to send those packets forever ? If the attack is made using perfectly formed, legitimate DNS packlets (or HTTP requests or whetever), can temporary mitigation measures continue forever even if they block legitimate requests ? Or is it general practioce for hackers to have short duration attacks to reduce the time available to track them down ? (similar to old movies where one had to hangup before the 2 minutes it took for police to trace a phone call). From large.hadron.collider at gmx.com Mon Oct 24 08:28:10 2016 From: large.hadron.collider at gmx.com (LHC) Date: Mon, 24 Oct 2016 08:28:10 +0000 Subject: Dyn DDoS this AM? In-Reply-To: <20161023224258.3569F576E7D7@rock.dv.isc.org> References: <580ABCF7.1030003@vaxination.ca> <20161023224258.3569F576E7D7@rock.dv.isc.org> Message-ID: <9B539B4C-BC53-4CBD-AE0E-AAEE99FE2CB7@gmx.com> All this TTL talk makes me think. Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? On October 23, 2016 3:42:58 PM PDT, Mark Andrews wrote: > >In message > >, Masood Ahmad Shah writes: >> > >> > > On Oct 21, 2016, at 6:35 PM, Eitan Adler >wrote: >> > > >> > > [...] >> > > >> > > In practice TTLs tend to be ignored on the public internet. In >past >> > > research I've been involved with browser[0] behavior was >effectively >> > > random despite the TTL set. >> > > >> > > [0] more specifically, the chain of DNS resolution and caching >down to >> > > the browser. >> > >> > >> > Yes, but that it can be both better and worse than your TTLs does >not >> > mean that you can ignore properly working implementations. >> > >> > If the other end device chain breaks you that's their fault and out >of >> > your control. If your own settings break you that's your fault. >> > >> >> +1 to what George wrote that we should make efforts to improve our >part of >> the network. There are ISPs that ignore TTL settings and only update >their >> cached records every two to three days or even more (particularly the >> smaller ones). OTOH, this results in your DNS data being inconsistent >but >> it???s very common to cache DNS records at multiple levels. It's an >effort >> that everyone needs to contribute to. > >For TTL there is a tension between being able to update with new >data and resiliance when servers are unreachable. For zone transfers >we have 3 timers refesh, retry and expire to deal with this tension. >If we were doing DNS from scratch there would be at least two ttl >values one for freshness and one for don't use past. > >Additionally a lot of the need for small TTL's is because clients >don't fail over to second addresses in a reasonable amount of time. >There is no reason for this other than poorly designed clients. A >client can failover using sub-second timers. We do this for Happy >Eyeballs. This strategy is viable for ALL connection attempts. > >Mark -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From large.hadron.collider at gmx.com Mon Oct 24 08:25:09 2016 From: large.hadron.collider at gmx.com (LHC) Date: Mon, 24 Oct 2016 08:25:09 +0000 Subject: Dyn DDoS this AM? In-Reply-To: <20161023224258.3569F576E7D7@rock.dv.isc.org> References: <580ABCF7.1030003@vaxination.ca> <20161023224258.3569F576E7D7@rock.dv.isc.org> Message-ID: <6FBDD46B-3FF1-423E-9037-E01D89C63063@gmx.com> All this TTL talk makes me think. Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? On October 23, 2016 3:42:58 PM PDT, Mark Andrews wrote: > >In message > >, Masood Ahmad Shah writes: >> > >> > > On Oct 21, 2016, at 6:35 PM, Eitan Adler >wrote: >> > > >> > > [...] >> > > >> > > In practice TTLs tend to be ignored on the public internet. In >past >> > > research I've been involved with browser[0] behavior was >effectively >> > > random despite the TTL set. >> > > >> > > [0] more specifically, the chain of DNS resolution and caching >down to >> > > the browser. >> > >> > >> > Yes, but that it can be both better and worse than your TTLs does >not >> > mean that you can ignore properly working implementations. >> > >> > If the other end device chain breaks you that's their fault and out >of >> > your control. If your own settings break you that's your fault. >> > >> >> +1 to what George wrote that we should make efforts to improve our >part of >> the network. There are ISPs that ignore TTL settings and only update >their >> cached records every two to three days or even more (particularly the >> smaller ones). OTOH, this results in your DNS data being inconsistent >but >> it???s very common to cache DNS records at multiple levels. It's an >effort >> that everyone needs to contribute to. > >For TTL there is a tension between being able to update with new >data and resiliance when servers are unreachable. For zone transfers >we have 3 timers refesh, retry and expire to deal with this tension. >If we were doing DNS from scratch there would be at least two ttl >values one for freshness and one for don't use past. > >Additionally a lot of the need for small TTL's is because clients >don't fail over to second addresses in a reasonable amount of time. >There is no reason for this other than poorly designed clients. A >client can failover using sub-second timers. We do this for Happy >Eyeballs. This strategy is viable for ALL connection attempts. > >Mark -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From rsk at gsp.org Mon Oct 24 11:43:24 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Mon, 24 Oct 2016 07:43:24 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <103874.1477290542@turing-police.cc.vt.edu> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <891588715.4653.1477155932958.JavaMail.mhammett@ThunderFuck> <580BD066.8050407@vaxination.ca> <826459920.4830.1477172878616.JavaMail.mhammett@ThunderFuck> <20161022223550.GA12610@esri.com> <580BF49C.5090209@vaxination.ca> <103874.1477290542@turing-police.cc.vt.edu> Message-ID: <20161024114324.GA15537@gsp.org> On Mon, Oct 24, 2016 at 02:29:02AM -0400, Valdis.Kletnieks at vt.edu wrote: > A few years ago, Vint Cerf gave a keynote speech at a conference, where he > claimed that there were 140 million pwned devices on the Internet - and this > was before IoT was itself a thing. > > Not one person in the security industry called bullshit and said the number > was too high. There were however a lot of people who thought Cerf had > significantly lowballed the estimate. It was January, 2007: Vint Cerf: one quarter of all computers part of a botnet http://arstechnica.com/news.ars/post/20070125-8707.html I thought, based on personal research and some discussions with other people interested in the same question, that 140M was a bit high at the time. Looking back, armed with more data and perspective, I think it was much too low. Nothing that has happened in the decade since gives me any reason to think the number has gone down. Many things have have happened in the decade since that give me reason to think the number has gone up -- significantly. And that's *before* I factor in the IoT. Speaking of which, I think IoT devices divide neatly into two categories: those that have been compromised, and those that are going to be. (It might be a while for some of the latter category to shift to the former: attackers find themselves in an incredibly target-rich environment and may perceive little need to move past the low-hanging fruit just yet.) So whatever the number is at this point -- 300M? 500M? -- it's enormous, it's going to get bigger, and it's going to get bigger quickly. In a relatively short time we've taken a system built to resist destruction by nuclear weapons and made it vulnerable to toasters. --- Jeff Jarmoc, October 21, 2016 ---rsk From josh at kyneticwifi.com Mon Oct 24 12:03:45 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Mon, 24 Oct 2016 07:03:45 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: You CAN actually block things, within reason. The caveat is you simply have to disclose it. There is a 'reasonable network management' clause. IANAL, please consult your telecommunications legal team. On Oct 24, 2016 1:25 AM, "Richard Holbo" wrote: > I run/manage the networks for several smallish (in the thousands of > customers) eyeball ISP's and I appreciate a nice "hey you've got a bot" or > "someone is scanning" me notice to my abuse emails. They are useful in > identifying crap that's going on, so for those of you who have the > resources to do that... I appreciate it, we do read them at my networks > and try to do something. > > That said... getting end users to actually fix the broken routers etc. etc. > is NOT easy. Very often we'll notify customers, they will _take their > stuff to the local computer repair guy_ ... or office depo.... and they > will run whatever auto scan they have and say it's all fine. Customer puts > it back in, it's still broke, and they call customer support and want us to > pay for the trip because _their_ expert says it's fine... > > IMHO since the advent of Net Neutrality... I cannot simply block all of X, > Y or Z at my edge and tell the customers it's for the best. I'd love to > block some stuff in and outbound to customers, but then the customer just > yells at us and files complaints with the PUC because _they have a right to > it_.. So those of you calling for Government interference... we've already > done that and it does not help. > > /rh > > On Sun, Oct 23, 2016 at 10:56 PM, John Weekes > wrote: > > > On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > > > >> > >> ... I've recorded > >>> about 2.4 million IP addresses involved in the last two months (a > number > >>> that is higher than the number of actual devices, since most seem to > >>> have dynamic IP addresses). The ISPs behind those IP addresses have > >>> received notifications via email... > >>> > >> Just curious... How well is that working out? > >> > > > > For the IoT botnets, most of the emails are ignored or rejected, because > > most go to providers who either quietly bitbucket them or flat-out reject > > all abuse emails. Most emails sent to mainland China, for instance, are > in > > that category (Hong Kong ISPs are somewhat better). > > > > For other botnets, such as those using compromised webservers running > > outdated phpMyAdmin installs at random hosts, harnessing spun-up services > > at reputable VPS providers (Amazon, Microsoft, Rackspace, etc.), or > > harnessing devices at large and small US and Canadian ISPs, we have had > > better luck. Usually, we don't hear a response back, but those emails are > > often forwarded to the end-user, who takes action (and may ask us for > help, > > which is how we know they are being forwarded). The fixes can enough to > > reduce attack volumes to more manageable levels. > > > > Kudos go out to the large and small ISPs and NSPs who have started > > policing SSDP and other reflection traffic, which we also send out some > > notifications for. In some cases, it may be that our emails spurred them > to > > notice how much damage those attacks were doing and how much it was > costing > > them to carry the attack traffic. > > > > I've tried this myself a few times in the past, when I've found things > >> that appear to be seriously compromised, and for my extensive trouble > >> I've mostly received back utter silence and no action. I remember that > >> after properly notifying security@ some large end-luser cable network > >> in the SouthEast (which shall remain nameless) I got back something > >> along the lines of "Thank you. We'll look into it." and was disgusted > >> to find, two months later, that the boxes in question were still utterly > >> pwned and in the exact same state they were two months prior, when I > >> had first reported them. > >> > > > > We do get our share of that, as well, unfortunately, along with our share > > of people who send angry responses calling the notifications spam (I > > disagree with them that sending a legitimate abuse notification to a > > publicly-posted, designated abuse account should be considered spam) or > who > > flame us for acting like "internet police". But, we persist. Some people > > change their minds after receiving multiple notifications or after we > > explain that DoS traffic costs them money and hurts their customers, who > > will be experiencing degraded service and may silently switch providers > > over it. > > > > I guess that's just an example of what somebody else already noted here, > >> i.e. that providers don't care to spend the time and/or effort and/or > >> money necessary to actually -do- anything about compromised boxes, and > >> anyway, they don't want to lose a paying customer. > >> > >> So, you know, let's just say for the sake of argument that right now, > >> today, I know about a botnet consiting of a quarter million popped > >> boxes, and that I have in-hand all of the relevant IPs, and that I > >> have no trouble finding contact email addresses for all of the relevant > >> ASNs. So then what? > >> > > > > I use scripts to send out an abuse notification to some percentage of the > > compromised hosts -- the ones sending some significant amount of the > > traffic. The notification includes a description of what we saw and > > timestamped example attack traffic, as interpreted by tcpdump. If further > > traffic is seen later from the same host, another notification will be > > sent, after a cool-off period. > > > > The emails are plain text and we don't try to use them as advertisement. > > We also don't force a link to be clicked to see more details or to > respond > > back. I don't like to receive such emails myself and have found that > those > > types are more likely to be ignored. > > > > The question is: Why should I waste my time informing all, or even any > >> of these ASNs about the popped boxes on their networks when (a) I am > >> not their customer... as many of them have been only too happy to > >> gleefully inform me in the past... and when (b) the vast majority > >> simply won't do anything with the information? > >> > > > > I'm not saying that everyone should send abuse notifications like we do, > > since it can be a big task. But, in response to someone wondering if > their > > network is being used for attacks, or asking how they could help to > police > > their own network, I am saying that making sure that inbound abuse > > notifications are arriving at the right place and being handled > > appropriately is important. > > > > And while we are on the subject, I just have to bring up one of my > >> biggest pet peeves. Why is it that every time some public-spirited > >> altrusitc well-meaning citizen such as myself reports any kind of a > >> problem to any kind of a company on the Internet, the report itself > >> gets immediately labeled and categorized as a "complaint". If I spend > >> some of -my- valuable time to helpfully try to let somebody else know > >> of a problem on their network, or with their web site, and if that > >> report gets categorized as a "complaint" then what does that make me? > >> A "complainer"?? > >> > >> I don't need this kind of abuse and denegration from people who I'm > >> trying to help. Like most other people, if I am in need of some > >> personal denegration and abuse... well... I have relatives for that. > >> > > > > There's a spectrum of people responding to these and some percentage are > > just jerks, as in real life. But, I like to think that the majority of at > > least NA providers are represented by professionals who just don't > respond > > out of courtesy because they don't want to flood our inboxes with simple > > acknowledgements. > > > > Those of us experiencing these attacks appreciate the community support, > > both from people like you who also send notifications and those who > handle > > the notifications on the receiving end. > > > > -John > > > From m4rtntns at gmail.com Mon Oct 24 12:05:33 2016 From: m4rtntns at gmail.com (Martin T) Date: Mon, 24 Oct 2016 15:05:33 +0300 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: Thank you all for the replies! I'll go with the solution where "ISP A" announces both /19 prefix and /24 prefix. Martin On Thu, Oct 20, 2016 at 1:16 AM, Matt Buford wrote: > On Mon, Oct 10, 2016 at 2:44 PM, Baldur Norddahl > wrote: > > >> Is that a real problem? In my experience a /24 is honoured almost >> universially. >> > > Here's a real-world issue I ran into with this. In this case, it isn't > that someone filtered /24s, but that they didn't have a full table (peering > routes only plus a default). This resulted in them following the less > specific route for traffic destined for the /24. > > A customer of mine was advertising only a /20 to me and then only a more > specific /24 was advertised from a remote site of theirs to a different > ISP. The customer did have a connection between their two sites, so in > theory it was OK if the traffic arrived on their "wrong" link, except that > the customer strongly didn't want this to happen, thus the inconsistent > routes. > > This customer found that the remote /24 was unable to access a large CDN > provider. This CDN told them that a traceroute to the /24 went to my > network (we peered at an exchange) and was then dropped. > > This seemed odd at first, as I confirmed I was not advertising the /24 to > them so why were they routing it to me? It turned out that the CDN > provider was running a peer-routes-only network with a default to their > transit. This meant that they saw the /20 from their peer (me) but never > saw the /24, since they carried no transit routes. This resulted in them > routing the entire /20 to me. > > My peering router was not willing to route traffic from a peering exchange > towards transit I had to pay for, so it was dropped. > > The customer's split advertisements didn't seem particularly unreasonable > or invalid, though perhaps they were not the preferred setup. It wasn't > unreasonable for me to not route from a peering exchange to transit. It > wasn't unreasonable of the CDN to have a peering-and-default routing > table. But, those three things together were not compatible. > > I called the customer and presented them with my findings and some > potential options to consider, and consistent advertisements was one of > those options. The customer strongly wanted incoming traffic to arrive > directly to the correct location so he didn't want to do that. I suggested > a possible compromise was for him to advertise both the /24 and /20 to me, > but use communities so that I never advertised his /24 to any upstreams or > peering exchanges. That way, if traffic for the /24 accidentally hit my > network like we were running into, I would route it to him and he could > pass it to the correct site. It meant that a little traffic would arrive > at the wrong site in his network and have to pass over his back-end link, > but at least it would be fairly limited. He didn't like this either. He > didn't want to split the /20 advertisement up to no longer cover the /24 > either, I think just because "I shouldn't have to do that!" > > The option the customer chose in the end was to use a community on his > advertisement to instruct my routers to no longer advertise his /20 to any > peering exchanges at all. That ensured the CDN didn't directly send me > anything for him (not the /24 or the /20), and the transit networks > in-between took care of making sure traffic to the /24 didn't accidentally > end up on my network. While I didn't find it very elegant to be shifting > traffic from peering exchanges to transit, it wasn't a significant amount > of traffic and it got him off my back. From lear at cisco.com Mon Oct 24 12:49:07 2016 From: lear at cisco.com (Eliot Lear) Date: Mon, 24 Oct 2016 14:49:07 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: <20161022125335.GA84013@ussenterprise.ufp.org> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> Message-ID: Hi Leo and all, Well, here we are together again ;-) Please see below. On 10/22/16 2:53 PM, Leo Bicknell wrote: > In a message written on Sat, Oct 22, 2016 at 07:34:55AM -0500, Mike Hammett wrote: >> "taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" > From https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/#more-36754 > > The part that should outrage everyone on this list: > > That's because while many of these devices allow users to change > the default usernames and passwords on a Web-based administration > panel that ships with the products, those machines can still be > reached via more obscure, less user-friendly communications services > called "Telnet" and "SSH." > > "The issue with these particular devices is that a user cannot > feasibly change this password," Flashpoints Zach Wikholm told > KrebsOnSecurity. "The password is hardcoded into the firmware, and > the tools necessary to disable it are not present. Even worse, the > web interface is not aware that these credentials even exist." > > As much as I hate to say it, what is needed is regulation. It could > be some form of self regulation, with retailers refusing to sell > products that aren't "certified" by some group. It could be full > blown government regulation. Perhaps a mix. We we discussed the last time, this is an opportunity. Let's not miss again. People have been mentioning BCP38. That BCP needs a dust-off. as mentioned, simply masking an address in a BOTNET attack is a relatively small component of the problem that often gets solved by accident with NATs. We have a broader set of issues to address, and anyone looking for a single silver bullet will be disappointed. The questions that need asking are these: * What is the responsibility of the end system (either side) and its developer/manufacturer? What are the things Thing developers should be doing? * What is the responsibility of the home router? How can home routers enable appropriate access for a device while stopping unwanted traffic? I won't repeat the ENTIRE thread, but draft-ietf-opsawg-mud-01.txt can help and I'll provide an example MUD file as a postscript to this message that would have stopped infection of some DVRs. * What is the responsibility of the provider access router? {You fill in this part} * What is the responsibility of the provider peering router? BCP38 filtering, use of ROAs, BGPSEC, and other fighting words ;-) * What is the responsibility of the consumer? Less is more, here, I think. * What is the responsibility of government? I would suggest that we have two or three documents to be written, which combined could be an update to BCP38. And the answers to each of these questions are going to evolve over time. That's okay. BCPs should be living documents. > > It's not a problem for a network operator to "solve", any more than > someone who builds roads can make an unsafe car safe. Yes, both > the network operator and rood operator play a role in building safe > infrastructure (BCP38, deformable barriers), but neither can do > anything for a manufacturer who builds a device that is wholely > deficient in the first place. > Yes. Eliot ps: here's that MUD file. It's possible to write it in more specific language. Probably not necessary. What is important is that someone fill in the class "http://dvr264.example.com/controller". That's not hard, but needs doing. What happens next is that the class gets expanded and access lists get installed, preferably on the home router. And this means that the home router needs to play a bigger role of limiting ingress even in the home. That can prevent reflection attacks, for instance. { "ietf-mud:meta-info": { "lastUpdate": "2016-10-23T14:11:52+02:00", "systeminfo": "DVR H.264", "cacheValidity": 1440 }, "ietf-acl:access-lists": { "ietf-acl:access-list": [ { "acl-name": "mud-10387-v4in", "acl-type": "ipv4-acl", "ietf-mud:packet-direction": "to-device", "access-list-entries": { "ace": [ { "rule-name": "clout0-in", "matches" : { "ietf-mud:direction-initiated" : "from-device" }, "actions": { "permit": [ null ] } }, { "rule-name": "entin0-in", "matches": { "ietf-mud:controller": "http://dvr264.example.com/controller", "ietf-mud:direction-initiated" : "to-device" }, "actions": { "permit": [ null ] } } ] } }, { "acl-name": "mud-10387-v4out", "acl-type": "ipv4-acl", "ietf-mud:packet-direction": "from-device", "access-list-entries": { "ace": [ { "rule-name": "clout0-in", "matches": { "ietf-mud:direction-initiated" : "from-device" }, "actions": { "permit": [ null ] } }, { "rule-name": "entin0-in", "matches": { "ietf-mud:controller": "http://dvr264.example.com/controller", "ietf-mud:direction-initiated" : "to-device" }, "actions": { "permit": [ null ] } } ] } } ] } } -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From cb.list6 at gmail.com Mon Oct 24 13:06:22 2016 From: cb.list6 at gmail.com (Ca By) Date: Mon, 24 Oct 2016 06:06:22 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> Message-ID: On Monday, October 24, 2016, Eliot Lear wrote: > Hi Leo and all, > > Well, here we are together again ;-) Please see below. > > > On 10/22/16 2:53 PM, Leo Bicknell wrote: > > In a message written on Sat, Oct 22, 2016 at 07:34:55AM -0500, Mike > Hammett wrote: > >> "taken all necessary steps to insure that none of the numerous specific > types of CCVT thingies that Krebs and others identified" > > From https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs- > powered-todays-massive-internet-outage/#more-36754 > > > > The part that should outrage everyone on this list: > > > > That's because while many of these devices allow users to change > > the default usernames and passwords on a Web-based administration > > panel that ships with the products, those machines can still be > > reached via more obscure, less user-friendly communications > services > > called "Telnet" and "SSH." > > > > "The issue with these particular devices is that a user cannot > > feasibly change this password," Flashpoints Zach Wikholm told > > KrebsOnSecurity. "The password is hardcoded into the firmware, > and > > the tools necessary to disable it are not present. Even worse, > the > > web interface is not aware that these credentials even exist." > > > > As much as I hate to say it, what is needed is regulation. It could > > be some form of self regulation, with retailers refusing to sell > > products that aren't "certified" by some group. It could be full > > blown government regulation. Perhaps a mix. > > We we discussed the last time, this is an opportunity. Let's not miss > again. People have been mentioning BCP38. That BCP needs a dust-off. > as mentioned, simply masking an address in a BOTNET attack is a > relatively small component of the problem that often gets solved by > accident with NATs. We have a broader set of issues to address, and > anyone looking for a single silver bullet will be disappointed. > > The questions that need asking are these: > > * What is the responsibility of the end system (either side) and its > developer/manufacturer? What are the things Thing developers should > be doing? > * What is the responsibility of the home router? How can home routers > enable appropriate access for a device while stopping unwanted > traffic? I won't repeat the ENTIRE thread, but > draft-ietf-opsawg-mud-01.txt can help and I'll provide an example > MUD file as a postscript to this message that would have stopped > infection of some DVRs. > * What is the responsibility of the provider access router? {You fill > in this part} > * What is the responsibility of the provider peering router? BCP38 > filtering, use of ROAs, BGPSEC, and other fighting words ;-) > * What is the responsibility of the consumer? Less is more, here, I > think. > * What is the responsibility of government? > > I would suggest that we have two or three documents to be written, which > combined could be an update to BCP38. And the answers to each of these > questions are going to evolve over time. That's okay. BCPs should be > living documents. > > > > > > It's not a problem for a network operator to "solve", any more than > > someone who builds roads can make an unsafe car safe. Yes, both > > the network operator and rood operator play a role in building safe > > infrastructure (BCP38, deformable barriers), but neither can do > > anything for a manufacturer who builds a device that is wholely > > deficient in the first place. > > > Yes. > > Eliot > > ps: here's that MUD file. It's possible to write it in more specific > language. Probably not necessary. What is important is that someone > fill in the class "http://dvr264.example.com/controller". That's not > hard, but needs doing. What happens next is that the class gets > expanded and access lists get installed, preferably on the home router. > And this means that the home router needs to play a bigger role of > limiting ingress even in the home. That can prevent reflection attacks, > for instance. > > { > "ietf-mud:meta-info": { > "lastUpdate": "2016-10-23T14:11:52+02:00", > "systeminfo": "DVR H.264", > "cacheValidity": 1440 > }, > "ietf-acl:access-lists": { > "ietf-acl:access-list": [ > { > "acl-name": "mud-10387-v4in", > "acl-type": "ipv4-acl", > "ietf-mud:packet-direction": "to-device", > "access-list-entries": { > "ace": [ > { > "rule-name": "clout0-in", > "matches" : { > "ietf-mud:direction-initiated" : "from-device" > }, > "actions": { > "permit": [ > null > ] > } > }, > { > "rule-name": "entin0-in", > "matches": { > "ietf-mud:controller": > "http://dvr264.example.com/controller", > "ietf-mud:direction-initiated" : "to-device" > }, > "actions": { > "permit": [ > null > ] > > } > } > ] > } > }, > { > "acl-name": "mud-10387-v4out", > "acl-type": "ipv4-acl", > "ietf-mud:packet-direction": "from-device", > "access-list-entries": { > "ace": [ > { > "rule-name": "clout0-in", > "matches": { > "ietf-mud:direction-initiated" : "from-device" > }, > "actions": { > "permit": [ > null > ] > } > }, > { > "rule-name": "entin0-in", > "matches": { > "ietf-mud:controller": "http://dvr264.example.com/controller", > "ietf-mud:direction-initiated" : "to-device" > }, > "actions": { > "permit": [ > null > ] > } > } > ] > } > } > ] > } > } > > Assuming MUD is successful in the ietf, the cpe lifecycle is 10 years before the needle moves. At which point the target will have morphed to something else. Also, nobody is going to pay for that feature. Just like the early days of ipv6, the economics were misaligned. in 10 years, the CPE will also be running PCP, where the bot tells the CPE to ignore all of MUD and open any arbitrary port it wants. Or, in 10 years we stopped using ipv4 (!!!!) wherein it is now unlikely to enumerate via the entire address space to find the telnet logins. My money is on dropping ipv4 as the most viable and most bang for the buck. It makes many of today's problems go away. Also, stopping support for ipv4 drops off the lowest of the low (like windows xp) From lear at cisco.com Mon Oct 24 13:22:03 2016 From: lear at cisco.com (Eliot Lear) Date: Mon, 24 Oct 2016 15:22:03 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> Message-ID: <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> Hi, On 10/24/16 3:06 PM, Ca By wrote: > > Assuming MUD is successful in the ietf, the cpe lifecycle is 10 years > before the needle moves. At which point the target will have morphed > to something else. Also, nobody is going to pay for that feature. Just > like the early days of ipv6, the economics were misaligned. We know of those who are planning to build, so maybe not so much. The function doesn't NEED to be in CPE, but it helps. And again, the CPE market is changing right now, so be careful about your assumptions. > > in 10 years, the CPE will also be running PCP, where the bot tells the > CPE to ignore all of MUD and open any arbitrary port it wants. One of the hidden villains in these attacks, by the way, is uPnP. The point is not for the device to self-assert, but for the manufacturer to assert. Apart from that PCP actually solves a slightly different problem. MUD can tackle interior connectivity, which PCP doesn't really address. And really that's what we need to address reflection attacks. Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From cb.list6 at gmail.com Mon Oct 24 14:03:14 2016 From: cb.list6 at gmail.com (Ca By) Date: Mon, 24 Oct 2016 07:03:14 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> Message-ID: On Mon, Oct 24, 2016 at 6:22 AM, Eliot Lear wrote: > Hi, > > > On 10/24/16 3:06 PM, Ca By wrote: > > > > Assuming MUD is successful in the ietf, the cpe lifecycle is 10 years > > before the needle moves. At which point the target will have morphed > > to something else. Also, nobody is going to pay for that feature. Just > > like the early days of ipv6, the economics were misaligned. > > We know of those who are planning to build, so maybe not so much. The > function doesn't NEED to be in CPE, but it helps. And again, the CPE > market is changing right now, so be careful about your assumptions. > > Please elaborate on concrete evidence to support your claim the CPE market is changing. > > > > in 10 years, the CPE will also be running PCP, where the bot tells the > > CPE to ignore all of MUD and open any arbitrary port it wants. > > One of the hidden villains in these attacks, by the way, is uPnP. The > point is not for the device to self-assert, but for the manufacturer to > assert. Apart from that PCP actually solves a slightly different > problem. MUD can tackle interior connectivity, which PCP doesn't really > address. And really that's what we need to address reflection attacks. > > Eliot > > From lguillory at reservetele.com Mon Oct 24 13:22:20 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Mon, 24 Oct 2016 13:22:20 +0000 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> Message-ID: <8D89F96B7AB1B84F9E049CC7BB91BF5CFAF87E4D@RTC-EXCH01.RESERVE.LDS> That only works if the /24 isn't coming from the middle of the /20 block and is on the front end or back end. Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Martin T Sent: Monday, October 24, 2016 7:06 AM To: Matt Buford; Baldur Norddahl Cc: nanog Subject: Re: nested prefixes in Internet Thank you all for the replies! I'll go with the solution where "ISP A" announces both /19 prefix and /24 prefix. Martin On Thu, Oct 20, 2016 at 1:16 AM, Matt Buford wrote: > On Mon, Oct 10, 2016 at 2:44 PM, Baldur Norddahl > > wrote: > > >> Is that a real problem? In my experience a /24 is honoured almost >> universially. >> > > Here's a real-world issue I ran into with this. In this case, it > isn't that someone filtered /24s, but that they didn't have a full > table (peering routes only plus a default). This resulted in them > following the less specific route for traffic destined for the /24. > > A customer of mine was advertising only a /20 to me and then only a > more specific /24 was advertised from a remote site of theirs to a > different ISP. The customer did have a connection between their two > sites, so in theory it was OK if the traffic arrived on their "wrong" > link, except that the customer strongly didn't want this to happen, > thus the inconsistent routes. > > This customer found that the remote /24 was unable to access a large > CDN provider. This CDN told them that a traceroute to the /24 went to > my network (we peered at an exchange) and was then dropped. > > This seemed odd at first, as I confirmed I was not advertising the /24 > to them so why were they routing it to me? It turned out that the CDN > provider was running a peer-routes-only network with a default to > their transit. This meant that they saw the /20 from their peer (me) > but never saw the /24, since they carried no transit routes. This > resulted in them routing the entire /20 to me. > > My peering router was not willing to route traffic from a peering > exchange towards transit I had to pay for, so it was dropped. > > The customer's split advertisements didn't seem particularly > unreasonable or invalid, though perhaps they were not the preferred > setup. It wasn't unreasonable for me to not route from a peering > exchange to transit. It wasn't unreasonable of the CDN to have a > peering-and-default routing table. But, those three things together were not compatible. > > I called the customer and presented them with my findings and some > potential options to consider, and consistent advertisements was one > of those options. The customer strongly wanted incoming traffic to > arrive directly to the correct location so he didn't want to do that. > I suggested a possible compromise was for him to advertise both the > /24 and /20 to me, but use communities so that I never advertised his > /24 to any upstreams or peering exchanges. That way, if traffic for > the /24 accidentally hit my network like we were running into, I would > route it to him and he could pass it to the correct site. It meant > that a little traffic would arrive at the wrong site in his network > and have to pass over his back-end link, but at least it would be > fairly limited. He didn't like this either. He didn't want to split > the /20 advertisement up to no longer cover the /24 either, I think just because "I shouldn't have to do that!" > > The option the customer chose in the end was to use a community on his > advertisement to instruct my routers to no longer advertise his /20 to > any peering exchanges at all. That ensured the CDN didn't directly > send me anything for him (not the /24 or the /20), and the transit > networks in-between took care of making sure traffic to the /24 didn't > accidentally end up on my network. While I didn't find it very > elegant to be shifting traffic from peering exchanges to transit, it > wasn't a significant amount of traffic and it got him off my back. From lear at cisco.com Mon Oct 24 14:46:27 2016 From: lear at cisco.com (Eliot Lear) Date: Mon, 24 Oct 2016 16:46:27 +0200 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> Message-ID: <37bdf646-338c-d924-2693-529dc435a65a@cisco.com> On 10/24/16 4:03 PM, Ca By wrote: > > Please elaborate on concrete evidence to support your claim the CPE > market is changing. If you can't see that then you're not paying attention. Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From cb.list6 at gmail.com Mon Oct 24 14:59:27 2016 From: cb.list6 at gmail.com (Ca By) Date: Mon, 24 Oct 2016 07:59:27 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <37bdf646-338c-d924-2693-529dc435a65a@cisco.com> References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> <37bdf646-338c-d924-2693-529dc435a65a@cisco.com> Message-ID: On Mon, Oct 24, 2016 at 7:46 AM, Eliot Lear wrote: > > > On 10/24/16 4:03 PM, Ca By wrote: > > > Please elaborate on concrete evidence to support your claim the CPE market > is changing. > > > If you can't see that then you're not paying attention. > > Eliot > > Not helpful. Reflects the weakness of your argument and uselessness of MUD. CB From lists at eitanadler.com Mon Oct 24 16:06:18 2016 From: lists at eitanadler.com (Eitan Adler) Date: Mon, 24 Oct 2016 09:06:18 -0700 Subject: Dyn DDoS this AM? In-Reply-To: <6FBDD46B-3FF1-423E-9037-E01D89C63063@gmx.com> References: <580ABCF7.1030003@vaxination.ca> <20161023224258.3569F576E7D7@rock.dv.isc.org> <6FBDD46B-3FF1-423E-9037-E01D89C63063@gmx.com> Message-ID: On 24 October 2016 at 01:25, LHC wrote: > All this TTL talk makes me think. > > Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? If clients can't get one TTL correct what makes you think they will get a more complicated two TTL system correct? -- Eitan Adler From mhuff at ox.com Mon Oct 24 17:05:53 2016 From: mhuff at ox.com (Matthew Huff) Date: Mon, 24 Oct 2016 17:05:53 +0000 Subject: BGP route instabilities Message-ID: We saw a slight uptick in routes today (at least since the last time I looked), but a large number of route flaps coming out of the APNIC region. Anyone else notice anything? ---- Matthew Huff???????????? | 1 Manhattanville Rd Director of Operations???| Purchase, NY 10577 From la at qrator.net Mon Oct 24 18:12:03 2016 From: la at qrator.net (Alexander Lyamin) Date: Mon, 24 Oct 2016 20:12:03 +0200 Subject: Dyn DDoS this AM? In-Reply-To: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction. I think its a right time to direct community attention to this document https://www.routingmanifesto.org/manrs/ It's work in progress. But its a good start. On Fri, Oct 21, 2016 at 5:48 PM, Patrick W. Gilmore wrote: > I cannot give additional info other than what?s been on ?public media?. > > However, I would very much like to say that this is a horrific trend on > the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can > Not Stand. See Krebs? on the Democratization of Censorship. See lots of > other things. > > To Dyn and everyone else being attacked: > The community is behind you. There are problems, but if we stick together, > we can beat these miscreants. > > To the miscreants: > You will not succeed. Search "churchill on the beaches?. It?s a bit > melodramatic, but it?s how I feel at this moment. > > To the rest of the community: > If you can help, please do. I know a lot of you are thinking ?what can I > do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, > that doesn?t help Mirai, but it still helps. There are many other things > you can do as well. > > But a lot of it is just willingness to help. When someone asks you to help > trace an attack, do not let the request sit for a while. Damage is being > done. Help your neighbor. When someone?s house is burning, your current > project, your lunch break, whatever else you are doing is almost certainly > less important. If we stick together and help each other, we can - we WILL > - win this war. If we are apathetic, we have already lost. > > > OK, enough motivational speaking for today. But take this to heart. Our > biggest problem is people thinking they cannot or do not want to help. > > -- > TTFN, > patrick > > > On Oct 21, 2016, at 10:55 AM, Chris Grundemann > wrote: > > > > Does anyone have any additional details? Seems to be over now, but I'm > very > > curious about the specifics of such a highly impactful attack (and it's > > timing following NANOG 68)... > > > > https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- > twitter-spotify-reddit/ > > > > -- > > @ChrisGrundemann > > http://chrisgrundemann.com > > -- Alexander Lyamin CEO | Qrator * Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la at qrator.net From ahebert at pubnix.net Mon Oct 24 18:38:58 2016 From: ahebert at pubnix.net (Alain Hebert) Date: Mon, 24 Oct 2016 14:38:58 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <108bf30f-d604-89da-28dc-e327c3216c12@pubnix.net> And its not the last time the big Tier(s) will refuse to do anything beside dropping the fault to the CPE vendors. People love circles. ----- Alain Hebert ahebert at pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/24/16 14:12, Alexander Lyamin wrote: > Its not a first time we have and large scale DDoS incident. > Its not a first time we have (a kind of) knee-jerk reaction. > > I think its a right time to direct community attention to this document > > https://www.routingmanifesto.org/manrs/ > > It's work in progress. But its a good start. > > > > On Fri, Oct 21, 2016 at 5:48 PM, Patrick W. Gilmore > wrote: > >> I cannot give additional info other than what?s been on ?public media?. >> >> However, I would very much like to say that this is a horrific trend on >> the Internet. The idea that someone can mention a DDoS then get DDoS?ed Can >> Not Stand. See Krebs? on the Democratization of Censorship. See lots of >> other things. >> >> To Dyn and everyone else being attacked: >> The community is behind you. There are problems, but if we stick together, >> we can beat these miscreants. >> >> To the miscreants: >> You will not succeed. Search "churchill on the beaches?. It?s a bit >> melodramatic, but it?s how I feel at this moment. >> >> To the rest of the community: >> If you can help, please do. I know a lot of you are thinking ?what can I >> do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, >> that doesn?t help Mirai, but it still helps. There are many other things >> you can do as well. >> >> But a lot of it is just willingness to help. When someone asks you to help >> trace an attack, do not let the request sit for a while. Damage is being >> done. Help your neighbor. When someone?s house is burning, your current >> project, your lunch break, whatever else you are doing is almost certainly >> less important. If we stick together and help each other, we can - we WILL >> - win this war. If we are apathetic, we have already lost. >> >> >> OK, enough motivational speaking for today. But take this to heart. Our >> biggest problem is people thinking they cannot or do not want to help. >> >> -- >> TTFN, >> patrick >> >>> On Oct 21, 2016, at 10:55 AM, Chris Grundemann >> wrote: >>> Does anyone have any additional details? Seems to be over now, but I'm >> very >>> curious about the specifics of such a highly impactful attack (and it's >>> timing following NANOG 68)... >>> >>> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- >> twitter-spotify-reddit/ >>> -- >>> @ChrisGrundemann >>> http://chrisgrundemann.com >> > From jared at puck.Nether.net Mon Oct 24 18:42:10 2016 From: jared at puck.Nether.net (Jared Mauch) Date: Mon, 24 Oct 2016 14:42:10 -0400 Subject: Dyn DDoS this AM? In-Reply-To: <108bf30f-d604-89da-28dc-e327c3216c12@pubnix.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <108bf30f-d604-89da-28dc-e327c3216c12@pubnix.net> Message-ID: <20161024184210.GB15308@puck.nether.net> On Mon, Oct 24, 2016 at 02:38:58PM -0400, Alain Hebert wrote: > And its not the last time the big Tier(s) will refuse to do anything > beside dropping the fault to the CPE vendors. I can say that we had to drop uRPF for technical reasons, namely not enough people ask their vendors about it so it is: a) not tested (at all) b) not performance rated c) lacks simple fixes The people who have the purchasing power are not the tier-1 carriers regardless. We push as hard as we can and end up with the compromises as a result. - Jared From jared at puck.Nether.net Mon Oct 24 18:43:06 2016 From: jared at puck.Nether.net (Jared Mauch) Date: Mon, 24 Oct 2016 14:43:06 -0400 Subject: Dyn DDoS this AM? In-Reply-To: <55cd5727-381e-5f71-7a53-ad99cdd27e7d@pubnix.net> References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> <55cd5727-381e-5f71-7a53-ad99cdd27e7d@pubnix.net> Message-ID: <20161024184306.GC15308@puck.nether.net> On Fri, Oct 21, 2016 at 12:30:44PM -0400, Alain Hebert wrote: > Rofl, > > Yeah good luck with that... 15+ years later and most of the actors > that could fix that, for the planete, still refuses to do anything. > > Now you can start the usual circular discussion that goes nowhere > after 3 days... > > PS: yeah usual BCP38 rant... but its friday. Not all attacks are BCP38 related. :-) - Jared -- Jared Mauch | pgp key available via finger from jared at puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine. From jfmezei_nanog at vaxination.ca Mon Oct 24 19:45:36 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Mon, 24 Oct 2016 15:45:36 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> <430335629.3600.1477139691877.JavaMail.mhammett@ThunderFuck> <20161022125335.GA84013@ussenterprise.ufp.org> <0354f8a1-081e-e662-d6f6-956eb3a07bd9@cisco.com> <37bdf646-338c-d924-2693-529dc435a65a@cisco.com> Message-ID: <580E64E0.9090907@vaxination.ca> Dumb question: If some camera, vaccum cleaner, toothbrush or refrigirator is behind NAT, can it do IP spoofing ? Won't the "from" address be replaced by the CPE router with the proper IP address assigned to that customer so that on the Internet itself, that packet will travel with a real IP routable back to the CPE ? Could mobile phones become a source of such attacks ? Depending on subscription, many are given actiual internet IPs and not NATted, so they could theoretically send packets with spoofed IPs. (would likely require rooted android phones, and how many of those are there ?) Second dumb question: If the number of infected devices in eastern USA is insufficient to have caused that DDoS, can one infer that the attack used an actual IP address instead of the anycast one in order to target the the easter USA hosts irrespective of the location of the infected device ? Could one operate such a host with the "real" IP address in a subnet that has its own BGP announcement, and when there is an attack, one would change the real IP to a different IP address in a different subnet, and drop the route announcement for the first subnet (making those attack packets unroutable at the origin). Is that a viable counter measure ? From duga95 at gmail.com Mon Oct 24 20:06:11 2016 From: duga95 at gmail.com (Julien) Date: Mon, 24 Oct 2016 15:06:11 -0500 Subject: BGP route instabilities In-Reply-To: References: Message-ID: Hi, Not sure if it?s related but we?re seing some of our route advertised on HKIX bouncing on different ISP. Julien. > On 24 Oct 2016, at 12:05, Matthew Huff wrote: > > We saw a slight uptick in routes today (at least since the last time I looked), but a large number of route flaps coming out of the APNIC region. Anyone else notice anything? > > ---- > Matthew Huff | 1 Manhattanville Rd > Director of Operations | Purchase, NY 10577 > > From web at typo.org Mon Oct 24 20:13:04 2016 From: web at typo.org (Wayne Bouchard) Date: Mon, 24 Oct 2016 13:13:04 -0700 Subject: MPLS in the campus Network? In-Reply-To: <6a210ce9-6694-86c3-1690-499767d8a669@seacom.mu> References: <7C7DDF18-2EC7-4414-AC09-9778D8D7B331@arbor.net> <9783dd82-fcc2-1ee2-7a64-d1062e6f2d91@seacom.mu> <9B25E653-391A-4195-ABDA-61FD0350607D@arbor.net> <6a210ce9-6694-86c3-1690-499767d8a669@seacom.mu> Message-ID: <20161024201304.GA39078@spider.typo.org> If the reason for L2 transport is purely customer driven and purely ptp, then a L2 VPN solution would be better than directly transporting the frames. If you don't have to bridge it directly, don't. Keep the core at layer 3 wherever possible. L2 can be very hard to debug when there are issues. On Thu, Oct 20, 2016 at 06:58:51PM +0200, Mark Tinka wrote: > > > On 20/Oct/16 18:45, Roland Dobbins wrote: > > > > > Sure - but it's probably worth revisiting the origins of those > > requirements, and whether there are better alternatives. > > Indeed. > > What we've seen is customers who prefer to manage their own IP layer, > and just need transport. These types of customers tend to be split > between EoDWDM and EoMPLS preferences. Whatever the case, their primary > requirement is control of their IP domain. > > What we're not seeing anymore is l3vpn requirements, particularly on the > back of on-premise IT infrastructure moving into the cloud. We see this > driving a lot of regular IP growth. > > Mark. --- Wayne Bouchard web at typo.org Network Dude http://www.typo.org/~web/ From edugas at unknowndevice.ca Mon Oct 24 20:15:26 2016 From: edugas at unknowndevice.ca (Eric Dugas) Date: Mon, 24 Oct 2016 20:15:26 +0000 Subject: Any Google Cloud people on the list? Message-ID: <2w0r6uwm7dawpmjort5qv60r3-2147483647@mailer.nylas.com> Any Google Cloud people or people really used to work with GC/GCI on the list? Please contact me off-list Thanks Eric ![](https://link.nylas.com/open/4sk3yzfka4ymj01d79p0ldahw/local- f01e8d07-1e4e?r=bmFub2dAbmFub2cub3Jn) From web at typo.org Mon Oct 24 20:23:39 2016 From: web at typo.org (Wayne Bouchard) Date: Mon, 24 Oct 2016 13:23:39 -0700 Subject: Dyn DDoS this AM? In-Reply-To: References: <67268F04-FC66-487E-8AA8-6BE4A6DBDA44@ianai.net> Message-ID: <20161024202338.GB39078@spider.typo.org> See, that's the thing... The key to victory here is to defeat the robots. Take away the anonymity of proxies and trojan amplifiers and enforcement gets a lot easier. Sadly, this war doesn't seem likely to be won anytime soon. Especially since there are State entities using (and even deploying) a number of these systems for use against other States and businesses and/or financial mechanisms. So rather than help the community solve the problem (for their own good, no less!), it is in their interests to perpetuate it. -Wayne On Fri, Oct 21, 2016 at 05:37:08PM -0400, Alain Hebert wrote: > Just a FYI, > > That "horrific trend" has been happening since some techie got > dissed on an IRC channel over 20 years ago. > > He used a bunch of hosted putters to ICMP flood the IRC server. > > Whatever the community is behind, until the carriers decide to wise > up this will keep happening, that is without talking about the > industries being developed around DDoSes events. > > Enjoy your weekend. ( I ain't on call anymore anyway =D ) > > ----- > Alain Hebert ahebert at pubnix.net > PubNIX Inc. > 50 boul. St-Charles > P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 > Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 > > On 10/21/16 11:52, Brian Davies via NANOG wrote: > > +1! > > > > Well said, Patrick. > > > > B > > > > On Friday, October 21, 2016, Patrick W. Gilmore wrote: > > > >> I cannot give additional info other than what???s been on ???public media???. > >> > >> However, I would very much like to say that this is a horrific trend on > >> the Internet. The idea that someone can mention a DDoS then get DDoS???ed Can > >> Not Stand. See Krebs??? on the Democratization of Censorship. See lots of > >> other things. > >> > >> To Dyn and everyone else being attacked: > >> The community is behind you. There are problems, but if we stick together, > >> we can beat these miscreants. > >> > >> To the miscreants: > >> You will not succeed. Search "churchill on the beaches???. It???s a bit > >> melodramatic, but it???s how I feel at this moment. > >> > >> To the rest of the community: > >> If you can help, please do. I know a lot of you are thinking ???what can I > >> do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, > >> that doesn???t help Mirai, but it still helps. There are many other things > >> you can do as well. > >> > >> But a lot of it is just willingness to help. When someone asks you to help > >> trace an attack, do not let the request sit for a while. Damage is being > >> done. Help your neighbor. When someone???s house is burning, your current > >> project, your lunch break, whatever else you are doing is almost certainly > >> less important. If we stick together and help each other, we can - we WILL > >> - win this war. If we are apathetic, we have already lost. > >> > >> > >> OK, enough motivational speaking for today. But take this to heart. Our > >> biggest problem is people thinking they cannot or do not want to help. > >> > >> -- > >> TTFN, > >> patrick > >> > >>> On Oct 21, 2016, at 10:55 AM, Chris Grundemann >> > wrote: > >>> Does anyone have any additional details? Seems to be over now, but I'm > >> very > >>> curious about the specifics of such a highly impactful attack (and it's > >>> timing following NANOG 68)... > >>> > >>> https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- > >> twitter-spotify-reddit/ > >>> -- > >>> @ChrisGrundemann > >>> http://chrisgrundemann.com > >> --- Wayne Bouchard web at typo.org Network Dude http://www.typo.org/~web/ From rfg at tristatelogic.com Mon Oct 24 20:24:59 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Mon, 24 Oct 2016 13:24:59 -0700 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <1722.1477340699@segfault.tristatelogic.com> In message , John Weekes wrote: >On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: jw>>> ... The ISPs behind those IP addresses have jw>>> received notifications via email... rfg>> Just curious... How well is that working out? > >For the IoT botnets, most of the emails are ignored or rejected, because >most go to providers who either quietly bitbucket them or flat-out >reject all abuse emails. Most emails sent to mainland China, for >instance, are in that category (Hong Kong ISPs are somewhat better)... So, given the apparently impracticality of trying to clean up all of these kinds of messes via the good old-fashioned tedious and laborious method of emailing the relevant providers and then just sort-of vaguely hoping that they will -do something- responsible with it, I am just sitting here trying to dream up some sort of generalized long-term fix for -all- of these IoT DDoS type problems. Maybe there just plain isn't any such thing as a general solution to the problem, because it may perhaps be just technically too complex. But I hope no one will begrudge me if I yearn for some sort of Grand Unified Field Theory of IoT security. So, I have a thought... probably worth what you paid for it... and I'm just brave enough to throw it out on the table and then everybody can pile on and tell me what an idiot I am, for this or that perfectly sound technical reason. (I'll say up front that I don't even pretend to understand many of the protocols in use these days, in particular UPnP, and to be frank, I'd never even heard of SSDP until yesterday. So I can't and won't begrudge anybody who tells me that I have my head up... ummm... in the clouds.) So anyway, here are the assumptions/assertions, perhaps wrong, which are my starting point: 1) I am not persuaded that IoT devices have a compelling need to them- selves initiate outbound TCP sessions, ever. (If I'm wrong about this, then I'm sure people here will tell me.) 2) Likewise, I am not persuaded that IoT devices have an absolute and compelling need to do very much in the way of UDP. Yes, I would like my smart XYZ device to always know what time it is, so, you know, a modest amount of NTP traffic is reasonable and to be expected. Other than that however, I don't see a compelling need. If you want to either control or get data out of your IoT device, you can make an inbound TCP connection to it. (Somebody will probably say "Oh, no. We need to stream real-time video out of some of these things, and for that we absolutely have to send the stuff via outbound high-bandwidth UDP." but I am not persuaded that this is either absolutely necessary or even Good, i.e. from the point of view of the legitimate security concerns of the owner of the device.) So, based on the above perhaps flawed assumptions, here is my idea. It is composed of two simple parts: 1) First, I will successfully complete my campaign to be elected King of the World. (Given the current poltical climate, worldwide, this should not be a problem, because I lie a lot.) 2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value. Remember, we're going to have a few billion of these devices online in the coming years. If even and modest subset of these can ever be tricked by an attacker into spewing non-rate-controlled traffic towards an attacker- selected target, then we're gonna have a problem. Regards, rfg From jared at puck.nether.net Mon Oct 24 20:41:39 2016 From: jared at puck.nether.net (Jared Mauch) Date: Mon, 24 Oct 2016 16:41:39 -0400 Subject: Spitballing IoT Security In-Reply-To: <1722.1477340699@segfault.tristatelogic.com> References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: Top posting to provide some clarity: 1) Many IoT devices are connected via some cloud service, think Nest (for example) 2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi, etc that phone out to a site via DHCP option or otherwise. 3) Many IoT devices are something like a set top box, these need access to a CDN or similar to get the content for the users. 4) Many IoT consumers don?t read NANOG, so they also won?t set up a firewall other than to know it is a service impairing tool that must be disabled for their devices to work. 5) There are few/no new lessons here compared to the default install of Redhat 3.0.3 from the late-1990s. I recall it by default installed INN a usenet news server. As a usenet operator, this baffled me as they had insufficient memory, storage etc to do this task, and left security surface quite wide. We must promote secure by default. This means some sort of secondary authentication such as a sticker on the device with default password or requiring the entering of a serial number or basic setup information to work. 6) Devices need to prompt for updates. Apple has sorted this nicely, people are prompted for supported upgrades, disjoint from carriers and other issues. Contrast this with other ecosystems where upgrades require extra steps or have a non-OEM partner involved (eg: Cell Carrier, Cable Co). These devices get less frequent updates, ostensibly for testing but also leave known issues exposed for months or years. 7) Malware can damage systems making them require extra steps to recover them. Whitehats may know some mitigation techniques, but can?t protect you either. Some people have taken a more aggressive approach, eg: https://gitlab.com/rav7teif/linux.wifatch They will block others from infecting your device until it?s rebooted. - Jared > On Oct 24, 2016, at 4:24 PM, Ronald F. Guilmette wrote: > > > In message , > John Weekes wrote: > >> On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > jw>>> ... The ISPs behind those IP addresses have > jw>>> received notifications via email... > rfg>> Just curious... How well is that working out? >> >> For the IoT botnets, most of the emails are ignored or rejected, because >> most go to providers who either quietly bitbucket them or flat-out >> reject all abuse emails. Most emails sent to mainland China, for >> instance, are in that category (Hong Kong ISPs are somewhat better)... > > So, given the apparently impracticality of trying to clean up all of these > kinds of messes via the good old-fashioned tedious and laborious method > of emailing the relevant providers and then just sort-of vaguely hoping > that they will -do something- responsible with it, I am just sitting here > trying to dream up some sort of generalized long-term fix for -all- of > these IoT DDoS type problems. > > Maybe there just plain isn't any such thing as a general solution to the > problem, because it may perhaps be just technically too complex. But I hope > no one will begrudge me if I yearn for some sort of Grand Unified Field > Theory of IoT security. > > So, I have a thought... probably worth what you paid for it... and I'm just > brave enough to throw it out on the table and then everybody can pile on > and tell me what an idiot I am, for this or that perfectly sound technical > reason. (I'll say up front that I don't even pretend to understand many > of the protocols in use these days, in particular UPnP, and to be frank, > I'd never even heard of SSDP until yesterday. So I can't and won't begrudge > anybody who tells me that I have my head up... ummm... in the clouds.) > > So anyway, here are the assumptions/assertions, perhaps wrong, which are my > starting point: > > 1) I am not persuaded that IoT devices have a compelling need to them- > selves initiate outbound TCP sessions, ever. (If I'm wrong about > this, then I'm sure people here will tell me.) > > 2) Likewise, I am not persuaded that IoT devices have an absolute and > compelling need to do very much in the way of UDP. Yes, I would > like my smart XYZ device to always know what time it is, so, you > know, a modest amount of NTP traffic is reasonable and to be expected. > Other than that however, I don't see a compelling need. If you want > to either control or get data out of your IoT device, you can make > an inbound TCP connection to it. > > (Somebody will probably say "Oh, no. We need to stream real-time > video out of some of these things, and for that we absolutely have > to send the stuff via outbound high-bandwidth UDP." but I am not > persuaded that this is either absolutely necessary or even Good, > i.e. from the point of view of the legitimate security concerns of > the owner of the device.) > > So, based on the above perhaps flawed assumptions, here is my idea. It is > composed of two simple parts: > > 1) First, I will successfully complete my campaign to be elected King > of the World. (Given the current poltical climate, worldwide, this > should not be a problem, because I lie a lot.) > > 2) Second, once elected I will decree that in future all new IoT devices, > and also all updates to firmware for existing IoT devices will have, > BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP > session initiation and which also (b) strictly rate-limits all other > protocols to some modest value. > > Remember, we're going to have a few billion of these devices online in the > coming years. If even and modest subset of these can ever be tricked by an > attacker into spewing non-rate-controlled traffic towards an attacker- > selected target, then we're gonna have a problem. > > > Regards, > rfg From Steve.Mikulasik at civeo.com Mon Oct 24 20:42:53 2016 From: Steve.Mikulasik at civeo.com (Steve Mikulasik) Date: Mon, 24 Oct 2016 20:42:53 +0000 Subject: Spitballing IoT Security In-Reply-To: <1722.1477340699@segfault.tristatelogic.com> References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: May as well throw in my idea here too. Can ISPs just block their clients from being reached by CNC servers? If we could get a service like Spamhaus for botnets and have service providers automatically blackhole those CNC IPs. Having this done at the tier 1 level would probably cause some pain to the botnets out there. Rather than pushing customers to secure their stuff (impossible), how about we try to stop communication to the CNC. For example, these guys https://zeustracker.abuse.ch/monitor.php keep track of Zeus, if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. It won't solve the problem, but it could put a dent in it. ISP's might like to implement it since it could cut down on transit costs due to DDoS traffic. -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Ronald F. Guilmette Sent: Monday, October 24, 2016 2:25 PM To: nanog at nanog.org Subject: Spitballing IoT Security In message , John Weekes wrote: >On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: jw>>> ... The ISPs behind those IP addresses have received notifications jw>>> via email... rfg>> Just curious... How well is that working out? > >For the IoT botnets, most of the emails are ignored or rejected, >because most go to providers who either quietly bitbucket them or >flat-out reject all abuse emails. Most emails sent to mainland China, >for instance, are in that category (Hong Kong ISPs are somewhat better)... So, given the apparently impracticality of trying to clean up all of these kinds of messes via the good old-fashioned tedious and laborious method of emailing the relevant providers and then just sort-of vaguely hoping that they will -do something- responsible with it, I am just sitting here trying to dream up some sort of generalized long-term fix for -all- of these IoT DDoS type problems. Maybe there just plain isn't any such thing as a general solution to the problem, because it may perhaps be just technically too complex. But I hope no one will begrudge me if I yearn for some sort of Grand Unified Field Theory of IoT security. So, I have a thought... probably worth what you paid for it... and I'm just brave enough to throw it out on the table and then everybody can pile on and tell me what an idiot I am, for this or that perfectly sound technical reason. (I'll say up front that I don't even pretend to understand many of the protocols in use these days, in particular UPnP, and to be frank, I'd never even heard of SSDP until yesterday. So I can't and won't begrudge anybody who tells me that I have my head up... ummm... in the clouds.) So anyway, here are the assumptions/assertions, perhaps wrong, which are my starting point: 1) I am not persuaded that IoT devices have a compelling need to them- selves initiate outbound TCP sessions, ever. (If I'm wrong about this, then I'm sure people here will tell me.) 2) Likewise, I am not persuaded that IoT devices have an absolute and compelling need to do very much in the way of UDP. Yes, I would like my smart XYZ device to always know what time it is, so, you know, a modest amount of NTP traffic is reasonable and to be expected. Other than that however, I don't see a compelling need. If you want to either control or get data out of your IoT device, you can make an inbound TCP connection to it. (Somebody will probably say "Oh, no. We need to stream real-time video out of some of these things, and for that we absolutely have to send the stuff via outbound high-bandwidth UDP." but I am not persuaded that this is either absolutely necessary or even Good, i.e. from the point of view of the legitimate security concerns of the owner of the device.) So, based on the above perhaps flawed assumptions, here is my idea. It is composed of two simple parts: 1) First, I will successfully complete my campaign to be elected King of the World. (Given the current poltical climate, worldwide, this should not be a problem, because I lie a lot.) 2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value. Remember, we're going to have a few billion of these devices online in the coming years. If even and modest subset of these can ever be tricked by an attacker into spewing non-rate-controlled traffic towards an attacker- selected target, then we're gonna have a problem. Regards, rfg From joquendo at e-fensive.net Mon Oct 24 20:53:25 2016 From: joquendo at e-fensive.net (J. Oquendo) Date: Mon, 24 Oct 2016 15:53:25 -0500 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: <20161024205325.GA72929@e-fensive.net> On Mon, 24 Oct 2016, Steve Mikulasik wrote: > if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. > That would involve someone lifting a finger and implement a config change. Much easier to implement BCP38 or was it RFC 4732? Would never work the moment someone has to lift a finger. /* I think I'll change my position on BCP38. It's pointless to try blocking spoofed source addresses because: * It doesn't solve every single problem * It means more effort for service providers * It requires more CPU processing power * Using it will generate smarter "black hats". https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132.html */ -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM "Where ignorance is our master, there is no possibility of real peace" - Dalai Lama 0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463 From johnl at iecc.com Mon Oct 24 12:40:28 2016 From: johnl at iecc.com (John Levine) Date: 24 Oct 2016 12:40:28 -0000 Subject: Death of the Internet, Film at 11 In-Reply-To: <580E64E0.9090907@vaxination.ca> Message-ID: <20161024124028.2180.qmail@ary.lan> >Dumb question: > >If some camera, vaccum cleaner, toothbrush or refrigirator is behind >NAT, can it do IP spoofing ? Won't the "from" address be replaced by >the CPE router with the proper IP address assigned to that customer so >that on the Internet itself, that packet will travel with a real IP >routable back to the CPE ? Depends on the way the NAT box works. But since Dyn-style attacks don't use IP spoofing, it doesn't really matter. >Could mobile phones become a source of such attacks ? Depends both on the phone and on the network. But since Dyn-style attacks don't use IP spoofing, it doesn't really matter. >If the number of infected devices in eastern USA is insufficient to have >caused that DDoS, can one infer that the attack used an actual IP >address instead of the anycast one in order to target the the eastern USA >hosts irrespective of the location of the infected device ? No. Anycast addresses are real IP addresses. There isn't a "real" address to attack. R's, John From m.waehlisch at fu-berlin.de Mon Oct 24 22:09:26 2016 From: m.waehlisch at fu-berlin.de (Matthias Waehlisch) Date: Tue, 25 Oct 2016 00:09:26 +0200 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: IoT is not a well-defined term. IoT implementations depend on system constraints. These constraints may relate to security (problems/solutions). It would be helpful to be more specific. See https://tools.ietf.org/html/rfc7228, for example. Cheers matthias On Mon, 24 Oct 2016, Jared Mauch wrote: > Top posting to provide some clarity: > > 1) Many IoT devices are connected via some cloud service, think Nest (for example) > 2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi, etc that > phone out to a site via DHCP option or otherwise. > 3) Many IoT devices are something like a set top box, these need access to a CDN > or similar to get the content for the users. > 4) Many IoT consumers don?t read NANOG, so they also won?t set up a firewall other > than to know it is a service impairing tool that must be disabled for their > devices to work. > 5) There are few/no new lessons here compared to the default install of Redhat 3.0.3 > from the late-1990s. I recall it by default installed INN a usenet news server. > As a usenet operator, this baffled me as they had insufficient memory, storage > etc to do this task, and left security surface quite wide. > > We must promote secure by default. This means some sort of secondary authentication > such as a sticker on the device with default password or requiring the entering of > a serial number or basic setup information to work. > 6) Devices need to prompt for updates. > > Apple has sorted this nicely, people are prompted for supported upgrades, disjoint > from carriers and other issues. Contrast this with other ecosystems where upgrades > require extra steps or have a non-OEM partner involved (eg: Cell Carrier, Cable Co). > > These devices get less frequent updates, ostensibly for testing but also leave known > issues exposed for months or years. > 7) Malware can damage systems making them require extra steps to recover them. Whitehats > may know some mitigation techniques, but can?t protect you either. Some people have > taken a more aggressive approach, eg: https://gitlab.com/rav7teif/linux.wifatch > > They will block others from infecting your device until it?s rebooted. > > - Jared > > > On Oct 24, 2016, at 4:24 PM, Ronald F. Guilmette wrote: > > > > > > In message , > > John Weekes wrote: > > > >> On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > > jw>>> ... The ISPs behind those IP addresses have > > jw>>> received notifications via email... > > rfg>> Just curious... How well is that working out? > >> > >> For the IoT botnets, most of the emails are ignored or rejected, because > >> most go to providers who either quietly bitbucket them or flat-out > >> reject all abuse emails. Most emails sent to mainland China, for > >> instance, are in that category (Hong Kong ISPs are somewhat better)... > > > > So, given the apparently impracticality of trying to clean up all of these > > kinds of messes via the good old-fashioned tedious and laborious method > > of emailing the relevant providers and then just sort-of vaguely hoping > > that they will -do something- responsible with it, I am just sitting here > > trying to dream up some sort of generalized long-term fix for -all- of > > these IoT DDoS type problems. > > > > Maybe there just plain isn't any such thing as a general solution to the > > problem, because it may perhaps be just technically too complex. But I hope > > no one will begrudge me if I yearn for some sort of Grand Unified Field > > Theory of IoT security. > > > > So, I have a thought... probably worth what you paid for it... and I'm just > > brave enough to throw it out on the table and then everybody can pile on > > and tell me what an idiot I am, for this or that perfectly sound technical > > reason. (I'll say up front that I don't even pretend to understand many > > of the protocols in use these days, in particular UPnP, and to be frank, > > I'd never even heard of SSDP until yesterday. So I can't and won't begrudge > > anybody who tells me that I have my head up... ummm... in the clouds.) > > > > So anyway, here are the assumptions/assertions, perhaps wrong, which are my > > starting point: > > > > 1) I am not persuaded that IoT devices have a compelling need to them- > > selves initiate outbound TCP sessions, ever. (If I'm wrong about > > this, then I'm sure people here will tell me.) > > > > 2) Likewise, I am not persuaded that IoT devices have an absolute and > > compelling need to do very much in the way of UDP. Yes, I would > > like my smart XYZ device to always know what time it is, so, you > > know, a modest amount of NTP traffic is reasonable and to be expected. > > Other than that however, I don't see a compelling need. If you want > > to either control or get data out of your IoT device, you can make > > an inbound TCP connection to it. > > > > (Somebody will probably say "Oh, no. We need to stream real-time > > video out of some of these things, and for that we absolutely have > > to send the stuff via outbound high-bandwidth UDP." but I am not > > persuaded that this is either absolutely necessary or even Good, > > i.e. from the point of view of the legitimate security concerns of > > the owner of the device.) > > > > So, based on the above perhaps flawed assumptions, here is my idea. It is > > composed of two simple parts: > > > > 1) First, I will successfully complete my campaign to be elected King > > of the World. (Given the current poltical climate, worldwide, this > > should not be a problem, because I lie a lot.) > > > > 2) Second, once elected I will decree that in future all new IoT devices, > > and also all updates to firmware for existing IoT devices will have, > > BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP > > session initiation and which also (b) strictly rate-limits all other > > protocols to some modest value. > > > > Remember, we're going to have a few billion of these devices online in the > > coming years. If even and modest subset of these can ever be tricked by an > > attacker into spewing non-rate-controlled traffic towards an attacker- > > selected target, then we're gonna have a problem. > > > > > > Regards, > > rfg > -- Matthias Waehlisch . Freie Universitaet Berlin, Computer Science .. http://www.cs.fu-berlin.de/~waehl From nanog at ics-il.net Mon Oct 24 22:17:43 2016 From: nanog at ics-il.net (Mike Hammett) Date: Mon, 24 Oct 2016 17:17:43 -0500 (CDT) Subject: Spitballing IoT Security In-Reply-To: <20161024205325.GA72929@e-fensive.net> References: <1722.1477340699@segfault.tristatelogic.com> <20161024205325.GA72929@e-fensive.net> Message-ID: <1512069667.452.1477347462606.JavaMail.mhammett@ThunderFuck> There's a buffer overrun in some software, so let's just remove all passwords (and keys), since they can get in anyway. Just pointing out flawed logic. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "J. Oquendo" To: "Steve Mikulasik" Cc: nanog at nanog.org Sent: Monday, October 24, 2016 3:53:25 PM Subject: Re: Spitballing IoT Security On Mon, 24 Oct 2016, Steve Mikulasik wrote: > if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. > That would involve someone lifting a finger and implement a config change. Much easier to implement BCP38 or was it RFC 4732? Would never work the moment someone has to lift a finger. /* I think I'll change my position on BCP38. It's pointless to try blocking spoofed source addresses because: * It doesn't solve every single problem * It means more effort for service providers * It requires more CPU processing power * Using it will generate smarter "black hats". https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132.html */ -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM "Where ignorance is our master, there is no possibility of real peace" - Dalai Lama 0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463 From hugo at slabnet.com Mon Oct 24 22:21:48 2016 From: hugo at slabnet.com (Hugo Slabbert) Date: Mon, 24 Oct 2016 15:21:48 -0700 Subject: Spitballing IoT Security In-Reply-To: <1512069667.452.1477347462606.JavaMail.mhammett@ThunderFuck> References: <1722.1477340699@segfault.tristatelogic.com> <20161024205325.GA72929@e-fensive.net> <1512069667.452.1477347462606.JavaMail.mhammett@ThunderFuck> Message-ID: <20161024222148.GA29899@bamboo.slabnet.com> It's possible you might have wanted to read the link for the context that pointed this out as sarcastic hyperbole, though the text as-is could (unfortunately) have been read as serious. -- Hugo Slabbert | email, xmpp/jabber: hugo at slabnet.com pgp key: B178313E | also on Signal On Mon 2016-Oct-24 17:17:43 -0500, Mike Hammett wrote: >There's a buffer overrun in some software, so let's just remove all passwords (and keys), since they can get in anyway. > > > > > >Just pointing out flawed logic. > > > > >----- >Mike Hammett >Intelligent Computing Solutions >http://www.ics-il.com > >Midwest-IX >http://www.midwest-ix.com > >----- Original Message ----- > >From: "J. Oquendo" >To: "Steve Mikulasik" >Cc: nanog at nanog.org >Sent: Monday, October 24, 2016 3:53:25 PM >Subject: Re: Spitballing IoT Security > >On Mon, 24 Oct 2016, Steve Mikulasik wrote: > >> if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. >> > >That would involve someone lifting a finger and implement >a config change. Much easier to implement BCP38 or was it >RFC 4732? Would never work the moment someone has to lift >a finger. > >/* >I think I'll change my position on BCP38. It's pointless to try >blocking spoofed source addresses because: > >* It doesn't solve every single problem >* It means more effort for service providers >* It requires more CPU processing power >* Using it will generate smarter "black hats". > >https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132.html > >*/ > > >-- >=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ >J. Oquendo >SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM > >"Where ignorance is our master, there is no possibility of >real peace" - Dalai Lama > >0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 >https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463 > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From nanog at ics-il.net Mon Oct 24 22:24:58 2016 From: nanog at ics-il.net (Mike Hammett) Date: Mon, 24 Oct 2016 17:24:58 -0500 (CDT) Subject: Spitballing IoT Security In-Reply-To: <20161024222148.GA29899@bamboo.slabnet.com> References: <1722.1477340699@segfault.tristatelogic.com> <20161024205325.GA72929@e-fensive.net> <1512069667.452.1477347462606.JavaMail.mhammett@ThunderFuck> <20161024222148.GA29899@bamboo.slabnet.com> Message-ID: <691759865.493.1477347896663.JavaMail.mhammett@ThunderFuck> Oh, yeah, list e-mail usually just gets skimmed through. No time for reading in detail or links. ;-) Sorry. :-\ ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Hugo Slabbert" To: "Mike Hammett" Cc: nanog at nanog.org Sent: Monday, October 24, 2016 5:21:48 PM Subject: Re: Spitballing IoT Security It's possible you might have wanted to read the link for the context that pointed this out as sarcastic hyperbole, though the text as-is could (unfortunately) have been read as serious. -- Hugo Slabbert | email, xmpp/jabber: hugo at slabnet.com pgp key: B178313E | also on Signal On Mon 2016-Oct-24 17:17:43 -0500, Mike Hammett wrote: >There's a buffer overrun in some software, so let's just remove all passwords (and keys), since they can get in anyway. > > > > > >Just pointing out flawed logic. > > > > >----- >Mike Hammett >Intelligent Computing Solutions >http://www.ics-il.com > >Midwest-IX >http://www.midwest-ix.com > >----- Original Message ----- > >From: "J. Oquendo" >To: "Steve Mikulasik" >Cc: nanog at nanog.org >Sent: Monday, October 24, 2016 3:53:25 PM >Subject: Re: Spitballing IoT Security > >On Mon, 24 Oct 2016, Steve Mikulasik wrote: > >> if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. >> > >That would involve someone lifting a finger and implement >a config change. Much easier to implement BCP38 or was it >RFC 4732? Would never work the moment someone has to lift >a finger. > >/* >I think I'll change my position on BCP38. It's pointless to try >blocking spoofed source addresses because: > >* It doesn't solve every single problem >* It means more effort for service providers >* It requires more CPU processing power >* Using it will generate smarter "black hats". > >https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132.html > >*/ > > >-- >=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ >J. Oquendo >SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM > >"Where ignorance is our master, there is no possibility of >real peace" - Dalai Lama > >0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 >https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463 > From emille at abccommunications.com Mon Oct 24 15:11:57 2016 From: emille at abccommunications.com (Emille Blanc) Date: Mon, 24 Oct 2016 08:11:57 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: <4FBAFC2ECF5D6244BA4A26C1C94A1E270D579C1CD9@exchange> -----Original Message----- >From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Richard Holbo >Sent: October-23-16 11:23 PM >To: John Weekes >Cc: NANOG >Subject: Re: Death of the Internet, Film at 11 > >I run/manage the networks for several smallish (in the thousands of >customers) eyeball ISP's and I appreciate a nice "hey you've got a bot" or >"someone is scanning" me notice to my abuse emails. They are useful in >identifying crap that's going on, so for those of you who have the >resources to do that... I appreciate it, we do read them at my networks >and try to do something. > >That said... getting end users to actually fix the broken routers etc. etc. >is NOT easy. Very often we'll notify customers, they will _take their >stuff to the local computer repair guy_ ... or office depo.... and they >will run whatever auto scan they have and say it's all fine. Customer puts >it back in, it's still broke, and they call customer support and want us to >pay for the trip because _their_ expert says it's fine... +1 a hundred times over. Educating customers is difficult to do, assuming they even want to take the time to be educated. I'm not just talking about the residential class here either. "Commercial IT shops" are not much better in most cases -staff tend to all but zone out when you try and explain anything more difficult than un-checking the NetBIOS box on their managed customer device(s), or try and explain why that DMZ checkbox is a bad idea. This usually plays out one of two ways at $dayjob; 1) If you apply too much force to the "You have broken equipment" statement, the customer becomes frustrated and move their broken equipment to $competing-operator's network, and you wind up with a black customer service mark. 2) Customer replaces the device at their expense (It's not _our_ CPE, so why would we?), so typically something cheaper and worse than what they replaced, and they still feel burned. But at least their service gets re-activated. I can recall at least a half-dozen scenarios where the customer actually takes up the problem with the manufacturer. In each of those cases, and they're effectively told to push off because the devices are more than 12mo old and outside the warranty period (paraphrasing customer statements and all that), or similar canned response. Hate to namedrop on a public list, but Linksys, D-Link, Buffalo, Netgear... "There's profit to be had." Granted, It's been a while, at least ~18 months since I've had any such feedback, so perhaps things are better in the wake of all the media attention such vendors have seen from time to time. One case with Asus ended positively after about a week of effort with phonecalls, so it's not all bad. But a week? Not a great turn-around assuming the volumes of devices that would potentially need to be patched/updated/whatever. Only the truly tech-savvy appreciate the advice to repair the problem, but they are probably 1 in 1,000, if that. From suzworldwide at gmail.com Mon Oct 24 17:10:16 2016 From: suzworldwide at gmail.com (Suzanne Woolf) Date: Mon, 24 Oct 2016 13:10:16 -0400 Subject: Dyn DDoS this AM? In-Reply-To: References: <580ABCF7.1030003@vaxination.ca> <20161023224258.3569F576E7D7@rock.dv.isc.org> <6FBDD46B-3FF1-423E-9037-E01D89C63063@gmx.com> Message-ID: <421420CB-5ABC-4CAB-900F-85BE1417A512@gmail.com> > On Oct 24, 2016, at 12:06 PM, Eitan Adler wrote: > > On 24 October 2016 at 01:25, LHC wrote: >> All this TTL talk makes me think. >> >> Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? > > If clients can't get one TTL correct what makes you think they will > get a more complicated two TTL system correct? > ?.To say nothing of resolvers that simply ignore server-side TTLs and set their own. For instance, https://www.icann.org/en/system/files/files/rssac-003-root-zone-ttls-21aug15-en.pdf ?RSSAC 003: RSSAC Report on Root Zone TTLs? will tell you far more than you really want to know about TTLs and caching behavior, and some of it is specific to the root zone, but one of the key observations is "Root zone TTLs appear to not matter to most clients.? Modern large-scale DNS is a fairly complex system. Speculating from here about how it behaved under attack in someone else?s network is interesting, and I look forward to more information from Dyn as they feel they can share it? but DDoS is a big enough fact of life for them and everyone else that if there was a simple answer, I think someone would be making a fortune on it already, or at least have filed the patents. Suzanne (speaking for myself) From nik at neko.id.au Mon Oct 24 18:38:36 2016 From: nik at neko.id.au (Nik Geyer) Date: Mon, 24 Oct 2016 18:38:36 +0000 Subject: BGP route instabilities In-Reply-To: References: Message-ID: Telstra Global was having some issues earlier, we've dropped our BGP sessions with them as a result, could be related. Sent from my iPhone > On 24 Oct 2016, at 1:08 PM, Matthew Huff wrote: > > We saw a slight uptick in routes today (at least since the last time I looked), but a large number of route flaps coming out of the APNIC region. Anyone else notice anything? > > ---- > Matthew Huff | 1 Manhattanville Rd > Director of Operations | Purchase, NY 10577 > > From owen at delong.com Tue Oct 25 02:14:55 2016 From: owen at delong.com (Owen DeLong) Date: Mon, 24 Oct 2016 19:14:55 -0700 Subject: nested prefixes in Internet In-Reply-To: <8D89F96B7AB1B84F9E049CC7BB91BF5CFAF87E4D@RTC-EXCH01.RESERVE.LDS> References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <8D89F96B7AB1B84F9E049CC7BB91BF5CFAF87E4D@RTC-EXCH01.RESERVE.LDS> Message-ID: How do you figure that? Owen > On Oct 24, 2016, at 06:22 , Luke Guillory wrote: > > That only works if the /24 isn't coming from the middle of the /20 block and is on the front end or back end. > > > > Luke Guillory > Network Operations Manager > > Tel: 985.536.1212 > Fax: 985.536.0300 > Email: lguillory at reservetele.com > > Reserve Telecommunications > 100 RTC Dr > Reserve, LA 70084 > > _________________________________________________________________________________________________ > > Disclaimer: > The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . > > -----Original Message----- > From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Martin T > Sent: Monday, October 24, 2016 7:06 AM > To: Matt Buford; Baldur Norddahl > Cc: nanog > Subject: Re: nested prefixes in Internet > > Thank you all for the replies! I'll go with the solution where "ISP A" > announces both /19 prefix and /24 prefix. > > > Martin > > On Thu, Oct 20, 2016 at 1:16 AM, Matt Buford wrote: >> On Mon, Oct 10, 2016 at 2:44 PM, Baldur Norddahl >> >> wrote: >> >> >>> Is that a real problem? In my experience a /24 is honoured almost >>> universially. >>> >> >> Here's a real-world issue I ran into with this. In this case, it >> isn't that someone filtered /24s, but that they didn't have a full >> table (peering routes only plus a default). This resulted in them >> following the less specific route for traffic destined for the /24. >> >> A customer of mine was advertising only a /20 to me and then only a >> more specific /24 was advertised from a remote site of theirs to a >> different ISP. The customer did have a connection between their two >> sites, so in theory it was OK if the traffic arrived on their "wrong" >> link, except that the customer strongly didn't want this to happen, >> thus the inconsistent routes. >> >> This customer found that the remote /24 was unable to access a large >> CDN provider. This CDN told them that a traceroute to the /24 went to >> my network (we peered at an exchange) and was then dropped. >> >> This seemed odd at first, as I confirmed I was not advertising the /24 >> to them so why were they routing it to me? It turned out that the CDN >> provider was running a peer-routes-only network with a default to >> their transit. This meant that they saw the /20 from their peer (me) >> but never saw the /24, since they carried no transit routes. This >> resulted in them routing the entire /20 to me. >> >> My peering router was not willing to route traffic from a peering >> exchange towards transit I had to pay for, so it was dropped. >> >> The customer's split advertisements didn't seem particularly >> unreasonable or invalid, though perhaps they were not the preferred >> setup. It wasn't unreasonable for me to not route from a peering >> exchange to transit. It wasn't unreasonable of the CDN to have a >> peering-and-default routing table. But, those three things together were not compatible. >> >> I called the customer and presented them with my findings and some >> potential options to consider, and consistent advertisements was one >> of those options. The customer strongly wanted incoming traffic to >> arrive directly to the correct location so he didn't want to do that. >> I suggested a possible compromise was for him to advertise both the >> /24 and /20 to me, but use communities so that I never advertised his >> /24 to any upstreams or peering exchanges. That way, if traffic for >> the /24 accidentally hit my network like we were running into, I would >> route it to him and he could pass it to the correct site. It meant >> that a little traffic would arrive at the wrong site in his network >> and have to pass over his back-end link, but at least it would be >> fairly limited. He didn't like this either. He didn't want to split >> the /20 advertisement up to no longer cover the /24 either, I think just because "I shouldn't have to do that!" >> >> The option the customer chose in the end was to use a community on his >> advertisement to instruct my routers to no longer advertise his /20 to >> any peering exchanges at all. That ensured the CDN didn't directly >> send me anything for him (not the /24 or the /20), and the transit >> networks in-between took care of making sure traffic to the /24 didn't >> accidentally end up on my network. While I didn't find it very >> elegant to be shifting traffic from peering exchanges to transit, it >> wasn't a significant amount of traffic and it got him off my back. From randy at psg.com Tue Oct 25 02:30:34 2016 From: randy at psg.com (Randy Bush) Date: Tue, 25 Oct 2016 11:30:34 +0900 Subject: Death of the Internet, Film at 11 In-Reply-To: <20161024124028.2180.qmail@ary.lan> References: <580E64E0.9090907@vaxination.ca> <20161024124028.2180.qmail@ary.lan> Message-ID: >> Could mobile phones become a source of such attacks ? > > Depends both on the phone and on the network. But since Dyn-style > attacks don't use IP spoofing, it doesn't really matter. J-F's question was not about ip spoofing, but rather the infected devices being behind nats. in the states, much broadband is not behind a cgn, but is behind home nats. more mobile is behind cgn [0]. cgns mean fewer visible attacking source addresses. it would be interesting to see the home-soho vs cgn distribution of attacks such as krebs and dyn. >> If the number of infected devices in eastern USA is insufficient to >> have caused that DDoS, can one infer that the attack used an actual >> IP address instead of the anycast one in order to target the the >> eastern USA hosts irrespective of the location of the infected >> device? > > No. Anycast addresses are real IP addresses. true. > There isn't a "real" address to attack. usually false. dns clusters have management interfaces. i suspect the congestion pattern attacking them would be different than that of attack on the anycast; but that is conjecture. randy -- 0 - to get an idea of the vast scale of cgn deployment see philipp's preso of our imc paper from ripe 75 From aaron at heyaaron.com Tue Oct 25 02:51:15 2016 From: aaron at heyaaron.com (Aaron C. de Bruyn) Date: Mon, 24 Oct 2016 19:51:15 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: On Sun, Oct 23, 2016 at 11:23 PM, Richard Holbo wrote: > That said... getting end users to actually fix the broken routers etc. etc. > is NOT easy. Very often we'll notify customers, they will _take their > stuff to the local computer repair guy_ ... or office depo.... and they > will run whatever auto scan they have and say it's all fine. Customer puts > it back in, it's still broke, and they call customer support and want us to > pay for the trip because _their_ expert says it's fine... Totally accurate. I have knowledge of a company in my area that does this. They install tons of 'home security' and 'business security' devices on the cheap. The regular course of action is to plug the device directly into the back of the ISP router and let uPNP handle everything because the installer knows *nothing* about networking, IP addresses, firewalls, routers, etc... The installer also part-times as a home repair tech, and the procedure for any suspected infection is: 1. Plug the infected computer into the business network 2. Boot it up 3. Install AVG Free and run a scan 4. Install Windows Updates 5. Return the computer with a $85 invoice. 6. When the machine comes back a few weeks later, it's obviously not because she failed to remove the virus, it's because the user got a *new* virus. 7. GOTO 1 8. The loop repeats a few times until the customer gets frustrated, then their machine is wiped and reinstalled. I think the only way out of this is for some organization to step up and start issuing competency certificates (Not CompTIA!) that show the tech has demonstrated a particular level of competence to handle IT. Maybe like the Michelin Star system or ASE certs for mechanics. 1-star for people that might be able to plug in power and ethernet, and on the other end a 5-star--where you'd trust them to work on your grandmother's pace-maker while it is in production. Re-test every 2-3 years. Maybe even a group modeled after a lot of 'open governance' projects that you see in open source today. Heck 'NANOG Certified Technician' *does* have a peculiar ring to it... Then have a huge marketing campaign to let home and small business to go to the website to find a local *qualified* technician. The only down-side is that it's ridiculously difficult to test for certain engineering qualities. Not trying to be rude here, but I'm sure lots of people on this list have run into the two types of techs out there: #1 is there for the paycheck. They know how to install Windows Updates and run a virus scan. They probably know one OS (usually Windows) well enough. If they click the mouse and reboot long enough they can get 2 or more computers to talk together. They show zero signs of improvement or change unless it affects their paycheck. #2 is there for the love and curiosity of learning, creating, and exploring. They are constantly learning new stuff, exploring, researching, and tinkering at home because the love figuring out How Stuff Works. (I've found the second type of tech become the best engineers.) The first type is what you run in to when it comes to all these crappy device installs--old vulnerable router, webcams with default password and uPNP forwards from the internet, and infected desktop machines. 10 years ago it was perfectly fine to install AVG and Windows Updates, but because they haven't kept up, they don't realize that doesn't cut it now-a-days. They probably don't know what firmware is, let alone that some of these devices can/should be upgraded. (I caught one installing a DVR based on Windows XP last week. I said "Isn't XP end of life?" "No, I just bought it last week." *facepalm*) Give the customer a reliable way to weed out the dead-wood and get a *good* technician, and most of them will gladly pay more. Or they will eventually after having no end of trouble with the first kind of tech. Sorry for the long rant, but it's either industry self-regulation or government regulation. Something will have to change. -A From lguillory at reservetele.com Tue Oct 25 03:33:43 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Tue, 25 Oct 2016 03:33:43 +0000 Subject: nested prefixes in Internet In-Reply-To: References: <72a17413-8ccf-1425-4953-277322e161af@gmail.com> <8737ka4j1q.fsf@mid.deneb.enyo.de> <67d56263-18da-18ff-3f1e-2e827ba981db@gmail.com> <3367963c-92ca-c5c5-10a5-b640672d35e0@gmail.com> <20161010172445.GD45065@excession.tpb.net> <8D89F96B7AB1B84F9E049CC7BB91BF5CFAF87E4D@RTC-EXCH01.RESERVE.LDS>, Message-ID: Misread the email. Ignore my ignorance. Sent from my iPad > On Oct 24, 2016, at 9:48 PM, Owen DeLong wrote: > > How do you figure that? > > Owen > >> On Oct 24, 2016, at 06:22 , Luke Guillory wrote: >> >> That only works if the /24 isn't coming from the middle of the /20 block and is on the front end or back end. >> >> >> >> Luke Guillory >> Network Operations Manager >> >> Tel: 985.536.1212 >> Fax: 985.536.0300 >> Email: lguillory at reservetele.com >> >> Reserve Telecommunications >> 100 RTC Dr >> Reserve, LA 70084 >> >> _________________________________________________________________________________________________ >> >> Disclaimer: >> The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . >> >> -----Original Message----- >> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Martin T >> Sent: Monday, October 24, 2016 7:06 AM >> To: Matt Buford; Baldur Norddahl >> Cc: nanog >> Subject: Re: nested prefixes in Internet >> >> Thank you all for the replies! I'll go with the solution where "ISP A" >> announces both /19 prefix and /24 prefix. >> >> >> Martin >> >>> On Thu, Oct 20, 2016 at 1:16 AM, Matt Buford wrote: >>> On Mon, Oct 10, 2016 at 2:44 PM, Baldur Norddahl >>> >>> wrote: >>> >>> >>>> Is that a real problem? In my experience a /24 is honoured almost >>>> universially. >>>> >>> >>> Here's a real-world issue I ran into with this. In this case, it >>> isn't that someone filtered /24s, but that they didn't have a full >>> table (peering routes only plus a default). This resulted in them >>> following the less specific route for traffic destined for the /24. >>> >>> A customer of mine was advertising only a /20 to me and then only a >>> more specific /24 was advertised from a remote site of theirs to a >>> different ISP. The customer did have a connection between their two >>> sites, so in theory it was OK if the traffic arrived on their "wrong" >>> link, except that the customer strongly didn't want this to happen, >>> thus the inconsistent routes. >>> >>> This customer found that the remote /24 was unable to access a large >>> CDN provider. This CDN told them that a traceroute to the /24 went to >>> my network (we peered at an exchange) and was then dropped. >>> >>> This seemed odd at first, as I confirmed I was not advertising the /24 >>> to them so why were they routing it to me? It turned out that the CDN >>> provider was running a peer-routes-only network with a default to >>> their transit. This meant that they saw the /20 from their peer (me) >>> but never saw the /24, since they carried no transit routes. This >>> resulted in them routing the entire /20 to me. >>> >>> My peering router was not willing to route traffic from a peering >>> exchange towards transit I had to pay for, so it was dropped. >>> >>> The customer's split advertisements didn't seem particularly >>> unreasonable or invalid, though perhaps they were not the preferred >>> setup. It wasn't unreasonable for me to not route from a peering >>> exchange to transit. It wasn't unreasonable of the CDN to have a >>> peering-and-default routing table. But, those three things together were not compatible. >>> >>> I called the customer and presented them with my findings and some >>> potential options to consider, and consistent advertisements was one >>> of those options. The customer strongly wanted incoming traffic to >>> arrive directly to the correct location so he didn't want to do that. >>> I suggested a possible compromise was for him to advertise both the >>> /24 and /20 to me, but use communities so that I never advertised his >>> /24 to any upstreams or peering exchanges. That way, if traffic for >>> the /24 accidentally hit my network like we were running into, I would >>> route it to him and he could pass it to the correct site. It meant >>> that a little traffic would arrive at the wrong site in his network >>> and have to pass over his back-end link, but at least it would be >>> fairly limited. He didn't like this either. He didn't want to split >>> the /20 advertisement up to no longer cover the /24 either, I think just because "I shouldn't have to do that!" >>> >>> The option the customer chose in the end was to use a community on his >>> advertisement to instruct my routers to no longer advertise his /20 to >>> any peering exchanges at all. That ensured the CDN didn't directly >>> send me anything for him (not the /24 or the /20), and the transit >>> networks in-between took care of making sure traffic to the /24 didn't >>> accidentally end up on my network. While I didn't find it very >>> elegant to be shifting traffic from peering exchanges to transit, it >>> wasn't a significant amount of traffic and it got him off my back. > From randy at psg.com Tue Oct 25 03:59:47 2016 From: randy at psg.com (Randy Bush) Date: Tue, 25 Oct 2016 12:59:47 +0900 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <580E64E0.9090907@vaxination.ca> <20161024124028.2180.qmail@ary.lan> Message-ID: > 0 - to get an idea of the vast scale of cgn deployment see philipp's > preso of our imc paper from ripe 75 let's try again. how about ripe 73. specifically, https://ripe73.ripe.net/archives/video/1244/ randy From bzs at TheWorld.com Tue Oct 25 04:37:59 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 00:37:59 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> Message-ID: <22542.57767.727193.640161@gargle.gargle.HOWL> On October 23, 2016 at 22:56 jw at nuclearfallout.net (John Weekes) wrote: > For the IoT botnets, most of the emails are ignored or rejected, because > most go to providers who either quietly bitbucket them or flat-out > reject all abuse emails. Most emails sent to mainland China, for > instance, are in that category (Hong Kong ISPs are somewhat better). As I've suggested before how much would you attribute this to a lack of English skills by recipients? Are they all sent in English? Just curious but one wonders what most here would do with an abuse complaint sent to them in Chinese? ????? -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From eyeronic.design at gmail.com Tue Oct 25 04:58:36 2016 From: eyeronic.design at gmail.com (Mike Hale) Date: Mon, 24 Oct 2016 21:58:36 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <22542.57767.727193.640161@gargle.gargle.HOWL> References: <97353.1477264758@segfault.tristatelogic.com> <22542.57767.727193.640161@gargle.gargle.HOWL> Message-ID: Run it through Google translate? On Oct 24, 2016 9:40 PM, wrote: > > On October 23, 2016 at 22:56 jw at nuclearfallout.net (John Weekes) wrote: > > For the IoT botnets, most of the emails are ignored or rejected, because > > most go to providers who either quietly bitbucket them or flat-out > > reject all abuse emails. Most emails sent to mainland China, for > > instance, are in that category (Hong Kong ISPs are somewhat better). > > As I've suggested before how much would you attribute this to a lack > of English skills by recipients? Are they all sent in English? > > Just curious but one wonders what most here would do with an abuse > complaint sent to them in Chinese? > > ????? > > -- > -Barry Shein > > Software Tool & Die | bzs at TheWorld.com | > http://www.TheWorld.com > Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD > The World: Since 1989 | A Public Information Utility | *oo* > From rfg at tristatelogic.com Tue Oct 25 05:30:19 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Mon, 24 Oct 2016 22:30:19 -0700 Subject: Dyn DDoS this AM? In-Reply-To: Message-ID: <3709.1477373419@segfault.tristatelogic.com> In message , Alexander Lyamin wrote: >Its not a first time we have and large scale DDoS incident. >Its not a first time we have (a kind of) knee-jerk reaction. I could be wrong, but I think its the first time I've turned on CNN and seen a "heat map" of the incident showing the entire NorthEast / New England area, all the way down to Washington, and parts of California all blanketed in red. So that part, at least, was, ya know, novel. Regards, rfg From mark.tinka at seacom.mu Tue Oct 25 05:39:28 2016 From: mark.tinka at seacom.mu (Mark Tinka) Date: Tue, 25 Oct 2016 07:39:28 +0200 Subject: Fwd: [apops] APRICOT 2017 Call for Presentations In-Reply-To: References: Message-ID: <6a51eaf9-d077-09e5-53f9-037e90b0fe43@seacom.mu> FYI. Mark. -------- Forwarded Message -------- Subject: [apops] APRICOT 2017 Call for Presentations Date: Mon, 17 Oct 2016 17:34:19 +0600 From: Philip Smith To: apops at apops.net Hi everyone, Asia Pacific Regional Internet Conference on Operational Technologies (APRICOT) 27th February - 2nd March 2017, Ho Chi Minh City, Vietnam https://2017.apricot.net CALL FOR PAPERS =============== The APRICOT 2017 Programme Committee is now seeking contributions for Presentations and Tutorials for the APRICOT 2017 Conference. We are looking for presenters who would: - Offer a technical tutorial on an appropriate topic; - Participate in the technical conference sessions as a speaker; - Convene and chair panel sessions of relevant topics. Please submit on-line at: http://papers.apricot.net/user/login.php?event=48 CONFERENCE MILESTONES --------------------- Call for Papers Opens: Now Draft Program Published: As Papers Confirmed Final Deadline for Submissions: 30 January 2016 Final Program Published: 6 February 2017 Final Slides Received: 13 February 2017 *NOTE THAT REGARDLESS OF DEADLINES SLOTS ARE FILLED ON A FIRST COME, FIRST SERVED BASIS* PROGRAMME MATERIAL ------------------ The APRICOT Conference Programme consists of three parts, these being the Peering Forum, Tutorials, and Conference Tracks. Topics proposed must be relevant to Internet Operations and Technologies: - IPv4 / IPv6 Routing and operations - IPv6 deployment and transition technologies - Internet backbone operations - ISP and Carrier services - IXPs and Peering - Research on Internet Operations and Deployment - Software Defined Networking / Network Function Virtualisaton - Network security issues (NSP-SEC, DDoS, Anti-Spam, Anti-Malware) - DNS / DNSSEC - Internet policy (Security, Regulation, Content Management, Addressing, etc) - Access and Transport Technologies, including Cable/DSL, 3G/LTE, wireless, metro ethernet, fibre, MPLS - Content & Service Delivery (Multicast, Voice, Video, "telepresence", Gaming) and Cloud Computing CfP SUBMISSION -------------- Draft slides for both tutorials and conference sessions MUST be provided with CfP submissions otherwise the Programme Committee will be unable to review the submission. For avoidance of doubt this means that submissions which do not include slides will be rejected immediately. For work in progress, the most current information available at time of submission is acceptable. All draft and complete slides must be submitted in PDF format only. Final slides are to be provided by the specified deadline for publication on the APRICOT website. Prospective presenters should note that the majority of speaking slots will be filled well before the final submission deadline. The PC may, at their discretion, retain a limited number of slots up to the final submission deadline for presentations that are exceptionally timely, important, or of critical operational importance. Every year we turn away submissions, due to filling up all available programme slots before the deadline. Presenters should endeavour to get material into the PC sooner rather than later. Any questions or concerns should be addressed to the Programme Committee by e-mail at: pc-chairs at apricot.net We look forward to receiving your presentation proposals. Mark Tinka & Philip Smith Co-Chairs, APRICOT 2017 Programme Committee -- _______________________________________________ apops mailing list apops at apops.net https://mailman.apnic.net/mailman/listinfo/apops Website: www.apops.net . From bzs at TheWorld.com Tue Oct 25 05:52:49 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 01:52:49 -0400 Subject: Spitballing IoT Security In-Reply-To: <1722.1477340699@segfault.tristatelogic.com> References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: <22542.62257.248353.588074@gargle.gargle.HOWL> On October 24, 2016 at 13:24 rfg at tristatelogic.com (Ronald F. Guilmette) wrote: > 1) First, I will successfully complete my campaign to be elected King > of the World. (Given the current poltical climate, worldwide, this > should not be a problem, because I lie a lot.) Too late. -Barry Shein, President & CEO, The World, president at TheWorld.com From jw at nuclearfallout.net Tue Oct 25 06:46:30 2016 From: jw at nuclearfallout.net (John Weekes) Date: Mon, 24 Oct 2016 23:46:30 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <22542.57767.727193.640161@gargle.gargle.HOWL> References: <97353.1477264758@segfault.tristatelogic.com> <22542.57767.727193.640161@gargle.gargle.HOWL> Message-ID: <5f095606-e91b-a16c-0a9f-8fa05e3d1de5@nuclearfallout.net> On 10/24/2016 9:37 PM, bzs at TheWorld.com wrote: > As I've suggested before how much would you attribute this to a lack > of English skills by recipients? I do not think that is a significant factor. Here are some points along those lines: - abuse at cnc-noc.net times out. It's not a matter of whether they know English; they just don't accept the email. - Some Hong Kong ISPs /do/ respond and ask questions. In English. (As does a sampling of other foreign ISPs around the world, including those in Japan, Europe, Russia, etc. -- but mainland China is consistently silent.) - The major Chinese players (including China Mobile, China Telecom, and China Unicom) are some of the largest companies in the world, with just China Mobile having 241,550 employees, according to their 2014 annual report. It is unlikely that they don't have internal translation capabilities. I also have no doubt that they have a large NOC, and they could have a large abuse team (but perhaps choose not to). Large teams are more likely to have some bilingual members, and English is a very common second language. - These large Chinese companies are global companies with PoPs inside the U.S, and peering with U.S. providers. They sell services to, and interact with, companies around the world, including in English. - I have had others tell me that engineers at these Chinese providers contact them for peering upgrades in English -- but that they ignore abuse concerns communicated over the same channels. - Knowing English is not necessary to read tcpdump output, recognize attack traffic, and check IP addresses. Recipients don't have to respond back, so that's mostly what they need. - It's not hard to use online translation services. - It's not hard to respond back and say "Use Mandarin" (or the equivalent, in their preferred language). - I tried sending emails to Russian providers in Russian for a time. I received quite a few responses back along the lines of "please just use English." This has made me think twice about trying to pre-translate. > Are they all sent in English? Currently, mine are. > Just curious but one wonders what most here would do with an abuse > complaint sent to them in Chinese? If I were to receive one in Chinese, I would personally paste it into Google Translate. That is what I do with Japanese complaints/responses, which are the main ones I see that aren't in English. Most others ISPs seem to use straight English, or both English and another language. -John From rfg at tristatelogic.com Tue Oct 25 08:10:31 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Tue, 25 Oct 2016 01:10:31 -0700 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <4246.1477383031@segfault.tristatelogic.com> In message , Jared Mauch wrote: >Top posting to provide some clarity: That's funny. Personally, I have always felt that top posting -destroys- clarity. But as Chaplin Tapman said in Catch-22 "I'm not here to judge you." >1) Many IoT devices are connected via some cloud service, think Nest Yea. And? Are you saying that there is no way to make that connection happen without a kernel that's going to allow unlimited outbound bandwidth to entirely arbitrary targets? >2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi >etc that phone out to a site via DHCP option or otherwise. I admit that I -was not- thinking about such things when I posted my trivial proposal, in large part because I'd never heard of either before now. But that's OK. I've just googled them and I don't think either type of device really negates my simple proposal. Via the simple expedient of redefining the scope of the problem being addressed, I would still like to claim that what I proposed may have some merit, but only for "real" IoT devices. I will now define what I'd like to call "real IoT" devices as "stuff that isn't either general purpose computers *or* stuff like routers." The Ruckus thing and that UBNT UniFi thing appear to me to fall more into the class of devices I'd prefer to call "routers", i.e. stuff whose job it is to move quite a lot of packets, on a routine basis, in the outbound (Internet) direction. (Yes, I'm cheating by definining and/or re-defining my terms, on-the-fly, to exclude stuff that I don't want to be bothered thinking about right now. But maybe that isn't such a bad thing, or even such an intellectually dishonest thing as it may at first appear. It's best not to bite off more than you can chew, and I think it might be useful to solve the problem for a big swath of common "IoT" devices that either are now, or that will soon be on the Internet, even if I can't offer you a solution for, you know, poverty and world hunger at the same time.) >3) Many IoT devices are something like a set top box, these need access >to a CDN or similar to get the content for the users. I thought a bit about that since I posted, and here's my response... Yeabut traffic for -those- things is goona be 99.99% inbound. My Roku needs to connect out to get stuff, and maybe it is even is doing that via TCP. (I really don't know, cuz I never even thought about it before.) But even if my Roku is sucking down content via TCP connections which *it* initiated (i.e. "outbound" TCP connections) if there were a low fixed, maximum -outbound- bandwidth rate limit implemented for this thing, directly in its kernel, I believe that it could still do everything it is supposed to do with no problem. So fuggetabout what I said about totally disallowing outbound TCP. For many IoT devices, actually and totally disallowing outbound TCP connections might be workable... and for those, it would be a prudent and useful kernel-enforced limitation...,but for many others, like my Roku, maybe not. But neither your refrigerator nor my Roku really ever has any need to be sending multi megabits per second outbound. They just don't. I'm -not- suggesting that we put all these things on a diet and give them only 9600 baud to work with. I'm just saying that if they ever hit more than, say, .2 Mbps outbound, in total, over any short period of time, then I think their kernels would be doing the world a favor if they displayed some sort of an error code (where feasible) and if they then started just dropping outbound packets on the floor until things settle down and the -requested- outbound bandwidth drops back below the hard limit. (Of course, all traffic to internal interfaces, local loopback, and RFC1918 addresses should be utterly exempt from the cap.) >4) Many IoT consumers don't read NANOG, so they also won't > set up a firewall other > than to know it is a service impairing tool that must be disabled for >their devices to work. Well, yea, there's that, but also, with IPv6, as I understand the plan, every single device, no matter how lowly or absurd, is gonna get itself plugged right straight into the neural backbone of the entire planet. None of this NAT router or firewall stuff anymore! That's so last millenium. So there's not gonna be nothin' in our bright bright future for Luci and Desi to either enable or disable anymore, even if they wanted to. But here it seems that you're agreeing with me, and even making the case for me. The device itself should be fail-safe. It shouldn't need outside help in order to avoid bursting into flames and endangering innocents like a 1970's Ford Pinto. >5) There are few/no new lessons here compared to the default install of >Redhat 3.0.3 > from the late-1990s. I recall it by default installed INN a usenet >news server. An IoT is -not- a general purpose computer. In the latter case, it is assumed that the owner will "pop the hood" when it comes to the software configuration. In the former case it is assumed that the owner -won't- be doing that, or will have only a very minimalist ability to do so. (CAUTION! Risk of electrical shock! There are NO user-servicable components in this box.) > We must promote secure by default. I love it when people agree with me. >This means some sort of secondary authentication > such as a sticker on the device with default password or requiring >the entering of > a serial number or basic setup information to work. Well, yea. That would be nice too. Force the user to set a unique password. How novel! Swell idea. Marvelous. I'm all for it. And we know that will work, absolutely 100%, until it doesn't, until the day when some pimply-faced teenager in Minsk figures out how to engineer a maliciously crafted JPEG, then uploads it to his Picassa album, and then gets his pal to spam out 50 million spams encouraging Joe Wankers everywhere to view the latest nude photos of Jennifer Lawrence on their Rokus, whereupon we'll all be right back where we were on Friday. >6) Devices need to prompt for updates. See above. That would be nice too. I'm not persuaded that it will actually solve the problem though. (Think about this too: If the software minions hired by the manufacturer to code up the original firmware were careless enough to have created a serious security flaw in the -original- firmware, what makes you think that they are suddenly going to develop superior coding skills and that, thus, any update released by the company will be any more secure than the original flawed version? And that's even assuming that the same guys who coded up the original firmware are tasked to do maintenance fixes, which is most companies, they're not. The original coders all get moved over to the latest project in development, and the company hires one or two 19 year old H1-Bs just off the boat to handle all software maintenance on the "old" products. So nine times out of ten, the guys tasked with fixing the security problem(s) in the "old" products are even more inept dumbasses than the ones who coded up the original security flaws in the first place. How many securty fixes have had to have -their own- subsequent security fixes applied? Plenty.) > Apple has sorted this nicely, people are prompted for supported >upgrades... Umm, I'm getting the book off my shelf that purports to tell me how to disagree without being diagreeable. But I can't seem to find it so I'll just say what I mean: Rubbish! "Apple has sorted this nicely"??? You must be joking! I've got an iPhone 3GS. I went into the local Apple store more than a year ago and asked them how I could get the bloody thing to read and display PDF files. They were very polite, and tried hard to avoid snickering directly in front of me. And they were, of course, only too happy to show me a brand new iPhone 6 while telling me how little it would probably cost in conjunction with "my plan". (Hint: I don't have a plan. I have GoPhone dollars, which don't cost me anything if I don't use them, which, in general, I don't. And if Apple really thinks that I'm going to shell out several hundred real bucks for a piece of electronics that might accidentally end up in the washing machine or simply get ripped off they got another thing coming.) So, it turns out that Apple can't (or rather won't) give me any version of any of the last -3- major releases of IOS for my phone, because they quite rightly have figured out that it is not in their economic interest to do so. So I now own the Apple equivalent of Windows XP. No security fixes, no nothing. So when you say "Apple has sorted this nicely', you'll just have to forgive me if I take issue with that. More to the point however, this illustrates part of the overall problem. Manufacturers and vendors sell stuff to you with the expectation (and hope?) that it's all gonna end up in a landfill someplace within 5 years. They certainly don't plan on releasing any security updates past those five years, and that's even assuming that the company itself lasts that long. (Many don't.) So, six years from today we'll all wake up one morning, turn on CNN, and find out that a maliciously crafted packet sent to 35 million vacation pet feeders has just resulted in the disapperaance from the Internet of the entire continent of South America. So Wolf Blitzer or Brian Krebs will pick up the phone to call the manufacturer to ask if they're going to be releasing a security update for this massive problem, only to find out that the phone number is dead and the company declared bankruptcy three years earlier. This is not a happy ending. If all of the *&^%$# damn stupid vacation pet feeders had originally shipped with outbound rate limits hard-coded in the kernel, maybe this could have been avoided. >7) Malware can damage systems making them require extra steps to recover >them. Whitehats > may know some mitigation techniques, but can't protect you I do so love to end on a note of agreement. I reiterate, fail-safe by design, and outbound rate limits for all IoT things to which such limits can be sensibly applied without damaging or materially degrading functionality... which is to say most of them. Regards, rfg From rfg at tristatelogic.com Tue Oct 25 08:28:11 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Tue, 25 Oct 2016 01:28:11 -0700 Subject: Death of the Internet, Film at 11 In-Reply-To: <4FBAFC2ECF5D6244BA4A26C1C94A1E270D579C1CD9@exchange> Message-ID: <4358.1477384091@segfault.tristatelogic.com> In message <4FBAFC2ECF5D6244BA4A26C1C94A1E270D579C1CD9 at exchange>, Emille Blanc wrote: >I can recall at least a half-dozen scenarios where the customer actually >takes up the problem with the manufacturer. In each of those cases, and >they're effectively told to push off because the devices are more than 12mo >old and outside the warranty period (paraphrasing customer statements and >all that), or similar canned response. Hate to namedrop on a public list, >but Linksys, D-Link, Buffalo, Netgear... "There's profit to be had." >Granted, It's been a while, at least ~18 months since I've had any such >feedback, so perhaps things are better in the wake of all the media >attention such vendors have seen from time to time. Not really. The fundamental economics have not changed. It pays to design and ship things. It doesn't pay to support them afterwards. This isn't going to change. It is common to include "goodwill" on the balance sheet, but unless I'm mistaken, for most companies this does not represent a significant fraction of shareholder equity. Regards, rfg From la at qrator.net Tue Oct 25 08:29:56 2016 From: la at qrator.net (Alexander Lyamin) Date: Tue, 25 Oct 2016 10:29:56 +0200 Subject: Dyn DDoS this AM? In-Reply-To: <3709.1477373419@segfault.tristatelogic.com> References: <3709.1477373419@segfault.tristatelogic.com> Message-ID: Yeah, it sucked to be a Dyn customer that day. However, if you had a backup dns provider, it wasnt that bad. You do realize that collateral effect scale is a property of a target and not attack? My point was that implementing MANRS, while isn't covering all of the spectrum of the attacks that made news this autumn will make at least some of them if not impossible, but harder to execute. And as I said - its work in progress. P.S. Jared Mauch notes regarding uRPF underperformance are correct, but it only shows how rarely its actually used in a real life. uRPF is more then feasible in terms of algorithmical complexity, and this means that bugs can be dealed with. On Tue, Oct 25, 2016 at 7:30 AM, Ronald F. Guilmette wrote: > > In message gmail.com>, > Alexander Lyamin wrote: > > >Its not a first time we have and large scale DDoS incident. > >Its not a first time we have (a kind of) knee-jerk reaction. > > I could be wrong, but I think its the first time I've turned > on CNN and seen a "heat map" of the incident showing the entire > NorthEast / New England area, all the way down to Washington, > and parts of California all blanketed in red. > > So that part, at least, was, ya know, novel. > > > Regards, > rfg > -- Alexander Lyamin CEO | Qrator * Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la at qrator.net From jfmezei_nanog at vaxination.ca Tue Oct 25 08:37:19 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Tue, 25 Oct 2016 04:37:19 -0400 Subject: Spitballing IoT Security In-Reply-To: <4246.1477383031@segfault.tristatelogic.com> References: <4246.1477383031@segfault.tristatelogic.com> Message-ID: <580F19BF.2070002@vaxination.ca> On 2016-10-25 04:10, Ronald F. Guilmette wrote: > If all of the *&^%$# damn stupid vacation pet feeders had originally shipped > with outbound rate limits hard-coded in the kernel, maybe this could have > been avoided. I view this differently. The problem is in allowing inbound connections and going as far as doing UPnP to tell the CPE router to open a inbound door to let hackers loging to that IoT pet feeder to turn it into an agressive DNS destroyer. Then again, you need to have the owner access the pet feeder from the remote beach to feed the dog. One way around this is for the pet feeder to initiate outbound connection to a central server, and have the pet onwer connect to that server to ask the server to send command to his pet feeder to feed the dog. This way, there need not be any inbound connection to the pet feeder. From aledm at qix.co.uk Tue Oct 25 08:49:41 2016 From: aledm at qix.co.uk (Aled Morris) Date: Tue, 25 Oct 2016 09:49:41 +0100 Subject: Spitballing IoT Security In-Reply-To: <580F19BF.2070002@vaxination.ca> References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> Message-ID: On 25 October 2016 at 09:37, Jean-Francois Mezei < jfmezei_nanog at vaxination.ca> wrote: > > One way around this is for the pet feeder to initiate outbound > connection to a central server, and have the pet onwer connect to that > server to ask the server to send command to his pet feeder to feed the dog. > This is pretty common but, IMHO, the worst solution to this problem. It creates a dependence on a cloud service which is typically undocumented (what protocol do they use? where is the server located, China?); a centralised service is a security risk in it's own right (crack one server, own all the pet feeders); and it is liable to disappear when the operator goes out of business, rendering all the products sold useless. A strength of IP is that it is fundamentally a peer-to-peer protocol, please don't break that. NAT broke it but IPv6 can fix it again. There's nothing wrong with accepting incoming connections if the device is secure. If your problem is security, fix that. Don't throw the baby out with the bath water. Aled From marcel.duregards at yahoo.fr Tue Oct 25 09:13:52 2016 From: marcel.duregards at yahoo.fr (marcel.duregards at yahoo.fr) Date: Tue, 25 Oct 2016 11:13:52 +0200 Subject: Internal vulnerability hosts scan to prevent DDOS Message-ID: <580F2250.5030100@yahoo.fr> Dear members, We are a small tier-2 isp and we would like to monitor any potentials risks at our customers ip (we do not want that our AS could be used as a source for DDOS). One of many measures is to scan, on a regular basis, all our IP (PI and PA) to detect any misconfigured customers hosts which could be used for DDOS. The scan should be able to detect misconfigured ddns resolver, ntp server, ssdp host, etc...and any future cool reflection protocol. If any misconfigured hosts is detected, an alarm should raise on an dashboard, and an email send to the customers contact (RIR info), and to our noc. Radar from qrator provide a similar service via email, but we need our own server/VM, with customizables features (like email in differents languages, recall management, acknowledgement request, history (some customers do not take any actions when we inform them, so we would like to have history to put pressure on them)). Does anybody have a solution for that ? Thank in advance for your input. Best regards, -- Marcel From rfg at tristatelogic.com Tue Oct 25 11:26:21 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Tue, 25 Oct 2016 04:26:21 -0700 Subject: Spitballing IoT Security In-Reply-To: <580F19BF.2070002@vaxination.ca> Message-ID: <5372.1477394781@segfault.tristatelogic.com> In message <580F19BF.2070002 at vaxination.ca>, Jean-Francois Mezei wrote: >One way around this is for the pet feeder to initiate outbound >connection to a central server, and have the pet onwer connect to that >server to ask the server to send command to his pet feeder to feed the dog. > >This way, there need not be any inbound connection to the pet feeder. Right. And even that one outbound connection (from the pet feeder to the central server) isn't going to need much in the way of bandwidth. So if the kernel wakes up and notices "Whoa! Wait a minute here! This box is sending out in excess of 100 kilobits per second!" then something is REALLY wrong here. Time to let that ethernet interface take a little nap. Likewise if the kernel notices that the box has just sent out in excess of, say, 100 outbound UDP packets with destination port set to 53 in the last 60 seconds. Time to send DNS to its room for a little "bad behavior" time out! Because in practice, for most or all of the kinds of IoT devices I can think of, (a) they shouldn't be sending out DNS -responses-, ever, and (b) if they are sending out more than, say, 100 outbound DNS -queries- per minute, then the only possible reason they would be doing so is that the box itself has been enslaved and is participating in a DNS reflection attack. The point is that for many (most?) types of IoT devices the engineers who designed the box should be able to tell... or at least predict... what the maximum theoretical outbound bandwidth usage and/or the maximum theoretical outbound packets per second for various protocols and ports ought to be... assuming that the box is functioning "normally", as the original design engineers intended, and not as some miscreant's slave. So then they, the original design engineers, can hard code limits into the kernel... limits that are, say, 25% above these worst case "normal operation" limits, thus allowing for a healthy margin of error. The result would be a box that can't be turned into a weapon, no matter what, at least not without loading up all new (and malicious) firmware, a task which should require at least some direct, hands-on manually fiddling (such as pressing and holding the reset button) in any event. I refer again to the Ford Pinto, and also to the Apollo 1 fire. I don't think that there's really a problem with engineering belt-and-suspenders safeguards into the firmware of IoT things. The only real problem is providing both companies and engineers with the motivation necessary to insure they will do so. Let's hope that nobody has to die before we find a way to manufacture that motivation. (A little temporary bad press for one Chinese manu- facturer of CCTV cameras isn't going to be sufficient, I'm afraid.) Regards, rfg From cboyd at gizmopartners.com Tue Oct 25 11:36:08 2016 From: cboyd at gizmopartners.com (Chris Boyd) Date: Tue, 25 Oct 2016 06:36:08 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: <22542.57767.727193.640161@gargle.gargle.HOWL> References: <97353.1477264758@segfault.tristatelogic.com> <22542.57767.727193.640161@gargle.gargle.HOWL> Message-ID: > On Oct 24, 2016, at 11:37 PM, bzs at TheWorld.com wrote: > > Just curious but one wonders what most here would do with an abuse > complaint sent to them in Chinese? I?ve received a few of these, and if the email included an IP address or domain name on our networks, I?d run the thing through Google Translate and try to figure out what they were on about. Not that hard. ?Chris From cboyd at gizmopartners.com Tue Oct 25 11:51:11 2016 From: cboyd at gizmopartners.com (Chris Boyd) Date: Tue, 25 Oct 2016 06:51:11 -0500 Subject: Spitballing IoT Security In-Reply-To: <4246.1477383031@segfault.tristatelogic.com> References: <4246.1477383031@segfault.tristatelogic.com> Message-ID: <0105FA70-BE62-4F5D-9366-5AE387081ED8@gizmopartners.com> > On Oct 25, 2016, at 3:10 AM, Ronald F. Guilmette wrote: > > An IoT is -not- a general purpose computer. In the latter case, it is > assumed that the owner will "pop the hood" when it comes to the software > configuration. Ah, but they are. In many cases you can ship a product faster and cheaper with an ARM based system running a stripped down Linux and some specialty I/O than building a properly hardened custom microcontroller. Source: Recently went through a round of proposals and bids for a small IoT type product. Also, you probably _don?t_ want the average consumer ?popping the hood? on their PC OS. They will screw something up. Worked in small business IT hell for 8 years, and that was the single most dangerous thing a customer could do. ?Chris From nanog at ics-il.net Tue Oct 25 11:51:10 2016 From: nanog at ics-il.net (Mike Hammett) Date: Tue, 25 Oct 2016 06:51:10 -0500 (CDT) Subject: Dyn DDoS this AM? In-Reply-To: References: <3709.1477373419@segfault.tristatelogic.com> Message-ID: <2061152687.1116.1477396267922.JavaMail.mhammett@ThunderFuck> Side note: I asked Mikrotik and they accepted the feature request of changing their uRPF setting from being universal on the machine to being per-interface (as the kernel supports). That would make it easier for Mikrotik end-user-facing routers to block crap right at the edge, allowing for strict facing customer and loose elsewhere. They haven't implemented it yet, but they accepted the request. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Alexander Lyamin" To: "Ronald F. Guilmette" Cc: "NANOG list" Sent: Tuesday, October 25, 2016 3:29:56 AM Subject: Re: Dyn DDoS this AM? Yeah, it sucked to be a Dyn customer that day. However, if you had a backup dns provider, it wasnt that bad. You do realize that collateral effect scale is a property of a target and not attack? My point was that implementing MANRS, while isn't covering all of the spectrum of the attacks that made news this autumn will make at least some of them if not impossible, but harder to execute. And as I said - its work in progress. P.S. Jared Mauch notes regarding uRPF underperformance are correct, but it only shows how rarely its actually used in a real life. uRPF is more then feasible in terms of algorithmical complexity, and this means that bugs can be dealed with. On Tue, Oct 25, 2016 at 7:30 AM, Ronald F. Guilmette wrote: > > In message gmail.com>, > Alexander Lyamin wrote: > > >Its not a first time we have and large scale DDoS incident. > >Its not a first time we have (a kind of) knee-jerk reaction. > > I could be wrong, but I think its the first time I've turned > on CNN and seen a "heat map" of the incident showing the entire > NorthEast / New England area, all the way down to Washington, > and parts of California all blanketed in red. > > So that part, at least, was, ya know, novel. > > > Regards, > rfg > -- Alexander Lyamin CEO | Qrator * Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la at qrator.net From jared at puck.Nether.net Tue Oct 25 12:48:31 2016 From: jared at puck.Nether.net (Jared Mauch) Date: Tue, 25 Oct 2016 08:48:31 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: <20161025124831.GB29492@puck.nether.net> On Tue, Oct 25, 2016 at 12:09:26AM +0200, Matthias Waehlisch wrote: > IoT is not a well-defined term. Agreed. This is why I call it Internet of Trash. > IoT implementations depend on system constraints. Of course, this is how you see LoWPAN pop up as a possible solution. > These constraints may relate to security (problems/solutions). Yes, and the lack of any bar being set is critical. The problem here isn't too many standards, it's the lack of a consistent one. > It would be helpful to be more specific. > See https://tools.ietf.org/html/rfc7228, for example. Perhaps time for an informational RFC or similar that can be cited as the low bar? The problem with best practices is they aren't, and that's clearly outlined in the most recent weeks events. - Jared > Cheers > matthias > > On Mon, 24 Oct 2016, Jared Mauch wrote: > > > Top posting to provide some clarity: > > > > 1) Many IoT devices are connected via some cloud service, think Nest (for example) > > 2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi, etc that > > phone out to a site via DHCP option or otherwise. > > 3) Many IoT devices are something like a set top box, these need access to a CDN > > or similar to get the content for the users. > > 4) Many IoT consumers don?t read NANOG, so they also won?t set up a firewall other > > than to know it is a service impairing tool that must be disabled for their > > devices to work. > > 5) There are few/no new lessons here compared to the default install of Redhat 3.0.3 > > from the late-1990s. I recall it by default installed INN a usenet news server. > > As a usenet operator, this baffled me as they had insufficient memory, storage > > etc to do this task, and left security surface quite wide. > > > > We must promote secure by default. This means some sort of secondary authentication > > such as a sticker on the device with default password or requiring the entering of > > a serial number or basic setup information to work. > > 6) Devices need to prompt for updates. > > > > Apple has sorted this nicely, people are prompted for supported upgrades, disjoint > > from carriers and other issues. Contrast this with other ecosystems where upgrades > > require extra steps or have a non-OEM partner involved (eg: Cell Carrier, Cable Co). > > > > These devices get less frequent updates, ostensibly for testing but also leave known > > issues exposed for months or years. > > 7) Malware can damage systems making them require extra steps to recover them. Whitehats > > may know some mitigation techniques, but can?t protect you either. Some people have > > taken a more aggressive approach, eg: https://gitlab.com/rav7teif/linux.wifatch > > > > They will block others from infecting your device until it?s rebooted. > > > > - Jared > > > > > On Oct 24, 2016, at 4:24 PM, Ronald F. Guilmette wrote: > > > > > > > > > In message , > > > John Weekes wrote: > > > > > >> On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: > > > jw>>> ... The ISPs behind those IP addresses have > > > jw>>> received notifications via email... > > > rfg>> Just curious... How well is that working out? > > >> > > >> For the IoT botnets, most of the emails are ignored or rejected, because > > >> most go to providers who either quietly bitbucket them or flat-out > > >> reject all abuse emails. Most emails sent to mainland China, for > > >> instance, are in that category (Hong Kong ISPs are somewhat better)... > > > > > > So, given the apparently impracticality of trying to clean up all of these > > > kinds of messes via the good old-fashioned tedious and laborious method > > > of emailing the relevant providers and then just sort-of vaguely hoping > > > that they will -do something- responsible with it, I am just sitting here > > > trying to dream up some sort of generalized long-term fix for -all- of > > > these IoT DDoS type problems. > > > > > > Maybe there just plain isn't any such thing as a general solution to the > > > problem, because it may perhaps be just technically too complex. But I hope > > > no one will begrudge me if I yearn for some sort of Grand Unified Field > > > Theory of IoT security. > > > > > > So, I have a thought... probably worth what you paid for it... and I'm just > > > brave enough to throw it out on the table and then everybody can pile on > > > and tell me what an idiot I am, for this or that perfectly sound technical > > > reason. (I'll say up front that I don't even pretend to understand many > > > of the protocols in use these days, in particular UPnP, and to be frank, > > > I'd never even heard of SSDP until yesterday. So I can't and won't begrudge > > > anybody who tells me that I have my head up... ummm... in the clouds.) > > > > > > So anyway, here are the assumptions/assertions, perhaps wrong, which are my > > > starting point: > > > > > > 1) I am not persuaded that IoT devices have a compelling need to them- > > > selves initiate outbound TCP sessions, ever. (If I'm wrong about > > > this, then I'm sure people here will tell me.) > > > > > > 2) Likewise, I am not persuaded that IoT devices have an absolute and > > > compelling need to do very much in the way of UDP. Yes, I would > > > like my smart XYZ device to always know what time it is, so, you > > > know, a modest amount of NTP traffic is reasonable and to be expected. > > > Other than that however, I don't see a compelling need. If you want > > > to either control or get data out of your IoT device, you can make > > > an inbound TCP connection to it. > > > > > > (Somebody will probably say "Oh, no. We need to stream real-time > > > video out of some of these things, and for that we absolutely have > > > to send the stuff via outbound high-bandwidth UDP." but I am not > > > persuaded that this is either absolutely necessary or even Good, > > > i.e. from the point of view of the legitimate security concerns of > > > the owner of the device.) > > > > > > So, based on the above perhaps flawed assumptions, here is my idea. It is > > > composed of two simple parts: > > > > > > 1) First, I will successfully complete my campaign to be elected King > > > of the World. (Given the current poltical climate, worldwide, this > > > should not be a problem, because I lie a lot.) > > > > > > 2) Second, once elected I will decree that in future all new IoT devices, > > > and also all updates to firmware for existing IoT devices will have, > > > BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP > > > session initiation and which also (b) strictly rate-limits all other > > > protocols to some modest value. > > > > > > Remember, we're going to have a few billion of these devices online in the > > > coming years. If even and modest subset of these can ever be tricked by an > > > attacker into spewing non-rate-controlled traffic towards an attacker- > > > selected target, then we're gonna have a problem. > > > > > > > > > Regards, > > > rfg > > > > > -- > Matthias Waehlisch > . Freie Universitaet Berlin, Computer Science > .. http://www.cs.fu-berlin.de/~waehl -- Jared Mauch | pgp key available via finger from jared at puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine. From netfortius at gmail.com Tue Oct 25 13:14:27 2016 From: netfortius at gmail.com (Stefan) Date: Tue, 25 Oct 2016 08:14:27 -0500 Subject: Other name resolution impacts, as by-products of the Dyn event (e.g. recursive queries limit reached) Message-ID: Have not seen any mentions yet, so thought of asking here: has anybody paid attention to other name resolution issues, even outside the Dyn hosted services directly sourced ones? I am talking about what we observed, as result of: https://community.infoblox.com/t5/Support-Central/Support-Central-KB-118-What-does-quot-no-more-recursive-clients/ba-p/6321 i.e. "active queue" filled with the Dyn caused event, but leaving thusly no "room", at times, for valid recursive queries, inclusive of internal name resolution, where the SOA sits in another tier (e.g. GSLBs). One possible mitigation for such scenarios is: https://community.infoblox.com/t5/Support-Central/Support-Central-KB-3451-Configuring-CLI-commands-for-Automated/ba-p/6327 Any other related events, on other platforms or configurations? ***Stefan From rsk at gsp.org Tue Oct 25 13:26:46 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Tue, 25 Oct 2016 09:26:46 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <85864.1477115622@segfault.tristatelogic.com> References: <85864.1477115622@segfault.tristatelogic.com> Message-ID: <20161025132646.GB16246@gsp.org> On Fri, Oct 21, 2016 at 10:53:42PM -0700, Ronald F. Guilmette wrote: > Recent events, like the Krebs DDoS and the even bigger OVH DDoS, and > today's events make it perfectly clear to even the most blithering of > blithering idiots that network operators, en mass, have to start scanning > their own networks for insecurities. And start monitoring their own networks for *outbound* attacks. Too many people focus exclusively on inbound attacks, never realizing that every attack inbound to them is outbound from somewhere else. ---rsk From bruce.curtis at ndsu.edu Tue Oct 25 15:18:14 2016 From: bruce.curtis at ndsu.edu (Bruce Curtis) Date: Tue, 25 Oct 2016 15:18:14 +0000 Subject: Spitballing IoT Security In-Reply-To: References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> Message-ID: <2927CD81-B785-45ED-A31D-C0946358A167@ndsu.edu> > On Oct 25, 2016, at 3:49 AM, Aled Morris wrote: > > On 25 October 2016 at 09:37, Jean-Francois Mezei < > jfmezei_nanog at vaxination.ca> wrote: >> >> One way around this is for the pet feeder to initiate outbound >> connection to a central server, and have the pet onwer connect to that >> server to ask the server to send command to his pet feeder to feed the dog. >> > > This is pretty common but, IMHO, the worst solution to this problem. > > It creates a dependence on a cloud service which is typically undocumented > (what protocol do they use? where is the server located, China?); a > centralised service is a security risk in it's own right (crack one server, > own all the pet feeders); and it is liable to disappear when the operator > goes out of business, rendering all the products sold useless. > > A strength of IP is that it is fundamentally a peer-to-peer protocol, > please don't break that. NAT broke it but IPv6 can fix it again. > > There's nothing wrong with accepting incoming connections if the device is > secure. If your problem is security, fix that. Don't throw the baby out > with the bath water. > > Aled How about SDP? SDP is most often implemented in a gateway in the network today but there is no reason it couldn?t be implemented in each IoT device. With SDP inbound connections are not allowed until they are authenticated by another box. A good quote from Gartner. "Through the end of 2017, at least 10% of enterprise organizations (up from less than 1% today) will leverage software-defined perimeter (SDP) technology to isolate sensitive environments." http://info.vidder.com/gartner-predicts-2016-security-solutions This is the presentation on SDP from the 2015 Internet2 Tech Exchange. http://meetings.internet2.edu/2015-technology-exchange/detail/10003978/ Videos explaining SDP. https://www.vidder.com/product-videos/ https://www.vidder.com/wp-content/uploads/2016/09/rethinking-connectivity.mp4 https://www.vidder.com/wp-content/uploads/2016/09/spa.mp4 SDP info from another vendor. https://www.cryptzone.com/forms/the-software-defined-perimeter-creating-an-invisible-infrastructure http://www.infosecurityeurope.com/__novadocuments/90951?v=635709327725830000 https://cloudsecurityalliance.org/group/software-defined-perimeter/ https://en.wikipedia.org/wiki/Software_Defined_Perimeter https://cloudsecurityalliance.org/media/news/cloud-security-alliance-to-host-third-software-defined-perimeter-sdp-hackathon-top-prize-of-10000-available/ " no one was able to circumvent even the first of the five SDP security controls layers (single packet authorization protocol), despite more than 5 billion packets being fired at the SDP.? https://www.vidder.com/resources/docs/CSA-Verizon-Vidder-Hackathon4-Reliability.pdf http://www.networkworld.com/article/3053561/security/learning-about-sdp-via-google-beyondcorp.html https://www.sdxcentral.com/articles/news/software-defined-perimeter-remains-undefeated-in-hackathon/2015/08/ --- Bruce Curtis bruce.curtis at ndsu.edu Certified NetAnalyst II 701-231-8527 North Dakota State University From bzs at TheWorld.com Tue Oct 25 18:27:07 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 14:27:07 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <97353.1477264758@segfault.tristatelogic.com> <22542.57767.727193.640161@gargle.gargle.HOWL> Message-ID: <22543.41979.515592.561510@gargle.gargle.HOWL> The problem is first you have to even recognize it's an abuse complaint worth spending another second over. For example one gets a lot of spam to abuse addresses, or I do anyhow. And much of it seems to be in some character set other than Latin-1. On October 24, 2016 at 21:58 eyeronic.design at gmail.com (Mike Hale) wrote: > Run it through Google translate? > > > On Oct 24, 2016 9:40 PM, wrote: > > > On October 23, 2016 at 22:56 jw at nuclearfallout.net (John Weekes) wrote: > ?> For the IoT botnets, most of the emails are ignored or rejected, because > ?> most go to providers who either quietly bitbucket them or flat-out > ?> reject all abuse emails. Most emails sent to mainland China, for > ?> instance, are in that category (Hong Kong ISPs are somewhat better). > > As I've suggested before how much would you attribute this to a lack > of English skills by recipients? Are they all sent in English? > > Just curious but one wonders what most here would do with an abuse > complaint sent to them in Chinese? > > ????? > > -- > ? ? ? ? -Barry Shein > > Software Tool & Die? ? | bzs at TheWorld.com? ? ? ? ? ? ?| http:// > www.TheWorld.com > Purveyors to the Trade | Voice: +1 617-STD-WRLD? ? ? ?| 800-THE-WRLD > The World: Since 1989? | A Public Information Utility | *oo* > -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Tue Oct 25 18:48:48 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 14:48:48 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <5f095606-e91b-a16c-0a9f-8fa05e3d1de5@nuclearfallout.net> References: <97353.1477264758@segfault.tristatelogic.com> <22542.57767.727193.640161@gargle.gargle.HOWL> <5f095606-e91b-a16c-0a9f-8fa05e3d1de5@nuclearfallout.net> Message-ID: <22543.43280.85931.31245@gargle.gargle.HOWL> On October 24, 2016 at 23:46 jw at nuclearfallout.net (John Weekes) wrote: > > Are they all sent in English? > > Currently, mine are. > > > Just curious but one wonders what most here would do with an abuse > > complaint sent to them in Chinese? > > If I were to receive one in Chinese, I would personally paste it into > Google Translate. That is what I do with Japanese complaints/responses, > which are the main ones I see that aren't in English. Most others ISPs > seem to use straight English, or both English and another language. As I said in a previous note first one would have to even recognize that a note written in Chinese (or Urdu for that matter, non-Latin-1) is a valid abuse note worth spending another moment looking at. I wonder how many have spam filters which pretty much block emails in non-Latin-1 character sets? It's certainly an easy setting on spamassassin for example, the easiest being to just choose English or maybe there's a Latin-1 choice and default score any other language and charset very high (i.e., as likely spam.) Even ignoring that possibility how many in other countries even agree with the assumptions underlying this complaint about their not reacting to various abuses? Maybe they just think such complaints are silly and/or way out of their hands (i.e., whoever is reading that abuse complaint)? In the US and similar countries we tend to use as reference points we tend to have evolved short-circuit mechanisms to respond to serious abuse events. For all I know all they can do is submit a 20 page highly stylized request to the Ministry of Information to consider among the 50,000 other requests from libraries, book publishers, minor political parties, etc at the next people's congress in August before any action, even an email response as it would imply a policy, can be taken. Which just might be a little frustrating to the front line. Or would be frustrating to us. For many of them it's just how things work, believe me when I say I've run into that powerless fatalism personally. Taking individual initiative is not a universal virtue. More importantly is there any attempt at a global meeting of the minds on what a notable problem and appropriate reaction, including responding to abuse reports, even is? My guess having dealt with some amount of international internet politics is: Many here might be very, very surprised if they tried. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Tue Oct 25 18:57:18 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 14:57:18 -0400 Subject: Spitballing IoT Security In-Reply-To: <4246.1477383031@segfault.tristatelogic.com> References: <4246.1477383031@segfault.tristatelogic.com> Message-ID: <22543.43790.908297.726367@gargle.gargle.HOWL> On October 25, 2016 at 01:10 rfg at tristatelogic.com (Ronald F. Guilmette) wrote: > > In message , > Jared Mauch wrote: > > >Top posting to provide some clarity: > > That's funny. Personally, I have always felt that top posting -destroys- > clarity. But as Chaplin Tapman said in Catch-22 "I'm not here to judge > you." FWIW I prefer top-posting as I assume anyone interested in the thread has read the note being responded to and it's only there for someone late to class or looking for perhaps a misunderstanding or mismatch between the new text and the quoted text. In my ideal world there wouldn't even be all this quoting sent back and forth. It'd just become a type of link perhaps like the HTML #tag syntax to highlight a specific region of text, which one could click if they need it. I'm aware that some MUA's will hide quoted text and only expand it if asked which is something of a one-sided compromise. One-sided in that a #tag scheme would require cooperation of some web site out there to format and hold the target links while just hiding quoted text can be done entirely within your phone or whatever. But on some lists I'm on, not this one particularly, there are a lot of notes going back and forth which are >1MB of nested quoting and the only new text is "I agree!" or "+1". Seems dumb but hey disk is cheap and so is talk. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Tue Oct 25 19:05:12 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Tue, 25 Oct 2016 15:05:12 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: <4358.1477384091@segfault.tristatelogic.com> References: <4FBAFC2ECF5D6244BA4A26C1C94A1E270D579C1CD9@exchange> <4358.1477384091@segfault.tristatelogic.com> Message-ID: <22543.44264.692741.965887@gargle.gargle.HOWL> On October 25, 2016 at 01:28 rfg at tristatelogic.com (Ronald F. Guilmette) wrote: > > The fundamental economics have not changed. It pays to design and ship > things. It doesn't pay to support them afterwards. This isn't going to > change. > > It is common to include "goodwill" on the balance sheet, but unless I'm > mistaken, for most companies this does not represent a significant > fraction of shareholder equity. I suppose that's the "one born every minute!" theory of product development. And there's truth to it no doubt. Marketing travels a lot faster and more efficiently than awareness of poor support or quality issues is another way to state that. Which is one reason why product areas tend to evolve regulatory structures both public (e.g., US FTC) and private (e.g., Underwriters Laboratory) thus getting some sort of quality and support push-back other than the bare-knuckle market. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From gary at baribault.net Tue Oct 25 17:41:17 2016 From: gary at baribault.net (Gary Baribault) Date: Tue, 25 Oct 2016 13:41:17 -0400 Subject: How to find all of an ISP's ASNs Message-ID: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Hi folks, how to I find all ASNs that belong to an ISP? I want to block access to my IoT cameras from the world other than the two local major ISPs (keeping last Friday in mind!) Gary B From colinj at gt86car.org.uk Tue Oct 25 19:27:27 2016 From: colinj at gt86car.org.uk (colin johnston) Date: Tue, 25 Oct 2016 20:27:27 +0100 Subject: How to find all of an ISP's ASNs In-Reply-To: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: > On 25 Oct 2016, at 18:41, Gary Baribault wrote: > > Hi folks, how to I find all ASNs that belong to an ISP? I want to block access to my IoT cameras from the world other than the two local major ISPs (keeping last Friday in mind!) > > Gary B > > ripe atlas has this info Colin From lguillory at reservetele.com Tue Oct 25 19:30:15 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Tue, 25 Oct 2016 19:30:15 +0000 Subject: How to find all of an ISP's ASNs In-Reply-To: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: <8D89F96B7AB1B84F9E049CC7BB91BF5CFAF8AFF0@RTC-EXCH01.RESERVE.LDS> Can search here as well. http://as-rank.caida.org/ Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Gary Baribault Sent: Tuesday, October 25, 2016 12:41 PM To: nanog at nanog.org Subject: How to find all of an ISP's ASNs Hi folks, how to I find all ASNs that belong to an ISP? I want to block access to my IoT cameras from the world other than the two local major ISPs (keeping last Friday in mind!) Gary B From rdobbins at arbor.net Tue Oct 25 19:34:15 2016 From: rdobbins at arbor.net (Roland Dobbins) Date: Wed, 26 Oct 2016 02:34:15 +0700 Subject: How to find all of an ISP's ASNs In-Reply-To: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: On 26 Oct 2016, at 0:41, Gary Baribault wrote: > other than the two local major ISPs (keeping last Friday in mind!) . . . why would you want to expose them to the public Internet at all? There are many, many reasons not to do so. ----------------------------------- Roland Dobbins From dhia at opendns.com Tue Oct 25 19:35:47 2016 From: dhia at opendns.com (Dhia Mahjoub) Date: Tue, 25 Oct 2016 12:35:47 -0700 Subject: How to find all of an ISP's ASNs In-Reply-To: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: Hello, You can use this http://www.cidr-report.org/as2.0/autnums.html Best Dhia Mahjoub, PhD Technical Leader, OpenDNS, part of Cisco On Tue, Oct 25, 2016 at 10:41 AM, Gary Baribault wrote: > Hi folks, how to I find all ASNs that belong to an ISP? I want to block > access to my IoT cameras from the world other than the two local major ISPs > (keeping last Friday in mind!) > > Gary B > > > From gary at baribault.net Tue Oct 25 22:42:43 2016 From: gary at baribault.net (Gary Baribault) Date: Tue, 25 Oct 2016 18:42:43 -0400 Subject: How to find all of an ISP's ASNs In-Reply-To: References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: <322ee572-4842-85f4-560d-ab7f752fe20a@baribault.net> Thanks folks, much appreciated Gary B On 25/10/16 03:35 PM, Dhia Mahjoub wrote: > Hello, > > You can use this > http://www.cidr-report.org/as2.0/autnums.html > > > Best > > > Dhia Mahjoub, PhD > Technical Leader, OpenDNS, part of Cisco > > > On Tue, Oct 25, 2016 at 10:41 AM, Gary Baribault > wrote: > > Hi folks, how to I find all ASNs that belong to an ISP? I want to > block access to my IoT cameras from the world other than the two > local major ISPs (keeping last Friday in mind!) > > Gary B > > > From larrysheldon at cox.net Tue Oct 25 23:54:22 2016 From: larrysheldon at cox.net (Larry Sheldon) Date: Tue, 25 Oct 2016 18:54:22 -0500 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> Message-ID: On 10/25/2016 08:26, Rich Kulawiec wrote: > On Fri, Oct 21, 2016 at 10:53:42PM -0700, Ronald F. Guilmette wrote: >> Recent events, like the Krebs DDoS and the even bigger OVH DDoS, and >> today's events make it perfectly clear to even the most blithering of >> blithering idiots that network operators, en mass, have to start scanning >> their own networks for insecurities. > > And start monitoring their own networks for *outbound* attacks. Too many > people focus exclusively on inbound attacks, never realizing that every > attack inbound to them is outbound from somewhere else. What is it? 20 years? since the first time I was banned from NANOG for saying that the world would be a nicer place if EVERY true router refused to forward a packet whose SOURCE could not be reached from the port question. (May not be stated clearly, but idea seems simple enough: If the proposed ICMP message would not be routed to the port the packet came from, the best plan is probably to log the event and drop the ICMP and the rogue packet on the floor.) -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From yang.yu.list at gmail.com Wed Oct 26 00:14:41 2016 From: yang.yu.list at gmail.com (Yang Yu) Date: Tue, 25 Oct 2016 19:14:41 -0500 Subject: How to find all of an ISP's ASNs In-Reply-To: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: as-set if they keep their routing registry updated? something like this http://bgp.he.net/irr/as-set/AS-RR-Res Normally I use IRR Explorer, but somehow the return is empty http://irrexplorer.nlnog.net/search/AS-RR-Res Yang On Tue, Oct 25, 2016 at 12:41 PM, Gary Baribault wrote: > Hi folks, how to I find all ASNs that belong to an ISP? I want to block > access to my IoT cameras from the world other than the two local major ISPs > (keeping last Friday in mind!) > > Gary B > > From hank at efes.iucc.ac.il Wed Oct 26 04:03:27 2016 From: hank at efes.iucc.ac.il (Hank Nussbacher) Date: Wed, 26 Oct 2016 07:03:27 +0300 Subject: How to find all of an ISP's ASNs In-Reply-To: References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: On 26/10/2016 03:14, Yang Yu wrote: > as-set if they keep their routing registry updated? > > something like this > http://bgp.he.net/irr/as-set/AS-RR-Res and if that doesn't work try: http://bgp.he.net/AS3356#_graph4 [replace the ASN with the ASN of your choice to see the interconnections.] -Hank > > Normally I use IRR Explorer, but somehow the return is empty > http://irrexplorer.nlnog.net/search/AS-RR-Res > > > Yang > > On Tue, Oct 25, 2016 at 12:41 PM, Gary Baribault wrote: >> Hi folks, how to I find all ASNs that belong to an ISP? I want to block >> access to my IoT cameras from the world other than the two local major ISPs >> (keeping last Friday in mind!) >> >> Gary B >> >> From Valdis.Kletnieks at vt.edu Wed Oct 26 05:30:02 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Wed, 26 Oct 2016 01:30:02 -0400 Subject: Death of the Internet, Film at 11 In-Reply-To: References: <85864.1477115622@segfault.tristatelogic.com> Message-ID: <22243.1477459802@turing-police.cc.vt.edu> On Tue, 25 Oct 2016 18:54:22 -0500, Larry Sheldon said: > What is it? 20 years? since the first time I was banned from NANOG for > saying that the world would be a nicer place if EVERY true router > refused to forward a packet whose SOURCE could not be reached from the > port question. (May not be stated clearly, but idea seems simple > enough: If the proposed ICMP message would not be routed to the port > the packet came from, the best plan is probably to log the event and > drop the ICMP and the rogue packet on the floor.) That's not going to work when there's asymmetric routing. Say you get an inbound packet from eth0 and the routing table says you should send it out on eth2. However, it has DF set and eth2 has a smaller MTU, so you need to send back an ICMP FRAG reply. Now, do you send it out, or do you create a PMTUD black hole by dropping the reply because your local table says the source is routed out eth1? Hint: there's a difference between strict uRPF and loose uRPF. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From rsk at gsp.org Wed Oct 26 12:06:34 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Wed, 26 Oct 2016 08:06:34 -0400 Subject: Spitballing IoT Security In-Reply-To: <1722.1477340699@segfault.tristatelogic.com> References: <1722.1477340699@segfault.tristatelogic.com> Message-ID: <20161026120634.GA20735@gsp.org> On Mon, Oct 24, 2016 at 01:24:59PM -0700, Ronald F. Guilmette wrote: > 2) Second, once elected I will decree that in future all new IoT devices, > and also all updates to firmware for existing IoT devices will have, > BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP > session initiation and which also (b) strictly rate-limits all other > protocols to some modest value. I like this idea. But unfortunately, I think it has no chance of succeeding. The makers of IoT devices are falling all over themselves to rush products to market as quickly as possible in order to maximize their profits. They have no time for security. They don't concern themselves with privacy implications. They don't run networks so they don't care about the impact their devices may have on them. They don't care about liability: many of them are effectively immune because suing them would mean trans-national litigation, which is tedious and expensive. (And even if they lost: they'd dissolve and reconstitute as another company the next day.) They don't even care about each other -- I'm pretty sure we're rapidly approaching the point where toasters will be used to attack garage door openers and washing machines. I think our working assumption should be that there will be zero cooperation from the IoT vendors. (Yeah, once in a while one might actually step up, but that will merely be a happy anomaly.) After all, why should they care? It doesn't impact their profits, and profits are all they care about. They're not the ones fielding support calls or frantically trying to stop a DoS or trying to work out a mitigation strategy or participating in this discussion thread. So they don't care. They don't have to. ---rsk From esr at thyrsus.com Wed Oct 26 12:30:43 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Wed, 26 Oct 2016 08:30:43 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161026120634.GA20735@gsp.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> Message-ID: <20161026123043.GA10916@thyrsus.com> Rich Kulawiec : > I think our working assumption should be that there will be zero cooperation > from the IoT vendors. (Yeah, once in a while one might actually step up, > but that will merely be a happy anomaly.) I agree. There is, however, a chokepoint we have more hope of getting decent software deployed to. I refer to home and small-business routers. OpenWRT and kin are already minor but significant players here. And there's an NRE-minimization aregument we can make for router manufacturers to use rebranded versions rather than rolling their own crappy firmware. I think the anti-IoT-flood strategy that makes the most sense is: 1. Push open-source firmware that doesn't suck to the vendors with a cost- and risk-minimization pitch. 2. Ship it with egress filters. (And telnet blocked.) It wouldn't be technically very difficult to make the firmware rate-limit outbound connections. Cute trick: if we unlimit any local IP address that is a port-forwarding target, most users will never notice because their browser sessions won't be effected. -- Eric S. Raymond From mel at beckman.org Wed Oct 26 15:45:11 2016 From: mel at beckman.org (Mel Beckman) Date: Wed, 26 Oct 2016 15:45:11 +0000 Subject: Spitballing IoT Security In-Reply-To: <20161026123043.GA10916@thyrsus.com> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026123043.GA10916@thyrsus.com> Message-ID: Eric, I agree that the home router is a viable choke point, and even though we can?t quickly roll out new firmware, if we had started this ten years ago we?d be done by now! So this is the ten-year plan, but still worth doing. I also really like the idea of offering open source options to vendors, many of whom seem to illegally take that privilege anyway. A key fast-path component, though, is in my opinion a new RFC for IoT security best practices, and probably some revisions to UPNP. The IoT RFC would spell out basic rules for safe devices: no back doors, no default passwords, no gratuitous inbound connections, etc. It would also make encryption a requirement, and limit how existing UPNP is deployed to prevent unnecessarily exposing vulnerable TCP/UDP ports to the wild. With this RFC in hand, and an appropriate splashy icon for vendor packaging (?RFC 9999 ThingSafe!?), vendors will have a competitive reason for compliance as a market differentiator, whether they deploy with open-source or proprietary code. This need not add a lot to R&D costs either, as enterprising embedded system designers will now be motivated to delivering packaged ThingSafe! solutions, such as embedded wireless controllers, cellular modems, etc. Your open-source approach seems to me something that could be started today, and that has no downside. The RFC could give it the imprimatur of a standard, which makes the plan easier for vendors to sell to their thrift-minded management. -mel > On Oct 26, 2016, at 5:30 AM, Eric S. Raymond wrote: > > Rich Kulawiec : >> I think our working assumption should be that there will be zero cooperation >> from the IoT vendors. (Yeah, once in a while one might actually step up, >> but that will merely be a happy anomaly.) > > I agree. > > There is, however, a chokepoint we have more hope of getting decent software > deployed to. I refer to home and small-business routers. OpenWRT and kin > are already minor but significant players here. And there's an NRE-minimization > aregument we can make for router manufacturers to use rebranded versions > rather than rolling their own crappy firmware. > > I think the anti-IoT-flood strategy that makes the most sense is: > > 1. Push open-source firmware that doesn't suck to the vendors with a > cost- and risk-minimization pitch. > > 2. Ship it with egress filters. (And telnet blocked.) > > It wouldn't be technically very difficult to make the firmware > rate-limit outbound connections. Cute trick: if we unlimit any > local IP address that is a port-forwarding target, most users > will never notice because their browser sessions won't be effected. > -- > Eric S. Raymond From bicknell at ufp.org Wed Oct 26 17:19:07 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Wed, 26 Oct 2016 10:19:07 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161026120634.GA20735@gsp.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> Message-ID: <20161026171907.GA84841@ussenterprise.ufp.org> In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: > The makers of IoT devices are falling all over themselves to rush products > to market as quickly as possible in order to maximize their profits. They > have no time for security. They don't concern themselves with privacy > implications. They don't run networks so they don't care about the impact > their devices may have on them. They don't care about liability: many of > them are effectively immune because suing them would mean trans-national > litigation, which is tedious and expensive. (And even if they lost: > they'd dissolve and reconstitute as another company the next day.) > They don't even care about each other -- I'm pretty sure we're rapidly > approaching the point where toasters will be used to attack garage door > openers and washing machines. You are correct. I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused. Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue. Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From jfmezei_nanog at vaxination.ca Wed Oct 26 17:30:38 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Wed, 26 Oct 2016 13:30:38 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161026171907.GA84841@ussenterprise.ufp.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <5810E83E.4080403@vaxination.ca> While I agree that fixing home routers is the best approach, something bugs me. If an IoT vendor doesn't even know that its devices have telnet or ssh enabled by default (and hence, no management interface to change passwords) and only focuses on the web interface it has added , then how come the kernel would be "UPnP" the telnet port to tell the router to send inbound telnet to that device ? And how do routers deal with multiple cameras each sending a "send port 23 requests to me" ? I can understand a computer sending a UPnP request when you start a game to tell router to forward inbound calls to a certain port to that computer/app. But for IoT devices that are on all the time, there should be static setup, not UPnP. From jordi.palet at consulintel.es Wed Oct 26 18:53:51 2016 From: jordi.palet at consulintel.es (JORDI PALET MARTINEZ) Date: Wed, 26 Oct 2016 20:53:51 +0200 Subject: Spitballing IoT Security In-Reply-To: <20161026171907.GA84841@ussenterprise.ufp.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: Exactly, I was arguing exactly the same with some folks this week during the RIPE meeting. The same way that certifications are needed to avoid radio interferences, etc., and if you don?t pass those certifications, you can?t sell the products in some countries (or regions in case of EU for example), authorities should make sure that those certifications have a broader scope, including security and probably some other features to ensure that in case something is discovered in the future, they can be updated. Yes, that means cost, but a few thousand dollars of certification price increase, among thousands of millions of devices of the same model being manufactured, means a few cents for each unit. Even if we speak about 1 dollar per each product being sold, it is much cheaper than the cost of not doing it and paying for damages, human resources, etc., when there is a security breach. Regards, Jordi -----Mensaje original----- De: NANOG en nombre de Leo Bicknell Organizaci?n: United Federation of Planets Responder a: Fecha: mi?rcoles, 26 de octubre de 2016, 19:19 Para: Asunto: Re: Spitballing IoT Security In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: > The makers of IoT devices are falling all over themselves to rush products > to market as quickly as possible in order to maximize their profits. They > have no time for security. They don't concern themselves with privacy > implications. They don't run networks so they don't care about the impact > their devices may have on them. They don't care about liability: many of > them are effectively immune because suing them would mean trans-national > litigation, which is tedious and expensive. (And even if they lost: > they'd dissolve and reconstitute as another company the next day.) > They don't even care about each other -- I'm pretty sure we're rapidly > approaching the point where toasters will be used to attack garage door > openers and washing machines. You are correct. I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused. Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue. Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ ********************************************** IPv4 is over Are you ready for the new Internet ? http://www.consulintel.es The IPv6 Company This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited. From esr at thyrsus.com Wed Oct 26 19:40:40 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Wed, 26 Oct 2016 15:40:40 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026123043.GA10916@thyrsus.com> Message-ID: <20161026194040.GA16649@thyrsus.com> Mel Beckman : > I also really like the idea of offering open source options to vendors, many of whom seem to illegally take that privilege anyway. A key fast-path component, though, is in my opinion a new RFC for IoT security best practices, and probably some revisions to UPNP. > > The IoT RFC would spell out basic rules for safe devices: no back doors, no default passwords, no gratuitous inbound connections, etc. It would also make encryption a requirement, and limit how existing UPNP is deployed to prevent unnecessarily exposing vulnerable TCP/UDP ports to the wild. With this RFC in hand, and an appropriate splashy icon for vendor packaging (?RFC 9999 ThingSafe!?), vendors will have a competitive reason for compliance as a market differentiator, whether they deploy with open-source or proprietary code. That is a good idea and I am officially adopting it as part of the Evil Master Plan for World Domination. :-) I may recruit you to help draft the RFC. -- Eric S. Raymond From deleskie at gmail.com Wed Oct 26 19:40:57 2016 From: deleskie at gmail.com (jim deleskie) Date: Wed, 26 Oct 2016 16:40:57 -0300 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: So device is certified, bug is found 2 years later. How does this help. The info to date is last week's issue was patched by the vendor in Sept 2015, I believe is what I read. We know bugs will creep in, (source anyone that has worked with code forever) Also certification assuming it would work, in what country, would I need one, per country I sell into? These are not the solutions you are looking for ( Jedi word play on purpose) On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < jordi.palet at consulintel.es> wrote: > Exactly, I was arguing exactly the same with some folks this week during > the RIPE meeting. > > The same way that certifications are needed to avoid radio interferences, > etc., and if you don?t pass those certifications, you can?t sell the > products in some countries (or regions in case of EU for example), > authorities should make sure that those certifications have a broader > scope, including security and probably some other features to ensure that > in case something is discovered in the future, they can be updated. > > Yes, that means cost, but a few thousand dollars of certification price > increase, among thousands of millions of devices of the same model being > manufactured, means a few cents for each unit. > > Even if we speak about 1 dollar per each product being sold, it is much > cheaper than the cost of not doing it and paying for damages, human > resources, etc., when there is a security breach. > > Regards, > Jordi > > > -----Mensaje original----- > De: NANOG en nombre de Leo Bicknell < > bicknell at ufp.org> > Organizaci?n: United Federation of Planets > Responder a: > Fecha: mi?rcoles, 26 de octubre de 2016, 19:19 > Para: > Asunto: Re: Spitballing IoT Security > > In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich > Kulawiec wrote: > > The makers of IoT devices are falling all over themselves to rush > products > > to market as quickly as possible in order to maximize their > profits. They > > have no time for security. They don't concern themselves with > privacy > > implications. They don't run networks so they don't care about the > impact > > their devices may have on them. They don't care about liability: > many of > > them are effectively immune because suing them would mean > trans-national > > litigation, which is tedious and expensive. (And even if they lost: > > they'd dissolve and reconstitute as another company the next day.) > > They don't even care about each other -- I'm pretty sure we're > rapidly > > approaching the point where toasters will be used to attack garage > door > > openers and washing machines. > > You are correct. > > I believe the answer is to have some sort of test scheme (UL > Labratories?) for basic security and updateability. Then federal > legislation is passed requiring any product being imported into the > country to be certified, or it is refused. > > Now when they rush to market and don't get certified they get $0 > and go out of business. Products are stopped at the boader, every > shipment is reviewed by authorities, and there is no cross boarder > suing issue. > > Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a > host of others have regulations that if you want to import a product > for sale it must be safe. It's not a new or novel concept, pretty > much every country has some scheme like it. > > -- > Leo Bicknell - bicknell at ufp.org > PGP keys at http://www.ufp.org/~bicknell/ > > > > > ********************************************** > IPv4 is over > Are you ready for the new Internet ? > http://www.consulintel.es > The IPv6 Company > > This electronic message contains information which may be privileged or > confidential. The information is intended to be for the use of the > individual(s) named above. If you are not the intended recipient be aware > that any disclosure, copying, distribution or use of the contents of this > information, including attached files, is prohibited. > > > > From jfmezei_nanog at vaxination.ca Wed Oct 26 19:52:58 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Wed, 26 Oct 2016 15:52:58 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <5811099A.4040307@vaxination.ca> re: having gadgets certified (aka UL/CSA for electric stuff). Devil is in the details. Who would certify it ? And who would set the standards for certification? How fast would those standards change? updated with each new attack? Would standards update require agreement of multiple parties who rarely agree? Consider vendor X who starts to develop product based on standards available in Oct 2016, but by the time he gets to market, standards have changed and his device no longer conforms? One of the beauties of the Internet is the freedom to innovate while keeping to the core basic IP packet delivery. Start to regulate it or add red tape and you start to hinder innovation. Perhaps the RFC mechanism to define best practices for standalone "IoT" devices might be a better mechanism. Those who build IP stacks to be used wholesale by gadget manufacturers could adhere to that RFC so that end products en up using a proper IP stack that doesn't easily allow the device to be "upgraded" to serve Dr Evil's botnet designed to take over the world. From mel at beckman.org Wed Oct 26 19:56:34 2016 From: mel at beckman.org (Mel Beckman) Date: Wed, 26 Oct 2016 19:56:34 +0000 Subject: Spitballing IoT Security In-Reply-To: <20161026194040.GA16649@thyrsus.com> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026123043.GA10916@thyrsus.com> , <20161026194040.GA16649@thyrsus.com> Message-ID: Why does everyone think the Master Plan for World Domination has to be Evil? :) -mel beckman > On Oct 26, 2016, at 12:40 PM, Eric S. Raymond wrote: > > Mel Beckman : >> I also really like the idea of offering open source options to vendors, many of whom seem to illegally take that privilege anyway. A key fast-path component, though, is in my opinion a new RFC for IoT security best practices, and probably some revisions to UPNP. >> >> The IoT RFC would spell out basic rules for safe devices: no back doors, no default passwords, no gratuitous inbound connections, etc. It would also make encryption a requirement, and limit how existing UPNP is deployed to prevent unnecessarily exposing vulnerable TCP/UDP ports to the wild. With this RFC in hand, and an appropriate splashy icon for vendor packaging (?RFC 9999 ThingSafe!?), vendors will have a competitive reason for compliance as a market differentiator, whether they deploy with open-source or proprietary code. > > That is a good idea and I am officially adopting it as part of the Evil > Master Plan for World Domination. :-) > > I may recruit you to help draft the RFC. > -- > Eric S. Raymond From matlockken at gmail.com Wed Oct 26 20:12:09 2016 From: matlockken at gmail.com (Ken Matlock) Date: Wed, 26 Oct 2016 14:12:09 -0600 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: As a relative 'outsider' I see a lot of finger-pointing and phrasing this as (effectively) someone else's fault. To me this is a failing on a number of levels all contributing to the problem. 1) The manufacturer - Backdoors, hidden accounts, remote access capabilities, no proper security testing. No enforcing of security updates. 2) The end-user - No initiative on the end-user's perspective to gain even a basic understanding of how the device works, connects, etc. Also no tools or understanding of how to recognize *which* of their many devices on the network might be compromised and participating in the botnet. (Only indication they get is maybe their internet is slow) 3) The service providers - No effective monitoring of outgoing traffic from the end users to identify botnets and DDoS in a real-time fashion I contend that all 3 levels have failed in this, and nothing has fundamentally changed (today it's IoT, before it was unpatched windows boxes, etc) in decades. We keep talking about the problem but very little actual action has occurred to *fix* the underlying issues. - Manufacturers need to be held accountable for devices that go on the internet (that includes *anything* that's connected. PCs, servers, routers, IoT devices, etc) - End users need to have ways to easily see what's going on over their local networks, to see botnet-like activity and DDoS participation (among other things) in a more real-time fashion - Service providers need to be much more proactive in watching for threats and identifying/blocking them at the source, not allowing the traffic to flow to your peers and making it someone else's problem. Right now there's a financial disincentive to doing this, in both real costs (standing up monitoring gear/etc), and imagined (my ISP is SPYING on me!). Until we fix all 3 of these main issues we're just going to keep going in the same set of circles we do every time a 'new' threat/vector comes in. Now, are these issues *easy*? Oh, heck no! Are they *cheap*? Once again, heck no! But to 'fix' this issue it will take all 3 levels being fixed. If we continue to keep pointing fingers at "the other guy" as the root of the problem we're inviting external forces (Legislation) to step in and 'fix' the problem for us (and it will just make it worse). My 2 cents (adjust for inflation) Ken On Wed, Oct 26, 2016 at 1:40 PM, jim deleskie wrote: > So device is certified, bug is found 2 years later. How does this help. > The info to date is last week's issue was patched by the vendor in Sept > 2015, I believe is what I read. We know bugs will creep in, (source anyone > that has worked with code forever) Also certification assuming it would > work, in what country, would I need one, per country I sell into? These > are not the solutions you are looking for ( Jedi word play on purpose) > > On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < > jordi.palet at consulintel.es> wrote: > > > Exactly, I was arguing exactly the same with some folks this week during > > the RIPE meeting. > > > > The same way that certifications are needed to avoid radio interferences, > > etc., and if you don?t pass those certifications, you can?t sell the > > products in some countries (or regions in case of EU for example), > > authorities should make sure that those certifications have a broader > > scope, including security and probably some other features to ensure that > > in case something is discovered in the future, they can be updated. > > > > Yes, that means cost, but a few thousand dollars of certification price > > increase, among thousands of millions of devices of the same model being > > manufactured, means a few cents for each unit. > > > > Even if we speak about 1 dollar per each product being sold, it is much > > cheaper than the cost of not doing it and paying for damages, human > > resources, etc., when there is a security breach. > > > > Regards, > > Jordi > > > > > > -----Mensaje original----- > > De: NANOG en nombre de Leo Bicknell < > > bicknell at ufp.org> > > Organizaci?n: United Federation of Planets > > Responder a: > > Fecha: mi?rcoles, 26 de octubre de 2016, 19:19 > > Para: > > Asunto: Re: Spitballing IoT Security > > > > In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich > > Kulawiec wrote: > > > The makers of IoT devices are falling all over themselves to rush > > products > > > to market as quickly as possible in order to maximize their > > profits. They > > > have no time for security. They don't concern themselves with > > privacy > > > implications. They don't run networks so they don't care about the > > impact > > > their devices may have on them. They don't care about liability: > > many of > > > them are effectively immune because suing them would mean > > trans-national > > > litigation, which is tedious and expensive. (And even if they > lost: > > > they'd dissolve and reconstitute as another company the next day.) > > > They don't even care about each other -- I'm pretty sure we're > > rapidly > > > approaching the point where toasters will be used to attack garage > > door > > > openers and washing machines. > > > > You are correct. > > > > I believe the answer is to have some sort of test scheme (UL > > Labratories?) for basic security and updateability. Then federal > > legislation is passed requiring any product being imported into the > > country to be certified, or it is refused. > > > > Now when they rush to market and don't get certified they get $0 > > and go out of business. Products are stopped at the boader, every > > shipment is reviewed by authorities, and there is no cross boarder > > suing issue. > > > > Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a > > host of others have regulations that if you want to import a product > > for sale it must be safe. It's not a new or novel concept, pretty > > much every country has some scheme like it. > > > > -- > > Leo Bicknell - bicknell at ufp.org > > PGP keys at http://www.ufp.org/~bicknell/ > > > > > > > > > > ********************************************** > > IPv4 is over > > Are you ready for the new Internet ? > > http://www.consulintel.es > > The IPv6 Company > > > > This electronic message contains information which may be privileged or > > confidential. The information is intended to be for the use of the > > individual(s) named above. If you are not the intended recipient be aware > > that any disclosure, copying, distribution or use of the contents of this > > information, including attached files, is prohibited. > > > > > > > > > From bzs at TheWorld.com Wed Oct 26 20:18:30 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Wed, 26 Oct 2016 16:18:30 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <22545.3990.314539.807281@gargle.gargle.HOWL> Re: certification of IoT devices analogous to UL etc Another potentially useful channel to give this idea legs are insurance companies, get them involved if possible. They underwrite the risks particularly liability risks for manufacturers. That's why "Underwriters Laboratory" is called that, ultimately it's an arm of the insurance industry. If the insurance companies tell a manufacturer they won't cover risk for any non-certified device the device will almost certainly be improved. Similar repercussions for wholesale and retail outlets who could decide to just not offer non-certified devices, for insurance reasons and otherwise. As to those who would try maneuvers such as bankrupting or using shell companies which are dissolved when liabilities occur such willful acts often allow "piercing of the corporate veil". That is, those individuals involved can be sued or held criminally liable personally and any such indemnification made worthless as a protection or defense. In the US, at least, if there's a pattern of such behavior, such as serially setting up shell corps and bankrupting them when trouble arises, the fearsome RICO statutes can be invoked. Basically they provide the added felonies of racketeering and engaging in a conspiracy to commit criminal acts. Not being located in the US may not be sufficient for evasion of prosecution, merely doing business in the US (e.g., selling one's products, establishing a "nexus") can make one a valid target for US (and other) law enforcement. The fly I see in all this ointment is that getting there could be a lot of honest work so who would do that and champion this idea? -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From marka at isc.org Wed Oct 26 20:58:00 2016 From: marka at isc.org (Mark Andrews) Date: Thu, 27 Oct 2016 07:58:00 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Wed, 26 Oct 2016 14:12:09 -0600." References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <20161026205800.7188D57B29B8@rock.dv.isc.org> In message , Ken Matlock writes: > As a relative 'outsider' I see a lot of finger-pointing and phrasing this > as (effectively) someone else's fault. > > To me this is a failing on a number of levels all contributing to the > problem. > > 1) The manufacturer - Backdoors, hidden accounts, remote access > capabilities, no proper security testing. No enforcing of security updates. > 2) The end-user - No initiative on the end-user's perspective to gain even > a basic understanding of how the device works, connects, etc. Also no tools > or understanding of how to recognize *which* of their many devices on the > network might be compromised and participating in the botnet. (Only > indication they get is maybe their internet is slow) > 3) The service providers - No effective monitoring of outgoing traffic from > the end users to identify botnets and DDoS in a real-time fashion > > I contend that all 3 levels have failed in this, and nothing has > fundamentally changed (today it's IoT, before it was unpatched windows > boxes, etc) in decades. We keep talking about the problem but very little > actual action has occurred to *fix* the underlying issues. Actually things have changed a lot in a positive direction. * Router manufactures are using device specific passwords. * Microsoft, Apple, Linux and *BSD issue regular fixes for their products and users do intall them. * My smart TV has automatic updates available and turned on. * Other products do the same. Now not everyone does this sort of thing yet, but we have examples and things don't blow up in the user's face very often. Even in the case the manufacture has tried to do the right thing. The problem had been identified and a fix issued. Now could this have been automated, yes. But it does show that what is required is getting through to manufactures and they are trying to reduce the problem. We need manufactures to have a working system to accept problem reports. We need manufactures to issue fixes to their products once they have been reported. This needs to happen for the entire expected lifetime of a product. We need the ability to have a third parties fix problems if a manufacture won't / is too slow. Getting this may require legislation changes. This may require manufactures to publish expected product lifetimes. > - Manufacturers need to be held accountable for devices that go on the > internet (that includes *anything* that's connected. PCs, servers, routers, > IoT devices, etc) > - End users need to have ways to easily see what's going on over their > local networks, to see botnet-like activity and DDoS participation (among > other things) in a more real-time fashion > - Service providers need to be much more proactive in watching for threats > and identifying/blocking them at the source, not allowing the traffic to > flow to your peers and making it someone else's problem. Right now there's > a financial disincentive to doing this, in both real costs (standing up > monitoring gear/etc), and imagined (my ISP is SPYING on me!). > > Until we fix all 3 of these main issues we're just going to keep going in > the same set of circles we do every time a 'new' threat/vector comes in. > > Now, are these issues *easy*? Oh, heck no! Are they *cheap*? Once again, > heck no! But to 'fix' this issue it will take all 3 levels being fixed. > > If we continue to keep pointing fingers at "the other guy" as the root of > the problem we're inviting external forces (Legislation) to step in and > 'fix' the problem for us (and it will just make it worse). > > My 2 cents (adjust for inflation) > Ken > > On Wed, Oct 26, 2016 at 1:40 PM, jim deleskie wrote: > > > So device is certified, bug is found 2 years later. How does this help. > > The info to date is last week's issue was patched by the vendor in Sept > > 2015, I believe is what I read. We know bugs will creep in, (source anyon= > e > > that has worked with code forever) Also certification assuming it would > > work, in what country, would I need one, per country I sell into? These > > are not the solutions you are looking for ( Jedi word play on purpose) > > > > On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < > > jordi.palet at consulintel.es> wrote: > > > > > Exactly, I was arguing exactly the same with some folks this week durin= > g > > > the RIPE meeting. > > > > > > The same way that certifications are needed to avoid radio interference= > s, > > > etc., and if you don=E2=80=99t pass those certifications, you can=E2=80= > =99t sell the > > > products in some countries (or regions in case of EU for example), > > > authorities should make sure that those certifications have a broader > > > scope, including security and probably some other features to ensure th= > at > > > in case something is discovered in the future, they can be updated. > > > > > > Yes, that means cost, but a few thousand dollars of certification price > > > increase, among thousands of millions of devices of the same model bein= > g > > > manufactured, means a few cents for each unit. > > > > > > Even if we speak about 1 dollar per each product being sold, it is much > > > cheaper than the cost of not doing it and paying for damages, human > > > resources, etc., when there is a security breach. > > > > > > Regards, > > > Jordi > > > > > > > > > -----Mensaje original----- > > > De: NANOG en nombre de Leo Bicknell < > > > bicknell at ufp.org> > > > Organizaci=C3=B3n: United Federation of Planets > > > Responder a: > > > Fecha: mi=C3=A9rcoles, 26 de octubre de 2016, 19:19 > > > Para: > > > Asunto: Re: Spitballing IoT Security > > > > > > In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich > > > Kulawiec wrote: > > > > The makers of IoT devices are falling all over themselves to rush > > > products > > > > to market as quickly as possible in order to maximize their > > > profits. They > > > > have no time for security. They don't concern themselves with > > > privacy > > > > implications. They don't run networks so they don't care about t= > he > > > impact > > > > their devices may have on them. They don't care about liability: > > > many of > > > > them are effectively immune because suing them would mean > > > trans-national > > > > litigation, which is tedious and expensive. (And even if they > > lost: > > > > they'd dissolve and reconstitute as another company the next day.= > ) > > > > They don't even care about each other -- I'm pretty sure we're > > > rapidly > > > > approaching the point where toasters will be used to attack garag= > e > > > door > > > > openers and washing machines. > > > > > > You are correct. > > > > > > I believe the answer is to have some sort of test scheme (UL > > > Labratories?) for basic security and updateability. Then federal > > > legislation is passed requiring any product being imported into the > > > country to be certified, or it is refused. > > > > > > Now when they rush to market and don't get certified they get $0 > > > and go out of business. Products are stopped at the boader, every > > > shipment is reviewed by authorities, and there is no cross boarder > > > suing issue. > > > > > > Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a > > > host of others have regulations that if you want to import a produc= > t > > > for sale it must be safe. It's not a new or novel concept, pretty > > > much every country has some scheme like it. > > > > > > -- > > > Leo Bicknell - bicknell at ufp.org > > > PGP keys at http://www.ufp.org/~bicknell/ > > > > > > > > > > > > > > > ********************************************** > > > IPv4 is over > > > Are you ready for the new Internet ? > > > http://www.consulintel.es > > > The IPv6 Company > > > > > > This electronic message contains information which may be privileged or > > > confidential. The information is intended to be for the use of the > > > individual(s) named above. If you are not the intended recipient be awa= > re > > > that any disclosure, copying, distribution or use of the contents of th= > is > > > information, including attached files, is prohibited. > > > > > > > > > > > > > > -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From jfmezei_nanog at vaxination.ca Wed Oct 26 21:10:44 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Wed, 26 Oct 2016 17:10:44 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161026205800.7188D57B29B8@rock.dv.isc.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> <20161026205800.7188D57B29B8@rock.dv.isc.org> Message-ID: <58111BD4.80403@vaxination.ca> On 2016-10-26 16:58, Mark Andrews wrote: > > Actually things have changed a lot in a positive direction. > > * Router manufactures are using device specific passwords. > * Microsoft, Apple, Linux and *BSD issue regular fixes for their > products and users do intall them. > * My smart TV has automatic updates available and turned on. > * Other products do the same. My smart TV not only hasn't gotten updates in years, but Sharp has stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere). When manufacturers provide a 2 year support on a device that will last 10 years, it is a problem which is why they really need to get it right when product is released and not rely on patches. With regards to liability. Good luck suing a chinese outfit that no longer exists. And pray tell, who gets to pay the millions of dollars of lawyer fees it will cost to sue that bankrupt company with no money ? From rfg at tristatelogic.com Wed Oct 26 21:25:00 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 14:25:00 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161026120634.GA20735@gsp.org> Message-ID: <11718.1477517100@segfault.tristatelogic.com> In message <20161026120634.GA20735 at gsp.org>, Rich Kulawiec wrote: >On Mon, Oct 24, 2016 at 01:24:59PM -0700, Ronald F. Guilmette wrote: >> 2) Second, once elected I will decree that in future all new IoT devices, >> and also all updates to firmware for existing IoT devices will have, >> BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP >> session initiation and which also (b) strictly rate-limits all other >> protocols to some modest value. > >I like this idea. But unfortunately, I think it has no chance of succeeding. > >The makers of IoT devices are falling all over themselves to rush products >to market as quickly as possible in order to maximize their profits. They >have no time for security. They don't concern themselves with privacy... Well, see, this is why I was clear at the outset that in order for this scheme to work, I'll first need to be elected King of the World. >From that high perch I will be able to decree, by fiat, that no Internet connectable device shall be sold or marketed *unless* it has been certified (i.e. by some reliable entity that knows how to test these things) to be incapable of being converted into a weapon, i.e. incapable of spewing unnecessarily large amounts of garbage at completely arbitrary targets, *even if* an attacker somehow manages to get a shell prompt. OK, so setting aside all frivolity now, how could this kind of restriction actually be achieved? Here's the thing: Any solution to these problems is going to come in two parts, technical and political. We here, by and large, are not politicians, but we can influence them and urge them towards solutions that are, workable, economically practical, and above all, technically effective. Or alternatively, we can leave them to flouder around on their own, in the dark. (We've all seen *that* movie before, and it isn't pretty. Think Clipper Chip and the recent push for crypto backdoors.) Left to their own devices, politicians routinely screw up technology regulation virtually every time. So the first order of business is for the industry itself to come up with a rational approach... and virtually immediately, because the window of opportunity is rapidly closing... for solving these IoT problems, and then get widspread agreement... or at least a lack of violent disagreement... within the industry itself. The industry can then speak with one voice to the politicians and regulators, who will then be effectively bound to doing the Right Thing. Sensible regulations, once enacted in one jurisdiction, tend to be contagious. In my own state of California, state regulation of various things, most notably air pollution, and the production thereof by cars, has eventually affected the entire North American auto market and beyond, in part because it is less economically palatable for manufacturers to design and ship multiple configurations of any one product, i.e. one which conforms to the regulations in jurisdiction X, and another that doesn't. In short, if sensible regulations requiring "safe" designs for IoT products were to come into force in one locale, it is not only possible, but actually quite likely that they would affect the whole market. If a given Far East manufacturer was required to have safety built into the kernel of its toasters in order to be able to sell said toasters, say, in the United States... or even just in California... would they really go to the trouble to strip out the additional "safety" part of their firmware when manufacturing what is essentially the same product, but destined for other markets? I think not. (A question for the audience: How has FCC regulation of the maximum power output of WiFi routers affected the worldwide market for such devices, over time? I honestly don't know, but I suspect that there has been a good effect, over time, on the whole worldwide market.) It may be difficult, even among technologists, to find common ground and agreement about what "IoT" things should and should not be able to do, or even, for that matter, to agree on the definition of "IoT". But after last Friday, and even before, I think that most of us know what we *do not* want them to be able to do, i.e. to send an unlimited percentage of their available bandwidth towards any arbitrary IP address. General purpose computers, and also routers, need to be able to do that, but your bird feeder, your lightbulb, your HDTV, your refrigerator and your home alarm system don't. So maybe that's a starting point. I proposed something which is at base, really rather simple, even if, in practice, the implementation details could get a bit complex. Basically, the proposal is that the kernels of all IoT devices should impose sensible limits on outbound bandwidth usage, consistant with each specific device's expected operational needs. It seems to me that this is not particularly different from other belt-and-suspenders approaches used in other safety critical systems, ranging from medical radiation treatment devices to nuclear power plants. Actual engineering of the firmware-imposed safety constraints needed in IoT devices will not, in my opinion, be very hard. In the absence of a King of the World to impose such a requirement on all manufacturers of IoT devices, I believe that it would be equally effective, in the long run, to get (U.S.) state-level regulations on the books, perhaps starting in California, just because we here in my home state have some experience going first with a lot of these kinds of things. A plausible alternatively would be to get the FCC on the case. (Obviously, the FCC already has a ton of experience in promulgating regulations whose goal is to prevent individual devices from behaving in ways that muck up the communications of other devices, so from that perspective at least, it seems like a good fit. Not that the FCC could be easily persuaded to take on this tar-baby, but they might.) So anyway, bottom line, I think this is do-able, both technically and politically, and also absolutely necessary. After the Krebs, OVH, and Dyn attacks, is anybody in their right mind willing to stand up, at this late date, and say that we can go on, as we have been, ignoring these problems and just constantly racing to build bigger pipes... a strategy which, by now, should be universally accepted as a self-defeating non-solution? Lincoln said "As our case is new, so we must think anew and act anew." If you hook up a device to your local telephone or cable company which sends fifty thousand volts down the line, you may fry your local distribution substation, but you're not going to fry the entire Eastern Seaboard or take down the world's largest e-commerce site for two hours. Even the popular news media, typically devoid of technical sophistication, now knows that the single organism that is the Internet is becoming more vlunerable *to itself* day by day. The time is ripe for clear-headed action and I do hope that we will see some. Regards, rfg From marka at isc.org Wed Oct 26 21:38:59 2016 From: marka at isc.org (Mark Andrews) Date: Thu, 27 Oct 2016 08:38:59 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Wed, 26 Oct 2016 14:25:00 -0700." <11718.1477517100@segfault.tristatelogic.com> References: <11718.1477517100@segfault.tristatelogic.com> Message-ID: <20161026213859.1C59257B2DE1@rock.dv.isc.org> In message <11718.1477517100 at segfault.tristatelogic.com>, "Ronald F. Guilmette" writes: > In short, if sensible regulations requiring "safe" designs for IoT products > were to come into force in one locale, it is not only possible, but > actually quite likely that they would affect the whole market. If a given > Far East manufacturer was required to have safety built into the kernel > of its toasters in order to be able to sell said toasters, say, in the > United States... or even just in California... would they really go to > the trouble to strip out the additional "safety" part of their firmware > when manufacturing what is essentially the same product, but destined > for other markets? I think not. (A question for the audience: How has > FCC regulation of the maximum power output of WiFi routers affected the > worldwide market for such devices, over time? I honestly don't know, but > I suspect that there has been a good effect, over time, on the whole > worldwide market.) FCC regulation has caused manufactures to do a US version and a rest of the world version. They have over regulated. A simple list for location should be enough with default on unknown which leaves Wifi off until set. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From Valdis.Kletnieks at vt.edu Wed Oct 26 21:57:45 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Wed, 26 Oct 2016 17:57:45 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <88042.1477519065@turing-police.cc.vt.edu> On Wed, 26 Oct 2016 20:53:51 +0200, JORDI PALET MARTINEZ said: > Even if we speak about 1 dollar per each product being sold, it is much > cheaper than the cost of not doing it and paying for damages, human resources, > etc., when there is a security breach. This only works if the company perceives a very real danger of having to pay for damages in case of a breach. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From rfg at tristatelogic.com Wed Oct 26 22:02:46 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 15:02:46 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161026123043.GA10916@thyrsus.com> Message-ID: <11878.1477519366@segfault.tristatelogic.com> In message <20161026123043.GA10916 at thyrsus.com>, "Eric S. Raymond" wrote: >There is, however, a chokepoint we have more hope of getting decent software >deployed to. I refer to home and small-business routers. OpenWRT and kin >are already minor but significant players here. And there's an NRE-minimization >aregument we can make for router manufacturers to use rebranded versions >rather than rolling their own crappy firmware. > >I think the anti-IoT-flood strategy that makes the most sense is: > >1. Push open-source firmware that doesn't suck to the vendors with a > cost- and risk-minimization pitch. > >2. Ship it with egress filters. (And telnet blocked.) So basically, this is a proposal to "fix" the problem for all IoT devices that are behind SOHO routers. I am compelled to note that the grand vision of the Home of the Future[tm], as it has been presented to me at least, looks rather more like this: http://p.globalsources.com/IMAGES/PDT/BIG/053/B1088622053.jpg i.e. a multitude of wall plates in every room, each one bristling with a multitude of RJ11 sockets into which all manner of shiny new IoT things will be directly plugged, thence to be issued their own IPv6 addresses directly via DHCP from the local provider. If I have misunderstood the general direction is which we are all heading, then I will be only too happy to be disabused of my incorrect notions. >It wouldn't be technically very difficult to make the firmware >rate-limit outbound connections... No, but if you do that to my desktop PeeCee, then I'm gonna be pretty mad at you. On the other hand, if you come up with a way for devices to somehow signal to an associated SOHO router that they are either (a) desktop PeeCees or, in the alternative, IoT devices, and if you should me that you method is (a) simple and (b) fool-proof and (c) hack-proof, then I'll be the first one to say that you're really on to something. Regards, rfg P.S. As noted in my prior post, the proplem of regulating IoT devices to insure that they do not exceed their reasonably expected operational limits, vis-a-vis outbound bandwidth usage, is in some ways reminicent of the FCC regulations which, ideally, prevent SOHO WiFi routers from exceeding reasonable power output limits designed to prevent them from interfering with other devices. Given that, and given that "OpenWRT and kin" often provide the end-user with readily accessible dials and knobs via which the user can force the device to *exceed* legal/FCC limits on power output, I am not persuaded that open source WiFi router firmware actually represents a shining example of a methodology to prevent inexpensive devices from behaving badly. From Valdis.Kletnieks at vt.edu Wed Oct 26 22:24:16 2016 From: Valdis.Kletnieks at vt.edu (Valdis.Kletnieks at vt.edu) Date: Wed, 26 Oct 2016 18:24:16 -0400 Subject: Spitballing IoT Security In-Reply-To: <11878.1477519366@segfault.tristatelogic.com> References: <11878.1477519366@segfault.tristatelogic.com> Message-ID: <89795.1477520656@turing-police.cc.vt.edu> On Wed, 26 Oct 2016 15:02:46 -0700, "Ronald F. Guilmette" said: > i.e. a multitude of wall plates in every room, each one bristling with a > multitude of RJ11 sockets into which all manner of shiny new IoT things > will be directly plugged, thence to be issued their own IPv6 addresses > directly via DHCP from the local provider. Actually, it seems to be going to wireless/bluetooth, and DHCP from the household router. Note that although a minor difference, it's one that can be leveraged. If we can change the dynamic from "plug it in and it Just Works" to "plug it in, and click the pop-up from your router confirming that you just added a device, and it Just Works after that", the battle is 3/4 won. The other 1/4 is the device initially telling the router what sort of device it is. - and we already know how to do that for USB and BlueTooth... > Given that, and given that "OpenWRT and kin" often provide the end-user > with readily accessible dials and knobs via which the user can force the > device to *exceed* legal/FCC limits on power output, I am not persuaded > that open source WiFi router firmware actually represents a shining > example of a methodology to prevent inexpensive devices from behaving badly. Given that out of the box, the default config is in bounds, and it requires actual user interaction to exceed the limits, and that we don't see a very large problem out in the wild, I think we have prior art for the concept that "shipped with default and clued user can reconfigure" is a workable design. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 484 bytes Desc: not available URL: From jfmezei_nanog at vaxination.ca Wed Oct 26 22:35:11 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Wed, 26 Oct 2016 18:35:11 -0400 Subject: Spitballing IoT Security In-Reply-To: <11878.1477519366@segfault.tristatelogic.com> References: <11878.1477519366@segfault.tristatelogic.com> Message-ID: <58112F9F.6060705@vaxination.ca> On 2016-10-26 18:02, Ronald F. Guilmette wrote: > http://p.globalsources.com/IMAGES/PDT/BIG/053/B1088622053.jpg > > i.e. a multitude of wall plates in every room, each one bristling with a > multitude of RJ11 sockets into which all manner of shiny new IoT things > will be directly plugged, thence to be issued their own IPv6 addresses You still need to have a SOHO router, which could simply block any incoming calls unless a port has been opened for a specific IP address. (or UPnP for computers). > P.S. As noted in my prior post, the proplem of regulating IoT devices to > insure that they do not exceed their reasonably expected operational limits, > vis-a-vis outbound bandwidth usage A camera showing the baby in 4K resolution along witgh sounds of him crying on dolby surround to the mother who is at work would likely saturate upload just as much as the virus sending DNS requests. This falls into the tonne of feathers weighting as much as a tonne of lead category. From rfg at tristatelogic.com Wed Oct 26 23:40:52 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 16:40:52 -0700 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <12301.1477525252@segfault.tristatelogic.com> In message Ken Matlock wrote: >- End users need to have ways to easily see what's going on over their >local networks, to see botnet-like activity and DDoS participation (among >other things) in a more real-time fashion This is an interesting point. I'm not actually an ISP guy, although I do play one on TV. Anyway, I hope nobody will begrudge me if I make a couple of brief points, and then ask a rather naive question. Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. My guess is that if any typical/average user is seen to be using more than, say, 1/10 of that amount of "up" bandwidth in any one given 10 minute time period, then something is really really REALLY wrong. Point: I am already signed up with various services which will send me automated emails whenever certain events occur, e.g. when the price of 2TB WD Black drives falls below my personal threshold value. Point: My ISP knows my email address. Question: Could ISPs set something up so that each customer broadband line is continuously and individually monitored, and so that an automated email would be automagically dashed off to the customer if that customer's "up" bandwidth in some time period exceeded a value which, for that ISP, is deemed "reasonable"? (I envision the hypothetical email messages in question would consist of a non-threatening warning to the customer which would include a link to a web page where there would be a list of common things end-lusers should check for in such circumstances, along with detailed and clear instructions for how to check for each, and also a "don't ever bother me with these warnings again" checkbox, and perhaps even a friendly slider where the end-luser could adjust his personal warning threshold value, for the future.) Of course, any ISP that desperately -never- wants to receive -any- end- luser support calls quite certainly won't like this scheme. But I'm not sure that that fact alone would utterly disqualify the idea from being useful in some contexts. The real question is: Is anything even remotely along these lines even possible with existing commonly used ISP infrastructure? (If not, then just forget I mentioned it.) Regards, rfg P.S. One possible big advantage to the kind of system described above is that if an ISP received a complaint about a given customer, alleging that the customer is running a bot, then the ISP could go and look at the warning settings for that customer. If that's already been set to "don't ever bother me', then the ISP can disconnect the customer, and when the customer inevitably saquaks about that, the ISP can respond and say "We told you so." From cboyd at gizmopartners.com Wed Oct 26 23:49:12 2016 From: cboyd at gizmopartners.com (Chris Boyd) Date: Wed, 26 Oct 2016 18:49:12 -0500 Subject: Spitballing IoT Security In-Reply-To: <12301.1477525252@segfault.tristatelogic.com> References: <12301.1477525252@segfault.tristatelogic.com> Message-ID: <184CFB86-A36F-40C9-A99D-72ED39C1BA8A@gizmopartners.com> > On Oct 26, 2016, at 6:40 PM, Ronald F. Guilmette wrote: > > Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. > My guess is that if any typical/average user is seen to be using more > than, say, 1/10 of that amount of "up" bandwidth in any one given 10 > minute time period, then something is really really REALLY wrong. Online backup service like Carbonite and Backblaze copy lots of data upstream. iPhone backups would probably saturate your line for a good chunk of 10 minutes. Even posting a bunch of photos could take that long. Oh, and bittorrent. ?Chris From marka at isc.org Wed Oct 26 23:56:24 2016 From: marka at isc.org (Mark Andrews) Date: Thu, 27 Oct 2016 10:56:24 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Wed, 26 Oct 2016 16:40:52 -0700." <12301.1477525252@segfault.tristatelogic.com> References: <12301.1477525252@segfault.tristatelogic.com> Message-ID: <20161026235624.732FC57BBBC3@rock.dv.isc.org> In message <12301.1477525252 at segfault.tristatelogic.com>, "Ronald F. Guilmette" writes: > > In message m> > Ken Matlock wrote: > > >- End users need to have ways to easily see what's going on over their > >local networks, to see botnet-like activity and DDoS participation (among > >other things) in a more real-time fashion > > This is an interesting point. > > I'm not actually an ISP guy, although I do play one on TV. Anyway, > I hope nobody will begrudge me if I make a couple of brief points, > and then ask a rather naive question. > > Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. > My guess is that if any typical/average user is seen to be using more > than, say, 1/10 of that amount of "up" bandwidth in any one given 10 > minute time period, then something is really really REALLY wrong. No. Just uploading a video to youtube would cause a false positive using that test. You need to know what "bad" traffic looks like to find it. Packets flowing != "bad traffic". Unusual != "bad traffic". Mark > Point: I am already signed up with various services which will send me > automated emails whenever certain events occur, e.g. when the price of > 2TB WD Black drives falls below my personal threshold value. > > Point: My ISP knows my email address. > > Question: Could ISPs set something up so that each customer broadband > line is continuously and individually monitored, and so that an automated > email would be automagically dashed off to the customer if that customer's > "up" bandwidth in some time period exceeded a value which, for that ISP, > is deemed "reasonable"? (I envision the hypothetical email messages in > question would consist of a non-threatening warning to the customer which > would include a link to a web page where there would be a list of common > things end-lusers should check for in such circumstances, along with > detailed and clear instructions for how to check for each, and also a > "don't ever bother me with these warnings again" checkbox, and perhaps > even a friendly slider where the end-luser could adjust his personal > warning threshold value, for the future.) > > Of course, any ISP that desperately -never- wants to receive -any- end- > luser support calls quite certainly won't like this scheme. But I'm not > sure that that fact alone would utterly disqualify the idea from being > useful in some contexts. > > The real question is: Is anything even remotely along these lines even > possible with existing commonly used ISP infrastructure? (If not, then just > forget I mentioned it.) > > > Regards, > rfg > > > P.S. One possible big advantage to the kind of system described above is > that if an ISP received a complaint about a given customer, alleging that > the customer is running a bot, then the ISP could go and look at the > warning settings for that customer. If that's already been set to > "don't ever bother me', then the ISP can disconnect the customer, and > when the customer inevitably saquaks about that, the ISP can respond and > say "We told you so." -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From rfg at tristatelogic.com Thu Oct 27 00:27:08 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 17:27:08 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161026205800.7188D57B29B8@rock.dv.isc.org> Message-ID: <12439.1477528028@segfault.tristatelogic.com> In message <20161026205800.7188D57B29B8 at rock.dv.isc.org>, Mark Andrews wrote: >Actually things have changed a lot in a positive direction. >... >* Microsoft, Apple, Linux and *BSD issue regular fixes for their > products and users do intall them. At the risk of repeating a point I have already made in this thread, please do let me know how I can obtain this month's security patches for my iPhone 3GS. (Note that Wikipedia says that this device was only formally discontinued by the manufacturer as of September 12, 2012, i.e. only slightly more than 4 short years ago. Nontheless, the current "security solution" for this product, as made available from the manufacturer... a manufacturer which is here being held up as a shining example of ernest social responsi- bility... is for me to contribute the entire device to my local landfill, where it will no doubt leach innumerable heavy metals into the soil for my children's children's children to enjoy.) >> - Manufacturers need to be held accountable for devices that go on the >> internet... My iPhone 3GS "goes on the Internet". Through no fauly of my own, it is also, apparently, destined in short order to "go onto" a landfill, if not here, then in China or India, where a pitiful plethora of shoeless and sad-eyed third-world waifs will spend their childhoods picking through the mand-made mountains of e-refuse in a daily and desperate search for of anything of value. http://motherboard.vice.com/read/much-of-americas-e-waste-recycling-is-a-sham In short, if the "good" companies, like Apple, are the solution to the problem, then I obviously misunderstood what "the problem" is, and would be obliged if someone (anyone) would re-phrase it for me in simpler terms, for the benefit of my limited little noggin. In lieu of that, for the moment I'd just like to emphasize again that it is my opinion that any "solution" to the now self-evident IoT problems which relies, even in the slightest, upon manufacturers providing a con- tinuous and timely stream of security updates is a fantasy. Wishful thinking, pure and simple. When even the "good" companies have built their fortunes and entire business models around convincing/forcing everyone to purchase "new and improved" units every two years, at a maximum, and when the same said companies stop issuing patches of any kind for products that have only departed the corporate price list three years earlier, then one shudders to even contemplate what the contribution of the "bad" companies will be to this ongoing catastrophy. Regards, rfg From brandon at rd.bbc.co.uk Thu Oct 27 00:28:26 2016 From: brandon at rd.bbc.co.uk (Brandon Butterworth) Date: Thu, 27 Oct 2016 01:28:26 +0100 Subject: Spitballing IoT Security In-Reply-To: <58111BD4.80403@vaxination.ca> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> <20161026205800.7188D57B29B8@rock.dv.isc.org> <58111BD4.80403@vaxination.ca> Message-ID: <20161027002826.GE8137@sunf10.rd.bbc.co.uk> On Wed Oct 26, 2016 at 05:10:44PM -0400, Jean-Francois Mezei wrote: > My smart TV not only hasn't gotten updates in years, but Sharp has > stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere). > > When manufacturers provide a 2 year support on a device that will last > 10 years, it is a problem which is why they really need to get it right > when product is released and not rely on patches. No chance of being right first time or ever but that's not a problem until it gets compromised > With regards to liability. Good luck suing a chinese outfit that no > longer exists. > > And pray tell, who gets to pay the millions of dollars of lawyer fees it > will cost to sue that bankrupt company with no money ? Nobody will. This is why a kill switch is needed. If you're going to IoT Safe mark things there needs to be a way to revoke it like with SSL certs So say devices, which phone home anyway, are required as part of getting the mark to check in with $version.$device.$manufacturer.iotsafe.com it's not much more than they do to check for new firmware already You don't want all those calling something central so delegate to manufacturers and if they go bust drop the deleagtion and serve it centrally. An ISP with problem devices can always fake it locally to drop them and spot the polling traffic when looking for them When the device checks in they can with a simple api check their version and if they're allowed to be on the general internet on not. If banned they go offline and maybe tell the user somehow if they can. The deal to get IoT safe rated is that everyone agree to this, the user will be told clearly that their thing will be removed from the net if the manufacturer doesn't keep it safe so it's clear they sue them if there is a problem (or accept it's so cheap they can throw it away if they go bust) Now there's tons of holes in that like an attacher turning that bit off, there may be better schemes I've not noticed for doing this already. All details, the idea is a back stop is needed for when all the other stuff fails. brandon From rfg at tristatelogic.com Thu Oct 27 01:01:49 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 18:01:49 -0700 Subject: Spitballing IoT Security In-Reply-To: <58111BD4.80403@vaxination.ca> Message-ID: <12573.1477530109@segfault.tristatelogic.com> In message <58111BD4.80403 at vaxination.ca>, Jean-Francois Mezei wrote: >My smart TV not only hasn't gotten updates in years, but Sharp has >stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere). A little more than 2 years ago, I bought a last-of-its-kind demo model of a 50 inch Panasonic Plasma TV which was on sale (due to having been discontinued by the manufacturer) from the local BestBuy. Not long after, once I got the thing home, I realized that the thing's understanding of current local time... important in conjunction with the on-screen TV guide... was locked to Eastern Standard Time, and there was no way to change it. (This was/is a bit of a problem for me, as I'm in PST/PDT.) I called up Panasonic and explained the whole thing to a first- level tech support minion. She had no solution to offer me. I insisted on speaking to a manager. A manager got on the line and I prroceeded to re-explain the whole issue to him. I said that I needed a firmware fix. He said that there was no way the company was going to develop a fix "just for you". Politely, I persisted and said that the TV firmware was self-evidently faulty. <> <> From marka at isc.org Thu Oct 27 01:26:49 2016 From: marka at isc.org (Mark Andrews) Date: Thu, 27 Oct 2016 12:26:49 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Wed, 26 Oct 2016 18:01:49 -0700." <12573.1477530109@segfault.tristatelogic.com> References: <12573.1477530109@segfault.tristatelogic.com> Message-ID: <20161027012649.6D62457BE664@rock.dv.isc.org> In message <12573.1477530109 at segfault.tristatelogic.com>, "Ronald F. Guilmette" writes: > > In message <58111BD4.80403 at vaxination.ca>, > Jean-Francois Mezei wrote: > > >My smart TV not only hasn't gotten updates in years, but Sharp has > >stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere). > > A little more than 2 years ago, I bought a last-of-its-kind demo > model of a 50 inch Panasonic Plasma TV which was on sale (due to > having been discontinued by the manufacturer) from the local BestBuy. > > Not long after, once I got the thing home, I realized that the > thing's understanding of current local time... important in > conjunction with the on-screen TV guide... was locked to Eastern > Standard Time, and there was no way to change it. (This was/is > a bit of a problem for me, as I'm in PST/PDT.) > > I called up Panasonic and explained the whole thing to a first- > level tech support minion. She had no solution to offer me. > I insisted on speaking to a manager. > > A manager got on the line and I prroceeded to re-explain the whole > issue to him. I said that I needed a firmware fix. He said that > there was no way the company was going to develop a fix "just for > you". Politely, I persisted and said that the TV firmware was > self-evidently faulty. Then you return it to the store as "Not Fit For Purpose" and demand your money back. Alternatively you pull them into court and have them honour the warrantee. Or you move to a juristiction with good consumer protection laws and sic the government onto them. Just because a model is discontinued it does not mean that warrantee issues do not need to be addressed. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From mel at beckman.org Thu Oct 27 01:44:27 2016 From: mel at beckman.org (Mel Beckman) Date: Thu, 27 Oct 2016 01:44:27 +0000 Subject: Spitballing IoT Security In-Reply-To: <12301.1477525252@segfault.tristatelogic.com> References: , <12301.1477525252@segfault.tristatelogic.com> Message-ID: People under appreciate the power of a million-strong IoT bot net. Just a few K per second from each bot becomes gigabits per second at the target. -mel > On Oct 26, 2016, at 4:41 PM, Ronald F. Guilmette wrote: > > > In message > Ken Matlock wrote: > >> - End users need to have ways to easily see what's going on over their >> local networks, to see botnet-like activity and DDoS participation (among >> other things) in a more real-time fashion > > This is an interesting point. > > I'm not actually an ISP guy, although I do play one on TV. Anyway, > I hope nobody will begrudge me if I make a couple of brief points, > and then ask a rather naive question. > > Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. > My guess is that if any typical/average user is seen to be using more > than, say, 1/10 of that amount of "up" bandwidth in any one given 10 > minute time period, then something is really really REALLY wrong. > > Point: I am already signed up with various services which will send me > automated emails whenever certain events occur, e.g. when the price of > 2TB WD Black drives falls below my personal threshold value. > > Point: My ISP knows my email address. > > Question: Could ISPs set something up so that each customer broadband > line is continuously and individually monitored, and so that an automated > email would be automagically dashed off to the customer if that customer's > "up" bandwidth in some time period exceeded a value which, for that ISP, > is deemed "reasonable"? (I envision the hypothetical email messages in > question would consist of a non-threatening warning to the customer which > would include a link to a web page where there would be a list of common > things end-lusers should check for in such circumstances, along with > detailed and clear instructions for how to check for each, and also a > "don't ever bother me with these warnings again" checkbox, and perhaps > even a friendly slider where the end-luser could adjust his personal > warning threshold value, for the future.) > > Of course, any ISP that desperately -never- wants to receive -any- end- > luser support calls quite certainly won't like this scheme. But I'm not > sure that that fact alone would utterly disqualify the idea from being > useful in some contexts. > > The real question is: Is anything even remotely along these lines even > possible with existing commonly used ISP infrastructure? (If not, then just > forget I mentioned it.) > > > Regards, > rfg > > > P.S. One possible big advantage to the kind of system described above is > that if an ISP received a complaint about a given customer, alleging that > the customer is running a bot, then the ISP could go and look at the > warning settings for that customer. If that's already been set to > "don't ever bother me', then the ISP can disconnect the customer, and > when the customer inevitably saquaks about that, the ISP can respond and > say "We told you so." From rfg at tristatelogic.com Thu Oct 27 03:44:35 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 20:44:35 -0700 Subject: Spitballing IoT Security In-Reply-To: <89795.1477520656@turing-police.cc.vt.edu> Message-ID: <13183.1477539875@segfault.tristatelogic.com> In message <89795.1477520656 at turing-police.cc.vt.edu>, Valdis.Kletnieks at vt.edu wrote: >> Given that, and given that "OpenWRT and kin" often provide the end-user >> with readily accessible dials and knobs via which the user can force the >> device to *exceed* legal/FCC limits on power output, I am not persuaded >> that open source WiFi router firmware actually represents a shining >> example of a methodology to prevent inexpensive devices from behaving badly. > >Given that out of the box, the default config is in bounds, and it requires >actual user interaction to exceed the limits, and that we don't see a very >large problem out in the wild, I think we have prior art for the concept >that "shipped with default and clued user can reconfigure" is a workable design. You're right, of course, and I didn't mean to be picking on DD-WRT or OpenWRT, both of which I have used and have great admiration and respect for. It's just that if it comes down to a choice between putting a big sign on something which says "Please keep your arms and legs inside the vehicle at all times" or actually building somewhat difficult-to-remove barriers which physically prevent people from dangling their arms and legs out, given what we now know about typical end-luser behaviour (e.g. not even changing default passwords), the latter seems probably preferable to the former. But perhaps this is all just a matter to be sorted out in the UI. DD-WRT and OpenWRT assume that users are adults and non-stupid, and I, for one, certainly prefer to be treated that way. But for garden variety consumers it might not be such a bad idea to first ask them to provide the cube root of 27, or the airspeed velocity of an unladen swallow, or the answer to Life, The Universe and Everything before allowing them to increase their WiFi transmit power above FCC legal limits, or before allowing them to disengage the handbrake on their Roomba outbound bandwidth limit. (Note to self: Patent idea: Intellectual CAPTCHAs... you must be at least this non-stupid in order to proceed past this point. HEADLINE: Sixty Eight Percent of American Adults Flunk Turing Test, Cannot Be Reliably Distinguished From Mindless Automatons -- Ninety Seven Percent For First-Line Tech Support Professionals :-) Regards, rfg From josh at kyneticwifi.com Thu Oct 27 03:54:17 2016 From: josh at kyneticwifi.com (Josh Reynolds) Date: Wed, 26 Oct 2016 22:54:17 -0500 Subject: Spitballing IoT Security In-Reply-To: <20161026171907.GA84841@ussenterprise.ufp.org> References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: i think this would be the most effective route proposed so far. May the force be with you :) On Wed, Oct 26, 2016 at 12:19 PM, Leo Bicknell wrote: > In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: >> The makers of IoT devices are falling all over themselves to rush products >> to market as quickly as possible in order to maximize their profits. They >> have no time for security. They don't concern themselves with privacy >> implications. They don't run networks so they don't care about the impact >> their devices may have on them. They don't care about liability: many of >> them are effectively immune because suing them would mean trans-national >> litigation, which is tedious and expensive. (And even if they lost: >> they'd dissolve and reconstitute as another company the next day.) >> They don't even care about each other -- I'm pretty sure we're rapidly >> approaching the point where toasters will be used to attack garage door >> openers and washing machines. > > You are correct. > > I believe the answer is to have some sort of test scheme (UL > Labratories?) for basic security and updateability. Then federal > legislation is passed requiring any product being imported into the > country to be certified, or it is refused. > > Now when they rush to market and don't get certified they get $0 > and go out of business. Products are stopped at the boader, every > shipment is reviewed by authorities, and there is no cross boarder > suing issue. > > Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a > host of others have regulations that if you want to import a product > for sale it must be safe. It's not a new or novel concept, pretty > much every country has some scheme like it. > > -- > Leo Bicknell - bicknell at ufp.org > PGP keys at http://www.ufp.org/~bicknell/ From rfg at tristatelogic.com Thu Oct 27 05:28:48 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Wed, 26 Oct 2016 22:28:48 -0700 Subject: Spitballing IoT Security In-Reply-To: <58112F9F.6060705@vaxination.ca> Message-ID: <13447.1477546128@segfault.tristatelogic.com> In message <58112F9F.6060705 at vaxination.ca>, Jean-Francois Mezei wrote: >A camera showing the baby in 4K resolution along witgh sounds of him >crying on dolby surround to the mother who is at work would likely >saturate upload just as much as the virus sending DNS requests. This >falls into the tonne of feathers weighting as much as a tonne of lead >category. Agreed. So the "solution" is either to define such devices out of the problem set (i.e. to say "that is not really an IoT type device") or else to find some other solution. Questions: Does the 4K baby monitor (or the 4K 7-11 parking lot cam) need to be sending its high-bandwidth outbound feed, arbitrarily, to absolutely anything? Could it instead reasonably be limited to sending its high- bitrate data _only_ back to just those clients which themselves had first made an inbound TCP connection to the thing? (Note: The video data itself wouldn't necessarily have to travel back to the client via the TCP session. It could be sent back to the client via a separate and parallel UDP data link directed back to the same IP that initiated, and that is currently holding open the TCP session. I think that this is kinda/sorta how FTP works, actually.) IoT devices that need to send a *lot* of data out can be programmed to only send such high-rate/high-bulk data to client IP addresses that currently have live TCP sessions open... ones which the clients them- selves initiated... and the kernel can be made to enforce this simple restriction. Problem solved. No DDoSing of arbitrary IP addreses here! Move along. Alternatively, in the model where the security camera needs, for whatever reason, silly or otherwise... to be the one that initiates an outbound connection (and then just blasts its data up through that) it seems to me that it should not be too awfully hard to minimally enforce, in the kernel, just step 1 of a kind of SMTP-ish protocol... one where the IoT device initiates the outbound connection, to an IP address of its choosing (perhaps after having done a DNS lookup to find it) and where the IoT device then does nothing unless and until it gets some kind of affirmative signal from the other side... like a 2xx banner/greeting which effectively says to the IoT device "I'm here, and you are Clear-To-Send." (Again, it isn't necessary for the IoT device to send the actual data stream up through this TCP connection. It could be sent via UDP, but only to the pre-verified "cloud" IP address that we have already established is willing to accept the bulk data.) Of course, it would be best if there were some sort of a standardized port number and protocol for this specific kind of IoT-to-Cloud interaction. It would surely cause problems to try to overload these semantics on top of, say, port 25. So, in summary, it isn't even necessary to define video cameras out of the "IoT" problem set. The problem is excessive outbound data flowing to perfectly arbitrary "victim" IP addresses. (And remember, even attack reflectors are victims too.) Given the problem statement, the solution is obvious: You gotta start building boxes that have kernel-enforced restrictions that fit one or the other of these two models: 1) Ordinary (non-cloud-oriented) things MUST either: 1a) be prevented from sending large amounts of data outbound AT ALL, ever, or else 1b) be prevented from send large amounts of data outbound *except* when explicitly requested to do so by some verified IP address... which is to say an IP address that has initiated and completed an inbound TCP handshake 2) Cloud-Oriented things MUST be prevented from sending "unsolicited" bulk data to any IP address other than one which has very explicitly consented to receive it, i.e. by accepting and completing an inbound TCP handshake on some specially reserved port, and perhaps also via some additional layered trivial RTS/CTS protocol. That's it. Simple, no? Implementation is of course, completely trivial, just as it is for all things that I myself don't actually have to write the code for. Regards, rfg From randy at psg.com Thu Oct 27 05:44:57 2016 From: randy at psg.com (Randy Bush) Date: Thu, 27 Oct 2016 14:44:57 +0900 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: actually, the one technical hack i liked the most so far was the suggestion to put throttling into openwrt/lede, as they are used for the base in much cpe. randy From lear at ofcourseimright.com Thu Oct 27 05:59:00 2016 From: lear at ofcourseimright.com (Eliot Lear) Date: Thu, 27 Oct 2016 07:59:00 +0200 Subject: Spitballing IoT Security In-Reply-To: <580F19BF.2070002@vaxination.ca> References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> Message-ID: Hi Jean-Francois, On 10/25/16 10:37 AM, Jean-Francois Mezei wrote: > On 2016-10-25 04:10, Ronald F. Guilmette wrote: > >> If all of the *&^%$# damn stupid vacation pet feeders had originally shipped >> with outbound rate limits hard-coded in the kernel, maybe this could have >> been avoided. > > I view this differently. > > The problem is in allowing inbound connections and going as far as doing > UPnP to tell the CPE router to open a inbound door to let hackers loging > to that IoT pet feeder to turn it into an agressive DNS destroyer. Well yes. uPnP is a problem precisely because it is some random device asserting on its own that it can be trusted to do what it wants. Had that assertion come from the manufacturer, at least you would know that the device was designed to require that sort of access.** > > Then again, you need to have the owner access the pet feeder from the > remote beach to feed the dog. > > One way around this is for the pet feeder to initiate outbound > connection to a central server, and have the pet onwer connect to that > server to ask the server to send command to his pet feeder to feed the dog. Precisely so. > > This way, there need not be any inbound connection to the pet feeder. > > But if instead of a pet feeder we're talking about a home file sharing system or a video camera where you don't want to share the feed into the cloud? There will be times when people want inbound connections. We need an architecture that supports them. Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From job at ntt.net Thu Oct 27 06:19:46 2016 From: job at ntt.net (Job Snijders) Date: Thu, 27 Oct 2016 08:19:46 +0200 Subject: Large BGP Communities beacon in the wild In-Reply-To: <20161011150156.GF4457@hanna.meerval.net> References: <20161011150156.GF4457@hanna.meerval.net> Message-ID: <20161027061946.GF37101@Vurt.local> Dear Internet, Through this beacon it was discovered that a vendor was squatting on BGP Path Attribute value 30. And another vendor sat on 31. So, a twisted turn of events, the Large BGP Communities effort has ended up with BGP Path Attribute value 32 - very befitting if you look at the very problem we're trying to solve :-) The beacon has been updated to use the new IANA assigned value, nothing else was changed. Hopefully we are in the clear this time around! Please verify if you can see 192.147.168.0/24 and 2001:67c:208c::/48 Kind regards, Job On Tue, Oct 11, 2016 at 05:01:56PM +0200, Job Snijders wrote: > Large BGP Communities are a novel way to signal information between > networks. An example of a Large BGP Communities is: 2914:4056024901:80. > > Large BGP Communities are composed of three 4-octet integers, separated > by something like a colon. This is easy to remember and accommodates > advanced routing policies in relation to 4-Byte ASNs. It is the tool that has > been missing since 4-octet ASNs were introduced. > > IANA has made an Early Allocation of the value 30 (LARGE_COMMUNITY) in > the "BGP Path Attributes" registry under the "Border Gateway Protocol > (BGP) Parameters" group. > > The draft can be read here: https://tools.ietf.org/html/draft-ietf-idr-large-community > > Additional information about Large BGP Communities can be found here: > http://largebgpcommunities.net/ > > Starting today (2016.10.11), the following two BGP beacons are available > to the general public, with AS_PATH 2914_15562$ > > Both these prefixes have a Large BGP Community attached: > > 2001:67c:208c::/48 > 192.147.168.0/24 > > Large BGP Community - 15562:1:1 > > The NLNOG RING BGP Looking Glass is running the latest version of BIRD > which understands the Large BGP Community Path Attribute. > > IPv4 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=192.147.168.0/24 > IPv6 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2001:67c:208c::/48 > > In theory, since this is an optional transitive BGP Path Attribute, all > the Looking Glass' peers should boomerang the Large Community back to > the LG. However we currently observe that 50 out of 75 peers propagate > the Large BGP Community to the LG. > > Relevant Router commands to see if you receive the attribute, or whether > one of intermediate networks has stripped the attribute from the route: > > IOS: show ip bgp path-attribute unknown > shows all prefixes with unknown path attributes. > > IOS #2 - like on route views: > route-views>sh ip bgp 192.147.168.0 > BGP routing table entry for 192.147.168.0/24, version 98399100 > Paths: (39 available, best #30, table default) > Not advertised to any peer > Refresh Epoch 1 > 701 2914 15562 > 137.39.3.55 from 137.39.3.55 (137.39.3.55) > Origin IGP, localpref 100, valid, external > unknown transitive attribute: flag 0xE0 type 0x1E length 0xC > value 0000 3CCA 0000 0001 0000 0001 > rx pathid: 0, tx pathid: 0 > > IOS-XR: (you must look at specific prefixes) > RP/0/RSP0/CPU0:Router#show bgp ipv6 unicast 2001:67c:208c::/48 unknown-attributes > BGP routing table entry for 2001:67c:208c::/48 > Community: 2914:370 2914:1206 2914:2203 2914:3200 > Unknown attributes have size 15 > Raw value: > e0 1e 0c 00 00 3c ca 00 00 00 01 00 00 00 01 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > JunOS: > user at JunOS-re6> show route 2001:67c:208c::/48 detail > 2001:67c:208c::/48 (1 entry, 1 announced) > AS path: 15562 I > Unrecognized Attributes: 15 bytes > Attr flags e0 code 1e: 00 00 3c ca 00 00 00 01 00 00 00 01 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > A note about router Configurations: > > Ensure you are not fitlering the path attributes, eg: > > JunOS: > [edit protocols bgp] > user at junos# delete drop-path-attributes 30 > > XR: > configure > router bgp YourASN > attribute-filter group ReallyBadIdea ! avoid creating bogons > no attribute 30 > ! > ! > > Contact persons: myself or Jared Mauch or NTT NOC. BGP Session > identifier 83.231.213.230 / 2001:728:0:5000::a92 AS 15562. > > Kind regards, > > Job > From marka at isc.org Thu Oct 27 08:49:39 2016 From: marka at isc.org (Mark Andrews) Date: Thu, 27 Oct 2016 19:49:39 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Wed, 26 Oct 2016 17:27:08 -0700." <12439.1477528028@segfault.tristatelogic.com> References: <12439.1477528028@segfault.tristatelogic.com> Message-ID: <20161027084939.5BDF457D0FFB@rock.dv.isc.org> In message <12439.1477528028 at segfault.tristatelogic.com>, "Ronald F. Guilmette" writes: > > In message <20161026205800.7188D57B29B8 at rock.dv.isc.org>, > Mark Andrews wrote: > > >Actually things have changed a lot in a positive direction. > >... > >* Microsoft, Apple, Linux and *BSD issue regular fixes for their > > products and users do intall them. > > At the risk of repeating a point I have already made in this thread, please > do let me know how I can obtain this month's security patches for my iPhone > 3GS. Your assuming that there is a need for a security update each month. The feature set is pretty stable at this point. > (Note that Wikipedia says that this device was only formally discontinued > by the manufacturer as of September 12, 2012, i.e. only slightly more > than 4 short years ago. Nontheless, the current "security solution" for > this product, as made available from the manufacturer... a manufacturer > which is here being held up as a shining example of ernest social responsi- > bility... is for me to contribute the entire device to my local landfill, > where it will no doubt leach innumerable heavy metals into the soil for > my children's children's children to enjoy.) Well the last update for the 3GS was iOS 6.1.6 in Feb 2014. > >> - Manufacturers need to be held accountable for devices that go on the > >> internet... > > My iPhone 3GS "goes on the Internet". > > Through no fauly of my own, it is also, apparently, destined in short order > to "go onto" a landfill, if not here, then in China or India, where a > pitiful plethora of shoeless and sad-eyed third-world waifs will spend > their childhoods picking through the mand-made mountains of e-refuse in a > daily and desperate search for of anything of value. > > http://motherboard.vice.com/read/much-of-americas-e-waste-recycling-is-a-sham > > In short, if the "good" companies, like Apple, are the solution to the problem, > then I obviously misunderstood what "the problem" is, and would be obliged > if someone (anyone) would re-phrase it for me in simpler terms, for the > benefit of my limited little noggin. > > In lieu of that, for the moment I'd just like to emphasize again that it > is my opinion that any "solution" to the now self-evident IoT problems > which relies, even in the slightest, upon manufacturers providing a con- > tinuous and timely stream of security updates is a fantasy. Wishful > thinking, pure and simple. When even the "good" companies have built > their fortunes and entire business models around convincing/forcing > everyone to purchase "new and improved" units every two years, at a > maximum, and when the same said companies stop issuing patches of any > kind for products that have only departed the corporate price list > three years earlier, then one shudders to even contemplate what the > contribution of the "bad" companies will be to this ongoing catastrophy. > > > Regards, > rfg -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From tim at pelican.org Thu Oct 27 08:53:31 2016 From: tim at pelican.org (tim at pelican.org) Date: Thu, 27 Oct 2016 09:53:31 +0100 (BST) Subject: Spitballing IoT Security In-Reply-To: <12301.1477525252@segfault.tristatelogic.com> References: <12301.1477525252@segfault.tristatelogic.com> Message-ID: <1477558411.730528979@apps.rackspace.com> On Thursday, 27 October, 2016 00:40, "Ronald F. Guilmette" said: > Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. > My guess is that if any typical/average user is seen to be using more > than, say, 1/10 of that amount of "up" bandwidth in any one given 10 > minute time period, then something is really really REALLY wrong. This sounds like a horrible view of the Internet as "TV, only with more funny cat pictures", where most users are in a second-tier that is only expected / allowed to consume. One of the reasons I'm very grateful for FTTC / VDSL is that I can finally get a useful upstream speed. Going from 10-14M downstream to 80M was very nice, but going from 1M to 20M upstream was an absolute game-changer. I back up to the cloud - and there are plenty of services that allow regular, non-technical users to do this. The initial run saturated my upstream for days, and the incrementals are sometimes 20 or 30 minute bursts. I wouldn't even have tried on ADSL. Every time I get back from a day out, or even more so from a holiday, I upload the photos from my PC to one or more cloud services. I'll max my uplink for anywhere between 10 minutes and an hour - on the old ADSL, it was easily an overnight task. Working from home, I can now work directly with files on network shares, rather than copying everything to the laptop before I leave the office and trying to sync changes when I get back. I know the exact figures for this case, but there are a *lot* of spikes over the course of a day. With ADSL, I could go and make tea every time I needed to save a large Word doc or Powerpoint back to the network. On top of that, I can spend anything up to 3 or 4 hours in videoconferences, which will have a steady stream of a few hundred Kb/s. Spotting atypical (or ideally malicious) traffic is a valid goal, but I think we need to be a whole lot smarter than "customer is using upstream". Regards, Tim. From mike.meredith at port.ac.uk Thu Oct 27 09:04:55 2016 From: mike.meredith at port.ac.uk (Mike Meredith) Date: Thu, 27 Oct 2016 10:04:55 +0100 Subject: Spitballing IoT Security In-Reply-To: References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> Message-ID: <20161027100455.3fe4cf14@scrofula.eps.is.port.ac.uk> On Thu, 27 Oct 2016 07:59:00 +0200, Eliot Lear may have written: > Well yes. uPnP is a problem precisely because it is some random device > asserting on its own that it can be trusted to do what it wants. Had From my own personal use (and I'm aware that this isn't a general solution), I'd like a device that sat on those uPnP requests until I logged into the admin interface to review them. Now if you could automate _me_ then it might become more generally useful :- uPnP(ssh, for admin access) -> f/w f/w -> uPnP device: Don't be silly. > But if instead of a pet feeder we're talking about a home file sharing > system or a video camera where you don't want to share the feed into the > cloud? There will be times when people want inbound connections. We > need an architecture that supports them. As someone who manages an application-based firewall, every problem looks like it would be easier to solve using an application-based firewall :) -- Mike Meredith, University of Portsmouth Principal Systems Engineer, Hostmaster, Security, and Timelord! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From bicknell at ufp.org Thu Oct 27 11:26:01 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Thu, 27 Oct 2016 04:26:01 -0700 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <20161027112601.GA17170@ussenterprise.ufp.org> In a message written on Wed, Oct 26, 2016 at 04:40:57PM -0300, jim deleskie wrote: > So device is certified, bug is found 2 years later. How does this help. > The info to date is last week's issue was patched by the vendor in Sept > 2015, I believe is what I read. We know bugs will creep in, (source anyone > that has worked with code forever) Also certification assuming it would > work, in what country, would I need one, per country I sell into? These > are not the solutions you are looking for ( Jedi word play on purpose) You're referencing a wider problem set than I am trying to solve. Problems I think consumer safety legislation can solve: * SSH and Telnet were enabled, but there was no notification in the UI that they were enabled and no way to turn them off. Requirements could be set to show all services in the UI and if they are on or off. * There was a hard coded user + pass that the consumer COULD NOT CHANGE, and did not display. Requirements could be set to never hard code an account. * That the system has a user-friendly way to update. "Click here to check for update." "Click here to install update." What consumer safety legislation can't do is insure a patch is made available at some point in the future. As for certification, I will point out minimally all of these products are already geting CE, UL, and FCC (if Wireless). They also have to meet other regulations (e.g. RoHS) to be imported. To really minimize burden, these security items could be added to one of the existing schemes so there is no additional org. But the idea that a certification per country is difficult is pretty much debunked by the fact that it is that way already, multiple times over in most cases. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From bicknell at ufp.org Thu Oct 27 11:29:40 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Thu, 27 Oct 2016 04:29:40 -0700 Subject: Spitballing IoT Security In-Reply-To: <12439.1477528028@segfault.tristatelogic.com> References: <20161026205800.7188D57B29B8@rock.dv.isc.org> <12439.1477528028@segfault.tristatelogic.com> Message-ID: <20161027112940.GB17170@ussenterprise.ufp.org> In a message written on Wed, Oct 26, 2016 at 05:27:08PM -0700, Ronald F. Guilmette wrote: > do let me know how I can obtain this month's security patches for my iPhone > 3GS. > > (Note that Wikipedia says that this device was only formally discontinued > by the manufacturer as of September 12, 2012, i.e. only slightly more > than 4 short years ago. Nontheless, the current "security solution" for > this product, as made available from the manufacturer... a manufacturer > which is here being held up as a shining example of ernest social responsi- > bility... is for me to contribute the entire device to my local landfill, > where it will no doubt leach innumerable heavy metals into the soil for > my children's children's children to enjoy.) Actually, they encourage you to trade it in, where it is used for replacement parts and/or recycled, see http://www.apple.com/iphone/trade-up/. If your device is too old for that program, they will still take it for free and recycle it in an enviornmentally friendly way, see http://www.apple.com/recycling/. No iPhone should ever end up in a landfill. If it does, it's your fault for not taking advantage of the free recycling. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From geoffk at geoffk.org Thu Oct 27 13:35:46 2016 From: geoffk at geoffk.org (Geoffrey Keating) Date: 27 Oct 2016 06:35:46 -0700 Subject: Spitballing IoT Security In-Reply-To: <12439.1477528028@segfault.tristatelogic.com> References: <20161026205800.7188D57B29B8@rock.dv.isc.org> <12439.1477528028@segfault.tristatelogic.com> Message-ID: "Ronald F. Guilmette" writes: > My iPhone 3GS "goes on the Internet". > > Through no fauly of my own, it is also, apparently, destined in short order > to "go onto" a landfill, if not here, then in China or India, where a > pitiful plethora of shoeless and sad-eyed third-world waifs will spend > their childhoods picking through the mand-made mountains of e-refuse in a > daily and desperate search for of anything of value. Please don't, bring it to your nearest Apple Store instead where it will be properly recycled, . From nanog at deltasly.com Thu Oct 27 03:46:36 2016 From: nanog at deltasly.com (knack) Date: Wed, 26 Oct 2016 22:46:36 -0500 Subject: Spitballing IoT Security In-Reply-To: References: <1722.1477340699@segfault.tristatelogic.com> <20161026120634.GA20735@gsp.org> <20161026171907.GA84841@ussenterprise.ufp.org> Message-ID: <5811789C.40209@deltasly.com> I agree wholeheartedly. ---- Yes, BCP (any relevant to your business), filtering, active tit-for-tat with abuse teams, calling out manufacturers, ISPs doing /anything/ (most already block 25 and 80, not that they give you the upload to bother with the latter and it's not necessarily for the good of the 'net as a whole) - they're all things we absolutely should be doing. That doesn't change the fact that all of those are just*ambulances in the valley*. If we're going to solve this, we need to be better as a species - we're about to the place where we can't progress much farther (without some sort of caste system nightmare - /The Diamond Age/ comes to mind) *until basic computing and good practice are as pervasive as the ability to read and write.* Hint: I don't mean 'can do an app on the smartphone' - real understanding and appreciation. I'm not saying everyone needs to be a savant, but a basic understanding of the technology you*^1 use *every.single.day* for almost *every.single.function* of your life isn't asking much, I don't think. The ability to think logically and problem solve is something that I see declining in even the brighter of the youths I've encountered in the past few years. This "it just works/should work" willfully (almost maliciously sometimes) ignorant mentality - pushed by vendors and craved by the overworked - is stunting our potential. Christ, the people who are *in charge of the world* (not necessarily those who /run/ it...but I'd be a good portion still) don't even understand the basics of how these machines, and this thing that facilitates the global economy, work. The root problem is much, much more significant than DoS attacks and spam. Maybe we need to start younger - I can't speak for all schools, but my 'computer course' was "here's Mavis Beacon - play games and...whatever" - I hope it's not [really | still] like that. Maybe we, the community, create and sponsor course material, maybe there's a push for more than Cisco Academy - at this point this knowledge a public safety issue and should be a respected part of the general education syllabus (too bad we're all too busy with standardized tests to care). Something so inherently part of everyday life cannot be just for the 'nerds' or even the especially interested, anymore. I don't know what to do about manufacturers - it's been a global race to bottom for years now. Someone mentioned a device certification earlier. It's something and a start at least, so I'm on board and willing to devote some time. I'm not sure this is something the community alone will be able to drive, silently, from the shadows. The cynic in me wants to throw in to buy a politician or two. The usual trick is to hit them where it hurts - in the wallet. Their wallets are so large these days (and constantly consolidating, lessening the chance of real change and competition) that I'm at a loss as to how. Maybe a slow increase in user-required configurations, decisions, and interaction...complete with logical explanations to help with said decisions? I don't know...that could affect profits this quarter (because who looks farther ahead than that...long term effects and progress aren't important anymore, right?). Pavlovian training? Seems a bit totalitarian. The /last/ thing I want is government (on the country or global scale) intervention..the 2nd to last is to use this upcoming metaphor (but I haven't a better one). Look at cars - in more places then not, it's damn near impossible to be a functional and contributory adult without a car; some might even call it a 'right' in the 1st world. Does that stop us from driver's ed courses**^2 in school? Do we not teach the basics of safe operation, maintenance, and even a bit about how it works under the hood (my school did)? Does the ubiquity of the automobile stop the removal of that (legal) ability for those who *endanger others* or otherwise abuse the driving privilege? No...no it doesn't. Granted, there are still those who are going to do what they're going to do - but that number is lessened (and some even come around to see the harm). That does *not* mean I think there should be a 'compu-tar license' - not at all. But it *does* mean that everyone should be taught responsible computing, the harms of carelessness, and the fun in knowing how these things work. Anyway - thanks for the rant (been bothering me for a while now)...I do believe we should address and minimize the symptoms as they appear, but without surgical attacks directed at the dark heart of the beast (be that people, intrinsically, or just our social norms) we're going to end up with either a horribly censored, totalitarian internet "app" regime, or burnt to the ground in chaos - too distracted by inane, emotionally infused, bullshit pumped forth day-to-day at an ADD inducing pace (meant to give us the ol' numb & dumb - I'll admit I succumb more often than I'd like - not trying to high-horse here), to notice the fires until it's too late to stomp them out. I never imagined we***^3 could become so dichotomous-ly obsessed yet ignorant. Yes, there will always be malicious people but, in the same way we convinced most of the world that sacrificing humans is murder and kinda wrong (and engaged them in at least a few active prevention tactics), we need to stigmatize -- really */really/* stigmatize (to the core of the soul) this bullshit. On a side note: giving 10 years to the guy who just wanted to tinker with "his own" (because we don't /own/ anything anymore, in a hyperbolic way) equipment isn't the way to do it. Everything seems overly fatalistic and over-dramatized until the moment it's not - how many disasters could have been prevented if people just listened to the engineers (ask the Challenger)? Of course, we're still prescribing antibiotics for virii in the face of MRSA and worse - hell Pompeii partied until they were literally dying in the streets...so let's drink up, add a little duct tape, and worry about it in a few years (/s). Or, we keep pushing, wherever possible, and maybe something will pop. Seldom do I wish to be proven incorrect - here I do. Contrary to what it may seem, I think we***^3 still have a chance. *1: That's /you, the generalization I'm referring to/, and not /you, the specific people reading this./ **2: Though, unfortunately due to that government intervention, we spent more time memorizing the specific BAC to age ratio to determine your fine and loss of license than honing basic knowledge and skills. ***3: Again, that's /we, as a society of 7 billion - call it a median or mode/ rather than /we, individuals in a set/. ---- (Disclaimer: I don't like speaking publicly, especially at this length (though I've cut out a good 60%, as I admit I have a rambling problem). I've spent the last week writing and re-writing versions of this; I still don't like it (both overly idealistic and fatalistic at the same time, and the "voice" is much harsher than I would have liked - the tradeoff of curtailing the rambling I suppose), but I had a strong reaction to this subject. ...And yes I even debated the disclaimer, as it's hokey as all getout...best I could muster was to move it to the end. My apologies if I've overlooked points below being covered previously in the thread - /thank you for the ear & I'm sorry/?). ~knack On 10/26/2016 3:12 PM, Ken Matlock wrote: > As a relative 'outsider' I see a lot of finger-pointing and phrasing this > as (effectively) someone else's fault. > > To me this is a failing on a number of levels all contributing to the > problem. > > 1) The manufacturer - Backdoors, hidden accounts, remote access > capabilities, no proper security testing. No enforcing of security updates. > 2) The end-user - No initiative on the end-user's perspective to gain even > a basic understanding of how the device works, connects, etc. Also no tools > or understanding of how to recognize *which* of their many devices on the > network might be compromised and participating in the botnet. (Only > indication they get is maybe their internet is slow) > 3) The service providers - No effective monitoring of outgoing traffic from > the end users to identify botnets and DDoS in a real-time fashion > > I contend that all 3 levels have failed in this, and nothing has > fundamentally changed (today it's IoT, before it was unpatched windows > boxes, etc) in decades. We keep talking about the problem but very little > actual action has occurred to *fix* the underlying issues. > > - Manufacturers need to be held accountable for devices that go on the > internet (that includes *anything* that's connected. PCs, servers, routers, > IoT devices, etc) > - End users need to have ways to easily see what's going on over their > local networks, to see botnet-like activity and DDoS participation (among > other things) in a more real-time fashion > - Service providers need to be much more proactive in watching for threats > and identifying/blocking them at the source, not allowing the traffic to > flow to your peers and making it someone else's problem. Right now there's > a financial disincentive to doing this, in both real costs (standing up > monitoring gear/etc), and imagined (my ISP is SPYING on me!). > > Until we fix all 3 of these main issues we're just going to keep going in > the same set of circles we do every time a 'new' threat/vector comes in. > > Now, are these issues *easy*? Oh, heck no! Are they *cheap*? Once again, > heck no! But to 'fix' this issue it will take all 3 levels being fixed. > > If we continue to keep pointing fingers at "the other guy" as the root of > the problem we're inviting external forces (Legislation) to step in and > 'fix' the problem for us (and it will just make it worse). > > My 2 cents (adjust for inflation) > Ken > > On Wed, Oct 26, 2016 at 1:40 PM, jim deleskie wrote: > >> So device is certified, bug is found 2 years later. How does this help. >> The info to date is last week's issue was patched by the vendor in Sept >> 2015, I believe is what I read. We know bugs will creep in, (source anyone >> that has worked with code forever) Also certification assuming it would >> work, in what country, would I need one, per country I sell into? These >> are not the solutions you are looking for ( Jedi word play on purpose) >> >> On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < >> jordi.palet at consulintel.es> wrote: >> >>> Exactly, I was arguing exactly the same with some folks this week during >>> the RIPE meeting. >>> >>> The same way that certifications are needed to avoid radio interferences, >>> etc., and if you don?t pass those certifications, you can?t sell the >>> products in some countries (or regions in case of EU for example), >>> authorities should make sure that those certifications have a broader >>> scope, including security and probably some other features to ensure that >>> in case something is discovered in the future, they can be updated. >>> >>> Yes, that means cost, but a few thousand dollars of certification price >>> increase, among thousands of millions of devices of the same model being >>> manufactured, means a few cents for each unit. >>> >>> Even if we speak about 1 dollar per each product being sold, it is much >>> cheaper than the cost of not doing it and paying for damages, human >>> resources, etc., when there is a security breach. >>> >>> Regards, >>> Jordi >>> >>> >>> -----Mensaje original----- >>> De: NANOG en nombre de Leo Bicknell < >>> bicknell at ufp.org> >>> Organizaci?n: United Federation of Planets >>> Responder a: >>> Fecha: mi?rcoles, 26 de octubre de 2016, 19:19 >>> Para: >>> Asunto: Re: Spitballing IoT Security >>> >>> In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich >>> Kulawiec wrote: >>> > The makers of IoT devices are falling all over themselves to rush >>> products >>> > to market as quickly as possible in order to maximize their >>> profits. They >>> > have no time for security. They don't concern themselves with >>> privacy >>> > implications. They don't run networks so they don't care about the >>> impact >>> > their devices may have on them. They don't care about liability: >>> many of >>> > them are effectively immune because suing them would mean >>> trans-national >>> > litigation, which is tedious and expensive. (And even if they >> lost: >>> > they'd dissolve and reconstitute as another company the next day.) >>> > They don't even care about each other -- I'm pretty sure we're >>> rapidly >>> > approaching the point where toasters will be used to attack garage >>> door >>> > openers and washing machines. >>> >>> You are correct. >>> >>> I believe the answer is to have some sort of test scheme (UL >>> Labratories?) for basic security and updateability. Then federal >>> legislation is passed requiring any product being imported into the >>> country to be certified, or it is refused. >>> >>> Now when they rush to market and don't get certified they get $0 >>> and go out of business. Products are stopped at the boader, every >>> shipment is reviewed by authorities, and there is no cross boarder >>> suing issue. >>> >>> Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a >>> host of others have regulations that if you want to import a product >>> for sale it must be safe. It's not a new or novel concept, pretty >>> much every country has some scheme like it. >>> >>> -- >>> Leo Bicknell - bicknell at ufp.org >>> PGP keys at http://www.ufp.org/~bicknell/ >>> >>> >>> >>> >>> ********************************************** >>> IPv4 is over >>> Are you ready for the new Internet ? >>> http://www.consulintel.es >>> The IPv6 Company >>> >>> This electronic message contains information which may be privileged or >>> confidential. The information is intended to be for the use of the >>> individual(s) named above. If you are not the intended recipient be aware >>> that any disclosure, copying, distribution or use of the contents of this >>> information, including attached files, is prohibited. >>> >>> >>> >>> From mel at beckman.org Thu Oct 27 14:02:16 2016 From: mel at beckman.org (Mel Beckman) Date: Thu, 27 Oct 2016 14:02:16 +0000 Subject: Spitballing IoT Security In-Reply-To: <20161027100455.3fe4cf14@scrofula.eps.is.port.ac.uk> References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> , <20161027100455.3fe4cf14@scrofula.eps.is.port.ac.uk> Message-ID: Requiring manual approval is an excellent idea for the ThingSafe RFC! -mel > On Oct 27, 2016, at 2:10 AM, Mike Meredith wrote: > > On Thu, 27 Oct 2016 07:59:00 +0200, Eliot Lear > may have written: >> Well yes. uPnP is a problem precisely because it is some random device >> asserting on its own that it can be trusted to do what it wants. Had > > From my own personal use (and I'm aware that this isn't a general > solution), I'd like a device that sat on those uPnP requests until I logged > into the admin interface to review them. Now if you could automate _me_ > then it might become more generally useful :- > > uPnP(ssh, for admin access) -> f/w > > f/w -> uPnP device: Don't be silly. > >> But if instead of a pet feeder we're talking about a home file sharing >> system or a video camera where you don't want to share the feed into the >> cloud? There will be times when people want inbound connections. We >> need an architecture that supports them. > > As someone who manages an application-based firewall, every problem looks > like it would be easier to solve using an application-based firewall :) > > -- > Mike Meredith, University of Portsmouth > Principal Systems Engineer, Hostmaster, Security, and Timelord! > From list at satchell.net Thu Oct 27 15:03:11 2016 From: list at satchell.net (Stephen Satchell) Date: Thu, 27 Oct 2016 08:03:11 -0700 Subject: Should abuse mailboxes have quotas? Message-ID: For the last couple of weeks, every single abuse mail I've tried to send to networks in a very short list of countries has bounced back with "mailbox exceeds quota". I take this to mean that there isn't someone actively reading, acting on, and deleting e-mail from abuse@. So my new rule is this: bounce an abuse e-mail message, sent to an abuse address announced for the netrange, and the ENTIRE NETRANGE gets put in my "reject forever" firewall. I've ask all my customers about this action, and all agree that it's reasonable, because an administration with an active abuse desk shouldn't ever have their abuse mailbox overflow. (Especially in this day of terabyte disks.) Or they need more people on their abuse desk. Or they need to eliminate the problem that generates so many abuse e-mails that it fills up their should-be-enormous mail queue. I'm tired of blatantly uncaring administrations. From morrowc.lists at gmail.com Thu Oct 27 15:08:25 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Thu, 27 Oct 2016 11:08:25 -0400 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 11:03 AM, Stephen Satchell wrote: > > I'm tired of blatantly uncaring administrations. > it's also totally possible that in some cases the mailbox for abuse@ got moved behind some orgs other mail systems... This happened numerous times at $PREVIOUS_EMPLOYER. When moving around ~200k mailboxes 1 special unicorn often gets mishandled :( we wouldn't find out until someone called in all complainy about how 'you never care about email... blah...' "Sure we care, but our mail-admin team sometimes breaks us, whoops!" ascribing malice is often unhelpful... Also, of course it's your network you can balkanize from the rest of the internet as much as you please. From owen at delong.com Thu Oct 27 15:47:08 2016 From: owen at delong.com (Owen DeLong) Date: Thu, 27 Oct 2016 08:47:08 -0700 Subject: Large BGP Communities beacon in the wild In-Reply-To: <20161027061946.GF37101@Vurt.local> References: <20161011150156.GF4457@hanna.meerval.net> <20161027061946.GF37101@Vurt.local> Message-ID: <270091A8-32B1-4519-A578-383C5A97332F@delong.com> I don?t mind the move to 32, but I hope the vendors are getting appropriately smacked for squatting and that those attributes are not allowed to be misappropriated by the vendors. We have a standards process for a reason and vendors simply squatting on numbers is a violation of that process which cannot be allowed to stand unless we wish to establish that as precedent and simply allow vendors to claim numbers as they wish. This already happened with the BSD community in their implementation of a pseudo-VRRP like capability and now two different vendors have abused BGP path attributes. This is not a good path for us to continue. Owen > On Oct 26, 2016, at 11:19 PM, Job Snijders wrote: > > Dear Internet, > > Through this beacon it was discovered that a vendor was squatting on BGP > Path Attribute value 30. And another vendor sat on 31. > > So, a twisted turn of events, the Large BGP Communities effort has ended up > with BGP Path Attribute value 32 - very befitting if you look at the very > problem we're trying to solve :-) > > The beacon has been updated to use the new IANA assigned value, nothing > else was changed. Hopefully we are in the clear this time around! > > Please verify if you can see 192.147.168.0/24 and 2001:67c:208c::/48 > > Kind regards, > > Job > > On Tue, Oct 11, 2016 at 05:01:56PM +0200, Job Snijders wrote: >> Large BGP Communities are a novel way to signal information between >> networks. An example of a Large BGP Communities is: 2914:4056024901:80. >> >> Large BGP Communities are composed of three 4-octet integers, separated >> by something like a colon. This is easy to remember and accommodates >> advanced routing policies in relation to 4-Byte ASNs. It is the tool that has >> been missing since 4-octet ASNs were introduced. >> >> IANA has made an Early Allocation of the value 30 (LARGE_COMMUNITY) in >> the "BGP Path Attributes" registry under the "Border Gateway Protocol >> (BGP) Parameters" group. >> >> The draft can be read here: https://tools.ietf.org/html/draft-ietf-idr-large-community >> >> Additional information about Large BGP Communities can be found here: >> http://largebgpcommunities.net/ >> >> Starting today (2016.10.11), the following two BGP beacons are available >> to the general public, with AS_PATH 2914_15562$ >> >> Both these prefixes have a Large BGP Community attached: >> >> 2001:67c:208c::/48 >> 192.147.168.0/24 >> >> Large BGP Community - 15562:1:1 >> >> The NLNOG RING BGP Looking Glass is running the latest version of BIRD >> which understands the Large BGP Community Path Attribute. >> >> IPv4 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=192.147.168.0/24 >> IPv6 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2001:67c:208c::/48 >> >> In theory, since this is an optional transitive BGP Path Attribute, all >> the Looking Glass' peers should boomerang the Large Community back to >> the LG. However we currently observe that 50 out of 75 peers propagate >> the Large BGP Community to the LG. >> >> Relevant Router commands to see if you receive the attribute, or whether >> one of intermediate networks has stripped the attribute from the route: >> >> IOS: show ip bgp path-attribute unknown >> shows all prefixes with unknown path attributes. >> >> IOS #2 - like on route views: >> route-views>sh ip bgp 192.147.168.0 >> BGP routing table entry for 192.147.168.0/24, version 98399100 >> Paths: (39 available, best #30, table default) >> Not advertised to any peer >> Refresh Epoch 1 >> 701 2914 15562 >> 137.39.3.55 from 137.39.3.55 (137.39.3.55) >> Origin IGP, localpref 100, valid, external >> unknown transitive attribute: flag 0xE0 type 0x1E length 0xC >> value 0000 3CCA 0000 0001 0000 0001 >> rx pathid: 0, tx pathid: 0 >> >> IOS-XR: (you must look at specific prefixes) >> RP/0/RSP0/CPU0:Router#show bgp ipv6 unicast 2001:67c:208c::/48 unknown-attributes >> BGP routing table entry for 2001:67c:208c::/48 >> Community: 2914:370 2914:1206 2914:2203 2914:3200 >> Unknown attributes have size 15 >> Raw value: >> e0 1e 0c 00 00 3c ca 00 00 00 01 00 00 00 01 >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> JunOS: >> user at JunOS-re6> show route 2001:67c:208c::/48 detail >> 2001:67c:208c::/48 (1 entry, 1 announced) >> AS path: 15562 I >> Unrecognized Attributes: 15 bytes >> Attr flags e0 code 1e: 00 00 3c ca 00 00 00 01 00 00 00 01 >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> A note about router Configurations: >> >> Ensure you are not fitlering the path attributes, eg: >> >> JunOS: >> [edit protocols bgp] >> user at junos# delete drop-path-attributes 30 >> >> XR: >> configure >> router bgp YourASN >> attribute-filter group ReallyBadIdea ! avoid creating bogons >> no attribute 30 >> ! >> ! >> >> Contact persons: myself or Jared Mauch or NTT NOC. BGP Session >> identifier 83.231.213.230 / 2001:728:0:5000::a92 AS 15562. >> >> Kind regards, >> >> Job >> From bicknell at ufp.org Thu Oct 27 16:47:57 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Thu, 27 Oct 2016 09:47:57 -0700 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: <20161027164757.GA29809@ussenterprise.ufp.org> In a message written on Thu, Oct 27, 2016 at 08:03:11AM -0700, Stephen Satchell wrote: > For the last couple of weeks, every single abuse mail I've tried to send > to networks in a very short list of countries has bounced back with > "mailbox exceeds quota". I take this to mean that there isn't someone > actively reading, acting on, and deleting e-mail from abuse@. Are there any ISP's left that read and respond to abuse@ in a timely fashion? I haven't seen one in at least a decade. Maybe I e-mail the wrong ones. -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From steve at blighty.com Thu Oct 27 16:55:07 2016 From: steve at blighty.com (Steve Atkins) Date: Thu, 27 Oct 2016 09:55:07 -0700 Subject: Should abuse mailboxes have quotas? In-Reply-To: <20161027164757.GA29809@ussenterprise.ufp.org> References: <20161027164757.GA29809@ussenterprise.ufp.org> Message-ID: <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> > On Oct 27, 2016, at 9:47 AM, Leo Bicknell wrote: > > In a message written on Thu, Oct 27, 2016 at 08:03:11AM -0700, Stephen Satchell wrote: >> For the last couple of weeks, every single abuse mail I've tried to send >> to networks in a very short list of countries has bounced back with >> "mailbox exceeds quota". I take this to mean that there isn't someone >> actively reading, acting on, and deleting e-mail from abuse@. > > Are there any ISP's left that read and respond to abuse@ in a timely > fashion? I haven't seen one in at least a decade. Maybe I e-mail the > wrong ones. Lots. There are also quite a lot that don't. There are also many who you can't easily tell. Mail to abuse@ doesn't bounce, their abuse issues aren't horrific relative to the size of their customer base, so they're doing something right. And yet they have persistently abusive customers who sit on their networks for years. I've met the abuse staff at quite a few of those, and they're doing good work, but it's only visible statistically, not on a per-incident level. If mail to abuse@ doesn't bounce, give them the benefit of the doubt until statistics say otherwise. Cheers, Steve From johnl at iecc.com Tue Oct 25 04:50:20 2016 From: johnl at iecc.com (John Levine) Date: 25 Oct 2016 04:50:20 -0000 Subject: Should abuse mailboxes have quotas? In-Reply-To: <20161027164757.GA29809@ussenterprise.ufp.org> Message-ID: <20161025045020.5600.qmail@ary.lan> >Are there any ISP's left that read and respond to abuse@ in a timely >fashion? I haven't seen one in at least a decade. Maybe I e-mail the >wrong ones. Or maybe you send reports that they can't act on. Mine are all in ARF format and ISPs reply and tell me they've acted on them all the time. In many cases they reports go into ticketing systems, so they'll get acted on but you don't get an answer from a person. That's fine with me, I'd rather they spend time swatting bad guys than composing mail by hand. R's, John From johnl at iecc.com Tue Oct 25 04:52:58 2016 From: johnl at iecc.com (John Levine) Date: 25 Oct 2016 04:52:58 -0000 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <20161025045258.5628.qmail@ary.lan> >Please don't, bring it to your nearest Apple Store instead where it >will be properly recycled, . My nearest Apple stores are 50 miles away. I'm not sure 100 miles in the car is a good tradeoff for one phone. From nevin at yahoo-inc.com Thu Oct 27 17:36:13 2016 From: nevin at yahoo-inc.com (Nevin Gonsalves) Date: Thu, 27 Oct 2016 17:36:13 +0000 (UTC) Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> Message-ID: <1215211938.2287995.1477589773824@mail.yahoo.com> :-) http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 thanks,-nevin From bicknell at ufp.org Thu Oct 27 17:42:52 2016 From: bicknell at ufp.org (Leo Bicknell) Date: Thu, 27 Oct 2016 10:42:52 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161025045258.5628.qmail@ary.lan> References: <20161025045258.5628.qmail@ary.lan> Message-ID: <20161027174252.GA31857@ussenterprise.ufp.org> In a message written on Tue, Oct 25, 2016 at 04:52:58AM -0000, John Levine wrote: > My nearest Apple stores are 50 miles away. I'm not sure 100 miles in > the car is a good tradeoff for one phone. Scroll down a bit further: "Tell us which device you have, and we?ll email you a prepaid mailing label. Once you?ve deleted your data, ship your device to us, and we?ll handle the rest." -- Leo Bicknell - bicknell at ufp.org PGP keys at http://www.ufp.org/~bicknell/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 811 bytes Desc: not available URL: From rfg at tristatelogic.com Thu Oct 27 18:02:45 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 11:02:45 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027084939.5BDF457D0FFB@rock.dv.isc.org> Message-ID: <15949.1477591365@segfault.tristatelogic.com> In message <20161027084939.5BDF457D0FFB at rock.dv.isc.org>, Mark Andrews wrote: >Well the last update for the 3GS was iOS 6.1.6 in Feb 2014. Bingo! Less than a year and a half after they stopped selling it, they effectively stopped supporting it. From rfg at tristatelogic.com Thu Oct 27 18:10:58 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 11:10:58 -0700 Subject: Spitballing IoT Security In-Reply-To: <1477558411.730528979@apps.rackspace.com> Message-ID: <15979.1477591858@segfault.tristatelogic.com> In message <1477558411.730528979 at apps.rackspace.com>, "tim at pelican.org" wrote: >...I back up to the cloud... Yes, I confess that this reasonable use case had not occured to me, and yes, it utterly negates what I was saying. (I myself am the paranoid type, so I -do not- back up -any- of my stuff or things to any storage which is not physically in a place where I myself can lay hands on it. Given all of the massive breaches that have been in the news over the past few years, I'm just not comfortable sending my data into the hands of others. But other people are obviously happy to do this, and who am I to tell them they shouldn't? It is 100% reasonable for end users to occasionally use a lot of outbound bandwidth. Period. I'm sorry that I wasted electrons suggesting otherwise.) Regards, rfg From toddunder at gmail.com Thu Oct 27 18:18:11 2016 From: toddunder at gmail.com (Todd Underwood) Date: Thu, 27 Oct 2016 14:18:11 -0400 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: to answer the actual question: all abuse mailboxes have quotas, either implicitly or explicitly. the amount of storage available to any given mailsystem is finite. technically correct. it's the best kind of correct. :-) t On Thu, Oct 27, 2016 at 11:03 AM, Stephen Satchell wrote: > For the last couple of weeks, every single abuse mail I've tried to send > to networks in a very short list of countries has bounced back with > "mailbox exceeds quota". I take this to mean that there isn't someone > actively reading, acting on, and deleting e-mail from abuse@. > > So my new rule is this: bounce an abuse e-mail message, sent to an > abuse address announced for the netrange, and the ENTIRE NETRANGE gets > put in my "reject forever" firewall. I've ask all my customers about > this action, and all agree that it's reasonable, because an > administration with an active abuse desk shouldn't ever have their abuse > mailbox overflow. (Especially in this day of terabyte disks.) > > Or they need more people on their abuse desk. > > Or they need to eliminate the problem that generates so many abuse > e-mails that it fills up their should-be-enormous mail queue. > > I'm tired of blatantly uncaring administrations. > From rfg at tristatelogic.com Thu Oct 27 18:26:57 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 11:26:57 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027112601.GA17170@ussenterprise.ufp.org> Message-ID: <16081.1477592817@segfault.tristatelogic.com> In message <20161027112601.GA17170 at ussenterprise.ufp.org>, Leo Bicknell wrote: >Problems I think consumer safety legislation can solve: > >* SSH and Telnet were enabled, but there was no notification in the UI > that they were enabled and no way to turn them off. Requirements > could be set to show all services in the UI and if they are on or > off. > >* There was a hard coded user + pass that the consumer COULD NOT CHANGE, > and did not display. Requirements could be set to never hard code an > account. > >* That the system has a user-friendly way to update. "Click here to > check for update." "Click here to install update." I say again, #3 is useless, unless and until you also have legislation that: *) Forces tech companies to never go bankrupt. *) Forces tech companies to -timely- issue security patches for all "critical" security issues (and good luck legally defining THAT). *) Forces tech companies to continue to issue security patches for as long as any "significant" number of the relevant devices remain actively in use, even if that turns out to be 20 years or more. You can force a company to implement a "user-friendly way to update", but what's the point of doing that if the company never issues any updates? I say again, the only way to solve these problems is if the devices are fundamentally secure by design, on the day they first ship to customers. Post-sale patching is an ad hoc and haphazard catch-as- catch-can solution at best, and it's not something that most manufacturers have -any- financial incentive to even do. They already got their money, on the day when the consumer bought the device. The rest is just an afterthought. Regards, rfg From goemon at sasami.anime.net Thu Oct 27 18:35:55 2016 From: goemon at sasami.anime.net (Dan Hollis) Date: Thu, 27 Oct 2016 11:35:55 -0700 (PDT) Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On Thu, 27 Oct 2016, Christopher Morrow wrote: > On Thu, Oct 27, 2016 at 11:03 AM, Stephen Satchell > wrote: >> I'm tired of blatantly uncaring administrations. > it's also totally possible that in some cases the mailbox for abuse@ got > moved behind some orgs other mail systems... This happened numerous times > at $PREVIOUS_EMPLOYER. When moving around ~200k mailboxes 1 special unicorn > often gets mishandled :( > > we wouldn't find out until someone called in all complainy about how 'you > never care about email... blah...' "Sure we care, but our mail-admin team > sometimes breaks us, whoops!" > > ascribing malice is often unhelpful... Also, of course it's your network > you can balkanize from the rest of the internet as much as you please. not so much malice as gross incompetence. running spamfilters on your abuse@ mailbox, really? that is, for those which actually have an abuse mailbox that doesn't bounce outright. -Dan From goemon at sasami.anime.net Thu Oct 27 18:37:47 2016 From: goemon at sasami.anime.net (Dan Hollis) Date: Thu, 27 Oct 2016 11:37:47 -0700 (PDT) Subject: Should abuse mailboxes have quotas? In-Reply-To: <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> References: <20161027164757.GA29809@ussenterprise.ufp.org> <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> Message-ID: On Thu, 27 Oct 2016, Steve Atkins wrote: > If mail to abuse@ doesn't bounce, give them the benefit > of the doubt until statistics say otherwise. I give them a couple weeks/months. The vast majority of them ignore, and allow the abuse to continue. It's amazing how quickly they respond when they get SBL'd though! Lightning quick. -Dan From edward.dore at freethought-internet.co.uk Thu Oct 27 18:41:28 2016 From: edward.dore at freethought-internet.co.uk (Edward Dore) Date: Thu, 27 Oct 2016 19:41:28 +0100 Subject: Spitballing IoT Security In-Reply-To: <15949.1477591365@segfault.tristatelogic.com> References: <15949.1477591365@segfault.tristatelogic.com> Message-ID: <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> On 27 Oct 2016, at 19:02, Ronald F. Guilmette wrote: > > > In message <20161027084939.5BDF457D0FFB at rock.dv.isc.org>, > Mark Andrews wrote: > >> Well the last update for the 3GS was iOS 6.1.6 in Feb 2014. > > Bingo! > > Less than a year and a half after they stopped selling it, they > effectively stopped supporting it. > At which point the 3GS was almost 5 years old (having originally been released in June 2009) and had been already superseded by the iPhone 4, 4S, 5 and 5S/5C. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From rfg at tristatelogic.com Thu Oct 27 18:55:38 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 11:55:38 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027112940.GB17170@ussenterprise.ufp.org> Message-ID: <16193.1477594538@segfault.tristatelogic.com> In message <20161027112940.GB17170 at ussenterprise.ufp.org>, Leo Bicknell wrote: >Actually, they encourage you to trade {your old iPhone} in... >... >If your device is too old for that program, they will still take >it for free and recycle it in an enviornmentally friendly way... OK, so good on them. I do compliment them for their apparent willingness to take back this pile of leachable heavy metals and do something responsible with it. But to come back to the point, what if I really don't -want- to give Apple another several hundred dollars this year? The baby needs shoes, the gas tank is empty, and maybe I just don't -have- $600+ dollars this month to further enrich their shareholders. My iPhone 3GS still works just fine, for the most part, so if I don't really need all of the new whiz bang features of the newer ones, why would I fork over big bucks to replace it? Just because TV commercials entice me to do so?? The problem is, as I have said, this device is now the Apple equivalent of Windows XP. There could be a horrendous collection of a dozen or more known critical security bugs in the thing by now, but as someone noted, the last update Apple issued for the thing was in Feb 2014. In the medical field, they use the term "orphan drugs" to refer to drugs that have such a low return on investment that no manufacturer has any interest in them anymore. We don't use that terminology in the tech field because it would be redundant. *Every* tech product either already is, or soon will be, an orphan. You can't *force* people to throw away or trade-in their old tech products, especially when, from the user's point of view, there doesn't -seem- to be anything wrong with them... like all of those pre- Sept. 2015 Internet video cameras. (Well, -in theory- you could force people to do this. You could legislate an Obamacare-esque tax which penalized everyone who -didn't- throw away or trade-in their old tech gadgets after, say, 4 years, but I don't think that would go down very well.) Regards, rfg From matlockken at gmail.com Thu Oct 27 18:56:02 2016 From: matlockken at gmail.com (Ken Matlock) Date: Thu, 27 Oct 2016 12:56:02 -0600 Subject: Spitballing IoT Security In-Reply-To: <16081.1477592817@segfault.tristatelogic.com> References: <20161027112601.GA17170@ussenterprise.ufp.org> <16081.1477592817@segfault.tristatelogic.com> Message-ID: And I contend that the device manufacturer is only one part in this. Yes, the manufacturers need to get better in securing their devices (that's never been in question). *But* the end users need to have better CPE that can do NetFlow/Sflow/etc in a near real-time fashion. This would allow the end-user to act as a check against the manufacturer(s) and see threats and DDoS packets originating from their gear in real-time (and on the customer's CPE they can get MAC or RFC1918 address to narrow it down better). *But* that doesn't let the SP's off the hook either. The SP needs to be a check against the end users as well, being able to do real-time (or near-real time) flow data export/analysis. Why isn't it done currently? Well, probably a few reasons (and more that I can't even imagine) 1) Cost - It's a real cost to put something like this in place, and upper management does not want to spend money on something with little to no return 2) Availability - How much SP gear even has the option to do any sort of flow export/analysis? 3) Competition - If I am SP 'A' and I allow my customers to participate in a DDoS against SP 'B' (who is a competitor of mine), that at least indirectly harms my competitor, and all I have to do is absolutely nothing, why would management in SP 'A' lift a finger to fix the problem? (Until the DDoS is directed at them). Fixing the current wave of 'IoT' devices and phones and Tv's etc is only putting a bandaid on a broken arm. It gives the illusion of progress, but the fact is the reason DDoS'es are still a problem (and honestly, they've been a problem for decades, IRC servers and Netsplits/channel takeovers/etc), is that each layer in the problem is pointing the finger at the other layers and declaring them the cause of the problem and washing their hands of it (not unlike current politics). Until we accept that it's *everyone's* problem and work to fix the things under our control and work as an advocate for the other layers, we will continue to suffer attacks. Ken > I say again, the only way to solve these problems is if the devices > are fundamentally secure by design, on the day they first ship to > customers. Post-sale patching is an ad hoc and haphazard catch-as- > catch-can solution at best, and it's not something that most manufacturers > have -any- financial incentive to even do. They already got their > money, on the day when the consumer bought the device. The rest is > just an afterthought. > > Regards, > rfg > From bzs at TheWorld.com Thu Oct 27 19:06:13 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Thu, 27 Oct 2016 15:06:13 -0400 Subject: Spitballing IoT Security In-Reply-To: <12573.1477530109@segfault.tristatelogic.com> References: <58111BD4.80403@vaxination.ca> <12573.1477530109@segfault.tristatelogic.com> Message-ID: <22546.20517.359554.656628@gargle.gargle.HOWL> Perhaps something which is needed is analogous to Maritime Law's "Law of Salvage". If a manufacturer abandons all support of a technical product then they lose various intellectual property rights which might prevent a third-party from providing support. Including reasonable assistance such as providing source code needed to support that product which could be provided to the third-party under NDA. But it can't just be refused. Perhaps this can be triggered by the sort of security concerns expressed here. This could be interesting since at least the US govt generally writes minimal terms of support into purchase contracts such as soonest end of life from time of purchase, soonest end of support thereafter, often several years. How that works beyond a vendor's bankruptcy is beyond the scope of this discussion but suffice it to say it's been considered. https://en.wikipedia.org/wiki/Law_of_salvage On October 26, 2016 at 18:01 rfg at tristatelogic.com (Ronald F. Guilmette) wrote: > > In message <58111BD4.80403 at vaxination.ca>, > Jean-Francois Mezei wrote: > > >My smart TV not only hasn't gotten updates in years, but Sharp has > >stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere). > > A little more than 2 years ago, I bought a last-of-its-kind demo > model of a 50 inch Panasonic Plasma TV which was on sale (due to > having been discontinued by the manufacturer) from the local BestBuy. > > Not long after, once I got the thing home, I realized that the > thing's understanding of current local time... important in > conjunction with the on-screen TV guide... was locked to Eastern > Standard Time, and there was no way to change it. (This was/is > a bit of a problem for me, as I'm in PST/PDT.) > > I called up Panasonic and explained the whole thing to a first- > level tech support minion. She had no solution to offer me. > I insisted on speaking to a manager. > > A manager got on the line and I prroceeded to re-explain the whole > issue to him. I said that I needed a firmware fix. He said that > there was no way the company was going to develop a fix "just for > you". Politely, I persisted and said that the TV firmware was > self-evidently faulty. > > > <> > <> -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From bzs at TheWorld.com Thu Oct 27 19:39:07 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Thu, 27 Oct 2016 15:39:07 -0400 Subject: Should abuse mailboxes have quotas? In-Reply-To: <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> References: <20161027164757.GA29809@ussenterprise.ufp.org> <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> Message-ID: <22546.22491.133208.5212@gargle.gargle.HOWL> FWIW abuse at whatever seems to be a favorite in many spammers' lists. I doubt that's their intent, seems like a good way to draw attention to the spam from people with access to blocking lists etc, so I'll guess they just blindly harvest web sites etc and abuse at whatever shows up frequently. That certainly doesn't help with the volume, some of that slips thru spam filters, it adds up. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From morrowc.lists at gmail.com Thu Oct 27 20:17:14 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Thu, 27 Oct 2016 16:17:14 -0400 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 2:35 PM, Dan Hollis wrote: > On Thu, 27 Oct 2016, Christopher Morrow wrote: > >> On Thu, Oct 27, 2016 at 11:03 AM, Stephen Satchell >> wrote: >> >>> I'm tired of blatantly uncaring administrations. >>> >> it's also totally possible that in some cases the mailbox for abuse@ got >> moved behind some orgs other mail systems... This happened numerous times >> at $PREVIOUS_EMPLOYER. When moving around ~200k mailboxes 1 special >> unicorn >> often gets mishandled :( >> >> we wouldn't find out until someone called in all complainy about how 'you >> never care about email... blah...' "Sure we care, but our mail-admin team >> sometimes breaks us, whoops!" >> >> ascribing malice is often unhelpful... Also, of course it's your network >> you can balkanize from the rest of the internet as much as you please. >> > > not so much malice as gross incompetence. > > running spamfilters on your abuse@ mailbox, really? that is, for those > which actually have an abuse mailbox that doesn't bounce outright. > > again, ascribing malice where it doesn't necessarily exist isn't helpful. From A.L.M.Buxey at lboro.ac.uk Thu Oct 27 20:25:52 2016 From: A.L.M.Buxey at lboro.ac.uk (Alan Buxey) Date: Thu, 27 Oct 2016 21:25:52 +0100 Subject: Spitballing IoT Security In-Reply-To: <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> References: <15949.1477591365@segfault.tristatelogic.com> <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> Message-ID: <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> Hi, >At which point the 3GS was almost 5 years old (having originally been >released in June 2009) and had been already superseded by the iPhone 4, >4S, 5 and 5S/5C. But the release of and presence of those phones does not make the older phone suddenly stop working. As noted, the phone might be obsolete to those people hungering for the latest tech but as a phone and web client etc it still works fine. ....and will continue doing so whilst the battery is okay. ... and then, with no updates it can be the next attack vector Which is the point. These things stay out there...like those winXP boxes. There are 2 choices 1) manufacturers are responsible for the devices. No longer caring for them? Recall them. Compensate the users. 2) stronger obsolescence. eg kill switch/firmware tombstoning/network connectivity function ending timebomb as a user of lots of legacy tech i find either option bad :/ alan From nanog at namor.ca Thu Oct 27 20:30:39 2016 From: nanog at namor.ca (J) Date: Thu, 27 Oct 2016 15:30:39 -0500 Subject: Should abuse mailboxes have quotas? In-Reply-To: <22546.22491.133208.5212@gargle.gargle.HOWL> References: <20161027164757.GA29809@ussenterprise.ufp.org> <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> <22546.22491.133208.5212@gargle.gargle.HOWL> Message-ID: <15807d65f42.fc28b465148492.6591941698600326830@namor.ca> I will admit, it's one of the faster ways I pick up on phishing campaigns against our users. So I'm not entirely against it. I'm in the camp of not replying to every report. ---- On Thu, 27 Oct 2016 14:39:07 -0500 <bzs at TheWorld.com> wrote ---- FWIW abuse at whatever seems to be a favorite in many spammers' lists. I doubt that's their intent, seems like a good way to draw attention to the spam from people with access to blocking lists etc, so I'll guess they just blindly harvest web sites etc and abuse at whatever shows up frequently. That certainly doesn't help with the volume, some of that slips thru spam filters, it adds up. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From marka at isc.org Thu Oct 27 20:42:58 2016 From: marka at isc.org (Mark Andrews) Date: Fri, 28 Oct 2016 07:42:58 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Thu, 27 Oct 2016 11:55:38 -0700." <16193.1477594538@segfault.tristatelogic.com> References: <16193.1477594538@segfault.tristatelogic.com> Message-ID: <20161027204258.CD18057D529E@rock.dv.isc.org> In message <16193.1477594538 at segfault.tristatelogic.com>, "Ronald F. Guilmette" writes: > > In message <20161027112940.GB17170 at ussenterprise.ufp.org>, > Leo Bicknell wrote: > > >Actually, they encourage you to trade {your old iPhone} in... > >... > >If your device is too old for that program, they will still take > >it for free and recycle it in an enviornmentally friendly way... > > OK, so good on them. I do compliment them for their apparent willingness > to take back this pile of leachable heavy metals and do something > responsible with it. > > But to come back to the point, what if I really don't -want- to give > Apple another several hundred dollars this year? The baby needs shoes, > the gas tank is empty, and maybe I just don't -have- $600+ dollars this > month to further enrich their shareholders. > > My iPhone 3GS still works just fine, for the most part, so if I don't > really need all of the new whiz bang features of the newer ones, why > would I fork over big bucks to replace it? Just because TV commercials > entice me to do so?? > > The problem is, as I have said, this device is now the Apple equivalent > of Windows XP. There could be a horrendous collection of a dozen or > more known critical security bugs in the thing by now, but as someone > noted, the last update Apple issued for the thing was in Feb 2014. But is there? Can you list a single security bug in iOS 6.1.6 that would require a iOS 6.1.7? Yes, it is annoying that iOS 10.x doesn't run on it so that you can't newer apps. > In the medical field, they use the term "orphan drugs" to refer to drugs > that have such a low return on investment that no manufacturer has any > interest in them anymore. We don't use that terminology in the tech > field because it would be redundant. *Every* tech product either already > is, or soon will be, an orphan. > > You can't *force* people to throw away or trade-in their old tech products, > especially when, from the user's point of view, there doesn't -seem- to be > anything wrong with them... like all of those pre- Sept. 2015 Internet video > cameras. (Well, -in theory- you could force people to do this. You could > legislate an Obamacare-esque tax which penalized everyone who -didn't- > throw away or trade-in their old tech gadgets after, say, 4 years, but I > don't think that would go down very well.) > > > Regards, > rfg -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From marka at isc.org Thu Oct 27 21:00:14 2016 From: marka at isc.org (Mark Andrews) Date: Fri, 28 Oct 2016 08:00:14 +1100 Subject: Spitballing IoT Security In-Reply-To: Your message of "Thu, 27 Oct 2016 21:25:52 +0100." <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> References: <15949.1477591365@segfault.tristatelogic.com> <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> Message-ID: <20161027210014.24E2557D53D3@rock.dv.isc.org> In message <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3 at lboro.ac.uk>, Alan Buxey write s: > Hi, > > > >At which point the 3GS was almost 5 years old (having originally been > >released in June 2009) and had been already superseded by the iPhone 4, > >4S, 5 and 5S/5C. > > But the release of and presence of those phones does not make the older > phone suddenly stop working. As noted, the phone might be obsolete to > those people hungering for the latest tech but as a phone and web client > etc it still works fine. ....and will continue doing so whilst the > battery is okay. ... and then, with no updates it can be the next attack > vector > > Which is the point. These things stay out there...like those winXP > boxes. There are 2 choices > > 1) manufacturers are responsible for the devices. No longer caring for > them? Recall them. Compensate the users. > > 2) stronger obsolescence. eg kill switch/firmware tombstoning/network > connectivity function ending timebomb > > as a user of lots of legacy tech i find either option bad :/ > > alan Or Apple could release iOS 6.1.7. There is nothing stopping Apple doing so. Apple are the ones preventing people running iOS 10.x on the 3GS. This puts the responsibilty on them to supply security fixes. All of the PC's running XP could run a newer version of the Windows regardless of whether they could run the latest version. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From jlewis at lewis.org Thu Oct 27 21:13:31 2016 From: jlewis at lewis.org (Jon Lewis) Date: Thu, 27 Oct 2016 17:13:31 -0400 (EDT) Subject: Spitballing IoT Security In-Reply-To: <16193.1477594538@segfault.tristatelogic.com> References: <16193.1477594538@segfault.tristatelogic.com> Message-ID: On Thu, 27 Oct 2016, Ronald F. Guilmette wrote: > My iPhone 3GS still works just fine, I still have a "functional" iPhone 3G (no S). I don't think AT&T will activate service on it at this point, and it's been relegated to iPod service when I do yard work. > You can't *force* people to throw away or trade-in their old tech products, > especially when, from the user's point of view, there doesn't -seem- to be > anything wrong with them... like all of those pre- Sept. 2015 Internet video > cameras. Sure you can. Just make the tech dependent on "the cloud" and when the device is too old, force retirement by no longer supporting it. That doesn't force it off the network (unless the final command from the cloud is "shut off [your network interface]?"), but it makes the user much more likely to toss it and replace it with something newer if they still want such a device. This is one of my bigger concerns every time I buy something that's "cloud controlled". Not so much that the manufacturer will force it's retirement, but "what happens if they go belly up, or just kill the division that supports my device?" A cloud controlled networked device, with no cloud, is not terribly useful. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From cb.list6 at gmail.com Thu Oct 27 21:25:18 2016 From: cb.list6 at gmail.com (Ca By) Date: Thu, 27 Oct 2016 14:25:18 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027204258.CD18057D529E@rock.dv.isc.org> References: <16193.1477594538@segfault.tristatelogic.com> <20161027204258.CD18057D529E@rock.dv.isc.org> Message-ID: On Thursday, October 27, 2016, Mark Andrews wrote: > > In message <16193.1477594538 at segfault.tristatelogic.com >, > "Ronald F. Guilmette" writes: > > > > In message <20161027112940.GB17170 at ussenterprise.ufp.org > >, > > Leo Bicknell > wrote: > > > > >Actually, they encourage you to trade {your old iPhone} in... > > >... > > >If your device is too old for that program, they will still take > > >it for free and recycle it in an enviornmentally friendly way... > > > > OK, so good on them. I do compliment them for their apparent willingness > > to take back this pile of leachable heavy metals and do something > > responsible with it. > > > > But to come back to the point, what if I really don't -want- to give > > Apple another several hundred dollars this year? The baby needs shoes, > > the gas tank is empty, and maybe I just don't -have- $600+ dollars this > > month to further enrich their shareholders. > > > > My iPhone 3GS still works just fine, for the most part, so if I don't > > really need all of the new whiz bang features of the newer ones, why > > would I fork over big bucks to replace it? Just because TV commercials > > entice me to do so?? > > > > The problem is, as I have said, this device is now the Apple equivalent > > of Windows XP. There could be a horrendous collection of a dozen or > > more known critical security bugs in the thing by now, but as someone > > noted, the last update Apple issued for the thing was in Feb 2014. > > But is there? Can you list a single security bug in iOS 6.1.6 that > would require a iOS 6.1.7? > > Well, ios 7 - 9.3.4 is in scope for this RCE https://blog.lookout.com/blog/2016/08/25/trident-pegasus/ And if you view jpegs, you may want to update to 10.1 https://threatpost.com/apple-patches-ios-flaw-exploitable-by-malicious-jpeg/121521/ Yes, it is annoying that iOS 10.x doesn't run on it so that you can't > newer apps. > > > In the medical field, they use the term "orphan drugs" to refer to drugs > > that have such a low return on investment that no manufacturer has any > > interest in them anymore. We don't use that terminology in the tech > > field because it would be redundant. *Every* tech product either already > > is, or soon will be, an orphan. > > > > You can't *force* people to throw away or trade-in their old tech > products, > > especially when, from the user's point of view, there doesn't -seem- to > be > > anything wrong with them... like all of those pre- Sept. 2015 Internet > video > > cameras. (Well, -in theory- you could force people to do this. You > could > > legislate an Obamacare-esque tax which penalized everyone who -didn't- > > throw away or trade-in their old tech gadgets after, say, 4 years, but I > > don't think that would go down very well.) > > > > > > Regards, > > rfg > -- > Mark Andrews, ISC > 1 Seymour St., Dundas Valley, NSW 2117, Australia > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org > > From jwbensley at gmail.com Thu Oct 27 21:28:21 2016 From: jwbensley at gmail.com (James Bensley) Date: Thu, 27 Oct 2016 22:28:21 +0100 Subject: Large BGP Communities beacon in the wild In-Reply-To: <270091A8-32B1-4519-A578-383C5A97332F@delong.com> References: <20161011150156.GF4457@hanna.meerval.net> <20161027061946.GF37101@Vurt.local> <270091A8-32B1-4519-A578-383C5A97332F@delong.com> Message-ID: On 27 October 2016 at 16:47, Owen DeLong wrote: > I don?t mind the move to 32, but I hope the vendors are getting appropriately smacked for squatting and that those attributes are not allowed to be misappropriated by the vendors. > > We have a standards process for a reason and vendors simply squatting on numbers is a violation of that process which cannot be allowed to stand unless we wish to establish that as precedent and simply allow vendors to claim numbers as they wish. > > This already happened with the BSD community in their implementation of a pseudo-VRRP like capability and now two different vendors have abused BGP path attributes. > > This is not a good path for us to continue. > > Owen Here here! Name and shame, it is not acceptable! James. From emille at abccommunications.com Thu Oct 27 21:30:14 2016 From: emille at abccommunications.com (Emille Blanc) Date: Thu, 27 Oct 2016 14:30:14 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027210014.24E2557D53D3@rock.dv.isc.org> References: <15949.1477591365@segfault.tristatelogic.com> <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> <20161027210014.24E2557D53D3@rock.dv.isc.org> Message-ID: <4FBAFC2ECF5D6244BA4A26C1C94A1E270D57B58EFF@exchange> (deleted for ambiguity) > > Which is the point. These things stay out there...like those winXP > > boxes. There are 2 choices > > > > 1) manufacturers are responsible for the devices. No longer caring for > > them? Recall them. Compensate the users. > > > > 2) stronger obsolescence. eg kill switch/firmware tombstoning/network > > connectivity function ending timebomb > > > > as a user of lots of legacy tech i find either option bad :/ > > > > alan > > Or Apple could release iOS 6.1.7. There is nothing stopping Apple doing > so. Apple are the ones preventing people running iOS 10.x on the 3GS. > This puts the responsibilty on them to supply security fixes. > > All of the PC's running XP could run a newer version of the Windows > regardless of whether they could run the latest version. Well, yes and no. As $newer_better_faster_stronger gains market share, there's no drive to be backwards compatible. iOS is no different from any other operating system in that regard, it's designed for hardware A, B, C, D's 1 through 4 (probably more, but I'm trying to be somewhat abstract). If it has to support E through Z also, for 12+ years of backwards compatibility, bad things can happen (bloat, instability, bugs). I don't get upset for example, when I transplant a Win2k or Win98 drive into a box built up with 3 year old hardware, of which not a single device is supported. That's not even taking into account the challenge of developing for different architectures. ARM, x86, PPC, AMD64, PowerISA, SPARC, to name a few. I won't even get into microcontrollers. Don't get me wrong. I'd love to update my 12 year old Macbook Pro to Sierra, but I've accepted that it, like most electronics, were almost certainly not engineered, let alone expected, to last even half that long. I'm reminded of that fact every time I open Youtube, and Flash Player spins both of its 2.33ghz Core2 Duo cores to 100% for a 460p video. Even then, I've had to stop updating Flash sometime around mid 2014, as any newer versions cease to function entirely. From edward.dore at freethought-internet.co.uk Thu Oct 27 21:32:28 2016 From: edward.dore at freethought-internet.co.uk (Edward Dore) Date: Thu, 27 Oct 2016 22:32:28 +0100 Subject: Spitballing IoT Security In-Reply-To: <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> References: <15949.1477591365@segfault.tristatelogic.com> <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> Message-ID: > On 27 Oct 2016, at 21:25, Alan Buxey wrote: > > Hi, > > >> At which point the 3GS was almost 5 years old (having originally been >> released in June 2009) and had been already superseded by the iPhone 4, >> 4S, 5 and 5S/5C. > > But the release of and presence of those phones does not make the older phone suddenly stop working. As noted, the phone might be obsolete to those people hungering for the latest tech but as a phone and web client etc it still works fine. ....and will continue doing so whilst the battery is okay. ... and then, with no updates it can be the next attack vector No, but at some point everything has to be discontinued. You can't reasonably expect manufacturers to continue to support their products indefinitely, particularly without recompense. To put it another way; are you willing to either pay more up front or some kind of ongoing fee in order to fund the manufacturer continuing to produce software updates for a device which is multiple years and multiple generations out of date? > > Which is the point. These things stay out there...like those winXP boxes. There are 2 choices > > 1) manufacturers are responsible for the devices. No longer caring for them? Recall them. Compensate the users. > > 2) stronger obsolescence. eg kill switch/firmware tombstoning/network connectivity function ending timebomb > > as a user of lots of legacy tech i find either option bad :/ > > alan Windows XP was released in October 2001 and finally killed in April 2014. Even the last service pack was released in April 2008. That's a pretty long life and I don't think it would be reasonable to expect Microsoft to continue to spend time and money supporting it any further. Users need to take some responsibility when it comes to making sure that their software (or firmware in the case of embedded devices) is still supported by the manufacturer. If you choose to use it past the end of the manufacturer's support, then you need to be prepared for the potential consequences of doing so, including that your service provider disconnects you from their network as your device(s) are participating in DoS attacks. Of course, the manufacturer needs to provide the user with some kind of reasonable expectation of the lifetime of a device so that they can make the appropriate plans to invest in a suitable replacement. In the case of Windows XP there has been a published official lifecycle for an extremely long time (since SP3 was released?). There was also a lot of press coverage before and after the end of support, so it shouldn't exactly come as a surprise to anyone. For the iPhone, new versions of iOS generally support the last 4-5 iterations of the hardware (I'm not sure if there is an official published policy about this), which is typically updated annually. Currently that is the iPhone 5/5C from September 2012, the iPhone 5S from September 2013, the iPhone 6/6+ from September 2014, the iPhone 6S/6S+/SE from September 2015 and the iPhone 7/7+ from September 2016. Edward -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From emille at abccommunications.com Thu Oct 27 21:39:15 2016 From: emille at abccommunications.com (Emille Blanc) Date: Thu, 27 Oct 2016 14:39:15 -0700 Subject: Spitballing IoT Security In-Reply-To: References: <16193.1477594538@segfault.tristatelogic.com> Message-ID: <4FBAFC2ECF5D6244BA4A26C1C94A1E270D57B58F07@exchange> >On Thu, 27 Oct 2016, Ronald F. Guilmette wrote: > >> My iPhone 3GS still works just fine, > >I still have a "functional" iPhone 3G (no S). I don't think AT&T will >activate service on it at this point, and it's been relegated to iPod >service when I do yard work. > >> You can't *force* people to throw away or trade-in their old tech products, >> especially when, from the user's point of view, there doesn't -seem- to be >> anything wrong with them... like all of those pre- Sept. 2015 Internet video >> cameras. > >Sure you can. Just make the tech dependent on "the cloud" and when the >device is too old, force retirement by no longer supporting it. That >doesn't force it off the network (unless the final command from the cloud >is "shut off [your network interface]?"), but it makes the user much more >likely to toss it and replace it with something newer if they still want >such a device. Or shut down the network that the phone(s) support. Anyone remember the analogue cell network shutdown? Or am I already that old? http://www.pcworld.com/article/142119/article.html Granted there were other problems this presented. Decreased coverage in areas for example is my favourite, as it opened the doors for such revolutionary pay-as-you-go-licensing features for base stations such as range-by-the-kilometre. But I think with this, I'm contributing to driving this thread off the topic of IoT security, and will now dive back into staring at some netflow data. From list at satchell.net Thu Oct 27 22:20:20 2016 From: list at satchell.net (Stephen Satchell) Date: Thu, 27 Oct 2016 15:20:20 -0700 Subject: Should abuse mailboxes have quotas? In-Reply-To: <15807d65f42.fc28b465148492.6591941698600326830@namor.ca> References: <20161027164757.GA29809@ussenterprise.ufp.org> <6A47CD3B-612F-474A-B810-70ADF0DFCB0D@blighty.com> <22546.22491.133208.5212@gargle.gargle.HOWL> <15807d65f42.fc28b465148492.6591941698600326830@namor.ca> Message-ID: <42825030-d93d-87cf-4cf6-c954423ba20b@satchell.net> On 10/27/2016 01:30 PM, J wrote: > I'm in the camp of not replying to every report. I was in that camp, too, when I was mail admin for a web host company. I wanted to spend my time fixing the flood, without having to take the time to reply. I figure the best reply is when the spamming stops. I have about 30 mail endpoints of my own that I monitor, so it becomes obvious real quick when someone takes action. :) From list at satchell.net Thu Oct 27 22:42:39 2016 From: list at satchell.net (Stephen Satchell) Date: Thu, 27 Oct 2016 15:42:39 -0700 Subject: Spitballing IoT Security -- Dancing around a solution In-Reply-To: References: <15949.1477591365@segfault.tristatelogic.com> <31DCBD1F-58B3-47E6-980A-93000264C4B8@freethought-internet.co.uk> <56B9ABD3-6911-42CB-9C9D-81FB33CA55C3@lboro.ac.uk> Message-ID: <6f1072d7-d165-a041-4a07-0e2b8ebca674@satchell.net> I've been following the discussion with quite a bit of interest. What had become crystal clear to me is that nobody here has been looking at the problem from the perspective of the manufacturer, particularly how they actually get product to marked. A la "Dilbert". The engineer's credo: "Why build it when you can buy it?" I doubt very seriously that manufacturers are starting completely from scratch when they design their IoT product. They buy this piece, they buy that piece, they buy this hunk of software, they buy these hardware components. Slap them together, and you have your product. That being the case, the question of "what happens when the company goes bankrupt" becomes less of an issue so long at the company who supplied the IP stack is still around. By government implementing some not-unpleasant rules, the companies can outsource the IP stack portion, including updates. Then the manufacturers can concentrate on the value add stuff. For durable goods like refrigerators and thermostats, you could require that the IP-capable part be in a plug-replaceable module, so that all the customer needs to do is go to Home Depot or wherever and get a replacement module. Instant update! The back end of the module would be a well-defined API so the manufacturer can do his product, divorced from the Net stuff. Indeed, it wouldn't take long for the various industry associations to codify what the modules should look like, both physically and electrically. The semiconductor industry did this big time in the TTL days. There is precedent. So what if your washing machine is working perfectly well 15 years into its lifecycle. You replace the network module and get the latest and greatest security updates. Light bulbs are harder, but even then there is an opportunity for someone to market an "industry standard" interface that can be upgraded easily and cheaply. By the original software vendor. Can someone say "IoTsoft"? From rfg at tristatelogic.com Thu Oct 27 23:24:07 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 16:24:07 -0700 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <17127.1477610647@segfault.tristatelogic.com> In message Ken Matlock wrote: >Fixing the current wave of 'IoT' devices and phones and Tv's etc is only >putting a bandaid on a broken arm. It gives the illusion of progress... >Until we accept that it's *everyone's* problem and work to fix the things >under our control and work as an advocate for the other layers, we will >continue to suffer attacks. Agreed. Even if we could snap our fingers and fix the whole morass that is the IoT problem tomorrow, that still wouldn't prevent dumb consumers from pulling their dusty old Windows XP laptops own out of their closets and hooking them up directly to the Internet. Nor would it do anything about the small ISPs that have "mailbox full" abuse@ addresses, or the even larger ISPs that allow deliberately spoofed packets out onto the public Internet, or the Tier 1s that still peer with known utterly irresponsible ASNs. But, ya know, you gotta start someplace. And we can't let the perfect be the enemy of the good. That just won't wash anymore, I think. Not after last Friday. I put forward what I think is a reasonbly modest scheme to try to get IoT things to place hard limits on their "unsolicited" packet output at the kernel level, and I'm going to go off now and try to find and then engage some Linux embedded kernel people and see what they think. Maybe the whole thing is a dumb idea and not worth persuing, but I'm not con- vinced of that yet. So I'll go off, investigate in some more appropriate forum, and report back here if/when I have anything useful to say. Hacking embedded kernels to make them fault-tolerant, even in the event of attackers getting a root shell prompt, isn't going to save the world from DDoS attacks, but it may be one small part of the solution. Regards, rfg P.S. In the scheme I proposed, I left out one additional nicety that embedded kernels could also do to enhance security, namely disabling raw sockets completely in the kernel. No normal IoT thing needs the ability to forge outbound packets. But I would be willing to bet my bottom dollar, right now, that if we poked around long enough we could surely find some easily break-in-able busybox-based thingies out there, right now, as we speak, into which a binary could dropped that would have no trouble at all opening raw outbound sockets. BCP38 for toasters anyone? From kmedcalf at dessus.com Thu Oct 27 23:55:19 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Thu, 27 Oct 2016 17:55:19 -0600 Subject: Spitballing IoT Security In-Reply-To: Message-ID: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> > > The problem is in allowing inbound connections and going as far as doing > > UPnP to tell the CPE router to open a inbound door to let hackers loging > > to that IoT pet feeder to turn it into an agressive DNS destroyer. > Well yes. uPnP is a problem precisely because it is some random device > asserting on its own that it can be trusted to do what it wants. Had > that assertion come from the manufacturer, at least you would know that > the device was designed to require that sort of access.** And why would anyone in their right mind trust the manufacturer to make this decision? Neither the device nor the manufacturer have the authority to make that decision ... ONLY the owner of the device has that authority, and once made the owner of the device is responsible for *all* consequences resulting from that decision. If the device itself makes the decision (because it is programmed to do so by the manufacturer), then the manufacturer is responsible for all the consequences resulting therefrom. End Of Line. From rfg at tristatelogic.com Fri Oct 28 00:17:09 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Thu, 27 Oct 2016 17:17:09 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161027204258.CD18057D529E@rock.dv.isc.org> Message-ID: <17304.1477613829@segfault.tristatelogic.com> In message <20161027204258.CD18057D529E at rock.dv.isc.org>, Mark Andrews wrote: >> The problem is, as I have said, this device is now the Apple equivalent >> of Windows XP. There could be a horrendous collection of a dozen or >> more known critical security bugs in the thing by now, but as someone >> noted, the last update Apple issued for the thing was in Feb 2014. > >But is there? Can you list a single security bug in iOS 6.1.6 that >would require a iOS 6.1.7? An entirely reasonable and logical question, Mark. I'll admit, it took me a bit of digging, but the answer would appear to be "yes": https://threatpost.com/apple-fixes-cookie-access-vulnerability-in-safari-on-billions-of-devices/112246/ Note that I have the latest and greatest IOS 6.1.6 on my 3GS. The Safari HTTP User-Agent string is apparently as follows: Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_6 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B500 Safari/8536.25 So, Q.E.D. ? Regards, rfg From mysidia at gmail.com Fri Oct 28 00:36:04 2016 From: mysidia at gmail.com (Jimmy Hess) Date: Thu, 27 Oct 2016 19:36:04 -0500 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 1:35 PM, Dan Hollis wrote: > not so much malice as gross incompetence. > running spamfilters on your abuse@ mailbox, really? that is, for those which > actually have an abuse mailbox that doesn't bounce outright. Sorry about that, many networks do perform standard filtering on messages to Abuse contacts based on DNS RBLs, SPF/DMARC policy enforcement, virus scans, etc, and do send a SMTP Reject on detected spam or malware. If your own mail server's IP appears on Spamhaus, then, Yes, you should expect any abuse reports you attempt to submit have a likelihood of being rejected as spam. Abuse/contact mailboxes are not special in this regard, and it would not be a good practice to leave unprotected. If anything.....: For many networks; files sent to abuse mailboxes are likely aliased to the normal mailbox of sysadmins who have access to high privileges. As such, these mailboxes may require even stronger protection than other accounts, because of increased risk (when a mistake is made). There is a reason that phone numbers, and not just e-mail addresses are listed in the WHOIS records...... If you get a SMTP reject, then call the the Abuse POC of the organization you need to report abuse from..... > -Dan -- -JH From larrysheldon at cox.net Fri Oct 28 00:43:52 2016 From: larrysheldon at cox.net (Larry Sheldon) Date: Thu, 27 Oct 2016 19:43:52 -0500 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <0hcw1u00h1cZc5601hd35k> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> Message-ID: <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: > :-) > http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 OH BOY! Omaha Taxpayers get to replace all the BGSs for their party venue boondoggle. Again. https://www.google.com/maps/place/CenturyLink+Center+Omaha/@41.2623782,-95.9281322,19z/data=!4m5!3m4!1s0x0:0xe896a8b5037ce4d0!8m2!3d41.2624226!4d-95.9282445 -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account. From jhellenthal at dataix.net Fri Oct 28 00:46:45 2016 From: jhellenthal at dataix.net (Jason Hellenthal) Date: Fri, 28 Oct 2016 00:46:45 +0000 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> Message-ID: <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> lol > On Oct 28, 2016, at 00:43, Larry Sheldon wrote: > > > > On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >> :-) >> http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 > > OH BOY! Omaha Taxpayers get to replace all the BGSs for their party venue boondoggle. Again. > > > https://www.google.com/maps/place/CenturyLink+Center+Omaha/@41.2623782,-95.9281322,19z/data=!4m5!3m4!1s0x0:0xe896a8b5037ce4d0!8m2!3d41.2624226!4d-95.9282445 > > -- > "Everybody is a genius. But if you judge a fish by > its ability to climb a tree, it will live its whole > life believing that it is stupid." > > --Albert Einstein > > From Larry's Cox account. -- Jason Hellenthal JJH48-ARIN From laszlo at heliacal.net Fri Oct 28 00:48:09 2016 From: laszlo at heliacal.net (Laszlo Hanyecz) Date: Fri, 28 Oct 2016 00:48:09 +0000 Subject: Spitballing IoT Security In-Reply-To: <17127.1477610647@segfault.tristatelogic.com> References: <17127.1477610647@segfault.tristatelogic.com> Message-ID: <97a7f937-c7b8-5fd5-81a0-100c163f551f@heliacal.net> On 2016-10-27 23:24, Ronald F. Guilmette wrote: > I put forward what I think is a reasonbly modest scheme to try to get > IoT things to place hard limits on their "unsolicited" packet output at > the kernel level, and I'm going to go off now and try to find and then > engage some Linux embedded kernel people and see what they think. Maybe > the whole thing is a dumb idea and not worth persuing, but I'm not con- > vinced of that yet. So I'll go off, investigate in some more appropriate > forum, and report back here if/when I have anything useful to say. > > Hacking embedded kernels to make them fault-tolerant, even in the event > of attackers getting a root shell prompt, isn't going to save the world > from DDoS attacks, but it may be one small part of the solution. > > > Regards, > rfg This doesn't make sense to me. When the device is compromised, the default software with the restrictions will just be reconfigured or replaced. This process is similar to installing DD-WRT, or even a simple update from the vendor, for example. Botnets download and install the software they require and often they close the original infection vector to prevent another botnet from reinfecting. Check out the Mirai source code that was posted. -Laszlo From list at satchell.net Fri Oct 28 00:48:47 2016 From: list at satchell.net (Stephen Satchell) Date: Thu, 27 Oct 2016 17:48:47 -0700 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On 10/27/2016 05:36 PM, Jimmy Hess wrote: > If you get a SMTP reject, then call the the Abuse POC of the organization you > need to report abuse from..... Not when the mailbox-full bounce is from a network in China, or India, or Pakistan, or Russia. Or a couple of other countries that seem to be spam havens. What is my ROI on the cost of the phone call. MUCH cheaper to put in netrange blocks, and less time, not to mention the language barriers. I can't run a telephone call through Google Translate... It's NOT MY JOB to accept packets from people who can't seem to follow the accepted rules of the Internet. I pine for the old NSF days, when a sysadmin had to be a BOFH to keep the local network connected to the 'Net at large. I don't want to suppress innovation as much as the next guy, but not at the expense of having to deal with childish morons and greedy bastards who think they can do what they want. Because no one disconnects them, they are sent the wrong message about what's permissible. (I'm grumpy because a couple of days ago I got tangled up in some medical gear, and ended up slamming my knee squarely into a marble floor. It hurts, it does.) From nanog at namor.ca Fri Oct 28 01:12:15 2016 From: nanog at namor.ca (J) Date: Thu, 27 Oct 2016 20:12:15 -0500 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: <15808d830f6.12a141683153750.8450151376399757667@namor.ca> Sorry about that, many networks do perform standard filtering on messages to Abuse contacts based on DNS RBLs, SPF/DMARC policy enforcement, virus scans, etc, and do send a SMTP Reject on detected spam or malware. I'll disagree, here. Sure, there are some basic considerations - but some of the major types of role accounts should have specific exceptions to allow the issues to actually reach the appropriate party. From bzs at TheWorld.com Fri Oct 28 03:59:10 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Thu, 27 Oct 2016 23:59:10 -0400 Subject: Spitballing IoT Security In-Reply-To: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> Message-ID: <22546.52494.516440.706525@gargle.gargle.HOWL> I suppose someone could modify this Mirai virus to instead inject antivirus software. I know, illegal. What would the manufacturers' response be if this virus had instead just shut down, possibly in some cases physically damaged the devices or otherwise caused them to cease functioning ever again (wiped all their software or broke their bootability), rather than just hijacked them for a while? One manufacturer agreed that half a million of their devices were involved in the attacks in a recent press release. And they apologize. What if half a million of their devices just suddenly stopped working, forever? I suspect that would require a somewhat more expensive response. They have to be thinking that sort of thing or else they're being pretty dumb. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From lear at ofcourseimright.com Fri Oct 28 04:09:17 2016 From: lear at ofcourseimright.com (Eliot Lear) Date: Fri, 28 Oct 2016 06:09:17 +0200 Subject: Spitballing IoT Security In-Reply-To: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> Message-ID: <68ea3900-1d2f-507c-1192-6c43fe1a5f1f@ofcourseimright.com> Hi Keith, On 10/28/16 1:55 AM, Keith Medcalf wrote: >>> The problem is in allowing inbound connections and going as far as doing >>> UPnP to tell the CPE router to open a inbound door to let hackers loging >>> to that IoT pet feeder to turn it into an agressive DNS destroyer. >> Well yes. uPnP is a problem precisely because it is some random device >> asserting on its own that it can be trusted to do what it wants. Had >> that assertion come from the manufacturer, at least you would know that >> the device was designed to require that sort of access.** > And why would anyone in their right mind trust the manufacturer to make this decision? Because the manufacturer designed the device and knows best as to what sort of access it will require. Consider that today most devices have unfettered outbound access, and many can arrange for unfettered inbound access. That's Not Good?. That doesn't mean that network administrators shouldn't be the kings and queens of their castles, but as I'm sure you well know, home users don't really know how to rule, and so they need some good defaults. Put it another way: you bring home a NEST and the first thing you the expert might do is read the net to figure out which ports to open. Are you really going to not open those ports? Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From goemon at sasami.anime.net Fri Oct 28 04:39:02 2016 From: goemon at sasami.anime.net (Dan Hollis) Date: Thu, 27 Oct 2016 21:39:02 -0700 (PDT) Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: On Thu, 27 Oct 2016, Jimmy Hess wrote: > On Thu, Oct 27, 2016 at 1:35 PM, Dan Hollis wrote: >> not so much malice as gross incompetence. >> running spamfilters on your abuse@ mailbox, really? that is, for those which >> actually have an abuse mailbox that doesn't bounce outright. > Sorry about that, many networks do perform standard filtering on > messages to Abuse contacts based on DNS RBLs, SPF/DMARC > policy enforcement, virus scans, etc, and do send a SMTP Reject on > detected spam or malware. This is a good way to get your block listed on RBLs. > For many networks; files sent to abuse mailboxes are likely aliased to the > normal mailbox of sysadmins who have access to high privileges. As such, > these mailboxes may require even stronger protection than other accounts, > because of increased risk (when a mistake is made). If anyone actually does this, it is incompetence beyond comprehension. > There is a reason that phone numbers, and not just e-mail addresses are listed > in the WHOIS records...... > > If you get a SMTP reject, then call the the Abuse POC of the organization you > need to report abuse from..... Again, good way to end up on RBLs. I encourage competitors to heavily filter their POCs. Oh yes, and also be sure your phone numbers are out of date. -Dan From Curtis at greenkey.net Thu Oct 27 21:38:30 2016 From: Curtis at greenkey.net (Curtis Doty) Date: Thu, 27 Oct 2016 14:38:30 -0700 Subject: How to find all of an ISP's ASNs In-Reply-To: References: <1e97a660-2c9d-15b3-9eda-e364c3fb7d3c@baribault.net> Message-ID: On Tue, Oct 25, 2016 at 9:03 PM, Hank Nussbacher wrote: > and if that doesn't work try: > http://bgp.he.net/AS3356#_graph4 > [replace the ASN with the ASN of your choice to see the interconnections.] > ?Doesn't always work--as it will only show upstream ASNs. For example, Comcast's backbone AS interconnects their regional ASNs. However, the regionals don't show up on http://bgp.he.net/AS7922#_graph4 so you'd need to find all of them first...with something like http://bgp.he.net/search?search[search]=Comcast and/or consult your favorite route server. Also Gary, keep in mind these aren't static. I.e. new AS are added/removed over time. And inferred policy (i.e. hub/spoke) could change too. But I'm still curious...how to you propose to filter by AS? And what if your neighbor (inside one of those permitted AS) is compromised? You've just re-exposed your IoT devices' soft white underbelly again. :-( ../C ? From morrowc.lists at gmail.com Fri Oct 28 06:00:15 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Fri, 28 Oct 2016 02:00:15 -0400 Subject: Large BGP Communities beacon in the wild In-Reply-To: References: <20161011150156.GF4457@hanna.meerval.net> <20161027061946.GF37101@Vurt.local> <270091A8-32B1-4519-A578-383C5A97332F@delong.com> Message-ID: On Thu, Oct 27, 2016 at 5:28 PM, James Bensley wrote: > > > Name and shame, it is not acceptable! > > read the IDR thread(1), the vendors in question actually self reported. I don't think 'shame' here is quite appropriate, but certainly owen's note about: "Hey, pls don't do this again" with the added: ""this is not a good path to continue" were noted by several folks on the IDR list. -chris 1: https://www.ietf.org/mail-archive/web/idr/current/msg16556.html From randy at psg.com Fri Oct 28 08:26:41 2016 From: randy at psg.com (Randy Bush) Date: Fri, 28 Oct 2016 17:26:41 +0900 Subject: Large BGP Communities beacon in the wild In-Reply-To: References: <20161011150156.GF4457@hanna.meerval.net> <20161027061946.GF37101@Vurt.local> <270091A8-32B1-4519-A578-383C5A97332F@delong.com> Message-ID: > read the IDR thread(1), the vendors in question actually self reported. > I don't think 'shame' here is quite appropriate, but certainly owen's note > about: "Hey, pls don't do this again" with the added: ""this is not a good > path to continue" were noted by several folks on the IDR list. luckily we have never seen something like this before. From thomas.mangin at exa-networks.co.uk Fri Oct 28 08:27:05 2016 From: thomas.mangin at exa-networks.co.uk (Exa) Date: Fri, 28 Oct 2016 09:27:05 +0100 Subject: [routing-wg] Large BGP Communities beacon in the wild In-Reply-To: <270091A8-32B1-4519-A578-383C5A97332F@delong.com> References: <20161011150156.GF4457@hanna.meerval.net> <20161027061946.GF37101@Vurt.local> <270091A8-32B1-4519-A578-383C5A97332F@delong.com> Message-ID: <94EA3C4A-616C-4A55-B253-5B9885DA8CB4@exa-networks.co.uk> Hello Owen, While I agree ( and cudos to Job for noticing the issue while the document is still at the draft stage), the current process for allocation is not developer friendly. For ExaBGP, I also had to squat a code point to implement draft-frs-bgp-operational-message. I doubt it will eve cause an operational issue, ExaBGP is not used to route packets in anyone's core, but but the current process gave me no choice: it takes implementation to find issues with drafts and/or interrop problems, unexpected behaviours or simply provide a feature to a crying customer even if a draft is never created / never reach RFC status. I remember reading a draft from John which attempted to allocate some code points for experimentation - my memory is fuzzy on the details sorry - but even then this would require re numbering which is not ideal. So perhaps in addition to recognising the squatting issue, "we" should look at how the current IETF process works and can be changed to improve experimentation. Thomas Sent from my iPad > On 27 Oct 2016, at 16:47, Owen DeLong wrote: > > I don?t mind the move to 32, but I hope the vendors are getting appropriately smacked for squatting and that those attributes are not allowed to be misappropriated by the vendors. > > We have a standards process for a reason and vendors simply squatting on numbers is a violation of that process which cannot be allowed to stand unless we wish to establish that as precedent and simply allow vendors to claim numbers as they wish. > > This already happened with the BSD community in their implementation of a pseudo-VRRP like capability and now two different vendors have abused BGP path attributes. > > This is not a good path for us to continue. > > Owen > >> On Oct 26, 2016, at 11:19 PM, Job Snijders wrote: >> >> Dear Internet, >> >> Through this beacon it was discovered that a vendor was squatting on BGP >> Path Attribute value 30. And another vendor sat on 31. >> >> So, a twisted turn of events, the Large BGP Communities effort has ended up >> with BGP Path Attribute value 32 - very befitting if you look at the very >> problem we're trying to solve :-) >> >> The beacon has been updated to use the new IANA assigned value, nothing >> else was changed. Hopefully we are in the clear this time around! >> >> Please verify if you can see 192.147.168.0/24 and 2001:67c:208c::/48 >> >> Kind regards, >> >> Job >> >>> On Tue, Oct 11, 2016 at 05:01:56PM +0200, Job Snijders wrote: >>> Large BGP Communities are a novel way to signal information between >>> networks. An example of a Large BGP Communities is: 2914:4056024901:80. >>> >>> Large BGP Communities are composed of three 4-octet integers, separated >>> by something like a colon. This is easy to remember and accommodates >>> advanced routing policies in relation to 4-Byte ASNs. It is the tool that has >>> been missing since 4-octet ASNs were introduced. >>> >>> IANA has made an Early Allocation of the value 30 (LARGE_COMMUNITY) in >>> the "BGP Path Attributes" registry under the "Border Gateway Protocol >>> (BGP) Parameters" group. >>> >>> The draft can be read here: https://tools.ietf.org/html/draft-ietf-idr-large-community >>> >>> Additional information about Large BGP Communities can be found here: >>> http://largebgpcommunities.net/ >>> >>> Starting today (2016.10.11), the following two BGP beacons are available >>> to the general public, with AS_PATH 2914_15562$ >>> >>> Both these prefixes have a Large BGP Community attached: >>> >>> 2001:67c:208c::/48 >>> 192.147.168.0/24 >>> >>> Large BGP Community - 15562:1:1 >>> >>> The NLNOG RING BGP Looking Glass is running the latest version of BIRD >>> which understands the Large BGP Community Path Attribute. >>> >>> IPv4 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=192.147.168.0/24 >>> IPv6 LG: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2001:67c:208c::/48 >>> >>> In theory, since this is an optional transitive BGP Path Attribute, all >>> the Looking Glass' peers should boomerang the Large Community back to >>> the LG. However we currently observe that 50 out of 75 peers propagate >>> the Large BGP Community to the LG. >>> >>> Relevant Router commands to see if you receive the attribute, or whether >>> one of intermediate networks has stripped the attribute from the route: >>> >>> IOS: show ip bgp path-attribute unknown >>> shows all prefixes with unknown path attributes. >>> >>> IOS #2 - like on route views: >>> route-views>sh ip bgp 192.147.168.0 >>> BGP routing table entry for 192.147.168.0/24, version 98399100 >>> Paths: (39 available, best #30, table default) >>> Not advertised to any peer >>> Refresh Epoch 1 >>> 701 2914 15562 >>> 137.39.3.55 from 137.39.3.55 (137.39.3.55) >>> Origin IGP, localpref 100, valid, external >>> unknown transitive attribute: flag 0xE0 type 0x1E length 0xC >>> value 0000 3CCA 0000 0001 0000 0001 >>> rx pathid: 0, tx pathid: 0 >>> >>> IOS-XR: (you must look at specific prefixes) >>> RP/0/RSP0/CPU0:Router#show bgp ipv6 unicast 2001:67c:208c::/48 unknown-attributes >>> BGP routing table entry for 2001:67c:208c::/48 >>> Community: 2914:370 2914:1206 2914:2203 2914:3200 >>> Unknown attributes have size 15 >>> Raw value: >>> e0 1e 0c 00 00 3c ca 00 00 00 01 00 00 00 01 >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> >>> JunOS: >>> user at JunOS-re6> show route 2001:67c:208c::/48 detail >>> 2001:67c:208c::/48 (1 entry, 1 announced) >>> AS path: 15562 I >>> Unrecognized Attributes: 15 bytes >>> Attr flags e0 code 1e: 00 00 3c ca 00 00 00 01 00 00 00 01 >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> >>> A note about router Configurations: >>> >>> Ensure you are not fitlering the path attributes, eg: >>> >>> JunOS: >>> [edit protocols bgp] >>> user at junos# delete drop-path-attributes 30 >>> >>> XR: >>> configure >>> router bgp YourASN >>> attribute-filter group ReallyBadIdea ! avoid creating bogons >>> no attribute 30 >>> ! >>> ! >>> >>> Contact persons: myself or Jared Mauch or NTT NOC. BGP Session >>> identifier 83.231.213.230 / 2001:728:0:5000::a92 AS 15562. >>> >>> Kind regards, >>> >>> Job > > From kmedcalf at dessus.com Fri Oct 28 13:13:26 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Fri, 28 Oct 2016 07:13:26 -0600 Subject: Spitballing IoT Security In-Reply-To: <68ea3900-1d2f-507c-1192-6c43fe1a5f1f@ofcourseimright.com> Message-ID: <10fef088ad619b44af30161f17715333@mail.dessus.com> On Thursday, 27 October, 2016 22:09, Eliot Lear said: > On 10/28/16 1:55 AM, Keith Medcalf wrote: > >>> The problem is in allowing inbound connections and going as far as > doing > >>> UPnP to tell the CPE router to open a inbound door to let hackers > loging > >>> to that IoT pet feeder to turn it into an agressive DNS destroyer. > >> Well yes. uPnP is a problem precisely because it is some random device > >> asserting on its own that it can be trusted to do what it wants. Had > >> that assertion come from the manufacturer, at least you would know that > >> the device was designed to require that sort of access.** > > And why would anyone in their right mind trust the manufacturer to make > > this decision? > Because the manufacturer designed the device and knows best as to what > sort of access it will require. Manufacturers of devices and Operating Systems (particularly Microsoft WIndows) have proven over and over and over again that they cannot be trusted to make that decision. One of the worst offenders, any versions of Windows subsequent to Windows XP, insists in dropping its knickers (opening the firewall) so that anything that wants to can fuck about with (connect to unrestricted from the internet) all the myriad of ever growing piles of shit included by Microsoft. Even if you close the firewall, the Manufacturer believes it knows better and changes your settings, without your permission. If you are stupid enough to run UPNP on your network, then all the drivel flarn filth is directly accessible from the internet (and beyond) without restriction. Preventing the manufacturer from doing that takes a *LOT* of *DEEP* surgery. I wish that Ballmer fellow would just up and die, and that damn indian too, even more so. If they got some help along those lines the world would be a lot better place. They are both total asshats and enemies of security and functionality everywhere. However, it is not just a microsoft thing -- ALL of them think they know better and they should all fuck off and die. > Consider that today most devices have > unfettered outbound access, and many can arrange for unfettered inbound > access. That's Not Good?. Yes, because that is what the device manufacturers have programmed the device to do and to have, and to go to inordinate lengths to ignore any directions from the OWNER to the contrary. They should all be strung up by their balls and dropped with dull rusty pinking shears! > That doesn't mean that network > administrators shouldn't be the kings and queens of their castles, but > as I'm sure you well know, home users don't really know how to rule, and > so they need some good defaults. What is wrong with OFF? That is a good default. > Put it another way: you bring home a NEST and the first thing you the > expert might do is read the net to figure out which ports to open. Are > you really going to not open those ports? First of all, I would NEVER bring home a NEST, nor would I ever allow a NEST or anything like it to be connected to my network. It is an evil device that does nothing of any use to me whatsoever. It is also dangerous and malicious and will not permit me to control the damn thing, nor to retrieve data from it. It is a hunk of useless shit. And no. Under no circumstances whatsoever do I open ports unless I know what they are for. And inbound port openings require proof of paid up indemnity insurance in the millions per incident (trillion in total). Therefore, no inbound ports get opened since no one has ever been able to satisfy this requirement. End of Line. From rsk at gsp.org Fri Oct 28 13:58:36 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Fri, 28 Oct 2016 09:58:36 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <16193.1477594538@segfault.tristatelogic.com> Message-ID: <20161028135836.GA13977@gsp.org> On Thu, Oct 27, 2016 at 05:13:31PM -0400, Jon Lewis wrote: > This is one of my bigger concerns every time I buy something that's "cloud > controlled". Not so much that the manufacturer will force it's retirement, > but "what happens if they go belly up, or just kill the division that > supports my device?" A cloud controlled networked device, with no cloud, is > not terribly useful. 1. This has happened already, e.g., Nest. It will happen again. Many times. 2. Such a device may not be terribly useful *to you*, but in its neglected, unremarked, unmaintained state it may become very useful to attackers. ---rsk From rsk at gsp.org Fri Oct 28 15:06:08 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Fri, 28 Oct 2016 11:06:08 -0400 Subject: Should abuse mailboxes have quotas? In-Reply-To: References: Message-ID: <20161028150608.GA22551@gsp.org> No. They should not. (Nor should they have spam or malware filters, since of course that's one of the things that people will forward as part of their complaints. Anyone using a sensible email client on a sensible platform will of course incur zero risk by handling either of those.) That said, and since abuse mailboxes have come up in the context of the ongoing IoT/DDoS discussion, let me point out that a fair amount of traffic on this list, on mailop, on dns-operations, on outages-discussion, on other lists, consists of queries of the form "how can I contact X about Y?" The traffic exists because X either has never read RFC 2142 or has just ignored it. A *lot* of our collective time has been wasted asking/answering these queries, and no doubt they represent the tip of the iceberg, as many folks don't bother (having found those addresses non-existent) or don't know where to ask. So now: A Rant (albeit a considered one, after two (2) cups of coffee): This is unacceptable. If you don't maintain the basic role addresses and pay attention to what shows up there, you're not a professional. You're not even a competent amateur. Of all the things we have to do, from unsnarling switches to diagnosing psychotic web/mail servers to dealing with WTF-grade announcements, maintaining role addresses is one of the easiest. It's also one of the best things to do, because traffic arriving there is quite often trying to tell you about problems that you have that you really, REALLY ought to be curious about. And in a time when we're all facing myriad threats, and relying on each other to communicate about them and address them in as close to real time as we can manage, it's inexcusable not to have at least the basics in place. (abuse@, hostmaster@, postmaster@, webmaster@, etc. as applicable to the services you provide) Other people are often doing your job for you *for free* and are sharing the results with you: you should be listening. Intently. I've heard all the whining excuses...and I dismiss them: "We get too much traffic @abuse" -- gosh, stop emitting/facilitating so much abuse and so many attacks, it'll decrease. "We can't reply to everything" -- see previous point. And learn how to use a real mail client. "We get too much spam" -- use procmail to bin incoming reports and triage the most likely non-spam ones first. [1] "People send us malware" -- to a very good first approximation, there are no such things as "email viruses". There are "Outlook viruses". Learn how to use a real mail client. "We don't speak language X" -- run it through Google translate, take your best shot at it. Most reports will be in your language anyway. (Note: if you are a multi-mega-million dollar company, then hire abuse and postmaster and hostmaster &etc. staff fluent in multiple languages. This is more important than your on-site massage therapist and gourmet chef.) "We don't have the time/personnel/budget" -- but magically you have the time, personnel, and budget to run an operation that's causing problems for other people. Also you have a market capitalization of $7.65 gazillion dollars and a gym on the second floor, so please spare me this one. "But X isn't doing it either" -- the "we're no worse than anyone else" excuse and subsequent race to the bottom. "You can call us on the phone" -- yeah, at 3 AM your local time, that'll work. Also I'll be dictating the contents of an email message, including the full headers. No, I don't mind trying to explain a hijacked network problem to your front-line support staff who will read their from their script and tell me to reboot the Windows box I've never had. Good use of my time. "We have a web form" -- that lets me paste 500 characters into a tiny box and requires 9 kinds of Javascript and captchas and other crap and doesn't allow me to keep copy of the message and doesn't facilitate a conversation and doesn't even work because my network is on fire (thanks in part to you) while email will at least get queued and retried at intervals. I also appreciate having to figure this out 14 times for 14 different operations rather than being able to just BCC the same message to all of them and get back to trying to put the fire out. Another good use of my time. "Another tired excuse here" -- if you invested the time you spend coming up with excuses into just doing it, you wouldn't be reading this rant. Mail to your role accounts is often coming either from (a) people your operation is attacking/abusing and/or (b) people who are graciously and generously trying to help you, despite (a). You owe them: (1) acknowledgement (2) investigation (3) action, if indicated (4) response/explanation (5) apology, if indicated (6) a thank-you You owe yourself: (7) remedial action to try to forestall a repeat occurrence and thus the need to keep repeating 1-6 ad infinitum This isn't hard. It's not complicated. We all solve problems far more difficult than this six times a day. If you don't do this, then YOU, and YOUR operation, are the problem. You're why we can't have nice things. And finally, if this appeal to basic professionalism and cooperation and responsibility hasn't gotten through, let me try self-interest: You should be doing this anyway because you're going to need other people to do it for you. Maybe not today. But tomorrow or the next day, when you're the target and you're desperately trying to get 37 other operations to see what's happening and stuff a sock in it before your stuff melts down, you're going to need it. And when that day comes, do you want to be thought of as the responsive, helpful, alert entity that helped others...or the blackhole that ignored role account email for years (or didn't even bother to accept it)? As the cooperative professional who discharged your basic responsibility to the Internet or as the worthless parasite who was happy to leech off everyone else's efforts but refused to make any of your own? I maintain role addresses. I pay attention to them. Every operation I've touched this century does the same, even the ones I'm no longer involved with. (And they'd better, because I check up on them, and if one day I find they're not, I will go back and kick their asses individually, thoroughly, AND in alphabetical order...because Wowbagger the Infinitely Prolonged is my role model.) If you need help: ask. I've done this a bunch of times, and so have other people. None of the solutions are perfect, but they don't have to be. They just have to work. End Rant. For now. More coffee is brewing. ---rsk [1] Procmail makes it pretty easy to winnow a lot of the wheat from the chaff. It's not perfect, but it's functional enough and when it's incrementally refined over time, it can be made to successively approximate the hypothetical "correct". Experience indicates that even if it misses (that is: fails to file some incoming traffic as a legitimate report) that enough other reports about the same problem will be filed correctly making it possible (a) to do steps 1-6 above and (b) improve the procmail rules for next time as part of step 7. The (b) part is important. Every investment in it makes the entire process better and thus reduces future workloads. A rather effective role-account-handling pipeline can be built on a single box using the 'nix OS and MTA of your choice, plus fetchmail, procmail, and Mailman. And a quality mail client: I strongly recommend mutt, as it's lightweight, fast, full-featured, and about as impervious to attack as a mail client can be, which is a good thing when you're handling a lot of known-hostile email. Hint: a procmail ruleset built from the email addresses of everyone who's sent something to nanog, mailop, dns-operations, outages-discussion, etc. over the last five years is a good start. A decent second pass includes the role addresses found in SOA records and the putative/likely role addresses of domains found in the first pass. Yes, that's a lot of procmail rules. Yes, that's what INCLUDERC is for. No, it's not a big deal: I'm running a procmail filter here with nearly 3000 rules *on a laptop* and the performance impact is negligible. From jxh at jxh.com Fri Oct 28 05:07:17 2016 From: jxh at jxh.com (Jim Hickstein) Date: Fri, 28 Oct 2016 00:07:17 -0500 Subject: Spitballing IoT Security In-Reply-To: <22546.52494.516440.706525@gargle.gargle.HOWL> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> Message-ID: <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> On 10/27/16 22:59, bzs at TheWorld.com wrote: > What would the manufacturers' response be if this virus had instead > just shut down, possibly in some cases physically damaged the devices > or otherwise caused them to cease functioning ever again (wiped all > their software or broke their bootability), rather than just hijacked > them for a while? A virus that kills its host (too much of the time) is not successful. From nanog at jima.us Fri Oct 28 17:22:00 2016 From: nanog at jima.us (Jima) Date: Fri, 28 Oct 2016 11:22:00 -0600 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> Message-ID: <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> >> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >>> :-) >>> http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 This is great! Except for all of their mutual customers who had circuits from both for redundancy. (See also: Level 3's and TWTC's mutual customers, and probably a long list of other M&A I'm not thinking of off-hand.) OK, I lied about it being great anyway. Jima From cscora at apnic.net Fri Oct 28 18:01:31 2016 From: cscora at apnic.net (Routing Analysis Role Account) Date: Sat, 29 Oct 2016 04:01:31 +1000 (AEST) Subject: Weekly Routing Table Report Message-ID: <20161028180131.1340DC55A1@thyme.apnic.net> This is an automated weekly mailing describing the state of the Internet Routing Table as seen from APNIC's router in Japan. The posting is sent to APOPS, NANOG, AfNOG, AusNOG, SANOG, PacNOG, SAFNOG, SdNOG, BJNOG, CaribNOG and the RIPE Routing WG. Daily listings are sent to bgp-stats at lists.apnic.net For historical data, please see http://thyme.rand.apnic.net. If you have any comments please contact Philip Smith . Routing Table Report 04:00 +10GMT Sat 29 Oct, 2016 Report Website: http://thyme.rand.apnic.net Detailed Analysis: http://thyme.rand.apnic.net/current/ Analysis Summary ---------------- BGP routing table entries examined: 619119 Prefixes after maximum aggregation (per Origin AS): 220712 Deaggregation factor: 2.81 Unique aggregates announced (without unneeded subnets): 300943 Total ASes present in the Internet Routing Table: 55082 Prefixes per ASN: 11.24 Origin-only ASes present in the Internet Routing Table: 36320 Origin ASes announcing only one prefix: 15327 Transit ASes present in the Internet Routing Table: 6532 Transit-only ASes present in the Internet Routing Table: 167 Average AS path length visible in the Internet Routing Table: 4.3 Max AS path length visible: 37 Max AS path prepend of ASN ( 55644) 31 Prefixes from unregistered ASNs in the Routing Table: 52 Unregistered ASNs in the Routing Table: 16 Number of 32-bit ASNs allocated by the RIRs: 15924 Number of 32-bit ASNs visible in the Routing Table: 12230 Prefixes from 32-bit ASNs in the Routing Table: 49420 Number of bogon 32-bit ASNs visible in the Routing Table: 281 Special use prefixes present in the Routing Table: 0 Prefixes being announced from unallocated address space: 352 Number of addresses announced to Internet: 2828482468 Equivalent to 168 /8s, 151 /16s and 55 /24s Percentage of available address space announced: 76.4 Percentage of allocated address space announced: 76.4 Percentage of available address space allocated: 100.0 Percentage of address space in use by end-sites: 98.3 Total number of prefixes smaller than registry allocations: 203450 APNIC Region Analysis Summary ----------------------------- Prefixes being announced by APNIC Region ASes: 156739 Total APNIC prefixes after maximum aggregation: 42750 APNIC Deaggregation factor: 3.67 Prefixes being announced from the APNIC address blocks: 170488 Unique aggregates announced from the APNIC address blocks: 69969 APNIC Region origin ASes present in the Internet Routing Table: 5173 APNIC Prefixes per ASN: 32.96 APNIC Region origin ASes announcing only one prefix: 1140 APNIC Region transit ASes present in the Internet Routing Table: 947 Average APNIC Region AS path length visible: 4.3 Max APNIC Region AS path length visible: 35 Number of APNIC region 32-bit ASNs visible in the Routing Table: 2444 Number of APNIC addresses announced to Internet: 758852036 Equivalent to 45 /8s, 59 /16s and 41 /24s APNIC AS Blocks 4608-4864, 7467-7722, 9216-10239, 17408-18431 (pre-ERX allocations) 23552-24575, 37888-38911, 45056-46079, 55296-56319, 58368-59391, 63488-64098, 64297-64395, 131072-137529 APNIC Address Blocks 1/8, 14/8, 27/8, 36/8, 39/8, 42/8, 43/8, 49/8, 58/8, 59/8, 60/8, 61/8, 101/8, 103/8, 106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8, 116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8, 123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8, 163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8, 203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8, 222/8, 223/8, ARIN Region Analysis Summary ---------------------------- Prefixes being announced by ARIN Region ASes: 187849 Total ARIN prefixes after maximum aggregation: 89675 ARIN Deaggregation factor: 2.09 Prefixes being announced from the ARIN address blocks: 193888 Unique aggregates announced from the ARIN address blocks: 89398 ARIN Region origin ASes present in the Internet Routing Table: 16158 ARIN Prefixes per ASN: 12.00 ARIN Region origin ASes announcing only one prefix: 5667 ARIN Region transit ASes present in the Internet Routing Table: 1712 Average ARIN Region AS path length visible: 3.8 Max ARIN Region AS path length visible: 23 Number of ARIN region 32-bit ASNs visible in the Routing Table: 1549 Number of ARIN addresses announced to Internet: 1105484960 Equivalent to 65 /8s, 228 /16s and 92 /24s ARIN AS Blocks 1-1876, 1902-2042, 2044-2046, 2048-2106 (pre-ERX allocations) 2138-2584, 2615-2772, 2823-2829, 2880-3153 3354-4607, 4865-5119, 5632-6655, 6912-7466 7723-8191, 10240-12287, 13312-15359, 16384-17407 18432-20479, 21504-23551, 25600-26591, 26624-27647, 29696-30719, 31744-33791 35840-36863, 39936-40959, 46080-47103 53248-55295, 62464-63487, 64198-64296, 393216-397212 ARIN Address Blocks 3/8, 4/8, 6/8, 7/8, 8/8, 9/8, 11/8, 12/8, 13/8, 15/8, 16/8, 17/8, 18/8, 19/8, 20/8, 21/8, 22/8, 23/8, 24/8, 26/8, 28/8, 29/8, 30/8, 32/8, 33/8, 34/8, 35/8, 38/8, 40/8, 44/8, 45/8, 47/8, 48/8, 50/8, 52/8, 53/8, 54/8, 55/8, 56/8, 57/8, 63/8, 64/8, 65/8, 66/8, 67/8, 68/8, 69/8, 70/8, 71/8, 72/8, 73/8, 74/8, 75/8, 76/8, 96/8, 97/8, 98/8, 99/8, 100/8, 104/8, 107/8, 108/8, 128/8, 129/8, 130/8, 131/8, 132/8, 134/8, 135/8, 136/8, 137/8, 138/8, 139/8, 140/8, 142/8, 143/8, 144/8, 146/8, 147/8, 148/8, 149/8, 152/8, 155/8, 156/8, 157/8, 158/8, 159/8, 160/8, 161/8, 162/8, 164/8, 165/8, 166/8, 167/8, 168/8, 169/8, 170/8, 172/8, 173/8, 174/8, 184/8, 192/8, 198/8, 199/8, 204/8, 205/8, 206/8, 207/8, 208/8, 209/8, 214/8, 215/8, 216/8, RIPE Region Analysis Summary ---------------------------- Prefixes being announced by RIPE Region ASes: 147174 Total RIPE prefixes after maximum aggregation: 72269 RIPE Deaggregation factor: 2.04 Prefixes being announced from the RIPE address blocks: 157973 Unique aggregates announced from the RIPE address blocks: 97288 RIPE Region origin ASes present in the Internet Routing Table: 18133 RIPE Prefixes per ASN: 8.71 RIPE Region origin ASes announcing only one prefix: 7797 RIPE Region transit ASes present in the Internet Routing Table: 3036 Average RIPE Region AS path length visible: 4.3 Max RIPE Region AS path length visible: 27 Number of RIPE region 32-bit ASNs visible in the Routing Table: 5080 Number of RIPE addresses announced to Internet: 708894976 Equivalent to 42 /8s, 64 /16s and 225 /24s RIPE AS Blocks 1877-1901, 2043, 2047, 2107-2136, 2585-2614 (pre-ERX allocations) 2773-2822, 2830-2879, 3154-3353, 5377-5631 6656-6911, 8192-9215, 12288-13311, 15360-16383 20480-21503, 24576-25599, 28672-29695 30720-31743, 33792-35839, 38912-39935 40960-45055, 47104-52223, 56320-58367 59392-61439, 61952-62463, 64396-64495 196608-207259 RIPE Address Blocks 2/8, 5/8, 25/8, 31/8, 37/8, 46/8, 51/8, 62/8, 77/8, 78/8, 79/8, 80/8, 81/8, 82/8, 83/8, 84/8, 85/8, 86/8, 87/8, 88/8, 89/8, 90/8, 91/8, 92/8, 93/8, 94/8, 95/8, 109/8, 141/8, 145/8, 151/8, 176/8, 178/8, 185/8, 188/8, 193/8, 194/8, 195/8, 212/8, 213/8, 217/8, LACNIC Region Analysis Summary ------------------------------ Prefixes being announced by LACNIC Region ASes: 62909 Total LACNIC prefixes after maximum aggregation: 12672 LACNIC Deaggregation factor: 4.96 Prefixes being announced from the LACNIC address blocks: 79235 Unique aggregates announced from the LACNIC address blocks: 37500 LACNIC Region origin ASes present in the Internet Routing Table: 2479 LACNIC Prefixes per ASN: 31.96 LACNIC Region origin ASes announcing only one prefix: 553 LACNIC Region transit ASes present in the Internet Routing Table: 593 Average LACNIC Region AS path length visible: 4.9 Max LACNIC Region AS path length visible: 22 Number of LACNIC region 32-bit ASNs visible in the Routing Table: 2886 Number of LACNIC addresses announced to Internet: 170733632 Equivalent to 10 /8s, 45 /16s and 48 /24s LACNIC AS Blocks 26592-26623, 27648-28671, 52224-53247, 61440-61951, 64099-64197, 262144-266652 + ERX transfers LACNIC Address Blocks 177/8, 179/8, 181/8, 186/8, 187/8, 189/8, 190/8, 191/8, 200/8, 201/8, AfriNIC Region Analysis Summary ------------------------------- Prefixes being announced by AfriNIC Region ASes: 14975 Total AfriNIC prefixes after maximum aggregation: 3338 AfriNIC Deaggregation factor: 4.49 Prefixes being announced from the AfriNIC address blocks: 17183 Unique aggregates announced from the AfriNIC address blocks: 6467 AfriNIC Region origin ASes present in the Internet Routing Table: 736 AfriNIC Prefixes per ASN: 23.35 AfriNIC Region origin ASes announcing only one prefix: 170 AfriNIC Region transit ASes present in the Internet Routing Table: 180 Average AfriNIC Region AS path length visible: 4.7 Max AfriNIC Region AS path length visible: 18 Number of AfriNIC region 32-bit ASNs visible in the Routing Table: 271 Number of AfriNIC addresses announced to Internet: 84162560 Equivalent to 5 /8s, 4 /16s and 56 /24s AfriNIC AS Blocks 36864-37887, 327680-328703 & ERX transfers AfriNIC Address Blocks 41/8, 102/8, 105/8, 154/8, 196/8, 197/8, APNIC Region per AS prefix count summary ---------------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5540 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3584 382 248 TPG-INTERNET-AP TPG Telecom Limited, AU 17974 2958 905 71 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9829 2694 1497 530 BSNL-NIB National Internet Backbone, IN 4766 2692 11132 739 KIXS-AS-KR Korea Telecom, KR 9808 2231 8781 42 CMNET-GD Guangdong Mobile Communication 4755 2050 429 215 TATACOMM-AS TATA Communications formerl 4808 1768 2291 520 CHINA169-BJ China Unicom Beijing Provin 45899 1603 1239 93 VNPT-AS-VN VNPT Corp, VN 24560 1558 520 227 AIRTELBROADBAND-AS-AP Bharti Airtel Ltd Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-APNIC ARIN Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 22773 3550 2965 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 6327 2986 1333 83 SHAW - Shaw Communications Inc., CA 18566 2197 405 110 MEGAPATH5-US - MegaPath Corporation, US 6389 2095 3671 39 BELLSOUTH-NET-BLK - BellSouth.net Inc., 20115 1946 1999 405 CHARTER-NET-HKY-NC - Charter Communicat 30036 1767 338 328 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 209 1725 5067 658 CENTURYLINK-US-LEGACY-QWEST - Qwest Com 6983 1679 848 227 ITCDELTA - Earthlink, Inc., US 701 1610 10658 670 UUNET - MCI Communications Services, In 7018 1483 20476 1048 ATT-INTERNET4 - AT&T Services, Inc., US Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-ARIN RIPE Region per AS prefix count summary --------------------------------------- ASN No of nets /20 equiv MaxAgg Description 39891 3329 169 15 ALJAWWALSTC-AS , SA 20940 2871 1079 2043 AKAMAI-ASN1 , US 34984 1987 326 358 TELLCOM-AS , TR 13188 1623 99 59 TRIOLAN , UA 12479 1376 1018 49 UNI2-AS , ES 8551 1204 377 46 BEZEQ-INTERNATIONAL-AS Bezeqint Interne 6849 1142 355 21 UKRTELNET , UA 8402 1113 544 15 CORBINA-AS Russia, RU 12389 950 1135 422 ROSTELECOM-AS , RU 6830 862 2755 463 LGI-UPC formerly known as UPC Broadband Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-RIPE LACNIC Region per AS prefix count summary ----------------------------------------- ASN No of nets /20 equiv MaxAgg Description 10620 3506 541 191 Telmex Colombia S.A., CO 8151 2286 3358 553 Uninet S.A. de C.V., MX 7303 1541 964 248 Telecom Argentina S.A., AR 6503 1444 437 54 Axtel, S.A.B. de C.V., MX 11830 1325 367 58 Instituto Costarricense de Electricidad 6147 1235 377 28 Telefonica del Peru S.A.A., PE 7738 1018 1882 40 Telemar Norte Leste S.A., BR 28573 1007 2264 166 CLARO S.A., BR 3816 978 473 195 COLOMBIA TELECOMUNICACIONES S.A. ESP, C 11172 906 126 76 Alestra, S. de R.L. de C.V., MX Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-LACNIC AfriNIC Region per AS prefix count summary ------------------------------------------ ASN No of nets /20 equiv MaxAgg Description 24863 1185 402 50 LINKdotNET-AS, EG 36903 680 342 128 MT-MPLS, MA 37611 672 67 6 Afrihost, ZA 36992 566 1373 25 ETISALAT-MISR, EG 8452 513 1472 15 TE-AS TE-AS, EG 37492 386 250 69 ORANGE-TN, TN 29571 364 37 12 CITelecom-AS, CI 24835 313 560 19 RAYA-AS, EG 15399 307 35 6 WANANCHI-KE, KE 2018 269 327 74 TENET-1, ZA Complete listing at http://thyme.rand.apnic.net/current/data-ASnet-AFRINIC Global Per AS prefix count summary ---------------------------------- ASN No of nets /20 equiv MaxAgg Description 4538 5540 4190 74 ERX-CERNET-BKB China Education and Rese 7545 3584 382 248 TPG-INTERNET-AP TPG Telecom Limited, AU 22773 3550 2965 146 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 10620 3506 541 191 Telmex Colombia S.A., CO 39891 3329 169 15 ALJAWWALSTC-AS , SA 6327 2986 1333 83 SHAW - Shaw Communications Inc., CA 17974 2958 905 71 TELKOMNET-AS2-AP PT Telekomunikasi Indo 20940 2871 1079 2043 AKAMAI-ASN1 , US 9829 2694 1497 530 BSNL-NIB National Internet Backbone, IN 4766 2692 11132 739 KIXS-AS-KR Korea Telecom, KR Complete listing at http://thyme.rand.apnic.net/current/data-ASnet Global Per AS Maximum Aggr summary ---------------------------------- ASN No of nets Net Savings Description 22773 3550 3404 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 7545 3584 3336 TPG-INTERNET-AP TPG Telecom Limited, AU 10620 3506 3315 Telmex Colombia S.A., CO 39891 3329 3314 ALJAWWALSTC-AS , SA 6327 2986 2903 SHAW - Shaw Communications Inc., CA 17974 2958 2887 TELKOMNET-AS2-AP PT Telekomunikasi Indo 9808 2231 2189 CMNET-GD Guangdong Mobile Communication 9829 2694 2164 BSNL-NIB National Internet Backbone, IN 18566 2197 2087 MEGAPATH5-US - MegaPath Corporation, US 6389 2095 2056 BELLSOUTH-NET-BLK - BellSouth.net Inc., Complete listing at http://thyme.rand.apnic.net/current/data-CIDRnet List of Unregistered Origin ASNs (Global) ----------------------------------------- Bad AS Designation Network Transit AS Description 65001 PRIVATE 5.143.176.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/20 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.192.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.200.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.208.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65001 PRIVATE 31.172.216.0/21 15468 KLGELECS-AS 38, Teatralnaya st 65000 PRIVATE 31.219.177.0/25 8966 ETISALAT-AS P.O. Box 1150, Dub 65000 PRIVATE 31.219.177.128/25 8966 ETISALAT-AS P.O. Box 1150, Dub 64855 PRIVATE 41.220.200.0/21 36945 mcelISP, MZ 65001 PRIVATE 46.237.32.0/20 13118 ASN-YARTELECOM PJSC Rostelecom Complete listing at http://thyme.rand.apnic.net/current/data-badAS Advertised Unallocated Addresses -------------------------------- Network Origin AS Description 27.100.7.0/24 56096 UNKNOWN 41.73.1.0/24 37004 -Reserved AS-, ZZ 41.73.2.0/24 37004 -Reserved AS-, ZZ 41.73.3.0/24 37004 -Reserved AS-, ZZ 41.73.4.0/24 37004 -Reserved AS-, ZZ 41.73.5.0/24 37004 -Reserved AS-, ZZ 41.73.6.0/24 37004 -Reserved AS-, ZZ 41.73.7.0/24 37004 -Reserved AS-, ZZ 41.73.8.0/24 37004 -Reserved AS-, ZZ 41.73.9.0/24 37004 -Reserved AS-, ZZ Complete listing at http://thyme.rand.apnic.net/current/data-add-IANA Number of prefixes announced per prefix length (Global) ------------------------------------------------------- /1:0 /2:0 /3:0 /4:0 /5:0 /6:0 /7:0 /8:16 /9:13 /10:36 /11:102 /12:273 /13:522 /14:1048 /15:1781 /16:13144 /17:7835 /18:12976 /19:25476 /20:38773 /21:40443 /22:71412 /23:60071 /24:343970 /25:451 /26:345 /27:293 /28:58 /29:34 /30:14 /31:1 /32:32 Advertised prefixes smaller than registry allocations ----------------------------------------------------- ASN No of nets Total ann. Description 39891 2896 3329 ALJAWWALSTC-AS , SA 6327 2782 2986 SHAW - Shaw Communications Inc., CA 22773 2770 3550 ASN-CXA-ALL-CCI-22773-RDC - Cox Communi 18566 2089 2197 MEGAPATH5-US - MegaPath Corporation, US 30036 1582 1767 MEDIACOM-ENTERPRISE-BUSINESS - Mediacom 10620 1433 3506 Telmex Colombia S.A., CO 6389 1393 2095 BELLSOUTH-NET-BLK - BellSouth.net Inc., 13188 1362 1623 TRIOLAN , UA 6983 1325 1679 ITCDELTA - Earthlink, Inc., US 34984 1272 1987 TELLCOM-AS , TR Complete listing at http://thyme.rand.apnic.net/current/data-sXXas-nos Number of /24s announced per /8 block (Global) ---------------------------------------------- 1:1541 2:800 4:23 5:2239 6:32 8:992 12:1805 13:50 14:1743 15:46 16:2 17:100 18:126 20:48 23:1876 24:1807 27:2355 31:1795 32:83 33:4 35:3 36:330 37:2388 38:1280 39:34 40:101 41:2985 42:463 43:1881 44:48 45:2212 46:2624 47:104 49:1214 50:928 51:14 52:557 53:2 54:351 55:7 56:4 57:40 58:1578 59:995 60:378 61:1837 62:1525 63:1936 64:4580 65:2175 66:4329 67:2255 68:1145 69:3303 70:1292 71:563 72:2078 74:2592 75:404 76:416 77:1428 78:1301 79:909 80:1309 81:1405 82:985 83:706 84:893 85:1590 86:479 87:1125 88:560 89:2053 90:222 91:6121 92:934 93:2360 94:2401 95:2487 96:555 97:345 98:945 99:89 100:147 101:1171 103:12713 104:2545 105:129 106:450 107:1432 108:777 109:2505 110:1277 111:1644 112:1128 113:1142 114:1091 115:1688 116:1672 117:1550 118:2070 119:1568 120:953 121:1075 122:2264 123:2030 124:1576 125:1822 128:693 129:451 130:415 131:1371 132:610 133:175 134:512 135:200 136:400 137:395 138:1799 139:435 140:598 141:453 142:676 143:928 144:734 145:166 146:949 147:660 148:1379 149:535 150:655 151:885 152:685 153:302 154:693 155:937 156:534 157:536 158:410 159:1273 160:555 161:721 162:2416 163:588 164:775 165:1129 166:348 167:1199 168:2317 169:671 170:2224 171:252 172:778 173:1810 174:768 175:667 176:1728 177:4145 178:2354 179:1197 180:2158 181:1929 182:2097 183:1006 184:826 185:7755 186:3251 187:2168 188:2226 189:1763 190:7859 191:1325 192:9100 193:5749 194:4479 195:3840 196:1811 197:1263 198:5604 199:5833 200:7168 201:3740 202:10155 203:9848 204:4504 205:2757 206:2997 207:3097 208:3993 209:3885 210:3873 211:2042 212:2749 213:2359 214:878 215:68 216:5738 217:1974 218:820 219:598 220:1643 221:878 222:708 223:1331 End of report From cheetahmorph at gmail.com Fri Oct 28 17:58:39 2016 From: cheetahmorph at gmail.com (Jippen) Date: Fri, 28 Oct 2016 10:58:39 -0700 Subject: Need to reach someone in Bell Canada Message-ID: Hello folks - I work for a ticketfly.com - a company that changed a lot of DNS records on wednesday, that are resolving correctly around the world... except Bell CA. If anyone here is @bell.ca and willing to beat their DNS servers briefly on my behalf, I would be very, very grateful. Currently racing against several coworkers stuck in the phone tree/tier 1 customer support From incudie at gmail.com Fri Oct 28 18:22:27 2016 From: incudie at gmail.com (Timothy Lister) Date: Fri, 28 Oct 2016 11:22:27 -0700 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net>, <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> Message-ID: So if this went through, how would it happen? Does 3356 (L3) absorb 209's (CL) infrastructure and slowly make customers change their peering config to hit 3356 instead? You make a good point, I have at least a couple clients that peer to both providers for redundancy. One of which just recently signed an agreement with CenturyLink for the sole purpose of fail over. -----Original Message----- Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed From: Jima To: >> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >>> :-) >>> http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 This is great! Except for all of their mutual customers who had circuits from both for redundancy. (See also: Level 3's and TWTC's mutual customers, and probably a long list of other M&A I'm not thinking of off-hand.) OK, I lied about it being great anyway. Jima -----Original Message----- Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed From: Jima To: From mel at beckman.org Fri Oct 28 19:18:43 2016 From: mel at beckman.org (Mel Beckman) Date: Fri, 28 Oct 2016 19:18:43 +0000 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net>, <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us>, Message-ID: <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> Level3 hasn't even finished migrating its TWTelecom customers to the L3 AS yes, and it's been years. So I don't think you can expect any faster transition for CL. -mel beckman > On Oct 28, 2016, at 2:16 PM, Timothy Lister wrote: > > So if this went through, how would it happen? Does 3356 (L3) absorb 209's > (CL) infrastructure and slowly make customers change their peering config > to hit 3356 instead? > > You make a good point, I have at least a couple clients that peer to both > providers for redundancy. One of which just recently signed an agreement > with CenturyLink for the sole purpose of fail over. > > -----Original Message----- > Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > Interweb is doomed > From: Jima > To: >>>> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >>>> :-) > http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 > > > This is great! Except for all of their mutual customers who had circuits > from both for redundancy. (See also: Level 3's and TWTC's mutual > customers, and probably a long list of other M&A I'm not thinking of > off-hand.) > > OK, I lied about it being great anyway. > > Jima > > > > -----Original Message----- > Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > Interweb is doomed > From: Jima > To: From lguillory at reservetele.com Fri Oct 28 19:23:19 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Fri, 28 Oct 2016 19:23:19 +0000 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net>, <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us>, Message-ID: <565881B4-E7DC-447B-B2D7-83EEE014ED8D@reservetele.com> And I'm sure it would go about as well as the TW integration went. Level3 is currently having issues, we lost BGP just a bit ago and also legacy voice trunks have been down since first thing this morning. Sent from my iPhone On Oct 28, 2016, at 2:17 PM, Timothy Lister > wrote: So if this went through, how would it happen? Does 3356 (L3) absorb 209's (CL) infrastructure and slowly make customers change their peering config to hit 3356 instead? You make a good point, I have at least a couple clients that peer to both providers for redundancy. One of which just recently signed an agreement with CenturyLink for the sole purpose of fail over. Luke Guillory Network Operations Manager [cid:image9bee0e.JPG at 604f5a4c.42a728e5] Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Web: www.rtconline.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. -----Original Message----- Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed From: Jima > To: > On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: :-) http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 This is great! Except for all of their mutual customers who had circuits from both for redundancy. (See also: Level 3's and TWTC's mutual customers, and probably a long list of other M&A I'm not thinking of off-hand.) OK, I lied about it being great anyway. Jima -----Original Message----- Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed From: Jima > To: > From joelja at bogus.com Fri Oct 28 19:24:38 2016 From: joelja at bogus.com (joel jaeggli) Date: Fri, 28 Oct 2016 12:24:38 -0700 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <0hcw1u00h1cZc5601hd35k> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> Message-ID: <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com> On 10/28/16 12:18 PM, Mel Beckman wrote: > Level3 hasn't even finished migrating its TWTelecom customers to the L3 AS yes, and it's been years. So I don't think you can expect any faster transition for CL. 3549 still exists... > -mel beckman > >> On Oct 28, 2016, at 2:16 PM, Timothy Lister wrote: >> >> So if this went through, how would it happen? Does 3356 (L3) absorb 209's >> (CL) infrastructure and slowly make customers change their peering config >> to hit 3356 instead? >> >> You make a good point, I have at least a couple clients that peer to both >> providers for redundancy. One of which just recently signed an agreement >> with CenturyLink for the sole purpose of fail over. >> >> -----Original Message----- >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - >> Interweb is doomed >> From: Jima >> To: >>>>> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >>>>> :-) >> http://www.wsj.com/articles/centurylink-in-advanced-talks-to-merge-with-level-3-communications-1477589011 >> >> >> This is great! Except for all of their mutual customers who had circuits >> from both for redundancy. (See also: Level 3's and TWTC's mutual >> customers, and probably a long list of other M&A I'm not thinking of >> off-hand.) >> >> OK, I lied about it being great anyway. >> >> Jima >> >> >> >> -----Original Message----- >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - >> Interweb is doomed >> From: Jima >> To: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From esr at thyrsus.com Fri Oct 28 19:45:36 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Fri, 28 Oct 2016 15:45:36 -0400 Subject: Yet another NTP security bug we fixed before the CVE issued Message-ID: <20161028194536.GA32089@thyrsus.com> http://forums.theregister.co.uk/forum/1/2016/10/28/researchers_tag_new_brace_of_bugs_in_ntp_but_theyre_fixable/ That'd be another CVE that NTPsec dodges before it's issued. We removed interleaved mode months ago because the code smelled bad and turned out to have an implementation error in the timestamp handling. On past performance, there'll be about a 75% chance each that we've pre-fixed the other new security bugs. -- Eric S. Raymond From jeff+nanog at waddellsolutions.com Fri Oct 28 20:30:21 2016 From: jeff+nanog at waddellsolutions.com (Jeff Waddell) Date: Fri, 28 Oct 2016 16:30:21 -0400 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com> Message-ID: We were on on 4323 - we are still peered to 4323 (from a config stand point) - but the world sees us thru 3549 It is a mess on convergence On Fri, Oct 28, 2016 at 3:24 PM, joel jaeggli wrote: > On 10/28/16 12:18 PM, Mel Beckman wrote: > > Level3 hasn't even finished migrating its TWTelecom customers to the L3 > AS yes, and it's been years. So I don't think you can expect any faster > transition for CL. > 3549 still exists... > > -mel beckman > > > >> On Oct 28, 2016, at 2:16 PM, Timothy Lister wrote: > >> > >> So if this went through, how would it happen? Does 3356 (L3) absorb > 209's > >> (CL) infrastructure and slowly make customers change their peering > config > >> to hit 3356 instead? > >> > >> You make a good point, I have at least a couple clients that peer to > both > >> providers for redundancy. One of which just recently signed an agreement > >> with CenturyLink for the sole purpose of fail over. > >> > >> -----Original Message----- > >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > >> Interweb is doomed > >> From: Jima > >> To: > >>>>> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: > >>>>> :-) > >> http://www.wsj.com/articles/centurylink-in-advanced-talks- > to-merge-with-level-3-communications-1477589011 > >> > >> > >> This is great! Except for all of their mutual customers who had circuits > >> from both for redundancy. (See also: Level 3's and TWTC's mutual > >> customers, and probably a long list of other M&A I'm not thinking of > >> off-hand.) > >> > >> OK, I lied about it being great anyway. > >> > >> Jima > >> > >> > >> > >> -----Original Message----- > >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > >> Interweb is doomed > >> From: Jima > >> To: > > > > From ahebert at pubnix.net Fri Oct 28 20:58:16 2016 From: ahebert at pubnix.net (Alain Hebert) Date: Fri, 28 Oct 2016 16:58:16 -0400 Subject: Need to reach someone in Bell Canada In-Reply-To: References: Message-ID: <65f8bda0-27b7-a9ab-bdc1-163c67199acf@pubnix.net> Good luck with that. ----- Alain Hebert ahebert at pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/28/16 13:58, Jippen wrote: > Hello folks - I work for a ticketfly.com - a company that changed a lot of > DNS records on wednesday, that are resolving correctly around the world... > except Bell CA. If anyone here is @bell.ca and willing to beat their DNS > servers briefly on my behalf, I would be very, very grateful. > > Currently racing against several coworkers stuck in the phone tree/tier 1 > customer support > From rfg at tristatelogic.com Fri Oct 28 21:40:23 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 28 Oct 2016 14:40:23 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 Message-ID: <21262.1477690823@segfault.tristatelogic.com> I just got a spam from 103.11.67.105. The containing /24 appears to be unallocated APNIC space. RIPE tools seem to say that AS18450 has been routing this block since around May 23rd. I see this kind of stuff almost every day now, it seems. And you know, there are days when I really do start to wonder "Has the Internet gone mad?" I'm going to call these turkeys right now and just ask them, point blank, what the bleep they think they're doing, routing unallocated APNIC space. But if history is any guide, this is probably going to turn out to be another one of these "absentee landlord" kinds of ASes, where all they have is an answering machine. I have to either laugh or cry when I see people posting here about the non-functionality of abuse@ email addresses, and then see other people saying "Well, this is why all ASes also have phone numbers." I wish I had a dollar for every AS I had ever tried to contact where -neither- the abuse@ address -nor- the phone number got me to any actual human being. Regards, rfg From math at sizone.org Fri Oct 28 22:05:10 2016 From: math at sizone.org (Ken Chase) Date: Fri, 28 Oct 2016 18:05:10 -0400 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21262.1477690823@segfault.tristatelogic.com> References: <21262.1477690823@segfault.tristatelogic.com> Message-ID: <20161028220510.GF14457@sizone.org> On Fri, Oct 28, 2016 at 02:40:23PM -0700, Ronald F. Guilmette said: >I'm going to call these turkeys right now and just ask them, point >blank, what the bleep they think they're doing, routing unallocated >APNIC space. Makin' phat stacks. One thing the RIRs could do is put pressure on AS's to not route these objects, and start producing daily public output scores for these orgs, and emailing them -- ultimately threatening them with de-reg of their assets if they dont stop this nonsense. Further more, could get the route db's involved in dereg threats. Is the politcal will there tho? Right now there's no stigma beyond nanog-l in being a bad actor from where I sit. /kc -- Ken Chase - math at sizone.org guelph canada From dclements at gmail.com Fri Oct 28 22:10:19 2016 From: dclements at gmail.com (Doug Clements) Date: Fri, 28 Oct 2016 18:10:19 -0400 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21262.1477690823@segfault.tristatelogic.com> References: <21262.1477690823@segfault.tristatelogic.com> Message-ID: How does one get ARIN to register resources to come up with this result? https://whois.arin.net/rest/nets;q=103.11.67.105 The /16 is APNIC but there are 2 subnets that appear to be allocated from ARIN. Having just typed 'whois 103.11.67.105' I completely missed the fact that the supernet was APNIC until I checked the web interface. --Doug On Fri, Oct 28, 2016 at 5:40 PM, Ronald F. Guilmette wrote: > > > I just got a spam from 103.11.67.105. The containing /24 appears to > be unallocated APNIC space. > > RIPE tools seem to say that AS18450 has been routing this block since > around May 23rd. > > I see this kind of stuff almost every day now, it seems. And you know, > there are days when I really do start to wonder "Has the Internet gone > mad?" > > I'm going to call these turkeys right now and just ask them, point > blank, what the bleep they think they're doing, routing unallocated > APNIC space. But if history is any guide, this is probably going to > turn out to be another one of these "absentee landlord" kinds of ASes, > where all they have is an answering machine. > > I have to either laugh or cry when I see people posting here about the > non-functionality of abuse@ email addresses, and then see other people > saying "Well, this is why all ASes also have phone numbers." > > I wish I had a dollar for every AS I had ever tried to contact where > -neither- the abuse@ address -nor- the phone number got me to any > actual human being. > > > Regards, > rfg > From beecher at beecher.cc Fri Oct 28 22:30:49 2016 From: beecher at beecher.cc (Tom Beecher) Date: Fri, 28 Oct 2016 18:30:49 -0400 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21262.1477690823@segfault.tristatelogic.com> References: <21262.1477690823@segfault.tristatelogic.com> Message-ID: Spammers are doing a great job abusing the gaps in the systems. Another common pattern in the last 12-14 months has been a combination of squatting on an AS, forging some business documentation, buying transit to an IX, and proceeding to hijack prefixes over bilateral peering sessions. Pain in the rear to catch, even worse when the IX and transit providers aren't receptive to do anything about it when it's brought to their attention because the business docs used to instantiate those services are 'good enough', and they have a fiduciary interest in _not_ disconnecting the IX port or circuit. This will continue to be the norm until prefix validation is standardized and in widespread use. On Fri, Oct 28, 2016 at 5:40 PM, Ronald F. Guilmette wrote: > > > I just got a spam from 103.11.67.105. The containing /24 appears to > be unallocated APNIC space. > > RIPE tools seem to say that AS18450 has been routing this block since > around May 23rd. > > I see this kind of stuff almost every day now, it seems. And you know, > there are days when I really do start to wonder "Has the Internet gone > mad?" > > I'm going to call these turkeys right now and just ask them, point > blank, what the bleep they think they're doing, routing unallocated > APNIC space. But if history is any guide, this is probably going to > turn out to be another one of these "absentee landlord" kinds of ASes, > where all they have is an answering machine. > > I have to either laugh or cry when I see people posting here about the > non-functionality of abuse@ email addresses, and then see other people > saying "Well, this is why all ASes also have phone numbers." > > I wish I had a dollar for every AS I had ever tried to contact where > -neither- the abuse@ address -nor- the phone number got me to any > actual human being. > > > Regards, > rfg > From baldur.norddahl at gmail.com Fri Oct 28 23:02:57 2016 From: baldur.norddahl at gmail.com (Baldur Norddahl) Date: Sat, 29 Oct 2016 01:02:57 +0200 Subject: IPv6 automatic reverse DNS Message-ID: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Hello Many service providers have IPv4 reverse DNS for all their IP addresses. If nothing is more relevant, this will often just be the IPv4 address hashed somehow and tagged to the ISP domain name. For some arcane reason it is important to have the forward DNS match the reverse DNS or some mail servers might reject your mails. However with IPv6 it is not practical to build such a complete reverse DNS zone. You could do a star entry but that would fail the reverse/forward match test. It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. Does any DNS server have that feature? Should we have it? Why not? I know of some arguments for: 1a) mail servers like it 1b) anti spam filters believe in the magic of checking forward/reverse match. 2) traceroute will be nicer 3) http://ipv6-test.com/ will give me 20/20 instead of 19/20 (yes that was what got me going on this post) 4) Output from "who" command on Unix will look nicer (maybe). Regards, Baldur From rfg at tristatelogic.com Fri Oct 28 23:07:15 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 28 Oct 2016 16:07:15 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <20161028220510.GF14457@sizone.org> Message-ID: <21547.1477696035@segfault.tristatelogic.com> In message <20161028220510.GF14457 at sizone.org>, Ken Chase wrote: >On Fri, Oct 28, 2016 at 02:40:23PM -0700, Ronald F. Guilmette said: > >I'm going to call these turkeys right now and just ask them, point > >blank, what the bleep they think they're doing, routing unallocated > >APNIC space. > >Makin' phat stacks. > >One thing the RIRs could do is put pressure on AS's to not route >these objects, Will never happen. The RiRs have been crystal clear, and also utterly consistant... "Not our job man! We am not the Internetz Police." The thing that really baffles me about this kind of thing is how this kind of crud can happen in the first place, and also, even more baffling, how it can persist for months on end without anybody even noticing. I'm looking at this: http://lg.ring.nlnog.net/prefix_detail/lg01/ipv4?q=103.11.67.105 which appears to provide a nice list of the rest of the netwits who are also to blame for this one particular singular bit of idiocy: AS2914 -- NTT America, Inc. AS1299 -- Telia Company AB AS12798 -- Ace Data Centers, Inc. AS174 -- Cogent Communications AS6939 -- Hurricane Electric, Inc. AS3491 -- PCCW Global AS7922 -- Comcast Cable Communications, LLC AS6762 -- Telecom Italia Sparkle / Seabone AS10026 -- Pacnet Global Ltd AS11798 -- Ace Data Centers, Inc. So, um, is it really the case that -none- of the above companies have even noticed that anything was amiss here, and that all have failed to do so for months on end? (Or did they notice, but then felt is wasn't their place to say anything about it?) Sorry if those are stupid or naive questions, but... "The more I know, the less I understand." -- Don Henley Is this just another one of these cases where everybody is responsible and thus, nobody is? Is it really the case that none of the above companies ever check that what their peers announce is consistant with any routing registry? I don't pretend to understand this stuff. Somebody please 'splain it to me. I'll be much obliged. Regards, rfg From nick at foobar.org Fri Oct 28 23:10:05 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 00:10:05 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21547.1477696035@segfault.tristatelogic.com> References: <21547.1477696035@segfault.tristatelogic.com> Message-ID: <5813DACD.3000309@foobar.org> Ronald F. Guilmette wrote: > Will never happen. The RiRs have been crystal clear, and also utterly > consistant... "Not our job man! We am not the Internetz Police." Ron, Maybe you could suggest some ideas about how the RIRs can stop someone from illegally squatting space? Nick From cb.list6 at gmail.com Fri Oct 28 23:22:03 2016 From: cb.list6 at gmail.com (Ca By) Date: Fri, 28 Oct 2016 16:22:03 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5813DACD.3000309@foobar.org> References: <21547.1477696035@segfault.tristatelogic.com> <5813DACD.3000309@foobar.org> Message-ID: On Friday, October 28, 2016, Nick Hilliard wrote: > Ronald F. Guilmette wrote: > > Will never happen. The RiRs have been crystal clear, and also utterly > > consistant... "Not our job man! We am not the Internetz Police." > > Ron, > > Maybe you could suggest some ideas about how the RIRs can stop someone > from illegally squatting space? > > Nick > If the space is unassigned, could the rir announce the space to park it to null0. And register it in spamhaus ? This would make the rir the custodian of the space in their possession CB From steve at blighty.com Fri Oct 28 23:28:44 2016 From: steve at blighty.com (Steve Atkins) Date: Fri, 28 Oct 2016 16:28:44 -0700 Subject: IPv6 automatic reverse DNS In-Reply-To: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Message-ID: <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> > On Oct 28, 2016, at 4:02 PM, Baldur Norddahl wrote: > > Hello > > Many service providers have IPv4 reverse DNS for all their IP addresses. If nothing is more relevant, this will often just be the IPv4 address hashed somehow and tagged to the ISP domain name. For some arcane reason it is important to have the forward DNS match the reverse DNS or some mail servers might reject your mails. > > However with IPv6 it is not practical to build such a complete reverse DNS zone. You could do a star entry but that would fail the reverse/forward match test. > > It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. > > Does any DNS server have that feature? It's easy enough to implement with plugins on some servers. > Should we have it? Meh. > Why not? Because having an automatically generated reverse DNS is a sign that the IP address is not really intended to be offering public services, rather it's a malware-infested end user machine. > > I know of some arguments for: > > 1a) mail servers like it ... because it's a sign that the mail is coming from a real mailserver configured by a competent admin, rather than being a random compromised machine. That's not the case if you're just synthesizing reverse DNS for arbitrary IP addresses on your network. > > 1b) anti spam filters believe in the magic of checking forward/reverse match. For the same reason as above. Spam filters are also often smart enough to recognize, and treat as dubious, synthesized reverse DNS. If you have synthesized reverse DNS on your smarthost you're likely to have a bad time, perhaps initially, perhaps the first time someone notices bad mail coming from it and doesn't recognize it as a legitimate smarthost. > > 2) traceroute will be nicer Most of those hosts a traceroute goes through should hopefully have stable IP addresses and meaningful, not synthesized, reverse DNS, I'd think. Consumer endpoints are the only ones where you might expect that not to be the case and synthesized reverse DNS might be an improvement there. > > 3) http://ipv6-test.com/ will give me 20/20 instead of 19/20 (yes that was what got me going on this post) > > 4) Output from "who" command on Unix will look nicer (maybe). > > Regards, > > Baldur Cheers, Steve From nick at foobar.org Fri Oct 28 23:31:35 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 00:31:35 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: References: <21547.1477696035@segfault.tristatelogic.com> <5813DACD.3000309@foobar.org> Message-ID: <5813DFD7.7030407@foobar.org> Ca By wrote: > If the space is unassigned, could the rir announce the space to park it > to null0. And register it in spamhaus ? > > This would make the rir the custodian of the space in their possession The space isn't unallocated. It's allocated, but the assignee hasn't announced it in the dfz. There are some statistics about unallocated space here: http://www.potaroo.net/tools/ipv4/index.html summary: this isn't the problem area. Nick From marka at isc.org Fri Oct 28 23:32:12 2016 From: marka at isc.org (Mark Andrews) Date: Sat, 29 Oct 2016 10:32:12 +1100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: Your message of "Sat, 29 Oct 2016 00:10:05 +0100." <5813DACD.3000309@foobar.org> References: <21547.1477696035@segfault.tristatelogic.com> <5813DACD.3000309@foobar.org> Message-ID: <20161028233212.D3AB657F9742@rock.dv.isc.org> In message <5813DACD.3000309 at foobar.org>, Nick Hilliard writes: > Ronald F. Guilmette wrote: > > Will never happen. The RiRs have been crystal clear, and also utterly > > consistant... "Not our job man! We am not the Internetz Police." > > Ron, > > Maybe you could suggest some ideas about how the RIRs can stop someone > from illegally squatting space? It's not the RIR's job. They already provide the framework for ISP's to do the job of policing route announcements themselves. ISP's just need to use that framework. > Nick -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka at isc.org From nick at foobar.org Fri Oct 28 23:33:18 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 00:33:18 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <20161028233212.D3AB657F9742@rock.dv.isc.org> References: <21547.1477696035@segfault.tristatelogic.com> <5813DACD.3000309@foobar.org> <20161028233212.D3AB657F9742@rock.dv.isc.org> Message-ID: <5813E03E.6060907@foobar.org> Mark Andrews wrote: > It's not the RIR's job. They already provide the framework for > ISP's to do the job of policing route announcements themselves. > ISP's just need to use that framework. Ron thinks otherwise. I'd like to understand what he thinks they can do to stop this. Nick From rfg at tristatelogic.com Fri Oct 28 23:36:45 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 28 Oct 2016 16:36:45 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: Message-ID: <21691.1477697805@segfault.tristatelogic.com> In message Doug Clements wrote: >How does one get ARIN to register resources to come up with this result? > >https://whois.arin.net/rest/nets;q=103.11.67.105 > >The /16 is APNIC but there are 2 subnets that appear to be allocated from >ARIN. Having just typed 'whois 103.11.67.105' I completely missed the fact >that the supernet was APNIC until I checked the web interface. Oh!! Wow!! I totally missed this also, i.e. that ARIN is showing an allocation for 103.11.64.0/22 to HostUs.Us in Texas. That's really weird, but even that doesn't either explain or excuse what still looks like an illicit squat (by an unrelated Los Angeles company) on the 103.11.67.0/24 block to me... perhaps one that's been re-sold to a spammer (which seems possible, given the spam I got). In my own defense, I didn't see the ARIN allocation because I have a normative process that I use for looking up IP addresses. It's hierarchical, and I always start with whatver whois.iana.org has to say. And it says that that 103.0.0.0/8 belongs to APNIC, so of course, I only looked at what whois.apnic.net had to say about 103.11.67.105. And it says that it's unallocated. (And apparently, data shown for announced prefixes on the bgp.he.net web site is also obtained in this same straightforward way, because it also is showing 103.11.67.0/24 as registered to "Asia Pacific Network Information Centre".) This isn't the first time I've wished that the right hand knew (or cared) what the left hand was doing. I've asked the folks at IANA about this sort of thing in the past, i.e. them giving pointers to the apparently wrong RiR whois server, and they just won't fix it. They just shrug and say "Not our problem man!" And in this case, maybe they're right. If APNIC gave two subparts of 103/8 to ARIN, it might have been helpful if their own whois server was made aware of that fact. Sigh. I have to keep reminding myself of what one friend of mine keeps on telling me... "Ron, there you go again, trying to think about these things logically." Regards, rfg From rfg at tristatelogic.com Fri Oct 28 23:53:29 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 28 Oct 2016 16:53:29 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5813DACD.3000309@foobar.org> Message-ID: <21745.1477698809@segfault.tristatelogic.com> In message <5813DACD.3000309 at foobar.org>, Nick Hilliard wrote: >Ronald F. Guilmette wrote: >> Will never happen. The RiRs have been crystal clear, and also utterly >> consistant... "Not our job man! We am not the Internetz Police." > >Ron, > >Maybe you could suggest some ideas about how the RIRs can stop someone >from illegally squatting space? Oh, don't get me wrong. I never said that I either could or would suggest how to convert RiRs into The Internet Police. Nor did I suggest that such a conversion would even be either prudent or advisable. (I am not persuaded that it would be.) We have a longstanding 20 or 30 year tradition/precedent and a division of labor that -does not- allocate to RiRs any responsibility for, or authority over anything to do with what routes people announce, and I am certainly not even nearly so presumptive as to believe that I either can or should try to roll back 30 years of history and ask everyone to start all over again and build governance structures anew, from scratch. (Doing so would be both silly and the very height of arrogance on my part.) I nontheless feel free to note, and to bemoan, the current utter lack of -any- authority which routinely notices apparent routing funny business and/or which works, on a routine basis, to try to put a stop to it all. I do not suggest that RiRs should be "minding the store" with respect to route announcements. I do think it would be helpful if -somebody- were doing so. My own occasional and srictly ad hoc efforts have only succeded in convincing me of how extensive the problem is, and how dire a need there is for a more rigorous solution. Regards, rfg From math at sizone.org Fri Oct 28 23:58:12 2016 From: math at sizone.org (Ken Chase) Date: Fri, 28 Oct 2016 19:58:12 -0400 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 Message-ID: <20161028235812.GA17103@sizone.org> On Sat, Oct 29, 2016 at 10:32:12AM +1100, Mark Andrews said: >It's not the RIR's job. They already provide the framework for >ISP's to do the job of policing route announcements themselves. >ISP's just need to use that framework. What incentive do the ISPs have to enforce any of this though? In fact, they're making money sending bits over these prefixes. What incentives could be created that the ISPs wont balk at as it might affect their accidental revenues from "oh, gee, I didnt know it was being squatted! " prefixes? /kc -- Ken Chase - math at sizone.org guelph canada From lguillory at reservetele.com Sat Oct 29 00:15:40 2016 From: lguillory at reservetele.com (Luke Guillory) Date: Sat, 29 Oct 2016 00:15:40 +0000 Subject: IPv6 automatic reverse DNS In-Reply-To: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Message-ID: Why not have DHCP update dns with both. Sent from my iPad > Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory at reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . On Oct 28, 2016, at 6:04 PM, Baldur Norddahl wrote: > > Hello > > Many service providers have IPv4 reverse DNS for all their IP addresses. If nothing is more relevant, this will often just be the IPv4 address hashed somehow and tagged to the ISP domain name. For some arcane reason it is important to have the forward DNS match the reverse DNS or some mail servers might reject your mails. > > However with IPv6 it is not practical to build such a complete reverse DNS zone. You could do a star entry but that would fail the reverse/forward match test. > > It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. > > Does any DNS server have that feature? Should we have it? Why not? > > I know of some arguments for: > > 1a) mail servers like it > > 1b) anti spam filters believe in the magic of checking forward/reverse match. > > 2) traceroute will be nicer > > 3) http://ipv6-test.com/ will give me 20/20 instead of 19/20 (yes that was what got me going on this post) > > 4) Output from "who" command on Unix will look nicer (maybe). > > Regards, > > Baldur From olivier.benghozi at wifirst.fr Sat Oct 29 00:21:49 2016 From: olivier.benghozi at wifirst.fr (Olivier Benghozi) Date: Sat, 29 Oct 2016 02:21:49 +0200 Subject: IPv6 automatic reverse DNS In-Reply-To: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Message-ID: Already available: KnotDNS. https://www.knot-dns.cz/docs/2.x/html/configuration.html#synth-record-automatic-forward-reverse-records Olivier > On 29 oct. 2016 ? 01:02, Baldur Norddahl wrote : > > It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. > > Does any DNS server have that feature? Should we have it? Why not? From jared at compuwizz.net Fri Oct 28 19:57:28 2016 From: jared at compuwizz.net (Jared Geiger) Date: Fri, 28 Oct 2016 12:57:28 -0700 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com> References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com> Message-ID: Savvis 3561 still exists on Centurylink's side too. 6 networks down to 1 ... How much of that fiber for each network was running in the same conduit to begin with anyway? Centurylink Qwest Savvis Level3 Global Crossing TWTC On Fri, Oct 28, 2016 at 12:24 PM, joel jaeggli wrote: > On 10/28/16 12:18 PM, Mel Beckman wrote: > > Level3 hasn't even finished migrating its TWTelecom customers to the L3 > AS yes, and it's been years. So I don't think you can expect any faster > transition for CL. > 3549 still exists... > > -mel beckman > > > >> On Oct 28, 2016, at 2:16 PM, Timothy Lister wrote: > >> > >> So if this went through, how would it happen? Does 3356 (L3) absorb > 209's > >> (CL) infrastructure and slowly make customers change their peering > config > >> to hit 3356 instead? > >> > >> You make a good point, I have at least a couple clients that peer to > both > >> providers for redundancy. One of which just recently signed an agreement > >> with CenturyLink for the sole purpose of fail over. > >> > >> -----Original Message----- > >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > >> Interweb is doomed > >> From: Jima > >> To: > >>>>> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: > >>>>> :-) > >> http://www.wsj.com/articles/centurylink-in-advanced-talks- > to-merge-with-level-3-communications-1477589011 > >> > >> > >> This is great! Except for all of their mutual customers who had circuits > >> from both for redundancy. (See also: Level 3's and TWTC's mutual > >> customers, and probably a long list of other M&A I'm not thinking of > >> off-hand.) > >> > >> OK, I lied about it being great anyway. > >> > >> Jima > >> > >> > >> > >> -----Original Message----- > >> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - > >> Interweb is doomed > >> From: Jima > >> To: > > > > From mel at beckman.org Sat Oct 29 00:32:18 2016 From: mel at beckman.org (Mel Beckman) Date: Sat, 29 Oct 2016 00:32:18 +0000 Subject: CenturyLink in Advanced Talks to Merge With Level 3 Communications - Interweb is doomed In-Reply-To: References: <1215211938.2287995.1477589773824.ref@mail.yahoo.com> <705cdfdf-b95d-8232-3239-b7c1662e2469@cox.net> <85FA41DE-2A6A-4473-B07D-87818E3CE7D2@dataix.net> <1fa1817b-170e-3d75-feca-a131922a3d7d@jima.us> <4ABF28B3-A98C-43AB-A86E-61FF9BB57960@beckman.org> <855eaf84-fedc-6798-b458-d0f55debdcfb@bogus.com>, Message-ID: <8235E7C8-1FA4-41FC-A9B8-68D714415C38@beckman.org> It's funny you should mention that. I just learned that our CL traffic rides on a single lambda is a Level3 fiber. Oddly, though, the cost to buy that same circuit directly from Level3 is twice as high. Which bodes ill for circuit pricing in the reduced-competition environment following the merger. A similar thing happened to us when L3 bought TWT. In both cases L3 says "but you're getting such a better network!" Alas, that turned out to be not the case with the TWT acquisition, as merger mishaps caused numerous outages. -mel > On Oct 28, 2016, at 7:25 PM, Jared Geiger wrote: > > Savvis 3561 still exists on Centurylink's side too. 6 networks down to 1 > ... How much of that fiber for each network was running in the same conduit > to begin with anyway? > > Centurylink > Qwest > Savvis > > Level3 > Global Crossing > TWTC > >> On Fri, Oct 28, 2016 at 12:24 PM, joel jaeggli wrote: >> >>> On 10/28/16 12:18 PM, Mel Beckman wrote: >>> Level3 hasn't even finished migrating its TWTelecom customers to the L3 >> AS yes, and it's been years. So I don't think you can expect any faster >> transition for CL. >> 3549 still exists... >>> -mel beckman >>> >>>> On Oct 28, 2016, at 2:16 PM, Timothy Lister wrote: >>>> >>>> So if this went through, how would it happen? Does 3356 (L3) absorb >> 209's >>>> (CL) infrastructure and slowly make customers change their peering >> config >>>> to hit 3356 instead? >>>> >>>> You make a good point, I have at least a couple clients that peer to >> both >>>> providers for redundancy. One of which just recently signed an agreement >>>> with CenturyLink for the sole purpose of fail over. >>>> >>>> -----Original Message----- >>>> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - >>>> Interweb is doomed >>>> From: Jima >>>> To: >>>>>>> On 10/27/2016 12:36, Nevin Gonsalves via NANOG wrote: >>>>>>> :-) >>>> http://www.wsj.com/articles/centurylink-in-advanced-talks- >> to-merge-with-level-3-communications-1477589011 >>>> >>>> >>>> This is great! Except for all of their mutual customers who had circuits >>>> from both for redundancy. (See also: Level 3's and TWTC's mutual >>>> customers, and probably a long list of other M&A I'm not thinking of >>>> off-hand.) >>>> >>>> OK, I lied about it being great anyway. >>>> >>>> Jima >>>> >>>> >>>> >>>> -----Original Message----- >>>> Re: CenturyLink in Advanced Talks to Merge With Level 3 Communications - >>>> Interweb is doomed >>>> From: Jima >>>> To: >> >> >> >> From rfg at tristatelogic.com Sat Oct 29 00:51:56 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Fri, 28 Oct 2016 17:51:56 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5813E03E.6060907@foobar.org> Message-ID: <21986.1477702316@segfault.tristatelogic.com> In message <5813E03E.6060907 at foobar.org>, Mark Andrews wrote: >Mark Andrews wrote: >> It's not the RIR's job. They already provide the framework for >> ISP's to do the job of policing route announcements themselves. >> ISP's just need to use that framework. > >Ron thinks otherwise. No, I don't. You have made a incorrect inference from the text of my actual comment. In my actual comment I merely noted that RIRs are in fact -not- the Internet Police, and that none of them have ever displayed even the slightest desire to become that (and indeed, when asked, they have, without exception, exhibited a clear desire -not- to be assigned any such role). These observations on my part are all merely recitations of well- established historical facts, all of which are easily verifiable by anyone with a browser. I made no comment at all about who, if anyone, should be tasked to take on the role of The Routing Police. And indeed, if asked, I would express some degree of skepticism about the ability of RIRs to even reliably execute their existing data base maintenance responsibilities to a level which I personally would find entirely satisfactory. (The apparent goofyness relating to 103.11.64.0/22 is just one very small example of this, there being also many other and more serious issues that I could also cite, if pressed, relating strictly to allocation functions and/or to WHOIS data base issues.) Given that I do not have an entirely unequivocal admiration for the quality and consistancy of the work that RIRs are already clearly responsible for, do you really believe that it would be my first choice to assign an entirely seperate but equally critical set of -new- authorities and responsibilities to the RiRs? If so, please allow me to disabuse you of that notion. (I am also and likewise not likely to support any effort any any part of the United States federal government to assign new authorities and responsibilities to the Office of Personnel Management.) Regards, rfg P.S. I may be wrong about this, but it has come to my attention that many, most, or all of the WHOIS records reflecting allocations made by the AFRINIC RIR are utterly devoid of either (a) information specifying the dates on which the relevant allocations were made or (b) email contact addresses for the relevant number resource registrants. I am, of course, utterly appalled by the apparent inability of this RIR to maintain a WHOIS data base which even approximates the modest and minimal level of relevant information commonly available from the WHOIS data bases of other and older RIRs. From kauer at biplane.com.au Sat Oct 29 01:04:03 2016 From: kauer at biplane.com.au (Karl Auer) Date: Sat, 29 Oct 2016 12:04:03 +1100 Subject: IPv6 automatic reverse DNS In-Reply-To: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Message-ID: <1477703043.2229.439.camel@biplane.com.au> On Sat, 2016-10-29 at 01:02 +0200, Baldur Norddahl wrote: > It should be simple to build a DNS server that will automatically? > generate a hostname value for every reverse lookup received, and also > be able to parse that hostname value to return the correct IPv6 > address on?forward lookups. > > Does any DNS server have that feature? Should we have it? Why not? Nominum's nameserver software has these features. Industrial strength nameservice, with lots of industrial-strength features, but at an industrial-strength price. I thought BIND had grown that feature, but I haven't used BIND for a while now, so maybe not. > 1b) anti spam filters believe in the magic of checking > forward/reverse?match. Someone in this thread said that only malware-infested end-users are behind IP addresses with no reverse lookup. Well - no. As long as we keep telling anyone who isn't running a full-bore commercial network to "consume, be silent, die", we are holding everyone back, including ourselves. It's fine to use no-reverse-lookup as a component of a spamminess score. It's not OK to use it as proof of spamminess. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer at biplane.com.au) http://www.biplane.com.au/kauer http://twitter.com/kauer389 GPG fingerprint: E00D 64ED 9C6A 8605 21E0 0ED0 EE64 2BEE CBCB C38B Old fingerprint: 3C41 82BE A9E7 99A1 B931 5AE7 7638 0147 2C3C 2AC4 From stenn at ntp.org Sat Oct 29 01:36:51 2016 From: stenn at ntp.org (Harlan Stenn) Date: Sat, 29 Oct 2016 01:36:51 +0000 Subject: Yet another NTP security bug we fixed before the CVE issued In-Reply-To: <20161028194536.GA32089@thyrsus.com> References: <20161028194536.GA32089@thyrsus.com> Message-ID: "Eric S. Raymond" writes: > ... Yawn. We disabled interleave a while ago. Interleave is the best way to get the next major step in accurate time using the NTP Protocol. Yes, it needs work. A reference implementation is where this work happens. Yes, we have another release about to happen. Mostly "security" bugs that folks will not see, if they're being at all responsible. Eric, you are loved and appreciated, and respected and admired. -- Harlan Stenn http://networktimefoundation.org - be a member! From steve at blighty.com Sat Oct 29 01:37:25 2016 From: steve at blighty.com (Steve Atkins) Date: Fri, 28 Oct 2016 18:37:25 -0700 Subject: IPv6 automatic reverse DNS In-Reply-To: <1477703043.2229.439.camel@biplane.com.au> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <1477703043.2229.439.camel@biplane.com.au> Message-ID: <2EFB5469-E3E5-4E00-A4B3-88C5FE8AA00A@blighty.com> > On Oct 28, 2016, at 6:04 PM, Karl Auer wrote: > >> 1b) anti spam filters believe in the magic of checking >> forward/reverse match. > > Someone in this thread said that only malware-infested end-users are > behind IP addresses with no reverse lookup. Well - no. As long as we > keep telling anyone who isn't running a full-bore commercial network to > "consume, be silent, die", we are holding everyone back, including > ourselves. If you send mail over IPv6 from an address with no reverse DNS you will see quite a lot of this sort of thing: 550 5.7.1 [*] Our system has detected that this message 5.7.1 does not meet IPv6 sending guidelines regarding PTR records and 5.7.1 authentication. Please review 5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more 5.7.1 information. > > It's fine to use no-reverse-lookup as a component of a spamminess > score. It's not OK to use it as proof of spamminess. People running large mailservers made that decision some time ago. Disagreeing with them won't make them accept your email. Cheers, Steve From list at satchell.net Sat Oct 29 02:05:41 2016 From: list at satchell.net (Stephen Satchell) Date: Fri, 28 Oct 2016 19:05:41 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <20161028233212.D3AB657F9742@rock.dv.isc.org> References: <21547.1477696035@segfault.tristatelogic.com> <5813DACD.3000309@foobar.org> <20161028233212.D3AB657F9742@rock.dv.isc.org> Message-ID: <4ec8117f-0995-e9d6-c23e-176e7b5fee5e@satchell.net> On 10/28/2016 04:32 PM, Mark Andrews wrote: > It's not the RIR's job. They already provide the framework for > ISP's to do the job of policing route announcements themselves. > ISP's just need to use that framework. Link to documentation on how to use that framework? From wesgeorge at puck.nether.net Sat Oct 29 02:08:24 2016 From: wesgeorge at puck.nether.net (Wesley George) Date: Fri, 28 Oct 2016 22:08:24 -0400 Subject: IPv6 automatic reverse DNS In-Reply-To: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> Message-ID: I'd recommend reviewing this document, and contributing as appropriate. I think it covers this pretty thoroughly today, but if there are missing considerations, now is the time to make sure that feedback is captured. https://tools.ietf.org/html/draft-ietf-dnsop-isp-ip6rdns-02 Wes George > On Oct 28, 2016, at 7:02 PM, Baldur Norddahl wrote: > > Hello > > Many service providers have IPv4 reverse DNS for all their IP addresses. If nothing is more relevant, this will often just be the IPv4 address hashed somehow and tagged to the ISP domain name. For some arcane reason it is important to have the forward DNS match the reverse DNS or some mail servers might reject your mails. > > However with IPv6 it is not practical to build such a complete reverse DNS zone. You could do a star entry but that would fail the reverse/forward match test. > > It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. > > Does any DNS server have that feature? Should we have it? Why not? > > I know of some arguments for: > > 1a) mail servers like it > > 1b) anti spam filters believe in the magic of checking forward/reverse match. > > 2) traceroute will be nicer > > 3) http://ipv6-test.com/ will give me 20/20 instead of 19/20 (yes that was what got me going on this post) > > 4) Output from "who" command on Unix will look nicer (maybe). > > Regards, > > Baldur -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mksmith at mac.com Sat Oct 29 02:45:40 2016 From: mksmith at mac.com (Michael Smith) Date: Fri, 28 Oct 2016 19:45:40 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21691.1477697805@segfault.tristatelogic.com> References: <21691.1477697805@segfault.tristatelogic.com> Message-ID: <5DE5D5A1-9760-43AA-9022-A10D5F1CE0DD@mac.com> I would use LACNIC?s whois server for these queries. They have info from all the registries, which is an amazing service that seems beyond the other RIRs. whois -h whois.lacnic.net 103.11.67.105 HostUS HOSTUS-IPV4-5 (NET-103-11-64-0-1) 103.11.64.0 - 103.11.67.255 Gaiacom, L.C. SOLVPS-103-11-67-0-24 (NET-103-11-67-0-1) 103.11.67.0 - 103.11.67.255 Mike > On Oct 28, 2016, at 4:36 PM, Ronald F. Guilmette wrote: > > > In message > Doug Clements wrote: > >> How does one get ARIN to register resources to come up with this result? >> >> https://whois.arin.net/rest/nets;q=103.11.67.105 >> >> The /16 is APNIC but there are 2 subnets that appear to be allocated from >> ARIN. Having just typed 'whois 103.11.67.105' I completely missed the fact >> that the supernet was APNIC until I checked the web interface. > > Oh!! Wow!! I totally missed this also, i.e. that ARIN is showing an > allocation for 103.11.64.0/22 to HostUs.Us in Texas. > > That's really weird, but even that doesn't either explain or excuse > what still looks like an illicit squat (by an unrelated Los Angeles > company) on the 103.11.67.0/24 block to me... perhaps one that's been > re-sold to a spammer (which seems possible, given the spam I got). > > In my own defense, I didn't see the ARIN allocation because I have a > normative process that I use for looking up IP addresses. It's > hierarchical, and I always start with whatver whois.iana.org has to > say. And it says that that 103.0.0.0/8 belongs to APNIC, so of course, > I only looked at what whois.apnic.net had to say about 103.11.67.105. > And it says that it's unallocated. (And apparently, data shown for > announced prefixes on the bgp.he.net web site is also obtained in this > same straightforward way, because it also is showing 103.11.67.0/24 as > registered to "Asia Pacific Network Information Centre".) > > This isn't the first time I've wished that the right hand knew (or cared) > what the left hand was doing. I've asked the folks at IANA about this > sort of thing in the past, i.e. them giving pointers to the apparently > wrong RiR whois server, and they just won't fix it. They just shrug and > say "Not our problem man!" And in this case, maybe they're right. If > APNIC gave two subparts of 103/8 to ARIN, it might have been helpful > if their own whois server was made aware of that fact. > > Sigh. I have to keep reminding myself of what one friend of mine keeps > on telling me... "Ron, there you go again, trying to think about these > things logically." > > > Regards, > rfg From Andrew.White2 at charter.com Sat Oct 29 03:03:54 2016 From: Andrew.White2 at charter.com (White, Andrew) Date: Sat, 29 Oct 2016 03:03:54 +0000 Subject: IPv6 automatic reverse DNS In-Reply-To: <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> Message-ID: <8b3af23d52f4423997b7e4428f3b9b81@SC58MEXGP034.CORP.CHARTERCOM.com> There are two competing drafts for synthetic rule-based PTR responses for IPv6 rDNS: Howard Lee, Time Warner Cable (now Charter) https://tools.ietf.org/html/draft-howard-isp-ip6rdns-08 J. Woodworth, CenturyLink https://datatracker.ietf.org/doc/draft-woodworth-bulk-rr/ Nominum and Xerocole/Akamai also have proprietary solutions to this in their Vantio AuthServ and AuthX products, respectively. It seems to me that it is still an open question whether the recommendations in RFC-1912 that any IP address that accesses the Internet should have a PTR and matching forward record. My personal thoughts are that the best solution would be an OPTIONAL standards-based method of generating DNS responses based on a ruleset if a specific zone record is not present, and that implementation of that requirement should be left to the developers of the auth nameserver software. Andrew Caveat: These thoughts are mine personally and do not represent any official position of Charter Communications. ??dr?w Whi?? Charter Network Operations - DAS DNS Desk: 314-394-9594 ? Cell: 314-452-4386 andrew.white2 at charter.com -----Original Message----- From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Steve Atkins Sent: Friday, October 28, 2016 6:29 PM To: NANOG list Subject: Re: IPv6 automatic reverse DNS > On Oct 28, 2016, at 4:02 PM, Baldur Norddahl wrote: > > Hello > > Many service providers have IPv4 reverse DNS for all their IP addresses. If nothing is more relevant, this will often just be the IPv4 address hashed somehow and tagged to the ISP domain name. For some arcane reason it is important to have the forward DNS match the reverse DNS or some mail servers might reject your mails. > > However with IPv6 it is not practical to build such a complete reverse DNS zone. You could do a star entry but that would fail the reverse/forward match test. > > It should be simple to build a DNS server that will automatically generate a hostname value for every reverse lookup received, and also be able to parse that hostname value to return the correct IPv6 address on forward lookups. > > Does any DNS server have that feature? It's easy enough to implement with plugins on some servers. > Should we have it? Meh. > Why not? Because having an automatically generated reverse DNS is a sign that the IP address is not really intended to be offering public services, rather it's a malware-infested end user machine. > > I know of some arguments for: > > 1a) mail servers like it ... because it's a sign that the mail is coming from a real mailserver configured by a competent admin, rather than being a random compromised machine. That's not the case if you're just synthesizing reverse DNS for arbitrary IP addresses on your network. > > 1b) anti spam filters believe in the magic of checking forward/reverse match. For the same reason as above. Spam filters are also often smart enough to recognize, and treat as dubious, synthesized reverse DNS. If you have synthesized reverse DNS on your smarthost you're likely to have a bad time, perhaps initially, perhaps the first time someone notices bad mail coming from it and doesn't recognize it as a legitimate smarthost. > > 2) traceroute will be nicer Most of those hosts a traceroute goes through should hopefully have stable IP addresses and meaningful, not synthesized, reverse DNS, I'd think. Consumer endpoints are the only ones where you might expect that not to be the case and synthesized reverse DNS might be an improvement there. > > 3) http://ipv6-test.com/ will give me 20/20 instead of 19/20 (yes that was what got me going on this post) > > 4) Output from "who" command on Unix will look nicer (maybe). > > Regards, > > Baldur Cheers, Steve From kauer at biplane.com.au Sat Oct 29 03:26:59 2016 From: kauer at biplane.com.au (Karl Auer) Date: Sat, 29 Oct 2016 14:26:59 +1100 Subject: IPv6 automatic reverse DNS In-Reply-To: <2EFB5469-E3E5-4E00-A4B3-88C5FE8AA00A@blighty.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <1477703043.2229.439.camel@biplane.com.au> <2EFB5469-E3E5-4E00-A4B3-88C5FE8AA00A@blighty.com> Message-ID: <1477711619.2229.473.camel@biplane.com.au> On Fri, 2016-10-28 at 18:37 -0700, Steve Atkins wrote: > > On Oct 28, 2016, at 6:04 PM, Karl Auer > > wrote: > > It's fine to use no-reverse-lookup as a component of a spamminess > > score. It's not OK to use it as proof of spamminess. > People running large mailservers made that decision some time > ago. Disagreeing with them won't make them accept your email. I didn't say it would. IMHO reverse lookups are excellent and useful. My only beef is with the idea that the absence of a reverse lookup entry has any useful meaning any more, or, in particular, is proof of spamminess. It would be interesting (and would alter my opinion) to see statistics of real spamminess positives ("is spam") dropping significantly if failed reverse lookups are removed from the calculation. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer at biplane.com.au) http://www.biplane.com.au/kauer http://twitter.com/kauer389 GPG fingerprint: E00D 64ED 9C6A 8605 21E0 0ED0 EE64 2BEE CBCB C38B Old fingerprint: 3C41 82BE A9E7 99A1 B931 5AE7 7638 0147 2C3C 2AC4 From bzs at TheWorld.com Sat Oct 29 05:14:34 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sat, 29 Oct 2016 01:14:34 -0400 Subject: Spitballing IoT Security In-Reply-To: <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> Message-ID: <22548.12346.733412.473362@gargle.gargle.HOWL> On October 28, 2016 at 00:07 jxh at jxh.com (Jim Hickstein) wrote: > On 10/27/16 22:59, bzs at TheWorld.com wrote: > > What would the manufacturers' response be if this virus had instead > > just shut down, possibly in some cases physically damaged the devices > > or otherwise caused them to cease functioning ever again (wiped all > > their software or broke their bootability), rather than just hijacked > > them for a while? > > A virus that kills its host (too much of the time) is not successful. Hmm. So now we assume we are dealing with rational actors? I suppose one can find some rational motives in bringing down Dyn but I don't see that it's all that different from bricking half a million (for starters) IoT devices. Thus far the goal just seems to be mayhem. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From esr at thyrsus.com Sat Oct 29 05:26:25 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Sat, 29 Oct 2016 01:26:25 -0400 Subject: Yet another NTP security bug we fixed before the CVE issued In-Reply-To: References: <20161028194536.GA32089@thyrsus.com> Message-ID: <20161029052625.GD3514@thyrsus.com> Harlan Stenn : > Interleave is the best way to get the next major step in accurate time > using the NTP Protocol. Yes, it needs work. A reference implementation > is where this work happens. Daniel Franke judges the interleave concept doesn't actually work well enough to be worth its code weight, and that Mills believed otherwise because of an error he failed to notice in the timestamp handling. I have not looked myself, but I have found Daniel very reliable when he says such things. > Yes, we have another release about to happen. Mostly "security" bugs > that folks will not see, if they're being at all responsible. They certainly won't see those bugs in NTPsec -- Daniel briefed me about 90 minutes ago, and even if we hadn't I knew we were pre-armored against 3/4ths of the CVEs that hit you guys this year. Might just have something to do with having removed 153KLOC of useless code and winding up with less than a third of the attack surface you guys have exposed. > Eric, you are loved and appreciated, and respected and admired. That's nice. It's a damn shame you didn't "admire" me (and my team members) enough to join forces with us when we were trying to avoid a fork, rather than fighting us and forcing one to happen. Your choice, your consequences. -- Eric S. Raymond From list at satchell.net Sat Oct 29 05:27:39 2016 From: list at satchell.net (Stephen Satchell) Date: Fri, 28 Oct 2016 22:27:39 -0700 Subject: Spitballing IoT Security In-Reply-To: <22548.12346.733412.473362@gargle.gargle.HOWL> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> Message-ID: <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> On 10/28/2016 10:14 PM, bzs at TheWorld.com wrote: > Thus far the goal just seems to be mayhem. Thus far, the goal on the part of the botnet opearators is to make money. The goal of the CUSTOMERS of the botnet operators? Who knows? From lear at ofcourseimright.com Sat Oct 29 06:37:56 2016 From: lear at ofcourseimright.com (Eliot Lear) Date: Sat, 29 Oct 2016 08:37:56 +0200 Subject: Spitballing IoT Security In-Reply-To: <20161027100455.3fe4cf14@scrofula.eps.is.port.ac.uk> References: <4246.1477383031@segfault.tristatelogic.com> <580F19BF.2070002@vaxination.ca> <20161027100455.3fe4cf14@scrofula.eps.is.port.ac.uk> Message-ID: <40134ffd-906b-7a44-49ca-e0b29e5ffe33@ofcourseimright.com> Hi Mike, On 10/27/16 11:04 AM, Mike Meredith wrote: > On Thu, 27 Oct 2016 07:59:00 +0200, Eliot Lear > may have written: >> Well yes. uPnP is a problem precisely because it is some random device >> asserting on its own that it can be trusted to do what it wants. Had > From my own personal use (and I'm aware that this isn't a general > solution), I'd like a device that sat on those uPnP requests until I logged > into the admin interface to review them. Now if you could automate _me_ > then it might become more generally useful :- You need to go further. It is no longer enough to tackle this problem simply as a firewall problem, because there are too many reflection-style attacks. Not only do you want to prevent devices from opening pinholes to the Internet, but you really want to know what they're going to be doing inside the home. And Quite frankly, I disagree that you want to nag the user unless it is absolutely necessary. To me, that means authorizing the device in the first place, and the access point having access to enough intelligence to know what sort of access is necessary for a device, given its purpose. > As someone who manages an application-based firewall, every problem looks > like it would be easier to solve using an application-based firewall :) I don't generally prefer application firewalls except in limited circumstances. Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From lear at ofcourseimright.com Sat Oct 29 06:44:01 2016 From: lear at ofcourseimright.com (Eliot Lear) Date: Sat, 29 Oct 2016 08:44:01 +0200 Subject: Spitballing IoT Security In-Reply-To: <0105FA70-BE62-4F5D-9366-5AE387081ED8@gizmopartners.com> References: <4246.1477383031@segfault.tristatelogic.com> <0105FA70-BE62-4F5D-9366-5AE387081ED8@gizmopartners.com> Message-ID: Hi Chris, On 10/25/16 1:51 PM, Chris Boyd wrote: >> On Oct 25, 2016, at 3:10 AM, Ronald F. Guilmette wrote: >> >> An IoT is -not- a general purpose computer. In the latter case, it is >> assumed that the owner will "pop the hood" when it comes to the software >> configuration. > Ah, but they are. In many cases you can ship a product faster and cheaper with an ARM based system running a stripped down Linux and some specialty I/O than building a properly hardened custom microcontroller. That something has a CPU doesn't tell you whether it is a general purpose computer. What tells you if a device is a general purpose is whether it is intended for particular uses or not (the key word there being "purpose"). More importantly, if you view every Thing as a general purpose computer you are missing an opportunity to impose an engineering constraint on the problem space. If that in turn let's you easily solve for the general case, you've had a huge win. Eliot -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: OpenPGP digital signature URL: From nick at foobar.org Sat Oct 29 09:18:39 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 10:18:39 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21691.1477697805@segfault.tristatelogic.com> References: <21691.1477697805@segfault.tristatelogic.com> Message-ID: <5814696F.3060106@foobar.org> Ronald F. Guilmette wrote: > I always start with whatver whois.iana.org has to > say. And it says that that 103.0.0.0/8 belongs to APNIC, so of course, > I only looked at what whois.apnic.net had to say about 103.11.67.105. yeah, this prefix was transferred from APNIC to ARIN. You can search for the details here: https://www.apnic.net/manage-ip/manage-resources/transfer-resources/transfer-logs There's a full log on their ftp site: ftp://ftp.apnic.net/public/transfers/apnic/transfer-apnic-latest No doubt other RIRs have their own transfer listings. > This isn't the first time I've wished that the right hand knew (or cared) > what the left hand was doing. I've asked the folks at IANA about this > sort of thing in the past, i.e. them giving pointers to the apparently > wrong RiR whois server, and they just won't fix it. It's not an IANA problem to fix. IANA handles the initial allocation to the RIR, but does not account for subsequent inter-RIR transfers. There are 5 RIRs, so 20 different ways for data to flow, and IANA is no longer authoritative for the address space once its been RIR-allocated. This excludes ERX space, which is another bundle of fun. I.e. you should no longer depend on whois.iana.org for accurate resource delegation information. The LACNIC whois server (whois.lacnic.net) appears to maintain pointer information, judging by a couple of queries. Nick From nick at foobar.org Sat Oct 29 09:40:20 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 10:40:20 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21986.1477702316@segfault.tristatelogic.com> References: <21986.1477702316@segfault.tristatelogic.com> Message-ID: <58146E84.3030000@foobar.org> Ronald F. Guilmette wrote: > In my actual comment I merely noted that RIRs are in fact -not- the > Internet Police, and that none of them have ever displayed even the > slightest desire to become that (and indeed, when asked, they have, > without exception, exhibited a clear desire -not- to be assigned any > such role). just to be clear: this is a bottom up position, not top-down. The registry roles of the RIRs exist by mandate of the communities they serve to provide a database of integer allocations and assignments. If there's been no inclination to become "Internet Police", it's because their memberships do not want their respective RIRs to take on this role. > Given that I do not have an entirely unequivocal admiration for the > quality and consistancy of the work that RIRs are already clearly > responsible for, do you really believe that it would be my first > choice to assign an entirely seperate but equally critical set of > -new- authorities and responsibilities to the RiRs? This will, of course, vary between RIRs. At least in the RIPE NCC service region, all allocations and assignments by the RIPE NCC are covered by written contractual links and complete records of these contracts are kept by the organisation. Sub-assignments by LIRs may not be as accurate. Other RIR service regions will have different policies. > P.S. I may be wrong about this, but it has come to my attention that > many, most, or all of the WHOIS records reflecting allocations made by > the AFRINIC RIR are utterly devoid of either (a) information specifying > the dates on which the relevant allocations were made or (b) email > contact addresses for the relevant number resource registrants. Works fine for me. Did you use the "-B" flag when querying the Afrinic irrdb? % whois -h whois.afrinic.net " -B x.x.x.x" Nick From drc at virtualized.org Sat Oct 29 10:09:02 2016 From: drc at virtualized.org (David Conrad) Date: Sat, 29 Oct 2016 18:09:02 +0800 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5814696F.3060106@foobar.org> References: <21691.1477697805@segfault.tristatelogic.com> <5814696F.3060106@foobar.org> Message-ID: <51AE5C5B-CAEB-48BF-AE79-85A72BAC40FB@virtualized.org> On Oct 29, 2016, at 5:18 PM, Nick Hilliard wrote: > There > are 5 RIRs, so 20 different ways for data to flow, and IANA is no longer > authoritative for the address space once its been RIR-allocated. While true, hopefully referrals in RDAP will address the need to identify registration information down to the leaves. > I.e. you should no longer depend on whois.iana.org for accurate resource > delegation information. Well, it should be accurate at the top-level delegation (albeit, the IANA Whois server only deals with /8s). Regards, -drc (speaking only for myself) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From wesgeorge at puck.nether.net Sat Oct 29 12:40:42 2016 From: wesgeorge at puck.nether.net (Wesley George) Date: Sat, 29 Oct 2016 08:40:42 -0400 Subject: IPv6 automatic reverse DNS In-Reply-To: <8b3af23d52f4423997b7e4428f3b9b81@SC58MEXGP034.CORP.CHARTERCOM.com> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> <8b3af23d52f4423997b7e4428f3b9b81@SC58MEXGP034.CORP.CHARTERCOM.com> Message-ID: <208C7FE5-1D75-46F6-99AD-5CE2921FEEF3@puck.nether.net> > On Oct 28, 2016, at 11:03 PM, White, Andrew wrote: > > There are two competing drafts for synthetic rule-based PTR responses for IPv6 rDNS: > > Howard Lee, Time Warner Cable (now Charter) > https://tools.ietf.org/html/draft-howard-isp-ip6rdns-08 > > J. Woodworth, CenturyLink > https://datatracker.ietf.org/doc/draft-woodworth-bulk-rr/ > At the risk of getting into IETF administrivia, a little clarification is important here: The first draft you mention above was replaced by the draft I referenced in my previous email. It is currently an adopted WG draft in DNSOP, moving toward working group last call as a consensus document., thus the window for capturing and incorporating feedback is closing soon. The second document does not appear to be associated with any IETF Working Group yet, but it also isn't competing with the first document. The first draft is informational status, discussing the issues and considerations surrounding this problem, of which generating on-the-fly reverse records is one possible solution. The second draft is a proposed standard defining *how* to generate those on-the-fly reverse records assuming one decides that is the right path to take in one's network, and would dovetail nicely via reference to section 2.5 of isp-ip6-rdns. Wes George -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Andrew.White2 at charter.com Sat Oct 29 13:43:05 2016 From: Andrew.White2 at charter.com (White, Andrew) Date: Sat, 29 Oct 2016 13:43:05 +0000 Subject: IPv6 automatic reverse DNS In-Reply-To: <208C7FE5-1D75-46F6-99AD-5CE2921FEEF3@puck.nether.net> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> <8b3af23d52f4423997b7e4428f3b9b81@SC58MEXGP034.CORP.CHARTERCOM.com> <208C7FE5-1D75-46F6-99AD-5CE2921FEEF3@puck.nether.net> Message-ID: Thanks for the clarification, Wes. Has anyone proposed the method of publishing v6 PTRs on-the-fly as addresses are observed passing through an ISP's router? Andrew ??dr?w Whi?? Charter Network Operations - DAS DNS Desk: 314-394-9594 ? Cell: 314-452-4386 andrew.white2 at charter.com -----Original Message----- From: Wesley George [mailto:wesgeorge at puck.nether.net] Sent: Saturday, October 29, 2016 7:41 AM To: White, Andrew Cc: Steve Atkins; NANOG list Subject: Re: IPv6 automatic reverse DNS > On Oct 28, 2016, at 11:03 PM, White, Andrew wrote: > > There are two competing drafts for synthetic rule-based PTR responses for IPv6 rDNS: > > Howard Lee, Time Warner Cable (now Charter) > https://tools.ietf.org/html/draft-howard-isp-ip6rdns-08 > > J. Woodworth, CenturyLink > https://datatracker.ietf.org/doc/draft-woodworth-bulk-rr/ > At the risk of getting into IETF administrivia, a little clarification is important here: The first draft you mention above was replaced by the draft I referenced in my previous email. It is currently an adopted WG draft in DNSOP, moving toward working group last call as a consensus document., thus the window for capturing and incorporating feedback is closing soon. The second document does not appear to be associated with any IETF Working Group yet, but it also isn't competing with the first document. The first draft is informational status, discussing the issues and considerations surrounding this problem, of which generating on-the-fly reverse records is one possible solution. The second draft is a proposed standard defining *how* to generate those on-the-fly reverse records assuming one decides that is the right path to take in one's network, and would dovetail nicely via reference to section 2.5 of isp-ip6-rdns. Wes George From kmedcalf at dessus.com Sat Oct 29 15:44:55 2016 From: kmedcalf at dessus.com (Keith Medcalf) Date: Sat, 29 Oct 2016 09:44:55 -0600 Subject: IPv6 automatic reverse DNS In-Reply-To: <2EFB5469-E3E5-4E00-A4B3-88C5FE8AA00A@blighty.com> Message-ID: <6b851012c81e334e8ffdbfba7d488672@mail.dessus.com> On Friday, 28 October, 2016 19:37, Steve Atkins wrote: > > On Oct 28, 2016, at 6:04 PM, Karl Auer wrote: > >> 1b) anti spam filters believe in the magic of checking > >> forward/reverse match. > > Someone in this thread said that only malware-infested end-users are > > behind IP addresses with no reverse lookup. Well - no. As long as we > > keep telling anyone who isn't running a full-bore commercial network to > > "consume, be silent, die", we are holding everyone back, including > > ourselves. > If you send mail over IPv6 from an address with no reverse DNS you > will see quite a lot of this sort of thing: > 550 5.7.1 [*] Our system has detected that this message > 5.7.1 does not meet IPv6 sending guidelines regarding PTR records and > 5.7.1 authentication. Please review > 5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for > more > 5.7.1 information. > > It's fine to use no-reverse-lookup as a component of a spamminess > > score. It's not OK to use it as proof of spamminess. > People running large mailservers made that decision some time > ago. Disagreeing with them won't make them accept your email. Actually, it was *long* before that. I think it is STD 1 or STD 2 -- requirements for connecting a host to the internet. All "deliberate" Internet hosts performing useful functions should have matching forward and reverse DNS and should expect to be labelled as "untrustworthy in the extreme" if they do not. Assigning meaning to the resolved DNS name (embeded parts) is what came much later. From bzs at TheWorld.com Sat Oct 29 17:27:48 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sat, 29 Oct 2016 13:27:48 -0400 Subject: Spitballing IoT Security In-Reply-To: <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> Message-ID: <22548.56340.234379.456600@gargle.gargle.HOWL> On October 28, 2016 at 22:27 list at satchell.net (Stephen Satchell) wrote: > On 10/28/2016 10:14 PM, bzs at TheWorld.com wrote: > > Thus far the goal just seems to be mayhem. > > Thus far, the goal on the part of the botnet opearators is to make > money. The goal of the CUSTOMERS of the botnet operators? Who knows? You're speaking in general terms, right? We don't know much anything about the perpetrators of these recent Krebs and Dyn attacks such as whether there was any DDoS for hire involved. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From esr at thyrsus.com Sat Oct 29 18:07:30 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Sat, 29 Oct 2016 14:07:30 -0400 Subject: Spitballing IoT Security In-Reply-To: <22548.56340.234379.456600@gargle.gargle.HOWL> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> <22548.56340.234379.456600@gargle.gargle.HOWL> Message-ID: <20161029180730.GA10801@thyrsus.com> bzs at TheWorld.com : > > On October 28, 2016 at 22:27 list at satchell.net (Stephen Satchell) wrote: > > On 10/28/2016 10:14 PM, bzs at TheWorld.com wrote: > > > Thus far the goal just seems to be mayhem. > > > > Thus far, the goal on the part of the botnet opearators is to make > > money. The goal of the CUSTOMERS of the botnet operators? Who knows? > > You're speaking in general terms, right? We don't know much anything > about the perpetrators of these recent Krebs and Dyn attacks such as > whether there was any DDoS for hire involved. We can deduce a lot from what didn't happen. You don't build or hire a botnet on Mirai's scale with pocket change. And the M.O. doesn't fit a criminal organization - no ransom demand, no attempt to steal data. That means the motive was prep for terrorism or cyberwar by a state-level actor. Bruce Schneier is right and is only saying what everybody else on the InfoSec side I've spoken with is thinking - the People's Liberation Army is the top suspect, with the Russian FSB operating through proxies in Bulgaria or Romania as a fairly distant second. Me, I think this fits the profile of a PLA probing attack perfectly. -- Eric S. Raymond From bzs at TheWorld.com Sat Oct 29 18:31:05 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sat, 29 Oct 2016 14:31:05 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161029180730.GA10801@thyrsus.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> <22548.56340.234379.456600@gargle.gargle.HOWL> <20161029180730.GA10801@thyrsus.com> Message-ID: <22548.60137.738949.850905@gargle.gargle.HOWL> On October 29, 2016 at 14:07 esr at thyrsus.com (Eric S. Raymond) wrote: > bzs at TheWorld.com : > > > > On October 28, 2016 at 22:27 list at satchell.net (Stephen Satchell) wrote: > > > On 10/28/2016 10:14 PM, bzs at TheWorld.com wrote: > > > > Thus far the goal just seems to be mayhem. > > > > > > Thus far, the goal on the part of the botnet opearators is to make > > > money. The goal of the CUSTOMERS of the botnet operators? Who knows? > > > > You're speaking in general terms, right? We don't know much anything > > about the perpetrators of these recent Krebs and Dyn attacks such as > > whether there was any DDoS for hire involved. > > We can deduce a lot from what didn't happen. > > You don't build or hire a botnet on Mirai's scale with pocket change. Do we know this or is this just a guess? The infamous 1988 Morris worm was also thought to be something similarly sinister for a short while until Bob Morris, Jr et al owned up to it just being an experiment by a couple of students gone out of control. Back around 1986 I accidentally brought down at least half the net by submitting a new hosts file (for Boston Univ) with an entry that tickled a bug in the hosts.txt->/etc/hosts code which everyone ran at midnight (whatever) causing a loop which filled /tmp (this would be unix hosts but by count they were by far most of the connected servers) and back then a full /tmp crashed unix and it often didn't come back up until a human intervened. Ok I doubt this was an accident, tho its scale could've been an accident, a prank gone wild. Anyhow what do we *know*? That the effect was large doesn't necessarily imply that it required a lot of resources. We live in a world rife with asymmetric warfare. A few boxcutters and 3,000+ people dead. > And the M.O. doesn't fit a criminal organization - no ransom demand, > no attempt to steal data. Same question. Would Dyn et al publicize ransom demands at this point? And even if not how do we rule out a prank or similar? Is there something specific about this attack which required significant resources? How significant? > > That means the motive was prep for terrorism or cyberwar by a > state-level actor. Bruce Schneier is right and is only saying what > everybody else on the InfoSec side I've spoken with is thinking - the > People's Liberation Army is the top suspect, with the Russian FSB > operating through proxies in Bulgaria or Romania as a fairly distant > second. Well, barring further details one can go anywhere with a few suppositions. > > Me, I think this fits the profile of a PLA probing attack perfectly. > -- > Eric S. Raymond -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From rfg at tristatelogic.com Sat Oct 29 18:42:27 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 11:42:27 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5814696F.3060106@foobar.org> Message-ID: <25352.1477766547@segfault.tristatelogic.com> In message <5814696F.3060106 at foobar.org>, Nick Hilliard wrote: >Ronald F. Guilmette wrote: >> I always start with whatver whois.iana.org has to >> say. And it says that that 103.0.0.0/8 belongs to APNIC, so of course, >> I only looked at what whois.apnic.net had to say about 103.11.67.105. > >yeah, this prefix was transferred from APNIC to ARIN. You can search >for the details here: > >https://www.apnic.net/manage-ip/manage-resources/transfer-resources/transfer-logs Oh, geeeezzzzz! ... Showing 1 to 10 of 1,823 entries >> This isn't the first time I've wished that the right hand knew (or cared) >> what the left hand was doing. I've asked the folks at IANA about this >> sort of thing in the past, i.e. them giving pointers to the apparently >> wrong RiR whois server, and they just won't fix it. > >It's not an IANA problem to fix. IANA handles the initial allocation... You are correct. In this case, it would have been helpful if APNIC's WHOIS server returned something, when queried about 103.11.67.105, that would include an explicit referral to the ARIN WHOIS server. I mean they obviously know all the transfers they've made. But I guess that somebody somwhere decided that that's just too much trouble. Regards, rfg From jfmezei_nanog at vaxination.ca Sat Oct 29 18:48:50 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sat, 29 Oct 2016 14:48:50 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161029180730.GA10801@thyrsus.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> <22548.56340.234379.456600@gargle.gargle.HOWL> <20161029180730.GA10801@thyrsus.com> Message-ID: <5814EF12.3080706@vaxination.ca> On 2016-10-29 14:07, Eric S. Raymond wrote: > You don't build or hire a botnet on Mirai's scale with pocket change. > And the M.O. doesn't fit a criminal organization - no ransom demand, > no attempt to steal data. it is wrong to underestimate script kiddies and open source code. It is wrong to underestimate a community that shares their own experiences with different devices. One contributes default password for brand X camera, one gives the defaults for brand Y router etc. Imagine someone writes code for university project to scan the network for improperly protected devices. That code, while designed as a security audit, could be integrated into something far nastier. At the end of the day, you may have plenty of open source information available to assemble this into something like Mirai. Yeah, there may be more sinister forces out there. The DYN attack may have been a "demo" of capabilities that will be part of threats/balckmail against other large players on the Internet. > everybody else on the InfoSec side I've spoken with is thinking - the > People's Liberation Army is the top suspect, with the Russian FSB > operating through proxies in Bulgaria or Romania as a fairly distant > second. Or some guy in Arkansas starting a new blackmail/extortion business, hoping to cash in on the software he put together. And if we're gonna talk conspiracies, include Trump. he publishes a "policy" on cyber attacks on a day, a couple days later a major cyber attack happens. Coincidence ? :-) I think the focus should be on preventing such attacks, and reducing their impacts when they happen and improving traceability tools as they happen. Speculating on who is reponsible doesn't do much to proect the internet against such attacks. From rfg at tristatelogic.com Sat Oct 29 19:02:40 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 12:02:40 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <58146E84.3030000@foobar.org> Message-ID: <25432.1477767760@segfault.tristatelogic.com> In message <58146E84.3030000 at foobar.org>, Nick Hilliard wrote: >> P.S. I may be wrong about this, but it has come to my attention that >> many, most, or all of the WHOIS records reflecting allocations made by >> the AFRINIC RIR are utterly devoid of either (a) information specifying >> the dates on which the relevant allocations were made or (b) email >> contact addresses for the relevant number resource registrants. > >Works fine for me. Did you use the "-B" flag when querying the Afrinic >irrdb? I wasn't talking about irrdb. I was just talking about the WHOIS records for IPv4 allocations within the AFRINIC region. Anyway, yes, I do believe that used the -B flag. But nontheless, I really did see some AFRINIC WHOIS records that had -no- email contacts, nor any date information. I will have to try to see if I can dredge those out again. But my overall point remains. If there were ever to be an election where we were all asked who we wanted to see become the once and future Routing Police, the RIRs would not be my own personal first choice. Regards, rfg From beecher at beecher.cc Sat Oct 29 19:35:15 2016 From: beecher at beecher.cc (Tom Beecher) Date: Sat, 29 Oct 2016 15:35:15 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161029180730.GA10801@thyrsus.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> <22548.56340.234379.456600@gargle.gargle.HOWL> <20161029180730.GA10801@thyrsus.com> Message-ID: "That means the motive was prep for terrorism or cyberwar by a state-level actor. " Or, quite possibly ( I would argue probably) it was marketing. Show off the capabilities of the botnet to garner more interest amongst those who pay for use of such things. On Sat, Oct 29, 2016 at 2:07 PM, Eric S. Raymond wrote: > bzs at TheWorld.com : > > > > On October 28, 2016 at 22:27 list at satchell.net (Stephen Satchell) wrote: > > > On 10/28/2016 10:14 PM, bzs at TheWorld.com wrote: > > > > Thus far the goal just seems to be mayhem. > > > > > > Thus far, the goal on the part of the botnet opearators is to make > > > money. The goal of the CUSTOMERS of the botnet operators? Who knows? > > > > You're speaking in general terms, right? We don't know much anything > > about the perpetrators of these recent Krebs and Dyn attacks such as > > whether there was any DDoS for hire involved. > > We can deduce a lot from what didn't happen. > > You don't build or hire a botnet on Mirai's scale with pocket change. > And the M.O. doesn't fit a criminal organization - no ransom demand, > no attempt to steal data. > > That means the motive was prep for terrorism or cyberwar by a > state-level actor. Bruce Schneier is right and is only saying what > everybody else on the InfoSec side I've spoken with is thinking - the > People's Liberation Army is the top suspect, with the Russian FSB > operating through proxies in Bulgaria or Romania as a fairly distant > second. > > Me, I think this fits the profile of a PLA probing attack perfectly. > -- > Eric S. Raymond > From bzs at TheWorld.com Sat Oct 29 19:54:35 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sat, 29 Oct 2016 15:54:35 -0400 Subject: Spitballing IoT Security In-Reply-To: References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <22548.12346.733412.473362@gargle.gargle.HOWL> <0de550e8-5196-a80a-c311-e2d555731ad9@satchell.net> <22548.56340.234379.456600@gargle.gargle.HOWL> <20161029180730.GA10801@thyrsus.com> Message-ID: <22548.65147.319577.701287@gargle.gargle.HOWL> On October 29, 2016 at 15:35 beecher at beecher.cc (Tom Beecher) wrote: > "That means the motive was prep for terrorism or cyberwar by a > state-level actor. " > > Or, quite possibly ( I would argue probably) it was marketing. Show off the > capabilities of the botnet to garner more interest amongst those who pay for > use of such things.? Supposedly Khalid Sheikh Mohammed's widely publicized video of him beheading Daniel Pearl was basically an ad for his group's for-hire mercenary services. Look at how ruthless we are! Didn't seem to lead to much of a career. However the one fly in this ointment is that the Mirai virus code has since been distributed. So unless there's some critical piece of that they've held back it's not much of a property. If that's true, that the virus code was subsequently distributed by the actors, it also raises doubts about a state actor. Why would they distribute the code? State actors tend to work in an atmosphere of secrecy unless they're flaunting their deeds. But, whatever, one doesn't know until one knows. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From nick at foobar.org Sat Oct 29 20:06:23 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 21:06:23 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <25432.1477767760@segfault.tristatelogic.com> References: <25432.1477767760@segfault.tristatelogic.com> Message-ID: <5815013F.2080502@foobar.org> Ronald F. Guilmette wrote: > I wasn't talking about irrdb. I was just talking about the WHOIS records > for IPv4 allocations within the AFRINIC region. afrinic, ripe ncc and apnic run a combined (+ partially authenticated) irrdb and whois server. > But my overall point remains. If there were ever to be an election where > we were all asked who we wanted to see become the once and future Routing > Police, the RIRs would not be my own personal first choice. Great, we're agreed then. So why do you keep on bringing them up in this context and criticising them whenever someone squats some block of address space? Nick From nick at foobar.org Sat Oct 29 20:28:35 2016 From: nick at foobar.org (Nick Hilliard) Date: Sat, 29 Oct 2016 21:28:35 +0100 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <25352.1477766547@segfault.tristatelogic.com> References: <25352.1477766547@segfault.tristatelogic.com> Message-ID: <58150673.5090201@foobar.org> Ronald F. Guilmette wrote: > Oh, geeeezzzzz! ... > > Showing 1 to 10 of 1,823 entries Yeah, get over it. Number resource transfers are a thing, and this number is only going to increase. > You are correct. In this case, it would have been helpful if APNIC's WHOIS > server returned something, when queried about 103.11.67.105, that would > include an explicit referral to the ARIN WHOIS server. I mean they > obviously know all the transfers they've made. > > But I guess that somebody somwhere decided that that's just too much > trouble. David Conrad already pointed out that this problem has been solved using RDAP which supports referrals. Try installing the nicinfo command from: https://github.com/arineng/nicinfo At a guess, I'd say referrals haven't been implemented in whois because the whois "protocol" is unfixably broken and unsuitable for distributed information sharing. Nick From A.L.M.Buxey at lboro.ac.uk Sat Oct 29 20:41:02 2016 From: A.L.M.Buxey at lboro.ac.uk (Alan Buxey) Date: Sat, 29 Oct 2016 21:41:02 +0100 Subject: Spitballing IoT Security In-Reply-To: <68ea3900-1d2f-507c-1192-6c43fe1a5f1f@ofcourseimright.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <68ea3900-1d2f-507c-1192-6c43fe1a5f1f@ofcourseimright.com> Message-ID: <0E2F9A27-D728-4F60-9748-1A62601476A9@lboro.ac.uk> Hi, Hi, >Put it another way: you bring home a NEST and the first thing you the >expert might do is read the net to figure out which ports to open. Are >you really going to not open those ports? Put onto its own isolated vlan with only internet access. Unfortunately no basic routers that are for the home come with such a setup by default. That's the first big win. alan From rfg at tristatelogic.com Sun Oct 30 00:32:19 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 17:32:19 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161029180730.GA10801@thyrsus.com> Message-ID: <26234.1477787539@segfault.tristatelogic.com> In message <20161029180730.GA10801 at thyrsus.com>, "Eric S. Raymond" wrote: >You don't build or hire a botnet on Mirai's scale with pocket change. Proof please? Sorry, but I am compelled to call B.S. on the above statement. This is a really important point that I, Krebs, and others have been trying to drive home: In an era when you've got a half million CCTV cams just lying around without even passwords on them, and in an era when nobody makes any fuss anymore about the dozens or hundreds or people and/or organizations (e.g. Shodan) that are out there scanning your box and my box and everybody's boxes, every damn day, you don't need to be either an omnious "state actor" or even SPECTER to assemble a truly massive packet weapon. Two kids with a modest amount of knowledge and a lot of time on their hands can do it from their mom's basement. It is comforting, for some, to think that this is not the case, just as it is, to this day, comforting, for some, to believe, based on scant evidence, that it -wasn't- just some lone nut case who killed President Kennedy. Psychologically, people have trouble coming to terms with great impactful tragedies unless they can be blamed on large, unseen, but enormously capable dark forces. And the actual available hard evidence relating to such events does not diminish the human yearning for a convenient comic book supervillain to pin it all on. >And the M.O. doesn't fit a criminal organization - no ransom demand, >no attempt to steal data. Allow me to refer you to an alternative possible motivation: https://en.wiktionary.org/wiki/lulz >That means the motive was prep for terrorism or cyberwar by a >state-level actor. Frankly, I am dismayed to see a well-known Internet persona with a respected name spreading this kind of absurd, alarmist, over-the-top, retorical fear- mongering inference, which is without clear basis in either fact or evidence. Even the hardest of the hard-core dyed-in-the-wool Clinton surrogates are too circumspect in their pronouncements (i.e. with respect to Russia's "obvious" connection to the DNC hack) to ever reach anything like this level of unfounded hyperbole. (And for the record, I am no Trump supporter either. I find myself equally disgusted when either side employs the currently fashionable verbal sleight-of-hand that politicians of all stripes have, of late, adopted whenever they want to say something without themselves having to take responsibility for its truth or accuracy. I get angry when I hear Clinton surrogates using the "Some people are saying..." dodge, e.g. when it comes to alleged nefarious Russian involvement with anything and everything evil, just as I do when Trump uses the exact same dodge in reference to... well... everything.) >Bruce Schneier is right and is only saying what >everybody else on the InfoSec side I've spoken with is thinking - the >People's Liberation Army is the top suspect, with the Russian FSB >operating through proxies in Bulgaria or Romania as a fairly distant >second. Yes, but I believe that Schneier was a bit more careful to separate the known facts from his personal speculations. In any case, all of this searching for who is to blame isn't contributing a damn thing towards actually fixing the problem. And if we really need to find someone to blame, I think we should all just look in the mirror. We, society, but especially those of us with more-than-average techno savvy, have for years been only too eager to lap up whatever whiz-bang new techno gadgets industry could crank out, with barely an afterthought given to the longer term implications, like security and also how the hell we are ever going to be able to recycle any of this s***. We've all been doing the exact same thing, since at least Windows 3.1 or earlier, and yet we continue to expect a different outcome. We eagerly grab for new capabilities and new gadgets, thinking about security last or, more often, not at all. In short, to quote Pogo, "We have met the enemy and he is us." Regards, rfg P.S. Even if the evidence were to support the view that only a superpower- level nation-state could have pulled off the Dyn attack... and I'm not at all persuaded that it does... it kills me that everyone seems to jump, within a millisecond, immediately from -that- unwarranted conclusion to the separate unwarranted conclusion that it must have been either Russia or China. Apparently, nobody even stops to consider the *other* elephant in the room, the one that stretches from sea to shining sea, and which itself has been heard to publically brag about its own cyber-offensive capabilities of late. In short, maybe our own guys did this. OK, so maybe this theory -is- worthy of le Carre, but that don't mean it ain't possible. I mean we aren't stupid. We don't build warehouses full of nuclear weapons without at least testing the design once or twice first, you know, to make sure they aren't all gonna end up being duds on impact. (Mike Rogers would probably lose his stripes -and- his pension if an actual cyber-confrontation came and it was revealed that nobody had ever actually tested any of our theoretical capabilities.) And when we do test our strategic weapons, we -don't- test them by dropping one on China, or Russia, or Iran, and then saying "Oh! Sorry. Please excuse us. Just testing." Doing that could come with consequences. So, what's worse? That Amazon and Twitter should be offline for a couple of hours in the middle of October (i.e. for a little test) or that any one of our many enemies should, you know, maybe take them down for days on end in early December, at the height of the shopping season, with us having no real/tested retaliatory capability? (Nutty conspiracy theorists might even suggest that staging a limited attack like this is a rather obvious way for certain three letter entities and/or parts of DoD to squeeze even more out of congress than the obscenely vast sums they are already getting, but I personally won't even go there. As I've noted, there are plenty of pragmatic and entirely non-nefarious/non-self-serving reasons why our own guys might have done a small/short practice run like this.) Anyway, when it comes to attribution, the bottom line is that all anybody has to do is to run their C&C through two or three levels of chained compromised socks proxies, e.g. in Tajikistan, Bolivia, and Singapore, and then, as a practical matter, nobody will ever be able to say for sure who you are. It's all just guesswork, and much of it, alas, isn't even all that educated. Who says that the Russians or the Chinese took down Dyn? Are these the same people who told us the fantasy... later retracted... that North Korea hacked Sony? Are these the same people who told us that Saddam absolutely positively had weapons of mass destruction? I would have hoped that all of us in this country (US) would have become just a little bit more skeptical of press reports and "expert" pronouncements by now. Remember the Maine! From rfg at tristatelogic.com Sun Oct 30 02:45:29 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 19:45:29 -0700 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5815013F.2080502@foobar.org> Message-ID: <26676.1477795529@segfault.tristatelogic.com> In message <5815013F.2080502 at foobar.org>, Nick Hilliard wrote: >> But my overall point remains. If there were ever to be an election where >> we were all asked who we wanted to see become the once and future Routing >> Police, the RIRs would not be my own personal first choice. > >Great, we're agreed then. So why do you keep on bringing them up in >this context and criticising them whenever someone squats some block of >address space? References please? *I* didn't introduce the topic of RIRs into this thread. It would appear that Ken Chase did that: http://mailman.nanog.org/pipermail/nanog/2016-October/088943.html Later on, I bemoaned what I still feel is a rather lousey WHOIS referrals system, among and between the various RIR WHOIS data bases... with respect to *allocations* (not route registrations)... and it was entirely appropriate for me to mention that, in this thread, as the problem most definitely did impact not only _my_ ability to figure out who the bleep, if anyone, 103.11.67.0/24 is actually registered to, but actually, anyone's ability to do so, including, apparently, bgp.he.net. But this criticism has/had nothing whatever to do, specifically, with either routing or the (hypothetical) Routing Police. If the totality of the RIR WHOIS data bases are needlessly difficult to extract accurate information out of, then this negatively affects *all* uses (and all users) of these data bases, whether one is investigating possible routing squats, or whether one is just trying to figure out who currently owns the block that all of your corporate intellectual property has just been surreptitiously exfiltrated to. Regards, rfg From rfg at tristatelogic.com Sun Oct 30 03:05:51 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 20:05:51 -0700 Subject: Death of WHOIS, Film at 11 In-Reply-To: <58150673.5090201@foobar.org> Message-ID: <26768.1477796751@segfault.tristatelogic.com> In message <58150673.5090201 at foobar.org>, Nick Hilliard wrote: >David Conrad already pointed out that this problem has been solved using >RDAP which supports referrals. Try installing the nicinfo command from: > >https://github.com/arineng/nicinfo > >At a guess, I'd say referrals haven't been implemented in whois because >the whois "protocol" is unfixably broken and unsuitable for distributed >information sharing. So basically, you're saying that the fact that port 43 is still open and still providing answers... known inaccurante answers... at all of the following places is just one big tease? whois.iana.org whois.arin.net whois.ripe.net whois.apnic.net whois.lacnic.net whois.afrinic.net So the overall game plan is to continue to have these things all give out inaccurate and/or misleanding answers until such time as all of the trusting old school hacks like me either die out or get the memo telling us to just stop using this stuff? If so, thanks for telling me. Nobody else has so far had the courtesy to do so. Regards, rfg P.S. Traditional WHOIS supports referrals. For an example, try this: whois -h whois.iana.org 1197 (Providing referrals in traditional WHOIS isn't exactly rocket surgery. The fact that certain RIRs may be too... umm... preoccupied to take the time to properly populate their data bases with such referrals notwithstanding.) From esr at thyrsus.com Sun Oct 30 04:43:42 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Sun, 30 Oct 2016 00:43:42 -0400 Subject: Spitballing IoT Security In-Reply-To: <26234.1477787539@segfault.tristatelogic.com> References: <20161029180730.GA10801@thyrsus.com> <26234.1477787539@segfault.tristatelogic.com> Message-ID: <20161030044342.GA18488@thyrsus.com> Ronald F. Guilmette : > Two kids with a modest amount of knowledge > and a lot of time on their hands can do it from their mom's basement. I in turn have to call BS on this. If it were really that easy, we'd be inundated by Mirais -- we'd have several attacks a *day*. -- Eric S. Raymond From rfg at tristatelogic.com Sun Oct 30 05:19:18 2016 From: rfg at tristatelogic.com (Ronald F. Guilmette) Date: Sat, 29 Oct 2016 22:19:18 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161030044342.GA18488@thyrsus.com> Message-ID: <27204.1477804758@segfault.tristatelogic.com> In message <20161030044342.GA18488 at thyrsus.com>, "Eric S. Raymond" wrote: >Ronald F. Guilmette : >> Two kids with a modest amount of knowledge >> and a lot of time on their hands can do it from their mom's basement. > >I in turn have to call BS on this. If it were really that easy, we'd >be inundated by Mirais -- we'd have several attacks a *day*. You need to get out more. http://www.nab.org/cybersecurity/Verisign-report-ddos-trends-Q22016.pdf It *is* happening every day. You just don't hear about it on CNN because a "little" 80Mbps DDoS isn't even worthy of a headline anymore, even though such an attack could CRUSH a local bank, and even many regional banks into utter oblivion. Now, where did I put those bitcoins... It's ransom time! Regards, rfg P.S. Of course, things were oh so much better, ya know, back in those idylic halcyon days a decade and a half ago... Denial of e-commerce Feb 10th 2000 http://www.economist.com/node/281531 "... The Computer Emergency Response Team of Carnegie Mellon University hears of roughly four DOS attacks a day..." Whew! I guess we need to count our blessings that insightful visionary industry leaders came forward, back in the early 00s, and spearheaded the global changes necessary to insure that DDoS attacks would become a thing of the past, and a distant memory. Oh! Wait! Nevermind. Sorry. I guess that I was dozing off and dreaming again. At the current rate of progress I think that I can confidently predict that the Internet industry ought to have this whole problem completely licked by the early 23rd century, you know, at the very latest. From esr at thyrsus.com Sun Oct 30 05:59:05 2016 From: esr at thyrsus.com (Eric S. Raymond) Date: Sun, 30 Oct 2016 01:59:05 -0400 Subject: Spitballing IoT Security In-Reply-To: <27204.1477804758@segfault.tristatelogic.com> References: <20161030044342.GA18488@thyrsus.com> <27204.1477804758@segfault.tristatelogic.com> Message-ID: <20161030055905.GA20998@thyrsus.com> Ronald F. Guilmette : > > In message <20161030044342.GA18488 at thyrsus.com>, > "Eric S. Raymond" wrote: > > >Ronald F. Guilmette : > >> Two kids with a modest amount of knowledge > >> and a lot of time on their hands can do it from their mom's basement. > > > >I in turn have to call BS on this. If it were really that easy, we'd > >be inundated by Mirais -- we'd have several attacks a *day*. > > > You need to get out more. > > http://www.nab.org/cybersecurity/Verisign-report-ddos-trends-Q22016.pdf > > It *is* happening every day. You just don't hear about it on CNN because > a "little" 80Mbps DDoS isn't even worthy of a headline anymore, even > though such an attack could CRUSH a local bank, and even many regional > banks into utter oblivion. > > Now, where did I put those bitcoins... It's ransom time! Don't change the subject. An effective DDoS against any single site is, though concerning, not a Mirai-class event. The difference matters, and you shouldn't be pretending it does to score rhetorical points. -- Eric S. Raymond From jw at nuclearfallout.net Sun Oct 30 06:16:26 2016 From: jw at nuclearfallout.net (John Weekes) Date: Sat, 29 Oct 2016 23:16:26 -0700 Subject: Spitballing IoT Security In-Reply-To: <20161030044342.GA18488@thyrsus.com> References: <20161029180730.GA10801@thyrsus.com> <26234.1477787539@segfault.tristatelogic.com> <20161030044342.GA18488@thyrsus.com> Message-ID: <068fc18f-4fc4-b7ff-193e-c4e79317cb20@nuclearfallout.net> On 10/29/2016 9:43 PM, Eric S. Raymond wrote: > I in turn have to call BS on this. If it were really that easy, we'd > be inundated by Mirais -- we'd have several attacks a*day*. Some of us are seeing many significant attacks a day. That's because botnets are frequently used to hit game servers and game players. In fact, the Mirai-targeted devices were not newly-seen; easily-exploited devices like older DVRs have been observed for years in attacks on game servers. The main difference in the recent botnet attacks (mostly, 2016) is that they have been larger and more frequent, likely because of incremental improvements to scanners (including in time-to-exploitation, which is important to building the botnet because these devices are so frequently rebooted) and payloads (to better block further exploitation by competitors). If you run a honeypot and take a look at what happens to one of these devices over time, you'll see an interesting tug-of-war between many different actors that are compromising them and running their own binaries. Reflection attacks are still common, as well, of course. Previously, those were the ones that created the largest flows. But, the higher-amplification-factor reflection attacks can be mostly mitigated upstream with basic ACLs (as long as the upstream is willing to help, and has the internal capacity to do it; many NSPs do not). It is not uncommon to see a botnet attack at the same time as a reflection attack. -John From nick at foobar.org Sun Oct 30 09:33:00 2016 From: nick at foobar.org (Nick Hilliard) Date: Sun, 30 Oct 2016 09:33:00 +0000 Subject: Death of WHOIS, Film at 11 In-Reply-To: <26768.1477796751@segfault.tristatelogic.com> References: <26768.1477796751@segfault.tristatelogic.com> Message-ID: <5815BE4C.2070305@foobar.org> Ronald F. Guilmette wrote: > So the overall game plan is to continue to have these things all give > out inaccurate and/or misleanding answers until such time as all of > the trusting old school hacks like me either die out or get the memo > telling us to just stop using this stuff? no, it's to move to a protocol which is fit for purpose, which "whois" is not. Nick From rsk at gsp.org Sun Oct 30 11:35:06 2016 From: rsk at gsp.org (Rich Kulawiec) Date: Sun, 30 Oct 2016 07:35:06 -0400 Subject: Spitballing IoT Security In-Reply-To: <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> Message-ID: <20161030113505.GA7295@gsp.org> On Fri, Oct 28, 2016 at 12:07:17AM -0500, Jim Hickstein wrote: > A virus that kills its host (too much of the time) is not successful. True. On the other hand: "Some men aren't looking for anything logical, like money. They can't be bought, bullied, reasoned, or negotiated with. Some men just want to watch the world burn." I have no doubt whatsoever that some of our adversaries fall squarely into this category. ---rsk From suzworldwide at gmail.com Sun Oct 30 13:47:56 2016 From: suzworldwide at gmail.com (Suzanne Woolf) Date: Sun, 30 Oct 2016 09:47:56 -0400 Subject: IPv6 automatic reverse DNS In-Reply-To: <208C7FE5-1D75-46F6-99AD-5CE2921FEEF3@puck.nether.net> References: <83b75e68-92dc-0fd4-3036-18d751f45715@gmail.com> <7987F35C-1D05-49C1-BB38-1CAD7788D180@blighty.com> <8b3af23d52f4423997b7e4428f3b9b81@SC58MEXGP034.CORP.CHARTERCOM.com> <208C7FE5-1D75-46F6-99AD-5CE2921FEEF3@puck.nether.net> Message-ID: <36FE1C10-4DDA-4A54-BE5B-CE18C185B7A2@gmail.com> Hi Wes, > On Oct 29, 2016, at 8:40 AM, Wesley George wrote: > > >> On Oct 28, 2016, at 11:03 PM, White, Andrew wrote: >> >> There are two competing drafts for synthetic rule-based PTR responses for IPv6 rDNS: >> >> Howard Lee, Time Warner Cable (now Charter) >> https://tools.ietf.org/html/draft-howard-isp-ip6rdns-08 >> >> J. Woodworth, CenturyLink >> https://datatracker.ietf.org/doc/draft-woodworth-bulk-rr/ >> > > At the risk of getting into IETF administrivia, a little clarification is important here: The first draft you mention above was replaced by the draft I referenced in my previous email. It is currently an adopted WG draft in DNSOP, moving toward working group last call as a consensus document., thus the window for capturing and incorporating feedback is closing soon. The second document does not appear to be associated with any IETF Working Group yet, but it also isn't competing with the first document. The first draft is informational status, discussing the issues and considerations surrounding this problem, of which generating on-the-fly reverse records is one possible solution. The second draft is a proposed standard defining *how* to generate those on-the-fly reverse records assuming one decides that is the right path to take in one's network, and would dovetail nicely via reference to section 2.5 of isp-ip6-rdns. This is exactly right, and thanks for the clear explanation of arcane IETF process?. Comments on https://www.ietf.org/id/draft-ietf-dnsop-isp-ip6rdns-02.txt can go to Lee or the WG mailing list, dnsop at ietf.org . We?re trying to make it useful for operators, so having operators comment is *really* good?. The WG felt quite strongly that the document shouldn?t be prescriptive as far as telling people they *should* do this, only some of the considerations about doing it if they wish to. John Woodworth?s bulk-rr document was discussed in the WG in the last IETF meeting (Berlin in July) and got enough interest that John was planning to keep working on it. It needs people committed to active review and discussion on it to become a WG document, which he hasn?t requested (yet), but if the idea seems useful to you, you should tell him. best, Suzanne (DNSOP co-chair, but not speaking for the WG or anyone else?.) From jxh at jxh.com Sun Oct 30 17:30:03 2016 From: jxh at jxh.com (Jim Hickstein) Date: Sun, 30 Oct 2016 12:30:03 -0500 Subject: Spitballing IoT Security In-Reply-To: <20161030113505.GA7295@gsp.org> References: <331654658bda924ab06f3eb56f62145c@mail.dessus.com> <22546.52494.516440.706525@gargle.gargle.HOWL> <161d4b6d-5529-a537-3153-27db58ce5e8e@jxh.com> <20161030113505.GA7295@gsp.org> Message-ID: <0b8b56ba-dcc4-478f-3843-76e5c1f15ebf@jxh.com> On 10/30/16 06:35, Rich Kulawiec wrote: > On Fri, Oct 28, 2016 at 12:07:17AM -0500, Jim Hickstein wrote: >> A virus that kills its host (too much of the time) is not successful. > > True. On the other hand: > > "Some men aren't looking for anything logical, like money. > They can't be bought, bullied, reasoned, or negotiated with. > Some men just want to watch the world burn." > > I have no doubt whatsoever that some of our adversaries fall squarely > into this category. i.e. vandalism. Agreed, and the respondent who brought up rational actors has pointed out where the computer "virus" analogy breaks down: biological viruses have no rational actor and no premeditated goal. Their success, as usually defined (maximum incidence in a population), emerges from the mathematics of their operation. DDoS attacks are ultimately caused by humans (so far) and while we may not know clearly their goals or the values that underlie them, they exist. This would seem to call for a different response. I wish I knew what. From jfmezei_nanog at vaxination.ca Sun Oct 30 17:52:47 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sun, 30 Oct 2016 13:52:47 -0400 Subject: FYI: Net Neutrality in Canada Message-ID: <5816336F.5050103@vaxination.ca> This is a heads up, the CRTC (Canada's FCC) is holding a week long hearing on net neutrality in Canada ("differential pricing" is the used). Canada has had its "ITMP" (Internet Traffic Management Practices) policy since 2009 which deals with unfair throttling, and now, we are arguing on zero rating and sponsored content stuff). It will be broadcasted at http://www.cpac.ca (at bottom of home page there should be a selection for the CRTC hearing). Note: you can choose between english, french of floor (untranslated). Days generally start at 09:00. Can end at any time. Either @CRTCeng or @CRTChearings will be tweeting links to presentations as each presentation begins. hashtag: #CRTC #Diffpricing Facebook did not wish to appear but was "invited". Last time the CRTC did that, it was with Netflix and Google and sparks flew ("you don't regulate us, we don't have to answer"). (It appears on Tuesday right after me, so they should be roughly ~ 10:30 or 11:00.) The agenda: http://www.crtc.gc.ca/Telecom/eng/HEARINGS/2016/ag31_10.htm The original Notice of Consultation: > http://crtc.gc.ca/eng/archive/2016/2016-192.htm?_ga=1.136052530.24154879.1433393531 And the record of the consultation: > https://services.crtc.gc.ca/pub/instances-proceedings/Default-defaut.aspx?EN=2016-192&Lang=eng&_ga=1.136052530.24154879.1433393531 Note: this started with a different proceeding in September 2015 2 parts 1 filings against Vid?otron who started zero rated music on its wireless service for music services that Vid?otron approved/selected. > https://services.crtc.gc.ca/pub/instances-proceedings/Default-Defaut.aspx?lang=eng&YA=2015&S=C&PA=t&PT=pt1&PST=a#201510735 From rod.beck at unitedcablecompany.com Sun Oct 30 18:20:06 2016 From: rod.beck at unitedcablecompany.com (Rod Beck) Date: Sun, 30 Oct 2016 18:20:06 +0000 Subject: Net Neutrality in Canada In-Reply-To: <5816336F.5050103@vaxination.ca> References: <5816336F.5050103@vaxination.ca> Message-ID: Hi Jean, What is the status of net neutrality in Canada? Regards, Roderick. ________________________________ From: NANOG on behalf of Jean-Francois Mezei Sent: Sunday, October 30, 2016 6:52 PM To: Nanog at nanog.org Subject: FYI: Net Neutrality in Canada This is a heads up, the CRTC (Canada's FCC) is holding a week long hearing on net neutrality in Canada ("differential pricing" is the used). Canada has had its "ITMP" (Internet Traffic Management Practices) policy since 2009 which deals with unfair throttling, and now, we are arguing on zero rating and sponsored content stuff). It will be broadcasted at http://www.cpac.ca (at bottom of home page [http://www.cpac.ca/wp-content/uploads/default_2016_social_seo_logo_500x230.jpg] CPAC - Cable Public Affairs Channel www.cpac.ca CPAC, the Cable Public Affairs Channel, is Canada's only privately-owned, commercial free, not for profit, bilingual television service. there should be a selection for the CRTC hearing). Note: you can choose between english, french of floor (untranslated). Days generally start at 09:00. Can end at any time. Either @CRTCeng or @CRTChearings will be tweeting links to presentations as each presentation begins. hashtag: #CRTC #Diffpricing Facebook did not wish to appear but was "invited". Last time the CRTC did that, it was with Netflix and Google and sparks flew ("you don't regulate us, we don't have to answer"). (It appears on Tuesday right after me, so they should be roughly ~ 10:30 or 11:00.) The agenda: http://www.crtc.gc.ca/Telecom/eng/HEARINGS/2016/ag31_10.htm The original Notice of Consultation: > http://crtc.gc.ca/eng/archive/2016/2016-192.htm?_ga=1.136052530.24154879.1433393531 And the record of the consultation: > https://services.crtc.gc.ca/pub/instances-proceedings/Default-defaut.aspx?EN=2016-192&Lang=eng&_ga=1.136052530.24154879.1433393531 Note: this started with a different proceeding in September 2015 2 parts 1 filings against Vid?otron who started zero rated music on its wireless service for music services that Vid?otron approved/selected. > https://services.crtc.gc.ca/pub/instances-proceedings/Default-Defaut.aspx?lang=eng&YA=2015&S=C&PA=t&PT=pt1&PST=a#201510735 From bzs at TheWorld.com Sun Oct 30 19:59:46 2016 From: bzs at TheWorld.com (bzs at TheWorld.com) Date: Sun, 30 Oct 2016 15:59:46 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161030055905.GA20998@thyrsus.com> References: <20161030044342.GA18488@thyrsus.com> <27204.1477804758@segfault.tristatelogic.com> <20161030055905.GA20998@thyrsus.com> Message-ID: <22550.20786.128542.370945@gargle.gargle.HOWL> Is this report reliable? I don't know off-hand: http://www.csoonline.com/article/3134721/security/amateurs-were-behind-the-dyn-inc-ddos-attack-report-says.html or: http://tinyurl.com/zb9mpy5 Amateurs were behind the Dyn Inc. DDoS attack, report says Flashpoint says that despite speculation, nothing they?ve seen points to political motivation or extortion Here is Flashpoint's actual report link: https://www.flashpoint-intel.com/action-analysis-mirai-botnet-attacks-dyn/ or http://tinyurl.com/hrhewxg "...In its investigation of Dyn DDoS attacks, Flashpoint discovered that the infrastructure used in the attack also targeted a well-known video game company. While there does not appear to have been any disruption of service, the targeting of a video game company is less indicative of hacktivists, state-actors, or social justice communities, and aligns more with the hackers that frequent online hacking forums. These hackers exist in their own tier, sometimes called ?script kiddies,? and are separate and distinct from hacktivists, organized crime, state-actors, and terrorist groups. They can be motivated by financial gain, but just as often will execute attacks such as these to show off, or to cause disruption and chaos for sport..." "...Flashpoint assesses with moderate confidence that these attacks were not financially or politically motivated..." P.S. not sure why I include tinyurls other than long URLs tend to get messed up in some MUAs and on rare occasion one has to retype one in and tinyurls are tiny. -- -Barry Shein Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* From morrowc.lists at gmail.com Sun Oct 30 20:02:56 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Sun, 30 Oct 2016 16:02:56 -0400 Subject: Death of WHOIS, Film at 11 In-Reply-To: <26768.1477796751@segfault.tristatelogic.com> References: <58150673.5090201@foobar.org> <26768.1477796751@segfault.tristatelogic.com> Message-ID: On Sat, Oct 29, 2016 at 11:05 PM, Ronald F. Guilmette wrote: > > known inaccurante answers > I'm betting that the operators of the named whois servers believe the information is as accurate as they can provide, right? From jfmezei_nanog at vaxination.ca Sun Oct 30 20:19:30 2016 From: jfmezei_nanog at vaxination.ca (Jean-Francois Mezei) Date: Sun, 30 Oct 2016 16:19:30 -0400 Subject: Net Neutrality in Canada In-Reply-To: References: <5816336F.5050103@vaxination.ca> Message-ID: <581655D2.2000505@vaxination.ca> On 2016-10-30 14:20, Rod Beck wrote: > Hi Jean, > > > What is the status of net neutrality in Canada? The Telecom Act has had a clasue against undue preference/discrimination, as well as a "cannot control content", but both have loopholes. (27(2) , a carrier can argue a preference/discrimination is not "undue", and for 36 (control of content), exemptions can be granted by CRTC. The 2009 ITMP framework was more about throttling and treating packets differently. In 2010, the CRTC decided to include wireless services into the ITMP framework, treating them as ISPs. But since then, incumbents have begun to zero rate stuff and there were 2 challenges. In 2013 (decided in 2015), Bell Canada was challenged for zero rating its own TV service on its own wireless service. CRTC decided Bell couldn't do that, but Bell went to Federal Court of Appeal, arguing its MobileTV offering was covered under the Broadcasting Act and not Telecom. Federal Court sided with CRTC, confirming that the content may have been Broadcasting but it was delivered over telecom. Despite this, Vid?otron launched Zero Rating for music in August 2015, and instead of deciding on this the same way it did for Bell, the CRTC decided to launch a wider public consultation on whether zero rating should be allowed or not. The hearing that will happen this week is a continuation of a process which saw 2 rounds of submissions as well as 2 interrogatories and included the record of the Vid?otron process from 2015. In a couple of weeks we have final replies and CRTC will take 4-6 months to rule on matter. Competition Bureau basically says that zero rating is OK unless the contrent being zero rated is owned by the ISP's organisation. Consumer groups state it isn't OK, and incumbents state it is OK and that there should simply be individual challenges whenc onsumers feel one package abuses 27(2) or 36. As side note: Telus hires Eisenach lobbyist to write pro-incumbent reports. He was also hired by the Trump campaign. Not sure if he will appear this week. From rod.beck at unitedcablecompany.com Sun Oct 30 21:21:32 2016 From: rod.beck at unitedcablecompany.com (Rod Beck) Date: Sun, 30 Oct 2016 21:21:32 +0000 Subject: Net Neutrality in Canada In-Reply-To: <581655D2.2000505@vaxination.ca> References: <5816336F.5050103@vaxination.ca> , <581655D2.2000505@vaxination.ca> Message-ID: Zero rating is probably pretty popular with end users and puts net neutrality advocates in a difficult position. It is an astute political move. The EU allowing zero ratings exceptions because it is popular. - R. ________________________________ From: Jean-Francois Mezei Sent: Sunday, October 30, 2016 9:19 PM To: Rod Beck; Nanog at nanog.org Subject: Re: Net Neutrality in Canada On 2016-10-30 14:20, Rod Beck wrote: > Hi Jean, > > > What is the status of net neutrality in Canada? The Telecom Act has had a clasue against undue preference/discrimination, as well as a "cannot control content", but both have loopholes. (27(2) , a carrier can argue a preference/discrimination is not "undue", and for 36 (control of content), exemptions can be granted by CRTC. The 2009 ITMP framework was more about throttling and treating packets differently. In 2010, the CRTC decided to include wireless services into the ITMP framework, treating them as ISPs. But since then, incumbents have begun to zero rate stuff and there were 2 challenges. In 2013 (decided in 2015), Bell Canada was challenged for zero rating its own TV service on its own wireless service. CRTC decided Bell couldn't do that, but Bell went to Federal Court of Appeal, arguing its MobileTV offering was covered under the Broadcasting Act and not Telecom. Federal Court sided with CRTC, confirming that the content may have been Broadcasting but it was delivered over telecom. Despite this, Vid?otron launched Zero Rating for music in August 2015, and instead of deciding on this the same way it did for Bell, the CRTC decided to launch a wider public consultation on whether zero rating should be allowed or not. The hearing that will happen this week is a continuation of a process which saw 2 rounds of submissions as well as 2 interrogatories and included the record of the Vid?otron process from 2015. In a couple of weeks we have final replies and CRTC will take 4-6 months to rule on matter. Competition Bureau basically says that zero rating is OK unless the contrent being zero rated is owned by the ISP's organisation. Consumer groups state it isn't OK, and incumbents state it is OK and that there should simply be individual challenges whenc onsumers feel one package abuses 27(2) or 36. As side note: Telus hires Eisenach lobbyist to write pro-incumbent reports. He was also hired by the Trump campaign. Not sure if he will appear this week. From brandon.yarnell at sendgrid.com Sun Oct 30 23:52:08 2016 From: brandon.yarnell at sendgrid.com (Brandon Yarnell) Date: Sun, 30 Oct 2016 17:52:08 -0600 Subject: AS6461 (Zayo) LAS->ORD Packet Loss Message-ID: Seeing some packet loss between LAS and ORD on the Zayo backbone. Long wait times / no answer on the ncc line, no auto-response/ticket creation after sending email. Anybody know whats up? Alerts cleared after taking the peers out of route. -- =========================== Brandon Yarnell Senior Network Engineer | Technical Operations O: (720) 515-3376 From starwars1070 at gmail.com Mon Oct 31 04:45:52 2016 From: starwars1070 at gmail.com (Samual Carman) Date: Mon, 31 Oct 2016 04:45:52 +0000 (UTC) Subject: Voip faxing Message-ID: <88BD2EB35BC7983D.6473A1CB-24FC-4B33-9281-7C1BD8E6531B@mail.outlook.com> Hello if this is not allowed please ignore and inform me that it not allowed however I could think of no better place for this? I would like to know why there has not been more wide support of the V.38 or v.17 v.34 or the old style g.711 protocols for faxing over voip? Based on my understanding it is an difficult in an IP based environment for faxing to work reliably however based on my understanding V.38 was suppose to fix that however the research I have conducted in regards to the support of these protocols is that there not widely implimted and besides G.711 which was designed for voice that proviso is somewhat implored on a wider scale? However I understand faxing is archaic but in some fields it is necessary? Thanks?Sam?Please excuse spelling and grammar as this was typed on my phone? Get Outlook for iOS From carlos at race.com Mon Oct 31 04:52:46 2016 From: carlos at race.com (Carlos Alcantar) Date: Mon, 31 Oct 2016 04:52:46 +0000 Subject: Voip faxing In-Reply-To: <88BD2EB35BC7983D.6473A1CB-24FC-4B33-9281-7C1BD8E6531B@mail.outlook.com> References: <88BD2EB35BC7983D.6473A1CB-24FC-4B33-9281-7C1BD8E6531B@mail.outlook.com> Message-ID: Hey Samual, you might want to check out the voice ops mailing list, might be a bit more relevant over there. https://puck.nether.net/mailman/listinfo/voiceops VoiceOps Info Page - Welcome to puck.nether.net puck.nether.net This is the VoiceOps Mailing list. This list is for discussions related to managing voice networks, both traditional and IP. The VOIP Operators' Group (VOG) charter ... Carlos Alcantar Race Communications / Race Team Member 1325 Howard Ave. #604, Burlingame, CA. 94010 Phone: +1 415 376 3314 / carlos at race.com / http://www.race.com ________________________________ From: NANOG on behalf of Samual Carman Sent: Sunday, October 30, 2016 9:45:52 PM To: nanog at nanog.org Subject: Voip faxing Hello if this is not allowed please ignore and inform me that it not allowed however I could think of no better place for this I would like to know why there has not been more wide support of the V.38 or v.17 v.34 or the old style g.711 protocols for faxing over voip Based on my understanding it is an difficult in an IP based environment for faxing to work reliably however based on my understanding V.38 was suppose to fix that however the research I have conducted in regards to the support of these protocols is that there not widely implimted and besides G.711 which was designed for voice that proviso is somewhat implored on a wider scale However I understand faxing is archaic but in some fields it is necessary Thanks Sam Please excuse spelling and grammar as this was typed on my phone Get Outlook for iOS From dougb at dougbarton.us Mon Oct 31 04:58:07 2016 From: dougb at dougbarton.us (Doug Barton) Date: Sun, 30 Oct 2016 21:58:07 -0700 Subject: Spitballing IoT Security In-Reply-To: <26234.1477787539@segfault.tristatelogic.com> References: <26234.1477787539@segfault.tristatelogic.com> Message-ID: <51187413-00f9-4353-c4d1-d1734a7c15b7@dougbarton.us> On 10/29/2016 05:32 PM, Ronald F. Guilmette wrote: > you don't need > to be either an omnious "state actor" or even SPECTER to assemble a > truly massive packet weapon. Please, it's SPECTRE .... show some respect From theodore at ciscodude.net Mon Oct 31 06:41:40 2016 From: theodore at ciscodude.net (Theodore Baschak) Date: Mon, 31 Oct 2016 01:41:40 -0500 Subject: AS6461 (Zayo) LAS->ORD Packet Loss In-Reply-To: References: Message-ID: I noticed similar for a couple hours to one host I monitor that took Zayo LGA -> ORD around the time you're describing as well. Only affected IPv4 (v6 takes same path), however that could have been a reverse path coming over something else masking the problem there. Theodore Baschak - AS395089 - Hextet Systems https://ciscodude.net/ - https://hextet.systems/ http://mbix.ca/ > On Oct 30, 2016, at 6:52 PM, Brandon Yarnell via NANOG wrote: > > Seeing some packet loss between LAS and ORD on the Zayo backbone. Long wait > times / no answer on the ncc line, no auto-response/ticket creation after > sending email. > > Anybody know whats up? Alerts cleared after taking the peers out of route. > > > > -- > > =========================== > > > Brandon Yarnell > Senior Network Engineer | Technical Operations > O: (720) 515-3376 From dot at dotat.at Mon Oct 31 10:57:00 2016 From: dot at dotat.at (Tony Finch) Date: Mon, 31 Oct 2016 10:57:00 +0000 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <25352.1477766547@segfault.tristatelogic.com> References: <25352.1477766547@segfault.tristatelogic.com> Message-ID: Ronald F. Guilmette wrote: > > You are correct. In this case, it would have been helpful if APNIC's WHOIS > server returned something, when queried about 103.11.67.105, that would > include an explicit referral to the ARIN WHOIS server. I mean they > obviously know all the transfers they've made. Yes, the state of whois referrals from RIRs is a bit of a mess. I have changed FreeBSD whois to rely more on referrals than built-in knowledge, and this mostly works. There are a couple of hacks to cope with awkward RIRs: AfriNIC's referrals are human-readable though they can be parsed if you assume the rubric is fixed; for RIPE, if the netname is NON-RIPE-NCC-MANAGED-ADDRESS-BLOCK it is treated as a referral to ARIN; there's a similar hack for APNIC's ERX-NETBLOCKs - but evidently this doesn't apply to more recently transferred net blocks :-( It's probably time to make whois use RDAP under the covers for address lookups. Bah. Tony. -- f.anthony.n.finch http://dotat.at/ - I xn--zr8h punycode Southeast Iceland: Westerly veering northwesterly 6 to gale 8, decreasing 4 or 5 for a time. Rough or very rough, occasionally high at first, then becoming moderate in west. Showers. Good, occasionally poor. From morrowc.lists at gmail.com Mon Oct 31 14:20:06 2016 From: morrowc.lists at gmail.com (Christopher Morrow) Date: Mon, 31 Oct 2016 10:20:06 -0400 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <21691.1477697805@segfault.tristatelogic.com> References: <21691.1477697805@segfault.tristatelogic.com> Message-ID: On Fri, Oct 28, 2016 at 7:36 PM, Ronald F. Guilmette wrote: > In my own defense, I didn't see the ARIN allocation because I have a > normative process that I use for looking up IP addresses. It's > hierarchical, and I always start with whatver whois.iana.org has to > say. And it says that that 103.0.0.0/8 belongs to APNIC, so of course, > I only looked at what whois.apnic.net had to say about 103.11.67.105. > And it says that it's unallocated. (And apparently, data shown for > announced prefixes on the bgp.he.net web site is also obtained in this > same straightforward way, because it also is showing 103.11.67.0/24 as > registered to "Asia Pacific Network Information Centre".) > In this new world of inter-rir transfers your process needs a revision. it's also not uncommon for hosting folks to allocate address space to non-local customers. From pierre at userid.org Mon Oct 31 17:14:10 2016 From: pierre at userid.org (Pierre Lamy) Date: Mon, 31 Oct 2016 13:14:10 -0400 Subject: Spitballing IoT Security In-Reply-To: <20161030044342.GA18488@thyrsus.com> References: <20161029180730.GA10801@thyrsus.com> <26234.1477787539@segfault.tristatelogic.com> <20161030044342.GA18488@thyrsus.com> Message-ID: On 30/10/2016 12:43 AM, Eric S. Raymond wrote: > Ronald F. Guilmette : >> Two kids with a modest amount of knowledge >> and a lot of time on their hands can do it from their mom's basement. > > I in turn have to call BS on this. If it were really that easy, we'd > be inundated by Mirais -- we'd have several attacks a *day*. > It's not easy, Mirai was closed source until the actor released it. We see a pattern again and again, where the bad guys find a private monetization strategy, milk it, and get out before too much attention is focused on just them. By dumping the code, the Mirai actors obfuscate attribution. Now that the code is public, we see a huge surge in dumb & pointless attacks against gaming/mod sites, Dyn, public schools and so on. We also see bad code "improvements" which were recently referenced. http://motherboard.vice.com/read/wannabe-hackers-are-adding-terrible-and-stupid-features-to-mirai The long term problem isn't any manufacturer or Mirai, it's going to be the long tail of IoT devices that never see a patch, deployed by people who don't know anything about security (nor should they need to... flame suit on). When millions of any type of device are put online, times thousands of products, it only takes one bad guy's "a-ha" moment for this to happen again. They'll milk it for a while, the attack vector/method will get pushed down to the skid level, and we'll see a massive increase in un-targeted attacks by those script kiddies until the cycle repeats. There's an endless fresh supply of script kiddies. While I agree with BCP38 etc, it wouldn't have prevented Mirai. Self-update functions at some point for these devices are going to get borked as well, such as a company going bust or forgetting to renew their auto-update target domain. If you can't get (thousands?) of major operators to deploy common sense security configurations, how will similar best practices be implemented by tens of thousands of manufacturers? Putting device regulations in one country won't impact the rest of the internet's connected devices either. Solutions...? Someone's going to have to take out a hammer, not a scalpel, for these issues. From amps at djlab.com Mon Oct 31 19:33:19 2016 From: amps at djlab.com (Randy) Date: Mon, 31 Oct 2016 12:33:19 -0700 Subject: Help interpret a strange traceroute? Message-ID: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> Happy monday all! Any idea how a traceroute (into my network) could end up this fubar'd? Discovered this wierd routing while investigating horrendously slow speeds (albeit no packet loss) to a particular ISP abroad. It's like - coming into us - the packets are taking every available path, simultaniously. Tracing back to him looks perfectly normal. Thank you in advance for any comments on or off list. Him to us: traceroute to mirror.ash.fastserv.com (208.85.240.29), 64 hops max, 52 byte packets 1 192.168.1.1 (192.168.1.1) 1.552 ms 1.522 ms 1.261 ms 2 br1.l.hiweb.ir (46.224.0.12) 17.750 ms 18.932 ms 18.811 ms 3 172.16.5.65 (172.16.5.65) 18.505 ms 20.044 ms 18.414 ms 4 int.gr1.s.hiweb.ir (46.224.0.125) 19.096 ms 18.447 ms 18.752 ms 5 int.gr1.l.hiweb.ir (46.224.0.117) 18.454 ms 18.768 ms 103.441 ms 6 10.21.252.166 (10.21.252.166) 66.308 ms 18.728 ms 18.867 ms 7 * * * 8 185.100.209.139 (185.100.209.139) 162.126 ms ae5.istanbul1.ist.seabone.net (93.186.132.212) 210.124 ms 126.208 ms 9 et7-1-0.franco71.fra.seabone.net (195.22.214.131) 102.694 ms if-ae-9-3.tcore1.pvu-paris.as6453.net (195.219.87.14) 189.955 ms 185.100.209.197 (185.100.209.197) 218.539 ms 10 te0-3-0-4.rcr21.b023101-0.lon01.atlas.cogentco.com (149.14.144.89) 201.223 ms 215.996 ms 185.100.209.1 (185.100.209.1) 164.381 ms 11 if-ae-2-2.tcore2.av2-amsterdam.as6453.net (195.219.194.6) 199.161 ms be2950.ccr22.lon01.atlas.cogentco.com (130.117.2.109) 216.648 ms if-ae-3-6.tcore1.l78-london.as6453.net (80.231.130.85) 271.750 ms 12 if-ae-4-2.thar1.njy-newark.as6453.net (80.231.130.34) 183.984 ms * be2814.ccr42.ams03.atlas.cogentco.com (130.117.0.141) 109.045 ms 13 if-ae-1-3.thar2.njy-newark.as6453.net (216.6.57.2) 207.371 ms be2870.ccr41.lon13.atlas.cogentco.com (154.54.58.173) 186.804 ms be12488.ccr42.lon13.atlas.cogentco.com (130.117.51.41) 190.236 ms 14 if-ae-14-14.tcore2.nto-new-york.as6453.net (66.198.111.126) 203.156 ms if-ae-9-2.tcore1.n75-new-york.as6453.net (63.243.128.122) 273.177 ms be2807.ccr42.dca01.atlas.cogentco.com (154.54.40.110) 235.632 ms 15 66.110.96.134 (66.110.96.134) 182.695 ms be2806.ccr41.dca01.atlas.cogentco.com (154.54.40.106) 278.593 ms 240.075 ms 16 66.110.96.142 (66.110.96.142) 245.454 ms be3084.ccr41.iad02.atlas.cogentco.com (154.54.30.66) 191.973 ms be3083.ccr41.iad02.atlas.cogentco.com (154.54.30.54) 240.740 ms 17 te0-0-0-0.nr13.b023801-0.iad01.atlas.cogentco.com (154.24.36.18) 257.027 ms be3083.ccr41.iad02.atlas.cogentco.com (154.54.30.54) 236.617 ms hu-1-3-0-2-cr02.newyork.ny.ibone.comcast.net (68.86.83.97) 221.186 ms 18 be-10102-cr02.ashburn.va.ibone.comcast.net (68.86.85.161) 194.766 ms 250.493 ms be-10203-cr01.newark.nj.ibone.comcast.net (68.86.85.185) 191.052 ms 19 be-10102-cr02.ashburn.va.ibone.comcast.net (68.86.85.161) 197.694 ms te0-0-2-0.nr11.b023801-0.iad01.atlas.cogentco.com (154.24.36.2) 266.914 ms 235.444 ms 20 te0-0-2-3.nr11.b023801-0.iad01.atlas.cogentco.com (154.24.36.6) 255.756 ms 38.88.249.210 (38.88.249.210) 216.677 ms te0-0-2-0.nr11.b023801-0.iad01.atlas.cogentco.com (154.24.36.2) 211.364 ms 21 208.85.240.29 (208.85.240.29) 274.692 ms 212.896 ms ash01-mls-dc-core-a.latisys.net (67.217.175.37) 208.976 ms And back to him: traceroute to 46.224.145.65 (46.224.145.65), 30 hops max, 60 byte packets 1 104.153.64.146 (104.153.64.146) 22.067 ms 22.049 ms * 2 xe-3-1-1.er1.iad10.us.above.net (208.185.24.1) 0.398 ms 0.392 ms 0.420 ms 3 zayo-tata.iad10.us.zip.zayo.com (64.125.13.170) 0.708 ms 0.580 ms 0.623 ms 4 if-ae-11-2.thar2.NJY-Newark.as6453.net (216.6.87.138) 92.900 ms if-ae-11-3.thar2.NJY-Newark.as6453.net (216.6.87.242) 92.802 ms if-ae-11-4.thar2.NJY-Newark.as6453.net (216.6.87.169) 97.177 ms 5 if-ae-1-3.thar1.NJY-Newark.as6453.net (216.6.57.1) 92.940 ms 93.008 ms 89.431 ms 6 if-ae-8-2.tcore1.LDN-London.as6453.net (66.198.70.174) 97.315 ms 96.049 ms 95.855 ms 7 if-ae-17-2.tcore1.L78-London.as6453.net (80.231.130.129) 98.074 ms if-ae-3-6.tcore1.PYE-Paris.as6453.net (80.231.130.86) 92.733 ms 92.898 ms 8 if-ae-2-2.tcore1.PVU-Paris.as6453.net (80.231.154.17) 93.636 ms if-ae-3-6.tcore1.PYE-Paris.as6453.net (80.231.130.86) 97.917 ms 98.363 ms 9 if-ae-2-2.tcore1.PVU-Paris.as6453.net (80.231.154.17) 100.909 ms if-ae-9-3.tcore2.FNM-Frankfurt.as6453.net (195.219.87.13) 92.055 ms 93.173 ms 10 195.219.87.46 (195.219.87.46) 208.894 ms 209.042 ms if-ae-9-2.tcore2.FNM-Frankfurt.as6453.net (195.219.87.9) 94.718 ms 11 * 195.219.87.46 (195.219.87.46) 209.640 ms int.gr1.s.hiweb.ir (46.224.0.118) 202.201 ms 12 int.cr1.s.hiweb.ir (46.224.0.126) 205.150 ms * * 13 * * * 14 int.gr1.s.hiweb.ir (46.224.0.118) 201.803 ms 201.799 ms int.cr1.s.hiweb.ir (46.224.0.126) 201.819 ms 15 int.cr1.s.hiweb.ir (46.224.0.126) 204.952 ms 198.969 ms * 16 46.224.145.65 (46.224.145.65) 210.507 ms 211.144 ms 206.998 ms From bill at herrin.us Mon Oct 31 19:42:33 2016 From: bill at herrin.us (William Herrin) Date: Mon, 31 Oct 2016 15:42:33 -0400 Subject: Help interpret a strange traceroute? In-Reply-To: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> References: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> Message-ID: On Mon, Oct 31, 2016 at 3:33 PM, Randy wrote: > Any idea how a traceroute (into my network) could end up this fubar'd? > Discovered this wierd routing while investigating horrendously slow speeds > (albeit no packet loss) to a particular ISP abroad. Hi Randy, This is per-packet load balancing. In the forward path the alternates are different lengths but the traceroute stops as soon as at least one of the paths reaches the destination. The return path is also engaged in per-packet load balancing but the paths are all the same length. Regards, Bill Herrin -- William Herrin ................ herrin at dirtside.com bill at herrin.us Owner, Dirtside Systems ......... Web: From olivier.benghozi at wifirst.fr Mon Oct 31 21:20:01 2016 From: olivier.benghozi at wifirst.fr (Olivier Benghozi) Date: Mon, 31 Oct 2016 22:20:01 +0100 Subject: Help interpret a strange traceroute? In-Reply-To: References: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> Message-ID: Hi Randy, ECMP loadbalancing is most frequently done on layer3+layer4 headers, and unixlike traceroute use UDP with increasing destination port number for each packet (usually starting at 33434), which allows to see the different available paths, as wrote William. Would you want/need to stick to only one traceroute path, you may use ICMP traceroute instead of UDP traceroute (no port in ICMP, so only layer 3 available to loadbalance, so all packets will go through the same interface). Usually it is achieved by using traceroute -I yourdest Windows tracert is ICMP only traceroute by the way. MTR tool is also ICMP based by default. Keep in mind that it looses some useful information, though (since you see only one path and don't decide which). So, you can also use UDP traceroute with fixed port (by example 33434 with no port increase), and try again the same traceroute with another destport (with fixed port too, by example 33435), which would display two different paths in a more readable way. RTFM is required since the options depend on your traceroute particular specie :) Olivier > On 31 oct. 2016 ? 20:42, William Herrin wrote : > > On Mon, Oct 31, 2016 at 3:33 PM, Randy wrote: >> Any idea how a traceroute (into my network) could end up this fubar'd? >> Discovered this wierd routing while investigating horrendously slow speeds >> (albeit no packet loss) to a particular ISP abroad. > > Hi Randy, > > This is per-packet load balancing. In the forward path the alternates > are different lengths but the traceroute stops as soon as at least one > of the paths reaches the destination. > > The return path is also engaged in per-packet load balancing but the > paths are all the same length. From selphie.keller at gmail.com Mon Oct 31 18:00:25 2016 From: selphie.keller at gmail.com (Selphie Keller) Date: Mon, 31 Oct 2016 12:00:25 -0600 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 Message-ID: Hi, I noticed this thread and wanted to provide some information, subnet 103.11.67.0/24 is not an illicit squat this subnet is apart of 103.11.64.0/22 which was transferred from APNIC to ARIN back this last February and is listed publicly at https://www.arin.net/knowledge/statistics/transfers.html within the "Inter-RIR Transfers to the ARIN Region", also WebNX AS18450 does have the LOA's on file for the subnet. I do agree with the others in this thread about the lack of WHOIS as looking up 103.11.67.0/24 does indeed provide very little information to go on so I can see how this could be misunderstood as a squat of the subnet due to the lack of whois information which is an updating issue ARIN/APNIC's part, hopefully can get this resolved so that ARIN shows the chain: APNIC -> 103.11.64.0/22 -> then to WebNX 103.11.67.0/24, which would show the full chain and a proper abuse contact for this subnet. As for the spamming/spam email part of this thread, please send the said spam email/emails with headers in question to abuse at webnx.com, this way we can investigate and sort it out. We do take spamming seriously and will work quickly to get it resolved. -Selphie K From bryan at shout.net Mon Oct 31 21:54:33 2016 From: bryan at shout.net (Bryan Holloway) Date: Mon, 31 Oct 2016 16:54:33 -0500 Subject: Help interpret a strange traceroute? In-Reply-To: References: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> Message-ID: On 10/31/16 4:20 PM, Olivier Benghozi wrote: > Hi Randy, > > > ECMP loadbalancing is most frequently done on layer3+layer4 headers, and unixlike traceroute use UDP with increasing destination port number for each packet (usually starting at 33434), which allows to see the different available paths, as wrote William. > > Would you want/need to stick to only one traceroute path, you may use ICMP traceroute instead of UDP traceroute (no port in ICMP, so only layer 3 available to loadbalance, so all packets will go through the same interface). > > Usually it is achieved by using traceroute -I yourdest > Windows tracert is ICMP only traceroute by the way. MTR tool is also ICMP based by default. > > Keep in mind that it looses some useful information, though (since you see only one path and don't decide which). > So, you can also use UDP traceroute with fixed port (by example 33434 with no port increase), and try again the same traceroute with another destport (with fixed port too, by example 33435), which would display two different paths in a more readable way. RTFM is required since the options depend on your traceroute particular specie :) > > > Olivier > >> On 31 oct. 2016 ? 20:42, William Herrin wrote : >> >> On Mon, Oct 31, 2016 at 3:33 PM, Randy wrote: >>> Any idea how a traceroute (into my network) could end up this fubar'd? >>> Discovered this wierd routing while investigating horrendously slow speeds >>> (albeit no packet loss) to a particular ISP abroad. >> >> Hi Randy, >> >> This is per-packet load balancing. In the forward path the alternates >> are different lengths but the traceroute stops as soon as at least one >> of the paths reaches the destination. >> >> The return path is also engaged in per-packet load balancing but the >> paths are all the same length. > This is also a handy tool that addresses the same issues: https://paris-traceroute.net/ From rod.beck at unitedcablecompany.com Mon Oct 31 23:15:56 2016 From: rod.beck at unitedcablecompany.com (Rod Beck) Date: Mon, 31 Oct 2016 23:15:56 +0000 Subject: Network Maps - Western Europe In-Reply-To: References: <346d50edd479d7c16a669bd3c02e755b@mailbox.fastserv.com> , Message-ID: Hi, I am trying to determine the physical diversity of the Zayo and Level3 networks vis-a-vis each other on the European racetrack - London/Amsterdam/Frankfurt/Paris/London. It is for a client of mine. Regards, Roderick. From nick at foobar.org Mon Oct 31 23:21:02 2016 From: nick at foobar.org (Nick Hilliard) Date: Mon, 31 Oct 2016 23:21:02 +0000 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: References: Message-ID: <5817D1DE.8060503@foobar.org> Selphie Keller wrote: > APNIC -> 103.11.64.0/22 -> then to WebNX 103.11.67.0/24, which would show > the full chain and a proper abuse contact for this subnet. the tl;dr on the thread scrollback was: 1. whois is irredeemably broken 2. use rdap, which supports referrals 3. open source RDAP client: https://github.com/arineng/nicinfo Nick From selphie.keller at gmail.com Mon Oct 31 23:43:57 2016 From: selphie.keller at gmail.com (Selphie Keller) Date: Mon, 31 Oct 2016 17:43:57 -0600 Subject: Another day, another illicit SQUAT - WebNX (AS18450) 103.11.67.0/24 In-Reply-To: <5817D1DE.8060503@foobar.org> References: <5817D1DE.8060503@foobar.org> Message-ID: Nick, Very cool, learn something new every day :) [root at stellarfrost(~)]> nicinfo 103.11.67.167 # NicInfo v.1.1.1 [ NOTICE ] Terms of Service 1 By using the ARIN RDAP/Whois service, you are agreeing to the RDAP/Whois Terms of Use About https://www.arin.net/whois_tou.html # Query type is IP4ADDR. Result type is IP. [ RESPONSE DATA ] 1= NET-103-11-67-0-1 |--- 1= Gaiacom, L.C. ( GL-299 ) | |--- 1= GCM NY NOC ( GNN-ARIN ) | `--- 2= GCM NET ABUSE ( GNA35-ARIN ) `--- 2= Los Angeles NOC ( LAN55-ARIN ) [ IP NETWORK ] Handle: NET-103-11-67-0-1 Start Address: 103.011.067.000 End Address: 103.011.067.255 IP Version: v4 Last Changed: Mon, 13 Jun 2016 15:20:51 -0700 Registration: Wed, 25 May 2016 17:17:12 -0700 [ ENTITY ] Handle: GL-299 Name: Gaiacom, L.C. Roles: Registrant Last Changed: Fri, 15 Aug 2014 11:26:53 -0700 Registration: Wed, 04 Dec 2013 13:01:12 -0800 [ ENTITY ] Handle: GNN-ARIN Name: GCM NY NOC Organization: GCM NY NOC Email: noc at gaiacom.net Phone: +1-310-421-9099 ( work, voice ) Phone: +1-310-421-9098 ( work, fax ) Roles: Noc, Technical, Administrative Status: Validated Last Changed: Sat, 20 Aug 2016 09:21:23 -0700 Registration: Tue, 26 Nov 2013 22:58:12 -0800 [ ENTITY ] Handle: GNA35-ARIN Name: GCM NET ABUSE Organization: GCM NET ABUSE Email: noc at maya.net Phone: +1-310-421-9099 ( work, voice ) Phone: +1-310-421-9098 ( work, fax ) Roles: Abuse Status: Validated Last Changed: Wed, 03 Aug 2016 13:51:02 -0700 Registration: Tue, 26 Nov 2013 23:39:45 -0800 [ ENTITY ] Handle: LAN55-ARIN Name: Los Angeles NOC Organization: Los Angeles NOC Email: noc at maya.net Phone: +1-213-587-7995 ( work, voice ) Phone: +1-213-587-7995 ( work, cell ) Phone: +1-213-587-7995 ( work, fax ) Roles: Technical, Noc Status: Validated Last Changed: Mon, 13 Jun 2016 15:14:38 -0700 Registration: Mon, 13 Jun 2016 15:14:38 -0700 # Use "nicinfo 1=" to show NET-103-11-67-0-1 # Use "nicinfo 1.1=" to show Gaiacom, L.C. ( GL-299 ) # Use "nicinfo 1.2=" to show Los Angeles NOC ( LAN55-ARIN ) # Use "nicinfo https://rdap.arin.net/registry/ip/103.011.067.000" to directly query this resource in the future. # Use "nicinfo -h" for help. On 31 October 2016 at 17:21, Nick Hilliard wrote: > Selphie Keller wrote: > > APNIC -> 103.11.64.0/22 -> then to WebNX 103.11.67.0/24, which would > show > > the full chain and a proper abuse contact for this subnet. > > the tl;dr on the thread scrollback was: > > 1. whois is irredeemably broken > 2. use rdap, which supports referrals > 3. open source RDAP client: https://github.com/arineng/nicinfo > > Nick >