Mailing List Archive: nsp: ipv6

Brekage due to Hurricane Electric/Internet2

Index | Next | Previous | View Threaded

fw at deneb

Oct 5, 2009, 12:27 PM

Post #1 of 12 (1511 views)
Permalink

Brekage due to Hurricane Electric/Internet2

Debian has received a report about a partition affecting access to
security.debian.org:

<http://lists.debian.org/debian-www/2009/10/msg00020.html>

According to <http://routerproxy.grnoc.iu.edu/internet2/>, the BGP
session to Hurricane Electric at Seattle is up, the prefixes are
there, but packets are dropped.

I wouldn't be surprised if this hists Internet2 downstreams pretty
hard.

fw at deneb

Oct 6, 2009, 10:06 AM

Post #2 of 12 (1454 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

* Florian Weimer:

> Debian has received a report about a partition affecting access to
> security.debian.org:
>
> <http://lists.debian.org/debian-www/2009/10/msg00020.html>
>
> According to <http://routerproxy.grnoc.iu.edu/internet2/>, the BGP
> session to Hurricane Electric at Seattle is up, the prefixes are
> there, but packets are dropped.

I've been told that the looking glass needs some knowledge about
Internet2's routing architecture to use properly, and that I had
misinterpreted its output. Sorry about that.

The site in Brazil for which the problems were initially reported
hasn't got global IPv6 transit, and it's likely that this is causing
the reachability issue (d'oh).

wmaton at ryouko

Oct 6, 2009, 4:11 PM

Post #3 of 12 (1455 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

On Mon, 5 Oct 2009, Florian Weimer wrote:

> Debian has received a report about a partition affecting access to
> security.debian.org:
>
> <http://lists.debian.org/debian-www/2009/10/msg00020.html>

+1 here. Coming from CANARIE-assigned address space the site is
unreachable. I suspect that global transit routes are being advertised to
some research networks but the traffic is getting tanked. For example,
security.debian.org resolves to 2001:4f8:8:36::6, but can't reach it:

stats 1216# traceroute6 2001:4f8:8:36::6
traceroute to 2001:4f8:8:36::6 (2001:4f8:8:36::6), 30 hops max, 40 byte
packets
1 olans.core.ottix.net (2001:410:90ff::1) 0.542 ms 0.745 ms 0.725 ms
2 border2.core.ottix.net (2001:478:235::7) 0.460 ms 0.562 ms 0.548 ms
3 canet5.gigafed.net (2001:478:149::13) 7.021 ms 7.756 ms 8.366 ms
4 2001:410:101:30::2 (2001:410:101:30::2) 71.428 ms 71.540 ms 71.772 ms
5 2001:320:1b00:1::1 (2001:320:1b00:1::1) 185.058 ms 185.172 ms 185.776 ms
6 * * *
7 * * *
8 * * *
9 * * *

etc.

> According to <http://routerproxy.grnoc.iu.edu/internet2/>, the BGP
> session to Hurricane Electric at Seattle is up, the prefixes are
> there, but packets are dropped.
>
> I wouldn't be surprised if this hists Internet2 downstreams pretty
> hard.

It might, but CANARIE have identified the problem elsewhere but can't seem
to pinpoint precisely where.

wfms

jogi at mur

Oct 7, 2009, 1:44 AM

Post #4 of 12 (1445 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

Hi all,

Our IPv6 upstream is the Austrian academic network.

William F. Maton Sotomayor schrieb:

> +1 here. Coming from CANARIE-assigned address space the site is
> unreachable. I suspect that global transit routes are being advertised
> to some research networks but the traffic is getting tanked. For
> example, security.debian.org resolves to 2001:4f8:8:36::6, but can't
> reach it:

Here it resolves to:

2001:a78::1a

1?: [LOCALHOST] pmtu 1500
1: v200-fe2-r1ko.mur.at 0.995ms
2: wien6.v6.aco.net 6.454ms
3: 2001:7f8:30:0:2:1:0:286 6.536ms
4: mchn-s2-rou-1030.eurorings.net 14.831ms
5: mchn-s2-rou-1030.eurorings.net asymm 4 14.818ms pmtu 1476
5: v6-2-v4.01.spxs.net 31. 69ms
6: 2001:a78::1a 39.783ms reached
Resume: pmtu 1476 hops 6 back 6

2001:8d8:2:1:6564:a62:0:2
1?: [LOCALHOST] pmtu 1500
1: v200-fe2-r1ko.mur.at 1. 41ms
2: wien6.v6.aco.net 6.772ms
3: wien6.v6.aco.net asymm 2 6.535ms pmtu 1480
3: tu-637.sar1.Amsterdam1.Level3.net 47.210ms
4: tu-637.sar1.Amsterdam1.Level3.net asymm 3 46.291ms pmtu 1450
4: tu-607.sar1.London1.Level3.net 54.438ms
5: 2001:7f8:4::cb9:1 asymm 6 55. 11ms
6: xe-4-1-0.lon11.ip6.tinet.net asymm 7 55.256ms
7: xe-8-2-0.ams10.ip6.tinet.net asymm 8 59.892ms
8: xe-2-2-0.ams20.ip.tinet.net asymm 9 59.219ms
9: schlund-gw.ip6.tinet.net asymm 14 193.410ms
10: te-4-1.bb-c.act.fra.de.oneandone.net asymm 15 201.553ms
11: te-1-3.bb-c.bs.kae.de.oneandone.net asymm 16 203.809ms
12: ae-1.gw-dists-a.bs.ka.oneandone.net asymm 17 203.263ms
13: wieck.debian.org asymm 18 204.294ms reached
Resume: pmtu 1450 hops 13 back 18

2001:a78::16

1?: [LOCALHOST] pmtu 1500
1: v200-fe2-r1ko.mur.at 1. 60ms
2: wien6.v6.aco.net 7. 79ms
3: 2001:7f8:30:0:2:1:0:286 7.196ms
4: mchn-s2-rou-1030.eurorings.net 16.251ms
5: mchn-s2-rou-1030.eurorings.net asymm 4 14.200ms pmtu 1476
5: v6-2-v4.01.spxs.net 31.430ms
6: 2001:a78::16 38.966ms reached
Resume: pmtu 1476 hops 6 back 6

The address you are getting is also reachable from our corner of the
network:

1?: [LOCALHOST] pmtu 1500
1: v200-fe2-r1ko.mur.at 1.120ms
2: wien6.v6.aco.net 8.654ms
3: wien6.v6.aco.net asymm 2 7. 57ms pmtu 1480
3: tu-637.sar1.Amsterdam1.Level3.net 46.557ms
4: tu-637.sar1.Amsterdam1.Level3.net asymm 3 46.167ms pmtu 1450
4: tu-618.sar1.Dallas1.Level3.net 179.218ms
5: tu-636.sar1.SanJose1.Level3.net 222.971ms
6: ISC.tu-616.sar1.SanJose1.Level3.net asymm 7 224.300ms
7: gig-2-0-0.r1.pao1.isc.org 225.536ms
8: int-0-0-0.r1.sjc3.isc.org 228.184ms
9: schein.debian.org 225.243ms reached
Resume: pmtu 1450 hops 9 back 9

Maybe this helps ... You might also check if you can see us (e.g.
debian.mur.at @ 2a02:3e0::14:80).

Cheers,
j.
--
NCC09 - Netart Community Convention 2009
What the net!
23.11.09 - 29.11.09 Graz/Austria https://wiki.mur.at/ncc09/

Attachments:

signature.asc (0.25 KB)

fw at deneb

Oct 7, 2009, 3:36 AM

Post #5 of 12 (1442 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

* William F. Maton Sotomayor:

> On Mon, 5 Oct 2009, Florian Weimer wrote:
>
>> Debian has received a report about a partition affecting access to
>> security.debian.org:
>>
>> <http://lists.debian.org/debian-www/2009/10/msg00020.html>
>
> +1 here. Coming from CANARIE-assigned address space the site is
> unreachable.

What is your source address? Can you reach 2001:8d8:2:1:6564:a62:0:2,
for instance?

> I suspect that global transit routes are being advertised to some
> research networks but the traffic is getting tanked.

Yes, this could be a routing leak. CANARIE should be able to resolve
it.

fw at deneb

Oct 7, 2009, 3:40 AM

Post #6 of 12 (1438 views)
Permalink

Re: Breakage due to Hurricane Electric/Internet2 [In reply to]

> Here it resolves to:
>
> 2001:a78::1a
>
> 1?: [LOCALHOST] pmtu 1500
> 1: v200-fe2-r1ko.mur.at 0.995ms
> 2: wien6.v6.aco.net 6.454ms

Are you located at a G�ANT downstream? Can you reach
2001:12f0:840:4092:204:acff:fe25:f5fb?

he at uninett

Oct 7, 2009, 4:44 AM

Post #7 of 12 (1292 views)
Permalink

Re: Breakage due to Hurricane Electric/Internet2 [In reply to]

> > Here it resolves to:
> >
> > 2001:a78::1a
> >
> > 1?: [LOCALHOST] pmtu 1500
> > 1: v200-fe2-r1ko.mur.at 0.995ms
> > 2: wien6.v6.aco.net 6.454ms
>
> Are you located at a G�ANT downstream? Can you reach
> 2001:12f0:840:4092:204:acff:fe25:f5fb?

Even though I'm not him, we're at a G�ANT downstream, and a
traceroute6 to that address trails off somewhere in Brazil,
(according to whois.lacnic.net):

% traceroute6 2001:12f0:840:4092:204:acff:fe25:f5fb
traceroute6 to 2001:12f0:840:4092:204:acff:fe25:f5fb (2001:12f0:840:4092:204:acff:fe25:f5fb) from 2001:700:1:0:21e:4fff:feed:ced, 64 hops max, 12 byte packets
1 uninett-gw 0.738 ms 0.777 ms 0.776 ms
2 teknobyen-gw2 4.375 ms 2.81 ms 2.018 ms
3 trd-gw1 0.778 ms 0.777 ms 6.827 ms
4 hovedbygget-gw1 2.432 ms 0.778 ms 0.777 ms
5 hovedbygget-gw4 0.777 ms 0.779 ms 0.777 ms
6 stolav-gw4 7.638 ms 7.563 ms 7.596 ms
7 stolav-gw1 7.813 ms 7.567 ms 7.617 ms
8 oslo-gw 10.824 ms 21.657 ms 7.787 ms
9 se-tug.nordu.net 15.616 ms 15.589 ms 15.493 ms
10 se-fre.nordu.net 15.982 ms 21.764 ms 15.939 ms
11 dk-ore.nordu.net 25.648 ms 25.612 ms 25.621 ms
12 nordunet.rt2.cop.dk.geant2.net 25.7 ms 25.692 ms 25.644 ms
13 so-7-3-0.rt1.fra.de.geant2.net 39.547 ms 39.693 ms 39.679 ms
14 so-6-2-0.rt1.gen.ch.geant2.net 47.857 ms 51.778 ms 47.776 ms
15 so-7-0-0.rt1.mad.es.geant2.net 69.869 ms 69.756 ms 69.889 ms
16 clara-gw.rt1.mad.es.geant2.net 184.347 ms 184.017 ms 184.216 ms
17 2001:1348::29 292.177 ms 292.174 ms 292.116 ms
18 2001:1348:1::2 300.53 ms 325.921 ms 292.639 ms
19 2001:12f0:0:fc::16 298.086 ms 298.917 ms 298.315 ms
20 2001:12f0:0:fc::21 318.522 ms 352.891 ms 318.153 ms
21 2001:12f0:0:3020::2 318.748 ms 318.556 ms 318.951 ms
22 2001:12f0:840:169::2 318.973 ms 318.866 ms 319.07 ms
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
^C
%

Some folks obviously don't beleive in ip6.arpa :)

Regards,

- H�vard

wmaton at ryouko

Oct 7, 2009, 5:34 AM

Post #8 of 12 (1447 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

>>> <http://lists.debian.org/debian-www/2009/10/msg00020.html>
>>
>> +1 here. Coming from CANARIE-assigned address space the site is
>> unreachable.
>
> What is your source address? Can you reach 2001:8d8:2:1:6564:a62:0:2,
> for instance?

We have address space from CAANRIE's network, so naturally it is the
reasearch network. However, some blocks allocated under that space also
have access to the global IPv6 network so those folks don't have a problem
reaching things. The ones that don't get in trouble:

stats 1219# traceroute 2001:8d8:2:1:6564:a62:0:2
traceroute to 2001:8d8:2:1:6564:a62:0:2 (2001:8d8:2:1:6564:a62:0:2), 30
hops max, 40 byte packets
1 olans.core.ottix.net (2001:410:90ff::1) 0.710 ms 0.788 ms 0.880 ms
2 border2.core.ottix.net (2001:478:235::7) 0.488 ms 0.482 ms 0.468 ms
3 canet5.gigafed.net (2001:478:149::13) 13.182 ms 13.664 ms 14.148 ms
4 2001:410:101:30::2 (2001:410:101:30::2) 70.721 ms 70.832 ms 71.430 ms
5 2001:320:1b00:1::1 (2001:320:1b00:1::1) 185.465 ms 185.575 ms 185.690 ms
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 linx.bb-c.the.lon.gb.oneandone.net (2001:7f8:4::2170:1) 685.270 ms 689.004 ms 695.486 ms
13 te-1-2.bb-c.nkf.ams.nl.oneandone.net (2001:8d8:0:2::6) 699.094 ms 708.692 ms 711.432 ms
14 te-4-1.bb-c.act.fra.de.oneandone.net (2001:8d8:0:2::a) 719.148 ms 722.634 ms 725.365 ms
15 te-3-3.bb-c.bs.kae.de.oneandone.net (2001:8d8:0:2::2a) 1026.029 ms
te-1-3.bb-c.bs.kae.de.oneandone.net (2001:8d8:0:2::12) 1016.646 ms
te-3-3.bb-c.bs.kae.de.oneandone.net (2001:8d8:0:2::2a) 730.999 ms
16 ae-1.gw-dists-b.bs.ka.oneandone.net (2001:8d8:0:4::11) 731.097 ms
ae-2.gw-dists-b.bs.ka.oneandone.net (2001:8d8:0:5::11) 731.451 ms
ae-1.gw-dists-b.bs.ka.oneandone.net (2001:8d8:0:4::11) 731.185 ms
17 wieck.debian.org (2001:8d8:2:1:6564:a62:0:2) 727.989 ms 721.742 ms
718.61 9 ms

Interesting. Today it's working.

>> I suspect that global transit routes are being advertised to some
>> research networks but the traffic is getting tanked.
>
> Yes, this could be a routing leak. CANARIE should be able to resolve
> it.

They are talking to KREONet who apparently is announcing those routes.
The above traceroute might be a fix in progress.

wfms

jogi at mur

Oct 7, 2009, 6:01 AM

Post #9 of 12 (1439 views)
Permalink

Re: Breakage due to Hurricane Electric/Internet2 [In reply to]

Hi,

Florian Weimer schrieb:

> Are you located at a GÉANT downstream? Can you reach
> 2001:12f0:840:4092:204:acff:fe25:f5fb?

had to do a traceroute6 -I but:

1 v201-fe2-r1ko.mur.at (2a02:3e0:1::1) 0.364 ms 0.321 ms 0.331 ms
2 wien6.v6.aco.net (2001:628::1) 4.284 ms 4.434 ms 4.426 ms
3 aconet.rt1.vie.at.geant2.net (2001:798:10:10dd::1) 49.511 ms
59.531 ms 59.532 ms
4 so-3-0-0.rt1.mil.it.geant2.net (2001:798:cc:1001:1e01::2) 50.693 ms
50.853 ms 50.850 ms
5 so-6-3-0.rt1.gen.ch.geant2.net (2001:798:cc:1201:1e01::1) 57.507 ms
2001:798:cc:1201:1e01::5 (2001:798:cc:1201:1e01::5) 58.781 ms
so-6-3-0.rt1.gen.ch.geant2.net (2001:798:cc:1201:1e01::1) 57.528 ms
6 so-7-0-0.rt1.mad.es.geant2.net (2001:798:cc:1201:1701::2) 80.994 ms
80.373 ms 80.587 ms
7 * * *
8 * * *
9 2001:1348:1::2 (2001:1348:1::2) 412.170 ms 412.209 ms 406.535 ms
10 2001:12f0:0:fc::16 (2001:12f0:0:fc::16) 406.512 ms 405.716 ms
404.130 ms
11 2001:12f0:0:fc::21 (2001:12f0:0:fc::21) 396.135 ms 381.061 ms
382.529 ms
12 2001:12f0:0:3020::2 (2001:12f0:0:3020::2) 382.290 ms 416.656 ms
416.944 ms
13 2001:12f0:840:169::2 (2001:12f0:840:169::2) 416.563 ms 416.929 ms
416.879 ms
14 2001:12f0:840:4092:204:acff:fe25:f5fb
(2001:12f0:840:4092:204:acff:fe25:f5fb) 416.870 ms 417.169 ms 417.170 ms

Cheers,
j.
--
NCC09 - Netart Community Convention 2009
What the net!
23.11.09 - 29.11.09 Graz/Austria https://wiki.mur.at/ncc09/

Attachments:

signature.asc (0.25 KB)

jabley at hopcount

Oct 8, 2009, 9:47 AM

Post #10 of 12 (1432 views)
Permalink

Re: Brekage due to Hurricane Electric/Internet2 [In reply to]

On 2009-10-07, at 00:11, William F. Maton Sotomayor wrote:

> On Mon, 5 Oct 2009, Florian Weimer wrote:
>
>> Debian has received a report about a partition affecting access to
>> security.debian.org:
>>
>> <http://lists.debian.org/debian-www/2009/10/msg00020.html>
>
> +1 here. Coming from CANARIE-assigned address space the site is
> unreachable. I suspect that global transit routes are being
> advertised to some research networks but the traffic is getting
> tanked. For example, security.debian.org resolves to
> 2001:4f8:8:36::6, but can't reach it:

Note that this debian server is hosted by ISC. If you're trying to
debug, it may be easiest and best to contact their netops staff at noc [at] isc
rather than trying to funnel the reports through debian volunteers.

Joe

jogi at mur

Oct 28, 2009, 2:20 PM

Post #11 of 12 (1283 views)
Permalink

Re: Breakage due to Hurricane Electric/Internet2 [In reply to]

Hey,

Quite awkward. This mail was on route for >20 days! According to how I
read the header, mail1.cluenet.de kept it for a while.

Havard Eidnes schrieb:
>>> Here it resolves to:
>>>
>>> 2001:a78::1a
>>>
>>> 1?: [LOCALHOST] pmtu 1500
>>> 1: v200-fe2-r1ko.mur.at 0.995ms
>>> 2: wien6.v6.aco.net 6.454ms
>> Are you located at a GÉANT downstream? Can you reach
>> 2001:12f0:840:4092:204:acff:fe25:f5fb?
>
> Even though I'm not him, we're at a GÉANT downstream, and a
> traceroute6 to that address trails off somewhere in Brazil,
> (according to whois.lacnic.net):

AFAIKS That's quite allright. We seem to have gotten the problems
fixed, thanx to the help from ACOnet people.

1?: [LOCALHOST] pmtu 1500
1: v200-fe2-r1ko.mur.at 1. 8ms
2: wien6.v6.aco.net 11.916ms
3: wien6.v6.aco.net asymm 2 6.605ms pmtu 1476
3: aconet.rt1.vie.at.geant2.net asymm 4 7.173ms
4: so-3-0-0.rt1.mil.it.geant2.net asymm 5 19.389ms
5: so-6-3-0.rt1.gen.ch.geant2.net asymm 6 27.533ms
6: so-7-0-0.rt1.mad.es.geant2.net asymm 7 49.501ms
7: clara-gw.rt1.mad.es.geant2.net 194.930ms
8: 2001:1348::29 302.328ms
9: 2001:1348:1::2 asymm 10 304.115ms
10: 2001:12f0:0:fc::16 asymm 11 308.389ms
11: 2001:12f0:0:fc::21 asymm 12 329. 63ms !H
Resume: pmtu 1476

The host seems to be unreachable at the moment, but it looks like I'm
almost there.

> % traceroute6 2001:12f0:840:4092:204:acff:fe25:f5fb
> traceroute6 to 2001:12f0:840:4092:204:acff:fe25:f5fb (2001:12f0:840:4092:204:acff:fe25:f5fb) from 2001:700:1:0:21e:4fff:feed:ced, 64 hops max, 12 byte packets
> 1 uninett-gw 0.738 ms 0.777 ms 0.776 ms
> 2 teknobyen-gw2 4.375 ms 2.81 ms 2.018 ms
> 3 trd-gw1 0.778 ms 0.777 ms 6.827 ms
> 4 hovedbygget-gw1 2.432 ms 0.778 ms 0.777 ms
> 5 hovedbygget-gw4 0.777 ms 0.779 ms 0.777 ms
> 6 stolav-gw4 7.638 ms 7.563 ms 7.596 ms
> 7 stolav-gw1 7.813 ms 7.567 ms 7.617 ms
> 8 oslo-gw 10.824 ms 21.657 ms 7.787 ms
> 9 se-tug.nordu.net 15.616 ms 15.589 ms 15.493 ms
> 10 se-fre.nordu.net 15.982 ms 21.764 ms 15.939 ms
> 11 dk-ore.nordu.net 25.648 ms 25.612 ms 25.621 ms
> 12 nordunet.rt2.cop.dk.geant2.net 25.7 ms 25.692 ms 25.644 ms
> 13 so-7-3-0.rt1.fra.de.geant2.net 39.547 ms 39.693 ms 39.679 ms
> 14 so-6-2-0.rt1.gen.ch.geant2.net 47.857 ms 51.778 ms 47.776 ms
> 15 so-7-0-0.rt1.mad.es.geant2.net 69.869 ms 69.756 ms 69.889 ms
> 16 clara-gw.rt1.mad.es.geant2.net 184.347 ms 184.017 ms 184.216 ms
> 17 2001:1348::29 292.177 ms 292.174 ms 292.116 ms
> 18 2001:1348:1::2 300.53 ms 325.921 ms 292.639 ms
> 19 2001:12f0:0:fc::16 298.086 ms 298.917 ms 298.315 ms
> 20 2001:12f0:0:fc::21 318.522 ms 352.891 ms 318.153 ms
> 21 2001:12f0:0:3020::2 318.748 ms 318.556 ms 318.951 ms
> 22 2001:12f0:840:169::2 318.973 ms 318.866 ms 319.07 ms
> 23 * * *
> 24 * * *
> 25 * * *
> 26 * * *
> 27 * * *
> 28 * * *
> 29 * * *
> ^C
> %
>
> Some folks obviously don't beleive in ip6.arpa :)

Well, all those 0.0.0... give me headaches too ;)

Cheers,
j.
--
NCC09 - Netart Community Convention 2009
What the net!
23.11.09 - 29.11.09 Graz/Austria https://wiki.mur.at/ncc09/

Attachments:

signature.asc (0.25 KB)

dr at cluenet

Oct 28, 2009, 5:06 PM

Post #12 of 12 (1278 views)
Permalink

Re: Breakage due to Hurricane Electric/Internet2 [In reply to]

On Wed, Oct 28, 2009 at 10:20:54PM +0100, Jogi Hofmueller wrote:
> Quite awkward. This mail was on route for >20 days! According to how I
> read the header, mail1.cluenet.de kept it for a while.

Yes, because the poster didn't post using his subscribed email address,
so the posting got held for approval. And I don't have much time lately
so it might take some (few) weeks until I get around to wade through a
couple thousand of held-for-moderation mails in the queue.

Will be fixed as soon as I have spam filtering in place at SMTP level.
And then folks will start complaining about false positives. The
spammers have won, they broke email. :-(

Now back to ontopic stuff. :)

Best regards,
Daniel

--
CLUE-RIPE -- Jabber: dr [at] cluenet -- dr [at] IRCne -- PGP: 0xA85C8AA0

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads