Mailing List Archive: nsp: juniper

MX960 JunOS recommendations

1 2

View All

Index | Next | Previous | View Threaded

andreir at gmail

Nov 5, 2009, 5:39 AM

Post #1 of 28 (1592 views)
Permalink

MX960 JunOS recommendations

Hello everybody,

We will be installing our first Juniper MX960 router in the network.
We were a 100% Cisco shop and this is our first Juniper router. The MX
will be a P/PE node
in a pretty standard MPLS backbone network surrounded by Cisco 7600
routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2
or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple
10GE links is a must. Does anyone have a recommendation for a stable
JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that
10.0 came out on the 4th of November but from our experience with
Cisco IOS releases we are reluctant to use the latest & greatest
release.

--
Andrei

"2+2=5, for extremely large values of 2 !"
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

cra at wpi

Nov 5, 2009, 10:04 AM

Post #2 of 28 (1560 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

phil at juniper

Nov 5, 2009, 10:27 AM

Post #3 of 28 (1559 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Derick Winkworth writes:
>9.3r4 indeed. Perhaps even 9.4r4 when that comes out.

FWIW: 9.5 has a number of scripting-related features, including
interactivity and remote RPC access. We've been working to ensure
that scripting PRs are backported to at least 9.5.

Thanks,
Phil
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

dwinkworth at att

Nov 5, 2009, 10:28 AM

Post #4 of 28 (1558 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

9.3r4 indeed. Perhaps even 9.4r4 when that comes out.

There are various issues and other things that have been nicely cleaned up in these releases. Supposedly there has been some process improvements made to how JUNOS is coded and that this will be reflected in the 10.0 release. I think there will greater confidence around the R1 and R2 releases going forward. We are aiming for the 10r2 or 10r3 release pending lab testing...

There is a new feature in 10 that automatically creates vlan interfaces based on dhcp authentication. Can't wait to test that.

There are some great new features in 10, if you haven't already read the release notes...

http://www.juniper.net/techpubs/en_US/junos10.0/information-products/topic-collections/release-notes/10/junos-release-notes-10.0.pdf

________________________________
From: Chuck Anderson <cra [at] wpi>
To: juniper-nsp [at] puck
Sent: Thu, November 5, 2009 12:04:57 PM
Subject: Re: [j-nsp] MX960 JunOS recommendations

On Thu, Nov 05, 2009 at 03:39:32PM +0200, Andrei Radu wrote:
> Hello everybody,
>
> We will be installing our first Juniper MX960 router in the network.
> We were a 100% Cisco shop and this is our first Juniper router. The MX
> will be a P/PE node
> in a pretty standard MPLS backbone network surrounded by Cisco 7600
> routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2
> or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple
> 10GE links is a must. Does anyone have a recommendation for a stable
> JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that
> 10.0 came out on the 4th of November but from our experience with
> Cisco IOS releases we are reluctant to use the latest & greatest
> release.

As a general rule, I'd recommend always running the newest release of
the newest Extended End-Of-Life version of JUNOS. Currently, that is
9.3R4.4 released in August.

10.0 will be out soon, and should be an E-EOL version, but I would say
hold off until R2 or R3 before upgrading unless you absolutely need a
feature only available in the new version.
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

mtinka at globaltransit

Nov 5, 2009, 7:35 PM

Post #5 of 28 (1552 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

On Friday 06 November 2009 02:28:32 am Derick Winkworth
wrote:

> There are some great new features in 10, if you haven't
> already read the release notes...

I'm particularly loving the 'interface-range' feature, which
should be great for the EX-series.

Cheers,

Mark.

Attachments:

signature.asc (0.82 KB)

ras at e-gerbil

Nov 5, 2009, 7:43 PM

Post #6 of 28 (1551 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

On Thu, Nov 05, 2009 at 03:39:32PM +0200, Andrei Radu wrote:
> Hello everybody,
>
> We will be installing our first Juniper MX960 router in the network.
> We were a 100% Cisco shop and this is our first Juniper router. The MX
> will be a P/PE node
> in a pretty standard MPLS backbone network surrounded by Cisco 7600
> routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2
> or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple
> 10GE links is a must. Does anyone have a recommendation for a stable
> JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that
> 10.0 came out on the 4th of November but from our experience with
> Cisco IOS releases we are reluctant to use the latest & greatest
> release.

For MX I'd say 9.3R4 for the conservative (we have loads of experience
running this, 9.3R3+ was the first release without major showstopper
bugs ON in well over a year before it, and 9.3R4 only made it better),
9.4R3 for the middle of the road (we've been running this on many
routers for a few months now, only a relatively modest amount of grief
and probably nothing you'd notice if you have to ask this question),
9.5R3 for the adventuresome (some major bugs in 9.5R1 but we're
currently baking 9.5R3 on a couple production routers and haven't seen
any issues thus far). 9.5 does have some scripting enhancements and
optimizations which are noticable, so if you're big on those it might be
worth a try. I don't currently have the testicular fortitude to try 9.6
outside of the lab (where all the real bugs are found :P), though I am
looking forward to the ISSU support for RSVP. For the super conservative
9.2R4 is relatively light on the pfe and counter bugs (unlike earier
revs of 9.2), but still has quite a few other unfixable issues until you
go to 9.3+ (like netconf and commit scripts will not play together at
all).

I didn't used to put JUNOS in the "you should really wait until R3
before you touch it" category, but given recent history I don't think
it's an irrational stance to take until they re-establish their track
record of putting out non catastrophically broken code on a regular
basis.

--
Richard A Steenbergen <ras [at] e-gerbil> http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

tima at transtelecom

Nov 10, 2009, 11:27 PM

Post #7 of 28 (1519 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
chain of few routers/links with ospf/ldp

bgp session occasionally goes down with notification timeout. Even when there is
no traffic at all and no physical errors

rollback to 9.3r3 helps though

JTAC still not confirmed it, but it easlily can be reprodused in lab

Phil Shafer wrote:
> Derick Winkworth writes:
>> 9.3r4 indeed. Perhaps even 9.4r4 when that comes out.
>
> FWIW: 9.5 has a number of scripting-related features, including
> interactivity and remote RPC access. We've been working to ensure
> that scripting PRs are backported to at least 9.5.
>
> Thanks,
> Phil
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

andreir at gmail

Nov 11, 2009, 12:15 AM

Post #8 of 28 (1516 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Thank you all for all the information, and thank you Richard for
taking the time to elaborate. We're going to use 9.3R4.4 mainly
because of the E-EOL (and that fact that it has all the features we
need :) ).

On Fri, Nov 6, 2009 at 5:43 AM, Richard A Steenbergen <ras [at] e-gerbil> wrote:
> On Thu, Nov 05, 2009 at 03:39:32PM +0200, Andrei Radu wrote:
>> Hello everybody,
>>
>> We will be installing our first Juniper MX960 router in the network.
>> We were a 100% Cisco shop and this is our first Juniper router. The MX
>> will be a P/PE node
>> in a pretty standard MPLS backbone network surrounded by Cisco 7600
>> routers, running OSPF, MPLS/LDP, BGP and PIM sparse-mode, no layer 2
>> or layer 3 VPNs whatsoever. Equal cost load-balancing over multiple
>> 10GE links is a must. Does anyone have a recommendation for a stable
>> JunOS version to run ? I'm currently looking at 9.5 or 9.6, I see that
>> 10.0 came out on the 4th of November but from our experience with
>> Cisco IOS releases we are reluctant to use the latest & greatest
>> release.
>
> For MX I'd say 9.3R4 for the conservative (we have loads of experience
> running this, 9.3R3+ was the first release without major showstopper
> bugs ON in well over a year before it, and 9.3R4 only made it better),
> 9.4R3 for the middle of the road (we've been running this on many
> routers for a few months now, only a relatively modest amount of grief
> and probably nothing you'd notice if you have to ask this question),
> 9.5R3 for the adventuresome (some major bugs in 9.5R1 but we're
> currently baking 9.5R3 on a couple production routers and haven't seen
> any issues thus far). 9.5 does have some scripting enhancements and
> optimizations which are noticable, so if you're big on those it might be
> worth a try. I don't currently have the testicular fortitude to try 9.6
> outside of the lab (where all the real bugs are found :P), though I am
> looking forward to the ISSU support for RSVP. For the super conservative
> 9.2R4 is relatively light on the pfe and counter bugs (unlike earier
> revs of 9.2), but still has quite a few other unfixable issues until you
> go to 9.3+ (like netconf and commit scripts will not play together at
> all).
>
> I didn't used to put JUNOS in the "you should really wait until R3
> before you touch it" category, but given recent history I don't think
> it's an irrational stance to take until they re-establish their track
> record of putting out non catastrophically broken code on a regular
> basis.
>
> --
> Richard A Steenbergen <ras [at] e-gerbil> http://www.e-gerbil.net/ras
> GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
>

--
Andrei

"2+2=5, for extremely large values of 2 !"
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

kszarkowicz at gmail

Nov 11, 2009, 12:36 AM

Post #9 of 28 (1518 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes.

//Krzysztof

-----Original Message-----
From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf Of
Tima Maryin
Sent: Wednesday, 11 November, 2009 8:28
To: juniper-nsp [at] puck
Subject: Re: [j-nsp] MX960 JunOS recommendations

9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
chain of few routers/links with ospf/ldp

bgp session occasionally goes down with notification timeout. Even when there is
no traffic at all and no physical errors

rollback to 9.3r3 helps though

JTAC still not confirmed it, but it easlily can be reprodused in lab

Phil Shafer wrote:
> Derick Winkworth writes:
>> 9.3r4 indeed. Perhaps even 9.4r4 when that comes out.
>
> FWIW: 9.5 has a number of scripting-related features, including
> interactivity and remote RPC access. We've been working to ensure
> that scripting PRs are backported to at least 9.5.
>
> Thanks,
> Phil
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

tima at transtelecom

Nov 11, 2009, 12:57 AM

Post #10 of 28 (1517 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

What did you mean by "inappropriately configured" ?

There are the same mtu settings everywhere and traffic passes quite well.
And ospf session goes up without problems.

And how comes that "inappropriately configured IP and MPLS MTU" work well on
9.3R3.8 ?

Krzysztof Szarkowicz wrote:
> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit nodes.
>
> //Krzysztof
>
> -----Original Message-----
> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf Of
> Tima Maryin
> Sent: Wednesday, 11 November, 2009 8:28
> To: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
> chain of few routers/links with ospf/ldp
>
> bgp session occasionally goes down with notification timeout. Even when there is
> no traffic at all and no physical errors
>
> rollback to 9.3r3 helps though
>
>
> JTAC still not confirmed it, but it easlily can be reprodused in lab
>
>
>
> Phil Shafer wrote:
>> Derick Winkworth writes:
>>> 9.3r4 indeed. Perhaps even 9.4r4 when that comes out.
>> FWIW: 9.5 has a number of scripting-related features, including
>> interactivity and remote RPC access. We've been working to ensure
>> that scripting PRs are backported to at least 9.5.
>>
>> Thanks,
>> Phil
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

kszarkowicz at gmail

Nov 11, 2009, 2:56 AM

Post #11 of 28 (1519 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Let me guess.

Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?

CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely
configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for
MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).

If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR
device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long
packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is
sent.

I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.

Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
terminated on CSCO on one side and JNPR on other side)?

//Krzysztof

-----Original Message-----
From: Tima Maryin [mailto:tima [at] transtelecom]
Sent: Wednesday, 11 November, 2009 9:57
To: kszarkowicz [at] gmail
Cc: juniper-nsp [at] puck
Subject: Re: [j-nsp] MX960 JunOS recommendations

What did you mean by "inappropriately configured" ?

There are the same mtu settings everywhere and traffic passes quite well.
And ospf session goes up without problems.

And how comes that "inappropriately configured IP and MPLS MTU" work well on
9.3R3.8 ?

Krzysztof Szarkowicz wrote:
> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
nodes.
>
> //Krzysztof
>
> -----Original Message-----
> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
Of
> Tima Maryin
> Sent: Wednesday, 11 November, 2009 8:28
> To: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
> chain of few routers/links with ospf/ldp
>
> bgp session occasionally goes down with notification timeout. Even when there is
> no traffic at all and no physical errors
>
> rollback to 9.3r3 helps though
>
>
> JTAC still not confirmed it, but it easlily can be reprodused in lab
>
>
>
> Phil Shafer wrote:
>> Derick Winkworth writes:
>>> 9.3r4 indeed. Perhaps even 9.4r4 when that comes out.
>> FWIW: 9.5 has a number of scripting-related features, including
>> interactivity and remote RPC access. We've been working to ensure
>> that scripting PRs are backported to at least 9.5.
>>
>> Thanks,
>> Phil
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
>
>

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

tima at transtelecom

Nov 11, 2009, 6:11 AM

Post #12 of 28 (1520 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Uhm, i see your point here.
We indeed have cisco - cisco - Jun - Jun setup

My cisco interface mtu = ip mtu = mpls mtu =9000
But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.

On Juniper i set interface mtu = cisco mtu +14 and it works fine!
And! As you say, it reports different mpls mtu value:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 8988
Protocol multiservice, MTU: Unlimited

As far as i understand "default mpls mtu" term (not sure that i _fully_
understand it though) it seems, Juniper supposes 3 labels maximum.
I dont see any reasons for device to drop packets which has 1 or 2 labels and
bigger than mpls mtu, but still ok from interface mtu point ov view.

As per your logic, device should drop all traffic that match such criteria but
it seems only bgp session keepalives and i didn't see any other problems

But still, i made an experiment on Juniper and cisco which has bgp session
between them.

cisco:
#sh mpls interfaces g 0/0 detail | i MTU
MTU = 9000
#sh ip int g 0/0 | i MTU
MTU is 9000 bytes
#sh run int g 0/0
Building configuration...

Current configuration : 212 bytes
!
interface GigabitEthernet0/0
description --- to 7606-2 ---
mtu 9000
ip address 10.3.13.2 255.255.255.0
load-interval 30
duplex full
speed 1000
media-type gbic
no negotiation auto
tag-switching ip
end

If i set mtu 9000 under family mpls and commit it, it looks like this:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 9000
Flags: Is-Primary, User-MTU
Protocol multiservice, MTU: Unlimited

and problem still persists

please let me know if you have any other ideas :)

p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
'default' (=8988) on juniper

Krzysztof Szarkowicz wrote:
> Let me guess.
>
> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>
> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely
> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for
> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>
> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR
> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long
> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is
> sent.
>
> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>
> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
> terminated on CSCO on one side and JNPR on other side)?
>
> //Krzysztof
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom]
> Sent: Wednesday, 11 November, 2009 9:57
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> What did you mean by "inappropriately configured" ?
>
> There are the same mtu settings everywhere and traffic passes quite well.
> And ospf session goes up without problems.
>
> And how comes that "inappropriately configured IP and MPLS MTU" work well on
> 9.3R3.8 ?
>
>
> Krzysztof Szarkowicz wrote:
>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
> nodes.
>> //Krzysztof
>>
>> -----Original Message-----
>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
> Of
>> Tima Maryin
>> Sent: Wednesday, 11 November, 2009 8:28
>> To: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
>> chain of few routers/links with ospf/ldp
>>
>> bgp session occasionally goes down with notification timeout. Even when there is
>> no traffic at all and no physical errors
>>
>> rollback to 9.3r3 helps though
>>
>>
>> JTAC still not confirmed it, but it easlily can be reprodused in lab
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

dwinkworth at att

Nov 11, 2009, 6:59 AM

Post #13 of 28 (1527 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

How about some debugs or traceoptions?

________________________________
From: Tima Maryin <tima [at] transtelecom>
To: kszarkowicz [at] gmail
Cc: juniper-nsp [at] puck
Sent: Wed, November 11, 2009 8:11:56 AM
Subject: Re: [j-nsp] MX960 JunOS recommendations

Uhm, i see your point here.
We indeed have cisco - cisco - Jun - Jun setup

My cisco interface mtu = ip mtu = mpls mtu =9000
But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.

On Juniper i set interface mtu = cisco mtu +14 and it works fine!
And! As you say, it reports different mpls mtu value:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 8988
Protocol multiservice, MTU: Unlimited

As far as i understand "default mpls mtu" term (not sure that i _fully_ understand it though) it seems, Juniper supposes 3 labels maximum.
I dont see any reasons for device to drop packets which has 1 or 2 labels and bigger than mpls mtu, but still ok from interface mtu point ov view.

As per your logic, device should drop all traffic that match such criteria but it seems only bgp session keepalives and i didn't see any other problems

But still, i made an experiment on Juniper and cisco which has bgp session between them.

cisco:
#sh mpls interfaces g 0/0 detail | i MTU
MTU = 9000
#sh ip int g 0/0 | i MTU
MTU is 9000 bytes
#sh run int g 0/0
Building configuration...

Current configuration : 212 bytes
!
interface GigabitEthernet0/0
description --- to 7606-2 ---
mtu 9000
ip address 10.3.13.2 255.255.255.0
load-interval 30
duplex full
speed 1000
media-type gbic
no negotiation auto
tag-switching ip
end

If i set mtu 9000 under family mpls and commit it, it looks like this:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 9000
Flags: Is-Primary, User-MTU
Protocol multiservice, MTU: Unlimited

and problem still persists

please let me know if you have any other ideas :)

p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 'default' (=8988) on juniper

Krzysztof Szarkowicz wrote:
> Let me guess.
>
> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>
> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not explicitely
> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link for
> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>
> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by JNPR
> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502 long
> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message is
> sent.
>
> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>
> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
> terminated on CSCO on one side and JNPR on other side)?
>
> //Krzysztof
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom] Sent: Wednesday, 11 November, 2009 9:57
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> What did you mean by "inappropriately configured" ?
>
> There are the same mtu settings everywhere and traffic passes quite well.
> And ospf session goes up without problems.
>
> And how comes that "inappropriately configured IP and MPLS MTU" work well on 9.3R3.8 ?
>
>
> Krzysztof Szarkowicz wrote:
>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
> nodes.
>> //Krzysztof
>>
>> -----Original Message-----
>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
> Of
>> Tima Maryin
>> Sent: Wednesday, 11 November, 2009 8:28
>> To: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over chain of few routers/links with ospf/ldp
>>
>> bgp session occasionally goes down with notification timeout. Even when there is no traffic at all and no physical errors
>>
>> rollback to 9.3r3 helps though
>>
>>
>> JTAC still not confirmed it, but it easlily can be reprodused in lab
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

sthaug at nethelp

Nov 11, 2009, 7:05 AM

Post #14 of 28 (1523 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

From: Tima Maryin <tima [at] transtelecom>
Subject: Re: [j-nsp] MX960 JunOS recommendations
Date: Wed, 11 Nov 2009 17:11:56 +0300

> Uhm, i see your point here.
> We indeed have cisco - cisco - Jun - Jun setup
>
>
> My cisco interface mtu = ip mtu = mpls mtu =9000
> But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.
>
>
> On Juniper i set interface mtu = cisco mtu +14 and it works fine!
> And! As you say, it reports different mpls mtu value:

Make sure that your IP MTU is the same on both Cisco and Juniper sides.
If you run IS-IS, make sure your CLNS MTU is the same on both Cisco and
Juniper sides.

People have been running interoperable Cisco and Juniper networks for
many years. This is not rocket science.

We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco
IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS
physical Ethernet interfaces. No need to explicitly set IP or CLNS
MTU.

(Also no need to specify anything at all for SDH/SONET interfaces...)

Steinar Haug, Nethelp consulting, sthaug [at] nethelp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

kszarkowicz at gmail

Nov 11, 2009, 11:36 AM

Post #15 of 28 (1570 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
since as per RFC4271, section 4:

The maximum message size is 4096 octets. All implementations are required to support this maximum
message size.

So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
perspective.

The only thing that comes in my mind, that there are some L2 switches in between and there is
something wrong with MTU on those switches. Worth to check.

Could you paste from the log the Notification message generated when the BGP session is tear down?

Thanks,
Krzysztof

-----Original Message-----
From: Tima Maryin [mailto:tima [at] transtelecom]
Sent: Wednesday, 11 November, 2009 15:12
To: kszarkowicz [at] gmail
Cc: juniper-nsp [at] puck
Subject: Re: [j-nsp] MX960 JunOS recommendations

Uhm, i see your point here.
We indeed have cisco - cisco - Jun - Jun setup

My cisco interface mtu = ip mtu = mpls mtu =9000
But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.

On Juniper i set interface mtu = cisco mtu +14 and it works fine!
And! As you say, it reports different mpls mtu value:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 8988
Protocol multiservice, MTU: Unlimited

As far as i understand "default mpls mtu" term (not sure that i _fully_
understand it though) it seems, Juniper supposes 3 labels maximum.
I dont see any reasons for device to drop packets which has 1 or 2 labels and
bigger than mpls mtu, but still ok from interface mtu point ov view.

As per your logic, device should drop all traffic that match such criteria but
it seems only bgp session keepalives and i didn't see any other problems

But still, i made an experiment on Juniper and cisco which has bgp session
between them.

cisco:
#sh mpls interfaces g 0/0 detail | i MTU
MTU = 9000
#sh ip int g 0/0 | i MTU
MTU is 9000 bytes
#sh run int g 0/0
Building configuration...

Current configuration : 212 bytes
!
interface GigabitEthernet0/0
description --- to 7606-2 ---
mtu 9000
ip address 10.3.13.2 255.255.255.0
load-interval 30
duplex full
speed 1000
media-type gbic
no negotiation auto
tag-switching ip
end

If i set mtu 9000 under family mpls and commit it, it looks like this:

> show interfaces xe-1/0/0 | match MTU
Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
None, Source filtering: Disabled,
Protocol inet, MTU: 9000
Protocol mpls, MTU: 9000
Flags: Is-Primary, User-MTU
Protocol multiservice, MTU: Unlimited

and problem still persists

please let me know if you have any other ideas :)

p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
'default' (=8988) on juniper

Krzysztof Szarkowicz wrote:
> Let me guess.
>
> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>
> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
explicitely
> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
for
> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>
> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
JNPR
> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
long
> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
is
> sent.
>
> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>
> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
> terminated on CSCO on one side and JNPR on other side)?
>
> //Krzysztof
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom]
> Sent: Wednesday, 11 November, 2009 9:57
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> What did you mean by "inappropriately configured" ?
>
> There are the same mtu settings everywhere and traffic passes quite well.
> And ospf session goes up without problems.
>
> And how comes that "inappropriately configured IP and MPLS MTU" work well on
> 9.3R3.8 ?
>
>
> Krzysztof Szarkowicz wrote:
>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
> nodes.
>> //Krzysztof
>>
>> -----Original Message-----
>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
> Of
>> Tima Maryin
>> Sent: Wednesday, 11 November, 2009 8:28
>> To: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
>> chain of few routers/links with ospf/ldp
>>
>> bgp session occasionally goes down with notification timeout. Even when there is
>> no traffic at all and no physical errors
>>
>> rollback to 9.3r3 helps though
>>
>>
>> JTAC still not confirmed it, but it easlily can be reprodused in lab

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

lordsith49 at hotmail

Nov 11, 2009, 6:10 PM

Post #16 of 28 (1512 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

I don't know if this will help because it has to deal with gigabit Ethernet interfaces but...

If you set a Cisco device to use frame size of MTU 9000 it does not count the 18 bytes for TCP headers. However, Juniper does count the 18 bytes when you set the MTU. So if the Cisco interface is set to mtu jumbo 9000, the Juniper end must have an mtu of 9018. This only seems to apply for physical interfaces on Juniper. For logical interfaces on Juniper the MTU remains 9000.

Jonathan

> From: kszarkowicz [at] gmail
> To: tima [at] transtelecom
> Date: Wed, 11 Nov 2009 20:36:43 +0100
> CC: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
> since as per RFC4271, section 4:
>
> The maximum message size is 4096 octets. All implementations are required to support this maximum
> message size.
>
> So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
> perspective.
>
> The only thing that comes in my mind, that there are some L2 switches in between and there is
> something wrong with MTU on those switches. Worth to check.
>
> Could you paste from the log the Notification message generated when the BGP session is tear down?
>
> Thanks,
> Krzysztof
>
>
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom]
> Sent: Wednesday, 11 November, 2009 15:12
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> Uhm, i see your point here.
> We indeed have cisco - cisco - Jun - Jun setup
>
>
> My cisco interface mtu = ip mtu = mpls mtu =9000
> But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.
>
>
> On Juniper i set interface mtu = cisco mtu +14 and it works fine!
> And! As you say, it reports different mpls mtu value:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 8988
> Protocol multiservice, MTU: Unlimited
>
>
> As far as i understand "default mpls mtu" term (not sure that i _fully_
> understand it though) it seems, Juniper supposes 3 labels maximum.
> I dont see any reasons for device to drop packets which has 1 or 2 labels and
> bigger than mpls mtu, but still ok from interface mtu point ov view.
>
> As per your logic, device should drop all traffic that match such criteria but
> it seems only bgp session keepalives and i didn't see any other problems
>
>
>
> But still, i made an experiment on Juniper and cisco which has bgp session
> between them.
>
> cisco:
> #sh mpls interfaces g 0/0 detail | i MTU
> MTU = 9000
> #sh ip int g 0/0 | i MTU
> MTU is 9000 bytes
> #sh run int g 0/0
> Building configuration...
>
> Current configuration : 212 bytes
> !
> interface GigabitEthernet0/0
> description --- to 7606-2 ---
> mtu 9000
> ip address 10.3.13.2 255.255.255.0
> load-interval 30
> duplex full
> speed 1000
> media-type gbic
> no negotiation auto
> tag-switching ip
> end
>
>
> If i set mtu 9000 under family mpls and commit it, it looks like this:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 9000
> Flags: Is-Primary, User-MTU
> Protocol multiservice, MTU: Unlimited
>
>
>
> and problem still persists
>
>
>
> please let me know if you have any other ideas :)
>
>
>
> p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
> 'default' (=8988) on juniper
>
>
>
>
>
>
>
>
> Krzysztof Szarkowicz wrote:
> > Let me guess.
> >
> > Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
> >
> > CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
> explicitely
> > configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
> for
> > MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
> >
> > If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
> > device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
> JNPR
> > device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
> long
> > packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
> is
> > sent.
> >
> > I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
> >
> > Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
> > terminated on CSCO on one side and JNPR on other side)?
> >
> > //Krzysztof
> >
> > -----Original Message-----
> > From: Tima Maryin [mailto:tima [at] transtelecom]
> > Sent: Wednesday, 11 November, 2009 9:57
> > To: kszarkowicz [at] gmail
> > Cc: juniper-nsp [at] puck
> > Subject: Re: [j-nsp] MX960 JunOS recommendations
> >
> > What did you mean by "inappropriately configured" ?
> >
> > There are the same mtu settings everywhere and traffic passes quite well.
> > And ospf session goes up without problems.
> >
> > And how comes that "inappropriately configured IP and MPLS MTU" work well on
> > 9.3R3.8 ?
> >
> >
> > Krzysztof Szarkowicz wrote:
> >> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
> > nodes.
> >> //Krzysztof
> >>
> >> -----Original Message-----
> >> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
> > Of
> >> Tima Maryin
> >> Sent: Wednesday, 11 November, 2009 8:28
> >> To: juniper-nsp [at] puck
> >> Subject: Re: [j-nsp] MX960 JunOS recommendations
> >>
> >> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
> >> chain of few routers/links with ospf/ldp
> >>
> >> bgp session occasionally goes down with notification timeout. Even when there is
> >> no traffic at all and no physical errors
> >>
> >> rollback to 9.3r3 helps though
> >>
> >>
> >> JTAC still not confirmed it, but it easlily can be reprodused in lab
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp

_________________________________________________________________
Bing brings you maps, menus, and reviews organized in one place.
http://www.bing.com/search?q=restaurants&form=MFESRP&publ=WLHMTAG&crea=TEXT_MFESRP_Local_MapsMenu_Resturants_1x1
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

swm at emanon

Nov 11, 2009, 6:33 PM

Post #17 of 28 (1550 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

I will assume that you mean ethernet headers instead of "TCP headers" ;)

You should be fine with 9014 though as only the header is counted. The
FCS shouldn't be counted in the total length at all.

Scott

Jonathan Call wrote:
> I don't know if this will help because it has to deal with gigabit Ethernet interfaces but...
>
> If you set a Cisco device to use frame size of MTU 9000 it does not count the 18 bytes for TCP headers. However, Juniper does count the 18 bytes when you set the MTU. So if the Cisco interface is set to mtu jumbo 9000, the Juniper end must have an mtu of 9018. This only seems to apply for physical interfaces on Juniper. For logical interfaces on Juniper the MTU remains 9000.
>
> Jonathan
>
>
>> From: kszarkowicz [at] gmail
>> To: tima [at] transtelecom
>> Date: Wed, 11 Nov 2009 20:36:43 +0100
>> CC: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
>> since as per RFC4271, section 4:
>>
>> The maximum message size is 4096 octets. All implementations are required to support this maximum
>> message size.
>>
>> So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
>> perspective.
>>
>> The only thing that comes in my mind, that there are some L2 switches in between and there is
>> something wrong with MTU on those switches. Worth to check.
>>
>> Could you paste from the log the Notification message generated when the BGP session is tear down?
>>
>> Thanks,
>> Krzysztof
>>
>>
>>
>> -----Original Message-----
>> From: Tima Maryin [mailto:tima [at] transtelecom]
>> Sent: Wednesday, 11 November, 2009 15:12
>> To: kszarkowicz [at] gmail
>> Cc: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> Uhm, i see your point here.
>> We indeed have cisco - cisco - Jun - Jun setup
>>
>>
>> My cisco interface mtu = ip mtu = mpls mtu =9000
>> But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.
>>
>>
>> On Juniper i set interface mtu = cisco mtu +14 and it works fine!
>> And! As you say, it reports different mpls mtu value:
>>
>> > show interfaces xe-1/0/0 | match MTU
>> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
>> None, Source filtering: Disabled,
>> Protocol inet, MTU: 9000
>> Protocol mpls, MTU: 8988
>> Protocol multiservice, MTU: Unlimited
>>
>>
>> As far as i understand "default mpls mtu" term (not sure that i _fully_
>> understand it though) it seems, Juniper supposes 3 labels maximum.
>> I dont see any reasons for device to drop packets which has 1 or 2 labels and
>> bigger than mpls mtu, but still ok from interface mtu point ov view.
>>
>> As per your logic, device should drop all traffic that match such criteria but
>> it seems only bgp session keepalives and i didn't see any other problems
>>
>>
>>
>> But still, i made an experiment on Juniper and cisco which has bgp session
>> between them.
>>
>> cisco:
>> #sh mpls interfaces g 0/0 detail | i MTU
>> MTU = 9000
>> #sh ip int g 0/0 | i MTU
>> MTU is 9000 bytes
>> #sh run int g 0/0
>> Building configuration...
>>
>> Current configuration : 212 bytes
>> !
>> interface GigabitEthernet0/0
>> description --- to 7606-2 ---
>> mtu 9000
>> ip address 10.3.13.2 255.255.255.0
>> load-interval 30
>> duplex full
>> speed 1000
>> media-type gbic
>> no negotiation auto
>> tag-switching ip
>> end
>>
>>
>> If i set mtu 9000 under family mpls and commit it, it looks like this:
>>
>> > show interfaces xe-1/0/0 | match MTU
>> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
>> None, Source filtering: Disabled,
>> Protocol inet, MTU: 9000
>> Protocol mpls, MTU: 9000
>> Flags: Is-Primary, User-MTU
>> Protocol multiservice, MTU: Unlimited
>>
>>
>>
>> and problem still persists
>>
>>
>>
>> please let me know if you have any other ideas :)
>>
>>
>>
>> p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
>> 'default' (=8988) on juniper
>>
>>
>>
>>
>>
>>
>>
>>
>> Krzysztof Szarkowicz wrote:
>>
>>> Let me guess.
>>>
>>> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>>>
>>> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
>>>
>> explicitely
>>
>>> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
>>>
>> for
>>
>>> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>>>
>>> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
>>> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
>>>
>> JNPR
>>
>>> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
>>>
>> long
>>
>>> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
>>>
>> is
>>
>>> sent.
>>>
>>> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>>>
>>> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
>>> terminated on CSCO on one side and JNPR on other side)?
>>>
>>> //Krzysztof
>>>
>>> -----Original Message-----
>>> From: Tima Maryin [mailto:tima [at] transtelecom]
>>> Sent: Wednesday, 11 November, 2009 9:57
>>> To: kszarkowicz [at] gmail
>>> Cc: juniper-nsp [at] puck
>>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>>
>>> What did you mean by "inappropriately configured" ?
>>>
>>> There are the same mtu settings everywhere and traffic passes quite well.
>>> And ospf session goes up without problems.
>>>
>>> And how comes that "inappropriately configured IP and MPLS MTU" work well on
>>> 9.3R3.8 ?
>>>
>>>
>>> Krzysztof Szarkowicz wrote:
>>>
>>>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
>>>>
>>> nodes.
>>>
>>>> //Krzysztof
>>>>
>>>> -----Original Message-----
>>>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
>>>>
>>> Of
>>>
>>>> Tima Maryin
>>>> Sent: Wednesday, 11 November, 2009 8:28
>>>> To: juniper-nsp [at] puck
>>>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>>>
>>>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
>>>> chain of few routers/links with ospf/ldp
>>>>
>>>> bgp session occasionally goes down with notification timeout. Even when there is
>>>> no traffic at all and no physical errors
>>>>
>>>> rollback to 9.3r3 helps though
>>>>
>>>>
>>>> JTAC still not confirmed it, but it easlily can be reprodused in lab
>>>>
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp [at] puck
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
>
> _________________________________________________________________
> Bing brings you maps, menus, and reviews organized in one place.
> http://www.bing.com/search?q=restaurants&form=MFESRP&publ=WLHMTAG&crea=TEXT_MFESRP_Local_MapsMenu_Resturants_1x1
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

pekkas at netcore

Nov 11, 2009, 11:39 PM

Post #18 of 28 (1511 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

On Wed, 11 Nov 2009, sthaug [at] nethelp wrote:
> We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco
> IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS
> physical Ethernet interfaces. No need to explicitly set IP or CLNS
> MTU.

Except when you enable VLANs on Ethernet, you need to crank it up to
4488.. A thing to keep in mind and/or to monitor using scripts.

--
Pekka Savola "You each name yourselves king, yet the
Netcore Oy kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

wjackson at sapphire

Nov 12, 2009, 12:03 AM

Post #19 of 28 (1513 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Hi do you have a PR Number for this issue?

-----Original Message-----
From: juniper-nsp-bounces [at] puck
[mailto:juniper-nsp-bounces [at] puck] On Behalf Of Tima Maryin
Sent: 11 November 2009 08:28
To: juniper-nsp [at] puck
Subject: Re: [j-nsp] MX960 JunOS recommendations

9.3R4.4 has a nasty bug which occures in setup when you have bgp session
over
chain of few routers/links with ospf/ldp

bgp session occasionally goes down with notification timeout. Even when
there is
no traffic at all and no physical errors

rollback to 9.3r3 helps though

JTAC still not confirmed it, but it easlily can be reprodused in lab

Phil Shafer wrote:
> Derick Winkworth writes:
>> 9.3r4 indeed. Perhaps even 9.4r4 when that comes out.
>
> FWIW: 9.5 has a number of scripting-related features, including
> interactivity and remote RPC access. We've been working to ensure
> that scripting PRs are backported to at least 9.5.
>
> Thanks,
> Phil
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

tima at transtelecom

Nov 12, 2009, 12:07 AM

Post #20 of 28 (1510 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

First of all thanks to all who cares :)
I'll reply one by one

Derick Winkworth wrote:
> How about some debugs or traceoptions?
>
>

traceoptions at last Jun says that box dosen't receive bgp notifications some
times. haven't tried any more yet

sthaug [at] nethelp wrote:
>
> Make sure that your IP MTU is the same on both Cisco and Juniper sides.
> If you run IS-IS, make sure your CLNS MTU is the same on both Cisco and
> Juniper sides.

IP mtu are the same, otherwise ospf do not come up

> People have been running interoperable Cisco and Juniper networks for
> many years. This is not rocket science.

Yeah, we installed several Juns into our network several months ago and this is
the only problem which we couldn't solve and rolled back to previous software

(well i do not count some rpd crashes on box with aggregated interfaces which we
can avoid for now. jtac evetually said that its PR439627. I can't read this
hidden PR, but its supposed to be fixed in 10.x and 9.3Rnextrelease )

Krzysztof Szarkowicz wrote:
> With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
> since as per RFC4271, section 4:
>
> The maximum message size is 4096 octets. All implementations are required to support this maximum
> message size.
>
> So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
> perspective.
>
> The only thing that comes in my mind, that there are some L2 switches in between and there is
> something wrong with MTU on those switches. Worth to check.

There are no switches between them
its
7301-geoptic-7606-tengig-t1600-tengig-mx960

Its lab setup. On the real network it was slightly different, but actually its
the same from this problem point of view

> Could you paste from the log the Notification message generated when the BGP session is tear down?

I didn't find any dependance from interfaces load or anything else.
It can be 3-4 gig load (like it was on real network) or empty (like its in
lab), bgp session may drop once per minute or stay up for 30 - 60 mins.
Cisco can be either GSR or 7301, Juniper can be mx or T.

There is nothing special in logs.
Thats the one from mx960:
Nov 12 06:18:31 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 307818660 snd_nxt: 307818660 snd_wnd: 16230
rcv_nxt: 614682635 rcv_adv: 614699019, hold timer 0
Nov 12 06:20:48 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 1301747029 snd_nxt: 1301747029 snd_wnd: 16211
rcv_nxt: 732160622 rcv_adv: 732177006, hold timer 0
Nov 12 06:22:53 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 2024212109 snd_nxt: 2024212109 snd_wnd: 16230
rcv_nxt: 3950965686 rcv_adv: 3950982070, hold timer 0
Nov 12 06:24:56 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 2363347692 snd_nxt: 2363347692 snd_wnd: 16230
rcv_nxt: 1449362513 rcv_adv: 1449378897, hold timer 0
Nov 12 06:59:09 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3704141975 snd_nxt: 3704141975 snd_wnd: 15985
rcv_nxt: 2261397920 rcv_adv: 2261414304, hold timer 0
Nov 12 07:01:19 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 1379635866 snd_nxt: 1379635866 snd_wnd: 16230
rcv_nxt: 612357774 rcv_adv: 612374158, hold timer 0
Nov 12 07:04:06 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3377139997 snd_nxt: 3377139997 snd_wnd: 16211
rcv_nxt: 544711184 rcv_adv: 544727568, hold timer 0
Nov 12 07:20:37 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3633708680 snd_nxt: 3633708680 snd_wnd: 16175
rcv_nxt: 1216109422 rcv_adv: 1216125806, hold timer 0
Nov 12 07:22:54 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 4034247055 snd_nxt: 4034247055 snd_wnd: 16211
rcv_nxt: 2010186633 rcv_adv: 2010203017, hold timer 0
Nov 12 07:25:00 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 38
rcvcc: 0 TCP state: 4, snd_una: 3122195868 snd_nxt: 3122195868 snd_wnd: 16268
rcv_nxt: 209999860 rcv_adv: 210016244, hold timer 0

>
> Thanks,
> Krzysztof
>
>
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom]
> Sent: Wednesday, 11 November, 2009 15:12
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> Uhm, i see your point here.
> We indeed have cisco - cisco - Jun - Jun setup
>
>
> My cisco interface mtu = ip mtu = mpls mtu =9000
> But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.
>
>
> On Juniper i set interface mtu = cisco mtu +14 and it works fine!
> And! As you say, it reports different mpls mtu value:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 8988
> Protocol multiservice, MTU: Unlimited
>
>
> As far as i understand "default mpls mtu" term (not sure that i _fully_
> understand it though) it seems, Juniper supposes 3 labels maximum.
> I dont see any reasons for device to drop packets which has 1 or 2 labels and
> bigger than mpls mtu, but still ok from interface mtu point ov view.
>
> As per your logic, device should drop all traffic that match such criteria but
> it seems only bgp session keepalives and i didn't see any other problems
>
>
>
> But still, i made an experiment on Juniper and cisco which has bgp session
> between them.
>
> cisco:
> #sh mpls interfaces g 0/0 detail | i MTU
> MTU = 9000
> #sh ip int g 0/0 | i MTU
> MTU is 9000 bytes
> #sh run int g 0/0
> Building configuration...
>
> Current configuration : 212 bytes
> !
> interface GigabitEthernet0/0
> description --- to 7606-2 ---
> mtu 9000
> ip address 10.3.13.2 255.255.255.0
> load-interval 30
> duplex full
> speed 1000
> media-type gbic
> no negotiation auto
> tag-switching ip
> end
>
>
> If i set mtu 9000 under family mpls and commit it, it looks like this:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 9000
> Flags: Is-Primary, User-MTU
> Protocol multiservice, MTU: Unlimited
>
>
>
> and problem still persists
>
>
>
> please let me know if you have any other ideas :)
>
>
>
> p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
> 'default' (=8988) on juniper
>
>
>
>
>
>
>
>
> Krzysztof Szarkowicz wrote:
>> Let me guess.
>>
>> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>>
>> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
> explicitely
>> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
> for
>> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>>
>> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
>> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
> JNPR
>> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
> long
>> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
> is
>> sent.
>>
>> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>>
>> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
>> terminated on CSCO on one side and JNPR on other side)?
>>
>> //Krzysztof
>>
>> -----Original Message-----
>> From: Tima Maryin [mailto:tima [at] transtelecom]
>> Sent: Wednesday, 11 November, 2009 9:57
>> To: kszarkowicz [at] gmail
>> Cc: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> What did you mean by "inappropriately configured" ?
>>
>> There are the same mtu settings everywhere and traffic passes quite well.
>> And ospf session goes up without problems.
>>
>> And how comes that "inappropriately configured IP and MPLS MTU" work well on
>> 9.3R3.8 ?
>>
>>
>> Krzysztof Szarkowicz wrote:
>>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
>> nodes.
>>> //Krzysztof
>>>
>>> -----Original Message-----
>>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
>> Of
>>> Tima Maryin
>>> Sent: Wednesday, 11 November, 2009 8:28
>>> To: juniper-nsp [at] puck
>>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>>
>>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
>>> chain of few routers/links with ospf/ldp
>>>
>>> bgp session occasionally goes down with notification timeout. Even when there is
>>> no traffic at all and no physical errors
>>>
>>> rollback to 9.3r3 helps though
>>>
>>>
>>> JTAC still not confirmed it, but it easlily can be reprodused in lab
>
>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

sthaug at nethelp

Nov 12, 2009, 12:12 AM

Post #21 of 28 (1523 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

> > We have standardized on 4470 byte MTU. This means mtu 4470 on Cisco
> > IOS physical Ethernet interfaces, and MTU 4484 on Juniper JunOS
> > physical Ethernet interfaces. No need to explicitly set IP or CLNS
> > MTU.
>
> Except when you enable VLANs on Ethernet, you need to crank it up to
> 4488.. A thing to keep in mind and/or to monitor using scripts.

Absolutely. We use quite a bit of dual tagging on Ethernet, so then we
need to crank it up to 4492. But all our backbone links are 4484 on the
Juniper side.

Steinar Haug, Nethelp consulting, sthaug [at] nethelp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

tima at transtelecom

Nov 12, 2009, 1:03 AM

Post #22 of 28 (1508 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

We have a case but i think they didn't make PR yet

William Jackson wrote:
>
> Hi do you have a PR Number for this issue?
>
>
> -----Original Message-----
> From: juniper-nsp-bounces [at] puck
> [mailto:juniper-nsp-bounces [at] puck] On Behalf Of Tima Maryin
> Sent: 11 November 2009 08:28
> To: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session
> over
> chain of few routers/links with ospf/ldp
>
> bgp session occasionally goes down with notification timeout. Even when
> there is
> no traffic at all and no physical errors
>
> rollback to 9.3r3 helps though
>
>
> JTAC still not confirmed it, but it easlily can be reprodused in lab

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

kszarkowicz at gmail

Nov 12, 2009, 4:45 AM

Post #23 of 28 (1525 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

JNPR send notification because of hold timer expired (meaning no BGP messages are received from the
neighbor) - this is correct behavior from BGP perspective.

Do you have logs on CSCO side for the same event? I assume you will see retransmission of UPDATE
message (not Keepalive message). This Update message is dropped somewhere on the path between CSCO
and JNPR. And CSCO retrsmits this message. Since UPDATE message is sent within Keepalive timer, no
Keepalives are sent.

The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs
somewhere in between.

You have to figure out (debugs, traceoptions, tcpdumps, whats ever) which device on the path is
dropping.

//Krzysztof

-----Original Message-----
From: Tima Maryin [mailto:tima [at] transtelecom]
Sent: Thursday, 12 November, 2009 9:07
To: kszarkowicz [at] gmail
Cc: juniper-nsp [at] puck
Subject: Re: [j-nsp] MX960 JunOS recommendations

First of all thanks to all who cares :)
I'll reply one by one

Derick Winkworth wrote:
> How about some debugs or traceoptions?
>
>

traceoptions at last Jun says that box dosen't receive bgp notifications some
times. haven't tried any more yet

sthaug [at] nethelp wrote:
>
> Make sure that your IP MTU is the same on both Cisco and Juniper sides.
> If you run IS-IS, make sure your CLNS MTU is the same on both Cisco and
> Juniper sides.

IP mtu are the same, otherwise ospf do not come up

> People have been running interoperable Cisco and Juniper networks for
> many years. This is not rocket science.

Yeah, we installed several Juns into our network several months ago and this is
the only problem which we couldn't solve and rolled back to previous software

(well i do not count some rpd crashes on box with aggregated interfaces which we
can avoid for now. jtac evetually said that its PR439627. I can't read this
hidden PR, but its supposed to be fixed in 10.x and 9.3Rnextrelease )

Krzysztof Szarkowicz wrote:
> With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
> since as per RFC4271, section 4:
>
> The maximum message size is 4096 octets. All implementations are required to support this maximum
> message size.
>
> So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
> perspective.
>
> The only thing that comes in my mind, that there are some L2 switches in between and there is
> something wrong with MTU on those switches. Worth to check.

There are no switches between them
its
7301-geoptic-7606-tengig-t1600-tengig-mx960

Its lab setup. On the real network it was slightly different, but actually its
the same from this problem point of view

> Could you paste from the log the Notification message generated when the BGP session is tear down?

I didn't find any dependance from interfaces load or anything else.
It can be 3-4 gig load (like it was on real network) or empty (like its in
lab), bgp session may drop once per minute or stay up for 30 - 60 mins.
Cisco can be either GSR or 7301, Juniper can be mx or T.

There is nothing special in logs.
Thats the one from mx960:
Nov 12 06:18:31 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 307818660 snd_nxt: 307818660 snd_wnd: 16230
rcv_nxt: 614682635 rcv_adv: 614699019, hold timer 0
Nov 12 06:20:48 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 1301747029 snd_nxt: 1301747029 snd_wnd: 16211
rcv_nxt: 732160622 rcv_adv: 732177006, hold timer 0
Nov 12 06:22:53 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 2024212109 snd_nxt: 2024212109 snd_wnd: 16230
rcv_nxt: 3950965686 rcv_adv: 3950982070, hold timer 0
Nov 12 06:24:56 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 2363347692 snd_nxt: 2363347692 snd_wnd: 16230
rcv_nxt: 1449362513 rcv_adv: 1449378897, hold timer 0
Nov 12 06:59:09 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3704141975 snd_nxt: 3704141975 snd_wnd: 15985
rcv_nxt: 2261397920 rcv_adv: 2261414304, hold timer 0
Nov 12 07:01:19 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 1379635866 snd_nxt: 1379635866 snd_wnd: 16230
rcv_nxt: 612357774 rcv_adv: 612374158, hold timer 0
Nov 12 07:04:06 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3377139997 snd_nxt: 3377139997 snd_wnd: 16211
rcv_nxt: 544711184 rcv_adv: 544727568, hold timer 0
Nov 12 07:20:37 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 3633708680 snd_nxt: 3633708680 snd_wnd: 16175
rcv_nxt: 1216109422 rcv_adv: 1216125806, hold timer 0
Nov 12 07:22:54 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 0
rcvcc: 0 TCP state: 4, snd_una: 4034247055 snd_nxt: 4034247055 snd_wnd: 16211
rcv_nxt: 2010186633 rcv_adv: 2010203017, hold timer 0
Nov 12 07:25:00 mskl04ra rpd[1080]: bgp_hold_timeout:3571: NOTIFICATION sent to
10.136.0.13 (Internal AS 20485): code 4 (Hold Timer Expired Error), Reason:
holdtime expired for 10.136.0.13 (Internal AS 20485), socket buffer sndcc: 38
rcvcc: 0 TCP state: 4, snd_una: 3122195868 snd_nxt: 3122195868 snd_wnd: 16268
rcv_nxt: 209999860 rcv_adv: 210016244, hold timer 0

>
> Thanks,
> Krzysztof
>
>
>
> -----Original Message-----
> From: Tima Maryin [mailto:tima [at] transtelecom]
> Sent: Wednesday, 11 November, 2009 15:12
> To: kszarkowicz [at] gmail
> Cc: juniper-nsp [at] puck
> Subject: Re: [j-nsp] MX960 JunOS recommendations
>
> Uhm, i see your point here.
> We indeed have cisco - cisco - Jun - Jun setup
>
>
> My cisco interface mtu = ip mtu = mpls mtu =9000
> But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.
>
>
> On Juniper i set interface mtu = cisco mtu +14 and it works fine!
> And! As you say, it reports different mpls mtu value:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 8988
> Protocol multiservice, MTU: Unlimited
>
>
> As far as i understand "default mpls mtu" term (not sure that i _fully_
> understand it though) it seems, Juniper supposes 3 labels maximum.
> I dont see any reasons for device to drop packets which has 1 or 2 labels and
> bigger than mpls mtu, but still ok from interface mtu point ov view.
>
> As per your logic, device should drop all traffic that match such criteria but
> it seems only bgp session keepalives and i didn't see any other problems
>
>
>
> But still, i made an experiment on Juniper and cisco which has bgp session
> between them.
>
> cisco:
> #sh mpls interfaces g 0/0 detail | i MTU
> MTU = 9000
> #sh ip int g 0/0 | i MTU
> MTU is 9000 bytes
> #sh run int g 0/0
> Building configuration...
>
> Current configuration : 212 bytes
> !
> interface GigabitEthernet0/0
> description --- to 7606-2 ---
> mtu 9000
> ip address 10.3.13.2 255.255.255.0
> load-interval 30
> duplex full
> speed 1000
> media-type gbic
> no negotiation auto
> tag-switching ip
> end
>
>
> If i set mtu 9000 under family mpls and commit it, it looks like this:
>
> > show interfaces xe-1/0/0 | match MTU
> Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback:
> None, Source filtering: Disabled,
> Protocol inet, MTU: 9000
> Protocol mpls, MTU: 9000
> Flags: Is-Primary, User-MTU
> Protocol multiservice, MTU: Unlimited
>
>
>
> and problem still persists
>
>
>
> please let me know if you have any other ideas :)
>
>
>
> p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it
> 'default' (=8988) on juniper
>
>
>
>
>
>
>
>
> Krzysztof Szarkowicz wrote:
>> Let me guess.
>>
>> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
>>
>> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
> explicitely
>> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
> for
>> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
>>
>> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the
CSCO
>> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
> JNPR
>> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
> long
>> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
> is
>> sent.
>>
>> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
>>
>> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
>> terminated on CSCO on one side and JNPR on other side)?
>>
>> //Krzysztof
>>
>> -----Original Message-----
>> From: Tima Maryin [mailto:tima [at] transtelecom]
>> Sent: Wednesday, 11 November, 2009 9:57
>> To: kszarkowicz [at] gmail
>> Cc: juniper-nsp [at] puck
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> What did you mean by "inappropriately configured" ?
>>
>> There are the same mtu settings everywhere and traffic passes quite well.
>> And ospf session goes up without problems.
>>
>> And how comes that "inappropriately configured IP and MPLS MTU" work well on
>> 9.3R3.8 ?
>>
>>
>> Krzysztof Szarkowicz wrote:
>>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
>> nodes.
>>> //Krzysztof
>>>
>>> -----Original Message-----
>>> From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf
>> Of
>>> Tima Maryin
>>> Sent: Wednesday, 11 November, 2009 8:28
>>> To: juniper-nsp [at] puck
>>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>>
>>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over
>>> chain of few routers/links with ospf/ldp
>>>
>>> bgp session occasionally goes down with notification timeout. Even when there is
>>> no traffic at all and no physical errors
>>>
>>> rollback to 9.3r3 helps though
>>>
>>>
>>> JTAC still not confirmed it, but it easlily can be reprodused in lab
>
>
>

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

ka at mork

Nov 12, 2009, 5:33 AM

Post #24 of 28 (1502 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

On Thu, Nov 12, 2009 at 01:45:22PM +0100, Krzysztof Szarkowicz wrote:
>
> The most common cause of dropping is mismatch of MPLS MTU, or L2 device with misconfigured MTUs
> somewhere in between.

Are you sur you mean MPLS MTU?

There is only one BGP session between a set of peers, even if you have
several address families. I always thought this used a regular L3
protocol like IP (could be exhanged with v6 in the future).

Kari
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

jof at thejof

Nov 12, 2009, 10:53 AM

Post #25 of 28 (1499 views)
Permalink

Re: MX960 JunOS recommendations [In reply to]

Excerpts from sthaug's message of Thu Nov 12 00:12:16 -0800 2009:
> Absolutely. We use quite a bit of dual tagging on Ethernet, so then we
> need to crank it up to 4492. But all our backbone links are 4484 on the
> Juniper side.

Is there a reason not to use 9000-bytes everywhere (accounting for
bridges/interfaces that count/don't count ethernet headers)?

--j
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads