Mailing List Archive: nsp: juniper

Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations?

Index | Next | Previous | View Threaded

chmorl at wm

Aug 17, 2012, 8:08 AM

Post #1 of 11 (1240 views)
Permalink

Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations?

We have had the unfortunate experience of having users plug in small
mini-switches into our network that have the capability of filtering out
(by-default) BPDUs while allowing other traffic through. The nightmare
situation is when a user plugs in such a switch accidentally into two of
our EX switches. Traffic will loop through the miscreant switch between
the two EXs and without BPDUs it just looks like MAC addresses keep moving
between the real source and the two EXs.

In an MX environment running VPLS, this problem can happen easily as there
are no BPDUs even to protect against loops in VPLS, particularly when your
VPLS domain ties into a Spanning Tree domain downstream where your
potential miscreant switch may appear.

I am curious to know if anyone has come up with strategies to kill these
loops for EXs running Spanning Tree and/or MXs running VPLS.
Rate-limiting may help, but it doesn't kill loops completely. I am
looking for ways to detect lots of MAC address moves (without polling for
them) and blocking those interfaces involved when those MAC moves exceed a
certain threshold via some trigger mechanism.

Assume Junos 10.4R10 or more recent.

Clarke Morledge
College of William and Mary
Information Technology - Network Engineering
Jones Hall (Room 18)
Williamsburg VA 23187
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

ab at lists

Aug 17, 2012, 8:22 AM

Post #2 of 11 (1210 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

Hi,

...on Fri, Aug 17, 2012 at 11:08:53AM -0400, Clarke Morledge wrote:
> switch accidentally into two of our EX switches. Traffic will loop
> through the miscreant switch between the two EXs and without BPDUs
> it just looks like MAC addresses keep moving between the real
> source and the two EXs.

Not that I've used it myself yet, but the aptly named MAC Move Limiting
might help here, though usefulness might depend on the actual topology::

http://www.juniper.net/techpubs/en_US/junos10.4/topics/concept/port-security-mac-limiting-and-mac-move-limiting.html

Alex.

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

JTyler at fiberutilities

Aug 17, 2012, 8:26 AM

Post #3 of 11 (1216 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

Quick google for "VPLS Multihoming" found me this:

http://www.juniper.net/techpubs/en_US/junos9.6/information-products/topic-collections/feature-guide/vpls-multihoming-bgp-signaling-solutions.html

Jensen Tyler
Sr Engineering Manager
Fiberutilities Group, LLC

-----Original Message-----
From: juniper-nsp-bounces [at] puck [mailto:juniper-nsp-bounces [at] puck] On Behalf Of Clarke Morledge
Sent: Friday, August 17, 2012 10:09 AM
To: juniper-nsp [at] puck
Subject: [j-nsp] Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations?

We have had the unfortunate experience of having users plug in small mini-switches into our network that have the capability of filtering out
(by-default) BPDUs while allowing other traffic through. The nightmare situation is when a user plugs in such a switch accidentally into two of our EX switches. Traffic will loop through the miscreant switch between the two EXs and without BPDUs it just looks like MAC addresses keep moving between the real source and the two EXs.

In an MX environment running VPLS, this problem can happen easily as there are no BPDUs even to protect against loops in VPLS, particularly when your VPLS domain ties into a Spanning Tree domain downstream where your potential miscreant switch may appear.

I am curious to know if anyone has come up with strategies to kill these loops for EXs running Spanning Tree and/or MXs running VPLS.
Rate-limiting may help, but it doesn't kill loops completely. I am looking for ways to detect lots of MAC address moves (without polling for
them) and blocking those interfaces involved when those MAC moves exceed a certain threshold via some trigger mechanism.

Assume Junos 10.4R10 or more recent.

Clarke Morledge
College of William and Mary
Information Technology - Network Engineering Jones Hall (Room 18) Williamsburg VA 23187 _______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

chmorl at wm

Aug 17, 2012, 8:45 AM

Post #4 of 11 (1213 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

On Fri, 17 Aug 2012, Jensen Tyler wrote:

> Quick google for "VPLS Multihoming" found me this:
>
> http://www.juniper.net/techpubs/en_US/junos9.6/information-products/topic-collections/feature-guide/vpls-multihoming-bgp-signaling-solutions.html
>
> Jensen Tyler
> Sr Engineering Manager
> Fiberutilities Group, LLC

Jensen,

VPLS multihoming assumes you are intentionally building out a loop-free
VPLS domain. My situation is when you have a downstream customer who
unintentionally introduces a loop in their Layer2 domain that causes MAC
learning table thrashing back inside your VPLS instance.

Thanks for the pointer though.

Clarke Morledge
College of William and Mary
Information Technology - Network Engineering
Jones Hall (Room 18)
Williamsburg VA 23187
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

wayne at tuckerlabs

Aug 17, 2012, 9:06 AM

Post #5 of 11 (1214 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

On Fri, Aug 17, 2012 at 8:08 AM, Clarke Morledge <chmorl [at] wm> wrote:
> We have had the unfortunate experience of having users plug in small
> mini-switches into our network that have the capability of filtering out
> (by-default) BPDUs while allowing other traffic through. The nightmare
> situation is when a user plugs in such a switch accidentally into two of our
> EX switches. Traffic will loop through the miscreant switch between the two
> EXs and without BPDUs it just looks like MAC addresses keep moving between
> the real source and the two EXs.

This is probably not the answer you're looking for, but the best
solution is to not bridge to your access switches. Everything in the
EX series is capable of routing, so why not take advantage of that
functionality?

Barring that, your options are: storm control, MAC limiting, and MAC
move limiting.

I've never found storm control to be that useful. It reduces the
volume of frames but usually not enough to cancel out all of the
negative effects.

MAC limiting with a reasonable MAC limit on a port can cause the port
to be disabled if too many addresses are seen coming from said port.

MAC move limiting is configured per VLAN. It can detect a layer 2
loop with a smaller number of MAC addresses than MAC limiting would,
but my concern has always been that (as far as I can tell) there's no
way to determine which interface would end up being disabled - it
would be bad to have it pick a trunk between your core switches
instead of the trunk to the IDF.

None of these will ever be as effective as routing.

> In an MX environment running VPLS, this problem can happen easily as there
> are no BPDUs even to protect against loops in VPLS, particularly when your
> VPLS domain ties into a Spanning Tree domain downstream where your potential
> miscreant switch may appear.

I believe there was a thread on here within the last month about an
event script for the MX platform that would do just that. Going back
to the first section, though, you should think thrice before doing
VPLS - Ivan PepeInjak has some good articles about the hazards of L2
across your wan on his blog.

HTH

:w
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

moua0100 at umn

Aug 17, 2012, 2:04 PM

Post #6 of 11 (1198 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

What about TRILL? Not sure if Juniper has jumped on the TRILL bandwagon yet.

--
Regards,
Ge Moua
Univ of Minn Alumnus
--

On 08/17/2012 11:06 AM, Wayne Tucker wrote:
> On Fri, Aug 17, 2012 at 8:08 AM, Clarke Morledge<chmorl [at] wm> wrote:
>> We have had the unfortunate experience of having users plug in small
>> mini-switches into our network that have the capability of filtering out
>> (by-default) BPDUs while allowing other traffic through. The nightmare
>> situation is when a user plugs in such a switch accidentally into two of our
>> EX switches. Traffic will loop through the miscreant switch between the two
>> EXs and without BPDUs it just looks like MAC addresses keep moving between
>> the real source and the two EXs.
> This is probably not the answer you're looking for, but the best
> solution is to not bridge to your access switches. Everything in the
> EX series is capable of routing, so why not take advantage of that
> functionality?
>
> Barring that, your options are: storm control, MAC limiting, and MAC
> move limiting.
>
> I've never found storm control to be that useful. It reduces the
> volume of frames but usually not enough to cancel out all of the
> negative effects.
>
> MAC limiting with a reasonable MAC limit on a port can cause the port
> to be disabled if too many addresses are seen coming from said port.
>
> MAC move limiting is configured per VLAN. It can detect a layer 2
> loop with a smaller number of MAC addresses than MAC limiting would,
> but my concern has always been that (as far as I can tell) there's no
> way to determine which interface would end up being disabled - it
> would be bad to have it pick a trunk between your core switches
> instead of the trunk to the IDF.
>
> None of these will ever be as effective as routing.
>
>
>> In an MX environment running VPLS, this problem can happen easily as there
>> are no BPDUs even to protect against loops in VPLS, particularly when your
>> VPLS domain ties into a Spanning Tree domain downstream where your potential
>> miscreant switch may appear.
> I believe there was a thread on here within the last month about an
> event script for the MX platform that would do just that. Going back
> to the first section, though, you should think thrice before doing
> VPLS - Ivan PepeInjak has some good articles about the hazards of L2
> across your wan on his blog.
>
> HTH
>
> :w
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

juniperdude at gmail

Aug 17, 2012, 4:20 PM

Post #7 of 11 (1197 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

Hi Clarke,

We pass through BPDUs through VPLS the MX'es- but yes, miscreant users / switches will always be a problem.

We do the following to every customer-facing VPLS instance, but only #3 would help you here:

1. Mac Limiting per VPLS Interface (100) (i.e per 'site')
2. Mac Limiting per VPLS (500)
3. Limit Broadcast/Unknown Unicast/Multicast Traffic (5 Mbit) into the VPLS

You can put on an input firewall filter which calls a 5 Mbit policer at [routing instances <vpls-name> forwarding-options family vpls <>] to start limiting this type of traffic into the 'bridge domain' at any time.

- CK.

On 18/08/2012, at 1:08 AM, Clarke Morledge <chmorl [at] wm> wrote:

> We have had the unfortunate experience of having users plug in small mini-switches into our network that have the capability of filtering out (by-default) BPDUs while allowing other traffic through. The nightmare situation is when a user plugs in such a switch accidentally into two of our EX switches. Traffic will loop through the miscreant switch between the two EXs and without BPDUs it just looks like MAC addresses keep moving between the real source and the two EXs.
>
> In an MX environment running VPLS, this problem can happen easily as there are no BPDUs even to protect against loops in VPLS, particularly when your VPLS domain ties into a Spanning Tree domain downstream where your potential miscreant switch may appear.
>
> I am curious to know if anyone has come up with strategies to kill these loops for EXs running Spanning Tree and/or MXs running VPLS. Rate-limiting may help, but it doesn't kill loops completely. I am looking for ways to detect lots of MAC address moves (without polling for them) and blocking those interfaces involved when those MAC moves exceed a certain threshold via some trigger mechanism.
>
> Assume Junos 10.4R10 or more recent.
>
> Clarke Morledge
> College of William and Mary
> Information Technology - Network Engineering
> Jones Hall (Room 18)
> Williamsburg VA 23187
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

chris.brown at acsalaska

Aug 19, 2012, 5:39 AM

Post #8 of 11 (1210 views)
Permalink

Re: Tricks for killing L2 loops in VPLS and STP "BPDU-less" situations? [In reply to]

One think I noticed when working with the BUM filter under VPLS instance
is that there is no way to declare a per instance policer that I could find.

Your can call the same filter/policer in multiple VPLS instances, but
the named policer is a single global instance. So, if you call the same
filter w/ 5Mbit policer in 20 instances it is not 20 seperate 5 mbit
policers, it is one policer shared across all.

I very much want to add a BUM policer by default to all VPLS instances,
but I really want to avoid creating a seperate filter and policer config
for each instance, when 95% of them would be running one of three
standard configs.

And yes, this was tested. 10.4R10, trio based (MX960s w/ MPC2 and MX80)
the policer was always shared.

On 8/17/12 3:20 PM, Chris Kawchuk wrote:
> Hi Clarke,
>
> We pass through BPDUs through VPLS the MX'es- but yes, miscreant users / switches will always be a problem.
>
> We do the following to every customer-facing VPLS instance, but only #3 would help you here:
>
> 1. Mac Limiting per VPLS Interface (100) (i.e per 'site')
> 2. Mac Limiting per VPLS (500)
> 3. Limit Broadcast/Unknown Unicast/Multicast Traffic (5 Mbit) into the VPLS
>
> You can put on an input firewall filter which calls a 5 Mbit policer at [routing instances <vpls-name> forwarding-options family vpls <>] to start limiting this type of traffic into the 'bridge domain' at any time.
>
> - CK.
>
>
> On 18/08/2012, at 1:08 AM, Clarke Morledge <chmorl [at] wm> wrote:
>
>> We have had the unfortunate experience of having users plug in small mini-switches into our network that have the capability of filtering out (by-default) BPDUs while allowing other traffic through. The nightmare situation is when a user plugs in such a switch accidentally into two of our EX switches. Traffic will loop through the miscreant switch between the two EXs and without BPDUs it just looks like MAC addresses keep moving between the real source and the two EXs.
>>
>> In an MX environment running VPLS, this problem can happen easily as there are no BPDUs even to protect against loops in VPLS, particularly when your VPLS domain ties into a Spanning Tree domain downstream where your potential miscreant switch may appear.
>>
>> I am curious to know if anyone has come up with strategies to kill these loops for EXs running Spanning Tree and/or MXs running VPLS. Rate-limiting may help, but it doesn't kill loops completely. I am looking for ways to detect lots of MAC address moves (without polling for them) and blocking those interfaces involved when those MAC moves exceed a certain threshold via some trigger mechanism.
>>
>> Assume Junos 10.4R10 or more recent.
>>
>> Clarke Morledge
>> College of William and Mary
>> Information Technology - Network Engineering
>> Jones Hall (Room 18)
>> Williamsburg VA 23187
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp [at] puck
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>

--
------------------------------------------------------------------------
Christopher E. Brown <chris.brown [at] acsalaska> desk (907) 550-8393
cell (907) 632-8492
IP Engineer - ACS
------------------------------------------------------------------------
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

chmorl at wm

Sep 17, 2012, 2:34 PM

Post #9 of 11 (1018 views)
Permalink

Re: MX Design [In reply to]

Johan,

Regarding your question about whether or not to run MSTP up to a pair of
MX distribution routers from downstream EX's, I wrestled with this for
awhile several years ago. I had a devil of a time trying to get Link
Aggregation (LAG) to work even between two MX routers reliably without
introducing other issues in the pre-10.4 days, so I gave up on it.

The idea of Multi-Chassis LAG active/active does sound inviting, but I
guess I am a bit shy still about LAG in general introducing weirdness and
bugs.

I really would like to see Juniper embrace IEEE802.1aq Shortest Path
Bridging or the IETF RBridges, but with the current emphasis on QFabric I
am not holding my breadth. At this point, waiting for BGP MPLS Based
MAC VPN is a more realistic down-the-road type of thing from Juniper.

In the meantime, we opted to bringing MSTP up into the MX routers and make
one of MX routers the root and the other a backup root. We lose use of
one of the links, but at least it has been very stable over the past few
years, and I stay away from proprietary solutions.

Clarke Morledge
College of William and Mary
Information Technology - Network Engineering
Jones Hall (Room 18)
Williamsburg VA 23187

_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

fs at wpi

Sep 17, 2012, 6:56 PM

Post #10 of 11 (1012 views)
Permalink

Re: MX Design [In reply to]

For what it's worth, we've been harassing pretty much everyone we talk to with
a juniper.net email address about SPB. I suspect the biggest limitation is
that nowhere in any EX or MX docs is there mention of support for 802.1ah
mac-in-mac, which is pretty necessary for SPB. That said, the one time we got
an answer we were told that Juniper was going to take a "wait and see"
approach, so there's at least a chance that if enough customers make enough
noise, they might start seriously working on it.

I, for one, have a dream where spanning tree, in all its incarnations, is
reduced to little more than a scary story told by crusty old admins to interns
and PFYs by the light of storage array lights. "Hey Mr. NetAdmin, tell us the
story of CareGroup again!"

Frank Sweetser fs at wpi.edu | For every problem, there is a solution that
Manager of Network Operations | is simple, elegant, and wrong.
Worcester Polytechnic Institute | - HL Mencken

On 9/17/2012 5:34 PM, Clarke Morledge wrote:
> Johan,
>
> Regarding your question about whether or not to run MSTP up to a pair of MX
> distribution routers from downstream EX's, I wrestled with this for awhile
> several years ago. I had a devil of a time trying to get Link Aggregation
> (LAG) to work even between two MX routers reliably without introducing other
> issues in the pre-10.4 days, so I gave up on it.
>
> The idea of Multi-Chassis LAG active/active does sound inviting, but I guess I
> am a bit shy still about LAG in general introducing weirdness and bugs.
>
> I really would like to see Juniper embrace IEEE802.1aq Shortest Path Bridging
> or the IETF RBridges, but with the current emphasis on QFabric I am not
> holding my breadth. At this point, waiting for BGP MPLS Based MAC VPN is a
> more realistic down-the-road type of thing from Juniper.
>
> In the meantime, we opted to bringing MSTP up into the MX routers and make one
> of MX routers the root and the other a backup root. We lose use of one of the
> links, but at least it has been very stable over the past few years, and I
> stay away from proprietary solutions.
>
> Clarke Morledge
> College of William and Mary
> Information Technology - Network Engineering
> Jones Hall (Room 18)
> Williamsburg VA 23187
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp [at] puck
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

bdale at comlinx

Sep 17, 2012, 8:29 PM

Post #11 of 11 (1015 views)
Permalink

Re: MX Design [In reply to]

On 18/09/2012, at 11:56 AM, Frank Sweetser <fs [at] WPI> wrote:

>
> For what it's worth, we've been harassing pretty much everyone we talk to with a juniper.net email address about SPB. I suspect the biggest limitation is that nowhere in any EX or MX docs is there mention of support for 802.1ah mac-in-mac, which is pretty necessary for SPB.

MX has had support for PBB (and thus 802.1ah) since 10.0:

http://www.juniper.net/techpubs/en_US/junos10.0/topics/example/pbb-eline-elan-mx-series-configuring.html

I too would like to see SPB get some support in Junos, or alternatively, JunosSDK getting some support for layer2 protocols, so someone else can have a crack at implementing it.

The pendulum of hype in the data centre appears to be swinging away from distributed control-plane and toward centralised everything (QFabric, Virtual-Chassis, SDN etc). Maybe there is still hope for the metro?

Ben
_______________________________________________
juniper-nsp mailing list juniper-nsp [at] puck
https://puck.nether.net/mailman/listinfo/juniper-nsp

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads