Mailing List Archive: nsp: extreme

Super strange issue

Index | Next | Previous | View Threaded

aaron at wholesaleinternet

Oct 29, 2007, 3:16 PM

Post #1 of 5 (2016 views)
Permalink

Super strange issue

Ok,

I've been trying to figure this out for a couple days now.

I have 2 BD6808's. Every 30 minutes ping times spike across them to
300-400ms for a minute and a half. During this time the Nettask on the
outside box jumps to 60-70. There are no traffic or pps anomalies.

I know this isn't a lot of info but anyone seen an issue like this before?

Aaron

gunjan.gandhi at ericsson

Oct 29, 2007, 6:21 PM

Post #2 of 5 (1907 views)
Permalink

Super strange issue [In reply to]

The first thing that comes to mind is a broadcast storm. Are you sure
there are no L2 loops in your topology?

What is the box connected to?
Are you running any QoS, ping is a very bad method of determining
traffic behaviors as it is always Best Effort.
You talk about a outside box, I assume this is a client connected to the
switch. Is it possible to run mirroring on that port and observe the
traffic patterns when the ping times spike?

Cheers
//Gunjan

-----Original Message-----
From: extreme-nsp-bounces [at] puck
[mailto:extreme-nsp-bounces at puck.nether.net] On Behalf Of Aaron Wendel
Sent: Tuesday, 30 October 2007 10:16 AM
To: extreme-nsp at puck.nether.net
Subject: [e-nsp] Super strange issue

Ok,

I've been trying to figure this out for a couple days now.

I have 2 BD6808's. Every 30 minutes ping times spike across them to
300-400ms for a minute and a half. During this time the Nettask on the
outside box jumps to 60-70. There are no traffic or pps anomalies.

I know this isn't a lot of info but anyone seen an issue like this
before?

Aaron

_______________________________________________
extreme-nsp mailing list
extreme-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/extreme-nsp

r.kerr at cranfield

Oct 30, 2007, 2:47 AM

Post #3 of 5 (1898 views)
Permalink

Super strange issue [In reply to]

On Mon, 2007-10-29 at 18:16 -0500, Aaron Wendel wrote:

> I've been trying to figure this out for a couple days now.

> I have 2 BD6808's. Every 30 minutes ping times spike across them to
> 300-400ms for a minute and a half. During this time the Nettask on the
> outside box jumps to 60-70. There are no traffic or pps anomalies.

> I know this isn't a lot of info but anyone seen an issue like this before?

Sounds very familiar, what's the size of your FDB like? I don't remember
the exact details, but if it gets full then you'll see spikes like that
whilst it tries to find some old entries to throw out to make room for
new ones.

Don't remember it ever occurring regularly every 30 minutes though. I
guess perhaps there is something else on the network that runs once
every 30 minutes and causes a lot of new FDB entries to be created?

--
Robert Kerr

aaron at wholesaleinternet

Oct 30, 2007, 9:42 AM

Post #4 of 5 (1898 views)
Permalink

Super strange issue [In reply to]

Thanks for the reply,

>The first thing that comes to mind is a
>broadcast storm. Are you sure
>there are no L2 loops in your topology?

Here's a rough flow:

Internet -- BD6808(1) -- BD6808(2) -- Other stuff

There are no L2 loops. Both BD's are connected by a single piece of fiber.
In addition all the ports are on separate VLANs and the boxes are routing
between those VLANs.

>What is the box connected to?

See above

>Are you running any QoS, ping is
>a very bad method of determining
>traffic behaviors as it is always Best Effort.

No QOS. Yes, I understand that Ping is a bad indication, especially on an
Extreme box. What it indicated though is the spike in CPU utilization which
is what I'm most concerned about.

>You talk about a outside box, I assume
>this is a client connected to the
>switch. Is it possible to run mirroring on
>that port and observe the
>traffic patterns when the ping times spike?

This is my next step. I think I've narrowed it down to a loop that's
created when a VLAN drops off the network. Packets bounce between the two
routers. I'm about 70% sure this is what's causing it. That every 30
minutes something is scanning my network and every vlan that's not active is
bouncing.

Aaron

aaron at wholesaleinternet

Oct 30, 2007, 9:52 AM

Post #5 of 5 (1909 views)
Permalink

Super strange issue [In reply to]

>Sounds very familiar, what's the size of your FDB like? I don't remember
>the exact details, but if it gets full then you'll see spikes like that
>whilst it tries to find some old entries to throw out to make room for
>new ones.

My FDB on both boxes is around 5000 entries. My ip-subnet-lookup fdb is
about 257000 entries.

>Don't remember it ever occurring regularly every 30 minutes though. I
>guess perhaps there is something else on the network that runs once
>every 30 minutes and causes a lot of new FDB entries to be created?

That's what I'm trying to figure out. :)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads