Mailing List Archive: Cisco: NSP

WS-X6704-10GE, WS-X6708-10GE

Index | Next | Previous | View Threaded

artem at aws-net

Mar 2, 2012, 12:13 AM

Post #1 of 15 (2164 views)
Permalink

WS-X6704-10GE, WS-X6708-10GE

Hi, List!

May be these questions was discussed earlier... can't find it...
Please give me some links than.

I'm tring to clarify my understanding of switching paths on these
line cards. From one point of view, Cisco docs says that if the
traffic should ingress via one port on the line card and then
should egress through another port on the same line card it will
never leave this line card. So it will be switched via internal
bus. Right?

At one of our POPs we have Cisco 7606-S chassis with the folowing:

Mod Ports Card Type Model
--- ----- -------------------------------------- ------------------
2 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE
3 48 CEF720 48 port 1000mb SFP WS-X6748-SFP
4 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX
5 2 Route Switch Processor 720 (Active) RSP720-3CXL-GE

At peaks we see total ingress thraffic on all 10GE ports around
30-32Gbps and increase of overruns on all 10GE ports.

Utilization of the fabric and forwarding performance are as folows:

Switch Fabric Resources
Bus utilization: current: 10%, peak was 51% at 21:31:26 EET Sat Feb
18 2012
Fabric utilization: Ingress Egress
Module Chanl Speed rate peak rate peak

2 0 20G 15% 46% @22:57 30Jan12 15% 49% @22:02
31Jan12
2 1 20G 15% 49% @20:08 01Feb12 15% 46% @20:14
25Feb12
3 0 20G 0% 2% @10:49 02Feb12 0% 1% @14:16
30Jan12
3 1 20G 1% 1% @14:16 30Jan12 0% 2% @19:25
30Jan12
4 0 20G 1% 16% @19:22 11Feb12 3% 11% @19:40
04Feb12
4 1 20G 3% 9% @22:23 31Jan12 1% 10% @19:23
11Feb12
5 0 20G 0% 1% @14:16 30Jan12 0% 2% @21:24
10Feb12
Switching mode: Module
Switching mode
2
compact
3
compact
4
compact
5
compact
a

L2 Forwarding Resources
MAC Table usage: Module Collisions Total Used
%Used
5 0 98304 292
1%

VPN CAM usage: Total Used
%Used
512 0
0%
L3 Forwarding Resources
Module FIB TCAM usage: Total
Used %Used
5 72 bits (IPv4, MPLS, EoM) 524288
13963 3%
144 bits (IP mcast, IPv6) 262144
316 1%

detail: Protocol Used
%Used
IPv4 7057
1%
MPLS 6897
1%
EoM 9
1%

IPv6 117
1%
IPv4 mcast 196
1%
IPv6 mcast 3
1%

Adjacency usage: Total Used
%Used
1048576 7759
1%

Forwarding engine load:
Module pps peak-pps
peak-time
5 1965007 10703659 21:31:21 EET Sat Feb
18 2012

Actually, typical PPS is about 5-6 millions at peak times in evening
and bus utilization is about 20%.

I made simple calculations and found that about 16Gbps switched via
internal bus at line card and about 14-15 Gbps switched via fabric.
So the problem is internal 16Gbps (atually 16Gbps+16Gbps?) bus on
linecard.

So, the possible solution seems to install additional 6704 line
card ad distribute links between them according to main traffic flows.

Is it correct that CFC is not an issue in this particular situation?
D-Bus is not overutilized yet. I agree that this is goog to install
DFC dauter cards (or even 6708 with DFC), but not now.

I kbow that WS-X6708 much better option (and it is DFC), but now we
have no possibility to replace all 6704 by 6708 ones.

Is it all correct or I'm missing something?

Is it possible somehow to disable switching via internal bus on linecard
and reroute all traffic via fabric?

Similar problem was found on another router with WS-X6708 line card.
After swapping some 10GE links between ports most part of traffic
starts to go via fabric. And overruns disappeared.

Thanks in advance!

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem [at] aws-net | http://www.aws-net.org.ua/~artem
artem [at] viklenko | JID: artem [at] jabber
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

A.L.M.Buxey at lboro

Mar 2, 2012, 2:03 AM

Post #2 of 15 (2093 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

without DFC cards, some work/decisions still have to go to the supervisor. DFC (distributed) is what gives your modules autonomy

alan

_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

artem at aws-net

Mar 2, 2012, 2:45 AM

Post #3 of 15 (2101 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 02.03.2012 12:03, Alan Buxey wrote:
> without DFC cards, some work/decisions still have to go to the supervisor. DFC (distributed) is what gives your modules autonomy
>
> alan
>

This is already clear. :) The only not-so-clear thing now is the
internals of these line cards.

Thank you!

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem [at] aws-net | http://www.aws-net.org.ua/~artem
artem [at] viklenko | JID: artem [at] jabber
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

dv at dv

Mar 2, 2012, 3:03 AM

Post #4 of 15 (2086 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

I had a simular problem a few months ago.
I saw overruns and loss of packets when much traffic flowed from one port
of 6704 to another port of the same card. (Actually it was port mirroring).

The problem was fixed by configuring "fabric buffer-reserve low" (or medium).

On Fri, 2 Mar 2012, Artyom Viklenko wrote:

> On 02.03.2012 12:03, Alan Buxey wrote:
>> without DFC cards, some work/decisions still have to go to the supervisor.
>> DFC (distributed) is what gives your modules autonomy
>>
>> alan
>>
>
> This is already clear. :) The only not-so-clear thing now is the internals of
> these line cards.
>
> Thank you!

--
Dmitry Valdov
CCIE #15379 (R&S and SP)
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

lukasz at bromirski

Mar 2, 2012, 3:07 AM

Post #5 of 15 (2069 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 2012-03-02 09:13, Artyom Viklenko wrote:

> I'm tring to clarify my understanding of switching paths on these
> line cards. From one point of view, Cisco docs says that if the
> traffic should ingress via one port on the line card and then
> should egress through another port on the same line card it will
> never leave this line card. So it will be switched via internal
> bus. Right?

No, and if it says so somewhere, please point it to the doc team
to fix it.

Both 6704 and 6708 have two complex of Fabric ASICs.
The 6708 you can see on figure 21 here:
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html

The port mappings for Fabric ASICs should be found in the hardware
installation notes under the 'Switch fabric connections' in the
tables for specific LC:
http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/hardware/Module_Installation/Mod_Install_Guide/02ethern.html#wp1048010

Essentially, traffic from one Fabric ASIC to the ports on the
other Fabric ASIC will go over the fabric itself. Only traffic
belonging the the same Fabric ASIC will be switched locally if of
course there's a DFC installed.

--
"There's no sense in being precise when | �ukasz Bromirski
you don't know what you're talking | jid:lbromirski [at] jabber
about." John von Neumann | http://lukasz.bromirski.net
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

artem at aws-net

Mar 2, 2012, 3:44 AM

Post #6 of 15 (2081 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 02.03.2012 13:07, �ukasz Bromirski wrote:
> On 2012-03-02 09:13, Artyom Viklenko wrote:
>
>> I'm tring to clarify my understanding of switching paths on these
>> line cards. From one point of view, Cisco docs says that if the
>> traffic should ingress via one port on the line card and then
>> should egress through another port on the same line card it will
>> never leave this line card. So it will be switched via internal
>> bus. Right?
>
> No, and if it says so somewhere, please point it to the doc team
> to fix it.
>
> Both 6704 and 6708 have two complex of Fabric ASICs.
> The 6708 you can see on figure 21 here:
> http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html
>
>
> The port mappings for Fabric ASICs should be found in the hardware
> installation notes under the 'Switch fabric connections' in the
> tables for specific LC:
> http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/hardware/Module_Installation/Mod_Install_Guide/02ethern.html#wp1048010
>
>
> Essentially, traffic from one Fabric ASIC to the ports on the
> other Fabric ASIC will go over the fabric itself. Only traffic
> belonging the the same Fabric ASIC will be switched locally if of
> course there's a DFC installed.

ok. Now I see

Switch Fabric Resources
Bus utilization: current: 13%, peak was 51% at 21:31:26 EET Sat Feb
18 2012
Fabric utilization: Ingress Egress
Module Chanl Speed rate peak rate peak

2 0 20G 23% 46% @22:57 30Jan12 18% 49% @22:02
31Jan12
2 1 20G 21% 49% @20:08 01Feb12 25% 46% @20:14
25Feb12

I.e. 4,6 Gbps on channel 0 and 4,2 Gbps on channel 1. No DFC on module.
Total input on all four 10GE ~ 19 Gbps. Fabric switching only 8,8 Gbps.
Similar approach I see on 6708 with DFC. Some part of traffic goes via
fabric and some in line card itself.

AFAIK, presense of DFC influence only forwarding decisions process (and
policyng, for example) but not on moving traffic itself?

Anyway, if traffic should be switched in ASIC what is the limitations
in terms of bandwidth or PPS?

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem [at] aws-net | http://www.aws-net.org.ua/~artem
artem [at] viklenko | JID: artem [at] jabber
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

saku at ytti

Mar 2, 2012, 3:50 AM

Post #7 of 15 (2083 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On (2012-03-02 12:07 +0100), Łukasz Bromirski wrote:

> Essentially, traffic from one Fabric ASIC to the ports on the
> belonging the the same Fabric ASIC will be switched locally if of
> course there's a DFC installed.

You don't need DFC for this, DFC has nothing to do with moving actual bits,
it is just for lookups.
So without DFC, you're still asking over DBUS from SUP PFC about egress,
but once answer from RBUS comes, you're copying inside the linecard the
packet to egress, without going through fabric.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

lukasz at bromirski

Mar 2, 2012, 3:57 AM

Post #8 of 15 (2078 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 2012-03-02 12:50, Saku Ytti wrote:
> On (2012-03-02 12:07 +0100), �ukasz Bromirski wrote:
>
>> Essentially, traffic from one Fabric ASIC to the ports on the
>> belonging the the same Fabric ASIC will be switched locally if of
>> course there's a DFC installed.
>
> You don't need DFC for this, DFC has nothing to do with moving actual bits,
> it is just for lookups.

That was my oversimplification. What I've meant to say, if the DFC
is installed the process will be "just as simple". For CFC, the
process of moving the data will be similar, but will require request
and answer from Sup over the shared bus.

> So without DFC, you're still asking over DBUS from SUP PFC about egress,
> but once answer from RBUS comes, you're copying inside the linecard the
> packet to egress, without going through fabric.

Yes.

--
"There's no sense in being precise when | �ukasz Bromirski
you don't know what you're talking | jid:lbromirski [at] jabber
about." John von Neumann | http://lukasz.bromirski.net
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

lukasz at bromirski

Mar 2, 2012, 4:17 AM

Post #9 of 15 (2108 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 2012-03-02 12:44, Artyom Viklenko wrote:

> Switch Fabric Resources
> Bus utilization: current: 13%, peak was 51% at 21:31:26 EET Sat Feb 18 2012
> Fabric utilization: Ingress Egress
> Module Chanl Speed rate peak rate peak
> 2 0 20G 23% 46% @22:57 30Jan12 18% 49% @22:02 31Jan12
> 2 1 20G 21% 49% @20:08 01Feb12 25% 46% @20:14 25Feb12
>
> I.e. 4,6 Gbps on channel 0 and 4,2 Gbps on channel 1. No DFC on module.
> Total input on all four 10GE ~ 19 Gbps. Fabric switching only 8,8 Gbps.
> Similar approach I see on 6708 with DFC. Some part of traffic goes via
> fabric and some in line card itself.

That's normal for "non-optimized" traffic patters, so in real life :)
You can check for example using NetFlow, if there are flows that
could be optimized within one Port ASIC on one LC. Some people decide
it's worth and do it, some skip it.

> AFAIK, presense of DFC influence only forwarding decisions process (and
> policyng, for example) but not on moving traffic itself?

Yes, see my answer to Ytti.

> Anyway, if traffic should be switched in ASIC what is the limitations
> in terms of bandwidth or PPS?

The bandwidth for 6704 is line rate of front ports, as it
connects using 2x20Gbit/s channels to the fabric. The DFC however
is limited to 48Mpps and the traffic through the fabric uses additional
headers.

So if you have 4 10GE ports doing forwarding for 64B packets fully
locally, it will be 4x14.8Mpps=59.2Mpps, while the DFC can only do
48Mpps.

Depending on your traffic profile, you'll either hit PPS limitation
of the DFC (or the centrally located PFC) or the bandwidth constrain
for the 64B packets (DDoS for example).

--
"There's no sense in being precise when | �ukasz Bromirski
you don't know what you're talking | jid:lbromirski [at] jabber
about." John von Neumann | http://lukasz.bromirski.net
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

artem at aws-net

Mar 2, 2012, 5:17 AM

Post #10 of 15 (2074 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 02.03.2012 13:03, Dmitry Valdov wrote:
>
>
>
> I had a simular problem a few months ago.
> I saw overruns and loss of packets when much traffic flowed from one
> port of 6704 to another port of the same card. (Actually it was port
> mirroring).
>
> The problem was fixed by configuring "fabric buffer-reserve low" (or
> medium).

Hm.. this increase space for incoming packets? Correct?
Interesting. Do I need to reload router after this command applied?

>
>
> On Fri, 2 Mar 2012, Artyom Viklenko wrote:
>
>> On 02.03.2012 12:03, Alan Buxey wrote:
>>> without DFC cards, some work/decisions still have to go to the
>>> supervisor. DFC (distributed) is what gives your modules autonomy
>>>
>>> alan
>>>
>>
>> This is already clear. :) The only not-so-clear thing now is the
>> internals of these line cards.
>>
>> Thank you!
>

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem [at] aws-net | http://www.aws-net.org.ua/~artem
artem [at] viklenko | JID: artem [at] jabber
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

dv at dv

Mar 2, 2012, 8:30 AM

Post #11 of 15 (2076 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

Hi!

No reload required.
I guess this command increases buffers between a card and fabric.

Well.. When a packet arrives.. It must come to the fabric and return back to
the card.. What happend when two packets arrive at the same time?

On Fri, 2 Mar 2012, Artyom Viklenko wrote:

> On 02.03.2012 13:03, Dmitry Valdov wrote:
>>
>>
>>
>> I had a simular problem a few months ago.
>> I saw overruns and loss of packets when much traffic flowed from one
>> port of 6704 to another port of the same card. (Actually it was port
>> mirroring).
>>
>> The problem was fixed by configuring "fabric buffer-reserve low" (or
>> medium).
>
> Hm.. this increase space for incoming packets? Correct?
> Interesting. Do I need to reload router after this command applied?
>
>>
>>
>> On Fri, 2 Mar 2012, Artyom Viklenko wrote:
>>
>>> On 02.03.2012 12:03, Alan Buxey wrote:
>>>> without DFC cards, some work/decisions still have to go to the
>>>> supervisor. DFC (distributed) is what gives your modules autonomy
>>>>
>>>> alan
>>>>
>>>
>>> This is already clear. :) The only not-so-clear thing now is the
>>> internals of these line cards.
>>>
>>> Thank you!
>>
>
>
> --
> Sincerely yours,
> Artyom Viklenko.
> -------------------------------------------------------
> artem [at] aws-net | http://www.aws-net.org.ua/~artem
> artem [at] viklenko | JID: artem [at] jabber
> FreeBSD: The Power to Serve - http://www.freebsd.org
>

--
Dmitry Valdov
CCIE #15379 (R&S and SP)
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

artem at aws-net

Mar 4, 2012, 10:59 AM

Post #12 of 15 (2024 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

02.03.2012 18:30, Dmitry Valdov пишет:
> Hi!
>
> No reload required.
> I guess this command increases buffers between a card and fabric.
>

So... it doesn't help much.
Last thing to try is to swap some ports.
Think we really hit hardware limitations between pairs of ports.

Anyway, thaks to all!

> Well.. When a packet arrives.. It must come to the fabric and return
> back to
> the card.. What happend when two packets arrive at the same time?
>
>
> On Fri, 2 Mar 2012, Artyom Viklenko wrote:
>
>> On 02.03.2012 13:03, Dmitry Valdov wrote:
>>>
>>>
>>>
>>> I had a simular problem a few months ago.
>>> I saw overruns and loss of packets when much traffic flowed from one
>>> port of 6704 to another port of the same card. (Actually it was port
>>> mirroring).
>>>
>>> The problem was fixed by configuring "fabric buffer-reserve low" (or
>>> medium).
>>
>> Hm.. this increase space for incoming packets? Correct?
>> Interesting. Do I need to reload router after this command applied?
>>
>>>
>>>
>>> On Fri, 2 Mar 2012, Artyom Viklenko wrote:
>>>
>>>> On 02.03.2012 12:03, Alan Buxey wrote:
>>>>> without DFC cards, some work/decisions still have to go to the
>>>>> supervisor. DFC (distributed) is what gives your modules autonomy
>>>>>
>>>>> alan
>>>>>
>>>>
>>>> This is already clear. :) The only not-so-clear thing now is the
>>>> internals of these line cards.
>>>>
>>>> Thank you!
>>>
>>
>>
>> --
>> Sincerely yours,
>> Artyom Viklenko.
>> -------------------------------------------------------
>> artem [at] aws-net | http://www.aws-net.org.ua/~artem
>> artem [at] viklenko | JID: artem [at] jabber
>> FreeBSD: The Power to Serve - http://www.freebsd.org
>>
>

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem [at] aws-net | http://www.aws-net.org.ua/~artem
artem [at] viklenko | ================================
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

taosysnet at gmail

Mar 5, 2012, 5:10 AM

Post #13 of 15 (2040 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

2012/3/2 �ukasz Bromirski <lukasz [at] bromirski>

> On 2012-03-02 09:13, Artyom Viklenko wrote:
>
> I'm tring to clarify my understanding of switching paths on these
>> line cards. From one point of view, Cisco docs says that if the
>> traffic should ingress via one port on the line card and then
>> should egress through another port on the same line card it will
>> never leave this line card. So it will be switched via internal
>> bus. Right?
>>
>
> No, and if it says so somewhere, please point it to the doc team
> to fix it.
>
> Both 6704 and 6708 have two complex of Fabric ASICs.
> The 6708 you can see on figure 21 here:
> http://www.cisco.com/en/US/**prod/collateral/switches/**
> ps5718/ps708/prod_white_**paper0900aecd80673385.html<http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html>
>
> suppose there are port A and port B belong to fabric channel 1 and port C
belongs to fabric channel 2.

how does traffic pass from port A to port B with/without DFC ?
and any difference for traffic passing from port A to port C ?

I am wondering whether the traffic pass through switch fabric ? Is switch
fabric a passive component?

thanks.

The port mappings for Fabric ASICs should be found in the hardware
> installation notes under the 'Switch fabric connections' in the
> tables for specific LC:
> http://www.cisco.com/en/US/**docs/switches/lan/**
> catalyst6500/hardware/Module_**Installation/Mod_Install_**
> Guide/02ethern.html#wp1048010<http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/hardware/Module_Installation/Mod_Install_Guide/02ethern.html#wp1048010>
>
> Essentially, traffic from one Fabric ASIC to the ports on the
> other Fabric ASIC will go over the fabric itself. Only traffic
> belonging the the same Fabric ASIC will be switched locally if of
> course there's a DFC installed.
>
> --
> "There's no sense in being precise when | �ukasz Bromirski
> you don't know what you're talking | jid:lbromirski [at] jabber
> about." John von Neumann | http://lukasz.bromirski.net
>
> ______________________________**_________________
> cisco-nsp mailing list cisco-nsp [at] puck
> https://puck.nether.net/**mailman/listinfo/cisco-nsp<https://puck.nether.net/mailman/listinfo/cisco-nsp>
> archive at http://puck.nether.net/**pipermail/cisco-nsp/<http://puck.nether.net/pipermail/cisco-nsp/>
>
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

lukasz at bromirski

Mar 6, 2012, 1:54 PM

Post #14 of 15 (1942 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On 2012-03-05 14:10, tao wrote:

> Both 6704 and 6708 have two complex of Fabric ASICs.
> The 6708 you can see on figure 21 here:
> http://www.cisco.com/en/US/__prod/collateral/switches/__ps5718/ps708/prod_white___paper0900aecd80673385.html
> <http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html>
>
> suppose there are port A and port B belong to fabric channel 1 and port C
> belongs to fabric channel 2.
>
> how does traffic pass from port A to port B with/without DFC ?

It goes up to the Port ASIC. CFC requests decision to be taken by
a PFC on the Supervisor and DFC can take the decision locally. It
is then looped back to port B. The CFC communication takes place
over the shared bus which is limiting factor for packets per second
(on each of it the CFC has to query PFC for decision). The limit
for such system is 15 or 30Mpps. The limit for DFC system is
48Mpps per linecard.

> and any difference for traffic passing from port A to port C ?

Yes, the CFC/DFC decision process will be the same, but the
traffic will leave the LC using channel 1, and then come back
to the LC using channel 2 to leave out of port C.

> I am wondering whether the traffic pass through switch fabric ? Is switch
> fabric a passive component?

Passive in what sense? It doesn't do any network operations as is,
but it's powered up and processing traffic by means of frames +
additional headers. Switch fabric physically is a either a compex of,
or in the newer versions, single pretty large ASIC on the Supervisor
under the heatsink.

--
"There's no sense in being precise when | �ukasz Bromirski
you don't know what you're talking | jid:lbromirski [at] jabber
about." John von Neumann | http://lukasz.bromirski.net
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

saku at ytti

Mar 7, 2012, 12:30 AM

Post #15 of 15 (1928 views)
Permalink

Re: WS-X6704-10GE, WS-X6708-10GE [In reply to]

On (2012-03-06 22:54 +0100), Łukasz Bromirski wrote:

> (on each of it the CFC has to query PFC for decision). The limit
> for such system is 15 or 30Mpps. The limit for DFC system is
> 48Mpps per linecard.

Just to help other people on the list understand where this difference
comes from.
The PFC in DFC and SUP are same, i.e. 48Mpps or so.

But SUP PFC under-performs DFC PFC is because we are unable to send work to
it.

DBUS is 62.5MHz and does 32B per cycle. IPv4 lookups are 2 cycles (64B) and
IPv6 (and MPLS) lookups are 3 cycles (96B).

So 62.5/2 = 31.25Mpps (IPv4) and 62.5/3 = 20.83Mpps (IPv6, MPLS).

There are many situation where circulation occurs, this will of course
halve the performance. Especially in use of tunnels and L3 MPLS VPN.

(As trivia, as far I understand, there probably isn't technical reason why
SUP couldn't do ~48Mpps of MPLS lookups by sending just 32B of packet,
you'd lose ability to do MPLS ECMP based on IP headers though. But I'm not
at all sure about this)
--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp [at] puck
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads