Mailing List Archive: nsp: foundry

Max BGP peers BigIron RX4

Index | Next | Previous | View Threaded

p.taks at nforce

Feb 22, 2011, 6:23 AM

Post #1 of 12 (1530 views)
Permalink

Max BGP peers BigIron RX4

Hi all,

While configuring a new peer on an BigIron RX4 we got the following error:
Error! BGP4 cannot allocate memory for peer 253

A PDF claims the following (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
BGPv4: Scalable to 4 million routes, 500 peers and 14,000 attributes with MR2 management module

Is there any reason why it wouldn't want to configure more peering sessions?

- I do not see any system-max setting for this

- As well as I do not see any full memory yet

Hope someone knows how to resolve this or what is causing this error.

sh memory
====================================================================
BigIron RX active MP slot 33:
Total SDRAM: 2147483648 bytes
Available Memory: 1646874624 bytes
Free Physical Pages: 401374 pages

Malloc statistics: total 500462813
os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0, diff: 12877
====================================================================
BigIron RX LP SL 1:
Total SDRAM: 536870912 bytes
Available Memory: 53039104 bytes
====================================================================
BigIron RX LP SL 3:
Total SDRAM: 536870912 bytes
Available Memory: 72671232 bytes
====================================================================
BigIron RX LP SL 4:
Total SDRAM: 536870912 bytes
Available Memory: 72654848 bytes

Any help is appreciated, thanks!

Pieter Taks

hardenrm at uiuc

Feb 22, 2011, 6:35 AM

Post #2 of 12 (1462 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Do a 'sh def val'.

My guess is you need to set the 'system-max' for max BGP peers.

/Ryan

On 02/22/2011 08:23 AM, Pieter Taks wrote:
> Hi all,
>
>
>
> While configuring a new peer on an BigIron RX4 we got the following error:
>
> /Error! BGP4 cannot allocate memory for peer 253/
>
>
>
> A PDF claims the following
> (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
>
> /BGPv4: //Scalable to 4 million routes, 500 peers and 14,000 attributes
> with MR2 management module/
>
>
>
> Is there any reason why it wouldn’t want to configure more peering sessions?
>
> - I do not see any system-max setting for this
>
> - As well as I do not see any full memory yet
>
>
>
> Hope someone knows how to resolve this or what is causing this error.
>
>
>
> sh memory
>
> ====================================================================
>
> BigIron RX active MP slot 33:
>
> Total SDRAM: 2147483648 bytes
>
> Available Memory: 1646874624 bytes
>
> Free Physical Pages: 401374 pages
>
>
>
> Malloc statistics: total 500462813
>
> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0,
> diff: 12877
>
> ====================================================================
>
> BigIron RX LP SL 1:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 53039104 bytes
>
> ====================================================================
>
> BigIron RX LP SL 3:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72671232 bytes
>
> ====================================================================
>
> BigIron RX LP SL 4:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72654848 bytes
>
>
>
>
>
> Any help is appreciated, thanks!
>
>
>
> Pieter Taks
>
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp [at] puck
> http://puck.nether.net/mailman/listinfo/foundry-nsp

--
Ryan M. Harden, BS, KC9IHX Office: 217-265-5192
CITES - Network Engineering Cell: 217-689-1363
2130 Digital Computer Lab Fax: 217-244-7089
1304 W. Springfield email: hardenrm [at] illinois
Urbana, IL 61801

University of Illinois at Urbana/Champaign - AS38
University of Illinois - ICCN - AS40387
_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck
http://puck.nether.net/mailman/listinfo/foundry-nsp

p.taks at nforce

Feb 22, 2011, 6:45 AM

Post #3 of 12 (1504 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi Ryan,

Thank you for replying so quickly.
I do not see any such value, as I thought of that as well. I am most likely really overlooking it (I hope).
Could it be the max session limit perhaps?

sh def values
sys log buffers:50 mac age time:300 sec telnet sessions:5

ip arp age:10 min bootp relay max hops:4 ip ttl:64 hops
ip addr per intf:24

when multicast enabled :
igmp group memb.:140 sec igmp query:60 sec

when ospf enabled :
ospf dead:40 sec ospf hello:10 sec ospf retrans:5 sec
ospf transit delay:1 sec

when bgp enabled :
bgp local pref.:100 bgp keep alive:60 sec bgp hold:180 sec
bgp metric:10 bgp local as:1 bgp cluster id:0
bgp ext. distance:20 bgp int. distance:200 bgp local distance:200

when IS-IS enabled :
isis hello interval:10 sec isis hello multiplier:3
isis port metric:10 isis priority:64
isis csnp-interval:10 sec isis default-metric:10
isis distance:115 isis lsp-gen-interval:10 sec
isis lsp-interval:33 msec isis lsp-refresh-interval:900 sec
isis max-lsp-lifetime:1200 sec isis maximum-paths:4
isis retransmit-interval:5 sec isis spf-interval:5 sec

System Parameters Default Maximum Current
mac 32768 65536 65536
vlan 512 4095 4095
spanning-tree 32 128 32
rstp 32 128 128
ip-arp 8192 65536 65536
ip-static-arp 2048 16384 2048
multicast-route 8192 153600 8192
dvmrp-route 2048 16384 2048
dvmrp-mcache 4096 4096 4096
pim-mcache 4096 4096 4096
igmp-max-group-addr 1024 4096 1024
ip-cache 204800 524288 524288
ip-route 204800 524288 524288
ip-subnet-port 24 128 24
virtual-interface 255 4095 4095
session-limit 32768 163840 32768
ip-filter-sys 4096 8192 4096
mgmt-port-acl-size 20 100 20
l2-acl-table-entries 64 256 64
vlan-multicast-flood 0 4095 0
ipv6-cache 65536 65536 65536
ipv6-route 65536 65536 65536

Thanks!

Pieter Taks

-----Oorspronkelijk bericht-----
Van: Ryan Harden [mailto:hardenrm [at] uiuc]
Verzonden: dinsdag 22 februari 2011 15:36
Aan: Pieter Taks
CC: foundry-nsp [at] puck
Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Do a 'sh def val'.

My guess is you need to set the 'system-max' for max BGP peers.

/Ryan

On 02/22/2011 08:23 AM, Pieter Taks wrote:
> Hi all,
>
>
>
> While configuring a new peer on an BigIron RX4 we got the following error:
>
> /Error! BGP4 cannot allocate memory for peer 253/
>
>
>
> A PDF claims the following
> (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
>
> /BGPv4: //Scalable to 4 million routes, 500 peers and 14,000
> attributes with MR2 management module/
>
>
>
> Is there any reason why it wouldn't want to configure more peering sessions?
>
> - I do not see any system-max setting for this
>
> - As well as I do not see any full memory yet
>
>
>
> Hope someone knows how to resolve this or what is causing this error.
>
>
>
> sh memory
>
> ====================================================================
>
> BigIron RX active MP slot 33:
>
> Total SDRAM: 2147483648 bytes
>
> Available Memory: 1646874624 bytes
>
> Free Physical Pages: 401374 pages
>
>
>
> Malloc statistics: total 500462813
>
> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0,
> diff: 12877
>
> ====================================================================
>
> BigIron RX LP SL 1:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 53039104 bytes
>
> ====================================================================
>
> BigIron RX LP SL 3:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72671232 bytes
>
> ====================================================================
>
> BigIron RX LP SL 4:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72654848 bytes
>
>
>
>
>
> Any help is appreciated, thanks!
>
>
>
> Pieter Taks
>
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp [at] puck
> http://puck.nether.net/mailman/listinfo/foundry-nsp

--
Ryan M. Harden, BS, KC9IHX Office: 217-265-5192
CITES - Network Engineering Cell: 217-689-1363
2130 Digital Computer Lab Fax: 217-244-7089
1304 W. Springfield email: hardenrm [at] illinois
Urbana, IL 61801

University of Illinois at Urbana/Champaign - AS38
University of Illinois - ICCN - AS40387

_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck
http://puck.nether.net/mailman/listinfo/foundry-nsp

p.taks at nforce

Feb 22, 2011, 6:47 AM

Post #4 of 12 (1468 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi Ryan,

Could it bet hat I need 2x MR2 or perhaps 3x SFM1?

sh module
==========================================================================
*** BigIron RX-4 CHASSIS ***

Module Status Ports Starting MAC
M1 (left): RX-BI-MR2 Management Module (High Value) Active
M2 (right):
F1: RX-BI-SFM1 Switch Fabric Module OK
F2: RX-BI-SFM1 Switch Fabric Module OK
F3: RX-BI-SFM1 Switch Fabric Module not present
S1: RX-BI-4XG 4-port 10GbE Module CARD_STATE_UP 4 000c.dbf5.1200
S2:
S3: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
S4: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
==========================================================================

Thank you for trying to help Ryan, perhaps it is a quick fix after all and I just overlooked the fact I need two MR2's or some other setting.

Pieter Taks

-----Oorspronkelijk bericht-----
Van: Ryan Harden [mailto:hardenrm [at] illinois]
Verzonden: dinsdag 22 februari 2011 15:44
Aan: Pieter Taks
CC: hardenrm [at] uiuc
Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hmm. Are you certain you have an M2 management module?

I don't have any RX's here to check against. We have many other flavors of Foundry/Brocade but none with anywhere near 250+ BGP peers.

I would suspect 'session limit' is more for traffic sessions than BGP sessions.

Sorry I couldn't help. I've found over the years that many never think to look or didn't know about 'sh def val' and had no idea that 'max capable' didn't mean 'max configured'. I was hoping it would be a quick fix. :)

/Ryan

On 02/22/2011 08:38 AM, Pieter Taks wrote:
> Hi Ryan,
>
> Thank you for replying so quickly.
> I do not see any such value, as I thought of that as well. I am most likely really overlooking it (I hope).
> Could it be the max session limit perhaps?
>
> sh def values
> sys log buffers:50 mac age time:300 sec telnet sessions:5
>
> ip arp age:10 min bootp relay max hops:4 ip ttl:64 hops
> ip addr per intf:24
>
> when multicast enabled :
> igmp group memb.:140 sec igmp query:60 sec
>
> when ospf enabled :
> ospf dead:40 sec ospf hello:10 sec ospf retrans:5 sec
> ospf transit delay:1 sec
>
> when bgp enabled :
> bgp local pref.:100 bgp keep alive:60 sec bgp hold:180 sec
> bgp metric:10 bgp local as:1 bgp cluster id:0
> bgp ext. distance:20 bgp int. distance:200 bgp local distance:200
>
> when IS-IS enabled :
> isis hello interval:10 sec isis hello multiplier:3
> isis port metric:10 isis priority:64
> isis csnp-interval:10 sec isis default-metric:10
> isis distance:115 isis lsp-gen-interval:10 sec
> isis lsp-interval:33 msec isis lsp-refresh-interval:900 sec
> isis max-lsp-lifetime:1200 sec isis maximum-paths:4
> isis retransmit-interval:5 sec isis spf-interval:5 sec
>
> System Parameters Default Maximum Current
> mac 32768 65536 65536
> vlan 512 4095 4095
> spanning-tree 32 128 32
> rstp 32 128 128
> ip-arp 8192 65536 65536
> ip-static-arp 2048 16384 2048
> multicast-route 8192 153600 8192
> dvmrp-route 2048 16384 2048
> dvmrp-mcache 4096 4096 4096
> pim-mcache 4096 4096 4096
> igmp-max-group-addr 1024 4096 1024
> ip-cache 204800 524288 524288
> ip-route 204800 524288 524288
> ip-subnet-port 24 128 24
> virtual-interface 255 4095 4095
> session-limit 32768 163840 32768
> ip-filter-sys 4096 8192 4096
> mgmt-port-acl-size 20 100 20
> l2-acl-table-entries 64 256 64
> vlan-multicast-flood 0 4095 0
> ipv6-cache 65536 65536 65536
> ipv6-route 65536 65536 65536
>
> Best regards,
>
> Pieter Taks
> NFOrce Entertainment BV
>
> Tel. +31 20 6919299
> Fax. +31 20 6919409
>
>
> -----Oorspronkelijk bericht-----
> Van: Ryan Harden [mailto:hardenrm [at] uiuc]
> Verzonden: dinsdag 22 februari 2011 15:36
> Aan: Pieter Taks
> CC: foundry-nsp [at] puck
> Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4
>
> Do a 'sh def val'.
>
> My guess is you need to set the 'system-max' for max BGP peers.
>
> /Ryan
>
> On 02/22/2011 08:23 AM, Pieter Taks wrote:
>> Hi all,
>>
>>
>>
>> While configuring a new peer on an BigIron RX4 we got the following error:
>>
>> /Error! BGP4 cannot allocate memory for peer 253/
>>
>>
>>
>> A PDF claims the following
>> (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
>>
>> /BGPv4: //Scalable to 4 million routes, 500 peers and 14,000
>> attributes with MR2 management module/
>>
>>
>>
>> Is there any reason why it wouldn't want to configure more peering sessions?
>>
>> - I do not see any system-max setting for this
>>
>> - As well as I do not see any full memory yet
>>
>>
>>
>> Hope someone knows how to resolve this or what is causing this error.
>>
>>
>>
>> sh memory
>>
>> ====================================================================
>>
>> BigIron RX active MP slot 33:
>>
>> Total SDRAM: 2147483648 bytes
>>
>> Available Memory: 1646874624 bytes
>>
>> Free Physical Pages: 401374 pages
>>
>>
>>
>> Malloc statistics: total 500462813
>>
>> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0,
>> diff: 12877
>>
>> ====================================================================
>>
>> BigIron RX LP SL 1:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 53039104 bytes
>>
>> ====================================================================
>>
>> BigIron RX LP SL 3:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 72671232 bytes
>>
>> ====================================================================
>>
>> BigIron RX LP SL 4:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 72654848 bytes
>>
>>
>>
>>
>>
>> Any help is appreciated, thanks!
>>
>>
>>
>> Pieter Taks
>>
>>
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp [at] puck
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>

--
Ryan M. Harden, BS, KC9IHX Office: 217-265-5192
CITES - Network Engineering Cell: 217-689-1363
2130 Digital Computer Lab Fax: 217-244-7089
1304 W. Springfield email: hardenrm [at] illinois
Urbana, IL 61801

University of Illinois at Urbana/Champaign - AS38
University of Illinois - ICCN - AS40387

_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck
http://puck.nether.net/mailman/listinfo/foundry-nsp

hardenrm at uiuc

Feb 22, 2011, 6:54 AM

Post #5 of 12 (1541 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

I would certainly hope you wouldn't need a second MR2 as I believe they
are to be fully redundant. Same goes with SFMs. The third is a spare (At
least in the MLX/XMR line.)

/Ryan

On 02/22/2011 08:47 AM, Pieter Taks wrote:
> Hi Ryan,
>
> Could it bet hat I need 2x MR2 or perhaps 3x SFM1?
>
> sh module
> ==========================================================================
> *** BigIron RX-4 CHASSIS ***
>
> Module Status Ports Starting MAC
> M1 (left): RX-BI-MR2 Management Module (High Value) Active
> M2 (right):
> F1: RX-BI-SFM1 Switch Fabric Module OK
> F2: RX-BI-SFM1 Switch Fabric Module OK
> F3: RX-BI-SFM1 Switch Fabric Module not present
> S1: RX-BI-4XG 4-port 10GbE Module CARD_STATE_UP 4 000c.dbf5.1200
> S2:
> S3: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
> S4: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
> ==========================================================================
>
> Thank you for trying to help Ryan, perhaps it is a quick fix after all and I just overlooked the fact I need two MR2's or some other setting.
>
> Pieter Taks
>
> -----Oorspronkelijk bericht-----
> Van: Ryan Harden [mailto:hardenrm [at] illinois]
> Verzonden: dinsdag 22 februari 2011 15:44
> Aan: Pieter Taks
> CC: hardenrm [at] uiuc
> Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4
>
> Hmm. Are you certain you have an M2 management module?
>
> I don't have any RX's here to check against. We have many other flavors of Foundry/Brocade but none with anywhere near 250+ BGP peers.
>
> I would suspect 'session limit' is more for traffic sessions than BGP sessions.
>
> Sorry I couldn't help. I've found over the years that many never think to look or didn't know about 'sh def val' and had no idea that 'max capable' didn't mean 'max configured'. I was hoping it would be a quick fix. :)
>
> /Ryan
>
> On 02/22/2011 08:38 AM, Pieter Taks wrote:
>> Hi Ryan,
>>
>> Thank you for replying so quickly.
>> I do not see any such value, as I thought of that as well. I am most likely really overlooking it (I hope).
>> Could it be the max session limit perhaps?
>>
>> sh def values
>> sys log buffers:50 mac age time:300 sec telnet sessions:5
>>
>> ip arp age:10 min bootp relay max hops:4 ip ttl:64 hops
>> ip addr per intf:24
>>
>> when multicast enabled :
>> igmp group memb.:140 sec igmp query:60 sec
>>
>> when ospf enabled :
>> ospf dead:40 sec ospf hello:10 sec ospf retrans:5 sec
>> ospf transit delay:1 sec
>>
>> when bgp enabled :
>> bgp local pref.:100 bgp keep alive:60 sec bgp hold:180 sec
>> bgp metric:10 bgp local as:1 bgp cluster id:0
>> bgp ext. distance:20 bgp int. distance:200 bgp local distance:200
>>
>> when IS-IS enabled :
>> isis hello interval:10 sec isis hello multiplier:3
>> isis port metric:10 isis priority:64
>> isis csnp-interval:10 sec isis default-metric:10
>> isis distance:115 isis lsp-gen-interval:10 sec
>> isis lsp-interval:33 msec isis lsp-refresh-interval:900 sec
>> isis max-lsp-lifetime:1200 sec isis maximum-paths:4
>> isis retransmit-interval:5 sec isis spf-interval:5 sec
>>
>> System Parameters Default Maximum Current
>> mac 32768 65536 65536
>> vlan 512 4095 4095
>> spanning-tree 32 128 32
>> rstp 32 128 128
>> ip-arp 8192 65536 65536
>> ip-static-arp 2048 16384 2048
>> multicast-route 8192 153600 8192
>> dvmrp-route 2048 16384 2048
>> dvmrp-mcache 4096 4096 4096
>> pim-mcache 4096 4096 4096
>> igmp-max-group-addr 1024 4096 1024
>> ip-cache 204800 524288 524288
>> ip-route 204800 524288 524288
>> ip-subnet-port 24 128 24
>> virtual-interface 255 4095 4095
>> session-limit 32768 163840 32768
>> ip-filter-sys 4096 8192 4096
>> mgmt-port-acl-size 20 100 20
>> l2-acl-table-entries 64 256 64
>> vlan-multicast-flood 0 4095 0
>> ipv6-cache 65536 65536 65536
>> ipv6-route 65536 65536 65536
>>
>> Best regards,
>>
>> Pieter Taks
>> NFOrce Entertainment BV
>>
>> Tel. +31 20 6919299
>> Fax. +31 20 6919409
>>
>>
>> -----Oorspronkelijk bericht-----
>> Van: Ryan Harden [mailto:hardenrm [at] uiuc]
>> Verzonden: dinsdag 22 februari 2011 15:36
>> Aan: Pieter Taks
>> CC: foundry-nsp [at] puck
>> Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4
>>
>> Do a 'sh def val'.
>>
>> My guess is you need to set the 'system-max' for max BGP peers.
>>
>> /Ryan
>>
>> On 02/22/2011 08:23 AM, Pieter Taks wrote:
>>> Hi all,
>>>
>>>
>>>
>>> While configuring a new peer on an BigIron RX4 we got the following error:
>>>
>>> /Error! BGP4 cannot allocate memory for peer 253/
>>>
>>>
>>>
>>> A PDF claims the following
>>> (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
>>>
>>> /BGPv4: //Scalable to 4 million routes, 500 peers and 14,000
>>> attributes with MR2 management module/
>>>
>>>
>>>
>>> Is there any reason why it wouldn't want to configure more peering sessions?
>>>
>>> - I do not see any system-max setting for this
>>>
>>> - As well as I do not see any full memory yet
>>>
>>>
>>>
>>> Hope someone knows how to resolve this or what is causing this error.
>>>
>>>
>>>
>>> sh memory
>>>
>>> ====================================================================
>>>
>>> BigIron RX active MP slot 33:
>>>
>>> Total SDRAM: 2147483648 bytes
>>>
>>> Available Memory: 1646874624 bytes
>>>
>>> Free Physical Pages: 401374 pages
>>>
>>>
>>>
>>> Malloc statistics: total 500462813
>>>
>>> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0,
>>> diff: 12877
>>>
>>> ====================================================================
>>>
>>> BigIron RX LP SL 1:
>>>
>>> Total SDRAM: 536870912 bytes
>>>
>>> Available Memory: 53039104 bytes
>>>
>>> ====================================================================
>>>
>>> BigIron RX LP SL 3:
>>>
>>> Total SDRAM: 536870912 bytes
>>>
>>> Available Memory: 72671232 bytes
>>>
>>> ====================================================================
>>>
>>> BigIron RX LP SL 4:
>>>
>>> Total SDRAM: 536870912 bytes
>>>
>>> Available Memory: 72654848 bytes
>>>
>>>
>>>
>>>
>>>
>>> Any help is appreciated, thanks!
>>>
>>>
>>>
>>> Pieter Taks
>>>
>>>
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp [at] puck
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>

--
Ryan M. Harden, BS, KC9IHX Office: 217-265-5192
CITES - Network Engineering Cell: 217-689-1363
2130 Digital Computer Lab Fax: 217-244-7089
1304 W. Springfield email: hardenrm [at] illinois
Urbana, IL 61801

University of Illinois at Urbana/Champaign - AS38
University of Illinois - ICCN - AS40387
_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck
http://puck.nether.net/mailman/listinfo/foundry-nsp

niels=foundry-nsp at bakker

Feb 22, 2011, 9:31 AM

Post #6 of 12 (1476 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

* p.taks [at] nforce (Pieter Taks) [Tue 22 Feb 2011, 15:24 CET]:
>While configuring a new peer on an BigIron RX4 we got the following error:
>Error! BGP4 cannot allocate memory for peer 253

I understand this is a limit introduced for the RX later than the
publishing date of the document you referred to. Adding an additional
MR won't alleviate this static limit.

If you're an AMS-IX member, you may want to investigate using the
route servers there to alleviate the need for direct BGP sessions as
a workaround.

-- Niels.

--
_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck
http://puck.nether.net/mailman/listinfo/foundry-nsp

mt at primats

Feb 23, 2011, 9:51 PM

Post #7 of 12 (1454 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi Pieter,

I think it's a limitation of management module - I had the same problem on NetIron: MLX module supports maximum 512 BGP sessions, XMR maximum 1024 sessions.

What management module do you have?

Maxim Tuliuk

On 22 Feb 2011, at 15:23, Pieter Taks wrote:

> Hi all,
>
> While configuring a new peer on an BigIron RX4 we got the following error:
> Error! BGP4 cannot allocate memory for peer 253
>
> A PDF claims the following (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
> BGPv4: Scalable to 4 million routes, 500 peers and 14,000 attributes with MR2 management module
>
> Is there any reason why it wouldn’t want to configure more peering sessions?
> - I do not see any system-max setting for this
> - As well as I do not see any full memory yet
>
> Hope someone knows how to resolve this or what is causing this error.
>
> sh memory
> ====================================================================
> BigIron RX active MP slot 33:
> Total SDRAM: 2147483648 bytes
> Available Memory: 1646874624 bytes
> Free Physical Pages: 401374 pages
>
> Malloc statistics: total 500462813
> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0, diff: 12877
> ====================================================================
> BigIron RX LP SL 1:
> Total SDRAM: 536870912 bytes
> Available Memory: 53039104 bytes
> ====================================================================
> BigIron RX LP SL 3:
> Total SDRAM: 536870912 bytes
> Available Memory: 72671232 bytes
> ====================================================================
> BigIron RX LP SL 4:
> Total SDRAM: 536870912 bytes
> Available Memory: 72654848 bytes
>
>
> Any help is appreciated, thanks!
>
> Pieter Taks
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp [at] puck
> http://puck.nether.net/mailman/listinfo/foundry-nsp

p.taks at nforce

Feb 24, 2011, 10:55 PM

Post #8 of 12 (1454 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi Maxim,

Thank you for thinking with me. The following modules are in place.

Is there anything I need to replace/add?

sh module
==========================================================================
*** BigIron RX-4 CHASSIS ***

Module Status Ports Starting MAC
M1 (left): RX-BI-MR2 Management Module (High Value) Active
M2 (right):
F1: RX-BI-SFM1 Switch Fabric Module OK
F2: RX-BI-SFM1 Switch Fabric Module OK
F3: RX-BI-SFM1 Switch Fabric Module not present
S1: RX-BI-4XG 4-port 10GbE Module CARD_STATE_UP 4 000c.dbf5.1200
S2:
S3: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
S4: RX-BI-2XG 2-port 10GbE Module CARD_STATE_UP 2 000c.dbf5.1200
==========================================================================

Pieter Taks

Verzonden: donderdag 24 februari 2011 7:59

Hi Pieter,

I think it's a limitation of management module - I had the same problem on NetIron: MLX module supports maximum 512 BGP sessions, XMR maximum 1024 sessions.

What management module do you have?

Maxim Tuliuk

jazzyjpz at gmail

Mar 4, 2011, 3:03 PM

Post #9 of 12 (1402 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi,

That's interesting.

Does the RX series support the 'show ip bgp debug' cmd list? It's hidden up
to the point of 'debug'. If you can get that far, take a look around in
there, especially 'show ip bgp debug memory' & just the plain 'show ip bgp
debug'. Maybe something will stick out.

I want to say the router dynamically allocates memory for certain fields up
to a certain dynamic max related to other parts of BGP like # of routes. Do
you have soft-reconfig turned on? Maybe try turning that off and see if it
frees up any more BGP peers. Brocade software can be a little dodgy at times
and not well documented.

Also, I wouldn't be surprised if it was just some sort of bug. We've had
stranger things happen when it comes to a bug on the Brocades.

Sincerely.

On Tue, Feb 22, 2011 at 8:23 AM, Pieter Taks <p.taks [at] nforce> wrote:

> Hi all,
>
>
>
> While configuring a new peer on an BigIron RX4 we got the following error:
>
> *Error! BGP4 cannot allocate memory for peer 253*
>
>
>
> A PDF claims the following (
> http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf
> ):
>
> *BGPv4: **Scalable to 4 million routes, 500 peers and 14,000 attributes
> with MR2 management module*
>
>
>
> Is there any reason why it wouldn’t want to configure more peering
> sessions?
>
> - I do not see any system-max setting for this
>
> - As well as I do not see any full memory yet
>
>
>
> Hope someone knows how to resolve this or what is causing this error.
>
>
>
> sh memory
>
> ====================================================================
>
> BigIron RX active MP slot 33:
>
> Total SDRAM: 2147483648 bytes
>
> Available Memory: 1646874624 bytes
>
> Free Physical Pages: 401374 pages
>
>
>
> Malloc statistics: total 500462813
>
> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0, diff:
> 12877
>
> ====================================================================
>
> BigIron RX LP SL 1:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 53039104 bytes
>
> ====================================================================
>
> BigIron RX LP SL 3:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72671232 bytes
>
> ====================================================================
>
> BigIron RX LP SL 4:
>
> Total SDRAM: 536870912 bytes
>
> Available Memory: 72654848 bytes
>
>
>
>
>
> Any help is appreciated, thanks!
>
>
>
> Pieter Taks
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp [at] puck
> http://puck.nether.net/mailman/listinfo/foundry-nsp
>

p.taks at nforce

Mar 9, 2011, 11:15 AM

Post #10 of 12 (1361 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi,

Thanks for reading with me and thinking what it could be. Even with all the soft-reconfigurations disabled I still got the same issue. Please see the debug outputs below, perhaps anyone else knows why this is limited to 252 BGP peers on an RX4?

In the sh ip bgp debug variables output it shows Maximum Peer Index Number:253. Is there a way to change this?
sh ip bgp debug variables
safi:0, &bgp:08fe0aac, enabled:1, operational:1, &dbg_mem=08fe1c4c, curr_afi:0
io_process_running:0, io_process_next_peer_number=0
in_long_loops 0, clear_all 0, timer 00000000, count 0
timer_enabled:1, timer_next_peer_number:1, 1s timer 1, short timer 1
scheduler id:5:2, ip:77.247.181.xxx/28, time=92219
bgp_tcb:08fe0b74 (0xf0090001, 0), tick_cnt=12, seconds=6717786
bgp_tcb6:f00a0001 (0x00000000, 0x00000019)
*peer:08fe0bb0, *peer_group:08fe1b5c, RIB_in_root_node:24a35960
Maximum Peer Index Number:253, check_nexthops:0 0
router_id:85.159.239.xx, configured:0, cluster_id:0.0.0.13, configured:1
route_is_router_reflector:0, client_to_client_reflection:1
networks:x08fe401c, aggregate:x08feb0b8
default_metric:4294967294, local_preference:100, keep_alive:60, hold_time:180
originate_default:0, originated:0
distance:20 200 200, fast_external_fallover=0
nexthop recur0, en_def:0, readvertise:1, auto_sum:0, synch:0
always_compare_med:1, compare_med_with_empty_aspath: 0, redistribute_ibgp:150911764, local_network_check_time_count:6
nexthop_cache_hit_count:159342982, nexthop_cache_miss_count:5142
system memory:536870912, total_allocated:104422684, bgp_defined_quota:2147483648

sh ip bgp debug
BGP Debug Information
Pid Size Address Total Used Free NoMem Errors #_pools p_unit
0 8 08fe3d38 43688 31090 12598 0 0 5 2000
1 16 08fe3d64 52426 37260 15166 0 0 5 2000
2 24 08fe3d90 224692 131923 92769 0 0 10 2000
3 32 08fe3dbc 116502 77986 38516 0 0 12 800
4 48 08fe3de8 20163 12144 8019 0 0 7 400
5 64 08fe3e14 3851 981 2870 0 0 5 200
6 96 08fe3e40 2617 397 2220 0 0 6 80
7 128 08fe3e6c 2480 1081 1399 0 0 8 40
8 256 08fe3e98 755 478 277 0 0 6 20
9 512 08fe3ec4 0 0 0 0 0 0 10
10 4096 08fe3ef0 0 0 0 0 0 0 10
11 48 08fe3f1c 362952 349541 13411 0 0 20 4000
12 36 08fe3f48 183498 160937 22561 0 0 8 8000
13 56 08fe3f74 699040 674509 24531 0 0 42 4000
14 84 08fe3fa0 250214 169516 80698 0 0 22 4000
15 96 08fe3fcc 0 0 0 0 0 0 4000
Total Memory Use for Route and Attributes Tables : 104328584
Memory Block Not Available Count : 0
Bad Memory Pool ID Count : 0
TCP buffers : 4096 0 0 0
BGP Tx Parameters : 5 30 3
BGP route update count : 5 (4) last:17h54m45s
event : (5:2) 77.247.181.xxx/28
BGP io semaphore take 54915185, yield 20300844, 14291686
Max timer process: l-5131 s-80740 (463), io: 88163 7431 us
io_rx_yield_time 0x08fe0acc, 1
1 sec timer value: 53364030, 21197655 TB
MP active: 1, standby up 0
Graceful_restart: enable 1, restart time 15, stale-routee 360, purge 600
Restarted 0, fwd 0, restart_up_time_count[0] 0

sh ip bgp debug memory
BGP_CLASS: 198964, BGP_PEER_CLASS: 120071, BGP_CONFIGURATION_CLASS: 3534
BGP_AS_PATH_ENTRY: 84, BGP_IPV6_AS_PATH_ENTRY: 96, BGP_AS_PATH_SEGMENTS: 14
BGP_NLRI_ENTRY: 56, BGP_PATRICIA_KEY_ADDRESS: 16, BGP_PATRICIA_NODE: 48
BGP_RIB_OUT_NLRI_ENTRY: 36, BGP_RIB_OUT_HOLDER: 28, BGP_WITHDRAWN_ROUTE_ENTRY: 34
BGP_NEXTHOP_ADDRESS: 16, BGP_NEXTHOP_ENTRY: 180, BGP_DAMPING_NLRI_ENTRY: 36
BGP_ROUTE_DAMPING_REUSE_LIST: 2052, BGP_ROUTE_DAMPING_BLOCK: 3522, BGP_DAMPENING: 37374
CU_BGP_ROUTE_MAP_SET_T: 234, CU_BGP_ROUTE_MAP_MATCH_T: 1222
BGP_MATCH_TAG: 65, BGP_SET_AS_PATH: 40, BGP_SET_COMMUNITY: 138
BGP_MATCH_UNION: 207, BGP_SET_UNION: 138
ROUTE_FILTER_MEMORY_ENTRY: 220, BGP_MATCH_CRITERIA: 216, BGP_SET_COMMAND: 147

sh ip bgp debug route-table
There are 674476 NLRIs in BGP Route Table, time 323 ms

Best regards,

Pieter Taks

Datum: Fri, 4 Mar 2011 17:03:03 -0600
Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hi,

That's interesting.

Does the RX series support the 'show ip bgp debug' cmd list? It's hidden up to the point of 'debug'. If you can get that far, take a look around in there, especially 'show ip bgp debug memory' & just the plain 'show ip bgp debug'. Maybe something will stick out.

I want to say the router dynamically allocates memory for certain fields up to a certain dynamic max related to other parts of BGP like # of routes. Do you have soft-reconfig turned on? Maybe try turning that off and see if it frees up any more BGP peers. Brocade software can be a little dodgy at times and not well documented.

Also, I wouldn't be surprised if it was just some sort of bug. We've had stranger things happen when it comes to a bug on the Brocades.

Sincerely.

On Tue, Feb 22, 2011 at 8:23 AM, Pieter Taks <p.taks [at] nforce<mailto:p.taks [at] nforce>> wrote:
Hi all,

While configuring a new peer on an BigIron RX4 we got the following error:
Error! BGP4 cannot allocate memory for peer 253

A PDF claims the following (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
BGPv4: Scalable to 4 million routes, 500 peers and 14,000 attributes with MR2 management module

Is there any reason why it wouldn’t want to configure more peering sessions?

- I do not see any system-max setting for this

- As well as I do not see any full memory yet

Hope someone knows how to resolve this or what is causing this error.

sh memory
====================================================================
BigIron RX active MP slot 33:
Total SDRAM: 2147483648 bytes
Available Memory: 1646874624 bytes
Free Physical Pages: 401374 pages

Malloc statistics: total 500462813
os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0, diff: 12877
====================================================================
BigIron RX LP SL 1:
Total SDRAM: 536870912 bytes
Available Memory: 53039104 bytes
====================================================================
BigIron RX LP SL 3:
Total SDRAM: 536870912 bytes
Available Memory: 72671232 bytes
====================================================================
BigIron RX LP SL 4:
Total SDRAM: 536870912 bytes
Available Memory: 72654848 bytes

Any help is appreciated, thanks!

Pieter Taks

_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck<mailto:foundry-nsp [at] puck>
http://puck.nether.net/mailman/listinfo/foundry-nsp

jazzyjpz at gmail

Mar 17, 2011, 3:06 PM

Post #11 of 12 (1368 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hello,

Hmmm, nice find w/the 'Maximum Peer Index Number' under the variables
output. I checked some of our Brocade routers and it appears this number is
dynamic and not static, but it's set ABOVE what our peer count is currently
at(1 router w/less peers was showing 40 above...a router with a lot more
peers had this at over 200 above current neighbor count). Being that you
have maxed this out, I'm guessing either the platform variably alters that
number UP TO a max of 253 or it has hit some sort of memory maximum from
somewhere else which is not allowing it to move beyond 253. I can't tell you
what is happening, but it's quite strange.

Let us know if you figure anything more out with this.

Sincerely.

On Wed, Mar 9, 2011 at 1:15 PM, Pieter Taks <p.taks [at] nforce> wrote:

> Hi,
>
> Thanks for reading with me and thinking what it could be. Even with all
> the soft-reconfigurations disabled I still got the same issue. Please see
> the debug outputs below, perhaps anyone else knows why this is limited to
> 252 BGP peers on an RX4?
>
> In the sh ip bgp debug variables output it shows Maximum Peer Index
> Number:253. Is there a way to change this?
> sh ip bgp debug variables
> safi:0, &bgp:08fe0aac, enabled:1, operational:1, &dbg_mem=08fe1c4c,
> curr_afi:0
> io_process_running:0, io_process_next_peer_number=0
> in_long_loops 0, clear_all 0, timer 00000000, count 0
> timer_enabled:1, timer_next_peer_number:1, 1s timer 1, short timer 1
> scheduler id:5:2, ip:77.247.181.xxx/28, time=92219
> bgp_tcb:08fe0b74 (0xf0090001, 0), tick_cnt=12, seconds=6717786
> bgp_tcb6:f00a0001 (0x00000000, 0x00000019)
> *peer:08fe0bb0, *peer_group:08fe1b5c, RIB_in_root_node:24a35960
> Maximum Peer Index Number:253, check_nexthops:0 0
> router_id:85.159.239.xx, configured:0, cluster_id:0.0.0.13, configured:1
> route_is_router_reflector:0, client_to_client_reflection:1
> networks:x08fe401c, aggregate:x08feb0b8
> default_metric:4294967294, local_preference:100, keep_alive:60,
> hold_time:180
> originate_default:0, originated:0
> distance:20 200 200, fast_external_fallover=0
> nexthop recur0, en_def:0, readvertise:1, auto_sum:0, synch:0
> always_compare_med:1, compare_med_with_empty_aspath: 0,
> redistribute_ibgp:150911764, local_network_check_time_count:6
> nexthop_cache_hit_count:159342982, nexthop_cache_miss_count:5142
> system memory:536870912, total_allocated:104422684,
> bgp_defined_quota:2147483648
>
> sh ip bgp debug
> BGP Debug Information
> Pid Size Address Total Used Free NoMem Errors #_pools p_unit
> 0 8 08fe3d38 43688 31090 12598 0 0 5 2000
> 1 16 08fe3d64 52426 37260 15166 0 0 5 2000
> 2 24 08fe3d90 224692 131923 92769 0 0 10 2000
> 3 32 08fe3dbc 116502 77986 38516 0 0 12 800
> 4 48 08fe3de8 20163 12144 8019 0 0 7 400
> 5 64 08fe3e14 3851 981 2870 0 0 5 200
> 6 96 08fe3e40 2617 397 2220 0 0 6 80
> 7 128 08fe3e6c 2480 1081 1399 0 0 8 40
> 8 256 08fe3e98 755 478 277 0 0 6 20
> 9 512 08fe3ec4 0 0 0 0 0 0 10
> 10 4096 08fe3ef0 0 0 0 0 0 0 10
> 11 48 08fe3f1c 362952 349541 13411 0 0 20 4000
> 12 36 08fe3f48 183498 160937 22561 0 0 8 8000
> 13 56 08fe3f74 699040 674509 24531 0 0 42 4000
> 14 84 08fe3fa0 250214 169516 80698 0 0 22 4000
> 15 96 08fe3fcc 0 0 0 0 0 0 4000
> Total Memory Use for Route and Attributes Tables : 104328584
> Memory Block Not Available Count : 0
> Bad Memory Pool ID Count : 0
> TCP buffers : 4096 0 0 0
> BGP Tx Parameters : 5 30 3
> BGP route update count : 5 (4) last:17h54m45s
> event : (5:2) 77.247.181.xxx/28
> BGP io semaphore take 54915185, yield 20300844, 14291686
> Max timer process: l-5131 s-80740 (463), io: 88163 7431 us
> io_rx_yield_time 0x08fe0acc, 1
> 1 sec timer value: 53364030, 21197655 TB
> MP active: 1, standby up 0
> Graceful_restart: enable 1, restart time 15, stale-routee 360, purge 600
> Restarted 0, fwd 0, restart_up_time_count[0] 0
>
> sh ip bgp debug memory
> BGP_CLASS: 198964, BGP_PEER_CLASS: 120071, BGP_CONFIGURATION_CLASS: 3534
> BGP_AS_PATH_ENTRY: 84, BGP_IPV6_AS_PATH_ENTRY: 96, BGP_AS_PATH_SEGMENTS: 14
> BGP_NLRI_ENTRY: 56, BGP_PATRICIA_KEY_ADDRESS: 16, BGP_PATRICIA_NODE: 48
> BGP_RIB_OUT_NLRI_ENTRY: 36, BGP_RIB_OUT_HOLDER: 28,
> BGP_WITHDRAWN_ROUTE_ENTRY: 34
> BGP_NEXTHOP_ADDRESS: 16, BGP_NEXTHOP_ENTRY: 180, BGP_DAMPING_NLRI_ENTRY: 36
> BGP_ROUTE_DAMPING_REUSE_LIST: 2052, BGP_ROUTE_DAMPING_BLOCK: 3522,
> BGP_DAMPENING: 37374
> CU_BGP_ROUTE_MAP_SET_T: 234, CU_BGP_ROUTE_MAP_MATCH_T: 1222
> BGP_MATCH_TAG: 65, BGP_SET_AS_PATH: 40, BGP_SET_COMMUNITY: 138
> BGP_MATCH_UNION: 207, BGP_SET_UNION: 138
> ROUTE_FILTER_MEMORY_ENTRY: 220, BGP_MATCH_CRITERIA: 216, BGP_SET_COMMAND:
> 147
>
> sh ip bgp debug route-table
> There are 674476 NLRIs in BGP Route Table, time 323 ms
>
> Best regards,
>
>
>
> Pieter Taks
>
>
> Datum: Fri, 4 Mar 2011 17:03:03 -0600
>
> Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4
>
> Hi,
>
> That's interesting.
>
> Does the RX series support the 'show ip bgp debug' cmd list? It's hidden up
> to the point of 'debug'. If you can get that far, take a look around in
> there, especially 'show ip bgp debug memory' & just the plain 'show ip bgp
> debug'. Maybe something will stick out.
>
> I want to say the router dynamically allocates memory for certain fields up
> to a certain dynamic max related to other parts of BGP like # of routes. Do
> you have soft-reconfig turned on? Maybe try turning that off and see if it
> frees up any more BGP peers. Brocade software can be a little dodgy at times
> and not well documented.
>
> Also, I wouldn't be surprised if it was just some sort of bug. We've had
> stranger things happen when it comes to a bug on the Brocades.
>
> Sincerely.
>
> On Tue, Feb 22, 2011 at 8:23 AM, Pieter Taks <p.taks [at] nforce> wrote:
>
>> Hi all,
>>
>>
>>
>> While configuring a new peer on an BigIron RX4 we got the following error:
>>
>> *Error! BGP4 cannot allocate memory for peer 253*
>>
>>
>>
>> A PDF claims the following (
>> http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf
>> ):
>>
>> *BGPv4: **Scalable to 4 million routes, 500 peers and 14,000 attributes
>> with MR2 management module*
>>
>>
>>
>> Is there any reason why it wouldn’t want to configure more peering
>> sessions?
>>
>> - I do not see any system-max setting for this
>>
>> - As well as I do not see any full memory yet
>>
>>
>>
>> Hope someone knows how to resolve this or what is causing this error.
>>
>>
>>
>> sh memory
>>
>> ====================================================================
>>
>> BigIron RX active MP slot 33:
>>
>> Total SDRAM: 2147483648 bytes
>>
>> Available Memory: 1646874624 bytes
>>
>> Free Physical Pages: 401374 pages
>>
>>
>>
>> Malloc statistics: total 500462813
>>
>> os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0,
>> diff: 12877
>>
>> ====================================================================
>>
>> BigIron RX LP SL 1:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 53039104 bytes
>>
>> ====================================================================
>>
>> BigIron RX LP SL 3:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 72671232 bytes
>>
>> ====================================================================
>>
>> BigIron RX LP SL 4:
>>
>> Total SDRAM: 536870912 bytes
>>
>> Available Memory: 72654848 bytes
>>
>>
>>
>>
>>
>> Any help is appreciated, thanks!
>>
>>
>>
>> Pieter Taks
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp [at] puck
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>
>

p.taks at nforce

Mar 19, 2011, 6:08 AM

Post #12 of 12 (1298 views)
Permalink

Re: Max BGP peers BigIron RX4 [In reply to]

Hi John,

I have found a work-around for it from Martijn and Johan who read these lists as well. They told me that when we change the max-vlans and the session-limit setting to a lower value it will create more "room" for the BGP sessions. I have tested this and it works properly. Keep in mind that in the newest release the max-vlan setting does not define the highest vlan number that is possible, it merely defines the maximum amount of vlans.

One thing I do wonder is what is the use for the session-limit, what does it actually "limit" and how can I read out how many session's the router is currently really needing/using.

And ofcourse it would be nice to be able to read out what now my maximum amount of BGP sessions is :)

Thanks for thinking with me. And I thought I should post this for others who search for this issue.

Pieter Taks

Van: John J <jazzyjpz [at] gmail<mailto:jazzyjpz [at] gmail>>
Datum: Thu, 17 Mar 2011 17:06:29 -0500
Aan: Pieter Taks <p.taks [at] nforce<mailto:p.taks [at] nforce>>
CC: "foundry-nsp [at] puck<mailto:foundry-nsp [at] puck>" <foundry-nsp [at] puck<mailto:foundry-nsp [at] puck>>
Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hello,

Hmmm, nice find w/the 'Maximum Peer Index Number' under the variables output. I checked some of our Brocade routers and it appears this number is dynamic and not static, but it's set ABOVE what our peer count is currently at(1 router w/less peers was showing 40 above...a router with a lot more peers had this at over 200 above current neighbor count). Being that you have maxed this out, I'm guessing either the platform variably alters that number UP TO a max of 253 or it has hit some sort of memory maximum from somewhere else which is not allowing it to move beyond 253. I can't tell you what is happening, but it's quite strange.

Let us know if you figure anything more out with this.

Sincerely.

On Wed, Mar 9, 2011 at 1:15 PM, Pieter Taks <p.taks [at] nforce<mailto:p.taks [at] nforce>> wrote:
Hi,

Thanks for reading with me and thinking what it could be. Even with all the soft-reconfigurations disabled I still got the same issue. Please see the debug outputs below, perhaps anyone else knows why this is limited to 252 BGP peers on an RX4?

In the sh ip bgp debug variables output it shows Maximum Peer Index Number:253. Is there a way to change this?
sh ip bgp debug variables
safi:0, &bgp:08fe0aac, enabled:1, operational:1, &dbg_mem=08fe1c4c, curr_afi:0
io_process_running:0, io_process_next_peer_number=0
in_long_loops 0, clear_all 0, timer 00000000, count 0
timer_enabled:1, timer_next_peer_number:1, 1s timer 1, short timer 1
scheduler id:5:2, ip:77.247.181.xxx/28, time=92219
bgp_tcb:08fe0b74 (0xf0090001, 0), tick_cnt=12, seconds=6717786
bgp_tcb6:f00a0001 (0x00000000, 0x00000019)
*peer:08fe0bb0, *peer_group:08fe1b5c, RIB_in_root_node:24a35960
Maximum Peer Index Number:253, check_nexthops:0 0
router_id:85.159.239.xx, configured:0, cluster_id:0.0.0.13, configured:1
route_is_router_reflector:0, client_to_client_reflection:1
networks:x08fe401c, aggregate:x08feb0b8
default_metric:4294967294, local_preference:100, keep_alive:60, hold_time:180
originate_default:0, originated:0
distance:20 200 200, fast_external_fallover=0
nexthop recur0, en_def:0, readvertise:1, auto_sum:0, synch:0
always_compare_med:1, compare_med_with_empty_aspath: 0, redistribute_ibgp:150911764, local_network_check_time_count:6
nexthop_cache_hit_count:159342982, nexthop_cache_miss_count:5142
system memory:536870912, total_allocated:104422684, bgp_defined_quota:2147483648

sh ip bgp debug
BGP Debug Information
Pid Size Address Total Used Free NoMem Errors #_pools p_unit
0 8 08fe3d38 43688 31090 12598 0 0 5 2000
1 16 08fe3d64 52426 37260 15166 0 0 5 2000
2 24 08fe3d90 224692 131923 92769 0 0 10 2000
3 32 08fe3dbc 116502 77986 38516 0 0 12 800
4 48 08fe3de8 20163 12144 8019 0 0 7 400
5 64 08fe3e14 3851 981 2870 0 0 5 200
6 96 08fe3e40 2617 397 2220 0 0 6 80
7 128 08fe3e6c 2480 1081 1399 0 0 8 40
8 256 08fe3e98 755 478 277 0 0 6 20
9 512 08fe3ec4 0 0 0 0 0 0 10
10 4096 08fe3ef0 0 0 0 0 0 0 10
11 48 08fe3f1c 362952 349541 13411 0 0 20 4000
12 36 08fe3f48 183498 160937 22561 0 0 8 8000
13 56 08fe3f74 699040 674509 24531 0 0 42 4000
14 84 08fe3fa0 250214 169516 80698 0 0 22 4000
15 96 08fe3fcc 0 0 0 0 0 0 4000
Total Memory Use for Route and Attributes Tables : 104328584
Memory Block Not Available Count : 0
Bad Memory Pool ID Count : 0
TCP buffers : 4096 0 0 0
BGP Tx Parameters : 5 30 3
BGP route update count : 5 (4) last:17h54m45s
event : (5:2) 77.247.181.xxx/28
BGP io semaphore take 54915185, yield 20300844, 14291686
Max timer process: l-5131 s-80740 (463), io: 88163 7431 us
io_rx_yield_time 0x08fe0acc, 1
1 sec timer value: 53364030, 21197655 TB
MP active: 1, standby up 0
Graceful_restart: enable 1, restart time 15, stale-routee 360, purge 600
Restarted 0, fwd 0, restart_up_time_count[0] 0

sh ip bgp debug memory
BGP_CLASS: 198964, BGP_PEER_CLASS: 120071, BGP_CONFIGURATION_CLASS: 3534
BGP_AS_PATH_ENTRY: 84, BGP_IPV6_AS_PATH_ENTRY: 96, BGP_AS_PATH_SEGMENTS: 14
BGP_NLRI_ENTRY: 56, BGP_PATRICIA_KEY_ADDRESS: 16, BGP_PATRICIA_NODE: 48
BGP_RIB_OUT_NLRI_ENTRY: 36, BGP_RIB_OUT_HOLDER: 28, BGP_WITHDRAWN_ROUTE_ENTRY: 34
BGP_NEXTHOP_ADDRESS: 16, BGP_NEXTHOP_ENTRY: 180, BGP_DAMPING_NLRI_ENTRY: 36
BGP_ROUTE_DAMPING_REUSE_LIST: 2052, BGP_ROUTE_DAMPING_BLOCK: 3522, BGP_DAMPENING: 37374
CU_BGP_ROUTE_MAP_SET_T: 234, CU_BGP_ROUTE_MAP_MATCH_T: 1222
BGP_MATCH_TAG: 65, BGP_SET_AS_PATH: 40, BGP_SET_COMMUNITY: 138
BGP_MATCH_UNION: 207, BGP_SET_UNION: 138
ROUTE_FILTER_MEMORY_ENTRY: 220, BGP_MATCH_CRITERIA: 216, BGP_SET_COMMAND: 147

sh ip bgp debug route-table
There are 674476 NLRIs in BGP Route Table, time 323 ms

Best regards,

Pieter Taks

Datum: Fri, 4 Mar 2011 17:03:03 -0600

Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hi,

That's interesting.

Does the RX series support the 'show ip bgp debug' cmd list? It's hidden up to the point of 'debug'. If you can get that far, take a look around in there, especially 'show ip bgp debug memory' & just the plain 'show ip bgp debug'. Maybe something will stick out.

I want to say the router dynamically allocates memory for certain fields up to a certain dynamic max related to other parts of BGP like # of routes. Do you have soft-reconfig turned on? Maybe try turning that off and see if it frees up any more BGP peers. Brocade software can be a little dodgy at times and not well documented.

Also, I wouldn't be surprised if it was just some sort of bug. We've had stranger things happen when it comes to a bug on the Brocades.

Sincerely.

On Tue, Feb 22, 2011 at 8:23 AM, Pieter Taks <p.taks [at] nforce<mailto:p.taks [at] nforce>> wrote:
Hi all,

While configuring a new peer on an BigIron RX4 we got the following error:
Error! BGP4 cannot allocate memory for peer 253

A PDF claims the following (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
BGPv4: Scalable to 4 million routes, 500 peers and 14,000 attributes with MR2 management module

Is there any reason why it wouldn’t want to configure more peering sessions?

- I do not see any system-max setting for this

- As well as I do not see any full memory yet

Hope someone knows how to resolve this or what is causing this error.

sh memory
====================================================================
BigIron RX active MP slot 33:
Total SDRAM: 2147483648 bytes
Available Memory: 1646874624 bytes
Free Physical Pages: 401374 pages

Malloc statistics: total 500462813
os_malloc count: 24059060, fail: 3; os_free count: 24046183, fail 0, diff: 12877
====================================================================
BigIron RX LP SL 1:
Total SDRAM: 536870912 bytes
Available Memory: 53039104 bytes
====================================================================
BigIron RX LP SL 3:
Total SDRAM: 536870912 bytes
Available Memory: 72671232 bytes
====================================================================
BigIron RX LP SL 4:
Total SDRAM: 536870912 bytes
Available Memory: 72654848 bytes

Any help is appreciated, thanks!

Pieter Taks

_______________________________________________
foundry-nsp mailing list
foundry-nsp [at] puck<mailto:foundry-nsp [at] puck>
http://puck.nether.net/mailman/listinfo/foundry-nsp

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads