Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Quagga: Users

BGP stuck in clearing state

 

 

Quagga users RSS feed   Index | Next | Previous | View Threaded


Preeti.Khurana at guavus

Mar 29, 2012, 5:33 AM

Post #1 of 3 (329 views)
Permalink
BGP stuck in clearing state

Hi,
I am seeing a strange issue of bgp flapping on my box. Am using quagga 0.99.16 version. The bgp peering is done with the route reflector handling some 100 routers. The flapping happens after 3-4 days of running bgpd process.

Show ip bgp summary at that point shows ::

show ip bgp summary
BGP router identifier 129.250.15.197, local AS number 65000
RIB entries 746327, using 68 MiB of memory
Peers 110, using 490 KiB of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
129.250.0.1 4 65000 456197 3921 0 0 0 10:32:22 Clearing
129.250.0.2 4 65000 495413 3921 0 0 0 10:29:11 Clearing
129.250.0.3 4 65000 776670 7632 0 0 0 10:45:36 Clearing
129.250.0.4 4 65000 795819 7631 0 0 0 10:41:22 Clearing
129.250.0.5 4 65000 471596 3921 0 0 0 10:29:39 Clearing
129.250.0.6 4 65000 598958 4149 0 0 0 06:11:52 29214
129.250.0.7 4 65000 774747 7632 0 0 0 10:42:31 Clearing
129.250.0.8 4 65000 777994 7630 0 0 0 10:42:26 Clearing
129.250.0.9 4 65000 448145 3921 0 0 0 10:31:33 Clearing
129.250.0.10 4 65000 746011 7632 0 0 0 10:44:17 Clearing
129.250.0.12 4 65000 772012 7631 0 0 0 10:45:27 Clearing
129.250.0.13 4 65000 757875 7632 0 0 0 10:45:15 Clearing
129.250.0.14 4 65000 467368 3921 0 0 0 10:29:33 Clearing
129.250.0.15 4 65000 774815 7632 0 0 0 10:44:45 Clearing
129.250.0.17 4 65000 883354 7632 0 0 0 10:42:56 Clearing
129.250.0.19 4 65000 462678 3921 0 0 0 10:31:45 Clearing
129.250.0.20 4 65000 485906 3921 0 0 0 10:32:16 Clearing
129.250.0.21 4 65000 478448 3921 0 0 0 10:32:40 Clearing
129.250.0.22 4 65000 461430 3921 0 0 0 10:31:39 Clearing
129.250.0.23 4 65000 456191 3921 0 0 0 10:32:34 Clearing
129.250.0.24 4 65000 837916 7632 0 0 0 10:43:59 Clearing
129.250.0.25 4 65000 832380 7632 0 0 0 10:41:29 Clearing
129.250.0.28 4 65000 786698 7631 0 0 0 10:40:44 Clearing
129.250.0.32 4 65000 796845 7632 0 0 0 10:40:34 Clearing
129.250.0.33 4 65000 452460 3921 0 0 0 10:29:45 Clearing
129.250.0.38 4 65000 797894 7632 0 0 0 10:43:56 Clearing
129.250.0.41 4 65000 457306 3922 0 0 0 10:28:55 Clearing



The status becomes clearing.



Also the memory utilization of bgpd process becomes huge. Reaches to 36 G from 4 G . The show memory output at that time is :: ( Look at the Link Node & Hash Bucket).

====================================================================================

System allocator statistics:
Total heap allocated: 0 bytes
Holding block headers: 0 bytes
Used small blocks: 0 bytes
Used ordinary blocks: 0 bytes
Free small blocks: 0 bytes
Free ordinary blocks: 0 bytes
Ordinary blocks: 1
Small blocks: 0
Holding blocks: 0
(see system documentation for 'mallinfo' for meaning)
-----------------------------
Temporary memory : 2
String vector : 13874
Vector : 6946
Vector index : 6946
Link List : 117
Link Node : 471093196
Thread : 454
Thread master : 1
Thread stats : 18
Thread function name : 333
VTY : 2
Buffer : 3
Buffer data : 1
Stream : 227
Stream data : 227
Stream FIFO : 111
Hash : 895
Hash Bucket : 6458673
Hash Index : 895
Route map : 1
Route map name : 1
Route map index : 1
Command desc : 6947
Socket union : 218
Privilege information : 2
Logging : 1
Zclient : 2
Work queue : 111
Work queue item : 471092973
Work queue name string : 111
Host config : 5

-----------------------------
BGP instance : 1
BGP listen socket details : 1
BGP peer : 111
BGP peer hostname : 111
Peer description : 109
BGP attribute : 6360361
BGP extra attributes : 5862758
BGP aspath : 93737
BGP aspath seg : 95515
BGP aspath segment data : 95515
BGP aspath str : 93737
-----------------------------
BGP table : 31
BGP node : 746327
BGP route : 37603790
BGP synchronise : 888
-----------------------------
community : 3802
community val : 3802
community str : 3802
-----------------------------
extcommunity : 47
extcommunity val : 47
extcommunity str : 47
-----------------------------
community-list handler : 1
-----------------------------
Cluster list : 708
Cluster list val : 708
-----------------------------
BGP Process queue : 449521269

BGP node clear queue : 21571704
====================================================================================



Does have anyone has a clue on what can be done to fix this or is it already fixed ?


Please copy me explicitly on this as I am not subscribed. ( preeti.khurana [at] guavus<mailto:preeti.khurana [at] guavus>)



Thanks
Preeti


Preeti.Khurana at guavus

Mar 29, 2012, 5:48 AM

Post #2 of 3 (333 views)
Permalink
Re: BGP stuck in clearing state [In reply to]

Also, am getting following logs ::



2012/03/28 05:16:20 BGP: 129.250.0.50 [FSM] Receive_NOTIFICATION_message

(Established->Clearing)

2012/03/28 05:16:52 BGP: [Event] BGP connection from host 129.250.0.50

2012/03/28 05:16:52 BGP: 129.250.0.50 [FSM] TCP_connection_open

(Active->OpenSent)

2012/03/28 05:16:57 BGP: 129.250.0.50 peer status is Clearing close connection

2012/03/28 05:16:57 BGP: 129.250.0.50 [Event] Accepting BGP peer delete

2012/03/28 05:18:12 BGP: [Event] BGP connection from host 129.250.0.50

2012/03/28 05:18:12 BGP: 129.250.0.50 [FSM] TCP_connection_open

(Active->OpenSent)

2012/03/28 05:18:17 BGP: 129.250.0.50 peer status is Clearing close connection





From: Preeti Khurana <preeti.khurana [at] guavus<mailto:preeti.khurana [at] guavus>>
Reply-To: Quagga Users Lists <quagga-users [at] lists<mailto:quagga-users [at] lists>>
Date: Thu, 29 Mar 2012 12:33:16 +0000
To: "quagga-users [at] lists<mailto:quagga-users [at] lists>" <quagga-users [at] lists<mailto:quagga-users [at] lists>>
Subject: [quagga-users 12831] BGP stuck in clearing state

Hi,
I am seeing a strange issue of bgp flapping on my box. Am using quagga 0.99.16 version. The bgp peering is done with the route reflector handling some 100 routers. The flapping happens after 3-4 days of running bgpd process.

Show ip bgp summary at that point shows ::

show ip bgp summary
BGP router identifier 129.250.15.197, local AS number 65000
RIB entries 746327, using 68 MiB of memory
Peers 110, using 490 KiB of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
129.250.0.1 4 65000 456197 3921 0 0 0 10:32:22 Clearing
129.250.0.2 4 65000 495413 3921 0 0 0 10:29:11 Clearing
129.250.0.3 4 65000 776670 7632 0 0 0 10:45:36 Clearing
129.250.0.4 4 65000 795819 7631 0 0 0 10:41:22 Clearing
129.250.0.5 4 65000 471596 3921 0 0 0 10:29:39 Clearing
129.250.0.6 4 65000 598958 4149 0 0 0 06:11:52 29214
129.250.0.7 4 65000 774747 7632 0 0 0 10:42:31 Clearing
129.250.0.8 4 65000 777994 7630 0 0 0 10:42:26 Clearing
129.250.0.9 4 65000 448145 3921 0 0 0 10:31:33 Clearing
129.250.0.10 4 65000 746011 7632 0 0 0 10:44:17 Clearing
129.250.0.12 4 65000 772012 7631 0 0 0 10:45:27 Clearing
129.250.0.13 4 65000 757875 7632 0 0 0 10:45:15 Clearing
129.250.0.14 4 65000 467368 3921 0 0 0 10:29:33 Clearing
129.250.0.15 4 65000 774815 7632 0 0 0 10:44:45 Clearing
129.250.0.17 4 65000 883354 7632 0 0 0 10:42:56 Clearing
129.250.0.19 4 65000 462678 3921 0 0 0 10:31:45 Clearing
129.250.0.20 4 65000 485906 3921 0 0 0 10:32:16 Clearing
129.250.0.21 4 65000 478448 3921 0 0 0 10:32:40 Clearing
129.250.0.22 4 65000 461430 3921 0 0 0 10:31:39 Clearing
129.250.0.23 4 65000 456191 3921 0 0 0 10:32:34 Clearing
129.250.0.24 4 65000 837916 7632 0 0 0 10:43:59 Clearing
129.250.0.25 4 65000 832380 7632 0 0 0 10:41:29 Clearing
129.250.0.28 4 65000 786698 7631 0 0 0 10:40:44 Clearing
129.250.0.32 4 65000 796845 7632 0 0 0 10:40:34 Clearing
129.250.0.33 4 65000 452460 3921 0 0 0 10:29:45 Clearing
129.250.0.38 4 65000 797894 7632 0 0 0 10:43:56 Clearing
129.250.0.41 4 65000 457306 3922 0 0 0 10:28:55 Clearing



The status becomes clearing.



Also the memory utilization of bgpd process becomes huge. Reaches to 36 G from 4 G . The show memory output at that time is :: ( Look at the Link Node & Hash Bucket).

====================================================================================

System allocator statistics:
Total heap allocated: 0 bytes
Holding block headers: 0 bytes
Used small blocks: 0 bytes
Used ordinary blocks: 0 bytes
Free small blocks: 0 bytes
Free ordinary blocks: 0 bytes
Ordinary blocks: 1
Small blocks: 0
Holding blocks: 0
(see system documentation for 'mallinfo' for meaning)
-----------------------------
Temporary memory : 2
String vector : 13874
Vector : 6946
Vector index : 6946
Link List : 117
Link Node : 471093196
Thread : 454
Thread master : 1
Thread stats : 18
Thread function name : 333
VTY : 2
Buffer : 3
Buffer data : 1
Stream : 227
Stream data : 227
Stream FIFO : 111
Hash : 895
Hash Bucket : 6458673
Hash Index : 895
Route map : 1
Route map name : 1
Route map index : 1
Command desc : 6947
Socket union : 218
Privilege information : 2
Logging : 1
Zclient : 2
Work queue : 111
Work queue item : 471092973
Work queue name string : 111
Host config : 5

-----------------------------
BGP instance : 1
BGP listen socket details : 1
BGP peer : 111
BGP peer hostname : 111
Peer description : 109
BGP attribute : 6360361
BGP extra attributes : 5862758
BGP aspath : 93737
BGP aspath seg : 95515
BGP aspath segment data : 95515
BGP aspath str : 93737
-----------------------------
BGP table : 31
BGP node : 746327
BGP route : 37603790
BGP synchronise : 888
-----------------------------
community : 3802
community val : 3802
community str : 3802
-----------------------------
extcommunity : 47
extcommunity val : 47
extcommunity str : 47
-----------------------------
community-list handler : 1
-----------------------------
Cluster list : 708
Cluster list val : 708
-----------------------------
BGP Process queue : 449521269

BGP node clear queue : 21571704
====================================================================================



Does have anyone has a clue on what can be done to fix this or is it already fixed ?


Please copy me explicitly on this as I am not subscribed. ( preeti.khurana [at] guavus<mailto:preeti.khurana [at] guavus>)



Thanks
Preeti
_______________________________________________ Quagga-users mailing list Quagga-users [at] lists<mailto:Quagga-users [at] lists> http://lists.quagga.net/mailman/listinfo/quagga-users


paul at jakma

Mar 29, 2012, 6:14 AM

Post #3 of 3 (306 views)
Permalink
Re: BGP stuck in clearing state [In reply to]

On Thu, 29 Mar 2012, Preeti Khurana wrote:

> Hi,

> I am seeing a strange issue of bgp flapping on my box. Am using quagga
> 0.99.16 version.

I can't think of any specific fixes that would make a difference, but it
could be a useful data-point to reproduce with the latest 0.99.20.1
release.

> Also the memory utilization of bgpd process becomes huge. Reaches to 36
> G from 4 G . The show memory output at that time is :: ( Look at the
> Link Node & Hash Bucket).

> Work queue item : 471092973

Sounds like it's struggling to process the route updates due to the
flapping. I'm sure "show work-queues" shows there's a lot of work waiting
to be done.

Are you using a low hold-time? Perhaps it's just too low to cope with
mass-resets.

regards,
--
Paul Jakma paul [at] jakma twitter: @pjakma PGP: 64A2FF6A
Fortune:
Personifiers of the world, unite! You have nothing to lose but Mr. Dignity!
-- Bernadette Bosky
_______________________________________________
Quagga-users mailing list
Quagga-users [at] lists
http://lists.quagga.net/mailman/listinfo/quagga-users

Quagga users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.