Mailing List Archive: mod_backhand: Wackamole

wackamole high cpu utilization on FreeBSD 4.8

Index | Next | Previous | View Threaded

bill at squadengine

Nov 17, 2003, 10:06 PM

Post #1 of 4 (902 views)
Permalink

wackamole high cpu utilization on FreeBSD 4.8

All-

After starting and running wackamole 2.0 on a 2 node spread group, after a
while it starts to run out of control and utilize as much CPU as it can. If
I look at /var/log/messages, I see the following.

Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
Nov 17 19:15:52 db1 named[29126]: deleting interface [66.151.149.110].53
Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
Nov 17 19:16:22 db1 last message repeated 513398 times
Nov 17 19:18:23 db1 last message repeated 2039749 times
Nov 17 19:28:25 db1 last message repeated 10119748 times
Nov 17 19:38:25 db1 last message repeated 10116695 times
Nov 17 19:48:26 db1 last message repeated 10123830 times
Nov 17 19:58:27 db1 last message repeated 10133955 times
Nov 17 20:08:28 db1 last message repeated 10107217 times
Nov 17 20:15:52 db1 last message repeated 7470197 times

I've seen this on either of the 2 nodes. Wackamole is the only application
I have using the spread group. Configuration files follow. Any ideas?

(btw, both nodes are dual homed... 2 phsyical interfaces em0 and em1, they
communicate over an internal network on 192.169.100.x, and face the clients
on 66.151.149.x)

Right now only db1 and web1 exist, but I wanted to have the planned nodes
created to limit the amount of disruption when bringing a new node online
in the future.

============
# cat /usr/local/etc/spread.conf
DangerousMonitor = false

Spread_Segment 192.168.100.255:4803 {
db1 192.168.100.10
web1 192.168.100.101
web2 192.168.100.102
web3 192.168.100.103
web4 192.168.100.104
web5 192.168.100.105
web6 192.168.100.106
}

DaemonUser = spread
DaemonGroup = spread

============
# cat /usr/local/etc/wackamole.conf
Spread = 4803
SpreadRetryInterval = 5s
Group = wack1
Control = /var/run/wack.it
Prefer None

VirtualInterfaces {
{ em0:66.151.149.101/32 }
{ em0:66.151.149.102/32 }
{ em0:66.151.149.103/32 }
{ em0:66.151.149.104/32 }
{ em0:66.151.149.105/32 }
{ em0:66.151.149.106/32 }
{ em0:66.151.149.107/32 }
{ em0:66.151.149.108/32 }
{ em0:66.151.149.109/32 }
{ em0:66.151.149.110/32 }
}
Arp-Cache = 90s

Notify {
arp-cache
}
balance {
# This field is the maximum number of IP addresses that will move
# from one wackamole to another during a round of balancing.
AcquisitionsPerRound = 1
# Time interval in each balancing round.
interval = 4s
}
# How long it takes us to mature
mature = 5s

--
Bill Plein
SquadEngine.com

jesus at omniti

Nov 18, 2003, 9:49 AM

Post #2 of 4 (828 views)
Permalink

wackamole high cpu utilization on FreeBSD 4.8 [In reply to]

On Nov 17, 2003, at 9:06 PM, Bill Plein wrote:
> Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
> Nov 17 19:15:52 db1 named[29126]: deleting interface
> [66.151.149.110].53
> Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
> Nov 17 19:16:22 db1 last message repeated 513398 times
> Nov 17 19:18:23 db1 last message repeated 2039749 times
> Nov 17 19:28:25 db1 last message repeated 10119748 times
> Nov 17 19:38:25 db1 last message repeated 10116695 times
> Nov 17 19:48:26 db1 last message repeated 10123830 times
> Nov 17 19:58:27 db1 last message repeated 10133955 times
> Nov 17 20:08:28 db1 last message repeated 10107217 times
> Nov 17 20:15:52 db1 last message repeated 7470197 times

What OS and version are you running?

That message happens when the wackatrl control socket becomes readable,
but accepting on it returns an error. Can you shut it down and delete
the /var/run/wack.it file and make sure it isn't touched by any other
program? Then start things back up again.

Are you using the wackatrl tool for anything and can you correlate
these problems with its use?

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on earth

bill at squadengine

Nov 18, 2003, 10:05 AM

Post #3 of 4 (823 views)
Permalink

wackamole high cpu utilization on FreeBSD 4.8 [In reply to]

My apologies!

FreeBSD 4.8, SMP kernel:

FreeBSD db1.squadengine.com 4.8-RELEASE-p4 FreeBSD 4.8-RELEASE-p4 #11: Sun
Sep 14 23:37:59 PDT
2003 root[at]test.squadengine.com:/usr/obj/usr/src/sys/SMP i386

I haven't run wackatrl at all, because frankly I am an admin, not a
programmer, and I haven't found any documentation on it (grin).

I'll give it all a try.

At 08:49 AM 11/18/2003 -0800, you wrote:
>On Nov 17, 2003, at 9:06 PM, Bill Plein wrote:
>>Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
>>Nov 17 19:15:52 db1 named[29126]: deleting interface [66.151.149.110].53
>>Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
>>Nov 17 19:16:22 db1 last message repeated 513398 times
>>Nov 17 19:18:23 db1 last message repeated 2039749 times
>>Nov 17 19:28:25 db1 last message repeated 10119748 times
>>Nov 17 19:38:25 db1 last message repeated 10116695 times
>>Nov 17 19:48:26 db1 last message repeated 10123830 times
>>Nov 17 19:58:27 db1 last message repeated 10133955 times
>>Nov 17 20:08:28 db1 last message repeated 10107217 times
>>Nov 17 20:15:52 db1 last message repeated 7470197 times
>
>What OS and version are you running?
>
>That message happens when the wackatrl control socket becomes readable,
>but accepting on it returns an error. Can you shut it down and delete the
>/var/run/wack.it file and make sure it isn't touched by any other
>program? Then start things back up again.
>
>Are you using the wackatrl tool for anything and can you correlate these
>problems with its use?
>
>// Theo Schlossnagle
>// Principal Engineer -- http://www.omniti.com/~jesus/
>// Postal Engine -- http://www.postalengine.com/
>// Ecelerity: fastest MTA on earth
>
>
>_______________________________________________
>wackamole-users mailing list
>wackamole-users[at]lists.backhand.org
>http://lists.backhand.org/mailman/listinfo/wackamole-users

--
Bill Plein
SquadEngine.com

bill at squadengine

Nov 19, 2003, 1:43 PM

Post #4 of 4 (817 views)
Permalink

wackamole high cpu utilization on FreeBSD 4.8 [In reply to]

>On Nov 17, 2003, at 9:06 PM, Bill Plein wrote:
>>Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
>>Nov 17 19:15:52 db1 named[29126]: deleting interface [66.151.149.110].53
>>Nov 17 19:15:52 db1 wackamole[41914]: error receiving wackatrl session
>>Nov 17 19:16:22 db1 last message repeated 513398 times
>>Nov 17 19:18:23 db1 last message repeated 2039749 times
>>Nov 17 19:28:25 db1 last message repeated 10119748 times
>>Nov 17 19:38:25 db1 last message repeated 10116695 times
>>Nov 17 19:48:26 db1 last message repeated 10123830 times
>>Nov 17 19:58:27 db1 last message repeated 10133955 times
>>Nov 17 20:08:28 db1 last message repeated 10107217 times
>>Nov 17 20:15:52 db1 last message repeated 7470197 times
>
>What OS and version are you running?
>
>That message happens when the wackatrl control socket becomes readable,
>but accepting on it returns an error. Can you shut it down and delete the
>/var/run/wack.it file and make sure it isn't touched by any other
>program? Then start things back up again.
>
>Are you using the wackatrl tool for anything and can you correlate these
>problems with its use?

Well, it is reproducing rather rapidly. After wackamole goes out of
control, I kill it, and delete /var/run/wack.it. There is nothing that is
touching it (single wackamole and spread instance on each node).

Is there a wackatrl command that can let me test this socket?

--
Bill Plein
SquadEngine.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com