Mailing List Archive: mod_backhand: Wackamole

Wackamole problem with arp cache on firewall/gateway

Index | Next | Previous | View Threaded

rama at pitties

Dec 11, 2003, 11:48 AM

Post #1 of 6 (1289 views)
Permalink

Wackamole problem with arp cache on firewall/gateway

Hello Wackamole users,

I have a two servers (1 is RLH 9.0 and 1 is RHL 7.3) and am using
wackamole 2.0.0 and spread 3.17.1. Each server has 3 Nics. I have
configured a single virtual IP on eth0.

Everything works perfect, except my firewall/gateway arp cache does not
get updated. My firewall is not running Wackamole but I thought the
Notify part of wackamole would tell my firewall to refresh it's arp
cache. If I manually delete the arp entry in my firewall everything
works after wackamole changes the owner of my virtual IP. Actually,
only the two machines running wackamole seem to have good arp caches.
Every other machine (solaris) on my subnet has a stale arp entry.

Here is my conf (with the IPs changed to protect the guilty):

# The Spread daemon we are going to connect to. It should be on the
local box
Spread = 4803
SpreadRetryInterval = 5s
# The group name
Group = wack1
# Named socket for online control
Control = /var/run/wack.it

# Denote the interface we prefer to have
#prefer eth0:10.3.4.5/8
#prefer { eth0:10.2.3.4/8 eth1:192.168.10.23/24 }

# In most cases, I just don't care. Let wackamole decide.
# If both servers are working, this server should have the virtual IP
Prefer { eth0:10.10.10.25/32 }

# List all the virtual interfaces (ALL of them)
VirtualInterfaces {
# The following two lines have the same effect
# en0:192.168.1.2/24
{ eth0:10.10.10.25/32 }
# This is how you say 2 or more IPs are to be treated as a single
# "set" or "virtual interface". If wackamole decides that this
# machine will manage it, you are ensured to get ALL the ips in the
# set.
# { en1:10.0.0.1/8 en0:192.168.35.64/26 }
}

# Collect and broadcast the IPs in our ARP table every so often
Arp-Cache = 90s

# List who we will notify
# Here the netblock (/24 or /28) can be deceptive. It is NOT a
netmask
# for a single IP. It is how one will describe that they want to
# notify ALL IPs in a segment.
Notify {
# Let's notify our router: ***** My firwall, 10.10.10.1's arp
cache becomes stale, this is my problem **********
eth0:10.10.10.1/32
# Notify out DB server on eth1
eth1:10.10.11.5/32
# 10.0.0.0 -> 10.0.0.255, but only 128 notifications/sec
eth0:10.10.10.0/24 throttle 128 #
***** appearantly this doesn't fix stale arp caches either on
solaris boxes on my subnet *******
# Wackamole shares arp-cache across machines, this says to
# notify every IP address in the aggregate shared arp-cache.
arp-cache
}
balance {
# This field is the maximum number of IP addresses that will move
# from one wackamole to another during a round of balancing.
AcquisitionsPerRound = all
# Time interval in each balancing round.
interval = 4s
}
# How long it takes us to mature
mature = 5s

Thank you,
-Rama McIntosh

jesus at omniti

Dec 11, 2003, 12:13 PM

Post #2 of 6 (1182 views)
Permalink

Wackamole problem with arp cache on firewall/gateway [In reply to]

On Dec 11, 2003, at 1:48 PM, Rama K. McIntosh wrote:
> Everything works perfect, except my firewall/gateway arp cache does
> not get updated. My firewall is not running Wackamole but I thought
> the Notify part of wackamole would tell my firewall to refresh it's
> arp cache. If I manually delete the arp entry in my firewall
> everything works after wackamole changes the owner of my virtual IP.
> Actually, only the two machines running wackamole seem to have good
> arp caches. Every other machine (solaris) on my subnet has a stale
> arp entry.
>
> Here is my conf (with the IPs changed to protect the guilty):
> Notify {
> # Let's notify our router: ***** My firwall, 10.10.10.1's arp
> cache becomes stale, this is my problem **********
> eth0:10.10.10.1/32

What kind of firewall is it?

So. Try adding eth0:0.0.0.0/32 and eth0:255.255.255.0/32 and and
eth0:networkaddress/32 and eth0:broadcastaddress/32 (where network
address or broadcast address is the IP network address and IP broadcast
address on that subnet, respectively).

Different devices and OSs respond differently to unsolicited ARP
responses. Some take "protective measures" to disable this to prevent
"arp-spoofing" attacks. On FreeBSD/NetBSD/OpenBSD I believe there is a
kernel option to allow/disallow this. Sometimes systems are more
receptive to unsolicited arp resopnses to IP network, IP broadcast, IP
subnet, or IP subnet broadcast addresses.

Check the logs on your firewall to see if it sees the ARP request but
is rejecting it.

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on earth

rama at pitties

Dec 11, 2003, 1:30 PM

Post #3 of 6 (1189 views)
Permalink

Wackamole problem with arp cache on firewall/gateway [In reply to]

Theo Schlossnagle wrote:

> What kind of firewall is it?

OpenBSD 3.4. I can't find anything in the log or pf log files. I'm
research for the kernel config option. If anyone knows it (and if it
can be set for only my interal interafce) I would appreciate it.

>
> So. Try adding eth0:0.0.0.0/32 and eth0:255.255.255.0/32 and and
> eth0:networkaddress/32 and eth0:broadcastaddress/32 (where network
> address or broadcast address is the IP network address and IP
> broadcast address on that subnet, respectively).

Tried this, no luck yet.

>
> Different devices and OSs respond differently to unsolicited ARP
> responses. Some take "protective measures" to disable this to prevent
> "arp-spoofing" attacks. On FreeBSD/NetBSD/OpenBSD I believe there is
> a kernel option to allow/disallow this. Sometimes systems are more
> receptive to unsolicited arp resopnses to IP network, IP broadcast, IP
> subnet, or IP subnet broadcast addresses.
>
> Check the logs on your firewall to see if it sees the ARP request but
> is rejecting it.

It looks like it may have worked once (I see some arp info overwritten
messgages). However, I can not seem to make it work consistantly.

Thanks,
-Rama

rama at pitties

Dec 13, 2003, 5:32 PM

Post #4 of 6 (1182 views)
Permalink

Wackamole problem with arp cache on firewall/gateway [In reply to]

I think this might be a bug/missing feature in Wackamole 2.0.0 After
much messing around I figured out arprelease
(http://arprelease.sourceforge.net/ ) had no problem remotely flushing
my firewall's arp table.

Thus I theorized it was a bug in wackamole and tried Wackamole from CVS
(as of this evening) and everything works fine! I did some diff's and
looked at some cvs logs but there were lots of changes and I was unable
to figure out what made the CVS version work and the 2.0.0 fail.

So My questions:

1. Is the Wackamole CVS version stable enough for production use?

2. Any idea why the CVS version can remotely update the arp cache on my
firewall, but 2.0.0 fails? (Note: With the 2.0.0 version , it seems to
be able to update the cache after it expires in 20 minutes. However,
the CVS version can do it instantly).

Thank you for your time,
-Rama

Theo Schlossnagle wrote:

>
> On Dec 11, 2003, at 1:48 PM, Rama K. McIntosh wrote:
>
>> Everything works perfect, except my firewall/gateway arp cache does
>> not get updated. My firewall is not running Wackamole but I thought
>> the Notify part of wackamole would tell my firewall to refresh it's
>> arp cache. If I manually delete the arp entry in my firewall
>> everything works after wackamole changes the owner of my virtual
>> IP. Actually, only the two machines running wackamole seem to have
>> good arp caches. Every other machine (solaris) on my subnet has a
>> stale arp entry.
>>
>> Here is my conf (with the IPs changed to protect the guilty):
>> Notify {
>> # Let's notify our router: ***** My firwall, 10.10.10.1's arp
>> cache becomes stale, this is my problem **********
>> eth0:10.10.10.1/32
>
>
> What kind of firewall is it?
>
> So. Try adding eth0:0.0.0.0/32 and eth0:255.255.255.0/32 and and
> eth0:networkaddress/32 and eth0:broadcastaddress/32 (where network
> address or broadcast address is the IP network address and IP
> broadcast address on that subnet, respectively).
>
> Different devices and OSs respond differently to unsolicited ARP
> responses. Some take "protective measures" to disable this to prevent
> "arp-spoofing" attacks. On FreeBSD/NetBSD/OpenBSD I believe there is
> a kernel option to allow/disallow this. Sometimes systems are more
> receptive to unsolicited arp resopnses to IP network, IP broadcast, IP
> subnet, or IP subnet broadcast addresses.
>
> Check the logs on your firewall to see if it sees the ARP request but
> is rejecting it.
>
> // Theo Schlossnagle
> // Principal Engineer -- http://www.omniti.com/~jesus/
> // Postal Engine -- http://www.postalengine.com/
> // Ecelerity: fastest MTA on earth
>
>
> _______________________________________________
> wackamole-users mailing list
> wackamole-users[at]lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/wackamole-users
>

jesus at omniti

Dec 13, 2003, 6:39 PM

Post #5 of 6 (1186 views)
Permalink

Wackamole problem with arp cache on firewall/gateway [In reply to]

On Dec 13, 2003, at 7:32 PM, Rama K. McIntosh wrote:
> So My questions:
>
> 1. Is the Wackamole CVS version stable enough for production use?

We use it in production. I would advise against using the embedded
perl interpreter feature without rigorously testing it in your
environment first.

> 2. Any idea why the CVS version can remotely update the arp cache on
> my firewall, but 2.0.0 fails? (Note: With the 2.0.0 version , it
> seems to be able to update the cache after it expires in 20 minutes.
> However, the CVS version can do it instantly).

72c70
< memcpy(cp, ze_mac, ETH_ALEN); cp+=ETH_ALEN;
---
> memcpy(cp, bc_mac, ETH_ALEN); cp+=ETH_ALEN;

Looks like the change. It now uses the bc_mac instead of ze mac for
spoofing (that is FF:FF:FF:FF:FF:FF instead of 00:00:00:00:00:00)

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on earth

rama at pitties

Dec 13, 2003, 7:44 PM

Post #6 of 6 (1179 views)
Permalink

Wackamole problem with arp cache on firewall/gateway [In reply to]

Thanks!

I tried to backport the bc_mac changes to 2.0.0 but still had problems
(could easily be my fault).

As the CVS versions is passing my tests I plan to just use it.

Thanks,
-Rama

Theo Schlossnagle wrote:

>
> On Dec 13, 2003, at 7:32 PM, Rama K. McIntosh wrote:
>
>> So My questions:
>>
>> 1. Is the Wackamole CVS version stable enough for production use?
>
>
> We use it in production. I would advise against using the embedded
> perl interpreter feature without rigorously testing it in your
> environment first.
>
>> 2. Any idea why the CVS version can remotely update the arp cache on
>> my firewall, but 2.0.0 fails? (Note: With the 2.0.0 version , it
>> seems to be able to update the cache after it expires in 20 minutes.
>> However, the CVS version can do it instantly).
>
>
> 72c70
> < memcpy(cp, ze_mac, ETH_ALEN); cp+=ETH_ALEN;
> ---
> > memcpy(cp, bc_mac, ETH_ALEN); cp+=ETH_ALEN;
>
> Looks like the change. It now uses the bc_mac instead of ze mac for
> spoofing (that is FF:FF:FF:FF:FF:FF instead of 00:00:00:00:00:00)
>
> // Theo Schlossnagle
> // Principal Engineer -- http://www.omniti.com/~jesus/
> // Postal Engine -- http://www.postalengine.com/
> // Ecelerity: fastest MTA on earth
>
>
> _______________________________________________
> wackamole-users mailing list
> wackamole-users[at]lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/wackamole-users
>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com