Mailing List Archive: exim: users

Cluster

Index | Next | Previous | View Threaded

pch0317 at gmail

Aug 13, 2009, 10:56 AM

Post #1 of 13 (1582 views)
Permalink

Cluster

Hi
Is exim capable to work in high-availability or load-balancing cluster?
If yes, where I can read about this. Please, give me a hint.

Thanks

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

pch0317 at gmail

Aug 13, 2009, 10:56 AM

Post #2 of 13 (1523 views)
Permalink

Cluster [In reply to]

exim-users at lists

Aug 13, 2009, 12:49 PM

Post #3 of 13 (1519 views)
Permalink

Re: Cluster [In reply to]

pch0317 wrote:

> Is exim capable to work in high-availability or load-balancing cluster?

Yes.

> If yes, where I can read about this. Please, give me a hint.

1.) Install Exim on each node in cluster
2.) Done.

Any sort of clustering or load balancing would be done outside of Exim.
high-availability is "built in" to SMTP by the ability to set multiple
MX records, and the ability to retry.

--
Mike Cardwell - IT Consultant and LAMP developer
Cardwell IT Ltd. (UK Reg'd Company #06920226) http://cardwellit.com/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

iane at sussex

Aug 14, 2009, 9:29 AM

Post #4 of 13 (1505 views)
Permalink

Re: Cluster [In reply to]

--On 13 August 2009 20:49:07 +0100 Mike Cardwell
<exim-users [at] lists> wrote:

> pch0317 wrote:
>
>> Is exim capable to work in high-availability or load-balancing cluster?
>
> Yes.
>
>> If yes, where I can read about this. Please, give me a hint.
>
> 1.) Install Exim on each node in cluster
> 2.) Done.
>
> Any sort of clustering or load balancing would be done outside of Exim.
> high-availability is "built in" to SMTP by the ability to set multiple
> MX records, and the ability to retry.
>

Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
don't use MX records, and don't retry even if the smtp server address is a
round robin in the DNS. At least, none of the servers that I've tried.

We have a cluster of four OSX Server servers with Exim. With three, Apple's
IP failover was just about managable. With four we switched to using
Wackamole on Spread. That works quite nicely, but occasionally a failover
throws up faults with ARP caches, leaving some of the IP addresses
unavailable.

As a result, we've recently put the MSA servers behind a CoyotePoint
equaliser. We publish a single IP address for MSA, and the equaliser does
load balancing and high availability. It regularly checks to see that it
can get an SMTP greeting from each of the servers - if not, then that
server doesn't get any traffic till its fixed.

Of course, the equaliser becomes a single point of failure... And so it
goes...

The equaliser doesn't fail as often as the ARP cache corruptions occur.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

addw at phcomp

Aug 14, 2009, 2:59 PM

Post #5 of 13 (1503 views)
Permalink

Re: Cluster [In reply to]

On Fri, Aug 14, 2009 at 05:29:12PM +0100, Ian Eiloart wrote:

> > Any sort of clustering or load balancing would be done outside of Exim.
> > high-availability is "built in" to SMTP by the ability to set multiple
> > MX records, and the ability to retry.

Although you would need some 'high availability' mail store that exim could
deliver to ...

As pointed out below: talking about exim is not quite the same as talking about
MS exchange; to get the same thing you would need to add mail storage (or use a MDA)
and something like imap/pop so that the MUAs could get mail. If you really are into
high availabilty then you perhaps want to choose cyrus over squirrel mail,
cyrus will handle the MDA & IMAP serving.

Cyrus is much harder to get going than many other IMAP servers.

> Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
> don't use MX records, and don't retry even if the smtp server address is a
> round robin in the DNS. At least, none of the servers that I've tried.

No, you prob need an aggegator or IP failover system (heartbeat, etc) to handle that.

> We have a cluster of four OSX Server servers with Exim. With three, Apple's
> IP failover was just about managable. With four we switched to using
> Wackamole on Spread. That works quite nicely, but occasionally a failover
> throws up faults with ARP caches, leaving some of the IP addresses
> unavailable.
>
> As a result, we've recently put the MSA servers behind a CoyotePoint
> equaliser. We publish a single IP address for MSA, and the equaliser does
> load balancing and high availability. It regularly checks to see that it
> can get an SMTP greeting from each of the servers - if not, then that
> server doesn't get any traffic till its fixed.
>
> Of course, the equaliser becomes a single point of failure... And so it
> goes...
>
> The equaliser doesn't fail as often as the ARP cache corruptions occur.

--
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256 http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
Past chairman of UKUUG: http://www.ukuug.org/
#include <std_disclaimer.h>

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

iane at sussex

Aug 17, 2009, 2:21 AM

Post #6 of 13 (1468 views)
Permalink

Re: Cluster [In reply to]

--On 14 August 2009 22:59:51 +0100 Alain Williams <addw [at] phcomp> wrote:

>
>> > Any sort of clustering or load balancing would be done outside of Exim.
>> > high-availability is "built in" to SMTP by the ability to set multiple
>> > MX records, and the ability to retry.
>
> Although you would need some 'high availability' mail store that exim
> could deliver to ...

That's not quite so important. Some MUAs may lose emails when they can't
deliver (especially poorly written web applications, for example). Or, they
might put the mail into a local queue that doesn't get tickled for a long
time - perhaps you're sending last minute instructions to your staff before
you go on leave for a fortnight, for example!

MTAs are vulnerable to DoS attacks, and problems with virus scanners can
bring the host down. Your mailstore (if on a separate host) should be more
stable because it doesn't face those threats. And, the consequence of lack
of availability is less severe - the data should still be there when you
next connect.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

iane at sussex

Aug 17, 2009, 2:23 AM

Post #7 of 13 (1467 views)
Permalink

Re: Cluster [In reply to]

--On 14 August 2009 17:29:12 +0100 Ian Eiloart <iane [at] sussex> wrote:

>
> Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
> don't use MX records, and don't retry even if the smtp server address is
> a round robin in the DNS. At least, none of the servers that I've tried.
>

Self-correction: I mean that none of the MUAs that I've tried will try
alternate SMTP servers, even if the DNS gives them several IP addresses in
a round robin.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

addw at phcomp

Aug 17, 2009, 2:36 AM

Post #8 of 13 (1469 views)
Permalink

Re: Cluster [In reply to]

On Mon, Aug 17, 2009 at 10:21:46AM +0100, Ian Eiloart wrote:
>
>
> --On 14 August 2009 22:59:51 +0100 Alain Williams <addw [at] phcomp> wrote:
>
> >
> >>> Any sort of clustering or load balancing would be done outside of Exim.
> >>> high-availability is "built in" to SMTP by the ability to set multiple
> >>> MX records, and the ability to retry.
> >
> >Although you would need some 'high availability' mail store that exim
> >could deliver to ...
>
> That's not quite so important. Some MUAs may lose emails when they can't
> deliver (especially poorly written web applications, for example). Or, they
> might put the mail into a local queue that doesn't get tickled for a long
> time - perhaps you're sending last minute instructions to your staff before
> you go on leave for a fortnight, for example!

I always run exim on the local host, even if the ''main'' exim MTA is just in
the next slot in the rack. That provides the reliability against the main MTA
being down for a bit.

--
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256 http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
Past chairman of UKUUG: http://www.ukuug.org/
#include <std_disclaimer.h>

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

addw at phcomp

Aug 17, 2009, 2:37 AM

Post #9 of 13 (1461 views)
Permalink

Re: Cluster [In reply to]

On Mon, Aug 17, 2009 at 10:23:47AM +0100, Ian Eiloart wrote:
>
>
> --On 14 August 2009 17:29:12 +0100 Ian Eiloart <iane [at] sussex> wrote:
>
> >
> > Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
> > don't use MX records, and don't retry even if the smtp server address is
> > a round robin in the DNS. At least, none of the servers that I've tried.
> >
>
> Self-correction: I mean that none of the MUAs that I've tried will try
> alternate SMTP servers, even if the DNS gives them several IP addresses in
> a round robin.

As I said: 127.0.0.1 is usually available, then rely on the local exim to get it right.

--
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256 http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
Past chairman of UKUUG: http://www.ukuug.org/
#include <std_disclaimer.h>

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim-users at lists

Aug 17, 2009, 2:51 AM

Post #10 of 13 (1460 views)
Permalink

Re: Cluster [In reply to]

Ian Eiloart wrote:

>> Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
>> don't use MX records, and don't retry even if the smtp server address is
>> a round robin in the DNS. At least, none of the servers that I've tried.
>
> Self-correction: I mean that none of the MUAs that I've tried will try
> alternate SMTP servers, even if the DNS gives them several IP addresses in
> a round robin.

My original point was that this is entirely outside the scope of Exim.
Exim listens on an IP for incoming connections. How you get that IP to
continue pointing at an Exim daemon is none of Exims concern.

--
Mike Cardwell - IT Consultant and LAMP developer
Cardwell IT Ltd. (UK Reg'd Company #06920226) http://cardwellit.com/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

iane at sussex

Aug 17, 2009, 3:11 AM

Post #11 of 13 (1463 views)
Permalink

Re: Cluster [In reply to]

--On 17 August 2009 10:37:58 +0100 Alain Williams <addw [at] phcomp> wrote:

> On Mon, Aug 17, 2009 at 10:23:47AM +0100, Ian Eiloart wrote:
>>
>>
>> --On 14 August 2009 17:29:12 +0100 Ian Eiloart <iane [at] sussex> wrote:
>>
>> >
>> > Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
>> > don't use MX records, and don't retry even if the smtp server address
>> > is a round robin in the DNS. At least, none of the servers that I've
>> > tried.
>> >
>>
>> Self-correction: I mean that none of the MUAs that I've tried will try
>> alternate SMTP servers, even if the DNS gives them several IP addresses
>> in a round robin.
>
> As I said: 127.0.0.1 is usually available, then rely on the local exim to
> get it right.
>

I'm kind of assuming that we're discussing the 99.9% of users that can't
possibly benefit from this advice because they don't have the technical
skills.

Apart from which, putting mail onto a local queue that doesn't get run for
a week is exactly the problem that I just described. All it takes is for
the local exim to get shut down with the host just before you go on
holiday. It's almost exactly equivalent to the MUA doing the queuing. We
switch that off by default on our local clients so that users are made
aware that their email hasn't been delivered.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

iane at sussex

Aug 17, 2009, 3:11 AM

Post #12 of 13 (1464 views)
Permalink

Re: Cluster [In reply to]

--On 17 August 2009 10:51:29 +0100 Mike Cardwell
<exim-users [at] lists> wrote:

> Ian Eiloart wrote:
>
>>> Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
>>> don't use MX records, and don't retry even if the smtp server address is
>>> a round robin in the DNS. At least, none of the servers that I've
>>> tried.
>>
>> Self-correction: I mean that none of the MUAs that I've tried will try
>> alternate SMTP servers, even if the DNS gives them several IP addresses
>> in a round robin.
>
> My original point was that this is entirely outside the scope of Exim.
> Exim listens on an IP for incoming connections. How you get that IP to
> continue pointing at an Exim daemon is none of Exims concern.
>

Right, but it is of concern to Exim administrators. This list is for them.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

addw at phcomp

Aug 17, 2009, 3:26 AM

Post #13 of 13 (1473 views)
Permalink

Re: Cluster [In reply to]

On Mon, Aug 17, 2009 at 11:11:04AM +0100, Ian Eiloart wrote:
>
>
> --On 17 August 2009 10:37:58 +0100 Alain Williams <addw [at] phcomp> wrote:
>
> >On Mon, Aug 17, 2009 at 10:23:47AM +0100, Ian Eiloart wrote:
> >>
> >>
> >>--On 14 August 2009 17:29:12 +0100 Ian Eiloart <iane [at] sussex> wrote:
> >>
> >>>
> >>> Which is fine for MX servers. Not so good for MSA. MUAs (user clients)
> >>> don't use MX records, and don't retry even if the smtp server address
> >>> is a round robin in the DNS. At least, none of the servers that I've
> >>> tried.
> >>>
> >>
> >>Self-correction: I mean that none of the MUAs that I've tried will try
> >>alternate SMTP servers, even if the DNS gives them several IP addresses
> >>in a round robin.
> >
> >As I said: 127.0.0.1 is usually available, then rely on the local exim to
> >get it right.
> >
>
> I'm kind of assuming that we're discussing the 99.9% of users that can't
> possibly benefit from this advice because they don't have the technical
> skills.

Who sets the machines up ?
You were talking about ''badly written web apps'' -- that implies a web server,
if someone has installed that on a machine then they can install a simple exim
that knows about 2 or 3 smart hosts that it punts everything to. Thus locally
delivered mail will get to the next hop in a few seconds ... assuming that
the local machine is connected to your local network.

> Apart from which, putting mail onto a local queue that doesn't get run for
> a week is exactly the problem that I just described. All it takes is for
> the local exim to get shut down with the host just before you go on
> holiday. It's almost exactly equivalent to the MUA doing the queuing. We
> switch that off by default on our local clients so that users are made
> aware that their email hasn't been delivered.

Not at all. The local MUA (as you said) will only try one smarthost, a local
exim will try however many you specify and succeed in less than a minute.

--
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256 http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
Past chairman of UKUUG: http://www.ukuug.org/
#include <std_disclaimer.h>

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads