Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ClamAV: devel

New ClamAV Milter + Patch

 

 

ClamAV devel RSS feed   Index | Next | Previous | View Threaded


christopher at gms

Mar 2, 2009, 8:04 AM

Post #1 of 7 (1698 views)
Permalink
New ClamAV Milter + Patch

Hi,

I have been looking at the RC1 for ClamAV. With the rewrite of the Milter it has removed one important (for us) feature. That is
the ability to notify clients that there mail has been intercepted.

As an ISP here in Luxembourg we stand liable if we intercept and remove mail without informing the client. Our current system
informs the recipient about filtered Virus mail and delivers UCE tagged (or delivered to a Spam folder).

I have had a look at the source to the new Milter and have seen that it has simplified the program and made it much simpler.
This is great. I looked at the possibility for re-adding the notification possibility and have had initial success with only a
small patch. The style of the patch, I believe, stays within the spirit of the re-write.

Before doing any further development, I wanted to get an idea if this might make it into the upstream code or if something like
this would end up being an in-house patch.

Regards

Chris


christopher at gms

Mar 2, 2009, 9:59 AM

Post #2 of 7 (1584 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

Patch now on Bugzilla (sorry for the mess, not use to the submission process).

Bug: 1448

Chris

Chris Moules wrote:
> Hi,
>
> I have been looking at the RC1 for ClamAV. With the rewrite of the
> Milter it has removed one important (for us) feature. That is the
> ability to notify clients that there mail has been intercepted.
>
> As an ISP here in Luxembourg we stand liable if we intercept and remove
> mail without informing the client. Our current system informs the
> recipient about filtered Virus mail and delivers UCE tagged (or
> delivered to a Spam folder).
>
> I have had a look at the source to the new Milter and have seen that it
> has simplified the program and made it much simpler. This is great. I
> looked at the possibility for re-adding the notification possibility and
> have had initial success with only a small patch. The style of the
> patch, I believe, stays within the spirit of the re-write.
>
> Before doing any further development, I wanted to get an idea if this
> might make it into the upstream code or if something like this would end
> up being an in-house patch.
>
> Regards
>
> Chris
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


acabng at digitalfuture

Mar 2, 2009, 8:29 PM

Post #3 of 7 (1585 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

Chris Moules wrote:
> As an ISP here in Luxembourg we stand liable if we intercept and remove
> mail without informing the client. Our current system informs the
> recipient about filtered Virus mail and delivers UCE tagged (or
> delivered to a Spam folder).

Hi Chris,

I think the quarantine queue and the x-headers already offer quite a
good base for postprocessing the infected messages and, standing the
very peculiar needs of each mail server and deployment environment, I'm
not very keen to add postprocessing functionalities to the milter unless
these are extremely generic. Failing that, I'd rather leave the
postprocessing entirely to the admin.

> I have had a look at the source to the new Milter and have seen that it
> has simplified the program and made it much simpler. This is great. I
> looked at the possibility for re-adding the notification possibility and
> have had initial success with only a small patch. The style of the
> patch, I believe, stays within the spirit of the re-write.

> Before doing any further development, I wanted to get an idea if this
> might make it into the upstream code or if something like this would end
> up being an in-house patch.

The patch is indeed small but it's not really about notifications. In
fact it destroys the malicious message replacing its content and
subjects with some static lines (BTW, what if the mail carries let's say
a mua-specific exploit in its headers?).
Now I'm pretty sure this works good enough in your very case, but
frankly I don't see such a feature of much interest for general use.

OTOH, a generic notification system possibly together with blackholing
(as in your case), 5xx'ing or tagging, would benefit much more users...
Except the milter interface is not really designed with such things in
mind, which re-introduces a series of issues like determining if the
recipient is to be notified (e.g. local vs remote), assembling the
message, forking, invoking sendmail, etc...

But again, if you consider that the very same effect can be achieved
with about 3 lines of code in a sitewide procmail recipe or in a cronned
shell/perl "mailq -qQ" parser, you would probably agree that doing it in
the milter is not the way to go.

Cheers,
-aCaB

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


christopher at gms

Mar 3, 2009, 12:44 AM

Post #4 of 7 (1580 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

aCaB wrote:
> Chris Moules wrote:
>> As an ISP here in Luxembourg we stand liable if we intercept and remove
>> mail without informing the client. Our current system informs the
>> recipient about filtered Virus mail and delivers UCE tagged (or
>> delivered to a Spam folder).
>
> Hi Chris,
>
> I think the quarantine queue and the x-headers already offer quite a
> good base for postprocessing the infected messages and, standing the
> very peculiar needs of each mail server and deployment environment, I'm
> not very keen to add postprocessing functionalities to the milter unless
> these are extremely generic. Failing that, I'd rather leave the
> postprocessing entirely to the admin.
>

Thanks for the feedback. I am using Postfix as MTA. Postfix does not have a
'quarantine queue' as such, it places messages on hold in the main queue.

This is my first ever work with a Milter and I am more sys-admin than developer.

>> I have had a look at the source to the new Milter and have seen that it
>> has simplified the program and made it much simpler. This is great. I
>> looked at the possibility for re-adding the notification possibility and
>> have had initial success with only a small patch. The style of the
>> patch, I believe, stays within the spirit of the re-write.
>
>> Before doing any further development, I wanted to get an idea if this
>> might make it into the upstream code or if something like this would end
>> up being an in-house patch.
>
> The patch is indeed small but it's not really about notifications. In
> fact it destroys the malicious message replacing its content and
> subjects with some static lines (BTW, what if the mail carries let's say
> a mua-specific exploit in its headers?).

The patch was not meant to be complete, just add base functionality along lines that
could lead to a 'proper' option. I just spent a few hours looking at the code and a
Milter programming guide. I kind of thought that this was somewhere between 'Accept'
and 'Blackhole'. We nuke the message but leave most of the headers intact so someone
could look up the source. I would have liked to add the name of the Virus and maybe the
original subject somewhere but this required lots more changes. Also reading the 'notification'
text from a file and supplying replace tags for things like 'From Address', 'Subject'
and 'Virus name'.

> Now I'm pretty sure this works good enough in your very case, but
> frankly I don't see such a feature of much interest for general use.
>
Fair enough, that is why I asked before doing more than a couple of hours.
I will look at the alternate options that you proposed.

> OTOH, a generic notification system possibly together with blackholing
> (as in your case), 5xx'ing or tagging, would benefit much more users...
That was the idea. I initally thought of options like "NotifyReject" or "NotifyBlackhole".
Never having anything to do with a Milter before, I hacked the most un-hack like solution
that I could with a Clam-like feel to the implementation on a Friday afternoon.

> Except the milter interface is not really designed with such things in
> mind, which re-introduces a series of issues like determining if the
> recipient is to be notified (e.g. local vs remote), assembling the
> message, forking, invoking sendmail, etc...
I noted that, I was looking for a neat way of getting information to the
destination without invoking an external call.

>
> But again, if you consider that the very same effect can be achieved
> with about 3 lines of code in a sitewide procmail recipe or in a cronned
> shell/perl "mailq -qQ" parser, you would probably agree that doing it in
> the milter is not the way to go.
I am not a procmail fanboy and as said, Postfix does not have a 'quarantine queue'.
-> mailq: fatal: -qQ is not implemented

I will re-appraise the options.

Thanks

Chris

>
> Cheers,
> -aCaB
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


clamav-devel at jubileegroup

Mar 3, 2009, 3:32 AM

Post #5 of 7 (1580 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

Hi there,

On Tue, 3 Mar 2009 aCaB wrote:

> Chris Moules wrote:
>
> > I have been looking at the RC1 for ClamAV. With the rewrite of the
> > Milter it has removed one important (for us) feature. That is the
> > ability to notify clients that there mail has been intercepted.
>
> I think the quarantine queue and the x-headers already offer quite a
> good base for postprocessing the infected messages ... I'd rather
> leave the postprocessing entirely to the admin ... if you consider
> that the very same effect can be achieved with about 3 lines of code
> in a sitewide procmail recipe or ...

What you say is true. However it would be much clearer that you
understand that your new milter breaks a lot of existing and even in
some cases carefully crafted and well tested code on many mailservers
if you had apologised for it once more instead of simply snipping the
part of the OP's mail which mentions it.

You can expect administrators who find themselves having to script
their way around your changes to wonder if there might be a package
which both provides equivalent functionality and has a stable API.

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


acabng at digitalfuture

Mar 3, 2009, 7:28 AM

Post #6 of 7 (1579 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

Chris Moules wrote:
> Thanks for the feedback. I am using Postfix as MTA. Postfix does not have a
> 'quarantine queue' as such, it places messages on hold in the main queue.
>
> This is my first ever work with a Milter and I am more sys-admin than developer.

Good job then!

>> OTOH, a generic notification system possibly together with blackholing
>> (as in your case), 5xx'ing or tagging, would benefit much more users...
> That was the idea. I initally thought of options like "NotifyReject" or "NotifyBlackhole".
> Never having anything to do with a Milter before, I hacked the most un-hack like solution
> that I could with a Clam-like feel to the implementation on a Friday afternoon.

The NotifyReject and NotifyBlackhole options would be something worth
looking into, imho.
But the approach in your patch is unfortunately not suited to handle
them. I say unfortunately because it's so minimal that it doesn't requre
much code nor to break a lot of stuff.
The problem is that in such a way you can basically generate a notices
only at eom time and only when blackholing. Even earlier (than eom)
blackholing cannot result in a notification being sent as we've got no
mail to rewrite yet.

>> Except the milter interface is not really designed with such things in
>> mind, which re-introduces a series of issues like determining if the
>> recipient is to be notified (e.g. local vs remote), assembling the
>> message, forking, invoking sendmail, etc...
> I noted that, I was looking for a neat way of getting information to the
> destination without invoking an external call.

If you ask me, the milter interface is horrible. IIRC the old milter had
to inwoke sendmail twice to generate notices: once to determine if the
recipient is local, once to send the actual notice.
In many scenarios this requires the milter to be running as a
privileged/trusted process.

>> But again, if you consider that the very same effect can be achieved
>> with about 3 lines of code in a sitewide procmail recipe or in a cronned
>> shell/perl "mailq -qQ" parser, you would probably agree that doing it in
>> the milter is not the way to go.
> I am not a procmail fanboy and as said, Postfix does not have a 'quarantine queue'.
> -> mailq: fatal: -qQ is not implemented

As a debian user and casual postfix admin I assumed procmail was the
default dropper anyway, but I see this is not the case. And yes, it's
unfortunate that postfix cannot yet handle the quarantine properly.

-aCaB
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


acabng at digitalfuture

Mar 3, 2009, 7:40 AM

Post #7 of 7 (1576 views)
Permalink
Re: New ClamAV Milter + Patch [In reply to]

G.W. Haywood wrote:
> What you say is true. However it would be much clearer that you
> understand that your new milter breaks a lot of existing and even in
> some cases carefully crafted and well tested code on many mailservers
> if you had apologised for it once more instead of simply snipping the
> part of the OP's mail which mentions it.

It does not break anything. In fact I've written a new milter without
touching the old one. If it suits you, go on and use it. If it doesn't
just use the old milter. Up to you.

And... apologies for?

> You can expect administrators who find themselves having to script
> their way around your changes to wonder if there might be a package
> which both provides equivalent functionality and has a stable API.

Again I dont. If you don't want to change, just don't.
Also, if you fell like maintaining the old milter please send us
patches. It's still sitting under /contrib and will always be available
there. I'll be committing them for you.

As a side note the new milter was announced on December the 5th. In
exactly 4 months i've received 1 bug report, promply fixed, and, as of
yesterday, 0 feature requests.
See http://lurker.clamav.net/message/20081205.152347.a7d7c9ee.en.html

Cheers,
-aCaB
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

ClamAV devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.