Mailing List Archive: SpamAssassin: users

My bash script to upload PDFinfo daily, safely

Index | Next | Previous | View Threaded

ichudov at Algebra

Jul 19, 2007, 10:02 PM

Post #1 of 7 (562 views)
Permalink

My bash script to upload PDFinfo daily, safely

#!/bin/bash

PM=`perl -MConfig -e 'print "$Config{installsitelib}"'`/Mail/SpamAssassin/Plugin/PDFInfo.pm
CF=/etc/mail/spamassassin/PDFInfo.cf

cp $PM $PM.bak || exit 1 # Probably I am not root...
cp $CF $CF.bak || exit 1 # same

echo Downloading, veryfying perl module and size of config file...

if wget -q -O $PM http://www.rulesemporium.com/plugins/PDFInfo.pm \
&& wget -q -O $CF http://www.rulesemporium.com/plugins/pdfinfo.cf \
&& perl -MMail::SpamAssassin::Plugin::PDFInfo -e print "Perl Module PDFInfo OK\n" \
&& test -s $CF ; then
echo Successfully downloaded $PM and $CF:
chmod 644 $PM $CF
ls -l $PM $CF
rm $CF.bak $PM.bak
echo Restarting SpamAssassin:
service spamassassin restart
exit 0
else
echo FAILED to download $PM and $CF
mv $CF.bak $CF
mv $PM.bak $PM
exit 1
fi

bob at proulx

Jul 20, 2007, 12:41 AM

Post #2 of 7 (544 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

Igor Chudov wrote:
> #!/bin/bash

Since there are no bash specific features this could be a standard
/bin/sh just as easily and then does not depend upon bash.

> PM=`perl -MConfig -e 'print "$Config{installsitelib}"'`/Mail/SpamAssassin/Plugin/PDFInfo.pm
> CF=/etc/mail/spamassassin/PDFInfo.cf
>
> cp $PM $PM.bak || exit 1 # Probably I am not root...
> cp $CF $CF.bak || exit 1 # same

Instead of backing up and removing these later I think it is better to
download the files into a temporary location and if valid then move
them into position.

> echo Downloading, veryfying perl module and size of config file...
>
> if wget -q -O $PM http://www.rulesemporium.com/plugins/PDFInfo.pm \
> && wget -q -O $CF http://www.rulesemporium.com/plugins/pdfinfo.cf \
> && perl -MMail::SpamAssassin::Plugin::PDFInfo -e print "Perl Module PDFInfo OK\n" \
> && test -s $CF ; then
> echo Successfully downloaded $PM and $CF:
> chmod 644 $PM $CF

There is an order of events problem as this stands. If the first wget
succeeds but the second wget or later commands fail then the chmod
never happens. If the chmod is needed (I don't think it should be)
then the first file is left in a bad state. If it is not needed then
this is simply redundant here.

> ls -l $PM $CF

Of course that is simply left over from debugging.

> rm $CF.bak $PM.bak

This should be done in a shell EXIT trap so that it always happens.

> echo Restarting SpamAssassin:
> service spamassassin restart

Hmm... spamassassin? Or spamc? I think your system is different
from mine. Best to double check.

> exit 0
> else
> echo FAILED to download $PM and $CF
> mv $CF.bak $CF
> mv $PM.bak $PM
> exit 1
> fi

This is untested and just typed in here off the top of my head but let
me suggest some improvements. Among other things the trap handling
here means that temporary files will not be left behind even if the
script is interrupted and errors are reported for each type of
possible error.

#!/bin/sh

PM=`perl -MConfig -e 'print "$Config{installsitelib}"'`/Mail/SpamAssassin/Plugin/PDFInfo.pm
CF=/etc/mail/spamassassin/PDFInfo.cf

trap 'rm -f $PMTMP $CFTMP' EXIT
PMTMP=$(mktemp -t pdfinfo.pm.XXXXXXXX) || exit 1
CFTMP=$(mktemp -t pdfinfo.cf.XXXXXXXX) || exit 1
chmod a+r $PMTMP $CFTMP

echo Downloading, veryfying perl module and size of config file...

if ! wget -q -O $PMTMP http://www.rulesemporium.com/plugins/PDFInfo.pm; then
echo FAILED to download http://www.rulesemporium.com/plugins/PDFInfo.pm
exit 1
fi
if ! wget -q -O $CFTMP http://www.rulesemporium.com/plugins/pdfinfo.cf; then
echo FAILED to download http://www.rulesemporium.com/plugins/pdfinfo.cf
exit 1
fi
if ! test -s $PMTMP ; then
echo ERROR the downloaded PDFInfo.pm file is zero sized
exit 1
fi
if ! test -s $CFTMP ; then
echo ERROR the downloaded pdfinfo.cf file is zero sized
exit 1
fi
if ! perl -cw $PMTMP; then
echo FAILED syntax check of new PDFInfo.pm module
exit 1
fi

echo Successfully downloaded $PM and $CF, installing:

mv $CFTMP $CF || exit 1 # Probably I am not root...
mv $PMTMP $PM || exit 1 # same

echo Restarting SpamAssassin:
service spamassassin restart

exit 0

sa-list at alexb

Jul 20, 2007, 1:12 AM

Post #3 of 7 (541 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

Guys

These are NOT AV signatures to be updated every day/hour, whatever

Hammering the site won't make it any better and not help to keep it going.

Seems some ppl still haven't understood that hammer-leeching spoils it
for everyone and themselves.

andy at xecu

Jul 20, 2007, 5:01 AM

Post #4 of 7 (539 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

On Fri, 20 Jul 2007, Yet Another Ninja wrote:

> Guys
>
> These are NOT AV signatures to be updated every day/hour, whatever
>
> Hammering the site won't make it any better and not help to keep it going.
>
> Seems some ppl still haven't understood that hammer-leeching spoils it for
> everyone and themselves.

People are still getting PDF spams?

I'm finding with a current version of SA and the sane security clamav
signatures, that none have gotten through for a few days now.

Do we really need a PDFinfo at this point?

Andy

---
Andy Dills
Xecunet, Inc.
www.xecu.net
301-682-9972
---

linux at matthias-keller

Jul 20, 2007, 6:23 AM

Post #5 of 7 (542 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

Andy Dills wrote:
> On Fri, 20 Jul 2007, Yet Another Ninja wrote:
>
>
>> Guys
>>
>> These are NOT AV signatures to be updated every day/hour, whatever
>>
>> Hammering the site won't make it any better and not help to keep it going.
>>
>> Seems some ppl still haven't understood that hammer-leeching spoils it for
>> everyone and themselves.
>>
>
> People are still getting PDF spams?
>
> I'm finding with a current version of SA and the sane security clamav
> signatures, that none have gotten through for a few days now.
>
> Do we really need a PDFinfo at this point?
>
Yes. Definitely.

I'm only using the signatures with the spamassassin plugin because I
found them to be too invasive. I dont allow anyone to block a mail at
SMTP level just because it contains ONE sentence. The sanesecurity
signatures also contain a LOT of single sentence rules which are too
risky for me... For example just becuase a mail contains '(ii) Email
ticket number' I dont want to block it.. but it will, according the the
current scam.ndb...

Thats why I only use these rules for scoring and I definitely also want
the pdfinfo scores in addition...
And.. It took them quite a while to score on the new encrypted pdfs
while they were already detected by the pdfinfo plugin

Matt

andy at xecu

Jul 20, 2007, 12:52 PM

Post #6 of 7 (540 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

On Fri, 20 Jul 2007, Matthias Keller wrote:

> I'm only using the signatures with the spamassassin plugin because I found
> them to be too invasive. I dont allow anyone to block a mail at SMTP level
> just because it contains ONE sentence. The sanesecurity signatures also
> contain a LOT of single sentence rules which are too risky for me... For
> example just becuase a mail contains '(ii) Email ticket number' I dont want to
> block it.. but it will, according the the current scam.ndb...

Hmm...with amavisd, the return from clamav can be treated like a SA rule.
You can score certain returns (regexable) with high scores, and
other returns with lower scores. Nobody said anything about blocking mail
at an SMTP level because it contains one sentence.

Andy

---
Andy Dills
Xecunet, Inc.
www.xecu.net
301-682-9972
---

oliver at fhsinternet

Jul 22, 2007, 8:46 AM

Post #7 of 7 (531 views)
Permalink

Re: My bash script to upload PDFinfo daily, safely [In reply to]

I have found SaneSecurity definitions to be VERY good - they hit about 60% of
my SPAM which is incredible given that they only match exact results (they
are not fuzzy). However this high percentage may be beacuse I am based in
the UK as is the author of the sanesecurity definitions. Also they tend to
hit already high scoring spam so they arn't a miracle spam fighting measure
though they are good.

My biggest concern was over possible false positives given that there is
only one person working on these definitions unlike the official ClamAV
signatures...

However I have yet to have any problems with them in the month that I have
been using them.

There are also two other sets of ClamAV signatures which I am now testing
(though these are not as good IMHO):

http://www.malware.com.br/ (various formats including ClamAV)
http://www.msrbl.com/site/ (ClamAV as well as RBLs)

As a solution to my own concerns over false positives I have changed from
virus scanning at SMTP time and have moved to using the ClamAV SpamAssassin
plugin:

http://wiki.apache.org/spamassassin/ClamAVPlugin

Rather than using the standard clamav.cf I have written my own which gives
different scores depending on what ClamAV signature found somthing:

loadplugin ClamAV clamav.pm
full CLAMAV eval:check_clamav()
describe CLAMAV Clam AntiVirus detected something...
score CLAMAV 0.001

# Look for specific types of ClamAV detections
header __CLAMAV_PHISH X-Spam-Virus =~ /Yes.{1,20}Phishing/i
header __CLAMAV_SANE X-Spam-Virus =~ /Yes.{1,20}Sanesecurity/i
header __CLAMAV_MBL X-Spam-Virus =~ /Yes.{1,20}MBL/
header __CLAMAV_MSRBL X-Spam-Virus =~ /Yes.{1,20}MSRBL/

# Give the above rules a very late priority so that they can see the output
# of previous rules - otherwise they don't work!
priority __CLAMAV_PHISH 9999
priority __CLAMAV_SANE 9999
priority __CLAMAV_MBL 9999
priority __CLAMAV_MSRBL 9999

# Work out what ClamAV detected and score accordingly
meta CLAMAV_VIRUS (CLAMAV && !__CLAMAV_PHISH && !__CLAMAV_SANE &&
!__CLAMAV_MBL && !__CLAMAV_MSRBL)
describe CLAMAV_VIRUS Virus found by ClamAV default signatures
score CLAMAV_VIRUS 20.0

meta CLAMAV_PHISH (CLAMAV && __CLAMAV_PHISH && !__CLAMAV_SANE)
describe CLAMAV_PHISH Phishing email found by ClamAV default signatures
score CLAMAV_PHISH 10.0

meta CLAMAV_SANE (CLAMAV && __CLAMAV_SANE)
describe CLAMAV_SANE SPAM found by ClamAV SaneSecurity signatures
score CLAMAV_SANE 7.5

meta CLAMAV_MBL (CLAMAV && __CLAMAV_MBL)
describe CLAMAV_MBL Malware found by ClamAV MBL signatures
score CLAMAV_MBL 7.5

meta CLAMAV_MSRBL (CLAMAV && __CLAMAV_MSRBL)
describe CLAMAV_MSRBL SPAM found by ClamAV MRSBL signatures
score CLAMAV_MSRBL 2.0

Hope this is of some help to someone...
--
View this message in context: http://www.nabble.com/My-bash-script-to-upload-PDFinfo-daily%2C-safely-tf4115144.html#a11732078
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com