Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: users

updates mirror is down

 

 

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded


mike at cchtml

Aug 24, 2011, 7:02 AM

Post #1 of 25 (3313 views)
Permalink
updates mirror is down

Hello,

For the past few days, my SpamAssassin instance has been trying to get
its updates from one mirror and the mirror is down.

Can someone contact the admin of the mirror or remove it from the mirror
list?

Mirror: daryl.dostech.ca

Thanks,
Michael


me at junc

Aug 24, 2011, 7:16 AM

Post #2 of 25 (3273 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 24 Aug 2011 09:02:58 -0500, Michael Cronenworth wrote:

> Mirror: daryl.dostech.ca

remove self the mirrored.by file

i remember this helped before, maybe also this time ?, could be a bug
in update mirrored.by that can be staled with mirrors that are not
mirror anymore


michael.scheidell at secnap

Aug 24, 2011, 7:24 AM

Post #3 of 25 (3271 views)
Permalink
Re: updates mirror is down [In reply to]

On 8/24/11 10:02 AM, Michael Cronenworth wrote:
> Hello,
>
> For the past few days, my SpamAssassin instance has been trying to get
> its updates from one mirror and the mirror is down.
>
> Can someone contact the admin of the mirror or remove it from the
> mirror list?
>
> Mirror: daryl.dostech.ca
>
pccm mirror is back up again.

> Thanks,
> Michael


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator


______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________


mike at cchtml

Aug 24, 2011, 7:26 AM

Post #4 of 25 (3270 views)
Permalink
Re: updates mirror is down [In reply to]

Michael Scheidell wrote:
> pccm mirror is back up again.

Huh?

$ wget daryl.dostech.ca
--2011-08-24 09:25:17-- http://daryl.dostech.ca/
Resolving daryl.dostech.ca... 71.164.246.108
Connecting to daryl.dostech.ca|71.164.246.108|:80...
(hangs forever)


mike at cchtml

Aug 24, 2011, 7:27 AM

Post #5 of 25 (3270 views)
Permalink
Re: updates mirror is down [In reply to]

Benny Pedersen wrote:
> remove self the mirrored.by file

Where is this file? I cannot find it.

I'm using SpamAssassin 3.3.2.


michael.scheidell at secnap

Aug 24, 2011, 7:29 AM

Post #6 of 25 (3267 views)
Permalink
Re: updates mirror is down [In reply to]

On 8/24/11 10:26 AM, Michael Cronenworth wrote:
> Michael Scheidell wrote:
>> pccm mirror is back up again.
>
> Huh?
>
> $ wget daryl.dostech.ca
> --2011-08-24 09:25:17-- http://daryl.dostech.ca/
> Resolving daryl.dostech.ca... 71.164.246.108
> Connecting to daryl.dostech.ca|71.164.246.108|:80...
> (hangs forever)
so, sa-update will call dostech, and when it fails, it will call pccm
mirror.

see MIRRORED.BY file.

http://daryl.dostech.ca/sa-update/asf/ weight=5
http://www.sa-update.pccc.com/ weight=5


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator


______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________


michael.scheidell at secnap

Aug 24, 2011, 7:31 AM

Post #7 of 25 (3273 views)
Permalink
Re: updates mirror is down [In reply to]

On 8/24/11 10:27 AM, Michael Cronenworth wrote:
> Benny Pedersen wrote:
>> remove self the mirrored.by file
>
> Where is this file? I cannot find it.
>
> I'm using SpamAssassin 3.3.2.
if you are trying to update this by hand, you are on your own.

just use sa-update (-D to watch)

it will delete MIRRORED.BY for you, pull a new one, and use it.


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator


______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________


mike at cchtml

Aug 24, 2011, 7:37 AM

Post #8 of 25 (3268 views)
Permalink
Re: updates mirror is down [In reply to]

Michael Scheidell wrote:
> if you are trying to update this by hand, you are on your own.
>
> just use sa-update (-D to watch)
>
> it will delete MIRRORED.BY for you, pull a new one, and use it.

I *am* using sa-update. sa-update is continuously failing.

Fedora 14 box. I'm using the default cron job, which calls sa-update.

I am not trying to update my definitions by hand.

Thanks,
Michael


michael.scheidell at secnap

Aug 24, 2011, 7:40 AM

Post #9 of 25 (3271 views)
Permalink
Re: updates mirror is down [In reply to]

On 8/24/11 10:37 AM, Michael Cronenworth wrote:
> Michael Scheidell wrote:
>> if you are trying to update this by hand, you are on your own.
>>
>> just use sa-update (-D to watch)
>>
>> it will delete MIRRORED.BY for you, pull a new one, and use it.
>
> I *am* using sa-update. sa-update is continuously failing.
>
sa-update doesn't pull a new one automatically?

use 'locate' to find it and delete it, or edit it and just keep pccm line.

Aug 24 09:53:06.636 [61273] dbg: generic: lint check of site pre files
succeeded, continuing with channel updates
Aug 24 09:53:06.649 [61273] dbg: channel: no MIRRORED.BY file available
Aug 24 09:53:09.065 [61273] dbg: http: GET request,
http://spamassassin.apache.org/updates/MIRRORED.BY
Aug 24 09:53:09.356 [61273] dbg: channel: MIRRORED.BY file retrieved
Aug 24 09:53:09.356 [61273] dbg: channel: reading MIRRORED.BY file
Aug 24 09:53:09.356 [61273] dbg: channel: found mirror
http://daryl.dostech.ca/sa-update/asf/ weight=5
Aug 24 09:53:09.356 [61273] dbg: channel: found mirror
http://www.sa-update.pccc.com/ weight=5
Aug 24 09:53:09.356 [61273] dbg: channel: selected mirror
http://daryl.dostech.ca/sa-update/asf

> Fedora 14 box. I'm using the default cron job, which calls sa-update.
>
on freebsd, its in
locate MIRRORED.BY

/var/db/spamassassin/3.003002/updates_spamassassin_org/MIRRORED.BY

> I am not trying to update my definitions by hand.
>
> Thanks,
> Michael


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator


______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________


mike at cchtml

Aug 24, 2011, 7:46 AM

Post #10 of 25 (3269 views)
Permalink
Re: updates mirror is down [In reply to]

Michael Scheidell wrote:
> sa-update doesn't pull a new one automatically?

Nope!

$ cat /var/lib/spamassassin/3.003002/updates_spamassassin_org/MIRRORED.BY
# test mirror: zone, cached via Coral
#http://buildbot.spamassassin.org.nyud.net:8090/updatestage/
http://daryl.dostech.ca/sa-update/asf/ weight=5
http://www.sa-update.pccc.com/ weight=5

What's disappointing is that when sa-update fails this is written to the
log:
http: GET http://daryl.dostech.ca/sa-update/asf/1160585.tar.gz request
failed: 500 read timeout: 500 read timeout
24-Aug-2011 05:18:11: SpamAssassin: Update processed successfully

Thanks for the tips. Looks like I've worked around it for now.

Michael


michael.scheidell at secnap

Aug 24, 2011, 7:48 AM

Post #11 of 25 (3268 views)
Permalink
Re: updates mirror is down [In reply to]

On 8/24/11 10:46 AM, Michael Cronenworth wrote:
> http://www.sa-update.pccc.com/ weight=5
question is... why didn't it pull from pccc.com?



--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator

______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________


guenther at rudersport

Aug 24, 2011, 8:41 AM

Post #12 of 25 (3257 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 2011-08-24 at 09:46 -0500, Michael Cronenworth wrote:
> What's disappointing is that when sa-update fails this is written to the
> log:
> http: GET http://daryl.dostech.ca/sa-update/asf/1160585.tar.gz request
> failed: 500 read timeout: 500 read timeout
> 24-Aug-2011 05:18:11: SpamAssassin: Update processed successfully

That last line is not SA.

And maybe the wrapper script that generates the success message actually
has reason to. Like a successful update from the second mirror, or any
other non-stock channel?


--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


axb.lists at gmail

Aug 24, 2011, 8:46 AM

Post #13 of 25 (3258 views)
Permalink
Re: updates mirror is down [In reply to]

On 2011-08-24 16:02, Michael Cronenworth wrote:
> Hello,
>
> For the past few days, my SpamAssassin instance has been trying to get
> its updates from one mirror and the mirror is down.
>
> Can someone contact the admin of the mirror or remove it from the mirror
> list?
>
> Mirror: daryl.dostech.ca


Would any of the biggish hosters reading this list be willing to donate
a small VPS to add a couple of update mirrors and offload Daryl's boxes?


mike at cchtml

Aug 24, 2011, 8:46 AM

Post #14 of 25 (3257 views)
Permalink
Re: updates mirror is down [In reply to]

Karsten Brńckelmann wrote:
> Like a successful update from the second mirror, or any
> other non-stock channel?

I'm thinking this is the case (second mirror) but the log file does not
clearly state that the bad mirror is ignored and it continues with a
good mirror for an update file.

To my eyes: sa-update failed. I get a message from cron showing me the
mirror had a read timeout. Nothing about it rolling on to a second mirror.

Is the cron job an SA problem or is it something the Fedora maintainer made?


guenther at rudersport

Aug 24, 2011, 9:14 AM

Post #15 of 25 (3258 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 2011-08-24 at 10:46 -0500, Michael Cronenworth wrote:
> Karsten Br├Ąckelmann wrote:
> > Like a successful update from the second mirror, or any
> > other non-stock channel?
>
> I'm thinking this is the case (second mirror) but the log file does not
> clearly state that the bad mirror is ignored and it continues with a
> good mirror for an update file.
>
> To my eyes: sa-update failed. I get a message from cron showing me the
> mirror had a read timeout. Nothing about it rolling on to a second mirror.
>
> Is the cron job an SA problem or is it something the Fedora maintainer made?

Most certainly distro specific.


--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


walterhurry at lavabit

Aug 24, 2011, 9:26 AM

Post #16 of 25 (3259 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 24 Aug 2011 18:14:49 +0200, Karsten Br├Ąckelmann wrote:

> On Wed, 2011-08-24 at 10:46 -0500, Michael Cronenworth wrote:
>> Karsten Br├Ąckelmann wrote:
>> > Like a successful update from the second mirror, or any other
>> > non-stock channel?
>>
>> I'm thinking this is the case (second mirror) but the log file does not
>> clearly state that the bad mirror is ignored and it continues with a
>> good mirror for an update file.
>>
>> To my eyes: sa-update failed. I get a message from cron showing me the
>> mirror had a read timeout. Nothing about it rolling on to a second
>> mirror.
>>
>> Is the cron job an SA problem or is it something the Fedora maintainer
>> made?
>
> Most certainly distro specific.

I don't think so. Fedora does not create a cron job for sa-update.
Perhaps OP would be well advised to inspect the script run by cron. My
guess is that it is something made locally.


mike at cchtml

Aug 24, 2011, 9:35 AM

Post #17 of 25 (3264 views)
Permalink
Re: updates mirror is down [In reply to]

Walter Hurry wrote:
> I don't think so. Fedora does not create a cron job for sa-update.
> Perhaps OP would be well advised to inspect the script run by cron. My
> guess is that it is something made locally.

The cron job:
10 4 * * * root /usr/share/spamassassin/sa-update.cron 2>&1 | tee -a
/var/log/sa-update.log

The cron job script:
/usr/share/spamassassin/sa-update.cron

I don't see anything in the upstream source tarball that relates to cron.

This is what I get in an email when the cron job fails:
http: GET http://daryl.dostech.ca/sa-update/asf/1160585.tar.gz request
failed: 500 read timeout: 500 read timeout

That's it. One line.

I'm fixing to create a distro bug report against the cron job to see a
more verbose output. If you feel it's something stupid I'm doing let me
know. However, I have not adjusted SpamAssassin in anyway. I'm not
calling it through a custom init script. I'm using all of the defaults
and I have not edited any configuration file for it. I did not create
the cron job either.


martin at gregorie

Aug 24, 2011, 9:53 AM

Post #18 of 25 (3253 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 2011-08-24 at 16:26 +0000, Walter Hurry wrote:

> I don't think so. Fedora does not create a cron job for sa-update.
> Perhaps OP would be well advised to inspect the script run by cron. My
> guess is that it is something made locally.
>
Fedora 13 and 15 do! I have copies of both running and both contain a
copy of the cron.d sa-update cron job, so its safe to assume that Fedora
14 did too.

However, earlier Fedora versions didn't include it. I know this because
I wrote my own sa_update weekly cron job in 2009 and its still lurking
in my CVS repository.


Martin


me at junc

Aug 24, 2011, 9:56 AM

Post #19 of 25 (3264 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 24 Aug 2011 17:46:47 +0200, Axb wrote:

> Would any of the biggish hosters reading this list be willing to
> donate a small VPS to add a couple of update mirrors and offload
> Daryl's boxes?

imho Daryl is not down becurse of bandwidth problems, but more mirrors
is still nice


walterhurry at lavabit

Aug 24, 2011, 10:12 AM

Post #20 of 25 (3261 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 24 Aug 2011 17:53:25 +0100, Martin Gregorie wrote:

> On Wed, 2011-08-24 at 16:26 +0000, Walter Hurry wrote:
>
>> I don't think so. Fedora does not create a cron job for sa-update.
>> Perhaps OP would be well advised to inspect the script run by cron. My
>> guess is that it is something made locally.
>>
> Fedora 13 and 15 do! I have copies of both running and both contain a
> copy of the cron.d sa-update cron job, so its safe to assume that Fedora
> 14 did too.
>
> However, earlier Fedora versions didn't include it. I know this because
> I wrote my own sa_update weekly cron job in 2009 and its still lurking
> in my CVS repository.

So they do; I stand corrected.
OP: Please accept my apologies.


martin at gregorie

Aug 24, 2011, 10:15 AM

Post #21 of 25 (3262 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 2011-08-24 at 11:35 -0500, Michael Cronenworth wrote:
> The cron job:
> 10 4 * * * root /usr/share/spamassassin/sa-update.cron 2>&1 | tee -a
> /var/log/sa-update.log
>
> The cron job script:
> /usr/share/spamassassin/sa-update.cron
>
Yep: I see exactly the same cron.d script in Fedora 13 and 15. I can't
check Fedora 14 because I replaced it with 15 last week.

IIRC I had about one occurrence of the same sa-update failure back when
this was first mentioned. Since then its just worked. Might you have
either a network configuration or an SA rules repository access controls
problem?

NOTES
=====
1) If/when you move to F15, be aware that it is using systemd in place
of the old system V init & daemon control system. The main ramification
is that, although 'service xxx start|stop' will still work OK in
scripts, any script that relies on 'service xxx status' will probably
fail. Its best to simply rewrite any scripts that use it to use
'systemctl' instead because that is the way ahead.

2) There is also an issue with UDEV rules that execute programs
in /usr/bin if you have /usr in a separate partition. This is best fixed
by moving /usr back into the root partition. Contact me offlist if you
need more detail on that.

Martin


Dan.McDonald at austinenergy

Aug 24, 2011, 10:16 AM

Post #22 of 25 (3261 views)
Permalink
Re: updates mirror is down [In reply to]

On Aug 24, 2011, at 11:27 AM, "Walter Hurry" <walterhurry [at] lavabit> wrote:

> On Wed, 24 Aug 2011 18:14:49 +0200, Karsten Br├â┬Ąckelmann wrote:
>
>> On Wed, 2011-08-24 at 10:46 -0500, Michael Cronenworth wrote:
>>
>> Most certainly distro specific.
>

I saw the same behavior on Mandriva Enterprise server 5.2, so I don't believe this to be distro specific.
> I don't think so. Fedora does not create a cron job for sa-update.
> Perhaps OP would be well advised to inspect the script run by cron. My
> guess is that it is something made locally.
>
>


jhardin at impsec

Aug 24, 2011, 1:32 PM

Post #23 of 25 (3249 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 24 Aug 2011, Axb wrote:

> On 2011-08-24 16:02, Michael Cronenworth wrote:
>> Hello,
>>
>> For the past few days, my SpamAssassin instance has been trying to get
>> its updates from one mirror and the mirror is down.
>>
>> Can someone contact the admin of the mirror or remove it from the mirror
>> list?
>>
>> Mirror: daryl.dostech.ca
>
> Would any of the biggish hosters reading this list be willing to donate a
> small VPS to add a couple of update mirrors and offload Daryl's boxes?

<cough>Coral</cough>

Granted a coralified reference to daryl.dostech.ca shouldn't be the _only_
reference, but putting it first woudl sure reduce his load.

--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin [at] impsec FALaholic #11174 pgpk -a jhardin [at] impsec
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The problem is when people look at Yahoo, slashdot, or groklaw and
jump from obvious and correct observations like "Oh my God, this
place is teeming with utter morons" to incorrect conclusions like
"there's nothing of value here". -- Al Petrofsky, in Y! SCOX
-----------------------------------------------------------------------
Today: the 1932nd anniversary of the destruction of Pompeii


guenther at rudersport

Aug 24, 2011, 1:47 PM

Post #24 of 25 (3247 views)
Permalink
Re: updates mirror is down [In reply to]

On Wed, 2011-08-24 at 13:32 -0700, John Hardin wrote:
> On Wed, 24 Aug 2011, Axb wrote:

> > Would any of the biggish hosters reading this list be willing to donate a
> > small VPS to add a couple of update mirrors and offload Daryl's boxes?
>
> <cough>Coral</cough>
>
> Granted a coralified reference to daryl.dostech.ca shouldn't be the _only_
> reference, but putting it first woudl sure reduce his load.

Speaking about coughing... Nope. Load balancing on mirrors is done by
weighting, order does not have any impact on priority.

Anyway, the SA PMC already got an offer for an additional mirror [1],
and the current issues with Daryl's mirror are expected to be fixed
soon-ish.


[1] Before Alex broadly poking at the larger players. ;)

--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


axb.lists at gmail

Aug 24, 2011, 2:32 PM

Post #25 of 25 (3250 views)
Permalink
Re: updates mirror is down [In reply to]

On 2011-08-24 22:32, John Hardin wrote:
> On Wed, 24 Aug 2011, Axb wrote:
>
>> On 2011-08-24 16:02, Michael Cronenworth wrote:
>>> Hello,
>>>
>>> For the past few days, my SpamAssassin instance has been trying to get
>>> its updates from one mirror and the mirror is down.
>>>
>>> Can someone contact the admin of the mirror or remove it from the mirror
>>> list?
>>>
>>> Mirror: daryl.dostech.ca
>>
>> Would any of the biggish hosters reading this list be willing to
>> donate a small VPS to add a couple of update mirrors and offload
>> Daryl's boxes?
>
> <cough>Coral</cough>

<blush>

Assuming Coral works... their Wiki isn't too well "Coralized" .-)

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.