Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ClamAV: users

Mirror Issues

 

 

ClamAV users RSS feed   Index | Next | Previous | View Threaded


alvarnell at mac

Jun 16, 2011, 11:50 PM

Post #1 of 7 (1155 views)
Permalink
Mirror Issues

Had a couple of US users who have complained about delays in updating their
defs which are obviously being caused by waiting on a couple of mirrors that
appear to have been down for several days.

- 194.47.250.218 clamav.df.lth.se at this writing has been down for 17 days.
- 69.163.100.14 clamav.theshell.com war recently added back to the list of
US mirrors and has been down ever since, over 10 days.

A typical log entry has dozens of entries that look like this:
nonblock_connect: connect timing out (30 secs)
Can't connect to port 80 of host database.clamav.net (IP: 194.47.250.218)

Yet when we check mirrors.dat it tells us:
Mirror #14
IP: 194.186.47.19
Successes: 14
Failures: 0
Last access: Tue May 31 21:11:08 2011
Ignore: No

This user has not inserted his country code. He is located in central US.
I have changed my freshclam.conf to include the US country code, but it
seems to make no difference.

I have been running "$ /Users/jvarnell/Desktop/\~Download/ClamXav\
Stuff/Check\ current\ database\ servers.command ; exit;"
periodically over the past three weeks and usually find one or both included
in my list of mirrors to check. At this moment, neither are.

So here are my questions of the day:

- How many days does a mirror have to be old or bad before it is taken off
the list of available servers? Put another way, when should I start asking
about it?

- Why are there so many (5) off-shore servers included in the list of
available US mirrors when there are 11 in-country servers?

- Why is a "connect timing out" error not considered a "Failure" for the
purpose of ignoring a server?

- How many failures before ignore?

- There have been issues with TheShell in the past and for awhile it was
taken off the list. Now it's been added back, but still doesn't seem to be
working.

I have attempted to summarize my discussions, but there are many more
details with one of these users here
<http://markallan.co.uk/BB/viewtopic.php?t=2349>


-Al-

--
Al Varnell
Mountain View, CA


_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


alvarnell at mac

Jun 17, 2011, 1:13 AM

Post #2 of 7 (1123 views)
Permalink
Mirror Issues [In reply to]

Sorry, please disregard my previous message and use this one instead.

I included the wrong log entry.

Also found that TheShell is back off the list. I saw that it was still
listed on the Status site but in running dig @ns1.clamav.net
db.us.big.clamav.net I see it's no longer on the active list. There are a
couple of other sites no longer on the list that are still shown on the
status site (clamav.catt.com & clamav.liquidweb.com) and at least one site
in the list not shown on the status site (unknown.lyonlabs.com).
------------------------------------------

Had a couple of US users who have complained about delays in updating their
defs which are obviously being caused by waiting on a couple of mirrors that
appear to have been down for several days.

- 194.47.250.218 clamav.df.lth.se at this writing has been down for 17 days.
- 69.163.100.14 clamav.theshell.com was recently added back to the list of
US mirrors, has been down ever since, over 10 days and now see it has been
removed again.

A typical log entry has dozens of entries that look like this:
nonblock_connect: connect timing out (30 secs)
Can't connect to port 80 of host database.clamav.net (IP: 194.47.250.218)

Yet when we check mirrors.dat it tells us:
Mirror #4
IP: 194.47.250.218
Successes: 11
Failures: 0
Last access: Thu May 26 17:37:43 2011
Ignore: No

This user has not inserted his country code and is located in central US. I
have changed my freshclam.conf to include the US country code, but it seems
to make no difference. I'm on the US west coast.

I have been running "$ host database.clamav.net" periodically over the past
three weeks and usually find one or both included in my list of mirrors to
check. At this moment, neither are.

So here are my questions of the day:

- How many days does a mirror have to be old or bad before it is taken off
the list of available servers? Put another way, when should I start asking
about it?

- Why are there so many (5) off-shore servers included in the list of
available US mirrors when there are around 11 in-country servers?

- Why is a "connect timing out" error not considered a "Failure" for the
purpose of ignoring a server?

- How many failures before ignore?

I have attempted to summarize my discussions, but there are many more
details with one of these users here
<http://markallan.co.uk/BB/viewtopic.php?t=2349>


-Al-

--
Al Varnell
Mountain View, CA


_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


tkojm at clamav

Jun 20, 2011, 6:35 AM

Post #3 of 7 (1118 views)
Permalink
Re: Mirror Issues [In reply to]

On Fri, 17 Jun 2011 01:13:35 -0700 Al Varnell <alvarnell [at] mac> wrote:

> - Why is a "connect timing out" error not considered a "Failure" for the
> purpose of ignoring a server?

Hi Al,

we changed the behavior (and don't treat connection issues as real
failures) some time ago. However, we're open for discussions on this
issue so please open a bug report (bugs.clamav.net) and we'll try to
find a good solution.

Thanks,

--
oo ..... Tomasz Kojm <tkojm [at] clamav>
(\/)\......... http://www.ClamAV.net/gpg/tkojm.gpg
\..........._ 0DCA5A08407D5288279DB43454822DC8985A444B
//\ /\ Mon Jun 20 15:30:15 CEST 2011
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


luca at clamav

Jun 21, 2011, 2:46 AM

Post #4 of 7 (1103 views)
Permalink
Re: Mirror Issues [In reply to]

Hello Al,

> - 194.47.250.218 clamav.df.lth.se at this writing has been down for 17 days.

this mirror is blocking our monitor tool but works just fine.
In other words the status page shows it's down even if it's actually up.

> - 69.163.100.14 clamav.theshell.com was recently added back to the list of
> US mirrors, has been down ever since, over 10 days and now see it has been
> removed again.

I removed it from the RR a long time ago. I'll remove it from the mirror
status page after it's declared definitely dead.

> - How many days does a mirror have to be old or bad before it is taken off
> the list of available servers? Put another way, when should I start asking
> about it?

it's removed from the DNS RR usually after 10 days.

> - Why are there so many (5) off-shore servers included in the list of
> available US mirrors when there are around 11 in-country servers?

because those 11 mirrors in US are not enough to handle all the traffic
generated in US.



Best regards

--
Luca Gibelli (luca _at_ clamav.net) ClamAV, a GPL anti-virus toolkit
[Tel] +39 0187 1851862 [Fax] +39 0187 1852252 [IM] nervous/jabber.linux.it
PGP key id 5EFC5582 @ any key-server || http://www.clamav.net/gpg/luca.gpg
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


alvarnell at mac

Jun 21, 2011, 11:54 PM

Post #5 of 7 (1102 views)
Permalink
Re: Mirror Issues [In reply to]

Luca,

Thanks for the detailed answers. Only one of them doesn't ring true to me.

On 6/21/11 2:46 AM, "Luca Gibelli" <luca [at] clamav> wrote:

>
> Hello Al,
>
>> - 194.47.250.218 clamav.df.lth.se at this writing has been down for 17 days.
>
> this mirror is blocking our monitor tool but works just fine.
> In other words the status page shows it's down even if it's actually up.
>
I do two scheduled updates per day and rarely do a manual update at any
other time. My last successful update from this server was on May 3 at
15:45.

All attempts to use that server since then have been connection failures
with a success on the very next mirror tried. All errors logged were:

connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused
Can't connect to port 80 of host db.US.clamav.net (IP: 194.47.250.218)

May 9 15:45
May 10 15:45
May 11 7:45 & 15:45
May 12 7:45 & 15:45
May 15 7:45 & 15:45
May 16 7:45
& 15:45 Twice, 30 seconds apart?
May 17 7:45
May 18 7:45 & 15:45
May 19 7:45 & 15:45
May 23 7:45
Jun 2 7:45
Jun 5 7:45
Jun 10 15:45
Jun 12 7:45 & 15:45
Jun 15 15:45

So over a period of more than a month every attempt (23) to contact this
server failed in the same manner and there were only two errors involving
server 209.222.131.222 during the same period. I may be jumping to
conclusions, but I think it really was down for a very long time.

It also seems strange to me that the rotation scheme would pick this server
to go to first so many times in a row.

I don't routinely check the status of servers, but I started to after the
user I mentioned began complaining. He claims to have had even more
failures than I did. At any rate, while I watched, the bad days continued
to increase until last weekend. Looking at it right now it says it has been
up for three days straight and is only 1 hour behind, so it would seem that
your monitoring tool is no longer being blocked.

I see just now that it has worked it's way up to the top of my rr list, so I
expect I'll be seeing something from it soon.


-Al-

--
Al Varnell
Mountain View, CA

>> - 69.163.100.14 clamav.theshell.com was recently added back to the list of
>> US mirrors, has been down ever since, over 10 days and now see it has been
>> removed again.
>
> I removed it from the RR a long time ago. I'll remove it from the mirror
> status page after it's declared definitely dead.
>
>> - How many days does a mirror have to be old or bad before it is taken off
>> the list of available servers? Put another way, when should I start asking
>> about it?
>
> it's removed from the DNS RR usually after 10 days.
>
>> - Why are there so many (5) off-shore servers included in the list of
>> available US mirrors when there are around 11 in-country servers?
>
> because those 11 mirrors in US are not enough to handle all the traffic
> generated in US.
>


_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


edwintorok at gmail

Oct 23, 2011, 9:00 AM

Post #6 of 7 (951 views)
Permalink
Re: Mirror issues [In reply to]

On 10/23/2011 05:33 PM, Jim Popovitch wrote:
> Is it my lack of clue, or are there a fair amount of mirror issues today?

I'm not seeing any issues with the mirror I use, what error messages do you see?

Best regards,
--Edwin
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml


jimpop at gmail

Oct 23, 2011, 12:22 PM

Post #7 of 7 (971 views)
Permalink
Re: Mirror issues [In reply to]

2011/10/23 Török Edwin <edwintorok [at] gmail>:
> On 10/23/2011 05:33 PM, Jim Popovitch wrote:
>> Is it my lack of clue, or are there a fair amount of mirror issues today?
>
> I'm not seeing any issues with the mirror I use, what error messages do you see?

Specifically it was connection issues (multiple mirrors) and "DNS
record is older than 3 hours" errors, earlier today. Seems to have
cleared up now.

-Jim P.

Oct 23 04:42:20 svr3 freshclam[2502]: Received signal: wake up
Oct 23 04:42:20 svr3 freshclam[2502]: ClamAV update process started at
Sun Oct 23 04:42:20 2011
Oct 23 04:42:20 svr3 freshclam[2502]: main.cld is up to date (version:
54, sigs: 1044387, f-level: 60, builder: sven)
Oct 23 04:42:20 svr3 freshclam[2502]: daily.cld is up to date
(version: 13841, sigs: 15898, f-level: 60, builder: mallan)
Oct 23 04:42:50 svr3 freshclam[2502]: nonblock_connect: connect timing
out (30 secs)
Oct 23 04:42:50 svr3 freshclam[2502]: Can't connect to port 80 of host
db.us.clamav.net (IP: 78.46.84.244)
Oct 23 04:42:50 svr3 freshclam[2502]: Trying host db.us.clamav.net
(155.98.64.87)...
Oct 23 04:42:51 svr3 freshclam[2502]: Downloading bytecode-149.cdiff [100%]
Oct 23 04:42:51 svr3 freshclam[2502]: bytecode.cld updated (version:
149, sigs: 39, f-level: 60, builder: edwin)
Oct 23 04:44:02 svr3 freshclam[2502]: Can't query
bytecode.149.62.1.0.155.98.64.87.ping.clamav.net
Oct 23 04:44:03 svr3 freshclam[2502]: Database updated (1060324
signatures) from db.us.clamav.net (IP: 155.98.64.87)
Oct 23 04:44:03 svr3 freshclam[2502]: Clamd successfully notified
about the update.
Oct 23 04:44:03 svr3 freshclam[2502]: --------------------------------------


Oct 23 07:44:06 svr3 freshclam[2502]: Received signal: wake up
Oct 23 07:44:06 svr3 freshclam[2502]: ClamAV update process started at
Sun Oct 23 07:44:06 2011
Oct 23 07:44:06 svr3 freshclam[2502]: DNS record is older than 3 hours.
Oct 23 07:44:06 svr3 freshclam[2502]: Invalid DNS reply. Falling back
to HTTP mode.
Oct 23 07:44:06 svr3 freshclam[2502]: Reading CVD header (main.cvd):
Oct 23 07:44:10 svr3 freshclam[2502]: connect_error:
getsockopt(SO_ERROR): fd=4 error=111: Connection refused
Oct 23 07:44:10 svr3 freshclam[2502]: Can't connect to port 80 of host
db.us.clamav.net (IP: 69.12.162.28)
Oct 23 07:44:10 svr3 freshclam[2502]: OK
Oct 23 07:44:10 svr3 freshclam[2502]: main.cld is up to date (version:
54, sigs: 1044387, f-level: 60, builder: sven)
Oct 23 07:44:10 svr3 freshclam[2502]: Reading CVD header (daily.cvd):
Oct 23 07:44:10 svr3 freshclam[2502]: OK
Oct 23 07:44:10 svr3 freshclam[2502]: daily.cld is up to date
(version: 13841, sigs: 15898, f-level: 60, builder: mallan)
Oct 23 07:44:10 svr3 freshclam[2502]: Reading CVD header (bytecode.cvd):
Oct 23 07:44:10 svr3 freshclam[2502]: OK
Oct 23 07:44:10 svr3 freshclam[2502]: bytecode.cld is up to date
(version: 149, sigs: 39, f-level: 60, builder: edwin)
Oct 23 07:44:12 svr3 freshclam[2502]: --------------------------------------


Oct 23 08:44:12 svr3 freshclam[2502]: Received signal: wake up
Oct 23 08:44:12 svr3 freshclam[2502]: ClamAV update process started at
Sun Oct 23 08:44:12 2011
Oct 23 08:44:12 svr3 freshclam[2502]: DNS record is older than 3 hours.
Oct 23 08:44:12 svr3 freshclam[2502]: Invalid DNS reply. Falling back
to HTTP mode.
Oct 23 08:44:12 svr3 freshclam[2502]: Reading CVD header (main.cvd):
Oct 23 08:44:16 svr3 freshclam[2502]: OK
Oct 23 08:44:16 svr3 freshclam[2502]: main.cld is up to date (version:
54, sigs: 1044387, f-level: 60, builder: sven)
Oct 23 08:44:16 svr3 freshclam[2502]: Reading CVD header (daily.cvd):
Oct 23 08:44:16 svr3 freshclam[2502]: OK (IMS)
Oct 23 08:44:16 svr3 freshclam[2502]: daily.cld is up to date
(version: 13841, sigs: 15898, f-level: 60, builder: mallan)
Oct 23 08:45:26 svr3 freshclam[2502]: Can't query
daily.13841.62.1.0.64.22.33.90.ping.clamav.net
Oct 23 08:45:26 svr3 freshclam[2502]: Reading CVD header (bytecode.cvd):
Oct 23 08:45:29 svr3 freshclam[2502]: OK (IMS)
Oct 23 08:45:29 svr3 freshclam[2502]: bytecode.cld is up to date
(version: 149, sigs: 39, f-level: 60, builder: edwin)
Oct 23 08:46:39 svr3 freshclam[2502]: Can't query
bytecode.149.62.1.0.64.22.33.90.ping.clamav.net
Oct 23 08:46:40 svr3 freshclam[2502]: --------------------------------------


Oct 23 09:12:14 svr3 freshclam[26400]: Update process interrupted
Oct 23 09:12:15 svr3 freshclam[28315]: freshclam daemon 0.97.2 (OS:
linux-gnu, ARCH: i386, CPU: i486)
Oct 23 09:12:15 svr3 freshclam[28315]: ClamAV update process started
at Sun Oct 23 09:12:15 2011
Oct 23 09:12:15 svr3 freshclam[28315]: DNS record is older than 3 hours.
Oct 23 09:12:15 svr3 freshclam[28315]: Invalid DNS reply. Falling back
to HTTP mode.
Oct 23 09:12:15 svr3 freshclam[28315]: Reading CVD header (main.cvd):
Oct 23 09:12:15 svr3 freshclam[28315]: connect_error:
getsockopt(SO_ERROR): fd=4 error=111: Connection refused
Oct 23 09:12:15 svr3 freshclam[28315]: Can't connect to port 80 of
host db.us.clamav.net (IP: 69.12.162.28)
Oct 23 09:12:15 svr3 freshclam[28315]: OK
Oct 23 09:12:15 svr3 freshclam[28315]: main.cld is up to date
(version: 54, sigs: 1044387, f-level: 60, builder: sven)
Oct 23 09:12:15 svr3 freshclam[28315]: Reading CVD header (daily.cvd):
Oct 23 09:12:15 svr3 freshclam[28315]: OK (IMS)
Oct 23 09:12:15 svr3 freshclam[28315]: daily.cld is up to date
(version: 13841, sigs: 15898, f-level: 60, builder: mallan)
Oct 23 09:13:25 svr3 freshclam[28315]: Can't query
daily.13841.62.1.0.155.98.64.87.ping.clamav.net
Oct 23 09:13:25 svr3 freshclam[28315]: Reading CVD header (bytecode.cvd):
Oct 23 09:13:25 svr3 freshclam[28315]: OK (IMS)
Oct 23 09:13:25 svr3 freshclam[28315]: bytecode.cld is up to date
(version: 149, sigs: 39, f-level: 60, builder: edwin)
Oct 23 09:14:35 svr3 freshclam[28315]: Can't query
bytecode.149.62.1.0.155.98.64.87.ping.clamav.net
Oct 23 09:14:36 svr3 freshclam[28315]: --------------------------------------

Oct 23 10:14:36 svr3 freshclam[28315]: Received signal: wake up
Oct 23 10:14:36 svr3 freshclam[28315]: ClamAV update process started
at Sun Oct 23 10:14:36 2011
Oct 23 10:14:38 svr3 freshclam[28315]: DNS record is older than 3 hours.
Oct 23 10:14:38 svr3 freshclam[28315]: Invalid DNS reply. Falling back
to HTTP mode.
Oct 23 10:14:38 svr3 freshclam[28315]: Reading CVD header (main.cvd):
Oct 23 10:14:40 svr3 freshclam[28315]: OK
Oct 23 10:14:40 svr3 freshclam[28315]: main.cld is up to date
(version: 54, sigs: 1044387, f-level: 60, builder: sven)
Oct 23 10:14:40 svr3 freshclam[28315]: Reading CVD header (daily.cvd):
Oct 23 10:14:40 svr3 freshclam[28315]: OK (IMS)
Oct 23 10:14:40 svr3 freshclam[28315]: daily.cld is up to date
(version: 13841, sigs: 15898, f-level: 60, builder: mallan)
Oct 23 10:15:50 svr3 freshclam[28315]: Can't query
daily.13841.62.1.0.78.46.84.244.ping.clamav.net
Oct 23 10:15:50 svr3 freshclam[28315]: Reading CVD header (bytecode.cvd):
Oct 23 10:15:50 svr3 freshclam[28315]: OK (IMS)
Oct 23 10:15:50 svr3 freshclam[28315]: bytecode.cld is up to date
(version: 149, sigs: 39, f-level: 60, builder: edwin)
Oct 23 10:17:00 svr3 freshclam[28315]: Can't query
bytecode.149.62.1.0.78.46.84.244.ping.clamav.net
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml

ClamAV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.