Mailing List Archive: Apache: Users

Apache killing server by using all swap

Index | Next | Previous | View Threaded

stargazer at blueyonder

May 12, 2008, 1:36 PM

Post #1 of 5 (369 views)
Permalink

Apache killing server by using all swap

Hi list, I have a RH server using Plesk with Apache 2.0.54. I can't
upgrade Apache via yum and am very reluctant to try manually
installing a newer version in case it breaks Plesk. The server hosts
about 60 domains and needed a daily restart due to it eating all the memory
then swap, which actually did report a zero free value. After lots of
investigaton I have found this bash script frees up the swap:

ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/);
print `ipcrm sem $a[1]`}'
service httpd restart

I run a 10 minute cron job which checks the value of swap as reported by
"free -m"
and if it is < 100M triggers this script.

The server has 750M Physical RAM. httpd.conf is the default apart from
whatever Plesk does to it. When I'm low on swap top shows nothing
obvious in that
theres no single process using the memory, there are usually about 8
httpd children
using about 2% each.

I'm really looking for help fixing this, if anyone can I'd be really
grateful -
just ask for whatever logs are needed.

Thanks!

- John

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd
" from the digest: users-digest-unsubscribe [at] httpd
For additional commands, e-mail: users-help [at] httpd

stargazer at blueyonder

May 14, 2008, 12:26 PM

Post #2 of 5 (377 views)
Permalink

Re: Apache killing server by using all swap [In reply to]

Stargazer wrote:
> Hi list, I have a RH server using Plesk with Apache 2.0.54. I can't
>
> ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/);
> print `ipcrm sem $a[1]`}'
> service httpd restart
>
Can I bumpt this please?
Could anyone suggest a command I can issue just when swap is near zero
so I can post back
the results here and hopefully find the culprit(s)? Thank you

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd
" from the digest: users-digest-unsubscribe [at] httpd
For additional commands, e-mail: users-help [at] httpd

i.galic at brainsware

May 14, 2008, 12:54 PM

Post #3 of 5 (342 views)
Permalink

Re: Apache killing server by using all swap [In reply to]

On Mon, 12 May 2008 21:36:20 +0100
Stargazer <stargazer [at] blueyonder> wrote:

> Hi list, I have a RH server using Plesk with Apache 2.0.54. I can't
> upgrade Apache via yum and am very reluctant to try manually
> installing a newer version in case it breaks Plesk. The server hosts
> about 60 domains and needed a daily restart due to it eating all the memory
> then swap, which actually did report a zero free value. After lots of
> investigaton I have found this bash script frees up the swap:
>
> ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/);
> print `ipcrm sem $a[1]`}'
> service httpd restart

This sounds more like a HACK, rather than a solution.

> The server has 750M Physical RAM. httpd.conf is the default apart from
> whatever Plesk does to it. When I'm low on swap top shows nothing
> obvious in that
> theres no single process using the memory, there are usually about 8
> httpd children
> using about 2% each.

750/8 ~ 93M each... that's awfully lot for a httpd child.
Are you sure you're not leaking memory anywhere?

Setting
http://httpd.apache.org/docs/2.0/mod/mpm_common.html#maxrequestsperchild
back to default (usually it's set to 0 == infinite), might show if it's
leaking or not.

Also take a look at http://httpd.apache.org/docs/2.0/misc/perf-tuning.html

> I'm really looking for help fixing this, if anyone can I'd be really
> grateful -
> just ask for whatever logs are needed.

If this doesn't help you, you might want to supply your MPM configuration,
also a listing of the used modules would be a good.

> Thanks!
>
> - John

So long,
Igor

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd
" from the digest: users-digest-unsubscribe [at] httpd
For additional commands, e-mail: users-help [at] httpd

nick at webthing

May 14, 2008, 1:06 PM

Post #4 of 5 (351 views)
Permalink

Re: Apache killing server by using all swap [In reply to]

>>
>> ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/);
>> print `ipcrm sem $a[1]`}'
>> service httpd restart
>
> This sounds more like a HACK, rather than a solution.

That's not swap it's cleaning up!

Almost certainly a (badly) buggy app leading to lots of crashes.
Or just leaking semaphores.

--
Nick Kew

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd
" from the digest: users-digest-unsubscribe [at] httpd
For additional commands, e-mail: users-help [at] httpd

stargazer at blueyonder

May 14, 2008, 11:43 PM

Post #5 of 5 (330 views)
Permalink

Re: Apache killing server by using all swap [In reply to]

Igor Galić wrote:
> On Mon, 12 May 2008 21:36:20 +0100
> Stargazer <stargazer [at] blueyonder> wrote:
>
> 750/8 ~ 93M each... that's awfully lot for a httpd child.
> Are you sure you're not leaking memory anywhere?
>
> Setting
> http://httpd.apache.org/docs/2.0/mod/mpm_common.html#maxrequestsperchild
> back to default (usually it's set to 0 == infinite), might show if it's
> leaking or not.
>
> Also take a look at http://httpd.apache.org/docs/2.0/misc/perf-tuning.html
>
>
Thanks for your help!
This is what its set to right now:

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# ServerLimit: maximum value for MaxClients for the lifetime of the server
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>

# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>

I read your link so I'll change the second MaxRequestsPerChild from 0 to
10000 and give it a day.
> If this doesn't help you, you might want to supply your MPM configuration,
> also a listing of the used modules would be a good.
LoadModule access_module modules/mod_access.so
LoadModule auth_module modules/mod_auth.so
LoadModule auth_anon_module modules/mod_auth_anon.so
LoadModule auth_dbm_module modules/mod_auth_dbm.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule ldap_module modules/mod_ldap.so
LoadModule auth_ldap_module modules/mod_auth_ldap.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
#LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule cern_meta_module modules/mod_cern_meta.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule asis_module modules/mod_asis.so
LoadModule info_module modules/mod_info.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule file_cache_module modules/mod_file_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule frontpage_module /usr/lib/httpd/modules/mod_frontpage.so
LoadModule jk_module /usr/lib/httpd/modules/mod_jk.so

Also, heres what free shows right now:

total used free shared buffers cached
Mem: 758464 750176 8288 0 2180 61428
-/+ buffers/cache: 686568 71896
Swap: 1052248 768892 283356

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd
" from the digest: users-digest-unsubscribe [at] httpd
For additional commands, e-mail: users-help [at] httpd

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads