
hans at moc
Jul 3, 2013, 12:09 AM
Post #5 of 10
(148 views)
Permalink
|
|
Re: Swap exceed & Child not responding to CLI, killing it
[In reply to]
|
|
Den 06/20/2013 04:51 PM, Reinis Rozitis skrev: >> Any hint or help much appreciated. > > Try setting sysctl vm.swappiness=0 Well, after trying that a couple of days, and upgrading to 3.0.4, I think I have hit a memory leak. Varnish keeps up eating mem and when there is no more mem I got a restart. Last one was: Child (6477) died signal=6 Child (6477) Panic message: Assert error in VGZ_NewGzip(), cache_gzip.c line 209:#012 Condition(Z_OK == i) not true.#012errno = 12 (Cannot allocate memory)... Is there anything in 'varnishstat -1' which can give a clue? client_conn 626603 9.22 Client connections accepted client_drop 0 0.00 Connection dropped, no sess/wrk client_req 2501923 36.83 Client requests received cache_hit 2135392 31.43 Cache hits cache_hitpass 2486 0.04 Cache hits for pass cache_miss 347931 5.12 Cache misses backend_conn 53645 0.79 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 1 0.00 Backend conn. failures backend_reuse 319326 4.70 Backend conn. reuses backend_toolate 17374 0.26 Backend conn. was closed backend_recycle 336723 4.96 Backend conn. recycles backend_retry 5 0.00 Backend conn. retry fetch_head 535 0.01 Fetch head fetch_length 357385 5.26 Fetch with Length fetch_chunked 11490 0.17 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 702 0.01 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 0 0.00 Fetch failed fetch_1xx 0 0.00 Fetch no body (1xx) fetch_204 0 0.00 Fetch no body (204) fetch_304 1243 0.02 Fetch no body (304) n_sess_mem 207 . N struct sess_mem n_sess 70 . N struct sess n_object 252984 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 253053 . N struct objectcore n_objecthead 58090 . N struct objecthead n_waitinglist 94 . N struct waitinglist n_vbc 19 . N struct vbc n_wrk 400 . N worker threads n_wrk_create 400 0.01 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 0 0.00 N worker threads limited n_wrk_lqueue 0 0.00 work request queue length n_wrk_queued 6 0.00 N queued work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 17 . N backends n_expired 94945 . N expired objects n_lru_nuked 0 . N LRU nuked objects n_lru_moved 1415668 . N LRU moved objects losthdr 1 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 2436792 35.87 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 626604 9.22 Total Sessions s_req 2501923 36.83 Total Requests s_pipe 1610 0.02 Total pipe s_pass 23428 0.34 Total pass s_fetch 371355 5.47 Total fetch s_hdrbytes 907219967 13354.04 Total header bytes s_bodybytes 21866125041 321863.59 Total body bytes sess_closed 107908 1.59 Session Closed sess_pipeline 4939 0.07 Session Pipeline sess_readahead 1677 0.02 Session Read Ahead sess_linger 2426224 35.71 Session Linger sess_herd 1424318 20.97 Session herd shm_records 118481149 1744.01 SHM records shm_writes 6661958 98.06 SHM writes shm_flushes 6 0.00 SHM flushes due to overflow shm_cont 6534 0.10 SHM MTX contention shm_cycles 53 0.00 SHM cycles through buffer sms_nreq 302 0.00 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 401612 . SMS bytes allocated sms_bfree 401612 . SMS bytes freed backend_req 371356 5.47 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_ban 298 . N total active bans n_ban_gone 277 . N total gone bans n_ban_add 298 0.00 N new bans added n_ban_retire 0 0.00 N old bans deleted n_ban_obj_test 32976 0.49 N objects tested n_ban_re_test 290209 4.27 N regexps tested against n_ban_dups 276 0.00 N duplicate bans removed hcb_nolock 2485461 36.59 HCB Lookups without lock hcb_lock 139934 2.06 HCB Lookups with lock hcb_insert 139934 2.06 HCB Inserts esi_errors 0 0.00 ESI parse errors (unlock) esi_warnings 0 0.00 ESI parse warnings (unlock) accept_fail 0 0.00 Accept failures client_drop_late 0 0.00 Connection dropped late uptime 67936 1.00 Client uptime dir_dns_lookups 0 0.00 DNS director lookups dir_dns_failed 0 0.00 DNS director failed lookups dir_dns_hit 0 0.00 DNS director cached lookups hit dir_dns_cache_full 0 0.00 DNS director full dnscache vmods 0 . Loaded VMODs n_gzip 131411 1.93 Gzip operations n_gunzip 295578 4.35 Gunzip operations sess_pipe_overflow 0 . Dropped sessions due to session pipe overflow LCK.sms.creat 1 0.00 Created locks LCK.sms.destroy 0 0.00 Destroyed locks LCK.sms.locks 906 0.01 Lock Operations LCK.sms.colls 0 0.00 Collisions LCK.smp.creat 0 0.00 Created locks LCK.smp.destroy 0 0.00 Destroyed locks LCK.smp.locks 0 0.00 Lock Operations LCK.smp.colls 0 0.00 Collisions LCK.sma.creat 1 0.00 Created locks LCK.sma.destroy 0 0.00 Destroyed locks LCK.sma.locks 994791 14.64 Lock Operations LCK.sma.colls 0 0.00 Collisions LCK.smf.creat 1 0.00 Created locks LCK.smf.destroy 0 0.00 Destroyed locks LCK.smf.locks 341171 5.02 Lock Operations LCK.smf.colls 0 0.00 Collisions LCK.hsl.creat 0 0.00 Created locks LCK.hsl.destroy 0 0.00 Destroyed locks LCK.hsl.locks 0 0.00 Lock Operations LCK.hsl.colls 0 0.00 Collisions LCK.hcb.creat 1 0.00 Created locks LCK.hcb.destroy 0 0.00 Destroyed locks LCK.hcb.locks 222504 3.28 Lock Operations LCK.hcb.colls 0 0.00 Collisions LCK.hcl.creat 0 0.00 Created locks LCK.hcl.destroy 0 0.00 Destroyed locks LCK.hcl.locks 0 0.00 Lock Operations LCK.hcl.colls 0 0.00 Collisions LCK.vcl.creat 1 0.00 Created locks LCK.vcl.destroy 0 0.00 Destroyed locks LCK.vcl.locks 3172 0.05 Lock Operations LCK.vcl.colls 0 0.00 Collisions LCK.stat.creat 1 0.00 Created locks LCK.stat.destroy 0 0.00 Destroyed locks LCK.stat.locks 626744 9.23 Lock Operations LCK.stat.colls 0 0.00 Collisions LCK.sessmem.creat 1 0.00 Created locks LCK.sessmem.destroy 0 0.00 Destroyed locks LCK.sessmem.locks 631760 9.30 Lock Operations LCK.sessmem.colls 0 0.00 Collisions LCK.wstat.creat 1 0.00 Created locks LCK.wstat.destroy 0 0.00 Destroyed locks LCK.wstat.locks 168421 2.48 Lock Operations LCK.wstat.colls 0 0.00 Collisions LCK.herder.creat 1 0.00 Created locks LCK.herder.destroy 0 0.00 Destroyed locks LCK.herder.locks 1 0.00 Lock Operations LCK.herder.colls 0 0.00 Collisions LCK.wq.creat 2 0.00 Created locks LCK.wq.destroy 0 0.00 Destroyed locks LCK.wq.locks 3206044 47.19 Lock Operations LCK.wq.colls 0 0.00 Collisions LCK.objhdr.creat 140007 2.06 Created locks LCK.objhdr.destroy 81917 1.21 Destroyed locks LCK.objhdr.locks 10343620 152.26 Lock Operations LCK.objhdr.colls 0 0.00 Collisions LCK.exp.creat 1 0.00 Created locks LCK.exp.destroy 0 0.00 Destroyed locks LCK.exp.locks 517536 7.62 Lock Operations LCK.exp.colls 0 0.00 Collisions LCK.lru.creat 2 0.00 Created locks LCK.lru.destroy 0 0.00 Destroyed locks LCK.lru.locks 354670 5.22 Lock Operations LCK.lru.colls 0 0.00 Collisions LCK.cli.creat 1 0.00 Created locks LCK.cli.destroy 0 0.00 Destroyed locks LCK.cli.locks 22642 0.33 Lock Operations LCK.cli.colls 0 0.00 Collisions LCK.ban.creat 1 0.00 Created locks LCK.ban.destroy 0 0.00 Destroyed locks LCK.ban.locks 544373 8.01 Lock Operations LCK.ban.colls 0 0.00 Collisions LCK.vbp.creat 1 0.00 Created locks LCK.vbp.destroy 0 0.00 Destroyed locks LCK.vbp.locks 13583 0.20 Lock Operations LCK.vbp.colls 0 0.00 Collisions LCK.vbe.creat 1 0.00 Created locks LCK.vbe.destroy 0 0.00 Destroyed locks LCK.vbe.locks 107279 1.58 Lock Operations LCK.vbe.colls 0 0.00 Collisions LCK.backend.creat 17 0.00 Created locks LCK.backend.destroy 0 0.00 Destroyed locks LCK.backend.locks 834359 12.28 Lock Operations LCK.backend.colls 0 0.00 Collisions SMF.s0.c_req 194123 2.86 Allocator requests SMF.s0.c_fail 0 0.00 Allocator failures SMF.s0.c_bytes 4399304704 64756.60 Bytes allocated SMF.s0.c_freed 3393925120 49957.68 Bytes freed SMF.s0.g_alloc 58453 . Allocations outstanding SMF.s0.g_bytes 1005379584 . Bytes outstanding SMF.s0.g_space 63419129856 . Bytes available SMF.s0.g_smf 58533 . N struct smf SMF.s0.g_smf_frag 3 . N small free smf SMF.s0.g_smf_large 77 . N large free smf SMA.Transient.c_req 550339 8.10 Allocator requests SMA.Transient.c_fail 0 0.00 Allocator failures SMA.Transient.c_bytes 40394887294 594602.09 Bytes allocated SMA.Transient.c_freed 30467874668 448479.08 Bytes freed SMA.Transient.g_alloc 340120 . Allocations outstanding SMA.Transient.g_bytes 9927012626 . Bytes outstanding SMA.Transient.g_space 0 . Bytes available VBE.web15(192.168.4.15,,80).vcls 1 . VCL references VBE.web15(192.168.4.15,,80).happy 0 . Happy health probes VBE.web25(192.168.4.25,,80).vcls 1 . VCL references VBE.web25(192.168.4.25,,80).happy 0 . Happy health probes VBE.web78(192.168.4.78,,80).vcls 1 . VCL references VBE.web78(192.168.4.78,,80).happy 0 . Happy health probes VBE.web78(192.168.4.78,,8081).vcls 1 . VCL references VBE.web78(192.168.4.78,,8081).happy 0 . Happy health probes VBE.web32(192.168.4.32,,80).vcls 1 . VCL references VBE.web32(192.168.4.32,,80).happy 0 . Happy health probes VBE.web53(192.168.4.53,,80).vcls 1 . VCL references VBE.web53(192.168.4.53,,80).happy 0 . Happy health probes VBE.web48(192.168.4.48,,80).vcls 1 . VCL references VBE.web48(192.168.4.48,,80).happy 0 . Happy health probes VBE.web42(192.168.4.42,,80).vcls 1 . VCL references VBE.web42(192.168.4.42,,80).happy 0 . Happy health probes VBE.web43(192.168.4.43,,80).vcls 1 . VCL references VBE.web43(192.168.4.43,,80).happy 0 . Happy health probes VBE.web45(192.168.4.45,,80).vcls 1 . VCL references VBE.web45(192.168.4.45,,80).happy 0 . Happy health probes VBE.web21(192.168.4.21,,80).vcls 1 . VCL references VBE.web21(192.168.4.21,,80).happy 0 . Happy health probes VBE.web62(192.168.4.62,,80).vcls 1 . VCL references VBE.web62(192.168.4.62,,80).happy 18446744073709551615 . Happy health probes VBE.web56(192.168.4.56,,80).vcls 1 . VCL references VBE.web56(192.168.4.56,,80).happy 0 . Happy health probes VBE.web30(192.168.4.30,,80).vcls 1 . VCL references VBE.web30(192.168.4.30,,80).happy 0 . Happy health probes VBE.web55(192.168.4.55,,80).vcls 1 . VCL references VBE.web55(192.168.4.55,,80).happy 0 . Happy health probes VBE.web75(192.168.4.75,,80).vcls 1 . VCL references VBE.web75(192.168.4.75,,80).happy 0 . Happy health probes VBE.web88(192.168.4.88,,80).vcls 1 . VCL references VBE.web88(192.168.4.88,,80).happy 0 . Happy health probes -- Venlig hilsen Hans Schou tel:46923438 _______________________________________________ varnish-misc mailing list varnish-misc [at] varnish-cache https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
|