Login | Register For Free | Help	Search this list this category for: (Advanced)

Mailing List Archive: SpamAssassin: devel [Bug 6793] PATCH reduce sa-awl memory usage   Index | Next | Previous | View Flat

bugzilla-daemon at bugzilla Apr 23, 2012, 4:12 PM Views: 504 Permalink [Bug 6793] PATCH reduce sa-awl memory usage https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6793 --- Comment #1 from Mark Martinec <Mark.Martinec [at] ijs> 2012-04-23 23:12:10 UTC --- > current version of sa-awl loads full database key list to memory before > showing any stats or performing maintenance. I believe it's obvious that > this behavior is undesirable and makes large databases impossible to handle I fully agree. Operations should not depend on being able to load a complete database into memory if the size of a database is potentially unlimited. > The patch below improves sa-awl scaling and responsiveness by scanning > database row-by-row basis instead of loading all keys to memory first. > Tested cleaning db with over 8 million rows. > For a cached db with 850K rows memory usage lowers from 1G to 6M, Great, thanks! > execution time is around 12% slower (26 vs. 29 sec), though. Despite Kevin's concern on the ML, I don't think this is worth introducing yet another option (and risking people using it without fully understanding the implications). One can make a program arbitrarily fast if correct and reliable operation is not mandated :) > I'm not a perl expert, please review. My only concern is deleting a key while the each() operator is active. Please see my alternative patch. -- Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug.

Subject User Time [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 4:12 PM     [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 4:13 PM     [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 11:22 PM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 12:47 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 1:57 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 2:08 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 2:22 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 3:22 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 3:38 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 5:48 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 6:12 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 6:19 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:33 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:38 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:40 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:58 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:20 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:27 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:37 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 9:37 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:35 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:39 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:39 AM         [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 11:03 AM

Index | Next | Previous | View Flat   SpamAssassin     announce     users     devel     commits

Interested in having your list archived? Contact Gossamer Threads
 

Web Applications & Managed Hosting Powered by Gossamer Threads Inc.