Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: devel
[Bug 6793] PATCH reduce sa-awl memory usage
 

Index | Next | Previous | View Flat


bugzilla-daemon at bugzilla

Apr 23, 2012, 4:12 PM


Views: 504
Permalink
[Bug 6793] PATCH reduce sa-awl memory usage

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6793

--- Comment #1 from Mark Martinec <Mark.Martinec [at] ijs> 2012-04-23 23:12:10 UTC ---
> current version of sa-awl loads full database key list to memory before
> showing any stats or performing maintenance. I believe it's obvious that
> this behavior is undesirable and makes large databases impossible to handle

I fully agree. Operations should not depend on being able to load a
complete database into memory if the size of a database is potentially
unlimited.

> The patch below improves sa-awl scaling and responsiveness by scanning
> database row-by-row basis instead of loading all keys to memory first.
> Tested cleaning db with over 8 million rows.
> For a cached db with 850K rows memory usage lowers from 1G to 6M,

Great, thanks!

> execution time is around 12% slower (26 vs. 29 sec), though.

Despite Kevin's concern on the ML, I don't think this is worth
introducing yet another option (and risking people using it
without fully understanding the implications).

One can make a program arbitrarily fast
if correct and reliable operation is not mandated :)

> I'm not a perl expert, please review.

My only concern is deleting a key while the each() operator is active.

Please see my alternative patch.

--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

Subject User Time
[Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 4:12 PM
    [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 4:13 PM
    [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 23, 2012, 11:22 PM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 12:47 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 1:57 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 2:08 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 2:22 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 3:22 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 3:38 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 5:48 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 6:12 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 6:19 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:33 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:38 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:40 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 7:58 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:20 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:27 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 8:37 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 9:37 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:35 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:39 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 10:39 AM
        [Bug 6793] PATCH reduce sa-awl memory usage bugzilla-daemon at bugzilla Apr 24, 2012, 11:03 AM

  Index | Next | Previous | View Flat
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.