Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: devel

3.3.0 plans

 

 

First page Previous page 1 2 3 Next page Last page  View All SpamAssassin devel RSS feed   Index | Next | Previous | View Threaded


jm at jmason

Mar 14, 2008, 8:36 AM

Post #1 of 60 (3409 views)
Permalink
3.3.0 plans

ok, what bugs really need to be fixed for 3.3.0? feel free to set
Priority on bugs on the 3.3.0 milestone. in my opinion this is the
only real blocker:

5752 2008-02-04 nor P1 Let's get rid of the default rules dir
and make sa-update mandatory

All the rest is just icing (and the odd non-blocker bug, but who's
counting ;).

Also, I would also like to get rid of the "rulesrc" external, and instead
just put its contents into each branch, separately. I don't think the
idea of sharing rules in this way between branches has worked out; in my
opinion it's caused more trouble than help (unanticipated dependencies,
complexity, SVN external = horrible anyway). Is anyone still attached to
this idea?

--j.


quanah at zimbra

Mar 14, 2008, 6:53 PM

Post #2 of 60 (3294 views)
Permalink
Re: 3.3.0 plans [In reply to]

--On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
wrote:

> ok, what bugs really need to be fixed for 3.3.0? feel free to set
> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> only real blocker:

<http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
please. :)


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration


jm at jmason

Mar 15, 2008, 3:43 AM

Post #3 of 60 (3299 views)
Permalink
Re: 3.3.0 plans [In reply to]

Quanah Gibson-Mount writes:
> --On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
> wrote:
>
> > ok, what bugs really need to be fixed for 3.3.0? feel free to set
> > Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> > only real blocker:
>
> <http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
> please. :)

unfortunately, there's no patch there, and it appears that BerkeleyDB
doesn't support upgrading of .db files anyway. unlikely to get in,
given that!

--j.


wtogami at redhat

May 16, 2008, 8:28 AM

Post #4 of 60 (3238 views)
Permalink
Re: 3.3.0 plans [In reply to]

Justin Mason wrote:
> Quanah Gibson-Mount writes:
>> --On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
>> wrote:
>>
>>> ok, what bugs really need to be fixed for 3.3.0? feel free to set
>>> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
>>> only real blocker:
>> <http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
>> please. :)
>
> unfortunately, there's no patch there, and it appears that BerkeleyDB
> doesn't support upgrading of .db files anyway. unlikely to get in,
> given that!
>
> --j.


How are the 3.3.0 plans coming along?

Warren


jm at jmason

May 16, 2008, 8:38 AM

Post #5 of 60 (3234 views)
Permalink
Re: 3.3.0 plans [In reply to]

Warren Togami writes:
> Justin Mason wrote:
> > Quanah Gibson-Mount writes:
> >> --On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
> >> wrote:
> >>
> >>> ok, what bugs really need to be fixed for 3.3.0? feel free to set
> >>> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> >>> only real blocker:
> >> <http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
> >> please. :)
> >
> > unfortunately, there's no patch there, and it appears that BerkeleyDB
> > doesn't support upgrading of .db files anyway. unlikely to get in,
> > given that!
>
> How are the 3.3.0 plans coming along?

Hmm. things have been pretty quiet -- I think we're all a bit distracted
by other, non-SpamAssassin-related stuff, unfortunately.

I've been meaning to try to push it along, the big work items are:

- fixing the distribution process to work without rules in the dist
tarball (since we'll be moving to a model where with distribute
without rules and they're downloaded by the admin on install).

- generating automated daily scores with Daryl. this is kind of complex
since it relies on having good rule-QA data coming in with the
"reuse=yes" flag set, and I'm not sure what our current status is
there.

I don't see anything else remaining in the 3.3.0 queue that are necessary
as much as those two.

--j.


wtogami at redhat

May 16, 2008, 8:47 AM

Post #6 of 60 (3238 views)
Permalink
Re: 3.3.0 plans [In reply to]

Justin Mason wrote:
> Warren Togami writes:
>> Justin Mason wrote:
>>> Quanah Gibson-Mount writes:
>>>> --On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
>>>> wrote:
>>>>
>>>>> ok, what bugs really need to be fixed for 3.3.0? feel free to set
>>>>> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
>>>>> only real blocker:
>>>> <http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
>>>> please. :)
>>> unfortunately, there's no patch there, and it appears that BerkeleyDB
>>> doesn't support upgrading of .db files anyway. unlikely to get in,
>>> given that!
>> How are the 3.3.0 plans coming along?
>
> Hmm. things have been pretty quiet -- I think we're all a bit distracted
> by other, non-SpamAssassin-related stuff, unfortunately.
>
> I've been meaning to try to push it along, the big work items are:
>
> - fixing the distribution process to work without rules in the dist
> tarball (since we'll be moving to a model where with distribute
> without rules and they're downloaded by the admin on install).
>
> - generating automated daily scores with Daryl. this is kind of complex
> since it relies on having good rule-QA data coming in with the
> "reuse=yes" flag set, and I'm not sure what our current status is
> there.
>

Should 5899 be fixed before 3.3.0? It seems to be a potential cause of
upgrade confusion.

Warren


jm at jmason

May 16, 2008, 8:56 AM

Post #7 of 60 (3243 views)
Permalink
Re: 3.3.0 plans [In reply to]

Warren Togami writes:
> Justin Mason wrote:
> > Warren Togami writes:
> >> Justin Mason wrote:
> >>> Quanah Gibson-Mount writes:
> >>>> --On Friday, March 14, 2008 3:36 PM +0000 Justin Mason <jm [at] jmason>
> >>>> wrote:
> >>>>
> >>>>> ok, what bugs really need to be fixed for 3.3.0? feel free to set
> >>>>> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> >>>>> only real blocker:
> >>>> <http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416> please please
> >>>> please. :)
> >>> unfortunately, there's no patch there, and it appears that BerkeleyDB
> >>> doesn't support upgrading of .db files anyway. unlikely to get in,
> >>> given that!
> >> How are the 3.3.0 plans coming along?
> >
> > Hmm. things have been pretty quiet -- I think we're all a bit distracted
> > by other, non-SpamAssassin-related stuff, unfortunately.
> >
> > I've been meaning to try to push it along, the big work items are:
> >
> > - fixing the distribution process to work without rules in the dist
> > tarball (since we'll be moving to a model where with distribute
> > without rules and they're downloaded by the admin on install).
> >
> > - generating automated daily scores with Daryl. this is kind of complex
> > since it relies on having good rule-QA data coming in with the
> > "reuse=yes" flag set, and I'm not sure what our current status is
> > there.
> >
>
> Should 5899 be fixed before 3.3.0? It seems to be a potential cause of
> upgrade confusion.

yes.


matthias at leisi

May 19, 2008, 3:09 AM

Post #8 of 60 (3230 views)
Permalink
Re: 3.3.0 plans [In reply to]

> - fixing the distribution process to work without rules in the dist
> tarball (since we'll be moving to a model where with distribute
> without rules and they're downloaded by the admin on install).

The longer I think about it, the less convinced I am that it is a good idea.

In environments requiring strict change control or that are otherwise
paranoid in terms of security, (direct) online updates are usually not
desired (or not possible, because eg the firewall concept forbids
mailservers from doing HTTP requests).

Thus actually *requiring* online update would make deployment in such
environments more complex (it would eg require repackaging *with* the
rules prior to deployment).

The same environments would also be rather reluctant to allow direct
online updates without at least some form of sanity check (maybe not only
lint, but also some messages being checked).

Although I'd like to see some clearer structure in how rules and rules
updates (of different sources) are handled, I suggest that some basic
ruleset ("rules du jour at the day of packing a release" or something
similar) be included with the tarball.

-- Matthias


jm at jmason

May 19, 2008, 9:48 AM

Post #9 of 60 (3213 views)
Permalink
Re: 3.3.0 plans [In reply to]

Matthias Leisi writes:
> > - fixing the distribution process to work without rules in the dist
> > tarball (since we'll be moving to a model where with distribute
> > without rules and they're downloaded by the admin on install).
>
> The longer I think about it, the less convinced I am that it is a good idea.
>
> In environments requiring strict change control or that are otherwise
> paranoid in terms of security, (direct) online updates are usually not
> desired (or not possible, because eg the firewall concept forbids
> mailservers from doing HTTP requests).
>
> Thus actually *requiring* online update would make deployment in such
> environments more complex (it would eg require repackaging *with* the
> rules prior to deployment).
>
> The same environments would also be rather reluctant to allow direct
> online updates without at least some form of sanity check (maybe not only
> lint, but also some messages being checked).
>
> Although I'd like to see some clearer structure in how rules and rules
> updates (of different sources) are handled, I suggest that some basic
> ruleset ("rules du jour at the day of packing a release" or something
> similar) be included with the tarball.

hi Matthias --

if you check the bug (can't recall the number right now), there's plans to
make a tarball available of _just_ the rules, alongside the distro at
release time. That way, that kind of users can dl the tarball and
install it using "sa-update --install SpamAssassin-rules-349584.tgz" (or
whatever).

Does that work?

--j.


matthias at leisi

May 19, 2008, 11:55 AM

Post #10 of 60 (3216 views)
Permalink
Re: 3.3.0 plans [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Justin Mason schrieb:

| if you check the bug (can't recall the number right now), there's plans to
| make a tarball available of _just_ the rules, alongside the distro at
| release time. That way, that kind of users can dl the tarball and
| install it using "sa-update --install SpamAssassin-rules-349584.tgz" (or
| whatever).

How would a typical distro handle this (or how are they thought to
handle it)? Pack the rules tarball at date of packaging? That should be
acceptable for most, as this would be basically the same as today (two
tarballs instead of one, but from the same source).

A rules package from the same source as the app package is definitely
acceptable. Would / should updated rules packages be made available
through the same channels as the initial one? (IMO, this *should* be
done - through distros, CPAN, ...).

- -- Matthias

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)

iD8DBQFIMc0nxbHw2nyi/okRAgIBAJ92gIAuBvsV51KucUMIrvgmpWEQPACeLHIQ
YlWoXASfBRA9/FhRM5cdOgk=
=ZjPM
-----END PGP SIGNATURE-----


jm at jmason

May 19, 2008, 12:35 PM

Post #11 of 60 (3228 views)
Permalink
Re: 3.3.0 plans [In reply to]

Matthias Leisi writes:
> Justin Mason schrieb:
>
> | if you check the bug (can't recall the number right now), there's plans to
> | make a tarball available of _just_ the rules, alongside the distro at
> | release time. That way, that kind of users can dl the tarball and
> | install it using "sa-update --install SpamAssassin-rules-349584.tgz" (or
> | whatever).
>
> How would a typical distro handle this (or how are they thought to
> handle it)? Pack the rules tarball at date of packaging? That should be
> acceptable for most, as this would be basically the same as today (two
> tarballs instead of one, but from the same source).

Yes, that's what I was thinking.

> A rules package from the same source as the app package is definitely
> acceptable. Would / should updated rules packages be made available
> through the same channels as the initial one? (IMO, this *should* be
> done - through distros, CPAN, ...).

I'm fine with doing that ;)

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752 is the bug.
could you add that recommendation there?

--j.


wtogami at redhat

Aug 25, 2008, 4:50 PM

Post #12 of 60 (3099 views)
Permalink
Re: 3.3.0 plans [In reply to]

Justin Mason wrote:
> ok, what bugs really need to be fixed for 3.3.0? feel free to set
> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> only real blocker:
>
> 5752 2008-02-04 nor P1 Let's get rid of the default rules dir
> and make sa-update mandatory
>
> All the rest is just icing (and the odd non-blocker bug, but who's
> counting ;).

It has been a while since this last update. What is the status of 3.3.0
now?

Warren Togami
wtogami [at] redhat


jm at jmason

Aug 28, 2008, 3:47 AM

Post #13 of 60 (3063 views)
Permalink
Re: 3.3.0 plans [In reply to]

Warren Togami writes:
> Justin Mason wrote:
> > ok, what bugs really need to be fixed for 3.3.0? feel free to set
> > Priority on bugs on the 3.3.0 milestone. in my opinion this is the
> > only real blocker:
> >
> > 5752 2008-02-04 nor P1 Let's get rid of the default rules dir
> > and make sa-update mandatory
> >
> > All the rest is just icing (and the odd non-blocker bug, but who's
> > counting ;).
>
> It has been a while since this last update. What is the status of 3.3.0
> now?

hi Warren --

There's been no real motion -- we've been infrequently bashing the odd bug
in the 3.3.0 list, but we have no concrete release schedule yet. Sorry
about that...

--j.


wtogami at redhat

Apr 13, 2009, 9:02 AM

Post #14 of 60 (2788 views)
Permalink
Re: 3.3.0 plans [In reply to]

On 08/28/2008 06:47 AM, Justin Mason wrote:
> Warren Togami writes:
>> Justin Mason wrote:
>>> ok, what bugs really need to be fixed for 3.3.0? feel free to set
>>> Priority on bugs on the 3.3.0 milestone. in my opinion this is the
>>> only real blocker:
>>>
>>> 5752 2008-02-04 nor P1 Let's get rid of the default rules dir
>>> and make sa-update mandatory
>>>
>>> All the rest is just icing (and the odd non-blocker bug, but who's
>>> counting ;).
>> It has been a while since this last update. What is the status of 3.3.0
>> now?
>
> hi Warren --
>
> There's been no real motion -- we've been infrequently bashing the odd bug
> in the 3.3.0 list, but we have no concrete release schedule yet. Sorry
> about that...
>
> --j.

How are things going now?

Warren Togami
wtogami [at] redhat


mkettler_sa at verizon

Apr 13, 2009, 5:48 PM

Post #15 of 60 (2801 views)
Permalink
Re: 3.3.0 plans [In reply to]

Warren Togami wrote:
> On 08/28/2008 06:47 AM, Justin Mason wrote:
>>
>> hi Warren --
>>
>> There's been no real motion -- we've been infrequently bashing the
>> odd bug
>> in the 3.3.0 list, but we have no concrete release schedule yet. Sorry
>> about that...
>>
>> --j.
>
> How are things going now?
>
> Warren Togami
> wtogami [at] redhat
>
Well, a first hit would be looking at bugzilla


https://issues.apache.org/SpamAssassin/buglist.cgi?bug_file_loc=&bug_file_loc_type=allwordssubstr&bug_id=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bugidtype=include&chfieldfrom=&chfieldto=Now&chfieldvalue=&email1=&email2=&emailtype1=substring&emailtype2=substring&field-1-0-0=target_milestone&field-1-1-0=bug_status&field0-0-0=noop&keywords=&keywords_type=allwords&long_desc=&long_desc_type=allwordssubstr&query_format=advanced&remaction=&short_desc=&short_desc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&target_milestone=3.3.0&type-1-0-0=anyexact&type-1-1-0=anyexact&type0-0-0=noop&value-1-0-0=3.3.0&value-1-1-0=NEW%2CASSIGNED%2CREOPENED&value0-0-0=&order=bugs.priority&query_based_on=

In general we've been a little light on dev effort lately.. perhaps we
need to start rounding up for a 3.3.0 release.


jm at jmason

Apr 14, 2009, 1:50 AM

Post #16 of 60 (2799 views)
Permalink
Re: 3.3.0 plans [In reply to]

On Tue, Apr 14, 2009 at 01:48, Matt Kettler <mkettler_sa [at] verizon> wrote:
> Warren Togami wrote:
>> On 08/28/2008 06:47 AM, Justin Mason wrote:
>>>
>>> hi Warren --
>>>
>>> There's been no real motion -- we've been infrequently bashing the
>>> odd bug
>>> in the 3.3.0 list, but we have no concrete release schedule yet.  Sorry
>>> about that...
>>>
>>> --j.
>>
>> How are things going now?
>>
>> Warren Togami
>> wtogami [at] redhat
>>
> Well, a first hit would be looking at bugzilla
>
>
> https://issues.apache.org/SpamAssassin/buglist.cgi?bug_file_loc=&bug_file_loc_type=allwordssubstr&bug_id=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bugidtype=include&chfieldfrom=&chfieldto=Now&chfieldvalue=&email1=&email2=&emailtype1=substring&emailtype2=substring&field-1-0-0=target_milestone&field-1-1-0=bug_status&field0-0-0=noop&keywords=&keywords_type=allwords&long_desc=&long_desc_type=allwordssubstr&query_format=advanced&remaction=&short_desc=&short_desc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&target_milestone=3.3.0&type-1-0-0=anyexact&type-1-1-0=anyexact&type0-0-0=noop&value-1-0-0=3.3.0&value-1-1-0=NEW%2CASSIGNED%2CREOPENED&value0-0-0=&order=bugs.priority&query_based_on=
>
> In general we've been a little light on dev effort lately.. perhaps we
> need to start rounding up for a 3.3.0 release.

yeah, I think we should.

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752
'Let's get rid of the default rules dir and make sa-update mandatory'

is the only bug that IMO _needs_ to be done. it'd be nice to get the
BerkeleyDB plugin in, too, though, and I'm sure there are a few
others.

we really need to get some tuits together ;)

--j.


Mark.Martinec+sa at ijs

Apr 14, 2009, 4:21 AM

Post #17 of 60 (2782 views)
Permalink
Re: 3.3.0 plans [In reply to]

On Tuesday 14 April 2009 10:50:23 Justin Mason wrote:
> > In general we've been a little light on dev effort lately.. perhaps we
> > need to start rounding up for a 3.3.0 release.
>
> yeah, I think we should.
>
> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752
> 'Let's get rid of the default rules dir and make sa-update mandatory'
>
> is the only bug that IMO _needs_ to be done. it'd be nice to get the
> BerkeleyDB plugin in, too, though, and I'm sure there are a few
> others.
>
> we really need to get some tuits together ;)

I agree it's about time to get 3.3.0 wrapped up. There is some useful new
code there along with a couple of bug fixes, just sitting there. People are
reluctant to use a non-released version, even though I'd say it is just as
stable if not more than 3.2.5.

As new problems are being reported faster than the old ones are being
closed, it would be an illusion to wait for a majority of open tickets
to be closed before a release.

I would too like to see the BerkeleyDB bayes backend to be sorted out
before the release. I made a couple of minor attempts to deal with the
more obvious issues, but it would be nice to get some help from Michael,
its author (or some other interested party), and some experience from
a production use. Bug 6046.

With DKIM ADSP rules I'm a bit stuck because 3.2.5 and 3.3 use the same
rules set (I'd like to add few adsp_override rules, drop now defunct
DKIM_POLICY_SIGNALL, DKIM_POLICY_SIGNSOME, DKIM_POLICY_TESTING;
and DKIM_VERIFIED now replaced by DKIM_VALID and DKIM_VALID_AU).
In principle the 'if version' could be used, but it gets unsightly.
If I remember right, we already heard suggestion (by JM?) to split again
the rules by version. Or maybe there could be a common subset, along
with a per-branch set.

I would like to get a dkim tests operational again. A few testing public keys
need to be published in a spamassassin.org zone (I know, Justin can push
them to DNS, I just need to provide them). This is the main item I need
to work on for 3.3.

Perhaps 3.3 is a good opportunity to make some Perl modules required
or phased out. One example that comes to mind is making Time::HiRes
a requirement - several SA modules are now jumping hoops and perform
suboptimally when they need to deal with integer seconds.

Another is perhaps to replace a Digest::SHA1 (which can only do a sha1)
with a Digest::SHA (which can do a sha1 as well as sha256).
The sha256 is required by a DKIM plugin, and Digest::SHA is upwards
compatible with Digest::SHA1. I can do some benchmarking if there
is a speed concern (sha1 is also used by Bayes).

Perhaps now the DomainKeys plugin could be removed, as its underlying
module is no longer supported, and its functionality is covered by a
DKIM plugin.

Mark


jm at jmason

Apr 14, 2009, 5:07 AM

Post #18 of 60 (2772 views)
Permalink
Re: 3.3.0 plans [In reply to]

On Tue, Apr 14, 2009 at 12:21, Mark Martinec <Mark.Martinec+sa [at] ijs> wrote:
> On Tuesday 14 April 2009 10:50:23 Justin Mason wrote:
>> > In general we've been a little light on dev effort lately.. perhaps we
>> > need to start rounding up for a 3.3.0 release.
>>
>> yeah, I think we should.
>>
>> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752
>> 'Let's get rid of the default rules dir and make sa-update mandatory'
>>
>> is the only bug that IMO _needs_ to be done. it'd be nice to get the
>> BerkeleyDB plugin in, too, though, and I'm sure there are a few
>> others.
>>
>> we really need to get some tuits together ;)
>
> I agree it's about time to get 3.3.0 wrapped up. There is some useful new
> code there along with a couple of bug fixes, just sitting there. People are
> reluctant to use a non-released version, even though I'd say it is just as
> stable if not more than 3.2.5.
>
> As new problems are being reported faster than the old ones are being
> closed, it would be an illusion to wait for a majority of open tickets
> to be closed before a release.

Yes, agreed. BTW, we generally use the Pri field in bugzilla to
determine importance of tickets to handle this issue. Here's a quick
guide to how to triage the 3.3.0 queue in our BZ:

- if a ticket has no patch, is a feature request or similar "I want a
pony" wishful thinking ;), and you think it's almost certainly not
going to be fixed for the release, move it to the "Future" milestone

- if a ticket *needs* to be fixed before release, set severity to "Blocker"

- if it should be fixed before release if possible, move Pri up to "1"

- if a ticket has a patch, or seems fixable for the release, set Pri
to somewhere between 2 and 4 based on your idea of its relative
importance

- if the ticket is a feature req with no suitable patch, set Pri to
"5", where it's probably not going to be fixed but is still "on the
radar".

You can then see the "queue" by searching against the 3.3.0 target milestone
and sorting by Pri.

I expect we'll release 3.3.0 with a lot of tickets in the Pri "3" to
"5" range. ;)


> I would too like to see the BerkeleyDB bayes backend to be sorted out
> before the release. I made a couple of minor attempts to deal with the
> more obvious issues, but it would be nice to get some help from Michael,
> its author (or some other interested party), and some experience from
> a production use. Bug 6046.

is already in that list with Pri=3, which is about right. I hope we
can get this working...

> With DKIM ADSP rules I'm a bit stuck because 3.2.5 and 3.3 use the same
> rules set (I'd like to add few adsp_override rules, drop now defunct
> DKIM_POLICY_SIGNALL, DKIM_POLICY_SIGNSOME, DKIM_POLICY_TESTING;
> and DKIM_VERIFIED now replaced by DKIM_VALID and DKIM_VALID_AU).
> In principle the 'if version' could be used, but it gets unsightly.
> If I remember right, we already heard suggestion (by JM?) to split again
> the rules by version. Or maybe there could be a common subset, along
> with a per-branch set.

I think we should split it again. the attempts to use a common subset of rules
has caused a *lot* of integration/dependency problems IMO. Here's what
I posted over a year ago(!):

'I would also like to get rid of the "rulesrc" external, and instead
just put its contents into each branch, separately. I don't think the
idea of sharing rules in this way between branches has worked out; in my
opinion it's caused more trouble than help (unanticipated dependencies,
complexity, SVN external = horrible anyway). Is anyone still attached to
this idea?'

there were no replies, so that sounds good ;)

> I would like to get a dkim tests operational again. A few testing public keys
> need to be published in a spamassassin.org zone (I know, Justin can push
> them to DNS, I just need to provide them). This is the main item I need
> to work on for 3.3.

is there a bug for that? there probably should be...

> Perhaps 3.3 is a good opportunity to make some Perl modules required
> or phased out. One example that comes to mind is making Time::HiRes
> a requirement - several SA modules are now jumping hoops and perform
> suboptimally when they need to deal with integer seconds.

+1, that seems ok to me. a lot of distros bundle Time::HiRes now.

> Another is perhaps to replace a Digest::SHA1 (which can only do a sha1)
> with a Digest::SHA (which can do a sha1 as well as sha256).
> The sha256 is required by a DKIM plugin, and Digest::SHA is upwards
> compatible with Digest::SHA1. I can do some benchmarking if there
> is a speed concern (sha1 is also used by Bayes).

I'd be more concerned about availability, esp whether or not it's
available in common distro packages (Ubuntu, Debian and Red Hat
particularly).

> Perhaps now the DomainKeys plugin could be removed, as its underlying
> module is no longer supported, and its functionality is covered by a
> DKIM plugin.

+1

--j.


mdorman at ironicdesign

Apr 14, 2009, 6:29 AM

Post #19 of 60 (2779 views)
Permalink
Re: 3.3.0 plans [In reply to]

> I would too like to see the BerkeleyDB bayes backend to be sorted out
> before the release. I made a couple of minor attempts to deal with the
> more obvious issues, but it would be nice to get some help from
> Michael, its author (or some other interested party), and some
> experience from a production use. Bug 6046.

Sorry, I've been swamped by work for the last couple of months.

I haven't had time to pull in your changes from a month or two ago,
which I want to look at, but I have done some work independently.

The solution to the consistency problems you were running into is, as
far as I can discern, to use transactions---which, to my eye, would
be perfectly fine: it slows things down a little bit, but it's still
faster than the full-on SQL back-ends.

Unfortunately, when I do that, I then have problems with BDB
complaining about lack of available transaction slots...even when
there's only ever been one or two transactions in play at one time.

(Oh, and you can only adjust the number of transaction slots using a
DB_CONFIG file, which is a whole other annoyance---that's just part of
the BDB API that isn't exposed in the perl module.)

Perhaps the combination of your changes and mine will make some forward
progress. I'm going to be very busy for the next couple of weeks, but
maybe I can carve out some time to reconcile our changes and start up
some tests.

Mike.


quanah at zimbra

Apr 14, 2009, 9:07 AM

Post #20 of 60 (2767 views)
Permalink
Re: 3.3.0 plans [In reply to]

--On Tuesday, April 14, 2009 9:29 AM -0400 Michael Alan Dorman
<mdorman [at] ironicdesign> wrote:

> Sorry, I've been swamped by work for the last couple of months.
>
> I haven't had time to pull in your changes from a month or two ago,
> which I want to look at, but I have done some work independently.
>
> The solution to the consistency problems you were running into is, as
> far as I can discern, to use transactions---which, to my eye, would
> be perfectly fine: it slows things down a little bit, but it's still
> faster than the full-on SQL back-ends.
>
> Unfortunately, when I do that, I then have problems with BDB
> complaining about lack of available transaction slots...even when
> there's only ever been one or two transactions in play at one time.
>
> (Oh, and you can only adjust the number of transaction slots using a
> DB_CONFIG file, which is a whole other annoyance---that's just part of
> the BDB API that isn't exposed in the perl module.)
>
> Perhaps the combination of your changes and mine will make some forward
> progress. I'm going to be very busy for the next couple of weeks, but
> maybe I can carve out some time to reconcile our changes and start up
> some tests.

I don't know if it would be helpful or not, but you may wish to look at the
OpenLDAP back-bdb and back-hdb backends, which use BDB extensively, and are
highly performant.

--Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration


mdorman at ironicdesign

Apr 14, 2009, 6:53 PM

Post #21 of 60 (2758 views)
Permalink
Re: 3.3.0 plans [In reply to]

> I don't know if it would be helpful or not, but you may wish to look
> at the OpenLDAP back-bdb and back-hdb backends, which use BDB
> extensively, and are highly performant.

It's a good recommendation.

I also read a post just this morning from a linux kernel hacker who's
working with BDB, who fingered RMW as causing problems for him, so I
may check that as well.

Mike.


Mark.Martinec+sa at ijs

Apr 14, 2009, 10:43 PM

Post #22 of 60 (2753 views)
Permalink
Re: 3.3.0 plans [In reply to]

On Wednesday 15 April 2009 03:53:17 Michael Alan Dorman wrote:
> I also read a post just this morning from a linux kernel hacker who's
> working with BDB, who fingered RMW as causing problems for him, so I
> may check that as well.

Btw, I'm using bdb with DB_INIT_CDB | DB_INIT_MPOOL
(cursor locking with Concurrent Data Store) in amavisd,
and it is very reliable. I also keep signals disabled around
each short code section operating under a lock.
No secondary keys in use though.

Mark


Mark.Martinec+sa at ijs

Apr 24, 2009, 1:13 PM

Post #23 of 60 (2481 views)
Permalink
Re: 3.3.0 plans [In reply to]

> > On Tuesday 14 April 2009 10:50:23 Justin Mason wrote:
> >> > In general we've been a little light on dev effort lately.. perhaps we
> >> > need to start rounding up for a 3.3.0 release.
> >>
> >> yeah, I think we should.

> > I agree it's about time to get 3.3.0 wrapped up. There is some useful new
> > code there along with a couple of bug fixes, just sitting there. People
> > are reluctant to use a non-released version, even though I'd say it is
> > just as stable if not more than 3.2.5.
> >
> > As new problems are being reported faster than the old ones are being
> > closed, it would be an illusion to wait for a majority of open tickets
> > to be closed before a release.

> I think we should split it again. the attempts to use a common subset of
> rules has caused a *lot* of integration/dependency problems IMO. Here's
> what I posted over a year ago(!):
>
> 'I would also like to get rid of the "rulesrc" external, and instead
> just put its contents into each branch, separately. I don't think the
> idea of sharing rules in this way between branches has worked out; in my
> opinion it's caused more trouble than help (unanticipated dependencies,
> complexity, SVN external = horrible anyway). Is anyone still attached to
> this idea?'
>
> there were no replies, so that sounds good ;)

Just needs to be done :)

> > I would like to get a dkim tests operational again. A few testing public
> > keys need to be published in a spamassassin.org zone (I know, Justin can
> > push them to DNS, I just need to provide them). This is the main item I
> > need to work on for 3.3.
>
> is there a bug for that? there probably should be...

Bug 6100, now open.
DKIM keys are now ready, test mail samples need to be prepared.

> > Perhaps 3.3 is a good opportunity to make some Perl modules required
> > or phased out. One example that comes to mind is making Time::HiRes
> > a requirement - several SA modules are now jumping hoops and perform
> > suboptimally when they need to deal with integer seconds.
>
> +1, that seems ok to me. a lot of distros bundle Time::HiRes now.

Bug 6095, now resolved.

> > Perhaps now the DomainKeys plugin could be removed, as its underlying
> > module is no longer supported, and its functionality is covered by a
> > DKIM plugin.
>
> +1

Bug 6098, now resolved.


Mark


wtogami at redhat

May 27, 2009, 9:10 PM

Post #24 of 60 (2151 views)
Permalink
Re: 3.3.0 plans [In reply to]

On 04/14/2009 07:21 AM, Mark Martinec wrote:
>
> I agree it's about time to get 3.3.0 wrapped up. There is some useful new
> code there along with a couple of bug fixes, just sitting there. People are
> reluctant to use a non-released version, even though I'd say it is just as
> stable if not more than 3.2.5.
>

Might it be feasible to get 3.3.0 out sometime during the Fedora 12 dev
cycle? It seems there is maybe 2-3 months. A fresh spamassassin making
this release might possibly be particularly more important than other
releases for unnamed reasons...

March 2008 Justin Mason mentions only two major bugs necessary to be
fixed before 3.3.0. Around that time 3.3.0 was considered to be
superior to 3.2.x. Is it still considered better than 3.2.x?

Might it be a good idea to set a target date? Time-based releases tend
to work well for projects. Those stated deadlines tend to be stretched
a bit in reality, but at least goals were set and people work toward
those goals.

Warren Togami
wtogami [at] redhat


jm at jmason

May 28, 2009, 12:48 AM

Post #25 of 60 (2149 views)
Permalink
Re: 3.3.0 plans [In reply to]

On Thu, May 28, 2009 at 05:10, Warren Togami <wtogami [at] redhat> wrote:
> On 04/14/2009 07:21 AM, Mark Martinec wrote:
>>
>> I agree it's about time to get 3.3.0 wrapped up. There is some useful new
>> code there along with a couple of bug fixes, just sitting there. People
>> are
>> reluctant to use a non-released version, even though I'd say it is just as
>> stable if not more than 3.2.5.
>>
>
> Might it be feasible to get 3.3.0 out sometime during the Fedora 12 dev
> cycle?  It seems there is maybe 2-3 months.  A fresh spamassassin making
> this release might possibly be particularly more important than other
> releases for unnamed reasons...
>
> March 2008 Justin Mason mentions only two major bugs necessary to be fixed
> before 3.3.0.  Around that time 3.3.0 was considered to be superior to
> 3.2.x.  Is it still considered better than 3.2.x?
>
> Might it be a good idea to set a target date?  Time-based releases tend to
> work well for projects.  Those stated deadlines tend to be stretched a bit
> in reality, but at least goals were set and people work toward those goals.

It would be quite possible to do this, given a sufficiently-dedicated
release guy pushing
it. It takes about 2 months to get through mass-checks, generate good scores,
etc. Unfortunately, I'm out for this job, as child #2 is about to be born any
day now ;)

BTW in the short term, if ppl want to help 3.3.0 release plans, I suggest trying
out the SVN trunk code as "dogfood". I've been doing this for the last couple
of years with very good results; I can't recall the last time I had to
restart my
spamd, for example, or spotted an FP...

--j.

First page Previous page 1 2 3 Next page Last page  View All SpamAssassin devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.