Mailing List Archive: Quagga: Dev

Quagga 0.99.21 freeze

Index | Next | Previous | View Threaded

equinox at diac24

Apr 17, 2012, 7:33 AM

Post #1 of 8 (265 views)
Permalink

Quagga 0.99.21 freeze

Hi guys,

there's now a freeze_0.99.21 tag on the master git. This is meant to
imply that we'd like to release this as 0.99.21 and people that have
testbeds are welcome to test the version.

Until the 0.99.21 release is made, only high-importance and reasonably
simple bugfixes & regression fixes will be merged.

A git tree very similar to this is alrady passing through large-scale
testing at OpenSourceRouting.org's facilities, therefore the release
will probably appear after a shortened testing period of about a week
or so.

Cheers,

David

Attachments:

signature.asc (0.22 KB)

thomas.r.henderson at boeing

Apr 26, 2012, 8:58 AM

Post #2 of 8 (226 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

> -----Original Message-----
> From: quagga-dev-bounces [at] lists [mailto:quagga-dev-
> bounces [at] lists] On Behalf Of David Lamparter
> Sent: Tuesday, April 17, 2012 7:33 AM
> To: quagga-dev [at] lists
> Subject: [quagga-dev 9196] Quagga 0.99.21 freeze
>
> Hi guys,
>
>
> there's now a freeze_0.99.21 tag on the master git. This is meant to
> imply that we'd like to release this as 0.99.21 and people that have
> testbeds are welcome to test the version.
>
> Until the 0.99.21 release is made, only high-importance and reasonably
> simple bugfixes & regression fixes will be merged.
>
> A git tree very similar to this is alrady passing through large-scale
> testing at OpenSourceRouting.org's facilities, therefore the release
> will probably appear after a shortened testing period of about a week
> or so.
>

David,
If you don't mind, I had a few questions about this pending release.

- can you clarify which bugs (if any) are being worked before the release? Are they in the tracker? I've noticed the persistence of a lot of bugs tagged 'blocker' that do not seem to block releases.

- what protocols are being tested, how are they being tested, and by whom? Is opensourcerouting.org or anyone else going to report back to the list on what tests have passed and failed (or even on what platforms/compilers it builds)?

- are you distributing any release notes that list the changes since the last release? I tend to see these in Paul's eventual release announcements but they do not seem to be maintained in the codebase. Would the project be interested in maintaining a top-level RELEASE_NOTES?

My sense is that you will release 0.99.21 some day in the near future unless anyone seriously complains, based on best-effort testing, but I'm wondering whether the project is applying any other criteria to making the release.

Thanks,
Tom

p.s. the URL in the HACKING.tex document http://wiki.quagga.net/index.php/Main/Processes is no longer resolvable and the page doesn't seem to exist in the sourceforge wiki.

_______________________________________________
Quagga-dev mailing list
Quagga-dev [at] lists
http://lists.quagga.net/mailman/listinfo/quagga-dev

mwinter at opensourcerouting

Apr 26, 2012, 10:03 AM

Post #3 of 8 (231 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

Thomas,

On Apr 26, 2012, at 8:58 AM, Henderson, Thomas R wrote:
>> [...]
>> there's now a freeze_0.99.21 tag on the master git. This is meant to
>> imply that we'd like to release this as 0.99.21 and people that have
>> testbeds are welcome to test the version.
>>
>> Until the 0.99.21 release is made, only high-importance and reasonably
>> simple bugfixes & regression fixes will be merged.
>>
>> A git tree very similar to this is alrady passing through large-scale
>> testing at OpenSourceRouting.org's facilities, therefore the release
>> will probably appear after a shortened testing period of about a week
>> or so.
>
> David,
> If you don't mind, I had a few questions about this pending release.

let me answer part of the questions

> - can you clarify which bugs (if any) are being worked before the release? Are they in the tracker? I've noticed the persistence of a lot of bugs tagged 'blocker' that do not seem to block releases.
>
> - what protocols are being tested, how are they being tested, and by whom?

We (opensourcerouting.org) are testing the following protocols:
RIPv1, RIPv2, RIPng, OSPFv2, OSPFv3, ISIS, BGP (ipv4 & ipv6)
(but with the limitation that all the IPv6 protocols are less thorough tested at this time - mainly because of our resources. We hope to have more intensive testing added very soon)

For testing coverage, we have 3 categories for our tests:
1) Protocol Compliance - We are using a commercial product (Ixia ANVL) here.
2) Protocol Fuzzer - Using a commercial prodocut (MU Dynamics) here.
3) Scale & Performance - Using our own set of tests.

For details, go to www.opensourcerouting.org/wiki/Testing+Efforts

Part 3 (Scale & Performance) is still lagging behind with implementation, but the testplan is online and if anyone has any suggestions to add there, then I very WELCOME any feedback.

We hope that other people run their own tests, but I can't talk for them. If someone else here does large tests and wants to coordinate anything then let me know.

> Is opensourcerouting.org or anyone else going to report back to the list on what tests have passed and failed (or even on what platforms/compilers it builds)?

We will before (or together) with the release. I just need to make it better readable (ie If I tell you that ANVL Test OSPF-15.2 fails, then this is useless for most of you until I give a decent description on the test).
We (at this time) don't file all the bugs in bugzilla as I still need to spend a lot of time to verify if actually Quagga is broken or just the test fails (because the test is incorrect) or the test is buggy. I don't want to flood bugzilla with lots of bugs which are not yet verified if they are real. But I filed some in the past and will file more after I verified them.

My goal would be to include a summary with failures in some release notes and detailed spreadsheets with the coverage on our wiki. I hope to have the initial spreadsheets online soon.

For some quick summary (only bug count), look at the slides from the talk I gave at the RIPE 64 meeting last week in Slovenia:
www.opensourcerouting.org/wiki/Presentations
Keep in mind that failure counts in the presentation means "test failed". It's not yet verified (for all) if this is a Quagga bug (see above)

- Martin Winter
(OpenSourceRouting.org)

> - are you distributing any release notes that list the changes since the last release? I tend to see these in Paul's eventual release announcements but they do not seem to be maintained in the codebase. Would the project be interested in maintaining a top-level RELEASE_NOTES?
>
> My sense is that you will release 0.99.21 some day in the near future unless anyone seriously complains, based on best-effort testing, but I'm wondering whether the project is applying any other criteria to making the release.
>
> Thanks,
> Tom
>
> p.s. the URL in the HACKING.tex document http://wiki.quagga.net/index.php/Main/Processes is no longer resolvable and the page doesn't seem to exist in the sourceforge wiki.
>
> _______________________________________________
> Quagga-dev mailing list
> Quagga-dev [at] lists
> http://lists.quagga.net/mailman/listinfo/quagga-dev

_______________________________________________
Quagga-dev mailing list
Quagga-dev [at] lists
http://lists.quagga.net/mailman/listinfo/quagga-dev

equinox at diac24

Apr 26, 2012, 11:27 AM

Post #4 of 8 (223 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

(answering the ones Martin didn't respond to)

On Thu, Apr 26, 2012 at 08:58:42AM -0700, Henderson, Thomas R wrote:
> - can you clarify which bugs (if any) are being worked before the
> release? Are they in the tracker? I've noticed the persistence of a
> lot of bugs tagged 'blocker' that do not seem to block releases.

While all of us contribute code on more or less random topics, the view
as a maintainer is primarily that of reviewing and integrating
submissions. As such, we don't have control over what gets fixed when.

Obviously we can and do write code, but our motivations for doing so
are, at the end of the day, defined by what we or our employers need.

We make releases when we feel that the codebase is at a point of
relative stability (i.e. larger changes have completed, no big open
holes that were not there before). Ideally, a new release is always
better than any older release. The "blocker" bug marking is probably
more important to an end user for now...

> - are you distributing any release notes that list the changes since
> the last release? I tend to see these in Paul's eventual release
> announcements but they do not seem to be maintained in the codebase.
> Would the project be interested in maintaining a top-level
> RELEASE_NOTES?

Hm. The idea is that you can look at the git subjects. I guess we
could collect rough summaries for the delta between releases?

-David

Attachments:

signature.asc (0.22 KB)

gdt at ir

Apr 26, 2012, 12:33 PM

Post #5 of 8 (221 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

> - are you distributing any release notes that list the changes since
> the last release? I tend to see these in Paul's eventual release
> announcements but they do not seem to be maintained in the codebase.
> Would the project be interested in maintaining a top-level
> RELEASE_NOTES?

Hm. The idea is that you can look at the git subjects. I guess we
could collect rough summaries for the delta between releases?

It would be good to maintain a NEWS file, in the gnu tradition, which is
a brief summary of user-facing changes (not bugfixes, but new features
and significant changes in behavior).

thomas.r.henderson at boeing

Apr 26, 2012, 12:46 PM

Post #6 of 8 (229 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

> -----Original Message-----
> From: quagga-dev-bounces [at] lists [mailto:quagga-dev-
> bounces [at] lists] On Behalf Of David Lamparter
> Sent: Thursday, April 26, 2012 11:27 AM
> To: Quagga development list
> Subject: [quagga-dev 9219] Re: Quagga 0.99.21 freeze
>
> (answering the ones Martin didn't respond to)
>
> On Thu, Apr 26, 2012 at 08:58:42AM -0700, Henderson, Thomas R wrote:
> > - can you clarify which bugs (if any) are being worked before the
> > release? Are they in the tracker? I've noticed the persistence of a
> > lot of bugs tagged 'blocker' that do not seem to block releases.
>
> While all of us contribute code on more or less random topics, the view
> as a maintainer is primarily that of reviewing and integrating
> submissions. As such, we don't have control over what gets fixed when.
>
> Obviously we can and do write code, but our motivations for doing so
> are, at the end of the day, defined by what we or our employers need.

I fully understand; however, I was mainly probing for more information about your post from last month [1] about migrating to new development processes. The criteria to move from 'start of release window' to 'release' did not seem to be well defined. On other projects, there is often a release manager who is tasked to deliver a release that supports new features without adding regressions, and sometimes there are blocker bugs that delay the release, and typically a projected day that the release will be made is announced (to help testers to know what date they have to get their tests done by), etc. I was wondering if anything of that sort applied now or would apply in the future.

>
> We make releases when we feel that the codebase is at a point of
> relative stability (i.e. larger changes have completed, no big open
> holes that were not there before). Ideally, a new release is always
> better than any older release. The "blocker" bug marking is probably
> more important to an end user for now...

that is good to know, thanks. Sometimes it is vague in trackers whether severity and priority fields in tracker are for users or maintainers to set. I'm not sure whether quagga provides guidance to bug reporters on how to set those fields.

>
> > - are you distributing any release notes that list the changes since
> > the last release? I tend to see these in Paul's eventual release
> > announcements but they do not seem to be maintained in the codebase.
> > Would the project be interested in maintaining a top-level
> > RELEASE_NOTES?
>
> Hm. The idea is that you can look at the git subjects. I guess we
> could collect rough summaries for the delta between releases?

A very verbose example of a project that does this is gcc:
http://gcc.gnu.org/gcc-4.2/changes.html

However, many projects just maintain a RELEASE_NOTES text file with the code. Besides announcing new features, this also helps to clearly identify API or behavioral changes to users, and to list what are the known supported platforms for each release.

- Tom

[1] http://lists.quagga.net/pipermail/quagga-dev/2012-March/009142.html

_______________________________________________
Quagga-dev mailing list
Quagga-dev [at] lists
http://lists.quagga.net/mailman/listinfo/quagga-dev

equinox at diac24

Apr 28, 2012, 1:05 PM

Post #7 of 8 (210 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

On Thu, Apr 26, 2012 at 12:46:55PM -0700, Henderson, Thomas R wrote:
> I fully understand; however, I was mainly probing for more information
> about your post from last month [1] about migrating to new development
> processes. The criteria to move from 'start of release window' to
> 'release' did not seem to be well defined. On other projects, there
> is often a release manager who is tasked to deliver a release that
> supports new features without adding regressions, and sometimes there
> are blocker bugs that delay the release, and typically a projected day
> that the release will be made is announced (to help testers to know
> what date they have to get their tests done by), etc. I was wondering
> if anything of that sort applied now or would apply in the future.

I guess, yes, with the testing from the OSR, these test results are
input for the release decision. Releases that add known regressions are
not acceptable; before releasing a version with new regressions there
will always be backing out the change that caused said regression.

Any test coverage can pretty much never be all-encompassing, so we're
working with whatever feedback we can get. If you depend on certain
things, feel free to cook up a small test setup yourself. We'll happily
take the input!

A projected day for the release to be made - well, there's a certain aim
at around 2 weeks after freeze, but no idea how much value to put on
that.

(I'll write a separate mail about 0.99.21 release status in a few
minutes)

> > Hm. The idea is that you can look at the git subjects. I guess we
> > could collect rough summaries for the delta between releases?
>
> A very verbose example of a project that does this is gcc:
> http://gcc.gnu.org/gcc-4.2/changes.html
>
> However, many projects just maintain a RELEASE_NOTES text file with
> the code. Besides announcing new features, this also helps to clearly
> identify API or behavioral changes to users, and to list what are the
> known supported platforms for each release.

Let's see what I can cook up for 0.99.21.

-David

Attachments:

signature.asc (0.22 KB)

equinox at diac24

Apr 28, 2012, 1:13 PM

Post #8 of 8 (211 views)
Permalink

Re: Quagga 0.99.21 freeze [In reply to]

On Tue, Apr 17, 2012 at 04:33:04PM +0200, David Lamparter wrote:
> there's now a freeze_0.99.21 tag on the master git. This is meant to
> imply that we'd like to release this as 0.99.21 and people that have
> testbeds are welcome to test the version.
>
> Until the 0.99.21 release is made, only high-importance and reasonably
> simple bugfixes & regression fixes will be merged.

Current status:

2 patches on savannah
2 more patches incoming (both build fixes to babeld/ picked from SRE)
1 bgpd issue (very odd-looking compiler warnings)
1 test suite issue (code not compiling / testsuite not updated)

ETA for 0.99.21 release is Tuesday next week, I'd say. Treat that date
as "crystal ball" statement please ;)

Cheers,

-David

Attachments:

signature.asc (0.22 KB)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads