Mailing List Archive: Maemo: Developers

QA process = bug fixing disincentive?

1 2 3

View All

Index | Next | Previous | View Threaded

andrew at bleb

Oct 31, 2009, 11:43 AM

Post #1 of 54 (2204 views)
Permalink

QA process = bug fixing disincentive?

Hi,

After working 'til stupid o'clock last night on a new version of Hermes, today someone's found a bug which'll impact a small number of people. The fix is trivial.

However, I find myself *not* wanting to fix it as it'll need to go through another round of testing.

Although the principle of resetting package karma to 0 made some sense - as any change could fundamentally break the package - it's also true that a one-line change to a relatively mature package probably won't change whether it's optified; its power usage profile; its package description and icon etc.

Perhaps there's an argument that some proportion of the karma earned should be retained, inversely related to length of time in -testing so far?

Thoughts welcome.

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

a.grandi at gmail

Oct 31, 2009, 11:55 AM

Post #2 of 54 (2127 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Hi,

2009/10/31 Andrew Flegg <andrew [at] bleb>:
> Hi,
>
> After working 'til stupid o'clock last night on a new version of Hermes, today someone's found a bug which'll impact a small number of people. The fix is trivial.
>
> However, I find myself *not* wanting to fix it as it'll need to go through another round of testing.
>
> Although the principle of resetting package karma to 0 made some sense - as any change could fundamentally break the package - it's also true that a one-line change to a relatively mature package probably won't change whether it's optified; its power usage profile; its package description and icon etc.
>
> Perhaps there's an argument that some proportion of the karma earned should be retained, inversely related to length of time in -testing so far?

I strongly agree with you. I've not released anything yet in Maemo
repository but I'll do it soon, I hope the whole experience won't be
too much frustrating :\

By the way, I've upgraded to Hermes 0.2 but I haven't used it yet,
what is the bug you're talking about?

Best regards,

--
Andrea Grandi
email: a.grandi [AT] gmail [DOT] com
website: http://www.andreagrandi.it
PGP Key: http://www.andreagrandi.it/pgp_key.asc
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Oct 31, 2009, 12:00 PM

Post #3 of 54 (2125 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Sat, Oct 31, 2009 at 18:55, Andrea Grandi <a.grandi [at] gmail> wrote:
>
> By the way, I've upgraded to Hermes 0.2 but I haven't used it yet,
> what is the bug you're talking about?

Some Facebook UIDs will now overflow MAXINT, and so I need to store it
in gconf as a long, rather than an int.

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

frank.banul at gmail

Oct 31, 2009, 12:26 PM

Post #4 of 54 (2114 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Hi,

I just threw away 5 karma to make some changes (but I think
worthwhile). I think the idea is that when there's many more users, 10
silly karma points will be nothing. Until then, have faith, or
something like that. :)

Frank

On Sat, Oct 31, 2009 at 1:43 PM, Andrew Flegg <andrew [at] bleb> wrote:
> Hi,
>
> After working 'til stupid o'clock last night on a new version of Hermes, today someone's found a bug which'll impact a small number of people. The fix is trivial.
>
> However, I find myself *not* wanting to fix it as it'll need to go through another round of testing.
>
> Although the principle of resetting package karma to 0 made some sense - as any change could fundamentally break the package - it's also true that a one-line change to a relatively mature package probably won't change whether it's optified; its power usage profile; its package description and icon etc.
>
> Perhaps there's an argument that some proportion of the karma earned should be retained, inversely related to length of time in -testing so far?
>
> Thoughts welcome.
>
> Cheers,
>
> Andrew
>
> --
> Andrew Flegg -- mailto:andrew [at] bleb� |� http://www.bleb.org/
> _______________________________________________
> maemo-developers mailing list
> maemo-developers [at] maemo
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Oct 31, 2009, 12:27 PM

Post #5 of 54 (2116 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Sat, Oct 31, 2009 at 19:26, Frank Banul <frank.banul [at] gmail> wrote:
>
> I just threw away 5 karma to make some changes (but I think
> worthwhile). I think the idea is that when there's many more users, 10
> silly karma points will be nothing. Until then, have faith, or
> something like that. :)

That's the theory, anyway :-)

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

jeremiah at jeremiahfoster

Oct 31, 2009, 12:38 PM

Post #6 of 54 (2113 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Oct 31, 2009, at 20:27, Andrew Flegg wrote:

> On Sat, Oct 31, 2009 at 19:26, Frank Banul <frank.banul [at] gmail>
> wrote:
>>
>> I just threw away 5 karma to make some changes (but I think
>> worthwhile). I think the idea is that when there's many more users,
>> 10
>> silly karma points will be nothing. Until then, have faith, or
>> something like that. :)

I wonder if we can preserve a package's karma if it only gets a
package upgrade. Lets say you only have a little typo in your app, you
fix that, and upload a new package with a new package version. Since
you didn't upload a new _app_ version, your karma is preserved.

This way we allow devs to make small changes and keep karma, make big
changes and bump your app version and start over at zero.

Does that sound worthwhile and implementable?

Jeremiah
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

maemo at csipa

Oct 31, 2009, 1:02 PM

Post #7 of 54 (2117 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Saturday 31 October 2009 19:43:40 Andrew Flegg wrote:
> After working 'til stupid o'clock last night on a new version of Hermes,
> today someone's found a bug which'll impact a small number of people. The
> fix is trivial.
> However, I find myself *not* wanting to fix it as it'll need to go through
> another round of testing.

Yeah, I know the feeling. Both as the person who has the silly UID that causes
problems and as a person who has/had stuff in -testing with 7 karma (collected
over a week) and had to seriously consider whether to let the existing
package go to extras or push the (not significantly altered, with minor fixes
and more user friendly defaults) version and start karma collection from 0.

There is a definitely a conflict there. I support Jeremiah's suggestion that
minor packaging/typo fixes that do not alter app functionality (e.g. when you
go from 1.0-maemo0 to 1.0-maemo1) should not reset app karma. Should require
some discipline so people would not abuse this, but still better than forcing
releases to be spaced 10+ days no matter how large the changes or how simple
the fix.

> Although the principle of resetting package karma to 0 made some sense - as
> any change could fundamentally break the package - it's also true that a
> one-line change to a relatively mature package probably won't change
> whether it's optified; its power usage profile; its package description and
> icon etc.

Yes, there is definitely a sense of throwing out the baby with the bathwater
here - as is, with a sufficiently mature app, NOT applying simple fixes will get
the app to the user quicker, and applying the fixes will keep the app AWAY from
the users.

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

a.grandi at gmail

Oct 31, 2009, 1:06 PM

Post #8 of 54 (2114 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Hi,

2009/10/31 Attila Csipa <maemo [at] csipa>:
> There is a definitely a conflict there. I support Jeremiah's suggestion that
> minor packaging/typo fixes that do not alter app functionality (e.g. when you
> go from 1.0-maemo0 to 1.0-maemo1) should not reset app karma. Should require
> some discipline so people would not abuse this, but still better than forcing
> releases to be spaced 10+ days no matter how large the changes or how simple
> the fix.

even more important: what if developer/users find a security bug that
should really be fixed as soon as possible?
In a normal Linux distribution, patched package is released and
available after few hours. Here it would pass lot of time before final
user can apply the patch.

--
Andrea Grandi
email: a.grandi [AT] gmail [DOT] com
website: http://www.andreagrandi.it
PGP Key: http://www.andreagrandi.it/pgp_key.asc
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

Igor.Stoppa at nokia

Oct 31, 2009, 1:28 PM

Post #9 of 54 (2130 views)
Permalink

RE: QA process = bug fixing disincentive? [In reply to]

Hi,
________________________________________
From: maemo-developers-bounces [at] maemo [maemo-developers-bounces [at] maemo] On Behalf Of ext Andrea Grandi [a.grandi [at] gmail]
Sent: 31 October 2009 22:06
To: Attila Csipa
Cc: maemo-developers [at] maemo
Subject: Re: QA process = bug fixing disincentive?

> even more important: what if developer/users find a security bug that
> should really be fixed as soon as possible?
> In a normal Linux distribution, patched package is released and
> available after few hours. Here it would pass lot of time before final
> user can apply the patch.

I think the problem here is that some braindead system has been introduced,
which doesn't account for the actual work being done.

So, if it's ok that somehow the karma goes down, it should be lowered
accordingly to the severity of the bug found.

Which also means that all the lost karma (plus some more?) should be re-instated
once the bug is fixed and the fix release and verified.

This would actually give an incentive to bugfixing.

igor
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

kees.jongenburger at gmail

Nov 1, 2009, 12:20 AM

Post #10 of 54 (2102 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Hi Andrew and Atilla,

On Sat, Oct 31, 2009 at 9:02 PM, Attila Csipa <maemo [at] csipa> wrote:
> On Saturday 31 October 2009 19:43:40 Andrew Flegg wrote:
>> After working 'til stupid o'clock last night on a new version of Hermes,
>> today someone's found a bug which'll impact a small number of people. The
>> fix is trivial.
>> However, I find myself *not* wanting to fix it as it'll need to go through
>> another round of testing.
>
> There is a definitely a conflict there. I support Jeremiah's suggestion that
> minor packaging/typo fixes that do not alter app functionality (e.g. when you
> go from 1.0-maemo0 to 1.0-maemo1) should not reset app karma. Should require
> some discipline so people would not abuse this, but still better than forcing
> releases to be spaced 10+ days no matter how large the changes or how simple
> the fix.

I fully agree there is a conflict of interest but this gets an event
more interesting problem when you scale up the numbers
imagine 100 app updates on 1.000.000 users on one day. At that point I
start not care anymore about the pain the developer has to go throe to
get is apps delivered. I don't care about the carma. I care about the
users who will get this update. Carma is out of the picture but still
you are pushing a "bugfree" fix to a lot of people and that might
benefit from testing.

Given the :extra-testing week-end" I wonder if the current chosen
scheme will scale up and how this can be improved. For the karma
problem I suggest that the extras testing karma is keps "as is" to
ensure testing happens but to find a solution for published apps otr
possibly even increase karma on republished apps because of "activity
and perseverance" ,

This all doesn't really solve the problem Andrew is facing:
-How will "security" updates be handled?(this is the same kind of
"small but required update")
-Can we use the extras-testing in a more end-user friendly manner
(asking the user on the device if he had problems).
-How do we scale up.

Greetings
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

henrik.hedberg at innologies

Nov 1, 2009, 12:43 AM

Post #11 of 54 (2102 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Igor.Stoppa [at] nokia wrote:

> I think the problem here is that some braindead system has been introduced,
> which doesn't account for the actual work being done.

And what is the biggest mistake here is that the new system has been
put into production before testing it at all.

Someone just came up with an idea of crowd sourcing the QA and used
random generator to set the parameters. Soon it was in real use before
any experiences of its functionaly had got.

I would have understood if the parameters had been really low (say,
1 day and 1 thumbs up) at the beginning, or there would have been a
separate repository (say, extras-selection) to test this new
functionality. The parameters could have been changed according to
success of the system and in parallel with the amount of testers (and
devices available).

The sad contradiction here is that developers are expected to
produce high quality outcomes and their results are put under QA, put
the maemo.org processes and tools are not! Those should have been
tested, for example, with a smaller group of developers before putting
into production system.

The maemo-extras testing marathon did amazing job. Thanks to
everybody who participated! However, it cannot be a permanent way to
overcome one of the biggest problems with the new system.

I am not totally against the new system, and I do recognize the
importance of quality assurance. I just hope that we can learn from the
past and react rapidly. Please, do not refer to the better future and
the possibility to have more users and testers later. Things should work
now!

Here are my suggestions now:

1) Fine tune the parameters: say, 5 days and 5 votes. These can be
changed later when the system is working (has enough testers).

2) Change the system so that user packages that are depended by another
user package are promoted automatically when the actual user package is
promoted (like non-user packages are promoted currently). For example,
when an user is testing Mauku, she is implicitely testing also the
microfeed package [1]).

3) Find a way to overcome the limitations when an upgraded package is an
important security fix.

And here are my older suggestions [2], which, I think, are still valid:

* Negative karma can be given _only_ if it based on the agreed QA
requirements. (Testers are still giving karma based on their subjective
thinking instead of QA requirements.)

* The package page should have a link to a bug tracker and the link must
be used! (Comments are stored into a wrong place currently. It is double
effort for a developer to track the packages interface in addition to
bugs.maemo.org.)

* Negative karma can be given _only_ with a link to a bug tracker having
a bug report about the show stopper. It may be either a new bug report
written by the tester or an old open bug report just referred in the
comment.

* Negative karma is automatically removed when the related bug report is
closed (fixed or other way resolved). (Currently, there is no way to
remove the negative karma (thumbs down) if the bug is fixed. Please,
note that the bug may be in the library code, and the bug in the package
is thus actually fixed when the library is fixed. So, there would not be
any need to update the application package every time.)

BR,

Henrik

P.S. I do not necessarily see that more testers will make things
easier and more workable. The more testers there are, the more
subjective evaluations we will get. If one tester just do not like the
graphics of an application he may give thumbs down, and even four other
testers giving thumbs up are needed to fix that misjudgement. I really
would like to see a discussion about the responsibilities of testers.
Should there be a mechanism to give negative karma to testers who are
not following the QA rules, or even way to ban them?

[1] http://talk.maemo.org/showthread.php?p=362575#post362575

[2]
http://lists.maemo.org/pipermail/maemo-developers/2009-September/020921.html

--
Henrik Hedberg - http://www.henrikhedberg.net/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

martin.grimme at gmail

Nov 1, 2009, 1:36 AM

Post #12 of 54 (2102 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Hi,

resetting Karma on a new version leads to one very bad issue, IMHO:

Developers of packages with some Karma will hold back bugfix-updates
until the unfixed version has reached extras.

This should be avoided.

Martin

2009/11/1, Henrik Hedberg <henrik.hedberg [at] innologies>:
> Igor.Stoppa [at] nokia wrote:
>
>> I think the problem here is that some braindead system has been
>> introduced,
>> which doesn't account for the actual work being done.
>
> And what is the biggest mistake here is that the new system has been
> put into production before testing it at all.
>
> Someone just came up with an idea of crowd sourcing the QA and used
> random generator to set the parameters. Soon it was in real use before
> any experiences of its functionaly had got.
>
> I would have understood if the parameters had been really low (say,
> 1 day and 1 thumbs up) at the beginning, or there would have been a
> separate repository (say, extras-selection) to test this new
> functionality. The parameters could have been changed according to
> success of the system and in parallel with the amount of testers (and
> devices available).
>
> The sad contradiction here is that developers are expected to
> produce high quality outcomes and their results are put under QA, put
> the maemo.org processes and tools are not! Those should have been
> tested, for example, with a smaller group of developers before putting
> into production system.
>
> The maemo-extras testing marathon did amazing job. Thanks to
> everybody who participated! However, it cannot be a permanent way to
> overcome one of the biggest problems with the new system.
>
> I am not totally against the new system, and I do recognize the
> importance of quality assurance. I just hope that we can learn from the
> past and react rapidly. Please, do not refer to the better future and
> the possibility to have more users and testers later. Things should work
> now!
>
> Here are my suggestions now:
>
> 1) Fine tune the parameters: say, 5 days and 5 votes. These can be
> changed later when the system is working (has enough testers).
>
> 2) Change the system so that user packages that are depended by another
> user package are promoted automatically when the actual user package is
> promoted (like non-user packages are promoted currently). For example,
> when an user is testing Mauku, she is implicitely testing also the
> microfeed package [1]).
>
> 3) Find a way to overcome the limitations when an upgraded package is an
> important security fix.
>
> And here are my older suggestions [2], which, I think, are still valid:
>
> * Negative karma can be given _only_ if it based on the agreed QA
> requirements. (Testers are still giving karma based on their subjective
> thinking instead of QA requirements.)
>
> * The package page should have a link to a bug tracker and the link must
> be used! (Comments are stored into a wrong place currently. It is double
> effort for a developer to track the packages interface in addition to
> bugs.maemo.org.)
>
> * Negative karma can be given _only_ with a link to a bug tracker having
> a bug report about the show stopper. It may be either a new bug report
> written by the tester or an old open bug report just referred in the
> comment.
>
> * Negative karma is automatically removed when the related bug report is
> closed (fixed or other way resolved). (Currently, there is no way to
> remove the negative karma (thumbs down) if the bug is fixed. Please,
> note that the bug may be in the library code, and the bug in the package
> is thus actually fixed when the library is fixed. So, there would not be
> any need to update the application package every time.)
>
> BR,
>
> Henrik
>
> P.S. I do not necessarily see that more testers will make things
> easier and more workable. The more testers there are, the more
> subjective evaluations we will get. If one tester just do not like the
> graphics of an application he may give thumbs down, and even four other
> testers giving thumbs up are needed to fix that misjudgement. I really
> would like to see a discussion about the responsibilities of testers.
> Should there be a mechanism to give negative karma to testers who are
> not following the QA rules, or even way to ban them?
>
> [1] http://talk.maemo.org/showthread.php?p=362575#post362575
>
> [2]
> http://lists.maemo.org/pipermail/maemo-developers/2009-September/020921.html
>
> --
> Henrik Hedberg - http://www.henrikhedberg.net/
> _______________________________________________
> maemo-developers mailing list
> maemo-developers [at] maemo
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

henrik.hedberg at innologies

Nov 1, 2009, 2:02 AM

Post #13 of 54 (2095 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Martin Grimme wrote:

> resetting Karma on a new version leads to one very bad issue, IMHO:
>
> Developers of packages with some Karma will hold back bugfix-updates
> until the unfixed version has reached extras.

Guilty as charged.

I have actually postponed the release of Mauku 2.0 beta 5, because I
have been waiting for beta 4 to go through the QA process. The sad thing
is that I am not happy with the beta 4, but I think there should be at
least one version in extras available for ordinary users. So, I just
promoted it into extras.

The beta 5 has been ready over two weeks and I have used it myself.
However, although the beta 4 just hit extras, I am not going to put beta
5 in testing just yet. I will develop it further, and not start the
testing process until I have more features implemented (actually, I
would have introduced those features in beta 6 or beta 7, but now those
features will appear in delayed beta 5).

On the other hand, the current QA process makes sure that you have
to think twice and test internally before you put your package into
testing. This may be a good thing. However, it makes incremental
development very hard or even impossible.

From my viewpoint, the biggest problem is not even the required
amount of karma but the quarantine period, which is way too long. If I
liked to release a new version once a week, the package would never hit
extras.

BR,

Henrik

--
Henrik Hedberg - http://www.henrikhedberg.net/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

jeremiah at jeremiahfoster

Nov 1, 2009, 3:49 PM

Post #14 of 54 (2085 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Nov 1, 2009, at 11:02, Henrik Hedberg wrote:

> Martin Grimme wrote:
>
>> resetting Karma on a new version leads to one very bad issue, IMHO:
>>
>> Developers of packages with some Karma will hold back bugfix-updates
>> until the unfixed version has reached extras.

This is a real problem that will have to be addressed.
>
> On the other hand, the current QA process makes sure that you have
> to think twice and test internally before you put your package into
> testing. This may be a good thing. However, it makes incremental
> development very hard or even impossible.
>
> From my viewpoint, the biggest problem is not even the required
> amount of karma but the quarantine period, which is way too long. If I
> liked to release a new version once a week, the package would never
> hit
> extras.

Ten days is the quarantine period in debian 'NEW', then your app goes
into 'testing' which takes roughly 22 months before it reaches
'stable'. If anything, the maemo quarantine is too short, not too long.

Jeremiah
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

marius.vollmer at nokia

Nov 2, 2009, 12:33 AM

Post #15 of 54 (2080 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

ext Attila Csipa <maemo [at] csipa> writes:

> Yes, there is definitely a sense of throwing out the baby with the bathwater
> here - as is, with a sufficiently mature app, NOT applying simple fixes will get
> the app to the user quicker, and applying the fixes will keep the app AWAY from
> the users.

I am not involved with all of this, so my opinions are very suspect, and
I haven't even read the QA process, but here goes anyway:

Shouldn't it be so that a package collects karma over its whole
lifetime, starting out at zero and then gaining from version to version
as it matures?

A high karma could mean that a package has many users/testers and that
we are confident that bugs are found quickly, that the developers have a
good track record of not screwing up and have demonstrated that they
understand the ins and outs of the Maemo platform (like power
management, package icons, etc), and other things that remain true even
when a package is changed.

The fitness of a concrete version of a package would be expressed as our
"confidence that it contains no critical bugs". That confidence is
proportional to

- time in extras-testing without critical bugs being reported, and

- confidence of the developer that the change doesn't break anything,
weighted by

- the karma that the package/developer has gathered with the previous
versions.

Then, we strike a balance between the confidence of having no critical
bugs and the urgency of the update. Urgent updates such as security
fixes can be risked with lower confidence.

Thus, a package with zero karma and a very confident developer might
still spent 10 days in extras-testing. Once the package has gained 10
karma points, it can leave extras-testing after two days, say, unless
the developer explicitly asks for more test exposure.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

riku.voipio at nokia

Nov 2, 2009, 6:42 AM

Post #16 of 54 (2072 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

ext Jeremiah Foster wrote:
> On Nov 1, 2009, at 11:02, Henrik Hedberg wrote:
>
>
>> Martin Grimme wrote:
>>
>>
>>> resetting Karma on a new version leads to one very bad issue, IMHO:
>>>
>>> Developers of packages with some Karma will hold back bugfix-updates
>>> until the unfixed version has reached extras.
>>>
>
> This is a real problem that will have to be addressed.

What need is a way to split bugfix changes and new major versions.

1) We must encourage developers to provide bug fixes often and be able
to deliver them quickly to endusers
2) New major versions still need QA testing - against regressions. Even
more than bugs endusers hate when things that used to work stopped working.

Now, I think it is impossible to automatically detect if a upload is
bugfix or "major" upgrade. Thus, we have to trust the developer to set
the major-/bugfix upgrade flag correctly somehere ( debian/changelog, a
menu item in the package promotion ui or whatever ).
The question then comes, can we trust the developers to do the right
thing and not abuse the "minor upgrade" to shove in any package with
shorter quarentee and less votes?
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anidel at gmail

Nov 2, 2009, 7:03 AM

Post #17 of 54 (2076 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Yup,

surely the system needs to be fixed.
I have some ideas, I don't like some of the ones already proposed, and
of some of them I do like some pieces, but not others and other pieces
could be improved..

But how do we do it ?

Here?
Talk?
Brainstorm?

I once stated that it's the developer that should have the last word
on whether to promote or not a package.
As Henry did with Mauku, I am doing the same for Xournal. I want it to
extras and then I will release a stupid minor bug fix.
And nothing will make me change my idea.
If there was a button "promote to Extras" I would have hit it already
and a bug fix would already be on its way to testing.

Probably there will never be a solution that makes everyone happy.
So put that button Promote to Extras there from day 0 (as soon as a
package hits Testing) and let the developer choose.
There's always time to pull an app from Extras if it behave wrong.

And, speaking of which, why don't we speak, again, about a system to
rate an application ONCE it's been installed FROM
inside App Manager so that a certain number of negative votes actually
automatically pulls the app out of Extras?

Developer in control (and happy), user in control (and happy as well).

Aniello
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Nov 2, 2009, 7:12 AM

Post #18 of 54 (2077 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Mon, Nov 2, 2009 at 15:03, Aniello Del Sorbo <anidel [at] gmail> wrote:
>
> But how do we do it ?
>
> Here?
> Talk?
> Brainstorm?

I'd say we've had more intelligent thought and discussion here than
I'd expect on this topic on talk; and I don't think the
single-threaded nature of brainstorm with the limited pot of options
is useful for an in-depth technical conversation trying to build
consensus.

> And, speaking of which, why don't we speak, again, about a system to
> rate an application ONCE it's been installed FROM
> inside App Manager so that a certain number of negative votes actually
> automatically pulls the app out of Extras?

Short version: it's not going to happen any time soon, if we want it
to be in the HAM shipped by Nokia.

If we want to build our own app, we need both the infrastructure on
the server and an easy to use app for rating stuff. More doable, and
Stskeeps was suggesting on #maemo that it picks up the app
starting/stopping to nag the user :-)

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrea at borgia

Nov 2, 2009, 7:35 AM

Post #19 of 54 (2070 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Aniello Del Sorbo ha scritto:

> As Henry did with Mauku, I am doing the same for Xournal. I want it to
> extras and then I will release a stupid minor bug fix.
> And nothing will make me change my idea.
> If there was a button "promote to Extras" I would have hit it already
> and a bug fix would already be on its way to testing.

Ideally, it should behave as a pipeline: a new entry in devel shouldn't
reset karma/quarantine for an earlier version in testing. Only when they
happen to be together in testing, you could perhaps automatically clear
the earlier version from the queue and restart counting.

Andrea.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

qole.tablet at gmail

Nov 2, 2009, 11:03 AM

Post #20 of 54 (2065 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

I really like Marius' ideas. A mature, high-karma app should be able to push
bugfixes through the system without all the QA hassles of a new, untested
app. Security fixes on high-karma apps should go straight to extras.

On Mon, Nov 2, 2009 at 7:12 AM, Andrew Flegg <andrew [at] bleb> wrote:

> I'd say we've had more intelligent thought and discussion here than
> I'd expect on this topic on talk

Grrr!

I hate that kind of talk. It only makes the problem worse.

My child is approaching school age, and I'm hearing the same thing all
around me as middle class moms want to keep their precious children away
from the "bad influences" of the poor kids in the local public schools, and
so they send their kids to private schools and impoverish the public schools
even more!

If you aren't part of the solution...

--
enthusiast, n. "One whose mind is wholly possessed and heated by what
engages it; one who is influenced by a peculiar fervor of mind; an ardent
and imaginative person."

andrew at bleb

Nov 2, 2009, 11:48 AM

Post #21 of 54 (2073 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Alan wrote:
> On Mon, Nov 2, 2009 at 7:12 AM, Andrew Flegg <andrew [at] bleb> wrote:
>
> > I'd say we've had more intelligent thought and discussion here than
> > I'd expect on this topic on talk
>
> Grrr!
>
> I hate that kind of talk. It only makes the problem worse.

What problem? This mailing list has a higher concentration of involved (and affected) developers than talk. You can have an intelligent discussion here without it devolving into petty spats or rampaging off topic. This is the official mailing list for the development of (and developers for) Maemo.

It is also easier, as someone strapped for time and wanting to make a positive contribution to Maemo, to spend time thinking of fundamental problems like this with a low-volume of well-formed ideas for debate and consensus. Mailing lists have advantages over fora, and vice versa.

Trying to corale all the discussion to one form and medium is just likely to lead frustration for everyone. This is one of the tenets of good knowledge management: there's no single technology which is perfect for all forms of discussion and storage.

> If you aren't part of the solution...

I think streamlining the process so that high quality software can get into Extras in a timely manner without severely disavowing developers from their "release often, release often" tendencies is more important than whether this should be discussed in the wiki, off a blog post, in a brainstorm or on tmo.

Fortunately, there's enough work for everyone - so please feel free to make sure anyone on tmo who can contribute comes here to do so. I don't want to limit the discussion but why should it be Not Here?

Cheers,

Andrew

PS. I wrote this on the train. There's no Maemo offline client for vBulletin ;-)

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

qole.tablet at gmail

Nov 2, 2009, 12:07 PM

Post #22 of 54 (2081 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

Your reply continues to sound like the middle class moms who argue for
private schools. How will our children ever get ahead if they go to that
school down the street? It is full of common children who will only slow our
gifted children down.

On Mon, Nov 2, 2009 at 11:48 AM, Andrew Flegg <andrew [at] bleb> wrote:

> Alan wrote:
> > On Mon, Nov 2, 2009 at 7:12 AM, Andrew Flegg <andrew [at] bleb> wrote:
> >
> > > I'd say we've had more intelligent thought and discussion here than
> > > I'd expect on this topic on talk
> >
> > Grrr!
> >
> > I hate that kind of talk. It only makes the problem worse.
>
> What problem? This mailing list has a higher concentration of involved (and
> affected) developers than talk. You can have an intelligent discussion here
> without it devolving into petty spats or rampaging off topic. This is the
> official mailing list for the development of (and developers for) Maemo.
>
> It is also easier, as someone strapped for time and wanting to make a
> positive contribution to Maemo, to spend time thinking of fundamental
> problems like this with a low-volume of well-formed ideas for debate and
> consensus. Mailing lists have advantages over fora, and vice versa.
>
> Trying to corale all the discussion to one form and medium is just likely
> to lead frustration for everyone. This is one of the tenets of good
> knowledge management: there's no single technology which is perfect for all
> forms of discussion and storage.
>
> > If you aren't part of the solution...
>
> I think streamlining the process so that high quality software can get into
> Extras in a timely manner without severely disavowing developers from their
> "release often, release often" tendencies is more important than whether
> this should be discussed in the wiki, off a blog post, in a brainstorm or on
> tmo.
>
> Fortunately, there's enough work for everyone - so please feel free to make
> sure anyone on tmo who can contribute comes here to do so. I don't want to
> limit the discussion but why should it be Not Here?
>
> Cheers,
>
> Andrew
>
> PS. I wrote this on the train. There's no Maemo offline client for
> vBulletin ;-)
>
> --
> Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
>
>

--
enthusiast, n. "One whose mind is wholly possessed and heated by what
engages it; one who is influenced by a peculiar fervor of mind; an ardent
and imaginative person."

rabelg5 at gmail

Nov 2, 2009, 12:33 PM

Post #23 of 54 (2064 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Nov 2, 2009, at 3:07 PM, Qole wrote:

> Your reply continues to sound like the middle class moms who argue
> for private schools. How will our children ever get ahead if they go
> to that school down the street? It is full of common children who
> will only slow our gifted children down.

This is a flawed analogy that leans more towards ad hominem than
productive discussion. :)
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

david at dgreaves

Nov 2, 2009, 1:12 PM

Post #24 of 54 (2071 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

> On Mon, Nov 2, 2009 at 11:48 AM, Andrew Flegg <andrew [at] bleb> wrote:
> Alan wrote:
> > Grrr!
> >
> > I hate that kind of talk. It only makes the problem worse.

> What problem? This mailing list has a higher concentration of
> involved (and affected) developers than talk. You can have an
> intelligent discussion here without it devolving into petty
> spats or rampaging off topic. This is the official mailing
> list for the development of (and developers for) Maemo.

[corrected the top-posting]
On Mon, 2009-11-02 at 12:07 -0800, Qole wrote:
> Your reply continues to sound like the middle class moms who argue for
> private schools. How will our children ever get ahead if they go to
> that school down the street? It is full of common children who will
> only slow our gifted children down.

Hmm, your argument sounds like something from talk :)
ie it's not technical, not related to the objective and just invites a
massive off-topic debate; all fine in a casual conversation but not in a
focussed discussion.

I'd suggest that the barrier to entry for a mailing list cf talk is
substantially less than the barrier to entry of a private school.

Anyone from talk is welcome to join the list provided they plan on
engaging in development oriented discussions (and yes, they do meander -
no-one cares *that* much).

David

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

jeremiah at jeremiahfoster

Nov 2, 2009, 2:07 PM

Post #25 of 54 (2070 views)
Permalink

Re: QA process = bug fixing disincentive? [In reply to]

On Nov 2, 2009, at 15:42, Riku Voipio wrote:

> ext Jeremiah Foster wrote:
>> On Nov 1, 2009, at 11:02, Henrik Hedberg wrote:
>>
>>
>>> Martin Grimme wrote:
>>>
>>>
>>>> resetting Karma on a new version leads to one very bad issue, IMHO:
>>>>
>>>> Developers of packages with some Karma will hold back bugfix-
>>>> updates
>>>> until the unfixed version has reached extras.
>>>>
>>
>> This is a real problem that will have to be addressed.
>
> What need is a way to split bugfix changes and new major versions.

I think this is an excellent way to distinguish between between those
apps that need karma reset and those that don't. There are a variety
of mechanisms to do this, all from setting a simple flag somewhere in
the changelog as suggested or even allowing part of the version string
to change and not truncate karma.

To beat the horse dead;

foo_1.0-1maemo0 -> bug fix -> foo_1.0-1maemo1 = All karma retained
foo_1.0-1maemo0 -> feature -> foo_1.1-1maemo0 = Karma set to zero

>
> 1) We must encourage developers to provide bug fixes often and be able
> to deliver them quickly to endusers

+1

> 2) New major versions still need QA testing - against regressions.
> Even
> more than bugs endusers hate when things that used to work stopped
> working.
>
> Now, I think it is impossible to automatically detect if a upload is
> bugfix or "major" upgrade. Thus, we have to trust the developer to set
> the major-/bugfix upgrade flag correctly somewhere ( debian/
> changelog, a
> menu item in the package promotion ui or whatever ).

I think we should be trusting the developers.

> The question then comes, can we trust the developers to do the right
> thing and not abuse the "minor upgrade" to shove in any package with
> shorter quarentee and less votes?

I don't see any other solution than trusting the developer and having
a QA process. If the QA finds a bug, the dev has to fix it. Once it
passes QA, we have to trust that the developer is working with the
system, not against it.

Jeremiah

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

1 2 3

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads