Mailing List Archive: Maemo: Developers

Testing nonsense

Index | Next | Previous | View Threaded

lists at harbaum

Nov 3, 2009, 6:59 AM

Post #1 of 16 (700 views)
Permalink

Testing nonsense

Hi,

there's another problem with the testing i am facing with gpxview: Nonsense ratings.
GPXView got a "thumbs down" for needing lots of porting to match the maemo6 gui.
Yes, harmattan! Why the heck should a fremantle program not be forwarded to
extras due to the fact that it will be hard to port it to qt (which is what that guy
is imho trying to say)?

I am considering to entirely ignore the test process until this testing/promotion thing
has actually proven to be useful. Dealing with people that just rate nonsense issues is
a) a waste of time and b) frustrating.

My proposals:
- Add links to the apps changelogs to the package rating page
- Add a small text telling the people what they are supposed to test (not harmattan
gui portability!!)
- Add a link to the bug tracker, so people can file appropriate bugs which can then
be processed in a useful manner

Till

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

liquid at gmail

Nov 3, 2009, 7:21 AM

Post #2 of 16 (687 views)
Permalink

Re: Testing nonsense [In reply to]

Till,

I totally agree, it is not part of the testing regime itself and as long as
an app is technically capable it passes the test.
the checklist has been defined here:

http://wiki.maemo.org/Extras-testing/QA_Checklist

Maemo 5 offers stock icons covering most regular use cases, but developers
> can use the icons they prefer as long as they respect copyrights. Broken
> icons can be a major bug stopping a release to Extras *but discussions
> about beauty/ugliness of a UI are out of scope in the QA process*.
>

I hope the testing interface can be adjusted to explain the purpose because
I see a lot of ui/feature creep suggestion in the comments.

Whilst the specific apps may not look/feel/perform exactly as we would like
it is wrong to block access because of this.

Use outside methods, discuss patches and changes with the developer for
future versions but we need to work together to get as many stable apps into
extras as possible.

(Incidentally, in the specific example you cite, I think its a typo for the
version)

Gary

On Tue, Nov 3, 2009 at 2:59 PM, Till Harbaum <lists [at] harbaum> wrote:

> Hi,
>
> there's another problem with the testing i am facing with gpxview: Nonsense
> ratings.
> GPXView got a "thumbs down" for needing lots of porting to match the maemo6
> gui.
> Yes, harmattan! Why the heck should a fremantle program not be forwarded to
> extras due to the fact that it will be hard to port it to qt (which is what
> that guy
> is imho trying to say)?
>
> I am considering to entirely ignore the test process until this
> testing/promotion thing
> has actually proven to be useful. Dealing with people that just rate
> nonsense issues is
> a) a waste of time and b) frustrating.
>
> My proposals:
> - Add links to the apps changelogs to the package rating page
> - Add a small text telling the people what they are supposed to test (not
> harmattan
> gui portability!!)
> - Add a link to the bug tracker, so people can file appropriate bugs which
> can then
> be processed in a useful manner
>
> Till
>
> _______________________________________________
> maemo-developers mailing list
> maemo-developers [at] maemo
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>

anidel at gmail

Nov 3, 2009, 7:23 AM

Post #3 of 16 (686 views)
Permalink

Re: Testing nonsense [In reply to]

2009/11/3 Till Harbaum <lists [at] harbaum>:
> Hi,
>
> there's another problem with the testing i am facing with gpxview: Nonsense ratings.
> GPXView got a "thumbs down" for needing lots of porting to match the maemo6 gui.
> Yes, harmattan! Why the heck should a fremantle program not be forwarded to
> extras due to the fact that it will be hard to port it to qt (which is what that guy
> is imho trying to say)?
>
> I am considering to entirely ignore the test process until this testing/promotion thing
> has actually proven to be useful. Dealing with people that just rate nonsense issues is
> a) a waste of time and b) frustrating.
>
> My proposals:
> - Add links to the apps changelogs to the package rating page
> - Add a small text telling the people what they are supposed to test (not harmattan
> gui portability!!)
> - Add a link to the bug tracker, so people can file appropriate bugs which can then
> be processed in a useful manner
>
> Till
>

Well he didn't say he thunbed it down because of what he said in the comment.

Maybe he thumbed it down for a reason and ALSO commented that it'll be
hard to adapt it
to Maemo 6 later on.

--
anidel
Sent from London, Eng, United Kingdom
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

henrik.hedberg at innologies

Nov 3, 2009, 7:32 AM

Post #4 of 16 (687 views)
Permalink

Re: Testing nonsense [In reply to]

Till Harbaum wrote:

> there's another problem with the testing i am facing with gpxview: Nonsense ratings.
> GPXView got a "thumbs down" for needing lots of porting to match the maemo6 gui.
> Yes, harmattan! Why the heck should a fremantle program not be forwarded to
> extras due to the fact that it will be hard to port it to qt (which is what that guy
> is imho trying to say)?
>
> I am considering to entirely ignore the test process until this testing/promotion thing
> has actually proven to be useful. Dealing with people that just rate nonsense issues is
> a) a waste of time and b) frustrating.

In addition, testers - whether they rate nonsense issues or not -
even get positive karma! It feels little unfair. I really would like to
see a discussion about the responsibilities and ethics of a tester, and
possible procedures to make sure that a tester is behaving as expected.

BR,

Henrik

--
Henrik Hedberg - http://www.henrikhedberg.net/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

aklapper at openismus

Nov 3, 2009, 7:49 AM

Post #5 of 16 (687 views)
Permalink

Re: Testing nonsense [In reply to]

Am Dienstag, den 03.11.2009, 15:59 +0100 schrieb Till Harbaum:
> - Add a link to the bug tracker, so people can file appropriate bugs which can then
> be processed in a useful manner

Possible already - add a Bugtracker field to the "control" file if I
remember correctly. Also see
http://wiki.maemo.org/Extras-testing/QA_Checklist#Lack_of_bug_reporting_database

andre
--
Andre Klapper (maemo.org bugmaster)

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

henrik.hedberg at innologies

Nov 3, 2009, 8:04 AM

Post #6 of 16 (687 views)
Permalink

Re: Testing nonsense [In reply to]

> 2009/11/3 Till Harbaum <lists [at] harbaum>:

>> there's another problem with the testing i am facing with gpxview:
Nonsense ratings.
>> GPXView got a "thumbs down" for needing lots of porting to match the
maemo6 gui.
>> Yes, harmattan! Why the heck should a fremantle program not be
forwarded to
>> extras due to the fact that it will be hard to port it to qt (which
is what that guy
>> is imho trying to say)?

Aniello Del Sorbo wrote:

> Well he didn't say he thunbed it down because of what he said in the
comment.
>
> Maybe he thumbed it down for a reason and ALSO commented that it'll be
> hard to adapt it
> to Maemo 6 later on.

The instructions for testers [1] says:

"Whenever you decide to vote an application down, please leave a comment
describing what issues you found. The best feedback is a bug number,
since this allow to track and discuss better the problems. Voting thumbs
down without any explanation doesn't help the developer getting better
software for you and the end users."

There is a long list of blockers ("must") for packages that
developers provide. Why are the instructions for testers just advices
("please", not even "should").

I think it should read:

"Whenever you decide to vote an application down, you MUST leave a
comment describing what issues you found. The best feedback is a bug
number, since this allow to track and discuss better the problems."

or even better (the commenting feature in packages interface is
overlapping thing with bugs.maemo.org):

"Whenever you decide to vote an application down, you MUST provide a
link to a bug report with severity major or higher."

Actually, I read some early postings about the subject (since
April). There were many good ideas (like linking into Bugzilla), but for
some reason we got this separate playground.

BR,

Henrik

[1] http://wiki.maemo.org/Extras-testing#Thumbs_Down

--
Henrik Hedberg - http://www.henrikhedberg.net/

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

ds2 at physik

Nov 3, 2009, 8:05 AM

Post #7 of 16 (687 views)
Permalink

Re: Testing nonsense [In reply to]

I am quite unhappy with the testing, too.

My package vncviewer has a blocking issue (Bugtracker field), which
should be marked on the package page!

http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/vncviewer/0.6.3-fremantle2

Not every developer is following the dev-list all the time. To me it is
not transparent, how to get into extras now. If I get ten thumb ups,
because no tester checked bugtracker it gets in?

Now I prepared a changed debian/control file, but if I upload to testing
I loose my 9 thumb ups. It is quite complicated for a developer to test
this application, as he needs a vnc server to access. Probably this is
the reason, why it takes a lot of time.

On the other hand it is totally to transparent, what is tested.

An other problem are security issues: How do you think a tester could
find a security issue in such an application? It is totaly impossible,
if you do not have access to a prepared vnc server. Should we assume the
vnc server to be prepared? Should we warn a user that prepared vnc
servers are not tested, so only use thrusted ones, or ...

I have the feeling, that the process is quite slow, without being much
better than having less testers.

OK, enough for now:-)

Am Dienstag, den 03.11.2009, 17:32 +0200 schrieb Henrik Hedberg:
> Till Harbaum wrote:
>
> > there's another problem with the testing i am facing with gpxview: Nonsense ratings.
> > GPXView got a "thumbs down" for needing lots of porting to match the maemo6 gui.
> > Yes, harmattan! Why the heck should a fremantle program not be forwarded to
> > extras due to the fact that it will be hard to port it to qt (which is what that guy
> > is imho trying to say)?
> >
> > I am considering to entirely ignore the test process until this testing/promotion thing
> > has actually proven to be useful. Dealing with people that just rate nonsense issues is
> > a) a waste of time and b) frustrating.
>
> In addition, testers - whether they rate nonsense issues or not -
> even get positive karma! It feels little unfair. I really would like to
> see a discussion about the responsibilities and ethics of a tester, and
> possible procedures to make sure that a tester is behaving as expected.
>
> BR,
>
> Henrik
>

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

aklapper at openismus

Nov 3, 2009, 8:17 AM

Post #8 of 16 (686 views)
Permalink

Re: Testing nonsense [In reply to]

Am Dienstag, den 03.11.2009, 17:05 +0100 schrieb ds:
> An other problem are security issues: How do you think a tester could
> find a security issue in such an application? It is totaly impossible,
> if you do not have access to a prepared vnc server. Should we assume the
> vnc server to be prepared? Should we warn a user that prepared vnc
> servers are not tested, so only use thrusted ones, or ...

This has been discussed here already a few days ago.
See for example
http://lists.maemo.org/pipermail/maemo-developers/2009-October/021861.html

andre
--
Andre Klapper (maemo.org bugmaster)

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

murrayc at murrayc

Nov 3, 2009, 8:34 AM

Post #9 of 16 (669 views)
Permalink

Re: Testing nonsense [In reply to]

On Tue, 2009-11-03 at 15:59 +0100, Till Harbaum wrote:
> there's another problem with the testing i am facing with gpxview:
> Nonsense ratings.
> GPXView got a "thumbs down" for needing lots of porting to match the
> maemo6 gui.
> Yes, harmattan! Why the heck should a fremantle program not be
> forwarded to
> extras due to the fact that it will be hard to port it to qt (which is
> what that guy
> is imho trying to say)?

It was a typo. I meant maemo 5.

--
murrayc [at] murrayc
www.murrayc.com
www.openismus.com

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

murrayc at murrayc

Nov 3, 2009, 8:47 AM

Post #10 of 16 (668 views)
Permalink

Re: Testing nonsense [In reply to]

On Tue, 2009-11-03 at 15:21 +0000, Gary Birkett wrote:
>
> I totally agree, it is not part of the testing regime itself and as
> long as
> an app is technically capable it passes the test.
> the checklist has been defined here:
>
> http://wiki.maemo.org/Extras-testing/QA_Checklist

Assuming that you are talking about ignoring UI issues, rather than just
being outraged (outraged!!) that I apparently wanted the app to be ready
for Maemo _6_:

I don't claim to know what the aims of the testing are. But I did see
this on the main testing page:
http://wiki.maemo.org/Extras-testing#The_extras-testing_QA_queue_.26_you
"
Offering good quality community software to owners of Maemo devices is a
top priority. We have a chance to show the world that open source
software developed by community projects can match commercial software
in terms of features, usability, reliability and fun. But we also face
the risk of getting maemo.org Extras associated with beta quality
software made by geeks for geeks only, without the last mile of
polishing.
"

So I've been
a) assuming that an application can't be impressive if if doesn't use
Maemo 5 widgets and generally follow the Maemo 5 UI layout.
b) assuming that the application maintainer would like some feedback.

(Note that the wiki is generally verbose, fragmented, repetitive and
hard to navigate around. That's not because it's a wiki.)

--
murrayc [at] murrayc
www.murrayc.com
www.openismus.com

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anidel at gmail

Nov 3, 2009, 8:56 AM

Post #11 of 16 (666 views)
Permalink

Re: Testing nonsense [In reply to]

2009/11/3 Murray Cumming <murrayc [at] murrayc>:
>
> I don't claim to know what the aims of the testing are. But I did see
> this on the main testing page:
> http://wiki.maemo.org/Extras-testing#The_extras-testing_QA_queue_.26_you
> "
> Offering good quality community software to owners of Maemo devices is a
> top priority. We have a chance to show the world that open source
> software developed by community projects can match commercial software
> in terms of features, usability, reliability and fun. But we also face
> the risk of getting maemo.org Extras associated with beta quality
> software made by geeks for geeks only, without the last mile of
> polishing.
> "
>

I missed this point while reading it.
And it convinced me to push to Extras Testing a new release of Xournal
no matter if it loses the 7 thumbs up it already got (plus the 3 it
got for a previous version)

:(

Oh well.. at least it'll be as much bug free as possible when it'll
finally land Extras :)

Thanks.

--
anidel
Sent from London, Eng, United Kingdom
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Nov 3, 2009, 9:02 AM

Post #12 of 16 (667 views)
Permalink

Re: Testing nonsense [In reply to]

On Tue, Nov 3, 2009 at 16:56, Aniello Del Sorbo <anidel [at] gmail> wrote:
>>
>
> I missed this point while reading it.
> And it convinced me to push to Extras Testing a new release of Xournal
> no matter if it loses the 7 thumbs up it already got (plus the 3 it
> got for a previous version)

I also think that the delay for new features is a good thing now, it
gives me at least 10 days whilst Hermes goes on its way to Extras of
waiting and not developing until 1am in the morning :-)

10 days off: woohoo!

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

rabelg5 at gmail

Nov 3, 2009, 9:58 AM

Post #13 of 16 (669 views)
Permalink

Re: Testing nonsense [In reply to]

On Nov 3, 2009, at 9:59 AM, Till Harbaum wrote:

> <snip/>

I'd just like to interject that any new process like this is going to
have growing pains. You have two options to deal with the kinks that
inevitably appear in an untried process, help to smooth them out (See
the "QA process = bug fixing disincentive?" thread) or drop into rant-
mode and succeed mostly in irritating and polarizing people while
doing little to assist with improving the process.

I see a lot of people picking the later method right now and find it a
bit disheartening.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

khertan at khertan

Nov 7, 2009, 4:08 AM

Post #14 of 16 (540 views)
Permalink

Re: Testing nonsense [In reply to]

>"Oh well.. at least it'll be as much bug free as possible when it'll
>finally land Extras :)"

If it could land it one day ...

--
Benoît HERVIER - http://khertan.net/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

thomas at mich

Nov 8, 2009, 5:18 PM

Post #15 of 16 (499 views)
Permalink

Re: Testing nonsense [In reply to]

I think having a bugzilla where thumbs-down == unresolved bug might
help. If the UI has an actual problem it should be reported, but
"invalid" or "wontfix" (yes, I know it could be abused) or any other
"resolved should negate any negative karma.

Also it should start at a -3 or something reasonable for applications
which were approved under Diablo as they aren't new. Or maybe -5 or
something for the first, but if it is an update it should not require
10 thumbs up.

(I also noted broad appeal applications can easily get 10, but
narrower ones might have trouble - it might be easier to require 75%
or 80% positive with a small quorum instead - geometric instead of
arithmetic).
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

vivainio at gmail

Nov 9, 2009, 3:30 AM

Post #16 of 16 (490 views)
Permalink

Re: Testing nonsense [In reply to]

On Tue, Nov 3, 2009 at 6:47 PM, Murray Cumming <murrayc [at] murrayc> wrote:

> in terms of features, usability, reliability and fun. But we also face
> the risk of getting maemo.org Extras associated with beta quality
> software made by geeks for geeks only, without the last mile of
> polishing.

Possibly off-topic, but what's the current recommended place for
software without that last mile of polishing (but with prospect of
getting moved to extras after widespread field use)?

Extras-devel isn't it, as it brings in library updates etc
(subsequently bricking your device). I'm thinking of something like
Ubuntu PPA's, that build the applications against the current stable
libraries.

--
Ville M. Vainio
http://tinyurl.com/vainio
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com