Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Gentoo: Dev

About tests needing internet connection to run

 

 

Gentoo dev RSS feed   Index | Next | Previous | View Threaded


pacho at gentoo

Jul 7, 2012, 5:38 AM

Post #1 of 5 (197 views)
Permalink
About tests needing internet connection to run

After reading:
https://bugs.gentoo.org/show_bug.cgi?id=424719
https://bugs.gentoo.org/show_bug.cgi?id=397973

Looks like there is not consensus about how to handle this cases,
probably a PROPERTIES variable for this would help :-/

Any ideas on this kind of issue?
Attachments: signature.asc (0.19 KB)


kentfredric at gmail

Jul 7, 2012, 7:33 AM

Post #2 of 5 (189 views)
Permalink
Re: About tests needing internet connection to run [In reply to]

On 8 July 2012 00:38, Pacho Ramos <pacho [at] gentoo> wrote:
> After reading:
> https://bugs.gentoo.org/show_bug.cgi?id=424719
> https://bugs.gentoo.org/show_bug.cgi?id=397973
>
> Looks like there is not consensus about how to handle this cases,
> probably a PROPERTIES variable for this would help :-/
>
> Any ideas on this kind of issue?

I've been handling it on the perl-experimental overlay by specifying
SRC_TEST="network"

SRC_TEST is a perl-module.eclass variable that controls weather or not
to run the packages inbuilt tests.

Its not anywhere in official capacity, but I think my approach is
sane-ish somewhat, just needs better native support in my opinion.

https://github.com/kentfredric/perl-experimental/blob/eclass-moretests/eclass/perl-module.eclass#L283

I think it would be nice to mask packages by their test failure
expectancy, for instance, mask packages that the tests are known to
fail on, or have tests turned on for all packages except packages
where failures are expected from tests. ( There are a few packages
which will always fail tests apparently, and it would be nice to
indicate as such in the ebuild ).

This way you can also probably opt for:
a) installing only packages which don't require network for their tests
b) only testing packages which don't require network for their tests

I've also thought it might be nice to have a way to enable testing
every time I install a ~amd64 package, instead of having a wide
spectrum "all or nothing" approach.





--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz


mgorny at gentoo

Jul 7, 2012, 8:21 AM

Post #3 of 5 (186 views)
Permalink
Re: About tests needing internet connection to run [In reply to]

On Sat, 07 Jul 2012 14:38:59 +0200
Pacho Ramos <pacho [at] gentoo> wrote:

> After reading:
> https://bugs.gentoo.org/show_bug.cgi?id=424719
> https://bugs.gentoo.org/show_bug.cgi?id=397973
>
> Looks like there is not consensus about how to handle this cases,
> probably a PROPERTIES variable for this would help :-/
>
> Any ideas on this kind of issue?

To be honest, I think the first thing to do would be fixing the test
suites to skip tests which fail due to internet connection being
unavailable. Well, there would still be question how to reliably
determine that...

--
Best regards,
Michał Górny
Attachments: signature.asc (0.31 KB)


tetromino at gentoo

Jul 7, 2012, 2:46 PM

Post #4 of 5 (183 views)
Permalink
Re: About tests needing internet connection to run [In reply to]

On Sat, Jul 7, 2012 at 11:21 AM, Michał Górny <mgorny [at] gentoo> wrote:
> To be honest, I think the first thing to do would be fixing the test
> suites to skip tests which fail due to internet connection being
> unavailable. Well, there would still be question how to reliably
> determine that...

For some packages, e.g. geocode-glib, which is basically a library for
calling a particular web service from C code, running the test suite
without network access is almost pointless. (Unless, of course, you
feel like implementing a clone of that web service just to run the
test suite.)

I don't like tests that need network access, but in a few cases, they
are the only to automatically verify that a package works.

-Alexandre.


mgorny at gentoo

Jul 8, 2012, 12:39 AM

Post #5 of 5 (176 views)
Permalink
Re: About tests needing internet connection to run [In reply to]

On Sat, 7 Jul 2012 17:46:49 -0400
Alexandre Rostovtsev <tetromino [at] gentoo> wrote:

> On Sat, Jul 7, 2012 at 11:21 AM, Michał Górny <mgorny [at] gentoo>
> wrote:
> > To be honest, I think the first thing to do would be fixing the test
> > suites to skip tests which fail due to internet connection being
> > unavailable. Well, there would still be question how to reliably
> > determine that...
>
> For some packages, e.g. geocode-glib, which is basically a library for
> calling a particular web service from C code, running the test suite
> without network access is almost pointless. (Unless, of course, you
> feel like implementing a clone of that web service just to run the
> test suite.)
>
> I don't like tests that need network access, but in a few cases, they
> are the only to automatically verify that a package works.

And 'skipped' tests simply mean that the test suite was unable to
verify whether the package works for one reason or another. Well, other
than build-time failures and a few possible runtime failures.

You just have to ensure that it correctly notices the difference
between 'no internet' and 'no matching API there'. Probably the domain
resolution failure should be the borderline.

Well, and I don't really mind having PROPERTIES about it. Some users
may actually want to know that tests could do better with internet
access.

--
Best regards,
Michał Górny
Attachments: signature.asc (0.31 KB)

Gentoo dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.