[PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Florian Weimer]

We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.

2017-11-13  Florian Weimer  <fweimer@redhat.com>

	[BZ #20826]
	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
	* posix/Makefile (tests): Remove tst-getaddrinfo5.
	(xtests): Add tst-getaddrinfo5.

diff --git a/posix/Makefile b/posix/Makefile
index 532f2c73c5..8b3632a3ba 100644
--- a/posix/Makefile
+++ b/posix/Makefile
@@ -91,7 +91,7 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 		   bug-getopt1 bug-getopt2 bug-getopt3 bug-getopt4 \
 		   bug-getopt5 tst-getopt_long1 bug-regex34 bug-regex35 \
 		   tst-pathconf tst-getaddrinfo4 tst-rxspencer-no-utf8 \
-		   tst-fnmatch3 bug-regex36 tst-getaddrinfo5 \
+		   tst-fnmatch3 bug-regex36 \
 		   tst-posix_spawn-fd tst-posix_spawn-setsid \
 		   tst-posix_fadvise tst-posix_fadvise64 \
 		   tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \
@@ -99,7 +99,7 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 tests-internal	:= bug-regex5 bug-regex20 bug-regex33 \
 		   tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \
 		   tst-glob_lstat_compat
-xtests		:= bug-ga2
+xtests		:= bug-ga2 tst-getaddrinfo5
 ifeq (yes,$(build-shared))
 test-srcs	:= globtest
 tests           += wordexp-test tst-exec tst-spawn tst-spawn2 tst-spawn3

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Aurelien Jarno]

On 2017-11-13 14:22, Florian Weimer wrote:
> We have additional coverage of getaddrinfo through the resolv tests, so
> the loss of default test coverage seems acceptable.
> 
> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
> 
> 	[BZ #20826]
> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
> 	(xtests): Add tst-getaddrinfo5.
> 

IMHO, this test should succeed even without network access, provided
that /etc/hosts contains an entry for localhost, which is something
pretty standard. However nss_files only consider exact matching, and
doesn't consider 'localhost.' being the same as 'localhost'.

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Carlos O'Donell]

On 11/14/2017 12:17 PM, Aurelien Jarno wrote:
> On 2017-11-13 14:22, Florian Weimer wrote:
>> We have additional coverage of getaddrinfo through the resolv tests, so
>> the loss of default test coverage seems acceptable.
>>
>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>
>> 	[BZ #20826]
>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>> 	(xtests): Add tst-getaddrinfo5.
>>
> 
> IMHO, this test should succeed even without network access, provided
> that /etc/hosts contains an entry for localhost, which is something
> pretty standard. However nss_files only consider exact matching, and
> doesn't consider 'localhost.' being the same as 'localhost'.
 
We will eventually run this test in a newtork namespace with a known
configured hosts file. Until that point I think it needs to remain an
xtest. We should not rely on the host being configured in any magical
way.

DJ is currently looking at running tests in a minimal namespace to
allow us to do exactly this kind of testing. If you have any interest
in that please reach out to DJ.

-- 
Cheers,
Carlos.

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Andreas Schwab]

On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:

> We have additional coverage of getaddrinfo through the resolv tests, so
> the loss of default test coverage seems acceptable.
>
> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>
> 	[BZ #20826]
> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
> 	(xtests): Add tst-getaddrinfo5.

tst-getaddrinfo4 has the same problem.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Florian Weimer]

On 11/15/2017 08:54 AM, Andreas Schwab wrote:
> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
> 
>> We have additional coverage of getaddrinfo through the resolv tests, so
>> the loss of default test coverage seems acceptable.
>>
>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>
>> 	[BZ #20826]
>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>> 	(xtests): Add tst-getaddrinfo5.
> 
> tst-getaddrinfo4 has the same problem.

Should we turn both into xtests?

Thanks,
Florian

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Andreas Schwab]

On Nov 15 2017, Florian Weimer <fweimer@redhat.com> wrote:

> On 11/15/2017 08:54 AM, Andreas Schwab wrote:
>> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
>>
>>> We have additional coverage of getaddrinfo through the resolv tests, so
>>> the loss of default test coverage seems acceptable.
>>>
>>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>>
>>> 	[BZ #20826]
>>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>>> 	(xtests): Add tst-getaddrinfo5.
>>
>> tst-getaddrinfo4 has the same problem.
>
> Should we turn both into xtests?

Either both or neither.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Florian Weimer]

[-- Attachment #1: Type: text/plain, Size: 789 bytes --]

On 11/15/2017 09:30 AM, Andreas Schwab wrote:
> On Nov 15 2017, Florian Weimer <fweimer@redhat.com> wrote:
> 
>> On 11/15/2017 08:54 AM, Andreas Schwab wrote:
>>> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
>>>
>>>> We have additional coverage of getaddrinfo through the resolv tests, so
>>>> the loss of default test coverage seems acceptable.
>>>>
>>>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>>>
>>>> 	[BZ #20826]
>>>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>>>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>>>> 	(xtests): Add tst-getaddrinfo5.
>>>
>>> tst-getaddrinfo4 has the same problem.
>>
>> Should we turn both into xtests?
> 
> Either both or neither.

Fair point.  I'm attaching an updated patch.

Thanks,
Florian

[-- Attachment #2: gai-xtests.patch --]
[-- Type: text/x-patch, Size: 1599 bytes --]


These tests need a working Internet connection with DNS.

We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.

2017-11-15  Florian Weimer  <fweimer@redhat.com>

	[BZ #20826]
	Turn posix/tst-getaddrinfo4, posix/tst-getaddrinfo5 into xtests
	due to Internet requirement.
	* posix/Makefile (tests): Remove tst-getaddrinfo4,
	tst-getaddrinfo5.
	(xtests): Add tst-getaddrinfo4, tst-getaddrinfo5.

diff --git a/posix/Makefile b/posix/Makefile
index 532f2c73c5..87e7ed7f9d 100644
--- a/posix/Makefile
+++ b/posix/Makefile
@@ -90,8 +90,8 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 		   tst-getaddrinfo3 tst-fnmatch2 tst-cpucount tst-cpuset \
 		   bug-getopt1 bug-getopt2 bug-getopt3 bug-getopt4 \
 		   bug-getopt5 tst-getopt_long1 bug-regex34 bug-regex35 \
-		   tst-pathconf tst-getaddrinfo4 tst-rxspencer-no-utf8 \
-		   tst-fnmatch3 bug-regex36 tst-getaddrinfo5 \
+		   tst-pathconf tst-rxspencer-no-utf8 \
+		   tst-fnmatch3 bug-regex36 \
 		   tst-posix_spawn-fd tst-posix_spawn-setsid \
 		   tst-posix_fadvise tst-posix_fadvise64 \
 		   tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \
@@ -99,7 +99,7 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 tests-internal	:= bug-regex5 bug-regex20 bug-regex33 \
 		   tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \
 		   tst-glob_lstat_compat
-xtests		:= bug-ga2
+xtests		:= bug-ga2 tst-getaddrinfo4 tst-getaddrinfo5
 ifeq (yes,$(build-shared))
 test-srcs	:= globtest
 tests           += wordexp-test tst-exec tst-spawn tst-spawn2 tst-spawn3

Re: [PATCH] posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

[Florian Weimer]

On 11/15/2017 11:29 AM, Florian Weimer wrote:
> 2017-11-15  Florian Weimer<fweimer@redhat.com>
> 
> 	[BZ #20826]
> 	Turn posix/tst-getaddrinfo4, posix/tst-getaddrinfo5 into xtests
> 	due to Internet requirement.
> 	* posix/Makefile (tests): Remove tst-getaddrinfo4,
> 	tst-getaddrinfo5.
> 	(xtests): Add tst-getaddrinfo4, tst-getaddrinfo5.

Any objections?  I'm going to commit this soon.

Thanks,
Florian