[PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[Carlos O'Donell]

DJ,

Given that we creat $(complocaledir) now we can cleanup an existing test
and remove the xmkdirp for support_complocaledir_prefix.

OK for master?

8< --- 8< --- 8<
From aee0b1425d57dfb3769b192a79d1d14cf02616f1 Mon Sep 17 00:00:00 2001
From: Carlos O'Donell <carlos@redhat.com>
Date: Tue, 28 Apr 2020 15:46:00 -0400
Subject: [PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

We automatically create $(complocaledir) in the testroot.root
now and so we don't need to create it in the test.
---
 locale/tst-localedef-path-norm.c | 41 +++++++++++++++-----------------
 1 file changed, 19 insertions(+), 22 deletions(-)

diff --git a/locale/tst-localedef-path-norm.c b/locale/tst-localedef-path-norm.c
index 0a77262484..5b81a2828d 100644
--- a/locale/tst-localedef-path-norm.c
+++ b/locale/tst-localedef-path-norm.c
@@ -92,15 +92,12 @@ do_test (void)
   /* We are running as root inside the container.  */
   prog = xasprintf ("%s/localedef", support_bindir_prefix);
 
-  /* Create the needed directories:
-     - We need the default compiled locale dir for default output.
-     - We need an arbitrary absolute path for localedef output.
-
-     Note: Writing to a non-default absolute path disables any kind
+  /* We need an arbitrary absolute path for localedef output.
+     Writing to a non-default absolute path disables any kind
      of path normalization since we expect the user wants the path
      exactly as they specified it.  */
-  xmkdirp (support_complocaledir_prefix, 0777);
-  xmkdirp ("/output", 0777);
+#define ABSDIR "/output"
+  xmkdirp (ABSDIR, 0777);
 
   /* It takes ~10 seconds to serially execute 9 localedef test.  We
      could run the compilations in parallel if we want to reduce test
@@ -109,7 +106,7 @@ do_test (void)
      tests saves disk space during testing.  */
 
   /* Test 1: Expected normalization.
-     Run localedef and expect output in /usr/lib/locale/en_US1.utf8,
+     Run localedef and expect output in $(complocaledir)/en_US1.utf8,
      with normalization changing UTF-8 to utf8.  */
   run_test ((struct test_closure)
 	    {
@@ -123,7 +120,7 @@ do_test (void)
 	    });
 
   /* Test 2: No normalization past '@'.
-     Run localedef and expect output in /usr/lib/locale/en_US2.utf8@tEsT,
+     Run localedef and expect output in $(complocaledir)/en_US2.utf8@tEsT,
      with normalization changing UTF-8@tEsT to utf8@tEsT (everything after
      @ is untouched).  */
   run_test ((struct test_closure)
@@ -138,7 +135,7 @@ do_test (void)
 	    });
 
   /* Test 3: No normalization past '@' despite period.
-     Run localedef and expect output in /usr/lib/locale/en_US3@tEsT.UTF-8,
+     Run localedef and expect output in $(complocaledir)/en_US3@tEsT.UTF-8,
      with normalization changing nothing (everything after @ is untouched)
      despite there being a period near the end.  */
   run_test ((struct test_closure)
@@ -153,7 +150,7 @@ do_test (void)
 	    });
 
   /* Test 4: Normalize numeric codeset by adding 'iso' prefix.
-     Run localedef and expect output in /usr/lib/locale/en_US4.88591,
+     Run localedef and expect output in $(complocaledir)/en_US4.88591,
      with normalization changing 88591 to iso88591.  */
   run_test ((struct test_closure)
 	    {
@@ -167,7 +164,7 @@ do_test (void)
 	    });
 
   /* Test 5: Don't add 'iso' prefix if first char is alpha.
-     Run localedef and expect output in /usr/lib/locale/en_US5.a88591,
+     Run localedef and expect output in $(complocaledir)/en_US5.a88591,
      with normalization changing nothing.  */
   run_test ((struct test_closure)
 	    {
@@ -181,7 +178,7 @@ do_test (void)
 	    });
 
   /* Test 6: Don't add 'iso' prefix if last char is alpha.
-     Run localedef and expect output in /usr/lib/locale/en_US6.88591a,
+     Run localedef and expect output in $(complocaledir)/en_US6.88591a,
      with normalization changing nothing.  */
   run_test ((struct test_closure)
 	    {
@@ -195,7 +192,7 @@ do_test (void)
 	    });
 
   /* Test 7: Don't normalize anything with an absolute path.
-     Run localedef and expect output in /output/en_US7.UTF-8,
+     Run localedef and expect output in ABSDIR/en_US7.UTF-8,
      with normalization changing nothing.  */
   run_test ((struct test_closure)
 	    {
@@ -203,13 +200,13 @@ do_test (void)
 			"--no-archive",
 			"-i", "en_US",
 			"-f", "UTF-8",
-			"/output/en_US7.UTF-8", NULL },
+			ABSDIR "/en_US7.UTF-8", NULL },
 	      .exp = "en_US7.UTF-8",
-	      .complocaledir = "/output"
+	      .complocaledir = ABSDIR
 	    });
 
   /* Test 8: Don't normalize anything with an absolute path.
-     Run localedef and expect output in /output/en_US8.UTF-8@tEsT,
+     Run localedef and expect output in ABSDIR/en_US8.UTF-8@tEsT,
      with normalization changing nothing.  */
   run_test ((struct test_closure)
 	    {
@@ -217,13 +214,13 @@ do_test (void)
 			"--no-archive",
 			"-i", "en_US",
 			"-f", "UTF-8",
-			"/output/en_US8.UTF-8@tEsT", NULL },
+			ABSDIR "/en_US8.UTF-8@tEsT", NULL },
 	      .exp = "en_US8.UTF-8@tEsT",
-	      .complocaledir = "/output"
+	      .complocaledir = ABSDIR
 	    });
 
   /* Test 9: Don't normalize anything with an absolute path.
-     Run localedef and expect output in /output/en_US9@tEsT.UTF-8,
+     Run localedef and expect output in ABSDIR/en_US9@tEsT.UTF-8,
      with normalization changing nothing.  */
   run_test ((struct test_closure)
 	    {
@@ -231,9 +228,9 @@ do_test (void)
 			"--no-archive",
 			"-i", "en_US",
 			"-f", "UTF-8",
-			"/output/en_US9@tEsT.UTF-8", NULL },
+			ABSDIR "/en_US9@tEsT.UTF-8", NULL },
 	      .exp = "en_US9@tEsT.UTF-8",
-	      .complocaledir = "/output"
+	      .complocaledir = ABSDIR
 	    });
 
   return 0;
-- 
2.21.1


-- 
Cheers,
Carlos.

Re: [PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[DJ Delorie]

"Carlos O'Donell" <carlos@redhat.com> writes:
> Given that we creat $(complocaledir) now we can cleanup an existing test
> and remove the xmkdirp for support_complocaledir_prefix.
>
> OK for master?

LGTM

Reviewed-by: DJ Delorie <dj@redhat.com>

> -  /* Create the needed directories:
> -     - We need the default compiled locale dir for default output.
> -     - We need an arbitrary absolute path for localedef output.
> -
> -     Note: Writing to a non-default absolute path disables any kind
> +  /* We need an arbitrary absolute path for localedef output.
> +     Writing to a non-default absolute path disables any kind

ok

>       exactly as they specified it.  */
> -  xmkdirp (support_complocaledir_prefix, 0777);

Makefile does this for us. Ok.

> -  xmkdirp ("/output", 0777);
> +#define ABSDIR "/output"
> +  xmkdirp (ABSDIR, 0777);

Same function, but parameterized.  Ok.  The rest of the patch "just"
uses this macro.

>  
>    /* Test 1: Expected normalization.
> -     Run localedef and expect output in /usr/lib/locale/en_US1.utf8,
> +     Run localedef and expect output in $(complocaledir)/en_US1.utf8,

Ok.

>    /* Test 2: No normalization past '@'.
> -     Run localedef and expect output in /usr/lib/locale/en_US2.utf8@tEsT,
> +     Run localedef and expect output in $(complocaledir)/en_US2.utf8@tEsT,

Ok^2

>    /* Test 3: No normalization past '@' despite period.
> -     Run localedef and expect output in /usr/lib/locale/en_US3@tEsT.UTF-8,
> +     Run localedef and expect output in $(complocaledir)/en_US3@tEsT.UTF-8,

Ok^3

>    /* Test 4: Normalize numeric codeset by adding 'iso' prefix.
> -     Run localedef and expect output in /usr/lib/locale/en_US4.88591,
> +     Run localedef and expect output in $(complocaledir)/en_US4.88591,

Ok^4

>    /* Test 5: Don't add 'iso' prefix if first char is alpha.
> -     Run localedef and expect output in /usr/lib/locale/en_US5.a88591,
> +     Run localedef and expect output in $(complocaledir)/en_US5.a88591,

Ok^5

>    /* Test 6: Don't add 'iso' prefix if last char is alpha.
> -     Run localedef and expect output in /usr/lib/locale/en_US6.88591a,
> +     Run localedef and expect output in $(complocaledir)/en_US6.88591a,

Ok^6

>    /* Test 7: Don't normalize anything with an absolute path.
> -     Run localedef and expect output in /output/en_US7.UTF-8,
> +     Run localedef and expect output in ABSDIR/en_US7.UTF-8,

Ok^7

>  			"-f", "UTF-8",
> -			"/output/en_US7.UTF-8", NULL },
> +			ABSDIR "/en_US7.UTF-8", NULL },

Ok.

>  	      .exp = "en_US7.UTF-8",
> -	      .complocaledir = "/output"
> +	      .complocaledir = ABSDIR

Ok.

>    /* Test 8: Don't normalize anything with an absolute path.
> -     Run localedef and expect output in /output/en_US8.UTF-8@tEsT,
> +     Run localedef and expect output in ABSDIR/en_US8.UTF-8@tEsT,

Ok^8

>  			"-f", "UTF-8",
> -			"/output/en_US8.UTF-8@tEsT", NULL },
> +			ABSDIR "/en_US8.UTF-8@tEsT", NULL },

Ok.

>  	      .exp = "en_US8.UTF-8@tEsT",
> -	      .complocaledir = "/output"
> +	      .complocaledir = ABSDIR

Ok.

>    /* Test 9: Don't normalize anything with an absolute path.
> -     Run localedef and expect output in /output/en_US9@tEsT.UTF-8,
> +     Run localedef and expect output in ABSDIR/en_US9@tEsT.UTF-8,

Ok^9

>  			"-f", "UTF-8",
> -			"/output/en_US9@tEsT.UTF-8", NULL },
> +			ABSDIR "/en_US9@tEsT.UTF-8", NULL },

Ok.

>  	      .exp = "en_US9@tEsT.UTF-8",
> -	      .complocaledir = "/output"
> +	      .complocaledir = ABSDIR

Ok.

Re: [PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[Carlos O'Donell]

On 4/30/20 3:31 PM, DJ Delorie wrote:
> "Carlos O'Donell" <carlos@redhat.com> writes:
>> Given that we creat $(complocaledir) now we can cleanup an existing test
>> and remove the xmkdirp for support_complocaledir_prefix.
>>
>> OK for master?
> 
> LGTM
> 
> Reviewed-by: DJ Delorie <dj@redhat.com>

Pushed as #4 in sequence.

-- 
Cheers,
Carlos.

Re: [PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[H.J. Lu]

On Thu, Apr 30, 2020 at 2:18 PM Carlos O'Donell via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> On 4/30/20 3:31 PM, DJ Delorie wrote:
> > "Carlos O'Donell" <carlos@redhat.com> writes:
> >> Given that we creat $(complocaledir) now we can cleanup an existing test
> >> and remove the xmkdirp for support_complocaledir_prefix.
> >>
> >> OK for master?
> >
> > LGTM
> >
> > Reviewed-by: DJ Delorie <dj@redhat.com>
>
> Pushed as #4 in sequence.
>

Does this cause

https://sourceware.org/bugzilla/show_bug.cgi?id=25908

-- 
H.J.

Re: [PATCH] locale/tst-localedef-path-norm: Don't create $(complocaledir)

[Carlos O'Donell]

On 5/2/20 2:15 PM, H.J. Lu wrote:
> On Thu, Apr 30, 2020 at 2:18 PM Carlos O'Donell via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
>>
>> On 4/30/20 3:31 PM, DJ Delorie wrote:
>>> "Carlos O'Donell" <carlos@redhat.com> writes:
>>>> Given that we creat $(complocaledir) now we can cleanup an existing test
>>>> and remove the xmkdirp for support_complocaledir_prefix.
>>>>
>>>> OK for master?
>>>
>>> LGTM
>>>
>>> Reviewed-by: DJ Delorie <dj@redhat.com>
>>
>> Pushed as #4 in sequence.
>>
> 
> Does this cause
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=25908

The testroot.root is created from testroot.pristine via a two step
process:

* all containerized tests depend on Makefile:$(objpfx)testroot.pristine/install.stamp
  - We are forced to create testroot.pristine/ first and that creates
    the $(complocaledir) directory also.

* all containerized tests rsync testroot.pristine into testroot.root to run from.
  - This is done internally by support/test-container.c.

Makefile:
583 $(tests-container) $(addsuffix /tests,$(subdirs)) : \
584                 $(objpfx)testroot.pristine/install.stamp
585 $(objpfx)testroot.pristine/install.stamp :
586         test -d $(objpfx)testroot.pristine || \
587           mkdir $(objpfx)testroot.pristine
588         # We need a working /bin/sh for some of the tests.
589         test -d $(objpfx)testroot.pristine/bin || \
590           mkdir $(objpfx)testroot.pristine/bin
591         # We need the compiled locale dir for localedef tests.
592         test -d $(objpfx)testroot.pristine/$(complocaledir) || \
593           mkdir -p $(objpfx)testroot.pristine/$(complocaledir)

By the time we run any container test we would have created the directory,
and so the test should see the created directory.

The new requirement for the $(complocaledir) invalides old testroot.pristine,
so you cannot incrementally build through this change. We do not have rootfs
dependencies that are fine-grained enough for this. You have to start from
a clean build or this will cause this test to fail. Did you start from a
clean build?

The alternative is that I would have to duplicate or move all install path
creation logic from Makefile to support/test-container.c, and stat them
unconditionally at each test (which is a waste). We could have some kind of
rootfs revision number we bump if we make incompatible layout changes to
force incremental build/check to work. That seems like overkill.

Can you reproduce this with a clean build/check?

-- 
Cheers,
Carlos.