[PATCH v2 0/2] Update Makefile fragments for DSO sorting tests

public inbox for libc-alpha@sourceware.org
 help / color / mirror / Atom feed

* [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests
@ 2021-11-08 15:42 H.J. Lu
  2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: H.J. Lu @ 2021-11-08 15:42 UTC (permalink / raw)
  To: libc-alpha; +Cc: Florian Weimer

One problem of using

$(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)

to build DSO sorting relocatable objects is that $< may not be the source
file with 3 "make -j 28" parallel builds on a machine with 112 cores.
When generating Makefile fragments at build time, the same file may be
generated more than once at the same time with parallel builds.

1. Update dso-ordering-test.py to put all sources for DSO sorting tests
in a single directory, dso-sort-tests-src, and compile relocatable objects
with

$(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
	$(compile.c) $(OUTPUT_OPTION)

2. Generate Makefile fragments for DSO sorting tests at configure time
to avoid generate them in the elf directory at build time.

H.J. Lu (2):
  dso-ordering-test.py: Put all sources in one directory [BZ #28550]
  elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]

 configure                    | 11 +++++++++
 configure.ac                 | 12 +++++++++
 elf/Makefile                 | 22 ++++-------------
 scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
 4 files changed, 62 insertions(+), 30 deletions(-)

-- 
2.33.1

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550]
  2021-11-08 15:42 [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
@ 2021-11-08 15:42 ` H.J. Lu
  2021-11-11 17:52   ` Chung-Lin Tang
  2021-11-15 18:50   ` Adhemerval Zanella
  2021-11-08 15:42 ` [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests " H.J. Lu
  2021-11-11 13:46 ` [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
  2 siblings, 2 replies; 15+ messages in thread
From: H.J. Lu @ 2021-11-08 15:42 UTC (permalink / raw)
  To: libc-alpha; +Cc: Florian Weimer

Put all sources for DSO sorting tests in the dso-sort-tests-src directory
and compile test relocatable objects with

$(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
	$(compile.c) $(OUTPUT_OPTION)

to avoid random $< values from $(before-compile) when compiling test
relocatable objects with

$(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)

for 3 "make -j 28" parallel builds on a machine with 112 cores at the
same time.

This partially fixes BZ #28550.
---
 scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/scripts/dso-ordering-test.py b/scripts/dso-ordering-test.py
index 944ee74052..bde0406be9 100644
--- a/scripts/dso-ordering-test.py
+++ b/scripts/dso-ordering-test.py
@@ -526,9 +526,13 @@ def process_testcase(t):
     base_test_name = t.test_name
     test_subdir = base_test_name + "-dir"
     testpfx = objpfx + test_subdir + "/"
+    test_srcdir = "dso-sort-tests-src/"
+    testpfx_src = objpfx + test_srcdir
 
     if not os.path.exists(testpfx):
         os.mkdir(testpfx)
+    if not os.path.exists(testpfx_src):
+        os.mkdir(testpfx_src)
 
     def find_objs_not_depended_on(t):
         objs_not_depended_on = []
@@ -595,6 +599,11 @@ def process_testcase(t):
         # Print out needed Makefile fragments for use in glibc/elf/Makefile.
         module_names = ""
         for o in test_descr.objs:
+            rule = ("$(objpfx)" + test_subdir + "/" + test_name
+                    + "-" + o + ".os: $(objpfx)" + test_srcdir
+                    + test_name + "-" + o + ".c\n"
+                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
+            makefile.write (rule)
             module_names += " " + test_subdir + "/" + test_name + "-" + o
         makefile.write("modules-names +=%s\n" % (module_names))
 
@@ -637,7 +646,7 @@ def process_testcase(t):
                         # object.  This only needs to be done at most once for
                         # an object name.
                         if not dep in fake_created:
-                            f = open(testpfx + test_name + "-" + dep
+                            f = open(testpfx_src + test_name + "-" + dep
                                      + ".FAKE.c", "w")
                             f.write(" \n")
                             f.close()
@@ -648,6 +657,12 @@ def process_testcase(t):
                                  % (test_name + "-" + dep + ".FAKE.so",
                                     ("$(objpfx)" + test_subdir + "/"
                                      + test_name + "-" + dep + ".so")))
+                            rule = ("$(objpfx)" + test_subdir + "/"
+                                    + test_name + "-" + dep + ".FAKE.os: "
+                                    "$(objpfx)" + test_srcdir
+                                    + test_name + "-" + dep + ".FAKE.c\n"
+                                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
+                            makefile.write (rule)
                             makefile.write \
                                 ("modules-names += %s\n"
                                  % (test_subdir + "/"
@@ -687,6 +702,10 @@ def process_testcase(t):
                       + test_descr.soname_map['#'] + ".so")
             ldflags += (" -Wl,-soname=" + soname)
         makefile.write("LDFLAGS-%s = %s\n" % (test_name, ldflags))
+        rule = ("$(objpfx)" + test_subdir + "/" + test_name + ".o: "
+                "$(objpfx)" + test_srcdir + test_name + ".c\n"
+                "\t$(compile.c) $(OUTPUT_OPTION)\n")
+        makefile.write (rule)
 
         not_depended_objs = find_objs_not_depended_on(test_descr)
         if not_depended_objs:
@@ -745,7 +764,7 @@ def process_testcase(t):
                      "  something_failed=true\n"
                      "else\n"
                      "  diff -wu ${common_objpfx}elf/%s/%s%s.output \\\n"
-                     "           ${common_objpfx}elf/%s/%s%s.exp\n"
+                     "           ${common_objpfx}elf/%s%s%s.exp\n"
                      "  if [ $? -ne 0 ]; then\n"
                      "    echo '%sFAIL: %s%s expected output comparison'\n"
                      "    something_failed=true\n"
@@ -753,14 +772,14 @@ def process_testcase(t):
                      "fi\n"
                      % (("X" if xfail else ""), test_name, tunable_descr,
                         test_subdir, test_name, tunable_sfx,
-                        test_subdir, base_test_name, exp_tunable_sfx,
+                        test_srcdir, base_test_name, exp_tunable_sfx,
                         ("X" if xfail else ""), test_name, tunable_descr))
 
         # Generate C files according to dependency and calling relations from
         # description string.
         for obj in test_descr.objs:
             src_name = test_name + "-" + obj + ".c"
-            f = open(testpfx + src_name, "w")
+            f = open(testpfx_src + src_name, "w")
             if obj in test_descr.callrefs:
                 called_objs = test_descr.callrefs[obj]
                 for callee in called_objs:
@@ -804,7 +823,7 @@ def process_testcase(t):
             f.close()
 
         # Open C file for writing main program
-        f = open(testpfx + test_name + ".c", "w")
+        f = open(testpfx_src + test_name + ".c", "w")
 
         # if there are some operations in main(), it means we need -ldl
         f.write("#include <stdio.h>\n")
@@ -885,7 +904,7 @@ def process_testcase(t):
             for obj in test_descr.objs:
                 src_name = test_name + "-" + obj + ".c"
                 obj_name = test_name + "-" + obj + ".os"
-                run_cmd([build_gcc, "-c", "-fPIC", testpfx + src_name,
+                run_cmd([build_gcc, "-c", "-fPIC", testpfx_src + src_name,
                           "-o", testpfx + obj_name])
 
             obj_processed = {}
@@ -903,10 +922,12 @@ def process_testcase(t):
                             deps.append(dep + ".FAKE")
                             if not dep in fake_created:
                                 base_name = testpfx + test_name + "-" + dep
+                                src_base_name = (testpfx_src + test_name
+                                                 + "-" + dep)
                                 cmd = [build_gcc, "-Wl,--no-as-needed",
                                        ("-Wl,-soname=" + base_name + ".so"),
                                        "-shared", base_name + ".FAKE.c",
-                                       "-o", base_name + ".FAKE.so"]
+                                       "-o", src_base_name + ".FAKE.so"]
                                 run_cmd(cmd)
                                 fake_created[dep] = True
                 dso_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
@@ -932,7 +953,7 @@ def process_testcase(t):
             main_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
                             deps)
             cmd = [build_gcc, "-Wl,--no-as-needed", "-o", testpfx + test_name,
-                   testpfx + test_name + ".c", "-L%s" % (os.getcwd()),
+                   testpfx_src + test_name + ".c", "-L%s" % (os.getcwd()),
                    "-Wl,-rpath-link=%s" % (os.getcwd())]
             if '#' in test_descr.soname_map:
                 soname = ("-Wl,-soname=" + testpfx + test_name + "-"
@@ -987,14 +1008,14 @@ def process_testcase(t):
         sfx = ""
         if r[0] != "":
             sfx = "-" + r[0].replace("=","_")
-        f = open(testpfx + t.test_name + sfx + ".exp", "w")
+        f = open(testpfx_src + t.test_name + sfx + ".exp", "w")
         (output, xfail) = r[1]
         f.write('%s' % output)
         f.close()
 
     # Create header part of top-level testcase shell script, to wrap execution
     # and output comparison together.
-    t.sh = open(testpfx + t.test_name + ".sh", "w")
+    t.sh = open(testpfx_src + t.test_name + ".sh", "w")
     t.sh.write("#!/bin/sh\n")
     t.sh.write("# Test driver for %s, generated by "
                 "dso-ordering-test.py\n" % (t.test_name))
@@ -1022,12 +1043,12 @@ def process_testcase(t):
         sfx = ""
         if r[0] != "":
             sfx = "-" + r[0].replace("=","_")
-        expected_output_files += " $(objpfx)%s/%s%s.exp" % (test_subdir,
+        expected_output_files += " $(objpfx)%s%s%s.exp" % (test_srcdir,
                                                             t.test_name, sfx)
     makefile.write \
-    ("$(objpfx)%s.out: $(objpfx)%s/%s.sh%s "
+    ("$(objpfx)%s.out: $(objpfx)%s%s.sh%s "
      "$(common-objpfx)support/test-run-command\n"
-     % (t.test_name, test_subdir, t.test_name,
+     % (t.test_name, test_srcdir, t.test_name,
         expected_output_files))
     makefile.write("\t$(SHELL) $< $(common-objpfx) '$(test-wrapper-env)' "
                     "'$(run-program-env)' > $@; $(evaluate-test)\n")
-- 
2.33.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550]
  2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
@ 2021-11-11 17:52   ` Chung-Lin Tang
  2021-11-11 18:21     ` H.J. Lu
  2021-11-15 18:50   ` Adhemerval Zanella
  1 sibling, 1 reply; 15+ messages in thread
From: Chung-Lin Tang @ 2021-11-11 17:52 UTC (permalink / raw)
  To: H.J. Lu, libc-alpha; +Cc: Florian Weimer



On 2021/11/8 11:42 PM, H.J. Lu via Libc-alpha wrote:
> Put all sources for DSO sorting tests in the dso-sort-tests-src directory
> and compile test relocatable objects with
> 
> $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
> 	$(compile.c) $(OUTPUT_OPTION)
> 
> to avoid random $< values from $(before-compile) when compiling test
> relocatable objects with
> 
> $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
> compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
> compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)

I kind of see what you mean, but it appears that the explicit compile rule is the main required addition?
If so, why the directory change to "dso-sort-tests-src"?
(it kind of adds more complexity to the already somewhat complex placement of files, at least to someone new to the setup)

Chung-Lin

> for 3 "make -j 28" parallel builds on a machine with 112 cores at the
> same time.
> 
> This partially fixes BZ #28550.
> ---
>   scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
>   1 file changed, 34 insertions(+), 13 deletions(-)
> 
> diff --git a/scripts/dso-ordering-test.py b/scripts/dso-ordering-test.py
> index 944ee74052..bde0406be9 100644
> --- a/scripts/dso-ordering-test.py
> +++ b/scripts/dso-ordering-test.py
> @@ -526,9 +526,13 @@ def process_testcase(t):
>       base_test_name = t.test_name
>       test_subdir = base_test_name + "-dir"
>       testpfx = objpfx + test_subdir + "/"
> +    test_srcdir = "dso-sort-tests-src/"
> +    testpfx_src = objpfx + test_srcdir
>   
>       if not os.path.exists(testpfx):
>           os.mkdir(testpfx)
> +    if not os.path.exists(testpfx_src):
> +        os.mkdir(testpfx_src)
>   
>       def find_objs_not_depended_on(t):
>           objs_not_depended_on = []
> @@ -595,6 +599,11 @@ def process_testcase(t):
>           # Print out needed Makefile fragments for use in glibc/elf/Makefile.
>           module_names = ""
>           for o in test_descr.objs:
> +            rule = ("$(objpfx)" + test_subdir + "/" + test_name
> +                    + "-" + o + ".os: $(objpfx)" + test_srcdir
> +                    + test_name + "-" + o + ".c\n"
> +                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +            makefile.write (rule)
>               module_names += " " + test_subdir + "/" + test_name + "-" + o
>           makefile.write("modules-names +=%s\n" % (module_names))
>   
> @@ -637,7 +646,7 @@ def process_testcase(t):
>                           # object.  This only needs to be done at most once for
>                           # an object name.
>                           if not dep in fake_created:
> -                            f = open(testpfx + test_name + "-" + dep
> +                            f = open(testpfx_src + test_name + "-" + dep
>                                        + ".FAKE.c", "w")
>                               f.write(" \n")
>                               f.close()
> @@ -648,6 +657,12 @@ def process_testcase(t):
>                                    % (test_name + "-" + dep + ".FAKE.so",
>                                       ("$(objpfx)" + test_subdir + "/"
>                                        + test_name + "-" + dep + ".so")))
> +                            rule = ("$(objpfx)" + test_subdir + "/"
> +                                    + test_name + "-" + dep + ".FAKE.os: "
> +                                    "$(objpfx)" + test_srcdir
> +                                    + test_name + "-" + dep + ".FAKE.c\n"
> +                                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +                            makefile.write (rule)
>                               makefile.write \
>                                   ("modules-names += %s\n"
>                                    % (test_subdir + "/"
> @@ -687,6 +702,10 @@ def process_testcase(t):
>                         + test_descr.soname_map['#'] + ".so")
>               ldflags += (" -Wl,-soname=" + soname)
>           makefile.write("LDFLAGS-%s = %s\n" % (test_name, ldflags))
> +        rule = ("$(objpfx)" + test_subdir + "/" + test_name + ".o: "
> +                "$(objpfx)" + test_srcdir + test_name + ".c\n"
> +                "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +        makefile.write (rule)
>   
>           not_depended_objs = find_objs_not_depended_on(test_descr)
>           if not_depended_objs:
> @@ -745,7 +764,7 @@ def process_testcase(t):
>                        "  something_failed=true\n"
>                        "else\n"
>                        "  diff -wu ${common_objpfx}elf/%s/%s%s.output \\\n"
> -                     "           ${common_objpfx}elf/%s/%s%s.exp\n"
> +                     "           ${common_objpfx}elf/%s%s%s.exp\n"
>                        "  if [ $? -ne 0 ]; then\n"
>                        "    echo '%sFAIL: %s%s expected output comparison'\n"
>                        "    something_failed=true\n"
> @@ -753,14 +772,14 @@ def process_testcase(t):
>                        "fi\n"
>                        % (("X" if xfail else ""), test_name, tunable_descr,
>                           test_subdir, test_name, tunable_sfx,
> -                        test_subdir, base_test_name, exp_tunable_sfx,
> +                        test_srcdir, base_test_name, exp_tunable_sfx,
>                           ("X" if xfail else ""), test_name, tunable_descr))
>   
>           # Generate C files according to dependency and calling relations from
>           # description string.
>           for obj in test_descr.objs:
>               src_name = test_name + "-" + obj + ".c"
> -            f = open(testpfx + src_name, "w")
> +            f = open(testpfx_src + src_name, "w")
>               if obj in test_descr.callrefs:
>                   called_objs = test_descr.callrefs[obj]
>                   for callee in called_objs:
> @@ -804,7 +823,7 @@ def process_testcase(t):
>               f.close()
>   
>           # Open C file for writing main program
> -        f = open(testpfx + test_name + ".c", "w")
> +        f = open(testpfx_src + test_name + ".c", "w")
>   
>           # if there are some operations in main(), it means we need -ldl
>           f.write("#include <stdio.h>\n")
> @@ -885,7 +904,7 @@ def process_testcase(t):
>               for obj in test_descr.objs:
>                   src_name = test_name + "-" + obj + ".c"
>                   obj_name = test_name + "-" + obj + ".os"
> -                run_cmd([build_gcc, "-c", "-fPIC", testpfx + src_name,
> +                run_cmd([build_gcc, "-c", "-fPIC", testpfx_src + src_name,
>                             "-o", testpfx + obj_name])
>   
>               obj_processed = {}
> @@ -903,10 +922,12 @@ def process_testcase(t):
>                               deps.append(dep + ".FAKE")
>                               if not dep in fake_created:
>                                   base_name = testpfx + test_name + "-" + dep
> +                                src_base_name = (testpfx_src + test_name
> +                                                 + "-" + dep)
>                                   cmd = [build_gcc, "-Wl,--no-as-needed",
>                                          ("-Wl,-soname=" + base_name + ".so"),
>                                          "-shared", base_name + ".FAKE.c",
> -                                       "-o", base_name + ".FAKE.so"]
> +                                       "-o", src_base_name + ".FAKE.so"]
>                                   run_cmd(cmd)
>                                   fake_created[dep] = True
>                   dso_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
> @@ -932,7 +953,7 @@ def process_testcase(t):
>               main_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
>                               deps)
>               cmd = [build_gcc, "-Wl,--no-as-needed", "-o", testpfx + test_name,
> -                   testpfx + test_name + ".c", "-L%s" % (os.getcwd()),
> +                   testpfx_src + test_name + ".c", "-L%s" % (os.getcwd()),
>                      "-Wl,-rpath-link=%s" % (os.getcwd())]
>               if '#' in test_descr.soname_map:
>                   soname = ("-Wl,-soname=" + testpfx + test_name + "-"
> @@ -987,14 +1008,14 @@ def process_testcase(t):
>           sfx = ""
>           if r[0] != "":
>               sfx = "-" + r[0].replace("=","_")
> -        f = open(testpfx + t.test_name + sfx + ".exp", "w")
> +        f = open(testpfx_src + t.test_name + sfx + ".exp", "w")
>           (output, xfail) = r[1]
>           f.write('%s' % output)
>           f.close()
>   
>       # Create header part of top-level testcase shell script, to wrap execution
>       # and output comparison together.
> -    t.sh = open(testpfx + t.test_name + ".sh", "w")
> +    t.sh = open(testpfx_src + t.test_name + ".sh", "w")
>       t.sh.write("#!/bin/sh\n")
>       t.sh.write("# Test driver for %s, generated by "
>                   "dso-ordering-test.py\n" % (t.test_name))
> @@ -1022,12 +1043,12 @@ def process_testcase(t):
>           sfx = ""
>           if r[0] != "":
>               sfx = "-" + r[0].replace("=","_")
> -        expected_output_files += " $(objpfx)%s/%s%s.exp" % (test_subdir,
> +        expected_output_files += " $(objpfx)%s%s%s.exp" % (test_srcdir,
>                                                               t.test_name, sfx)
>       makefile.write \
> -    ("$(objpfx)%s.out: $(objpfx)%s/%s.sh%s "
> +    ("$(objpfx)%s.out: $(objpfx)%s%s.sh%s "
>        "$(common-objpfx)support/test-run-command\n"
> -     % (t.test_name, test_subdir, t.test_name,
> +     % (t.test_name, test_srcdir, t.test_name,
>           expected_output_files))
>       makefile.write("\t$(SHELL) $< $(common-objpfx) '$(test-wrapper-env)' "
>                       "'$(run-program-env)' > $@; $(evaluate-test)\n")
> 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550]
  2021-11-11 17:52   ` Chung-Lin Tang
@ 2021-11-11 18:21     ` H.J. Lu
  0 siblings, 0 replies; 15+ messages in thread
From: H.J. Lu @ 2021-11-11 18:21 UTC (permalink / raw)
  To: Chung-Lin Tang; +Cc: GNU C Library, Florian Weimer

On Thu, Nov 11, 2021 at 9:52 AM Chung-Lin Tang <cltang@codesourcery.com> wrote:
>
>
>
> On 2021/11/8 11:42 PM, H.J. Lu via Libc-alpha wrote:
> > Put all sources for DSO sorting tests in the dso-sort-tests-src directory
> > and compile test relocatable objects with
> >
> > $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
> >       $(compile.c) $(OUTPUT_OPTION)
> >
> > to avoid random $< values from $(before-compile) when compiling test
> > relocatable objects with
> >
> > $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
> > compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
> > compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)
>
> I kind of see what you mean, but it appears that the explicit compile rule is the main required addition?
> If so, why the directory change to "dso-sort-tests-src"?
> (it kind of adds more complexity to the already somewhat complex placement of files, at least to someone new to the setup)
>

If the generated files are placed in the same directory where the object files
are, they will match the default rule:

$(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)

which is the root cause of the parallel build problem.


-- 
H.J.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550]
  2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
  2021-11-11 17:52   ` Chung-Lin Tang
@ 2021-11-15 18:50   ` Adhemerval Zanella
  1 sibling, 0 replies; 15+ messages in thread
From: Adhemerval Zanella @ 2021-11-15 18:50 UTC (permalink / raw)
  To: libc-alpha, H.J. Lu



On 08/11/2021 12:42, H.J. Lu via Libc-alpha wrote:
> Put all sources for DSO sorting tests in the dso-sort-tests-src directory
> and compile test relocatable objects with
> 
> $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
> 	$(compile.c) $(OUTPUT_OPTION)
> 
> to avoid random $< values from $(before-compile) when compiling test
> relocatable objects with
> 
> $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
> compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
> compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)
> 
> for 3 "make -j 28" parallel builds on a machine with 112 cores at the
> same time.
> 
> This partially fixes BZ #28550.

LGTM, thanks.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>

> ---
>  scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
>  1 file changed, 34 insertions(+), 13 deletions(-)
> 
> diff --git a/scripts/dso-ordering-test.py b/scripts/dso-ordering-test.py
> index 944ee74052..bde0406be9 100644
> --- a/scripts/dso-ordering-test.py
> +++ b/scripts/dso-ordering-test.py
> @@ -526,9 +526,13 @@ def process_testcase(t):
>      base_test_name = t.test_name
>      test_subdir = base_test_name + "-dir"
>      testpfx = objpfx + test_subdir + "/"
> +    test_srcdir = "dso-sort-tests-src/"
> +    testpfx_src = objpfx + test_srcdir
>  
>      if not os.path.exists(testpfx):
>          os.mkdir(testpfx)
> +    if not os.path.exists(testpfx_src):
> +        os.mkdir(testpfx_src)
>  
>      def find_objs_not_depended_on(t):
>          objs_not_depended_on = []
> @@ -595,6 +599,11 @@ def process_testcase(t):
>          # Print out needed Makefile fragments for use in glibc/elf/Makefile.
>          module_names = ""
>          for o in test_descr.objs:
> +            rule = ("$(objpfx)" + test_subdir + "/" + test_name
> +                    + "-" + o + ".os: $(objpfx)" + test_srcdir
> +                    + test_name + "-" + o + ".c\n"
> +                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +            makefile.write (rule)
>              module_names += " " + test_subdir + "/" + test_name + "-" + o
>          makefile.write("modules-names +=%s\n" % (module_names))
>  
> @@ -637,7 +646,7 @@ def process_testcase(t):
>                          # object.  This only needs to be done at most once for
>                          # an object name.
>                          if not dep in fake_created:
> -                            f = open(testpfx + test_name + "-" + dep
> +                            f = open(testpfx_src + test_name + "-" + dep
>                                       + ".FAKE.c", "w")
>                              f.write(" \n")
>                              f.close()
> @@ -648,6 +657,12 @@ def process_testcase(t):
>                                   % (test_name + "-" + dep + ".FAKE.so",
>                                      ("$(objpfx)" + test_subdir + "/"
>                                       + test_name + "-" + dep + ".so")))
> +                            rule = ("$(objpfx)" + test_subdir + "/"
> +                                    + test_name + "-" + dep + ".FAKE.os: "
> +                                    "$(objpfx)" + test_srcdir
> +                                    + test_name + "-" + dep + ".FAKE.c\n"
> +                                    "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +                            makefile.write (rule)
>                              makefile.write \
>                                  ("modules-names += %s\n"
>                                   % (test_subdir + "/"
> @@ -687,6 +702,10 @@ def process_testcase(t):
>                        + test_descr.soname_map['#'] + ".so")
>              ldflags += (" -Wl,-soname=" + soname)
>          makefile.write("LDFLAGS-%s = %s\n" % (test_name, ldflags))
> +        rule = ("$(objpfx)" + test_subdir + "/" + test_name + ".o: "
> +                "$(objpfx)" + test_srcdir + test_name + ".c\n"
> +                "\t$(compile.c) $(OUTPUT_OPTION)\n")
> +        makefile.write (rule)
>  
>          not_depended_objs = find_objs_not_depended_on(test_descr)
>          if not_depended_objs:
> @@ -745,7 +764,7 @@ def process_testcase(t):
>                       "  something_failed=true\n"
>                       "else\n"
>                       "  diff -wu ${common_objpfx}elf/%s/%s%s.output \\\n"
> -                     "           ${common_objpfx}elf/%s/%s%s.exp\n"
> +                     "           ${common_objpfx}elf/%s%s%s.exp\n"
>                       "  if [ $? -ne 0 ]; then\n"
>                       "    echo '%sFAIL: %s%s expected output comparison'\n"
>                       "    something_failed=true\n"
> @@ -753,14 +772,14 @@ def process_testcase(t):
>                       "fi\n"
>                       % (("X" if xfail else ""), test_name, tunable_descr,
>                          test_subdir, test_name, tunable_sfx,
> -                        test_subdir, base_test_name, exp_tunable_sfx,
> +                        test_srcdir, base_test_name, exp_tunable_sfx,
>                          ("X" if xfail else ""), test_name, tunable_descr))
>  
>          # Generate C files according to dependency and calling relations from
>          # description string.
>          for obj in test_descr.objs:
>              src_name = test_name + "-" + obj + ".c"
> -            f = open(testpfx + src_name, "w")
> +            f = open(testpfx_src + src_name, "w")
>              if obj in test_descr.callrefs:
>                  called_objs = test_descr.callrefs[obj]
>                  for callee in called_objs:
> @@ -804,7 +823,7 @@ def process_testcase(t):
>              f.close()
>  
>          # Open C file for writing main program
> -        f = open(testpfx + test_name + ".c", "w")
> +        f = open(testpfx_src + test_name + ".c", "w")
>  
>          # if there are some operations in main(), it means we need -ldl
>          f.write("#include <stdio.h>\n")
> @@ -885,7 +904,7 @@ def process_testcase(t):
>              for obj in test_descr.objs:
>                  src_name = test_name + "-" + obj + ".c"
>                  obj_name = test_name + "-" + obj + ".os"
> -                run_cmd([build_gcc, "-c", "-fPIC", testpfx + src_name,
> +                run_cmd([build_gcc, "-c", "-fPIC", testpfx_src + src_name,
>                            "-o", testpfx + obj_name])
>  
>              obj_processed = {}
> @@ -903,10 +922,12 @@ def process_testcase(t):
>                              deps.append(dep + ".FAKE")
>                              if not dep in fake_created:
>                                  base_name = testpfx + test_name + "-" + dep
> +                                src_base_name = (testpfx_src + test_name
> +                                                 + "-" + dep)
>                                  cmd = [build_gcc, "-Wl,--no-as-needed",
>                                         ("-Wl,-soname=" + base_name + ".so"),
>                                         "-shared", base_name + ".FAKE.c",
> -                                       "-o", base_name + ".FAKE.so"]
> +                                       "-o", src_base_name + ".FAKE.so"]
>                                  run_cmd(cmd)
>                                  fake_created[dep] = True
>                  dso_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
> @@ -932,7 +953,7 @@ def process_testcase(t):
>              main_deps = map(lambda d: testpfx + test_name + "-" + d + ".so",
>                              deps)
>              cmd = [build_gcc, "-Wl,--no-as-needed", "-o", testpfx + test_name,
> -                   testpfx + test_name + ".c", "-L%s" % (os.getcwd()),
> +                   testpfx_src + test_name + ".c", "-L%s" % (os.getcwd()),
>                     "-Wl,-rpath-link=%s" % (os.getcwd())]
>              if '#' in test_descr.soname_map:
>                  soname = ("-Wl,-soname=" + testpfx + test_name + "-"
> @@ -987,14 +1008,14 @@ def process_testcase(t):
>          sfx = ""
>          if r[0] != "":
>              sfx = "-" + r[0].replace("=","_")
> -        f = open(testpfx + t.test_name + sfx + ".exp", "w")
> +        f = open(testpfx_src + t.test_name + sfx + ".exp", "w")
>          (output, xfail) = r[1]
>          f.write('%s' % output)
>          f.close()
>  
>      # Create header part of top-level testcase shell script, to wrap execution
>      # and output comparison together.
> -    t.sh = open(testpfx + t.test_name + ".sh", "w")
> +    t.sh = open(testpfx_src + t.test_name + ".sh", "w")
>      t.sh.write("#!/bin/sh\n")
>      t.sh.write("# Test driver for %s, generated by "
>                  "dso-ordering-test.py\n" % (t.test_name))
> @@ -1022,12 +1043,12 @@ def process_testcase(t):
>          sfx = ""
>          if r[0] != "":
>              sfx = "-" + r[0].replace("=","_")
> -        expected_output_files += " $(objpfx)%s/%s%s.exp" % (test_subdir,
> +        expected_output_files += " $(objpfx)%s%s%s.exp" % (test_srcdir,
>                                                              t.test_name, sfx)
>      makefile.write \
> -    ("$(objpfx)%s.out: $(objpfx)%s/%s.sh%s "
> +    ("$(objpfx)%s.out: $(objpfx)%s%s.sh%s "
>       "$(common-objpfx)support/test-run-command\n"
> -     % (t.test_name, test_subdir, t.test_name,
> +     % (t.test_name, test_srcdir, t.test_name,
>          expected_output_files))
>      makefile.write("\t$(SHELL) $< $(common-objpfx) '$(test-wrapper-env)' "
>                      "'$(run-program-env)' > $@; $(evaluate-test)\n")
> 

Ok.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
  2021-11-08 15:42 [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
  2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
@ 2021-11-08 15:42 ` H.J. Lu
  2021-11-15 18:59   ` Adhemerval Zanella
  2021-11-11 13:46 ` [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
  2 siblings, 1 reply; 15+ messages in thread
From: H.J. Lu @ 2021-11-08 15:42 UTC (permalink / raw)
  To: libc-alpha; +Cc: Florian Weimer

Generate Makefile fragment for DSO sorting tests at configure time.

This partially fixes BZ #28550.
---
 configure    | 11 +++++++++++
 configure.ac | 12 ++++++++++++
 elf/Makefile | 22 +++++-----------------
 3 files changed, 28 insertions(+), 17 deletions(-)

diff --git a/configure b/configure
index 2f9adca064..91993da3b9 100755
--- a/configure
+++ b/configure
@@ -8484,3 +8484,14 @@ if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then
 $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;}
 fi
 
+
+if test "$have_tunables" = yes; then
+  test -d elf || mkdir elf
+  objdir=`pwd`
+  rm -f elf/dso-sort-tests.mk
+  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
+   do
+    $PYTHON $srcdir/scripts/dso-ordering-test.py \
+	--description-file $srcdir/elf/$d --objpfx $objdir/elf/
+   done) > elf/dso-sort-tests.mk
+fi
diff --git a/configure.ac b/configure.ac
index 7eb4239359..faf70b930c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1901,3 +1901,15 @@ echo "$config_vars" >> config.make;;
 esac
 test -d bits || mkdir bits]],[[config_vars='$config_vars']])
 AC_OUTPUT
+
+dnl Generate Makefile fragments for DSO sorting tests.
+if test "$have_tunables" = yes; then
+  test -d elf || mkdir elf
+  objdir=`pwd`
+  rm -f elf/dso-sort-tests.mk
+  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
+   do
+    $PYTHON $srcdir/scripts/dso-ordering-test.py \
+	--description-file $srcdir/elf/$d --objpfx $objdir/elf/
+   done) > elf/dso-sort-tests.mk
+fi
diff --git a/elf/Makefile b/elf/Makefile
index 41c19668c3..6995ccaa0f 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -366,6 +366,11 @@ modules-names = testobj1 testobj2 testobj3 testobj4 testobj5 testobj6 \
 		tst-auxvalmod \
 		tst-dlmopen-gethostbyname-mod tst-ro-dynamic-mod \
 
+ifeq (yes,$(have-tunables))
+# Include Makefile fragments for DSO sorting tests.
+include $(objpfx)dso-sort-tests.mk
+endif
+
 # Most modules build with _ISOMAC defined, but those filtered out
 # depend on internal headers.
 modules-names-tests = $(filter-out ifuncmod% tst-tlsmod%,\
@@ -486,23 +491,6 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
 		 $(objpfx)tst-unused-dep-cmp.out
 endif
 
-# DSO sorting tests:
-# The dso-ordering-test.py script generates testcase source files in $(objpfx),
-# creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
-# Makefile fragment to be included.
-define include_dsosort_tests
-$(objpfx)$(1).generated-makefile: $(1)
-	$(PYTHON) $(..)scripts/dso-ordering-test.py \
-	--description-file $$< --objpfx $(objpfx) --output-makefile $$@
-include $(objpfx)$(1).generated-makefile
-endef
-
-# Generate from each testcase description file
-ifeq (yes,$(have-tunables))
-$(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
-$(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
-endif
-
 check-abi: $(objpfx)check-abi-ld.out
 tests-special += $(objpfx)check-abi-ld.out
 update-abi: update-abi-ld
-- 
2.33.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
  2021-11-08 15:42 ` [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests " H.J. Lu
@ 2021-11-15 18:59   ` Adhemerval Zanella
  2021-11-15 19:51     ` H.J. Lu
  0 siblings, 1 reply; 15+ messages in thread
From: Adhemerval Zanella @ 2021-11-15 18:59 UTC (permalink / raw)
  To: H.J. Lu, libc-alpha; +Cc: Florian Weimer



On 08/11/2021 12:42, H.J. Lu via Libc-alpha wrote:
> Generate Makefile fragment for DSO sorting tests at configure time.
> 
> This partially fixes BZ #28550.

I am not sure if configure.ac is the right place to generate the Makefile
fragment required for tests.  Can we use 'avoid-generated' instead?


diff --git a/elf/Makefile b/elf/Makefile
index a311c3e23c..8556105a20 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
                 $(objpfx)tst-unused-dep-cmp.out
 endif
 
+ifndef avoid-generated
 # DSO sorting tests:
 # The dso-ordering-test.py script generates testcase source files in $(objpfx),
 # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
@@ -507,6 +508,7 @@ ifeq (yes,$(have-tunables))
 $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
 $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
 endif
+endif
 
 check-abi: $(objpfx)check-abi-ld.out
 tests-special += $(objpfx)check-abi-ld.out


> ---
>  configure    | 11 +++++++++++
>  configure.ac | 12 ++++++++++++
>  elf/Makefile | 22 +++++-----------------
>  3 files changed, 28 insertions(+), 17 deletions(-)
> 
> diff --git a/configure b/configure
> index 2f9adca064..91993da3b9 100755
> --- a/configure
> +++ b/configure
> @@ -8484,3 +8484,14 @@ if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then
>  $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;}
>  fi
>  
> +
> +if test "$have_tunables" = yes; then
> +  test -d elf || mkdir elf
> +  objdir=`pwd`
> +  rm -f elf/dso-sort-tests.mk
> +  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
> +   do
> +    $PYTHON $srcdir/scripts/dso-ordering-test.py \
> +	--description-file $srcdir/elf/$d --objpfx $objdir/elf/
> +   done) > elf/dso-sort-tests.mk
> +fi
> diff --git a/configure.ac b/configure.ac
> index 7eb4239359..faf70b930c 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1901,3 +1901,15 @@ echo "$config_vars" >> config.make;;
>  esac
>  test -d bits || mkdir bits]],[[config_vars='$config_vars']])
>  AC_OUTPUT
> +
> +dnl Generate Makefile fragments for DSO sorting tests.
> +if test "$have_tunables" = yes; then
> +  test -d elf || mkdir elf
> +  objdir=`pwd`
> +  rm -f elf/dso-sort-tests.mk
> +  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
> +   do
> +    $PYTHON $srcdir/scripts/dso-ordering-test.py \
> +	--description-file $srcdir/elf/$d --objpfx $objdir/elf/
> +   done) > elf/dso-sort-tests.mk
> +fi
> diff --git a/elf/Makefile b/elf/Makefile
> index 41c19668c3..6995ccaa0f 100644
> --- a/elf/Makefile
> +++ b/elf/Makefile
> @@ -366,6 +366,11 @@ modules-names = testobj1 testobj2 testobj3 testobj4 testobj5 testobj6 \
>  		tst-auxvalmod \
>  		tst-dlmopen-gethostbyname-mod tst-ro-dynamic-mod \
>  
> +ifeq (yes,$(have-tunables))
> +# Include Makefile fragments for DSO sorting tests.
> +include $(objpfx)dso-sort-tests.mk
> +endif
> +
>  # Most modules build with _ISOMAC defined, but those filtered out
>  # depend on internal headers.
>  modules-names-tests = $(filter-out ifuncmod% tst-tlsmod%,\
> @@ -486,23 +491,6 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
>  		 $(objpfx)tst-unused-dep-cmp.out
>  endif
>  
> -# DSO sorting tests:
> -# The dso-ordering-test.py script generates testcase source files in $(objpfx),
> -# creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> -# Makefile fragment to be included.
> -define include_dsosort_tests
> -$(objpfx)$(1).generated-makefile: $(1)
> -	$(PYTHON) $(..)scripts/dso-ordering-test.py \
> -	--description-file $$< --objpfx $(objpfx) --output-makefile $$@
> -include $(objpfx)$(1).generated-makefile
> -endef
> -
> -# Generate from each testcase description file
> -ifeq (yes,$(have-tunables))
> -$(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
> -$(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
> -endif
> -
>  check-abi: $(objpfx)check-abi-ld.out
>  tests-special += $(objpfx)check-abi-ld.out
>  update-abi: update-abi-ld
> 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
  2021-11-15 18:59   ` Adhemerval Zanella
@ 2021-11-15 19:51     ` H.J. Lu
  2021-11-15 20:18       ` Adhemerval Zanella
  0 siblings, 1 reply; 15+ messages in thread
From: H.J. Lu @ 2021-11-15 19:51 UTC (permalink / raw)
  To: Adhemerval Zanella; +Cc: GNU C Library, Florian Weimer

On Mon, Nov 15, 2021 at 10:59 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 08/11/2021 12:42, H.J. Lu via Libc-alpha wrote:
> > Generate Makefile fragment for DSO sorting tests at configure time.
> >
> > This partially fixes BZ #28550.
>
> I am not sure if configure.ac is the right place to generate the Makefile
> fragment required for tests.  Can we use 'avoid-generated' instead?
>
>
> diff --git a/elf/Makefile b/elf/Makefile
> index a311c3e23c..8556105a20 100644
> --- a/elf/Makefile
> +++ b/elf/Makefile
> @@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
>                  $(objpfx)tst-unused-dep-cmp.out
>  endif
>
> +ifndef avoid-generated
>  # DSO sorting tests:
>  # The dso-ordering-test.py script generates testcase source files in $(objpfx),
>  # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> @@ -507,6 +508,7 @@ ifeq (yes,$(have-tunables))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
>  endif
> +endif

It doesn't work.  "make check -j28" on a machine with 112 cores failed:

/export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/dso-sort-tests-1.def.generated-makefile:938:
warning: ignoring old recipe for target
'/export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/tst-dso-ordering9-dir/tst-dso-ordering9_25-bacde-b.so'
...
make[3]: *** [Makefile:483: elf/tests] Error 2

I couldn't find the exact error message in a very long make log file.

>  check-abi: $(objpfx)check-abi-ld.out
>  tests-special += $(objpfx)check-abi-ld.out
>
>
> > ---
> >  configure    | 11 +++++++++++
> >  configure.ac | 12 ++++++++++++
> >  elf/Makefile | 22 +++++-----------------
> >  3 files changed, 28 insertions(+), 17 deletions(-)
> >
> > diff --git a/configure b/configure
> > index 2f9adca064..91993da3b9 100755
> > --- a/configure
> > +++ b/configure
> > @@ -8484,3 +8484,14 @@ if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then
> >  $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;}
> >  fi
> >
> > +
> > +if test "$have_tunables" = yes; then
> > +  test -d elf || mkdir elf
> > +  objdir=`pwd`
> > +  rm -f elf/dso-sort-tests.mk
> > +  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
> > +   do
> > +    $PYTHON $srcdir/scripts/dso-ordering-test.py \
> > +     --description-file $srcdir/elf/$d --objpfx $objdir/elf/
> > +   done) > elf/dso-sort-tests.mk
> > +fi
> > diff --git a/configure.ac b/configure.ac
> > index 7eb4239359..faf70b930c 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -1901,3 +1901,15 @@ echo "$config_vars" >> config.make;;
> >  esac
> >  test -d bits || mkdir bits]],[[config_vars='$config_vars']])
> >  AC_OUTPUT
> > +
> > +dnl Generate Makefile fragments for DSO sorting tests.
> > +if test "$have_tunables" = yes; then
> > +  test -d elf || mkdir elf
> > +  objdir=`pwd`
> > +  rm -f elf/dso-sort-tests.mk
> > +  (for d in dso-sort-tests-1.def dso-sort-tests-2.def
> > +   do
> > +    $PYTHON $srcdir/scripts/dso-ordering-test.py \
> > +     --description-file $srcdir/elf/$d --objpfx $objdir/elf/
> > +   done) > elf/dso-sort-tests.mk
> > +fi
> > diff --git a/elf/Makefile b/elf/Makefile
> > index 41c19668c3..6995ccaa0f 100644
> > --- a/elf/Makefile
> > +++ b/elf/Makefile
> > @@ -366,6 +366,11 @@ modules-names = testobj1 testobj2 testobj3 testobj4 testobj5 testobj6 \
> >               tst-auxvalmod \
> >               tst-dlmopen-gethostbyname-mod tst-ro-dynamic-mod \
> >
> > +ifeq (yes,$(have-tunables))
> > +# Include Makefile fragments for DSO sorting tests.
> > +include $(objpfx)dso-sort-tests.mk
> > +endif
> > +
> >  # Most modules build with _ISOMAC defined, but those filtered out
> >  # depend on internal headers.
> >  modules-names-tests = $(filter-out ifuncmod% tst-tlsmod%,\
> > @@ -486,23 +491,6 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
> >                $(objpfx)tst-unused-dep-cmp.out
> >  endif
> >
> > -# DSO sorting tests:
> > -# The dso-ordering-test.py script generates testcase source files in $(objpfx),
> > -# creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> > -# Makefile fragment to be included.
> > -define include_dsosort_tests
> > -$(objpfx)$(1).generated-makefile: $(1)
> > -     $(PYTHON) $(..)scripts/dso-ordering-test.py \
> > -     --description-file $$< --objpfx $(objpfx) --output-makefile $$@
> > -include $(objpfx)$(1).generated-makefile
> > -endef
> > -
> > -# Generate from each testcase description file
> > -ifeq (yes,$(have-tunables))
> > -$(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
> > -$(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
> > -endif
> > -
> >  check-abi: $(objpfx)check-abi-ld.out
> >  tests-special += $(objpfx)check-abi-ld.out
> >  update-abi: update-abi-ld
> >



--
H.J.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
  2021-11-15 19:51     ` H.J. Lu
@ 2021-11-15 20:18       ` Adhemerval Zanella
  2021-11-16  4:48         ` [PATCH] elf: Use a temporary file to generate Makefile fragments " H.J. Lu
  0 siblings, 1 reply; 15+ messages in thread
From: Adhemerval Zanella @ 2021-11-15 20:18 UTC (permalink / raw)
  To: H.J. Lu; +Cc: GNU C Library, Florian Weimer



On 15/11/2021 16:51, H.J. Lu wrote:
> On Mon, Nov 15, 2021 at 10:59 AM Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
>>
>>
>>
>> On 08/11/2021 12:42, H.J. Lu via Libc-alpha wrote:
>>> Generate Makefile fragment for DSO sorting tests at configure time.
>>>
>>> This partially fixes BZ #28550.
>>
>> I am not sure if configure.ac is the right place to generate the Makefile
>> fragment required for tests.  Can we use 'avoid-generated' instead?
>>
>>
>> diff --git a/elf/Makefile b/elf/Makefile
>> index a311c3e23c..8556105a20 100644
>> --- a/elf/Makefile
>> +++ b/elf/Makefile
>> @@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
>>                  $(objpfx)tst-unused-dep-cmp.out
>>  endif
>>
>> +ifndef avoid-generated
>>  # DSO sorting tests:
>>  # The dso-ordering-test.py script generates testcase source files in $(objpfx),
>>  # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
>> @@ -507,6 +508,7 @@ ifeq (yes,$(have-tunables))
>>  $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
>>  $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
>>  endif
>> +endif
> 
> It doesn't work.  "make check -j28" on a machine with 112 cores failed:
> 
> /export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/dso-sort-tests-1.def.generated-makefile:938:
> warning: ignoring old recipe for target
> '/export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/tst-dso-ordering9-dir/tst-dso-ordering9_25-bacde-b.so'
> ...
> make[3]: *** [Makefile:483: elf/tests] Error 2
> 
> I couldn't find the exact error message in a very long make log file.
This is usual way to create other auto-generated Makefile fragments, such as
sysd-sycalls and time64-compat.mk.  Maybe the below helps (I forgot to use 
'-include' and add the fragments on postclean-generated).


diff --git a/elf/Makefile b/elf/Makefile
index a311c3e23c..0e6d7ffb51 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
                 $(objpfx)tst-unused-dep-cmp.out
 endif
 
+ifndef avoid-generated
 # DSO sorting tests:
 # The dso-ordering-test.py script generates testcase source files in $(objpfx),
 # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
@@ -499,7 +500,7 @@ define include_dsosort_tests
 $(objpfx)$(1).generated-makefile: $(1)
        $(PYTHON) $(..)scripts/dso-ordering-test.py \
        --description-file $$< --objpfx $(objpfx) --output-makefile $$@
-include $(objpfx)$(1).generated-makefile
+-include $(objpfx)$(1).generated-makefile
 endef
 
 # Generate from each testcase description file
@@ -507,6 +508,10 @@ ifeq (yes,$(have-tunables))
 $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
 $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
 endif
+endif
+
+postclean-generated += $(objpfx)/dso-sort-tests-2.generated-makefile \
+                      $(objpfx)/dso-sort-tests-2.generated-makefile
 
 check-abi: $(objpfx)check-abi-ld.out
 tests-special += $(objpfx)check-abi-ld.out

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ #28550]
  2021-11-15 20:18       ` Adhemerval Zanella
@ 2021-11-16  4:48         ` H.J. Lu
  2021-11-16 12:52           ` Adhemerval Zanella
  0 siblings, 1 reply; 15+ messages in thread
From: H.J. Lu @ 2021-11-16  4:48 UTC (permalink / raw)
  To: Adhemerval Zanella; +Cc: GNU C Library, Florian Weimer

[-- Attachment #1: Type: text/plain, Size: 3405 bytes --]

On Mon, Nov 15, 2021 at 12:18 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 15/11/2021 16:51, H.J. Lu wrote:
> > On Mon, Nov 15, 2021 at 10:59 AM Adhemerval Zanella
> > <adhemerval.zanella@linaro.org> wrote:
> >>
> >>
> >>
> >> On 08/11/2021 12:42, H.J. Lu via Libc-alpha wrote:
> >>> Generate Makefile fragment for DSO sorting tests at configure time.
> >>>
> >>> This partially fixes BZ #28550.
> >>
> >> I am not sure if configure.ac is the right place to generate the Makefile
> >> fragment required for tests.  Can we use 'avoid-generated' instead?
> >>
> >>
> >> diff --git a/elf/Makefile b/elf/Makefile
> >> index a311c3e23c..8556105a20 100644
> >> --- a/elf/Makefile
> >> +++ b/elf/Makefile
> >> @@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
> >>                  $(objpfx)tst-unused-dep-cmp.out
> >>  endif
> >>
> >> +ifndef avoid-generated
> >>  # DSO sorting tests:
> >>  # The dso-ordering-test.py script generates testcase source files in $(objpfx),
> >>  # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> >> @@ -507,6 +508,7 @@ ifeq (yes,$(have-tunables))
> >>  $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
> >>  $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
> >>  endif
> >> +endif
> >
> > It doesn't work.  "make check -j28" on a machine with 112 cores failed:
> >
> > /export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/dso-sort-tests-1.def.generated-makefile:938:
> > warning: ignoring old recipe for target
> > '/export/users/hjl/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/elf/tst-dso-ordering9-dir/tst-dso-ordering9_25-bacde-b.so'
> > ...
> > make[3]: *** [Makefile:483: elf/tests] Error 2
> >
> > I couldn't find the exact error message in a very long make log file.
> This is usual way to create other auto-generated Makefile fragments, such as
> sysd-sycalls and time64-compat.mk.  Maybe the below helps (I forgot to use
> '-include' and add the fragments on postclean-generated).
>
>
> diff --git a/elf/Makefile b/elf/Makefile
> index a311c3e23c..0e6d7ffb51 100644
> --- a/elf/Makefile
> +++ b/elf/Makefile
> @@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
>                  $(objpfx)tst-unused-dep-cmp.out
>  endif
>
> +ifndef avoid-generated
>  # DSO sorting tests:
>  # The dso-ordering-test.py script generates testcase source files in $(objpfx),
>  # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> @@ -499,7 +500,7 @@ define include_dsosort_tests
>  $(objpfx)$(1).generated-makefile: $(1)
>         $(PYTHON) $(..)scripts/dso-ordering-test.py \
>         --description-file $$< --objpfx $(objpfx) --output-makefile $$@
> -include $(objpfx)$(1).generated-makefile
> +-include $(objpfx)$(1).generated-makefile
>  endef
>
>  # Generate from each testcase description file
> @@ -507,6 +508,10 @@ ifeq (yes,$(have-tunables))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
>  endif
> +endif
> +
> +postclean-generated += $(objpfx)/dso-sort-tests-2.generated-makefile \
> +                      $(objpfx)/dso-sort-tests-2.generated-makefile
>
>  check-abi: $(objpfx)check-abi-ld.out
>  tests-special += $(objpfx)check-abi-ld.out

This works for me.

Thanks.

-- 
H.J.

[-- Attachment #2: 0001-elf-Use-a-temporary-file-to-generate-Makefile-fragme.patch --]
[-- Type: text/x-patch, Size: 1006 bytes --]

From a928b52c036bb888797b3e81014f441f0d876b65 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Mon, 15 Nov 2021 16:28:39 -0800
Subject: [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ
 #28550]

Use a temporary file to generate Makefile fragments for DSO sorting tests
and use -include on them.

This partially fixes BZ #28550.
---
 elf/Makefile | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/elf/Makefile b/elf/Makefile
index a311c3e23c..57374fc690 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -498,8 +498,9 @@ endif
 define include_dsosort_tests
 $(objpfx)$(1).generated-makefile: $(1)
 	$(PYTHON) $(..)scripts/dso-ordering-test.py \
-	--description-file $$< --objpfx $(objpfx) --output-makefile $$@
-include $(objpfx)$(1).generated-makefile
+	--description-file $$< --objpfx $(objpfx) --output-makefile $$@T
+	mv $$@T $$@
+-include $(objpfx)$(1).generated-makefile
 endef
 
 # Generate from each testcase description file
-- 
2.33.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ #28550]
  2021-11-16  4:48         ` [PATCH] elf: Use a temporary file to generate Makefile fragments " H.J. Lu
@ 2021-11-16 12:52           ` Adhemerval Zanella
  2021-11-16 13:15             ` H.J. Lu
  0 siblings, 1 reply; 15+ messages in thread
From: Adhemerval Zanella @ 2021-11-16 12:52 UTC (permalink / raw)
  To: H.J. Lu; +Cc: GNU C Library, Florian Weimer



On 16/11/2021 01:48, H.J. Lu wrote:
> 
> This works for me.
> 
> Thanks.
> 

> From a928b52c036bb888797b3e81014f441f0d876b65 Mon Sep 17 00:00:00 2001
> From: "H.J. Lu" <hjl.tools@gmail.com>
> Date: Mon, 15 Nov 2021 16:28:39 -0800
> Subject: [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ
>  #28550]
> 
> Use a temporary file to generate Makefile fragments for DSO sorting tests
> and use -include on them.
> 
> This partially fixes BZ #28550.
> ---
>  elf/Makefile | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/elf/Makefile b/elf/Makefile
> index a311c3e23c..57374fc690 100644
> --- a/elf/Makefile
> +++ b/elf/Makefile
> @@ -498,8 +498,9 @@ endif
>  define include_dsosort_tests
>  $(objpfx)$(1).generated-makefile: $(1)
>  	$(PYTHON) $(..)scripts/dso-ordering-test.py \
> -	--description-file $$< --objpfx $(objpfx) --output-makefile $$@
> -include $(objpfx)$(1).generated-makefile
> +	--description-file $$< --objpfx $(objpfx) --output-makefile $$@T
> +	mv $$@T $$@
> +-include $(objpfx)$(1).generated-makefile
>  endef
>  
>  # Generate from each testcase description file
> -- 
> 2.33.1

I think you need the below on top of your patch.  The 'avoid-generated' and
'postclean-generated' are required so a 'make clean' removes the autogenerated
fragments so a subsequent 'make' regenerates them.

diff --git a/elf/Makefile b/elf/Makefile
index 57374fc690..31759279d4 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
                 $(objpfx)tst-unused-dep-cmp.out
 endif
 
+ifndef avoid-generated
 # DSO sorting tests:
 # The dso-ordering-test.py script generates testcase source files in $(objpfx),
 # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
@@ -508,6 +509,9 @@ ifeq (yes,$(have-tunables))
 $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
 $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
 endif
+endif
+postclean-generated += $(objpfx)/dso-sort-tests-2.generated-makefile \
+                      $(objpfx)/dso-sort-tests-2.generated-makefile
 
 check-abi: $(objpfx)check-abi-ld.out
 tests-special += $(objpfx)check-abi-ld.out

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ #28550]
  2021-11-16 12:52           ` Adhemerval Zanella
@ 2021-11-16 13:15             ` H.J. Lu
  0 siblings, 0 replies; 15+ messages in thread
From: H.J. Lu @ 2021-11-16 13:15 UTC (permalink / raw)
  To: Adhemerval Zanella; +Cc: GNU C Library, Florian Weimer

On Tue, Nov 16, 2021 at 4:52 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 16/11/2021 01:48, H.J. Lu wrote:
> >
> > This works for me.
> >
> > Thanks.
> >
>
> > From a928b52c036bb888797b3e81014f441f0d876b65 Mon Sep 17 00:00:00 2001
> > From: "H.J. Lu" <hjl.tools@gmail.com>
> > Date: Mon, 15 Nov 2021 16:28:39 -0800
> > Subject: [PATCH] elf: Use a temporary file to generate Makefile fragments [BZ
> >  #28550]
> >
> > Use a temporary file to generate Makefile fragments for DSO sorting tests
> > and use -include on them.
> >
> > This partially fixes BZ #28550.
> > ---
> >  elf/Makefile | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/elf/Makefile b/elf/Makefile
> > index a311c3e23c..57374fc690 100644
> > --- a/elf/Makefile
> > +++ b/elf/Makefile
> > @@ -498,8 +498,9 @@ endif
> >  define include_dsosort_tests
> >  $(objpfx)$(1).generated-makefile: $(1)
> >       $(PYTHON) $(..)scripts/dso-ordering-test.py \
> > -     --description-file $$< --objpfx $(objpfx) --output-makefile $$@
> > -include $(objpfx)$(1).generated-makefile
> > +     --description-file $$< --objpfx $(objpfx) --output-makefile $$@T
> > +     mv $$@T $$@
> > +-include $(objpfx)$(1).generated-makefile
> >  endef
> >
> >  # Generate from each testcase description file
> > --
> > 2.33.1
>
> I think you need the below on top of your patch.  The 'avoid-generated' and
> 'postclean-generated' are required so a 'make clean' removes the autogenerated
> fragments so a subsequent 'make' regenerates them.

Fixed in the v2 patch.

Thanks.

> diff --git a/elf/Makefile b/elf/Makefile
> index 57374fc690..31759279d4 100644
> --- a/elf/Makefile
> +++ b/elf/Makefile
> @@ -491,6 +491,7 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
>                  $(objpfx)tst-unused-dep-cmp.out
>  endif
>
> +ifndef avoid-generated
>  # DSO sorting tests:
>  # The dso-ordering-test.py script generates testcase source files in $(objpfx),
>  # creating a $(objpfx)<testcase-name>-dir for each testcase, and creates a
> @@ -508,6 +509,9 @@ ifeq (yes,$(have-tunables))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-1.def))
>  $(eval $(call include_dsosort_tests,dso-sort-tests-2.def))
>  endif
> +endif
> +postclean-generated += $(objpfx)/dso-sort-tests-2.generated-makefile \
> +                      $(objpfx)/dso-sort-tests-2.generated-makefile
>
>  check-abi: $(objpfx)check-abi-ld.out
>  tests-special += $(objpfx)check-abi-ld.out



-- 
H.J.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests
  2021-11-08 15:42 [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
  2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
  2021-11-08 15:42 ` [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests " H.J. Lu
@ 2021-11-11 13:46 ` H.J. Lu
  2021-11-11 14:05   ` Adhemerval Zanella
  2 siblings, 1 reply; 15+ messages in thread
From: H.J. Lu @ 2021-11-11 13:46 UTC (permalink / raw)
  To: GNU C Library; +Cc: Florian Weimer

On Mon, Nov 8, 2021 at 7:42 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> One problem of using
>
> $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
> compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
> compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)
>
> to build DSO sorting relocatable objects is that $< may not be the source
> file with 3 "make -j 28" parallel builds on a machine with 112 cores.
> When generating Makefile fragments at build time, the same file may be
> generated more than once at the same time with parallel builds.
>
> 1. Update dso-ordering-test.py to put all sources for DSO sorting tests
> in a single directory, dso-sort-tests-src, and compile relocatable objects
> with
>
> $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
>         $(compile.c) $(OUTPUT_OPTION)
>
> 2. Generate Makefile fragments for DSO sorting tests at configure time
> to avoid generate them in the elf directory at build time.
>
> H.J. Lu (2):
>   dso-ordering-test.py: Put all sources in one directory [BZ #28550]
>   elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
>
>  configure                    | 11 +++++++++
>  configure.ac                 | 12 +++++++++
>  elf/Makefile                 | 22 ++++-------------
>  scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
>  4 files changed, 62 insertions(+), 30 deletions(-)
>
> --
> 2.33.1
>

Any comments on the v2 patches?  "make check -j28" has been failing for me
on machines with many cores.   I'd like to resolve it soon.

-- 
H.J.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests
  2021-11-11 13:46 ` [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
@ 2021-11-11 14:05   ` Adhemerval Zanella
  2021-11-15 14:43     ` H.J. Lu
  0 siblings, 1 reply; 15+ messages in thread
From: Adhemerval Zanella @ 2021-11-11 14:05 UTC (permalink / raw)
  To: H.J. Lu, GNU C Library; +Cc: Florian Weimer



On 11/11/2021 10:46, H.J. Lu via Libc-alpha wrote:
> On Mon, Nov 8, 2021 at 7:42 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>>
>> One problem of using
>>
>> $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
>> compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
>> compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)
>>
>> to build DSO sorting relocatable objects is that $< may not be the source
>> file with 3 "make -j 28" parallel builds on a machine with 112 cores.
>> When generating Makefile fragments at build time, the same file may be
>> generated more than once at the same time with parallel builds.
>>
>> 1. Update dso-ordering-test.py to put all sources for DSO sorting tests
>> in a single directory, dso-sort-tests-src, and compile relocatable objects
>> with
>>
>> $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
>>         $(compile.c) $(OUTPUT_OPTION)
>>
>> 2. Generate Makefile fragments for DSO sorting tests at configure time
>> to avoid generate them in the elf directory at build time.
>>
>> H.J. Lu (2):
>>   dso-ordering-test.py: Put all sources in one directory [BZ #28550]
>>   elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
>>
>>  configure                    | 11 +++++++++
>>  configure.ac                 | 12 +++++++++
>>  elf/Makefile                 | 22 ++++-------------
>>  scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
>>  4 files changed, 62 insertions(+), 30 deletions(-)
>>
>> --
>> 2.33.1
>>
> 
> Any comments on the v2 patches?  "make check -j28" has been failing for me
> on machines with many cores.   I'd like to resolve it soon.
> 

It is on my list, just sorting out Florian comments to rtld-audit patchset.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests
  2021-11-11 14:05   ` Adhemerval Zanella
@ 2021-11-15 14:43     ` H.J. Lu
  0 siblings, 0 replies; 15+ messages in thread
From: H.J. Lu @ 2021-11-15 14:43 UTC (permalink / raw)
  To: Adhemerval Zanella; +Cc: GNU C Library, Florian Weimer

On Thu, Nov 11, 2021 at 6:05 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 11/11/2021 10:46, H.J. Lu via Libc-alpha wrote:
> > On Mon, Nov 8, 2021 at 7:42 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> >>
> >> One problem of using
> >>
> >> $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c)
> >> compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags)
> >> compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS)
> >>
> >> to build DSO sorting relocatable objects is that $< may not be the source
> >> file with 3 "make -j 28" parallel builds on a machine with 112 cores.
> >> When generating Makefile fragments at build time, the same file may be
> >> generated more than once at the same time with parallel builds.
> >>
> >> 1. Update dso-ordering-test.py to put all sources for DSO sorting tests
> >> in a single directory, dso-sort-tests-src, and compile relocatable objects
> >> with
> >>
> >> $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c
> >>         $(compile.c) $(OUTPUT_OPTION)
> >>
> >> 2. Generate Makefile fragments for DSO sorting tests at configure time
> >> to avoid generate them in the elf directory at build time.
> >>
> >> H.J. Lu (2):
> >>   dso-ordering-test.py: Put all sources in one directory [BZ #28550]
> >>   elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550]
> >>
> >>  configure                    | 11 +++++++++
> >>  configure.ac                 | 12 +++++++++
> >>  elf/Makefile                 | 22 ++++-------------
> >>  scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++----------
> >>  4 files changed, 62 insertions(+), 30 deletions(-)
> >>
> >> --
> >> 2.33.1
> >>
> >
> > Any comments on the v2 patches?  "make check -j28" has been failing for me
> > on machines with many cores.   I'd like to resolve it soon.
> >
>
> It is on my list, just sorting out Florian comments to rtld-audit patchset.

This bug has been causing false negative results with "makc check -jN"
on machines with many cores.  I will check it in on Tuesday if there are
no objections.

Thanks.

-- 
H.J.

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-11-16 13:15 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-08 15:42 [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
2021-11-08 15:42 ` [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] H.J. Lu
2021-11-11 17:52   ` Chung-Lin Tang
2021-11-11 18:21     ` H.J. Lu
2021-11-15 18:50   ` Adhemerval Zanella
2021-11-08 15:42 ` [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests " H.J. Lu
2021-11-15 18:59   ` Adhemerval Zanella
2021-11-15 19:51     ` H.J. Lu
2021-11-15 20:18       ` Adhemerval Zanella
2021-11-16  4:48         ` [PATCH] elf: Use a temporary file to generate Makefile fragments " H.J. Lu
2021-11-16 12:52           ` Adhemerval Zanella
2021-11-16 13:15             ` H.J. Lu
2021-11-11 13:46 ` [PATCH v2 0/2] Update Makefile fragments for DSO sorting tests H.J. Lu
2021-11-11 14:05   ` Adhemerval Zanella
2021-11-15 14:43     ` H.J. Lu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).