public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug go/104832] New: gccgo / libgo Reproducibility Problem
@ 2022-03-08  1:54 toolybird at tuta dot io
  2022-03-08  8:10 ` [Bug go/104832] " schwab@linux-m68k.org
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: toolybird at tuta dot io @ 2022-03-08  1:54 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104832

            Bug ID: 104832
           Summary: gccgo / libgo Reproducibility Problem
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: go
          Assignee: ian at airs dot com
          Reporter: toolybird at tuta dot io
                CC: cmang at google dot com
  Target Milestone: ---

TL;DR -> Different binaries are produced depending on whether gccgo is already
installed on the host or not.

Background: I'm looking into broad issues of reproducibility in the context of
bootstrapping whole toolchains.

I first noticed this problem with gcc-11.2.0 but it's still present on GCC
trunk.

To reproduce, you will need to have the same version of gccgo installed on your
host in /usr.

For example, with current GCC trunk:

$ cat gcc/BASE-VER
$ 12.0.1

Step 1:
bootstrap GCC with `--enable-languages=c,c++,go --prefix=/usr
--libexecdir=/usr/lib' then install into a temp DESTDIR.

Step 2:
Uninstall gccgo from the host, or you can fudge it with:

$ sudo mv /usr/lib/go /usr/lib/go.XX

Step 3:
bootstrap GCC again with `--enable-languages=c,c++,go --prefix=/usr
--libexecdir=/usr/lib' then install into a 2nd temp DESTDIR.

Step 4: diff / compare the binaries in the DESTDIRs and you will see
differences.

Delving deeper and diffing the gcc-build dirs, there are a number of object
files affected. A list sorted with `ls -altr' is at the end of this report.

Here is a demonstration with "libgo/fmt.o" (although I suspect the real problem
is further up the chain):

$ cd gcc-build/x86_64-pc-linux-gnu/libgo

$ /build/gcc/src/gcc-build/./gcc/gccgo -B/build/gcc/src/gcc-build/./gcc/
-minline-all-stringops -O2 -g -I . -c -fgo-pkgpath=fmt
../../../gcc/libgo/go/fmt/doc.go ../../../gcc/libgo/go/fmt/errors.go
../../../gcc/libgo/go/fmt/format.go ../../../gcc/libgo/go/fmt/print.go
../../../gcc/libgo/go/fmt/scan.go -o fmt-test1.o

$ sudo mv /usr/lib/go /usr/lib/go.XX

$ /build/gcc/src/gcc-build/./gcc/gccgo -B/build/gcc/src/gcc-build/./gcc/
-minline-all-stringops -O2 -g -I . -c -fgo-pkgpath=fmt
../../../gcc/libgo/go/fmt/doc.go ../../../gcc/libgo/go/fmt/errors.go
../../../gcc/libgo/go/fmt/format.go ../../../gcc/libgo/go/fmt/print.go
../../../gcc/libgo/go/fmt/scan.go -o fmt-test2.o

$ diff fmt-test1.o fmt-test2.o
Binary files fmt-test1.o and fmt-test2.o differ

The differences are in the ".go_export" section of the ELF file.

Using "strings" helps to visualize the differences:

$ strings fmt-test1.o > 1.txt
$ strings fmt-test2.o > 2.txt
$ diff -u 1.txt 2.txt
--- 1.txt       2022-03-08 11:07:01.577545105 +1100
+++ 2.txt       2022-03-08 11:07:06.364263769 +1100
@@ -860,7 +860,7 @@
 import utf8 unicode/utf8 "unicode/utf8"
 init fmt fmt..import errors errors..import cpu internal_1cpu..import oserror
internal_1oserror..import poll internal_1poll..import reflectlite
internal_1reflectlite..import testlog internal_1testlog..import io io..import
fs io_1fs..import math math..import os os..import path path..import reflect
reflect..import runtime runtime..import sort sort..import strconv
strconv..import sync sync..import syscall syscall..import time time..import
unicode unicode..import abi ~internal_1abi bytealg ~internal_1bytealg fmtsort
~internal_1fmtsort goarch ~internal_1goarch goexperiment
~internal_1goexperiment goos ~internal_1goos itoa ~internal_1itoa race
~internal_1race execenv ~internal_1syscall_1execenv unix
~internal_1syscall_1unix unsafeheader ~internal_1unsafeheader bits ~math_1bits
atomic ~runtime_1internal_1atomic math ~runtime_1internal_1math sys
~runtime_1internal_1sys atomic ~sync_1atomic utf8 ~unicode_1utf8
 init_graph 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13 0 14 0 15 0
16 0 17 0 18 0 19 1 2 1 5 1 13 3 1 3 2 3 5 3 13 4 1 4 2 4 3 4 5 4 7 4 13 4 16 4
17 4 18 5 2 5 13 6 2 6 13 6 16 7 1 7 2 7 5 7 13 7 16 8 1 8 2 8 3 8 5 8 7 8 11 8
13 8 14 8 16 8 17 8 18 9 2 10 1 10 2 10 3 10 4 10 5 10 6 10 7 10 8 10 11 10 13
10 14 10 16 10 17 10 18 11 1 11 2 11 5 11 13 12 1 12 2 12 5 12 9 12 13 12 15 12
16 12 19 13 2 14 2 14 5 14 13 15 1 15 2 15 5 15 9 15 13 16 2 16 13 17 1 17 2 17
3 17 5 17 13 17 16 18 1 18 2 18 3 18 5 18 13 18 16 18 17
-types 41 7 29 30 29 27 25 28 1067 3172 30 335 26 30 34 35 30 88 92 47 88 47 34
34 18 18 19 115 171 21 22 195 60 46 83 61 73 295 73 44 74 22
+types 41 7 29 30 29 27 25 28 1067 3172 30 335 26 30 34 35 30 88 92 47 88 47 38
34 18 18 19 115 171 21 22 195 60 46 83 61 73 295 73 44 74 22
 type 1 "Formatter" <type 31>
 type 2 "GoStringer" <type 32>
 type 3 "ScanState" <type 36>
@@ -972,7 +972,7 @@
 type 18 (? <type 29>, ? <type -21>) <type -11>
 type 19 (? <type 29>, ? <type -16>, ? <type -20>, ? <type -15>, ? <type -15>)
<type 29>
 type 20 (? <type -10>, ? <type -3>) <type -10>
-type 21 (? <type -21>) <type -15>
+type 21 (? <type -21>) (? <type -15>)
 type 22 (? <type -16>) <type -19>
 type 23 *<type 7>
 type 24 *<type 8>
@@ -1067,7 +1067,7 @@
   $ret10 = $convert(<type -15>, $false) //577
   return //577
  } //0

-checksum 22D2C8F1A933DC5F8B8E981917B8ADA04DEB2FC1
+checksum A916C151E089AA7EC237CBEBBE2B293A9595478C
 ScanState's Read should not be called. Use ReadRune
 fmt: scanning called UnreadRune with no rune available
 0123456789ABCDEFX


strace shows the host dirs being accessed:

First case:
newfstatat(AT_FDCWD, "/build/gcc/src/gcc-build/./gcc/go/12.0.1",
0x7ffc48250da0, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1/x86_64-pc-linux-gnu",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1/x86_64-pc-linux-gnu",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0

Second case:
newfstatat(AT_FDCWD, "/build/gcc/src/gcc-build/./gcc/go/12.0.1",
0x7ffee9c2a960, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/../lib/go/12.0.1", 0x7ffee9c2a960, 0) = -1 ENOENT
(No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/../lib/go/12.0.1", 0x7ffee9c2a960, 0) = -1
ENOENT (No such file or directory)

Any thoughts?
Thanks


gcc-build/x86_64-pc-linux-gnu/libgo/.libs/runtime.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/syscall.o
gcc-build/x86_64-pc-linux-gnu/libgo/syscall.o
gcc-build/x86_64-pc-linux-gnu/libgo/index/suffixarray.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/os.o
gcc-build/x86_64-pc-linux-gnu/libgo/os.o
gcc-build/x86_64-pc-linux-gnu/libgo/.libs/fmt.o
gcc-build/x86_64-pc-linux-gnu/libgo/fmt.o
gcc-build/x86_64-pc-linux-gnu/libgo/image/.libs/gif.o
gcc-build/x86_64-pc-linux-gnu/libgo/image/gif.o
gcc-build/x86_64-pc-linux-gnu/libgo/archive/.libs/zip.o
gcc-build/x86_64-pc-linux-gnu/libgo/archive/zip.o
gcc-build/x86_64-pc-linux-gnu/libgo/debug/.libs/macho.o
gcc-build/x86_64-pc-linux-gnu/libgo/text/.libs/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/testing.o
gcc-build/x86_64-pc-linux-gnu/libgo/text/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/go/ast.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/mail.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/mail.o
gcc-build/x86_64-pc-linux-gnu/libgo/html/.libs/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/html/template.o
gcc-build/x86_64-pc-linux-gnu/libgo/crypto/.libs/x509.o
gcc-build/x86_64-pc-linux-gnu/libgo/crypto/tls.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/http.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/http.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/.libs/rpc.o
gcc-build/x86_64-pc-linux-gnu/libgo/net/rpc.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/modfetch.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/help.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/workcmd.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/work.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/clean.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/get.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/envcmd.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/bug.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/modget.o
gcc-build/x86_64-pc-linux-gnu/libgo/cmd/go/internal/test.o

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-03-27  2:27 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-08  1:54 [Bug go/104832] New: gccgo / libgo Reproducibility Problem toolybird at tuta dot io
2022-03-08  8:10 ` [Bug go/104832] " schwab@linux-m68k.org
2022-03-08 21:54 ` ian at airs dot com
2022-03-09  5:24 ` toolybird at tuta dot io
2022-03-09  6:10 ` ian at airs dot com
2022-03-09  6:36 ` toolybird at tuta dot io
2022-03-09  6:37 ` toolybird at tuta dot io
2022-03-27  2:27 ` toolybird at tuta dot io

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).