Go patch committed: Implement new syscall package

Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

This patch is a rewrite of the syscall package in the Go library.  This
rewrite moves it from libgo/syscalls to libgo/go/syscall, to more
closely match the master Go library.  More importantly, it changes most
library calls to call new entersyscall and exitsyscall functions.  These
functions currently do nothing.  However, they are a step toward
multiplexing multiple goroutines onto a single thread, which will make
the implementation of goroutines more efficient.  When multiplexing
goroutines, it is of course essential to know when a goroutine is
calling a library function which may block.  This patch makes that
possible.

There are still some existing calls to possibly blocking library
functions in other parts of the library.  Those will also have to be
updated.

It's possible that this patch will once again break the Solaris and Irix
support.  I've tried to ensure that I didn't make any stupid errors, but
I haven't done any actual testing.  Sorry about any problems.

Bootstrapped and ran Go testsuite on x86_64-unknown-linux-gnu.
Committed to mainline.

Ian


[-- Attachment #2: patch --]
[-- Type: application/x-bzip, Size: 38298 bytes --]

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

Ian,

> It's possible that this patch will once again break the Solaris and Irix
> support.  I've tried to ensure that I didn't make any stupid errors, but
> I haven't done any actual testing.  Sorry about any problems.

it did (as expected :-), but in easy to fix ways (at least for getting
libgo to compile again):

* go/syscall/wait.c doesn't compile on Solaris (cannot test IRIX right
  now since bootstrap is broken):

/vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c: In function 'TrapCause':
/vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:98:22: error: unused parameter 'w' [-Werror=unused-parameter]

  I'm marking the arg as unused to work around this since including it
  in #ifndef __linux__ seemed uglier.  I'll check if I can find the trap
  cause from the wait status, but a quick check of Solaris manpages and
  headers found nothing (at least not via wait* functions).

* sys/syscall/exec_unix.go doesn't compile either:

/vol/gcc/src/hg/trunk/local/libgo/go/syscall/exec_unix.go:163:21: error: reference to undefined name '_PTRACE_TRACEME'

  forkAndExecInChild uses _PTRACE_TRACEME, but mksysinfo only emitted
  PTRACE_TRACEME if _PTRACE_TRACEME is missing.

Unfortunately, testsuite results are a mess: all link tests fail like
this:

Undefined			first referenced
 symbol  			    in file
flock                               /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/libgo/.libs/libgo.so
ld: fatal: symbol referencing errors. No output written to a.out
collect2: error: ld returned 1 exit status
FAIL: asn1

The 64-bit tests are even worse:

Undefined			first referenced
 symbol  			    in file
flock                               /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/amd64/libgo/.libs/libgo.so
ptrace                              /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/amd64/libgo/.libs/libgo.so
ld: fatal: symbol referencing errors. No output written to a.out
collect2: error: ld returned 1 exit status
FAIL: asn1

I've not yet checked how to avoid this, but at least the ptrace stuff
worked (as in: didn't try to use 64-bit ptrace which doesn't exist)
before.

	Rainer


2011-10-25  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	libgo:
	* go/syscall/wait.c (TrapCause): Declare w unused.
	* mksysinfo.sh: Emit _PTRACE_TRACEME if missing.

diff --git a/libgo/go/syscall/wait.c b/libgo/go/syscall/wait.c
--- a/libgo/go/syscall/wait.c
+++ b/libgo/go/syscall/wait.c
@@ -95,7 +95,7 @@ extern int TrapCause (uint32_t *w)
   __asm__ ("libgo_syscall.syscall.TrapCause.N32_libgo_syscall.syscall.WaitStatus");
 
 int
-TrapCause (uint32_t *w)
+TrapCause (uint32_t *w __attribute__ ((unused)))
 {
 #ifndef __linux__
   return -1;
diff --git a/libgo/mksysinfo.sh b/libgo/mksysinfo.sh
--- a/libgo/mksysinfo.sh
+++ b/libgo/mksysinfo.sh
@@ -251,7 +251,7 @@ if ! grep '^const PTRACE_EVENT_EXIT' ${O
   echo "const PTRACE_EVENT_EXIT = 6" >> ${OUT}
 fi
 if ! grep '^const _PTRACE_TRACEME' ${OUT} > /dev/null 2>&1; then
-  echo "const PTRACE_TRACEME = 0" >> ${OUT}
+  echo "const _PTRACE_TRACEME = 0" >> ${OUT}
 fi
 
 # The registers returned by PTRACE_GETREGS.  This is probably


-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> it did (as expected :-), but in easy to fix ways (at least for getting
> libgo to compile again):

Thanks.  I committed your patch.


> Unfortunately, testsuite results are a mess: all link tests fail like
> this:
>
> Undefined			first referenced
>  symbol  			    in file
> flock                               /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/libgo/.libs/libgo.so
> ld: fatal: symbol referencing errors. No output written to a.out
> collect2: error: ld returned 1 exit status
> FAIL: asn1
>
> The 64-bit tests are even worse:
>
> Undefined			first referenced
>  symbol  			    in file
> flock                               /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/amd64/libgo/.libs/libgo.so
> ptrace                              /var/gcc/regression/trunk/10-gcc/build/i386-pc-solaris2.10/amd64/libgo/.libs/libgo.so
> ld: fatal: symbol referencing errors. No output written to a.out
> collect2: error: ld returned 1 exit status
> FAIL: asn1
>
> I've not yet checked how to avoid this, but at least the ptrace stuff
> worked (as in: didn't try to use 64-bit ptrace which doesn't exist)
> before.

I committed this patch to mainline to try to fix these problems.  Thanks
for testing.  For this patch I bootstrapped and ran Go testsuite on
x86_64-unknown-linux-gnu.

Ian


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 4954 bytes --]

diff -r f02b62d1dcea libgo/go/syscall/exec_stubs.go
--- a/libgo/go/syscall/exec_stubs.go	Tue Oct 25 10:45:29 2011 -0700
+++ b/libgo/go/syscall/exec_stubs.go	Tue Oct 25 10:56:37 2011 -0700
@@ -17,3 +17,7 @@
 func Wait4(pid int, wstatus *WaitStatus, options int, rusage *Rusage) (wpid int, errno int) {
 	return -1, ENOSYS;
 }
+
+func raw_ptrace(request int, pid int, addr *byte, data *byte) int {
+	return ENOSYS
+}
diff -r f02b62d1dcea libgo/go/syscall/exec_unix.go
--- a/libgo/go/syscall/exec_unix.go	Tue Oct 25 10:45:29 2011 -0700
+++ b/libgo/go/syscall/exec_unix.go	Tue Oct 25 10:56:37 2011 -0700
@@ -16,9 +16,6 @@
 //sysnb	raw_fork() (pid Pid_t, errno int)
 //fork() Pid_t
 
-//sysnb raw_ptrace(request int, pid int, addr *byte, data *byte) (errno int)
-//ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
-
 //sysnb raw_setsid() (errno int)
 //setsid() Pid_t
 
diff -r f02b62d1dcea libgo/go/syscall/libcall_irix.go
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/libgo/go/syscall/libcall_irix.go	Tue Oct 25 10:56:37 2011 -0700
@@ -0,0 +1,8 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package syscall
+
+//sysnb raw_ptrace(request int, pid int, addr *byte, data *byte) (errno int)
+//ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
diff -r f02b62d1dcea libgo/go/syscall/libcall_linux.go
--- a/libgo/go/syscall/libcall_linux.go	Tue Oct 25 10:45:29 2011 -0700
+++ b/libgo/go/syscall/libcall_linux.go	Tue Oct 25 10:56:37 2011 -0700
@@ -29,6 +29,9 @@
 //sys	ptrace(request int, pid int, addr uintptr, data uintptr) (errno int)
 //ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
 
+//sysnb raw_ptrace(request int, pid int, addr *byte, data *byte) (errno int)
+//ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
+
 func ptracePeek(req int, pid int, addr uintptr, out []byte) (count int, errno int) {
 	// The peek requests are machine-size oriented, so we wrap it
 	// to retrieve arbitrary-length data.
@@ -192,6 +195,9 @@
 //sys	Fchownat(dirfd int, path string, uid int, gid int, flags int) (errno int)
 //fchownat(dirfd int, path *byte, owner Uid_t, group Gid_t, flags int) int
 
+//sys	Flock(fd int, how int) (errno int)
+//flock(fd int, how int) int
+
 // FIXME: mksysinfo statfs
 // //sys	Fstatfs(fd int, buf *Statfs_t) (errno int)
 // //fstatfs(fd int, buf *Statfs_t) int
diff -r f02b62d1dcea libgo/go/syscall/libcall_posix.go
--- a/libgo/go/syscall/libcall_posix.go	Tue Oct 25 10:45:29 2011 -0700
+++ b/libgo/go/syscall/libcall_posix.go	Tue Oct 25 10:56:37 2011 -0700
@@ -199,9 +199,6 @@
 //sys	Fdatasync(fd int) (errno int)
 //fdatasync(fd int) int
 
-//sys	Flock(fd int, how int) (errno int)
-//flock(fd int, how int) int
-
 //sys	Fsync(fd int) (errno int)
 //fsync(fd int) int
 
diff -r f02b62d1dcea libgo/go/syscall/libcall_solaris_386.go
--- a/libgo/go/syscall/libcall_solaris_386.go	Tue Oct 25 10:45:29 2011 -0700
+++ b/libgo/go/syscall/libcall_solaris_386.go	Tue Oct 25 10:56:37 2011 -0700
@@ -7,3 +7,6 @@
 // 32-bit Solaris 2/x86 needs to use _nuname internally, cf. <sys/utsname.h>.
 //sysnb	Uname(buf *Utsname) (errno int)
 //_nuname(buf *Utsname) int
+
+//sysnb raw_ptrace(request int, pid int, addr *byte, data *byte) (errno int)
+//ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
diff -r f02b62d1dcea libgo/go/syscall/libcall_solaris_amd64.go
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/libgo/go/syscall/libcall_solaris_amd64.go	Tue Oct 25 10:56:37 2011 -0700
@@ -0,0 +1,10 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package syscall
+
+// 64-bit ptrace(3C) doesn't exist
+func raw_ptrace(request int, pid int, addr *byte, data *byte) int {
+	return ENOSYS
+}
diff -r f02b62d1dcea libgo/go/syscall/libcall_solaris_sparc.go
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/libgo/go/syscall/libcall_solaris_sparc.go	Tue Oct 25 10:56:37 2011 -0700
@@ -0,0 +1,8 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package syscall
+
+//sysnb raw_ptrace(request int, pid int, addr *byte, data *byte) (errno int)
+//ptrace(request int, pid Pid_t, addr *byte, data *byte) _C_long
diff -r f02b62d1dcea libgo/go/syscall/libcall_solaris_sparc64.go
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/libgo/go/syscall/libcall_solaris_sparc64.go	Tue Oct 25 10:56:37 2011 -0700
@@ -0,0 +1,10 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package syscall
+
+// 64-bit ptrace(3C) doesn't exist
+func raw_ptrace(request int, pid int, addr *byte, data *byte) int {
+	return ENOSYS
+}

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

Ian,

> I committed this patch to mainline to try to fix these problems.  Thanks
> for testing.  For this patch I bootstrapped and ran Go testsuite on
> x86_64-unknown-linux-gnu.

with this patch, go and libgo results on Solaris 10 and 11/x86 are back
to normal, and Solaris 10 and 11/SPARC bootstraps are currently
running.  Thanks.

There's one problem left: with Solaris nawk, building libcalls.go fails
(from Solaris 8 to 11 inclusive):

nawk -f /vol/gcc/src/hg/trunk/local/libgo/go/syscall/mksyscall.awk ${files} > li
bcalls.go.tmp
nawk: syntax error at source line 47
 context is
            if (match($0, "//sys(nb)?[  ]*([a-zA-Z0-9_]+)\\(([^()]*)\\) >>>  *(\
\(([^()]+)\\))?", <<<  gosig) == 0) {
nawk: illegal statement at source line 47
nawk: syntax error at source line 58
        missing }
make[4]: *** [s-libcalls] Error 2

I don't yet see what's wrong with the pattern, and gawk does accept it.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> There's one problem left: with Solaris nawk, building libcalls.go fails
> (from Solaris 8 to 11 inclusive):
>
> nawk -f /vol/gcc/src/hg/trunk/local/libgo/go/syscall/mksyscall.awk ${files} > li
> bcalls.go.tmp
> nawk: syntax error at source line 47
>  context is
>             if (match($0, "//sys(nb)?[  ]*([a-zA-Z0-9_]+)\\(([^()]*)\\) >>>  *(\
> \(([^()]+)\\))?", <<<  gosig) == 0) {
> nawk: illegal statement at source line 47
> nawk: syntax error at source line 58
>         missing }
> make[4]: *** [s-libcalls] Error 2
>
> I don't yet see what's wrong with the pattern, and gawk does accept it.

Hmmm, I don't have a copy of nawk.  Is it possible that it doesn't like
" *" == <space>*?  You could try "[ ]*".

Ian

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

Ian,

> Hmmm, I don't have a copy of nawk.  Is it possible that it doesn't like
> " *" == <space>*?  You could try "[ ]*".

the problem is another one: using /usr/xpg4/bin/awk, I find:

/usr/xpg4/bin/awk: line 47 (NR=32): wrong number of arguments to function "m"

nawk(1) only documents match(s,ere) (i.e. two args), and the gawk docs
state:

`match(STRING, REGEXP [, ARRAY])'
[...]
     The ARRAY argument to `match' is a `gawk' extension.  In
     compatibility mode (*note Options::), using a third argument is a
     fatal error.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> the problem is another one: using /usr/xpg4/bin/awk, I find:
>
> /usr/xpg4/bin/awk: line 47 (NR=32): wrong number of arguments to function "m"
>
> nawk(1) only documents match(s,ere) (i.e. two args), and the gawk docs
> state:
>
> `match(STRING, REGEXP [, ARRAY])'
> [...]
>      The ARRAY argument to `match' is a `gawk' extension.  In
>      compatibility mode (*note Options::), using a third argument is a
>      fatal error.

Ah.  I took a look at the nawk man page, and I don't see any evidence
that it supports submatches at all.  How annoying.

Ian

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> the problem is another one: using /usr/xpg4/bin/awk, I find:
>
> /usr/xpg4/bin/awk: line 47 (NR=32): wrong number of arguments to function "m"
>
> nawk(1) only documents match(s,ere) (i.e. two args), and the gawk docs
> state:
>
> `match(STRING, REGEXP [, ARRAY])'
> [...]
>      The ARRAY argument to `match' is a `gawk' extension.  In
>      compatibility mode (*note Options::), using a third argument is a
>      fatal error.

I committed this patch which should fix this problem.  Bootstrapped and
ran Go testsuite on x86_64-unknown-linux-gnu.

Ian


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 3109 bytes --]

diff -r 5f4f4eae5cd9 libgo/go/syscall/mksyscall.awk
--- a/libgo/go/syscall/mksyscall.awk	Wed Oct 26 16:19:57 2011 -0700
+++ b/libgo/go/syscall/mksyscall.awk	Wed Oct 26 21:53:05 2011 -0700
@@ -44,33 +44,63 @@
 	blocking = 1
     }
 
-    if (match($0, "//sys(nb)?[ 	]*([a-zA-Z0-9_]+)\\(([^()]*)\\) *(\\(([^()]+)\\))?", gosig) == 0) {
+    line = $0
+
+    if (match(line, "//sys(nb)?[ 	]*[a-zA-Z0-9_]+\\([^()]*\\) *(\\(([^()]+)\\))?") == 0) {
 	print "unmatched line:", $0 | "cat 1>&2"
 	status = 1
 	next
     }
 
-    gofnname = gosig[2]
-    gofnparams = gosig[3]
-    gofnresults = gosig[5]
+    # Sets a[1] = //sysnb, a[2] == function name.
+    split(line, a, "[ 	(]*")
+    gofnname = a[2]
+
+    off = match(line, "\\([^()]*\\)")
+    end = index(substr(line, off, length(line) - off + 1), ")")
+    gofnparams = substr(line, off + 1, end - 2)
+
+    line = substr(line, off + end, length(line) - (off + end) + 1)
+    off = match(line, "\\([^()]*\\)")
+    if (off == 0) {
+	gofnresults = ""
+    } else {
+	end = index(substr(line, off, length(line) - off + 1), ")")
+	gofnresults = substr(line, off + 1, end - 2)
+    }
 
     getline
+    line = $0
 
-    if (match($0, "//([a-zA-Z0-9_]+)\\(([^()]*)\\) *(.*)$", csig) == 0) {
+    if (match(line, "//[a-zA-Z0-9_]+\\([^()]*\\)") == 0) {
 	print "unmatched C line", $0, "after", gofnname | "cat 1>&2"
 	status = 1
 	next
     }
 
-    cfnname = csig[1]
-    cfnparams = csig[2]
-    cfnresult = csig[3]
+    split(line, a, "[ 	(]*")
+    cfnname = substr(a[1], 3, length(a[1]) - 2)
+
+    off = match(line, "\\([^()]*\\)")
+    end = index(substr(line, off, length(line) - off + 1), ")")
+    cfnparams = substr(line, off + 1, end - 2)
+
+    line = substr(line, off + end + 1, length(line) - (off + end) + 1)
+    while (substr(line, 1, 1) == " ") {
+	line = substr(line, 2, length(line) - 1)
+    }
+    end = index(line, " ")
+    if (end != 0) {
+	line = substr(line, 1, end)
+    }
+    cfnresult = line
 
     printf("// Automatically generated wrapper for %s/%s\n", gofnname, cfnname)
     printf("func c_%s(%s) %s%s__asm__(\"%s\")\n",
 	   cfnname, cfnparams, cfnresult, cfnresult == "" ? "" : " ", cfnname)
-    printf("func %s(%s) %s%s{\n",
-	   gofnname, gofnparams, gosig[4], gosig[4] == "" ? "" : " ")
+    printf("func %s(%s) %s%s%s%s{\n",
+	   gofnname, gofnparams, gofnresults == "" ? "" : "(", gofnresults,
+	   gofnresults == "" ? "" : ")", gofnresults == "" ? "" : " ")
 
     if (blocking) {
 	print "\tentersyscall()"
@@ -91,22 +121,22 @@
 	    args = args ", "
 	}
 
-	if (match(goargs[goarg], "^([^ ]*) ([^ ]*)$", goparam) == 0) {
+	if (split(goargs[goarg], a) != 2) {
 	    print loc, "bad parameter:", goargs[goarg] | "cat 1>&2"
 	    status = 1
 	    next
 	}
 
-	goname = goparam[1]
-	gotype = goparam[2]
+	goname = a[1]
+	gotype = a[2]
 
-	if (match(cargs[carg], "^([^ ]*) ([^ ]*)$", cparam) == 0) {
+	if (split(cargs[carg], a) != 2) {
 	    print loc, "bad C parameter:", cargs[carg] | "cat 1>&2"
 	    status = 1
 	    next
 	}
 
-	ctype = cparam[2]
+	ctype = a[2]
 
 	if (gotype ~ /^\*/) {
 	    if (gotype != ctype) {

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

Ian,

> I committed this patch which should fix this problem.  Bootstrapped and
> ran Go testsuite on x86_64-unknown-linux-gnu.

thanks, but this is not enough:

nawk: syntax error at source line 173
 context is
         ([^ >>>  ]*)$", <<<  cparam) == 0) {
nawk: illegal statement at source line 173
nawk: syntax error at source line 179

and there is another instance on l.210.  I haven't tried fixing this
myself since I'm fighting with other issues.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

Ian,

>> I committed this patch which should fix this problem.  Bootstrapped and
>> ran Go testsuite on x86_64-unknown-linux-gnu.
>
> thanks, but this is not enough:
>
> nawk: syntax error at source line 173
>  context is
>          ([^ >>>  ]*)$", <<<  cparam) == 0) {
> nawk: illegal statement at source line 173
> nawk: syntax error at source line 179
>
> and there is another instance on l.210.  I haven't tried fixing this
> myself since I'm fighting with other issues.

even if I work around this by installing gawk 4.0.0 on Solaris 8/9, I
run into another issue:

/vol/gcc/src/hg/trunk/local/libgo/go/syscall/errstr_nor.go:22:8: error: referenc
e to undefined name 'libc_strerror'
make[4]: *** [syscall/syscall.lo] Error 1

Replacing libc_strerror (which doesn't exist anywhere) by strerror isn't
enough, though:

/vol/gcc/src/hg/trunk/local/libgo/go/syscall/errstr_nor.go:22:2: error: variable has no type
/vol/gcc/src/hg/trunk/local/libgo/go/syscall/errstr_nor.go:22:2: error: incompatible type in initialization (non-value used as value)
make[2]: *** [syscall/syscall.lo] Error 1

I couldn't figure out what's wrong here; I'll need considerable more
time with the Go tutorial etc.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> Ian,
>
>> I committed this patch which should fix this problem.  Bootstrapped and
>> ran Go testsuite on x86_64-unknown-linux-gnu.
>
> thanks, but this is not enough:
>
> nawk: syntax error at source line 173
>  context is
>          ([^ >>>  ]*)$", <<<  cparam) == 0) {
> nawk: illegal statement at source line 173
> nawk: syntax error at source line 179
>
> and there is another instance on l.210.  I haven't tried fixing this
> myself since I'm fighting with other issues.

Whoops, my patch was incomplete.  Sorry about that.  Fixed by this
patch.  Bootstrapped on x86_64-unknown-linux-gnu and saw that it did not
change the code generated by the script.  Committed to mainline.

Ian


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 748 bytes --]

diff -r a880b911554e libgo/go/syscall/mksyscall.awk
--- a/libgo/go/syscall/mksyscall.awk	Fri Oct 28 15:05:12 2011 -0700
+++ b/libgo/go/syscall/mksyscall.awk	Mon Oct 31 14:41:12 2011 -0700
@@ -170,7 +170,7 @@
 	    printf("\t}\n")
 
 	    ++carg
-	    if (match(cargs[carg], "^([^ ]*) ([^ ]*)$", cparam) == 0) {
+	    if (split(cargs[carg], cparam) != 2) {
 		print loc, "bad C parameter:", cargs[carg] | "cat 1>&2"
 		status = 1
 		next
@@ -207,7 +207,7 @@
 	}
 	usedr = 0
 	for (goresult = 1; goresults[goresult] != ""; goresult++) {
-	    if (match(goresults[goresult], "^([^ ]*) ([^ ]*)$", goparam) == 0) {
+	    if (split(goresults[goresult], goparam) != 2) {
 		print loc, "bad result:", goresults[goresult] | "cat 1>&2"
 		status = 1
 		next

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/errstr_nor.go:22:8: error: referenc
> e to undefined name 'libc_strerror'
> make[4]: *** [syscall/syscall.lo] Error 1

Sorry about that.  I thought I had tested that, but evidently not.

Fixed like so.  Committed to mainline.

Ian


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 555 bytes --]

diff -r 56a1bd1d907a libgo/go/syscall/errstr_nor.go
--- a/libgo/go/syscall/errstr_nor.go	Mon Oct 31 14:41:55 2011 -0700
+++ b/libgo/go/syscall/errstr_nor.go	Mon Oct 31 14:53:22 2011 -0700
@@ -11,7 +11,7 @@
 	"unsafe"
 )
 
-//sysnb	strerror(errnum int) *byte
+//sysnb	strerror(errnum int) (buf *byte)
 //strerror(errnum int) *byte
 
 var errstr_lock sync.Mutex
@@ -19,7 +19,7 @@
 func Errstr(errno int) string {
 	errstr_lock.Lock()
 
-	bp := libc_strerror(errno)
+	bp := strerror(errno)
 	b := (*[1000]byte)(unsafe.Pointer(bp))
 	i := 0
 	for b[i] != 0 {

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

[-- Attachment #1: Type: text/plain, Size: 1370 bytes --]

Ian Lance Taylor <iant@google.com> writes:

> It's possible that this patch will once again break the Solaris and Irix
> support.  I've tried to ensure that I didn't make any stupid errors, but
> I haven't done any actual testing.  Sorry about any problems.

Now IRIX finally bootstrap again, I had to make two adjustments to have
libgo build there.

* go/syscall/wait.c doesn't compile:

/vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:11:0: error: "__EXTENSIONS__" redefined [-Werror]
/vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:1:0: note: this is the location of the previous definition

  I've wrapped the __EXTENSIONS__ definition in #ifndef/#endif, but
  think this is the wrong approach: definitions of _GNU_SOURCE,
  __EXTENSIONS__ and other platform-specific stuff should go into
  configure.ac or mksysinfo.sh, not individual sources.  There are more
  instances of this problem, but they don't hurt me on IRIX.

* There's a redefinition of IPMreq now:

sysinfo.go:3089:6: error: redefinition of 'IPMreq'
/vol/gcc/src/hg/trunk/local/libgo/go/syscall/socket_irix.go:81:6: note: previous definition of 'IPMreq' was here

  For some reason, sysinfo.go now gets the right definition, so the one
  in socket_irix.go isn't necessary any longer.

make check is still running, but it seems that all 64-bit tests fail, as
they do on Solaris/x86.

	Rainer



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: irix-libgo.patch --]
[-- Type: text/x-patch, Size: 1202 bytes --]

diff --git a/libgo/go/syscall/socket_irix.go b/libgo/go/syscall/socket_irix.go
--- a/libgo/go/syscall/socket_irix.go
+++ b/libgo/go/syscall/socket_irix.go
@@ -74,17 +74,9 @@ func BindToDevice(fd int, device string)
 	return ENOSYS
 }
 
-// struct ip_mreg is provived in <netinet/in.h>, but protected with _SGIAPI.
-// This could be enabled with -D_SGI_SOURCE, but conflicts with
-// -D_XOPEN_SOURCE=500 required for msg_control etc. in struct msghgr, so
-// simply provide it here.
-type IPMreq struct {
-	Multiaddr [4]byte
-	Interface [4]byte
-}
-
-// Similarly, <netdb.h> only provides struct addrinfo, AI_* and EAI_* if
-// _NO_XOPEN4 && _NO_XOPEN5.
+// <netdb.h> only provides struct addrinfo, AI_* and EAI_* if  _NO_XOPEN4
+// && _NO_XOPEN5, but -D_XOPEN_SOURCE=500 is required for msg_control etc.
+// in struct msghgr, so simply provide them here.
 type Addrinfo struct {
 	Ai_flags int32
 	Ai_family int32
diff --git a/libgo/go/syscall/wait.c b/libgo/go/syscall/wait.c
--- a/libgo/go/syscall/wait.c
+++ b/libgo/go/syscall/wait.c
@@ -8,7 +8,9 @@
    OS-independent.  */
 
 #define _GNU_SOURCE
+#ifndef __EXTENSIONS__
 #define __EXTENSIONS__
+#endif
 
 #include <stdint.h>
 #include <sys/wait.h>

[-- Attachment #3: Type: text/plain, Size: 144 bytes --]



-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> Now IRIX finally bootstrap again, I had to make two adjustments to have
> libgo build there.
>
> * go/syscall/wait.c doesn't compile:
>
> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:11:0: error: "__EXTENSIONS__" redefined [-Werror]
> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:1:0: note: this is the location of the previous definition
>
>   I've wrapped the __EXTENSIONS__ definition in #ifndef/#endif, but
>   think this is the wrong approach: definitions of _GNU_SOURCE,
>   __EXTENSIONS__ and other platform-specific stuff should go into
>   configure.ac or mksysinfo.sh, not individual sources.  There are more
>   instances of this problem, but they don't hurt me on IRIX.

Makes sense.  I have committed this patch to try to clean this up a bit.
Bootstrapped and ran Go testsuite on x86_64-unknown-linux-gnu.

Ian


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 2435 bytes --]

diff -r 870b56a3d07e libgo/Makefile.am
--- a/libgo/Makefile.am	Fri Dec 16 06:45:11 2011 -0800
+++ b/libgo/Makefile.am	Tue Dec 20 09:18:16 2011 -0800
@@ -37,7 +37,7 @@
 ACLOCAL_AMFLAGS = -I ./config -I ../config
 
 AM_CFLAGS = -fexceptions -fplan9-extensions $(SPLIT_STACK) $(WARN_CFLAGS) \
-	$(STRINGOPS_FLAG) \
+	$(STRINGOPS_FLAG) $(OSCFLAGS) \
 	-I $(srcdir)/../libgcc -I $(MULTIBUILDTOP)../../gcc/include
 
 if USING_SPLIT_STACK
diff -r 870b56a3d07e libgo/configure.ac
--- a/libgo/configure.ac	Fri Dec 16 06:45:11 2011 -0800
+++ b/libgo/configure.ac	Tue Dec 20 09:18:16 2011 -0800
@@ -277,23 +277,24 @@
 AC_SUBST(GO_SYSCALL_OS_FILE)
 AC_SUBST(GO_SYSCALL_OS_ARCH_FILE)
 
-dnl Some targets need special flags to build sysinfo.go.
+dnl Special flags used to generate sysinfo.go.
+OSCFLAGS="-D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
 case "$target" in
     mips-sgi-irix6.5*)
 	# IRIX 6 needs _XOPEN_SOURCE=500 for the XPG5 version of struct
 	# msghdr in <sys/socket.h>.
-	OSCFLAGS='-D_XOPEN_SOURCE=500'
+	OSCFLAGS="$OSCFLAGS -D_XOPEN_SOURCE=500"
 	;;
     *-*-solaris2.[[89]])
 	# Solaris 8/9 need this so struct msghdr gets the msg_control
 	# etc. fields in <sys/socket.h> (_XPG4_2).
-	OSCFLAGS='-D_XOPEN_SOURCE=500 -D_XOPEN_SOURCE_EXTENDED -D__EXTENSIONS__'
+	OSCFLAGS="$OSCFLAGS -D_XOPEN_SOURCE=500 -D_XOPEN_SOURCE_EXTENDED -D__EXTENSIONS__"
 	;;
     *-*-solaris2.1[[01]])
 	# Solaris 10+ needs this so struct msghdr gets the msg_control
 	# etc. fields in <sys/socket.h> (_XPG4_2).  _XOPEN_SOURCE=500 as
 	# above doesn't work with C99.
-	OSCFLAGS='-D_XOPEN_SOURCE=600 -D__EXTENSIONS__'
+	OSCFLAGS="$OSCFLAGS -D_XOPEN_SOURCE=600 -D__EXTENSIONS__"
 	;;
 esac
 AC_SUBST(OSCFLAGS)
diff -r 870b56a3d07e libgo/go/syscall/wait.c
--- a/libgo/go/syscall/wait.c	Fri Dec 16 06:45:11 2011 -0800
+++ b/libgo/go/syscall/wait.c	Tue Dec 20 09:18:16 2011 -0800
@@ -7,9 +7,6 @@
    We use C code to extract the wait status so that we can easily be
    OS-independent.  */
 
-#define _GNU_SOURCE
-#define __EXTENSIONS__
-
 #include <stdint.h>
 #include <sys/wait.h>
 
diff -r 870b56a3d07e libgo/mksysinfo.sh
--- a/libgo/mksysinfo.sh	Fri Dec 16 06:45:11 2011 -0800
+++ b/libgo/mksysinfo.sh	Tue Dec 20 09:18:16 2011 -0800
@@ -25,10 +25,6 @@
 cat > sysinfo.c <<EOF
 #include "config.h"
 
-#define _GNU_SOURCE
-#define _LARGEFILE_SOURCE
-#define _FILE_OFFSET_BITS 64
-
 #include <sys/types.h>
 #include <dirent.h>
 #include <errno.h>

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> * There's a redefinition of IPMreq now:
>
> sysinfo.go:3089:6: error: redefinition of 'IPMreq'
> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/socket_irix.go:81:6: note: previous definition of 'IPMreq' was here
>
>   For some reason, sysinfo.go now gets the right definition, so the one
>   in socket_irix.go isn't necessary any longer.

It changed because mksysinfo.sh now unconditionally defines IPMreq.  I
did that because it is needed in order to compile the net package, and
RTEMS doesn't define it at all.  We can get away with this on Irix
because the definition is pretty much the same on all systems.

I committed your patch to socket_irix.go.

Thank.

Ian

Re: Go patch committed: Implement new syscall package

[Rainer Orth]

[-- Attachment #1: Type: text/plain, Size: 2227 bytes --]

Ian Lance Taylor <iant@google.com> writes:

> Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:
>
>> Now IRIX finally bootstrap again, I had to make two adjustments to have
>> libgo build there.
>>
>> * go/syscall/wait.c doesn't compile:
>>
>> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:11:0: error: "__EXTENSIONS__" redefined [-Werror]
>> /vol/gcc/src/hg/trunk/local/libgo/go/syscall/wait.c:1:0: note: this is the location of the previous definition
>>
>>   I've wrapped the __EXTENSIONS__ definition in #ifndef/#endif, but
>>   think this is the wrong approach: definitions of _GNU_SOURCE,
>>   __EXTENSIONS__ and other platform-specific stuff should go into
>>   configure.ac or mksysinfo.sh, not individual sources.  There are more
>>   instances of this problem, but they don't hurt me on IRIX.
>
> Makes sense.  I have committed this patch to try to clean this up a bit.
> Bootstrapped and ran Go testsuite on x86_64-unknown-linux-gnu.

Thanks.  Unfortunately, this broke bootstrap on Solaris 10 and 11:

/var/gcc/regression/trunk/11-gcc/build/./gcc/include-fixed/sys/feature_tests.h:367:2: error: #error "Compiler or options invalid; UNIX 03 and POSIX.1-2001 applications 	require the use of c99"
make[4]: *** [go-main.o] Error 1

Compiling with _XOPEN_SOURCE=600 only works with -std=gnu99 (c99 alone
breaks due to the uses of asm).  Initially, I meant to apply this
globally to match what mksysinfo.sh does, but unfortunately that broke
x86_64-unknown-linux-gnu bootstrap: libgo.so fails to link:

.libs/go-byte-array-to-string.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:580: multiple definition of `__pthread_cleanup_routine'
.libs/go-append.o:/usr/include/pthread.h:580: first defined here
.libs/go-defer.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:580: multiple definition of `__pthread_cleanup_routine'
.libs/go-append.o:/usr/include/pthread.h:580: first defined here

and many more.  I've therefore settled for the follwing patch, which
allowed i386-pc-solaris2.11 bootstrap to complete.

	Rainer


2011-12-22  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	libgo:
	* configure.ac (OSCFLAGS): Add -std=gnu99 for *-*-solaris2.1[01].
	* configure: Regenerate.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: sol2-libgo-gnu99.patch --]
[-- Type: text/x-patch, Size: 660 bytes --]

# HG changeset patch
# Parent b90f1ceca568f0c01951cbefd59496ef565a9096
Compile libgo as C99

diff --git a/libgo/configure.ac b/libgo/configure.ac
--- a/libgo/configure.ac
+++ b/libgo/configure.ac
@@ -292,9 +292,9 @@ case "$target" in
 	;;
     *-*-solaris2.1[[01]])
 	# Solaris 10+ needs this so struct msghdr gets the msg_control
-	# etc. fields in <sys/socket.h> (_XPG4_2).  _XOPEN_SOURCE=500 as
+	# etc. fields in <sys/socket.h> (_XPG4_2).  _XOPEN_SOURCE=600 as
 	# above doesn't work with C99.
-	OSCFLAGS="$OSCFLAGS -D_XOPEN_SOURCE=600 -D__EXTENSIONS__"
+	OSCFLAGS="$OSCFLAGS -std=gnu99 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__"
 	;;
 esac
 AC_SUBST(OSCFLAGS)

[-- Attachment #3: Type: text/plain, Size: 144 bytes --]



-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Go patch committed: Implement new syscall package

[Ian Lance Taylor]

Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> writes:

> Thanks.  Unfortunately, this broke bootstrap on Solaris 10 and 11:
>
> /var/gcc/regression/trunk/11-gcc/build/./gcc/include-fixed/sys/feature_tests.h:367:2: error: #error "Compiler or options invalid; UNIX 03 and POSIX.1-2001 applications 	require the use of c99"
> make[4]: *** [go-main.o] Error 1
>
> Compiling with _XOPEN_SOURCE=600 only works with -std=gnu99 (c99 alone
> breaks due to the uses of asm).  Initially, I meant to apply this
> globally to match what mksysinfo.sh does, but unfortunately that broke
> x86_64-unknown-linux-gnu bootstrap: libgo.so fails to link:
>
> .libs/go-byte-array-to-string.o: In function `__pthread_cleanup_routine':
> /usr/include/pthread.h:580: multiple definition of `__pthread_cleanup_routine'
> .libs/go-append.o:/usr/include/pthread.h:580: first defined here
> .libs/go-defer.o: In function `__pthread_cleanup_routine':
> /usr/include/pthread.h:580: multiple definition of `__pthread_cleanup_routine'
> .libs/go-append.o:/usr/include/pthread.h:580: first defined here
>
> and many more.  I've therefore settled for the follwing patch, which
> allowed i386-pc-solaris2.11 bootstrap to complete.

Thanks.  Patch committed.

Ian