Re: asm-generic/unistd.h and glibc use of NR_ipc

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Chris Metcalf]

 (Adding libc-ports to the cc list.)

On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
> Chris Metcalf is using the generic unistd.h file on the tile architecture
> and has a glibc port that should be easily portable to all future
> architectures. There are a few of them getting ready to be merged
> now (c6x, lm32, nios2, and some people have contacted me privately
> for architectures I cannot name).
>
> I expect that all of them will just use the same syscall ABI and glibc
> port.

I'm happy to provide some previews of the work I'm doing to interested parties.

One question for the libc folks is nomenclature.  For now I'm using
"sysdeps/unix/sysv/linux/generic" to hold the sources that are meant to be
used for any architecture that uses Arnd's <asm-generic/unistd.h> along
with the other asm-generic headers, e.g. <asm-generic/stat.h>.   I've
appended a list of the files that I've put in that directory to this
email.  Many of them are there just for handling missing "standard"
syscalls in <asm-generic/unistd.h>; for example the "readlink" syscall
becomes "readlinkat" in dl-origin.c, etc.

Is this the best name to use for the directory, though?  In particular,
glibc already uses "sysdeps/generic" in a slightly special sense to provide
the fallback if no sysdeps override is available for a given file.  But the
Linux naming of "generic unistd.h" is pretty commonplace, so I assume that
in the "linux/" hierarchy it is reasonably clear what that subdirectory means.

In practice, of course, there's still lots of architecture-specific code
that is needed above and beyond the generic Linux subdirectory.  Our 32-bit
platform ("tile/tilepro") has an "Implies" file with
unix/sysv/linux/generic and unix/sysv/linux/generic/wordsize-32; our 64-bit
platform ("tile/tilegx") has an "Implies" file with unix/sysv/linux/generic
and unix/sysv/linux/wordsize-64.  Beyond that we have another 30-odd files
in sysdeps/unix/sysv/linux/tile to implement the non-NPTL part of our glibc
support, plus 50-odd files in sysdeps/tile.

I haven't provided an "nptl/sysdeps/unix/sysv/linux/generic/" directory
since there wasn't anything obviously generic about the "tile" support I
added for NPTL.

Another point where I'd appreciate guidance from libc-alpha is the sysctl()
and ustat() APIs.  The corresponding system calls are missing from
<asm-generic/unistd.h>, since they are deprecated and their functionality
is better provided by other means (/proc/sys, fstat).  So I've simply had
them return -1 with errno == ENOSYS.  Is there any reason to think they
merit more substantial work?  One could imagine baking in some horrible
mapping of "integer names" into path components for a sysctl()
implementation and reading /proc/sys to provide results, or walking all of
the mount points looking for a matching device number to pass a name to
fstat(), but I'm not sure it's worth the bloat to the library.

I have a few patches for the platform-independent glibc code, mostly for
things which are exposed in our platform but not in others.  E.g.
__readlink() in include/unistd.h returns the wrong type (int not ssize_t),
and some of the standard Linux implementations of things like fxstat64.c
need to be completely wrapped with "#ifndef XSTAT_IS_XSTAT64" so that
fxstat.c can provide the alias, since all the stat structures are the same
everywhere.  I will submit some separate patches to libc-alpha for this.

Thanks in advance for any feedback.  Please keep me cc'ed on any
discussions of the generic Linux syscall ABI.  Thanks!

sysdeps/unix/sysv/linux/generic/access.c
sysdeps/unix/sysv/linux/generic/bits/fcntl.h
sysdeps/unix/sysv/linux/generic/bits/sem.h
sysdeps/unix/sysv/linux/generic/bits/stat.h
sysdeps/unix/sysv/linux/generic/bits/typesizes.h
sysdeps/unix/sysv/linux/generic/chmod.c
sysdeps/unix/sysv/linux/generic/chown.c
sysdeps/unix/sysv/linux/generic/lxstat.c
sysdeps/unix/sysv/linux/generic/xstat.c
sysdeps/unix/sysv/linux/generic/Versions
sysdeps/unix/sysv/linux/generic/fxstatat.c
sysdeps/unix/sysv/linux/generic/lchown.c
sysdeps/unix/sysv/linux/generic/link.c
sysdeps/unix/sysv/linux/generic/readlink.c
sysdeps/unix/sysv/linux/generic/open.c
sysdeps/unix/sysv/linux/generic/open64.c
sysdeps/unix/sysv/linux/generic/pipe.c
sysdeps/unix/sysv/linux/generic/poll.c
sysdeps/unix/sysv/linux/generic/recv.c
sysdeps/unix/sysv/linux/generic/rmdir.c
sysdeps/unix/sysv/linux/generic/send.c
sysdeps/unix/sysv/linux/generic/symlink.c
sysdeps/unix/sysv/linux/generic/syscalls.list
sysdeps/unix/sysv/linux/generic/sysdep.h
sysdeps/unix/sysv/linux/generic/unlink.c
sysdeps/unix/sysv/linux/generic/xmknod.c
sysdeps/unix/sysv/linux/generic/sysctl.c
sysdeps/unix/sysv/linux/generic/readlink_chk.c
sysdeps/unix/sysv/linux/generic/ustat.c
sysdeps/unix/sysv/linux/generic/wordsize-32/fcntl.c
sysdeps/unix/sysv/linux/generic/wordsize-32/fstatfs.c
sysdeps/unix/sysv/linux/generic/wordsize-32/ftruncate.c
sysdeps/unix/sysv/linux/generic/wordsize-32/fxstat.c
sysdeps/unix/sysv/linux/generic/wordsize-32/llseek.c
sysdeps/unix/sysv/linux/generic/wordsize-32/lseek.c
sysdeps/unix/sysv/linux/generic/wordsize-32/mmap.c
sysdeps/unix/sysv/linux/generic/wordsize-32/posix_fadvise.c
sysdeps/unix/sysv/linux/generic/wordsize-32/sendfile.c
sysdeps/unix/sysv/linux/generic/wordsize-32/statfs.c
sysdeps/unix/sysv/linux/generic/wordsize-32/truncate.c
sysdeps/unix/sysv/linux/generic/brk.c
sysdeps/unix/sysv/linux/generic/dl-origin.c
sysdeps/unix/sysv/linux/generic/umount.c
sysdeps/unix/sysv/linux/generic/creat.c
sysdeps/unix/sysv/linux/generic/pause.c
sysdeps/unix/sysv/linux/generic/Makefile
sysdeps/unix/sysv/linux/generic/select.c
sysdeps/unix/sysv/linux/generic/epoll_create.c
sysdeps/unix/sysv/linux/generic/epoll_wait.c
sysdeps/unix/sysv/linux/generic/inotify_init.c
sysdeps/unix/sysv/linux/generic/dup2.c
sysdeps/unix/sysv/linux/generic/mkdir.c
sysdeps/unix/sysv/linux/generic/rename.c
sysdeps/unix/sysv/linux/generic/utimes.c

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Joseph S. Myers]

On Tue, 28 Sep 2010, Chris Metcalf wrote:

>  (Adding libc-ports to the cc list.)
> 
> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
> > Chris Metcalf is using the generic unistd.h file on the tile architecture
> > and has a glibc port that should be easily portable to all future
> > architectures. There are a few of them getting ready to be merged
> > now (c6x, lm32, nios2, and some people have contacted me privately
> > for architectures I cannot name).

(Removing libc-alpha as my comments aren't relevant to that list.)

Two of the three seem rather surprising to me as regards glibc ports being 
ready to merge - maybe kernel ports are ready, however, it isn't clear to 
me what (kernel or glibc) the original comment was referring to.

C6X has no MMU so a glibc port isn't a possibility.  There are indeed 
uClibc and GCC ports we will be contributing in due course - but the 
linker support for shared objects is still in development (I contributed 
the static linking binutils support upstream earlier this year) and is 
required for building anything for uClinux userspace for this platform.

As for Nios II, we have a fully functional glibc port at CodeSourcery.  
It is I believe all our own work so there are no legal obstacles to our 
contributing it to the FSF - but upstream contribution would not really be 
useful without the GCC and binutils ports, and those ports (to which we 
added PIC and TLS support among other things) are based on code from 
Altera which is not covered by an FSF copyright assignment, so while the 
code is available we have no current plans for Nios II upstream 
contribution.

http://www.codesourcery.com/sgpp/portal/datasheet?target_arch=Nios+II&target_os=GNU%2FLinux

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Arnd Bergmann]

On Tuesday 28 September 2010, Joseph S. Myers wrote:
> (Removing libc-alpha as my comments aren't relevant to that list.)
> 
> Two of the three seem rather surprising to me as regards glibc ports being 
> ready to merge - maybe kernel ports are ready, however, it isn't clear to 
> me what (kernel or glibc) the original comment was referring to.

I was talking about kernel ports.
 
> C6X has no MMU so a glibc port isn't a possibility.  There are indeed 
> uClibc and GCC ports we will be contributing in due course - but the 
> linker support for shared objects is still in development (I contributed 
> the static linking binutils support upstream earlier this year) and is 
> required for building anything for uClinux userspace for this platform.

Right. Obviously the same question exists for uClibc though: The C6X syscall
ABI will need to be rewritten (as in deleted and replaced by generic
code) in the kernel in order to get merged, so the uClibc port will
have to gain the same kind of generic API that we need for glibc.

> As for Nios II, we have a fully functional glibc port at CodeSourcery.  
> It is I believe all our own work so there are no legal obstacles to our 
> contributing it to the FSF - but upstream contribution would not really be 
> useful without the GCC and binutils ports, and those ports (to which we 
> added PIC and TLS support among other things) are based on code from 
> Altera which is not covered by an FSF copyright assignment, so while the 
> code is available we have no current plans for Nios II upstream 
> contribution.
> 
> http://www.codesourcery.com/sgpp/portal/datasheet?target_arch=Nios+II&target_os=GNU%2FLinux

Same thing here. I wouldn't call the glibc port "fully functional" because
it does not cover a mainline Linux and getting the architecture tree
upstream means replacing the syscall ABI with a proper one.

Once the kernel parts are upstream and use the generic syscall ABI, only
rudimentary architecture support in glibc would be needed.

	Arnd

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Joseph S. Myers]

On Tue, 28 Sep 2010, Arnd Bergmann wrote:

> Once the kernel parts are upstream and use the generic syscall ABI, only
> rudimentary architecture support in glibc would be needed.

In my experience very little of a glibc port relates to the particular set 
of syscalls present on a particular architecture (and what does often 
consists of one-line files that #include the file for i386 or some other 
convenient architecture); changing a port from an old syscall ABI to the 
generic one would mean making it use the new set of generic files in 
preference to some existing sysdeps/unix/sysv/linux files, and maybe 
having fewer bits/ headers if some of the architecture differences there 
are avoided, but no real differences to any of the bits that aren't purely 
mechanical such as the implementation of each architecture's TLS ABI, 
other pieces of the ELF ABI, the set of atomic operations available, 
startup code, longjmp, profiling, clone, the interface used to call an 
individual syscall, etc. - remember that you can't build any vaguely 
recent glibc without full shared library and NPTL support.

A lot of the *changes made to maintain an existing port* relate to 
constants in the duplicated bits/* headers which it might be good to 
duplicate less if future architectures share values for all the constants, 
but that's not where most of the complexity in the port lies.  MIPS is 
probably the worst case for code dealing with syscall ABI peculiarities 
(given the three different ABIs, and n64 having different userspace stat 
and stat64 structures), but even there they aren't much of a source of 
complexity (MIPS ELF peculiarities are much more an issue).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1773 bytes --]

On Tuesday, September 28, 2010 09:05:13 Chris Metcalf wrote:
> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
> > Chris Metcalf is using the generic unistd.h file on the tile architecture
> > and has a glibc port that should be easily portable to all future
> > architectures. There are a few of them getting ready to be merged
> > now (c6x, lm32, nios2, and some people have contacted me privately
> > for architectures I cannot name).

c6x lacks a MMU, so i dont see how it could possibly use glibc.  same for 
lm32.  nios2 at least has an optional MMU, so it'd be usable some of the time.  
so no, i dont see a generic unistd.h glibc port being useful for them.

> One question for the libc folks is nomenclature.  For now I'm using
> "sysdeps/unix/sysv/linux/generic" to hold the sources that are meant to be
> used for any architecture that uses Arnd's <asm-generic/unistd.h> along
> with the other asm-generic headers, e.g. <asm-generic/stat.h>.   I've
> appended a list of the files that I've put in that directory to this
> email.  Many of them are there just for handling missing "standard"
> syscalls in <asm-generic/unistd.h>; for example the "readlink" syscall
> becomes "readlinkat" in dl-origin.c, etc.
> 
> Is this the best name to use for the directory, though?  In particular,
> glibc already uses "sysdeps/generic" in a slightly special sense to provide
> the fallback if no sysdeps override is available for a given file.  But the
> Linux naming of "generic unistd.h" is pretty commonplace, so I assume that
> in the "linux/" hierarchy it is reasonably clear what that subdirectory
> means.

i think "generic" in the context of "linux" is ok, but about the only real 
answer you'd get is if it comes from drepper/roland.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1090 bytes --]

On Tuesday, September 28, 2010 12:13:56 Arnd Bergmann wrote:
> On Tuesday 28 September 2010, Joseph S. Myers wrote:
> > C6X has no MMU so a glibc port isn't a possibility.  There are indeed
> > uClibc and GCC ports we will be contributing in due course - but the
> > linker support for shared objects is still in development (I contributed
> > the static linking binutils support upstream earlier this year) and is
> > required for building anything for uClinux userspace for this platform.
> 
> Right. Obviously the same question exists for uClibc though: The C6X
> syscall ABI will need to be rewritten (as in deleted and replaced by
> generic code) in the kernel in order to get merged, so the uClibc port
> will have to gain the same kind of generic API that we need for glibc.

i dont think uClibc needs nearly as much work as glibc.  it already goes out 
of its way to avoid code duplication whenever possible (and then some).  the 
position glibc takes wrt copy & paste at the file level makes me ill and so i 
tend to double down in my uClibc efforts there.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Joseph S. Myers]

On Tue, 28 Sep 2010, Mike Frysinger wrote:

> On Tuesday, September 28, 2010 09:05:13 Chris Metcalf wrote:
> > On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
> > > Chris Metcalf is using the generic unistd.h file on the tile architecture
> > > and has a glibc port that should be easily portable to all future
> > > architectures. There are a few of them getting ready to be merged
> > > now (c6x, lm32, nios2, and some people have contacted me privately
> > > for architectures I cannot name).
> 
> c6x lacks a MMU, so i dont see how it could possibly use glibc.  same for 
> lm32.  nios2 at least has an optional MMU, so it'd be usable some of the time.  
> so no, i dont see a generic unistd.h glibc port being useful for them.

A generic port would be useful for new architectures with MMU - but the 
notion that "only rudimentary architecture support in glibc would be 
needed" is mistaken.  Note Chris's figures "another 30-odd files in 
sysdeps/unix/sysv/linux/tile to implement the non-NPTL part of our glibc 
support, plus 50-odd files in sysdeps/tile" - hardly rudimentary.  Those 
seem the right order of magnitude, if larger than the minimum; the Nios II 
port currently has 59 new files in sysdeps directories, most of which I 
expect would still be needed with the generic syscall ABI.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Arnd Bergmann]

On Tuesday 28 September 2010 18:41:41 Joseph S. Myers wrote:
> In my experience very little of a glibc port relates to the particular set 
> of syscalls present on a particular architecture (and what does often 
> consists of one-line files that #include the file for i386 or some other 
> convenient architecture); changing a port from an old syscall ABI to the 
> generic one would mean making it use the new set of generic files in 
> preference to some existing sysdeps/unix/sysv/linux files, and maybe 
> having fewer bits/ headers if some of the architecture differences there 
> are avoided, but no real differences to any of the bits that aren't purely 
> mechanical such as the implementation of each architecture's TLS ABI, 
> other pieces of the ELF ABI, the set of atomic operations available, 
> startup code, longjmp, profiling, clone, the interface used to call an 
> individual syscall, etc. - remember that you can't build any vaguely 
> recent glibc without full shared library and NPTL support.
> 
> A lot of the changes made to maintain an existing port relate to 
> constants in the duplicated bits/* headers which it might be good to 
> duplicate less if future architectures share values for all the constants, 
> but that's not where most of the complexity in the port lies.  MIPS is 
> probably the worst case for code dealing with syscall ABI peculiarities 
> (given the three different ABIs, and n64 having different userspace stat 
> and stat64 structures), but even there they aren't much of a source of 
> complexity (MIPS ELF peculiarities are much more an issue).

Good point. As a kernel developer I tend to see glibc too much as
just a front-end for the system calls, but of course you are right
that it is much more that requires a lot of hard work for an architecture
port.
I did not mean to discredit the good work that has gone into the nios2
port or any part of upstream glibc.

The question that Linas was asking is specifically about the syscall
ABI and this is what I'm mostly interested in as well. A significant
part of the maintainance work in the kernel is just keeping up with
syscall ABI changes, which now gets minimized for new architectures.

	Arnd

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Christoph Hellwig]

On Tue, Sep 28, 2010 at 09:05:13AM -0400, Chris Metcalf wrote:
> Another point where I'd appreciate guidance from libc-alpha is the sysctl()
> and ustat() APIs.  The corresponding system calls are missing from
> <asm-generic/unistd.h>, since they are deprecated and their functionality
> is better provided by other means (/proc/sys, fstat).  So I've simply had
> them return -1 with errno == ENOSYS.  Is there any reason to think they
> merit more substantial work?  One could imagine baking in some horrible
> mapping of "integer names" into path components for a sysctl()
> implementation and reading /proc/sys to provide results, or walking all of
> the mount points looking for a matching device number to pass a name to
> fstat(), but I'm not sure it's worth the bloat to the library.

fstat does not replace ustat.  ustat is a statf-subsystem by dev_t and
is not replaced by anything.  xfsprogs for example uses it to check if
a given dev_t is currently mounted.  Please add it to the generic code.

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Arnd Bergmann]

On Wednesday 29 September 2010, Christoph Hellwig wrote:
> On Tue, Sep 28, 2010 at 09:05:13AM -0400, Chris Metcalf wrote:
> > Another point where I'd appreciate guidance from libc-alpha is the sysctl()
> > and ustat() APIs.  The corresponding system calls are missing from
> > <asm-generic/unistd.h>, since they are deprecated and their functionality
> > is better provided by other means (/proc/sys, fstat).  So I've simply had
> > them return -1 with errno == ENOSYS.  Is there any reason to think they
> > merit more substantial work?  One could imagine baking in some horrible
> > mapping of "integer names" into path components for a sysctl()
> > implementation and reading /proc/sys to provide results, or walking all of
> > the mount points looking for a matching device number to pass a name to
> > fstat(), but I'm not sure it's worth the bloat to the library.
> 
> fstat does not replace ustat.  ustat is a statf-subsystem by dev_t and
> is not replaced by anything.  xfsprogs for example uses it to check if
> a given dev_t is currently mounted.  Please add it to the generic code.
> 

I didn't put it into the generic list because the man page claims

 "ustat() is deprecated and has only been provided for compatibility.
  All new programs should use statfs(2) instead."

I suppose that means we should change the man page as well, right?

	Arnd

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Chris Metcalf]

 On 9/28/2010 1:36 PM, Linas Vepstas wrote:
> On 28 September 2010 08:05, Chris Metcalf <cmetcalf@tilera.com> wrote:
>> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
>>> Chris Metcalf is using the generic unistd.h file on the tile architecture
>>> and has a glibc port that should be easily portable to all future
>>> architectures. There are a few of them getting ready to be merged
>>> now (c6x, lm32, nios2, and some people have contacted me privately
>>> for architectures I cannot name).
>>>
>> I'm happy to provide some previews of the work I'm doing to interested parties.
> Yes, this is exactly what I'm looking for!  I'd love to "preview" the
> code; it'll take a good long while to get everything built and tested,
> so feeback might take a while ...
>
> Any chance that these could be split out as a distinct set of patches,
> taken apart from the tilera arch port, and submitted upstream
> independently (to glibc and/or ports collection)?  I'll help w/ patch
> preparation...

At this point I'm probably a week away from having something with the
obvious bugs all fixed, at which point I can start submitting it back
upstream.  The few patches I have to common infrastructure I'll just submit
to libc-alpha as git patches, then I'll probably just send a two-part patch
set to libc-ports, one with the linux/generic stuff, and one with the
tile-specific stuff.  (It looks like changes are never made as "git pull"
requests on the glibc lists, just as emailed unidiffs -- someone correct me
if I'm wrong.)

Linas, if you can wait a week I'd rather have you see the stuff I believe
to be working 100% to get maximum use out of you as a reviewer :-)  If you
need it sooner, of course, I'm happy to let you have something now.

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Linas Vepstas]

Hi Chris,

A question, and a patch follow:

On 28 September 2010 08:05, Chris Metcalf <cmetcalf@tilera.com> wrote:
>  (Adding libc-ports to the cc list.)
>
> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
>> Chris Metcalf is using the generic unistd.h file on the tile architecture
>> and has a glibc port that should be easily portable to all future
>> architectures. There are a few of them getting ready to be merged
>> now (c6x, lm32, nios2, and some people have contacted me privately
>> for architectures I cannot name).
>>
>> I expect that all of them will just use the same syscall ABI and glibc
>> port.
>
> I'm happy to provide some previews of the work I'm doing to interested parties.
[...]

> sysdeps/unix/sysv/linux/generic/access.c
> sysdeps/unix/sysv/linux/generic/bits/fcntl.h
> sysdeps/unix/sysv/linux/generic/bits/sem.h
> sysdeps/unix/sysv/linux/generic/bits/stat.h
> sysdeps/unix/sysv/linux/generic/bits/typesizes.h
> sysdeps/unix/sysv/linux/generic/chmod.c
> sysdeps/unix/sysv/linux/generic/chown.c
> sysdeps/unix/sysv/linux/generic/lxstat.c
> sysdeps/unix/sysv/linux/generic/xstat.c
> sysdeps/unix/sysv/linux/generic/Versions
> sysdeps/unix/sysv/linux/generic/fxstatat.c
> sysdeps/unix/sysv/linux/generic/lchown.c
> sysdeps/unix/sysv/linux/generic/link.c
> sysdeps/unix/sysv/linux/generic/readlink.c
> sysdeps/unix/sysv/linux/generic/open.c
> sysdeps/unix/sysv/linux/generic/open64.c
> sysdeps/unix/sysv/linux/generic/pipe.c
> sysdeps/unix/sysv/linux/generic/poll.c
> sysdeps/unix/sysv/linux/generic/recv.c
> sysdeps/unix/sysv/linux/generic/rmdir.c
> sysdeps/unix/sysv/linux/generic/send.c
> sysdeps/unix/sysv/linux/generic/symlink.c
> sysdeps/unix/sysv/linux/generic/syscalls.list
> sysdeps/unix/sysv/linux/generic/sysdep.h
> sysdeps/unix/sysv/linux/generic/unlink.c
> sysdeps/unix/sysv/linux/generic/xmknod.c
> sysdeps/unix/sysv/linux/generic/sysctl.c
> sysdeps/unix/sysv/linux/generic/readlink_chk.c
> sysdeps/unix/sysv/linux/generic/ustat.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/fcntl.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/fstatfs.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/ftruncate.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/fxstat.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/llseek.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/lseek.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/mmap.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/posix_fadvise.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/sendfile.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/statfs.c
> sysdeps/unix/sysv/linux/generic/wordsize-32/truncate.c
> sysdeps/unix/sysv/linux/generic/brk.c
> sysdeps/unix/sysv/linux/generic/dl-origin.c
> sysdeps/unix/sysv/linux/generic/umount.c
> sysdeps/unix/sysv/linux/generic/creat.c
> sysdeps/unix/sysv/linux/generic/pause.c
> sysdeps/unix/sysv/linux/generic/Makefile
> sysdeps/unix/sysv/linux/generic/select.c
> sysdeps/unix/sysv/linux/generic/epoll_create.c
> sysdeps/unix/sysv/linux/generic/epoll_wait.c
> sysdeps/unix/sysv/linux/generic/inotify_init.c
> sysdeps/unix/sysv/linux/generic/dup2.c
> sysdeps/unix/sysv/linux/generic/mkdir.c
> sysdeps/unix/sysv/linux/generic/rename.c
> sysdeps/unix/sysv/linux/generic/utimes.c

I note that this hasn't been merged into glibc-2.13, any particular
explaintation
for that?

Anyway, it seems that pthread cancellation pints are missing
from a couple of the functions.  e.g. for send, recv, one needs
the following:

Index: glibc-2.13/sysdeps/unix/sysv/linux/generic/send.c
===================================================================
--- glibc-2.13.orig/sysdeps/unix/sysv/linux/generic/send.c	2011-03-10
10:26:48.000000000 -0600
+++ glibc-2.13/sysdeps/unix/sysv/linux/generic/send.c	2011-03-10
10:26:55.000000000 -0600
@@ -18,12 +18,25 @@

 #include <stdlib.h>
 #include <sys/types.h>
-#include <sysdep.h>
+#include <sysdep-cancel.h>
 #include <libc-symbols.h>

 ssize_t __libc_send(int sockfd, const void *buffer, size_t len, int flags)
 {
-  return INLINE_SYSCALL (sendto, 6, sockfd, buffer, len, flags, NULL, 0);
+  ssize_t result;
+
+  if (SINGLE_THREAD_P)
+    result = INLINE_SYSCALL (sendto, 6, sockfd, buffer, len, flags, NULL, 0);
+  else
+    {
+      int oldtype = LIBC_CANCEL_ASYNC ();
+
+      result = INLINE_SYSCALL (sendto, 6, sockfd, buffer, len, flags, NULL, 0);
+
+      LIBC_CANCEL_RESET (oldtype);
+    }
+
+  return result;
 }

 strong_alias (__libc_send, __send)

and likewise for recv ...

--linas

Re: asm-generic/unistd.h and glibc use of NR_ipc

[Chris Metcalf]

On 3/10/2011 11:46 AM, Linas Vepstas wrote:
> Hi Chris,
>
> A question, and a patch follow:
>
> On 28 September 2010 08:05, Chris Metcalf <cmetcalf@tilera.com> wrote:
>>  (Adding libc-ports to the cc list.)
>>
>> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
>>> Chris Metcalf is using the generic unistd.h file on the tile architecture
>>> and has a glibc port that should be easily portable to all future
>>> architectures. There are a few of them getting ready to be merged
>>> now (c6x, lm32, nios2, and some people have contacted me privately
>>> for architectures I cannot name).
>>>
>>> I expect that all of them will just use the same syscall ABI and glibc
>>> port.
>> [...]
> I note that this hasn't been merged into glibc-2.13, any particular
> explaintation for that?

Just that I haven't had the time -- I've been more focused on the kernel
community rather than glibc.  I'm still planning to do it when I have some
time!

> Anyway, it seems that pthread cancellation pints are missing
> from a couple of the functions.  e.g. for send, recv, one needs
> the following:

Thanks, that seems very plausible.  I'll update my copy of those sources so
it will be in the eventual submission to libc-ports.

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com

linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Linas Vepstas]

On Tue, Sep 28, 2010 at 09:05:13AM -0400, Chris Metcalf wrote:
>  (Adding libc-ports to the cc list.)
> 
> On 9/28/2010 4:40 AM, Arnd Bergmann wrote:
> > Chris Metcalf is using the generic unistd.h file on the tile architecture
> > and has a glibc port that should be easily portable to all future
> > architectures. There are a few of them getting ready to be merged
> > now (c6x, lm32, nios2, and some people have contacted me privately
> > for architectures I cannot name).
> >
> > I expect that all of them will just use the same syscall ABI and glibc
> > port.
> 
> I'm happy to provide some previews of the work I'm doing to interested parties.
> 

I've been using this for quite a while & am very happy with it.  
Works like a charm.  I'd like to add a file to the collection: 
a linux-generic bits/mman.h.  It should complement the linux
kernel asm-generic mman files precisely.  

I spotted one thing that looks questionable: the asm-generic/mman-common.h
file has

#define MAP_TYPE 0xf     /* Mask for type of mapping.  */

but on examination, this seems to me like it should be 0x3f 
(at least) or maybe 0xff, so that, as a mask, it includes MAP_ANON.
I dunno. I just harmonized with the kernel files.

Consider the below as a formal submssion for glibc; FSF has my
copyright assignments on file.

Changelog:

2010-08-16  Linas Vepstas  <linasvepstas@gmail.com>

	* sysdeps/unix/sysv/linux/generic/bits/mman.h: New file.

---
 sysdeps/unix/sysv/linux/generic/bits/mman.h |  114 ++++++++++++++++++++++++++++
 1 file changed, 114 insertions(+)

Index: glibc-2.13/sysdeps/unix/sysv/linux/generic/bits/mman.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ glibc-2.13/sysdeps/unix/sysv/linux/generic/bits/mman.h	2011-04-13 12:46:55.000000000 -0500
@@ -0,0 +1,114 @@
+/* Definitions for Linux/POSIX memory map interface.
+   Copyright (C) 1997,2000,2003,2005,2006,2008,2009,2010,2011
+   Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, write to the Free
+   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+   02111-1307 USA.  */
+
+#ifndef _SYS_MMAN_H
+# error "Never use <bits/mman.h> directly; include <sys/mman.h> instead."
+#endif
+
+/* These are the bits used by generic Linux syscall interfaces.
+   The following definitions are essentially identical to those
+   in asm-generic/mman.h and asm-generic/mman-common.h.  We can't
+   include those headers directly, because they are not namespace-clean.
+   That is, they don't have __USE_GNU, __USE_BSD, etc. set up. */
+
+/* Protections are chosen from these bits, OR'd together.  The
+   implementation does not necessarily support PROT_EXEC or PROT_WRITE
+   without PROT_READ.  The only guarantees are that no writing will be
+   allowed without PROT_WRITE and no access will be allowed for PROT_NONE. */
+
+#define PROT_READ	0x1		/* Page can be read.  */
+#define PROT_WRITE	0x2		/* Page can be written.  */
+#define PROT_EXEC	0x4		/* Page can be executed.  */
+#define PROT_SEM	0x8		/* Page may be used for atomic ops. */
+#define PROT_NONE	0x0		/* Page can not be accessed.  */
+#define PROT_GROWSDOWN	0x01000000	/* Extend change to start of
+					   growsdown vma (mprotect only).  */
+#define PROT_GROWSUP	0x02000000	/* Extend change to start of
+					   growsup vma (mprotect only).  */
+
+/* Sharing types (must choose one and only one of these).  */
+#define MAP_SHARED	0x01		/* Share changes.  */
+#define MAP_PRIVATE	0x02		/* Changes are private.  */
+
+/* Mapping type (must choose one and only one of these).  */
+#ifdef __USE_BSD
+# define MAP_FILE	0
+# define MAP_ANON	0x20		/* Don't use a file.  */
+# define MAP_TYPE	0x0f		/* Mask for type of mapping.  */
+# ifdef __USE_MISC
+#  define MAP_ANONYMOUS	MAP_ANON	/* Linux name.  */
+# endif
+#endif
+
+/* Other flags.  */
+#define MAP_FIXED	0x10		/* Interpret addr exactly.  */
+#ifdef __USE_MISC
+# define MAP_GROWSDOWN	0x00100		/* Stack-like segment.  */
+# define MAP_DENYWRITE	0x00800		/* ETXTBSY */
+# define MAP_EXECUTABLE	0x01000		/* Mark it as an executable.  */
+# define MAP_LOCKED	0x02000		/* Lock the mapping.  */
+# define MAP_NORESERVE	0x04000		/* Don't check for reservations.  */
+# define MAP_POPULATE	0x08000		/* Populate (prefault) pagetables.  */
+# define MAP_NONBLOCK	0x10000		/* Do not block on IO.  */
+# define MAP_STACK	0x20000		/* Allocation is for a stack.  */
+# define MAP_HUGETLB	0x40000		/* Create huge page mapping.  */
+#endif
+
+/* Flags to `msync'.  */
+#define MS_ASYNC	1	/* Sync memory asynchronously.  */
+#define MS_INVALIDATE	2	/* Invalidate the caches.  */
+#define MS_SYNC		4	/* Synchronous memory sync.  */
+
+/* Flags for `mlockall'.  */
+#define MCL_CURRENT	1	/* Lock all currently mapped pages.  */
+#define MCL_FUTURE	2	/* Lock all additions to address space.  */
+
+/* Flags for `mremap'.  */
+#ifdef __USE_GNU
+# define MREMAP_MAYMOVE	1	/* Mapping address may change.  */
+# define MREMAP_FIXED	2	/* Fifth argument sets new address.  */
+#endif
+
+/* Advice to `madvise'.  */
+#ifdef __USE_BSD
+# define MADV_NORMAL	   0	/* No further special treatment.  */
+# define MADV_RANDOM	   1	/* Expect random page references.  */
+# define MADV_SEQUENTIAL   2	/* Expect sequential page references.  */
+# define MADV_WILLNEED	   3	/* Will need these pages.  */
+# define MADV_DONTNEED	   4	/* Don't need these pages.  */
+# define MADV_REMOVE	   9	/* Remove these pages and resources.  */
+# define MADV_DONTFORK	   10	/* Do not inherit across fork.  */
+# define MADV_DOFORK	   11	/* Do inherit across fork.  */
+# define MADV_MERGEABLE	   12	/* KSM may merge identical pages.  */
+# define MADV_UNMERGEABLE  13	/* KSM may not merge identical pages.  */
+# define MADV_HUGEPAGE	   14	/* Worth backing with hugepages.  */
+# define MADV_NOHUGEPAGE   15	/* Not worth backing with hugepages.  */
+# define MADV_HWPOISON	   100	/* Poison a page for testing.  */
+# define MADV_SOFT_OFFLINE 101	/* Soft offline page for testing */
+#endif
+
+/* The POSIX people had to invent similar names for the same things.  */
+#ifdef __USE_XOPEN2K
+# define POSIX_MADV_NORMAL	0 /* No further special treatment.  */
+# define POSIX_MADV_RANDOM	1 /* Expect random page references.  */
+# define POSIX_MADV_SEQUENTIAL	2 /* Expect sequential page references.  */
+# define POSIX_MADV_WILLNEED	3 /* Will need these pages.  */
+# define POSIX_MADV_DONTNEED	4 /* Don't need these pages.  */
+#endif

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Ulrich Drepper]

That belongs in the prots files, not glibc proper.

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Arnd Bergmann]

On Sunday 17 April 2011, Ulrich Drepper wrote:
> That belongs in the prots files, not glibc proper.

Do you mean just that file, or the entire
sysdeps/unix/sysv/linux/generic directory?

	Arnd

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Ulrich Drepper]

On Sun, Apr 17, 2011 at 12:54, Arnd Bergmann <arnd@arndb.de> wrote:
> Do you mean just that file, or the entire
> sysdeps/unix/sysv/linux/generic directory?

Everything.  I don't even want to know about it.  People who care
should subscribe to the ports email list.

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Linas Vepstas]

On 17 April 2011 12:01, Ulrich Drepper <drepper@gmail.com> wrote:
> On Sun, Apr 17, 2011 at 12:54, Arnd Bergmann <arnd@arndb.de> wrote:
>> Do you mean just that file, or the entire
>> sysdeps/unix/sysv/linux/generic directory?
>
> Everything.  I don't even want to know about it.  People who care
> should subscribe to the ports email list.

Would any of the existing arches ever be interested in using asm-generic?

--linas

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Ulrich Drepper]

On Sun, Apr 17, 2011 at 14:35, Linas Vepstas <linasvepstas@gmail.com> wrote:
> Would any of the existing arches ever be interested in using asm-generic?

No.

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Linas Vepstas]

Hi,

On 17 April 2011 12:01, Ulrich Drepper <drepper@gmail.com> wrote:
> On Sun, Apr 17, 2011 at 12:54, Arnd Bergmann <arnd@arndb.de> wrote:
>> Do you mean just that file, or the entire
>> sysdeps/unix/sysv/linux/generic directory?
>
> Everything.  I don't even want to know about it.  People who care
> should subscribe to the ports email list.
>

So what can I do to get the generic syscall wrappers into ports?
I've been using these for the last 6 months & they've been spotless
(well, OK, 1 or 2 trite bugs...)  I'm using these to leverage a port to
a new processor, the qualcomm hexagon; the port is done
(better than 99% of the glibc test cases pass; gd/ie tls support and
everything). I'd send the patches yesterday, except that the lawyers
haven't yet recovered from their strange gplv3-disease.  Anyway,
to lay the groundwork, I'd like to see  the generic-syscalls code
in the port git repo.

--linas

BTW,  http://www.gnu.org/s/libc/ports.html doesn't point at any reasonable
url for the sourceware ports git repo, etc.

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Chris Metcalf]

On 4/17/2011 3:23 PM, Linas Vepstas wrote:
> Hi,
>
> On 17 April 2011 12:01, Ulrich Drepper <drepper@gmail.com> wrote:
>> On Sun, Apr 17, 2011 at 12:54, Arnd Bergmann <arnd@arndb.de> wrote:
>>> Do you mean just that file, or the entire
>>> sysdeps/unix/sysv/linux/generic directory?
>> Everything.  I don't even want to know about it.  People who care
>> should subscribe to the ports email list.
>>
> So what can I do to get the generic syscall wrappers into ports?
> I've been using these for the last 6 months & they've been spotless
> (well, OK, 1 or 2 trite bugs...)  I'm using these to leverage a port to
> a new processor, the qualcomm hexagon; the port is done
> (better than 99% of the glibc test cases pass; gd/ie tls support and
> everything). I'd send the patches yesterday, except that the lawyers
> haven't yet recovered from their strange gplv3-disease.  Anyway,
> to lay the groundwork, I'd like to see  the generic-syscalls code
> in the port git repo.
>
> --linas
>
> BTW,  http://www.gnu.org/s/libc/ports.html doesn't point at any reasonable
> url for the sourceware ports git repo, etc.

Just to stay in the loop here, I'll mention that the current version of the
Tilera "sysdeps/unix/sysv/linux/generic" hierarchy along with the rest of
the "tile" support is on our website along with build instructions.  Having
pushed the kernel code a while back, we were gearing up to do binutils/gcc
as the next step, but I'm happy to submit the glibc work earlier, if I can
make some time.  Linas's enthusiasm is very motivating in this regard :-)

The web site is http://www.tilera.com/scm/.

There are also a few fixes for the generic code that we will pass by Ulrich
(e.g. __readlink in include/unistd.h should return ssize_t) and some fairly
minor tweaks (a few places where the asm-generic API can be supported at no
cost to other architectures).  The rest is all in the
sysdeps/unix/sysv/linux/generic hierarchy, which obviously would go to
glibc-ports.

I will be out on vacation for the next week or so but will likely have
email access.

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Joseph S. Myers]

On Sun, 17 Apr 2011, Linas Vepstas wrote:

> So what can I do to get the generic syscall wrappers into ports?

Someone needs to submit a patch that is fully up to date with the APIs 
provided by the existing libc headers for other Linux architectures.  I'm 
not aware of such a patch against current ports git having been submitted.  
I ignored the bits/mman.h submission as an individual header makes no 
sense without the rest of that tree; the tree needs to be submitted as a 
whole and an individual header submission must be a submission for 
inclusion in some externally maintained tree rather than anything directly 
relevant to libc-ports.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Joseph S. Myers]

On Mon, 18 Apr 2011, Chris Metcalf wrote:

> Just to stay in the loop here, I'll mention that the current version of the
> Tilera "sysdeps/unix/sysv/linux/generic" hierarchy along with the rest of
> the "tile" support is on our website along with build instructions.  Having
> pushed the kernel code a while back, we were gearing up to do binutils/gcc
> as the next step, but I'm happy to submit the glibc work earlier, if I can
> make some time.  Linas's enthusiasm is very motivating in this regard :-)

Note also that I don't see an assignment/disclaimer for either Tilera or 
Qualcomm on file at the FSF (Linas has a personal assignment, but no 
employer assignment or disclaimer).  We won't be able to review these 
changes until the documents are on file for the patch authors and/or any 
relevant employers, as applicable (and the FSF has reportedly been rather 
slow at processing assignments lately, so getting them sent to the FSF 
sooner rather than later is a good idea).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: linux/generic/bits/mmap.h [was Re: asm-generic/unistd.h and glibc use of NR_ipc

[Chris Metcalf]

On 4/20/2011 6:47 PM, Joseph S. Myers wrote:
> On Mon, 18 Apr 2011, Chris Metcalf wrote:
>
>> Just to stay in the loop here, I'll mention that the current version of the
>> Tilera "sysdeps/unix/sysv/linux/generic" hierarchy along with the rest of
>> the "tile" support is on our website along with build instructions.  Having
>> pushed the kernel code a while back, we were gearing up to do binutils/gcc
>> as the next step, but I'm happy to submit the glibc work earlier, if I can
>> make some time.  Linas's enthusiasm is very motivating in this regard :-)
> Note also that I don't see an assignment/disclaimer for either Tilera or 
> Qualcomm on file at the FSF (Linas has a personal assignment, but no 
> employer assignment or disclaimer).  We won't be able to review these 
> changes until the documents are on file for the patch authors and/or any 
> relevant employers, as applicable (and the FSF has reportedly been rather 
> slow at processing assignments lately, so getting them sent to the FSF 
> sooner rather than later is a good idea).

Donald Robertson at FSF is preparing an assignment form for Tilera at the
moment, so we'll push that through as soon as we receive it.  Thanks again
for the heads up about possible delays in the process.

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com