public inbox for libc-alpha@sourceware.org
 help / color / mirror / Atom feed
* [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd
@ 2022-12-12 11:46 Sergey Bugaev
  2022-12-12 11:46 ` [RFC PATCH 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
                   ` (3 more replies)
  0 siblings, 4 replies; 15+ messages in thread
From: Sergey Bugaev @ 2022-12-12 11:46 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

This is an attempt to add O_TMPFILE (from Linux) and SHM_ANON (from FreeBSD)
to the Hurd.

The Hurd already has the very handly dir_mkfile API that lets you create
anonymous/unnamed files -- ones that can either be used as just tmp files,
given to other processes via fork or IPC, or linked into the filesystem to
become regular, named files.

This little gem is used by glibc internally to implement atomic creation/
replacement of some files (/etc/hostname and the like, symlinks, etc), as well
as for the tmpfile () function (which makes a tmp file under $TMPDIR), but is
not otherwise exposed to user programs through a Unix-level API; you have to
drop down to Mach/Hurd level if you want to use it.

But there is a Unix-level API for this functionality in Linux: the O_TMPFILE
flag to open () & co, where you pass the parent directory path w/o a basename
to open () along with O_TMPFILE, and that makes an anonymous file "in that
directory's filesystem". So I thought it would make sense to expose the Hurd's
dir_mkfile by supporting the same O_TMPFILE flag in our open ().

Then, Linux also has memfd_create (), which makes a temp file without touching
any fs at all. This API is widely used in the Wayland ecosystem as the
recommended way to create shared memory.  But such an approach would not work
as-is on the Hurd: in order for there to be an fd, there has to be a server
somewhere, servicing that fd. We can't just make an fd out of "pure memory" --
it may be an fd to /hurd/tmpfs, but that /hurd/tmpfs needs to exist and be
accessible. So being usable in an empty chroot is not going to happen anyway,
unless we start spawning our own instances of /hurd/tmpfs for each memfd, which
sounds like a terrible idea.

And so in that light, the FreeBSD alternative to memfd_create () -- namely
SHM_ANON -- sounds much more approachable to me, while being, well, a bit less
widely supported in the Wayland ecosystem than memfd, but still quite popular.
We already implement shm by creating files under /dev/shm/ (which should
normally be a tmpfs, but for some reason on my Debian GNU/Hurd box it does not
seem to be?), so it seems only natural to use the just-introduced O_TMPFILE to
create anonymous shm files there.

So that is my motivation for adding O_TMPFILE and SHM_ANON. That being said,
maybe we don't actually want these? tmpfile () already gets you most of the way
there, and the fact that it creates a file under /tmp/ and not /dev/shm/ is not
*that* important.

As for the implementation: basically all of the SHM_ANON implementation, except
the very definition, ended up in the generic / POSIX version of shm_open () and
__shm_get_name (), predicated on defined (SHM_ANON) && defined (O_TMPFILE).
This sounds problematic: while there is indeed nothing Hurd-specific about the
implementation, and any port that supports O_TMPFILE and wants to support
SHM_ANON could use these code paths, what if another port wants to implement
SHM_ANON differently? Should I make a separate copy of shm_open.c in
sysdeps/mach/hurd instead of modifying the generic version?

I've used the following simple program to test shm_open () with SHM_ANON:

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

int main() {
    int fd = shm_open(SHM_ANON, O_RDWR | O_CREAT, 0600);
    printf("Opened fd %d\n", fd);
    write(fd, "hello", 5);
    lseek(fd, SEEK_SET, 0);
    char buffer[6] = { 0 };
    read(fd, buffer, 5);
    printf("Read %s\n", buffer);
}

and the following to test O_TMPFILE:

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <error.h>
#include <errno.h>

int main() {
    int fd = open("/tmp", O_TMPFILE | O_RDWR, 0700);
    if (fd < 0)
        error(1, errno, "open(O_TMPFILE)");
    write(fd, "Hello", 5);
    int r = fchown(fd, -1, 24);
    if (r < 0)
        error(0, errno, "fchown");
    r = linkat(fd, "", AT_FDCWD, "/tmp/hello.txt", AT_EMPTY_PATH);
    if (r < 0)
        error(1, errno, "linkat");
}

After running the latter, /tmp/hello.txt should appear, owned by $(id -u):24,
(where 24 is the group 'cdrom' on my system, chosen arbitrarily among those
that my user is a member of), containing "hello". Importantly, it does so
atomically, i.e. these properties are immediately visible once it appears.

Sergey Bugaev (3):
  hurd: Consolidate file_name_lookup implementation
  hurd: Implement O_TMPFILE
  hurd: Implement SHM_ANON

 hurd/hurdlookup.c                 | 10 +----
 hurd/lookup-at.c                  | 69 ++++++++++++++++++++++++-------
 posix/shm-directory.c             | 25 +++++++++--
 rt/shm_open.c                     |  5 +++
 sysdeps/mach/hurd/bits/fcntl.h    |  5 +++
 sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++
 6 files changed, 112 insertions(+), 27 deletions(-)
 create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h

-- 
2.38.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [RFC PATCH 1/3] hurd: Consolidate file_name_lookup implementation
  2022-12-12 11:46 [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
@ 2022-12-12 11:46 ` Sergey Bugaev
  2022-12-12 11:46 ` [RFC PATCH 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 15+ messages in thread
From: Sergey Bugaev @ 2022-12-12 11:46 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

Instead of __file_name_lookup_at delegating to __file_name_lookup
in simple cases, make __file_name_lookup_at deal with both cases, and
have __file_name_lookup simply wrap __file_name_lookup_at.
---
 hurd/hurdlookup.c | 10 ++--------
 hurd/lookup-at.c  | 51 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 37 insertions(+), 24 deletions(-)

diff --git a/hurd/hurdlookup.c b/hurd/hurdlookup.c
index 7babac7d..08166a2c 100644
--- a/hurd/hurdlookup.c
+++ b/hurd/hurdlookup.c
@@ -16,6 +16,7 @@
    <https://www.gnu.org/licenses/>.  */
 
 #include <hurd.h>
+#include <hurd/fd.h>
 #include <hurd/lookup.h>
 #include <string.h>
 #include <fcntl.h>
@@ -220,14 +221,7 @@ weak_alias (__hurd_directory_name_split, hurd_directory_name_split)
 file_t
 __file_name_lookup (const char *file_name, int flags, mode_t mode)
 {
-  error_t err;
-  file_t result;
-
-  err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
-				 file_name, flags, mode & ~_hurd_umask,
-				 &result);
-
-  return err ? (__hurd_fail (err), MACH_PORT_NULL) : result;
+  return __file_name_lookup_at (AT_FDCWD, 0, file_name, flags, mode);
 }
 weak_alias (__file_name_lookup, file_name_lookup)
 
diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
index 0c247a64..2b84dfd6 100644
--- a/hurd/lookup-at.c
+++ b/hurd/lookup-at.c
@@ -36,9 +36,6 @@ __file_name_lookup_at (int fd, int at_flags,
   if (err)
     return (__hurd_fail (err), MACH_PORT_NULL);
 
-  if (fd == AT_FDCWD || file_name[0] == '/')
-    return __file_name_lookup (file_name, flags, mode);
-
   if (empty != 0 && file_name[0] == '\0')
     {
       enum retry_type doretry;
@@ -56,22 +53,44 @@ __file_name_lookup_at (int fd, int at_flags,
       return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
     }
 
-  file_t startdir;
-  error_t use_init_port (int which, error_t (*operate) (mach_port_t))
+  if (fd == AT_FDCWD || file_name[0] == '/')
     {
-      return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
-	      : _hurd_ports_use (which, operate));
+      err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
+                                     file_name, flags, mode & ~_hurd_umask,
+                                     &result);
+      if (err)
+        {
+          __hurd_fail (err);
+          return MACH_PORT_NULL;
+        }
+    }
+  else
+    {
+      file_t startdir;
+      /* We need to look the file up relative to the given directory (and
+         not our cwd).  For this to work, we supply our own wrapper for
+         _hurd_ports_use, which replaces cwd with our startdir.  */
+      error_t use_init_port (int which, error_t (*operate) (mach_port_t))
+        {
+          return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
+	          : _hurd_ports_use (which, operate));
+        }
+
+      err = HURD_DPORT_USE (fd, (startdir = port,
+                                 __hurd_file_name_lookup (&use_init_port,
+                                                          &__getdport, NULL,
+                                                          file_name,
+                                                          flags,
+                                                          mode & ~_hurd_umask,
+                                                          &result)));
+      if (err)
+        {
+          __hurd_dfail (fd, err);
+          return MACH_PORT_NULL;
+        }
     }
 
-  err = HURD_DPORT_USE (fd, (startdir = port,
-			     __hurd_file_name_lookup (&use_init_port,
-						      &__getdport, NULL,
-						      file_name,
-						      flags,
-						      mode & ~_hurd_umask,
-						      &result)));
-
-  return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
+  return result;
 }
 
 file_t
-- 
2.38.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [RFC PATCH 2/3] hurd: Implement O_TMPFILE
  2022-12-12 11:46 [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
  2022-12-12 11:46 ` [RFC PATCH 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
@ 2022-12-12 11:46 ` Sergey Bugaev
  2023-01-29 23:25   ` Samuel Thibault
  2022-12-12 11:46 ` [RFC PATCH " Sergey Bugaev
  2023-01-29 23:31 ` [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Samuel Thibault
  3 siblings, 1 reply; 15+ messages in thread
From: Sergey Bugaev @ 2022-12-12 11:46 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

This is a flag that causes open () to create a new, unnamed file in the
same filesystem as the given directory. The file descriptor can be
simply used in the creating process as a temporary file, or shared with
children processes via fork (), or sent over a Unix socket. The file can
be left anonymous, in which case it will be deleted from the backing
file system once all copies of the file descriptor are closed, or given
a permanent name with a linkat () call, such as the following:

int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700);
/* Do something with the file... */
linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH);

In between creating the file and linking it to the file system, it is
possible to set the file content, mode, ownership, author, and other
attributes, so that the file visibly appears in the file system (perhaps
replacing another file) atomically, with all of its attributes already
set up.

The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to
user programs. Previously, dir_mkfile was used by glibc internally, in
particular for implementing tmpfile (), but not exposed to user programs
through a Unix-level API.

O_TMPFILE was initially introduced by Linux. This implementation is
intended to be compatible with the Linux implementation, except that the
O_EXCL flag is not given the special meaning when used together with
O_TMPFILE, unlike on Linux.
---
 hurd/lookup-at.c               | 20 ++++++++++++++++++++
 sysdeps/mach/hurd/bits/fcntl.h |  5 +++++
 2 files changed, 25 insertions(+)

diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
index 2b84dfd6..42ac8c42 100644
--- a/hurd/lookup-at.c
+++ b/hurd/lookup-at.c
@@ -29,6 +29,7 @@ __file_name_lookup_at (int fd, int at_flags,
   error_t err;
   file_t result;
   int empty = at_flags & AT_EMPTY_PATH;
+  int orig_flags;
 
   at_flags &= ~AT_EMPTY_PATH;
 
@@ -53,6 +54,10 @@ __file_name_lookup_at (int fd, int at_flags,
       return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
     }
 
+  orig_flags = flags;
+  if ((flags & __O_TMPFILE) == __O_TMPFILE)
+    flags = O_DIRECTORY;
+
   if (fd == AT_FDCWD || file_name[0] == '/')
     {
       err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
@@ -90,6 +95,21 @@ __file_name_lookup_at (int fd, int at_flags,
         }
     }
 
+  if ((orig_flags & __O_TMPFILE) == __O_TMPFILE)
+    {
+      /* What we have looked up is not the file iteself, but actually
+         the directory to create the file in.  Do that now.  */
+      file_t dir = result;
+
+      err = __dir_mkfile (dir, orig_flags & ~O_TMPFILE, mode, &result);
+      __mach_port_deallocate (__mach_task_self (), dir);
+      if (err)
+        {
+          __hurd_fail (err);
+          return MACH_PORT_NULL;
+        }
+    }
+
   return result;
 }
 
diff --git a/sysdeps/mach/hurd/bits/fcntl.h b/sysdeps/mach/hurd/bits/fcntl.h
index 17dcb384..b898a0c5 100644
--- a/sysdeps/mach/hurd/bits/fcntl.h
+++ b/sysdeps/mach/hurd/bits/fcntl.h
@@ -123,6 +123,11 @@
 # define O_CLOEXEC	0x00400000 /* Set FD_CLOEXEC.  */
 #endif
 
+#ifdef __USE_GNU
+# define __O_TMPFILE	0x00800000 /* Make a new unnamed file.  */
+# define O_TMPFILE	(__O_TMPFILE | O_DIRECTORY)
+#endif
+
 
 /* Controlling terminal flags.  These are understood only by `open',
    and are not preserved once the file has been opened.  */
-- 
2.38.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [RFC PATCH 3/3] hurd: Implement SHM_ANON
  2022-12-12 11:46 [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
  2022-12-12 11:46 ` [RFC PATCH 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
  2022-12-12 11:46 ` [RFC PATCH 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
@ 2022-12-12 11:46 ` Sergey Bugaev
  2023-01-29 23:31 ` [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Samuel Thibault
  3 siblings, 0 replies; 15+ messages in thread
From: Sergey Bugaev @ 2022-12-12 11:46 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

This adds a special SHM_ANON value that can be passed into shm_open ()
in place of a name. When called in this way, shm_open () will create a
new anonymous shared memory file. The file will be created in the same
way that other shared memory files are created (i.e., under /dev/shm/),
except that it is not given a name and therefore cannot be reached from
the file system, nor by other calls to shm_open (). This is accomplished
by utilizing O_TMPFILE.

This is intended to be compatible with FreeBSD's API of the same name.
---
 posix/shm-directory.c             | 25 +++++++++++++++++++++----
 rt/shm_open.c                     |  5 +++++
 sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 4 deletions(-)
 create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h

diff --git a/posix/shm-directory.c b/posix/shm-directory.c
index 76ea825b..12cc683e 100644
--- a/posix/shm-directory.c
+++ b/posix/shm-directory.c
@@ -23,17 +23,34 @@
 #include <alloc_buffer.h>
 #include <shm-directory.h>
 #include <string.h>
+#include <sys/mman.h>
+#include <fcntl.h>
 
 int
 __shm_get_name (struct shmdir_name *result, const char *name, bool sem_prefix)
 {
+  struct alloc_buffer buffer;
+  size_t namelen;
+
+  buffer = alloc_buffer_create (result->name, sizeof (result->name));
+  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
+
+#if defined (SHM_ANON) && defined (O_TMPFILE)
+  if (name == SHM_ANON)
+    {
+      /* For SHM_ANON, we want shm_open () to pass O_TMPFILE to open (),
+         with SHMDIR itself as the path.  So, leave it at that.  */
+      alloc_buffer_add_byte (&buffer, 0);
+      if (alloc_buffer_has_failed (&buffer))
+        return -1;
+      return 0;
+    }
+#endif
+
   while (name[0] == '/')
     ++name;
-  size_t namelen = strlen (name);
+  namelen = strlen (name);
 
-  struct alloc_buffer buffer
-    = alloc_buffer_create (result->name, sizeof (result->name));
-  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
   if (sem_prefix)
     alloc_buffer_copy_bytes (&buffer, "sem.", strlen ("sem."));
   alloc_buffer_copy_bytes (&buffer, name, namelen + 1);
diff --git a/rt/shm_open.c b/rt/shm_open.c
index 11cb7cc7..6524776e 100644
--- a/rt/shm_open.c
+++ b/rt/shm_open.c
@@ -23,6 +23,7 @@
 #include <shlib-compat.h>
 #include <shm-directory.h>
 #include <unistd.h>
+#include <sys/mman.h>
 
 /* Open shared memory object.  */
 int
@@ -36,6 +37,10 @@ __shm_open (const char *name, int oflag, mode_t mode)
     }
 
   oflag |= O_NOFOLLOW | O_CLOEXEC;
+#if defined (SHM_ANON) && defined (O_TMPFILE)
+  if (name == SHM_ANON)
+    oflag |= O_TMPFILE;
+#endif
 
   int fd = __open64_nocancel (dirname.name, oflag, mode);
   if (fd == -1 && __glibc_unlikely (errno == EISDIR))
diff --git a/sysdeps/mach/hurd/bits/mman_ext.h b/sysdeps/mach/hurd/bits/mman_ext.h
new file mode 100644
index 00000000..f022826e
--- /dev/null
+++ b/sysdeps/mach/hurd/bits/mman_ext.h
@@ -0,0 +1,25 @@
+/* System-specific extensions of <sys/mman.h>, Hurd version.
+   Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#ifndef _SYS_MMAN_H
+# error "Never include <bits/mman_ext.h> directly; use <sys/mman.h> instead."
+#endif
+
+#ifdef __USE_GNU
+# define SHM_ANON	((const char *) 1)
+#endif /* __USE_GNU  */
-- 
2.38.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 2/3] hurd: Implement O_TMPFILE
  2022-12-12 11:46 ` [RFC PATCH 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
@ 2023-01-29 23:25   ` Samuel Thibault
  2023-01-30  9:53     ` Sergey Bugaev
  0 siblings, 1 reply; 15+ messages in thread
From: Samuel Thibault @ 2023-01-29 23:25 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Hello,

Sergey Bugaev, le lun. 12 déc. 2022 14:46:35 +0300, a ecrit:
> diff --git a/sysdeps/mach/hurd/bits/fcntl.h b/sysdeps/mach/hurd/bits/fcntl.h
> index 17dcb384..b898a0c5 100644
> --- a/sysdeps/mach/hurd/bits/fcntl.h
> +++ b/sysdeps/mach/hurd/bits/fcntl.h
> @@ -123,6 +123,11 @@
>  # define O_CLOEXEC	0x00400000 /* Set FD_CLOEXEC.  */
>  #endif
>  
> +#ifdef __USE_GNU
> +# define __O_TMPFILE	0x00800000 /* Make a new unnamed file.  */
> +# define O_TMPFILE	(__O_TMPFILE | O_DIRECTORY)
> +#endif

I don't think we need the __O_TMPFILE variant, only the O_TMPFILE one?

Linux uses __ variants just because it has per-arch definitions.

Samuel

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd
  2022-12-12 11:46 [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
                   ` (2 preceding siblings ...)
  2022-12-12 11:46 ` [RFC PATCH " Sergey Bugaev
@ 2023-01-29 23:31 ` Samuel Thibault
  3 siblings, 0 replies; 15+ messages in thread
From: Samuel Thibault @ 2023-01-29 23:31 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Hello,

Sergey Bugaev, le lun. 12 déc. 2022 14:46:33 +0300, a ecrit:
> Then, Linux also has memfd_create (), which makes a temp file without touching
> any fs at all. This API is widely used in the Wayland ecosystem as the
> recommended way to create shared memory.  But such an approach would not work
> as-is on the Hurd: in order for there to be an fd, there has to be a server
> somewhere, servicing that fd.

Can't glibc itself serve it?

The drawback is that it'd only keep working while the creator is
running.

> We can't just make an fd out of "pure memory" -- it may be an fd to
> /hurd/tmpfs, but that /hurd/tmpfs needs to exist and be accessible. So
> being usable in an empty chroot is not going to happen anyway, unless
> we start spawning our own instances of /hurd/tmpfs for each memfd,
> which sounds like a terrible idea.

For a really-working memfd, it'd have to be so anyway.

> And so in that light, the FreeBSD alternative to memfd_create () -- namely
> SHM_ANON -- sounds much more approachable to me, while being, well, a bit less
> widely supported in the Wayland ecosystem than memfd, but still quite popular.
> We already implement shm by creating files under /dev/shm/

That has the benefit of factorizing the server part, indeed.

> As for the implementation: basically all of the SHM_ANON implementation, except
> the very definition, ended up in the generic / POSIX version of shm_open () and
> __shm_get_name (), predicated on defined (SHM_ANON) && defined (O_TMPFILE).
> This sounds problematic: while there is indeed nothing Hurd-specific about the
> implementation, and any port that supports O_TMPFILE and wants to support
> SHM_ANON could use these code paths, what if another port wants to implement
> SHM_ANON differently? Should I make a separate copy of shm_open.c in
> sysdeps/mach/hurd instead of modifying the generic version?

I would say that we can afford not foreseeing this, and wait for the
case to happen to see how to refactor things?

Thanks for this! We'll be able to commit things after the 2.37 release.
Samuel

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 2/3] hurd: Implement O_TMPFILE
  2023-01-29 23:25   ` Samuel Thibault
@ 2023-01-30  9:53     ` Sergey Bugaev
  2023-01-30  9:59       ` Samuel Thibault
  0 siblings, 1 reply; 15+ messages in thread
From: Sergey Bugaev @ 2023-01-30  9:53 UTC (permalink / raw)
  To: Samuel Thibault; +Cc: bug-hurd, libc-alpha

On Mon, Jan 30, 2023 at 2:25 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
>
> Hello,
>
> Sergey Bugaev, le lun. 12 déc. 2022 14:46:35 +0300, a ecrit:
> > diff --git a/sysdeps/mach/hurd/bits/fcntl.h b/sysdeps/mach/hurd/bits/fcntl.h
> > index 17dcb384..b898a0c5 100644
> > --- a/sysdeps/mach/hurd/bits/fcntl.h
> > +++ b/sysdeps/mach/hurd/bits/fcntl.h
> > @@ -123,6 +123,11 @@
> >  # define O_CLOEXEC   0x00400000 /* Set FD_CLOEXEC.  */
> >  #endif
> >
> > +#ifdef __USE_GNU
> > +# define __O_TMPFILE 0x00800000 /* Make a new unnamed file.  */
> > +# define O_TMPFILE   (__O_TMPFILE | O_DIRECTORY)
> > +#endif
>
> I don't think we need the __O_TMPFILE variant, only the O_TMPFILE one?
>
> Linux uses __ variants just because it has per-arch definitions.

This was an attempt to mimic the Linux port's behavior, where it
automatically or's in O_DIRECTORY when you specify O_TMPFILE. Not that
it has any real meaning (at least for us) since my
__file_name_lookup_at () passes O_DIRECTORY automatically (instead of
any other flags) when O_TMPFILE is used. So should I remove this and
just have

# define O_TMPFILE 0x00800000 /* Make a new unnamed file.  */

?

Thanks for taking a look!

Sergey

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 2/3] hurd: Implement O_TMPFILE
  2023-01-30  9:53     ` Sergey Bugaev
@ 2023-01-30  9:59       ` Samuel Thibault
  2023-01-30 12:52         ` [PATCH v2 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
                           ` (3 more replies)
  0 siblings, 4 replies; 15+ messages in thread
From: Samuel Thibault @ 2023-01-30  9:59 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Sergey Bugaev, le lun. 30 janv. 2023 12:53:02 +0300, a ecrit:
> So should I remove this and just have
> 
> # define O_TMPFILE 0x00800000 /* Make a new unnamed file.  */
> 
> ?

Yes, I believe we don't need the __ variant.

Samuel

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 0/3] O_TMPFILE and SHM_ANON for the Hurd
  2023-01-30  9:59       ` Samuel Thibault
@ 2023-01-30 12:52         ` Sergey Bugaev
  2023-01-30 12:52         ` [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 15+ messages in thread
From: Sergey Bugaev @ 2023-01-30 12:52 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

Changes since v1:
- Add forgotten Signed-off-by
- Fix a typo (thanks Amos!)
- Drop the separate __O_TMPFILE definition

Re-tested on i686-gnu, still works.

Sergey Bugaev (3):
  hurd: Consolidate file_name_lookup implementation
  hurd: Implement O_TMPFILE
  hurd: Implement SHM_ANON

 hurd/hurdlookup.c                 | 10 +----
 hurd/lookup-at.c                  | 70 ++++++++++++++++++++++++-------
 posix/shm-directory.c             | 25 +++++++++--
 rt/shm_open.c                     |  5 +++
 sysdeps/mach/hurd/bits/fcntl.h    |  4 ++
 sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++
 6 files changed, 112 insertions(+), 27 deletions(-)
 create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h

-- 
2.39.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation
  2023-01-30  9:59       ` Samuel Thibault
  2023-01-30 12:52         ` [PATCH v2 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
@ 2023-01-30 12:52         ` Sergey Bugaev
  2023-02-01 19:06           ` Samuel Thibault
  2023-01-30 12:52         ` [PATCH v2 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
  2023-01-30 12:52         ` [PATCH v2 3/3] hurd: Implement SHM_ANON Sergey Bugaev
  3 siblings, 1 reply; 15+ messages in thread
From: Sergey Bugaev @ 2023-01-30 12:52 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

Instead of __file_name_lookup_at delegating to __file_name_lookup
in simple cases, make __file_name_lookup_at deal with both cases, and
have __file_name_lookup simply wrap __file_name_lookup_at.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 hurd/hurdlookup.c | 10 ++--------
 hurd/lookup-at.c  | 51 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 37 insertions(+), 24 deletions(-)

diff --git a/hurd/hurdlookup.c b/hurd/hurdlookup.c
index 49eac443..3cfe444f 100644
--- a/hurd/hurdlookup.c
+++ b/hurd/hurdlookup.c
@@ -16,6 +16,7 @@
    <https://www.gnu.org/licenses/>.  */
 
 #include <hurd.h>
+#include <hurd/fd.h>
 #include <hurd/lookup.h>
 #include <string.h>
 #include <fcntl.h>
@@ -220,14 +221,7 @@ weak_alias (__hurd_directory_name_split, hurd_directory_name_split)
 file_t
 __file_name_lookup (const char *file_name, int flags, mode_t mode)
 {
-  error_t err;
-  file_t result;
-
-  err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
-				 file_name, flags, mode & ~_hurd_umask,
-				 &result);
-
-  return err ? (__hurd_fail (err), MACH_PORT_NULL) : result;
+  return __file_name_lookup_at (AT_FDCWD, 0, file_name, flags, mode);
 }
 weak_alias (__file_name_lookup, file_name_lookup)
 
diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
index 6f30a067..25dab5a1 100644
--- a/hurd/lookup-at.c
+++ b/hurd/lookup-at.c
@@ -36,9 +36,6 @@ __file_name_lookup_at (int fd, int at_flags,
   if (err)
     return (__hurd_fail (err), MACH_PORT_NULL);
 
-  if (fd == AT_FDCWD || file_name[0] == '/')
-    return __file_name_lookup (file_name, flags, mode);
-
   if (empty != 0 && file_name[0] == '\0')
     {
       enum retry_type doretry;
@@ -56,22 +53,44 @@ __file_name_lookup_at (int fd, int at_flags,
       return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
     }
 
-  file_t startdir;
-  error_t use_init_port (int which, error_t (*operate) (mach_port_t))
+  if (fd == AT_FDCWD || file_name[0] == '/')
     {
-      return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
-	      : _hurd_ports_use (which, operate));
+      err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
+                                     file_name, flags, mode & ~_hurd_umask,
+                                     &result);
+      if (err)
+        {
+          __hurd_fail (err);
+          return MACH_PORT_NULL;
+        }
+    }
+  else
+    {
+      file_t startdir;
+      /* We need to look the file up relative to the given directory (and
+         not our cwd).  For this to work, we supply our own wrapper for
+         _hurd_ports_use, which replaces cwd with our startdir.  */
+      error_t use_init_port (int which, error_t (*operate) (mach_port_t))
+        {
+          return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
+	          : _hurd_ports_use (which, operate));
+        }
+
+      err = HURD_DPORT_USE (fd, (startdir = port,
+                                 __hurd_file_name_lookup (&use_init_port,
+                                                          &__getdport, NULL,
+                                                          file_name,
+                                                          flags,
+                                                          mode & ~_hurd_umask,
+                                                          &result)));
+      if (err)
+        {
+          __hurd_dfail (fd, err);
+          return MACH_PORT_NULL;
+        }
     }
 
-  err = HURD_DPORT_USE (fd, (startdir = port,
-			     __hurd_file_name_lookup (&use_init_port,
-						      &__getdport, NULL,
-						      file_name,
-						      flags,
-						      mode & ~_hurd_umask,
-						      &result)));
-
-  return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
+  return result;
 }
 
 file_t
-- 
2.39.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 2/3] hurd: Implement O_TMPFILE
  2023-01-30  9:59       ` Samuel Thibault
  2023-01-30 12:52         ` [PATCH v2 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
  2023-01-30 12:52         ` [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
@ 2023-01-30 12:52         ` Sergey Bugaev
  2023-02-01 22:34           ` Samuel Thibault
  2023-01-30 12:52         ` [PATCH v2 3/3] hurd: Implement SHM_ANON Sergey Bugaev
  3 siblings, 1 reply; 15+ messages in thread
From: Sergey Bugaev @ 2023-01-30 12:52 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

This is a flag that causes open () to create a new, unnamed file in the
same filesystem as the given directory. The file descriptor can be
simply used in the creating process as a temporary file, or shared with
children processes via fork (), or sent over a Unix socket. The file can
be left anonymous, in which case it will be deleted from the backing
file system once all copies of the file descriptor are closed, or given
a permanent name with a linkat () call, such as the following:

int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700);
/* Do something with the file... */
linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH);

In between creating the file and linking it to the file system, it is
possible to set the file content, mode, ownership, author, and other
attributes, so that the file visibly appears in the file system (perhaps
replacing another file) atomically, with all of its attributes already
set up.

The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to
user programs. Previously, dir_mkfile was used by glibc internally, in
particular for implementing tmpfile (), but not exposed to user programs
through a Unix-level API.

O_TMPFILE was initially introduced by Linux. This implementation is
intended to be compatible with the Linux implementation, except that the
O_EXCL flag is not given the special meaning when used together with
O_TMPFILE, unlike on Linux.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 hurd/lookup-at.c               | 21 +++++++++++++++++++++
 sysdeps/mach/hurd/bits/fcntl.h |  4 ++++
 2 files changed, 25 insertions(+)

diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
index 25dab5a1..88c83779 100644
--- a/hurd/lookup-at.c
+++ b/hurd/lookup-at.c
@@ -29,6 +29,7 @@ __file_name_lookup_at (int fd, int at_flags,
   error_t err;
   file_t result;
   int empty = at_flags & AT_EMPTY_PATH;
+  int orig_flags;
 
   at_flags &= ~AT_EMPTY_PATH;
 
@@ -53,6 +54,10 @@ __file_name_lookup_at (int fd, int at_flags,
       return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
     }
 
+  orig_flags = flags;
+  if (flags & O_TMPFILE)
+    flags = O_DIRECTORY;
+
   if (fd == AT_FDCWD || file_name[0] == '/')
     {
       err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
@@ -90,6 +95,22 @@ __file_name_lookup_at (int fd, int at_flags,
         }
     }
 
+  if (orig_flags & O_TMPFILE)
+    {
+      /* What we have looked up is not the file itself, but actually
+         the directory to create the file in.  Do that now.  */
+      file_t dir = result;
+
+      err = __dir_mkfile (dir, orig_flags & ~(O_TMPFILE | O_DIRECTORY),
+                          mode, &result);
+      __mach_port_deallocate (__mach_task_self (), dir);
+      if (err)
+        {
+          __hurd_fail (err);
+          return MACH_PORT_NULL;
+        }
+    }
+
   return result;
 }
 
diff --git a/sysdeps/mach/hurd/bits/fcntl.h b/sysdeps/mach/hurd/bits/fcntl.h
index 970f79b8..c24a819e 100644
--- a/sysdeps/mach/hurd/bits/fcntl.h
+++ b/sysdeps/mach/hurd/bits/fcntl.h
@@ -123,6 +123,10 @@
 # define O_CLOEXEC	0x00400000 /* Set FD_CLOEXEC.  */
 #endif
 
+#ifdef __USE_GNU
+# define O_TMPFILE	0x00800000 /* Make a new unnamed file.  */
+#endif
+
 
 /* Controlling terminal flags.  These are understood only by `open',
    and are not preserved once the file has been opened.  */
-- 
2.39.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 3/3] hurd: Implement SHM_ANON
  2023-01-30  9:59       ` Samuel Thibault
                           ` (2 preceding siblings ...)
  2023-01-30 12:52         ` [PATCH v2 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
@ 2023-01-30 12:52         ` Sergey Bugaev
  2023-02-01 22:36           ` Samuel Thibault
  3 siblings, 1 reply; 15+ messages in thread
From: Sergey Bugaev @ 2023-01-30 12:52 UTC (permalink / raw)
  To: bug-hurd, libc-alpha, samuel.thibault; +Cc: Sergey Bugaev

This adds a special SHM_ANON value that can be passed into shm_open ()
in place of a name. When called in this way, shm_open () will create a
new anonymous shared memory file. The file will be created in the same
way that other shared memory files are created (i.e., under /dev/shm/),
except that it is not given a name and therefore cannot be reached from
the file system, nor by other calls to shm_open (). This is accomplished
by utilizing O_TMPFILE.

This is intended to be compatible with FreeBSD's API of the same name.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 posix/shm-directory.c             | 25 +++++++++++++++++++++----
 rt/shm_open.c                     |  5 +++++
 sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 4 deletions(-)
 create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h

diff --git a/posix/shm-directory.c b/posix/shm-directory.c
index 47e2355d..86d9fd8e 100644
--- a/posix/shm-directory.c
+++ b/posix/shm-directory.c
@@ -23,17 +23,34 @@
 #include <alloc_buffer.h>
 #include <shm-directory.h>
 #include <string.h>
+#include <sys/mman.h>
+#include <fcntl.h>
 
 int
 __shm_get_name (struct shmdir_name *result, const char *name, bool sem_prefix)
 {
+  struct alloc_buffer buffer;
+  size_t namelen;
+
+  buffer = alloc_buffer_create (result->name, sizeof (result->name));
+  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
+
+#if defined (SHM_ANON) && defined (O_TMPFILE)
+  if (name == SHM_ANON)
+    {
+      /* For SHM_ANON, we want shm_open () to pass O_TMPFILE to open (),
+         with SHMDIR itself as the path.  So, leave it at that.  */
+      alloc_buffer_add_byte (&buffer, 0);
+      if (alloc_buffer_has_failed (&buffer))
+        return -1;
+      return 0;
+    }
+#endif
+
   while (name[0] == '/')
     ++name;
-  size_t namelen = strlen (name);
+  namelen = strlen (name);
 
-  struct alloc_buffer buffer
-    = alloc_buffer_create (result->name, sizeof (result->name));
-  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
   if (sem_prefix)
     alloc_buffer_copy_bytes (&buffer, "sem.", strlen ("sem."));
   alloc_buffer_copy_bytes (&buffer, name, namelen + 1);
diff --git a/rt/shm_open.c b/rt/shm_open.c
index 48970bbc..6c1f4d60 100644
--- a/rt/shm_open.c
+++ b/rt/shm_open.c
@@ -23,6 +23,7 @@
 #include <shlib-compat.h>
 #include <shm-directory.h>
 #include <unistd.h>
+#include <sys/mman.h>
 
 /* Open shared memory object.  */
 int
@@ -36,6 +37,10 @@ __shm_open (const char *name, int oflag, mode_t mode)
     }
 
   oflag |= O_NOFOLLOW | O_CLOEXEC;
+#if defined (SHM_ANON) && defined (O_TMPFILE)
+  if (name == SHM_ANON)
+    oflag |= O_TMPFILE;
+#endif
 
   int fd = __open64_nocancel (dirname.name, oflag, mode);
   if (fd == -1 && __glibc_unlikely (errno == EISDIR))
diff --git a/sysdeps/mach/hurd/bits/mman_ext.h b/sysdeps/mach/hurd/bits/mman_ext.h
new file mode 100644
index 00000000..f022826e
--- /dev/null
+++ b/sysdeps/mach/hurd/bits/mman_ext.h
@@ -0,0 +1,25 @@
+/* System-specific extensions of <sys/mman.h>, Hurd version.
+   Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#ifndef _SYS_MMAN_H
+# error "Never include <bits/mman_ext.h> directly; use <sys/mman.h> instead."
+#endif
+
+#ifdef __USE_GNU
+# define SHM_ANON	((const char *) 1)
+#endif /* __USE_GNU  */
-- 
2.39.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation
  2023-01-30 12:52         ` [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
@ 2023-02-01 19:06           ` Samuel Thibault
  0 siblings, 0 replies; 15+ messages in thread
From: Samuel Thibault @ 2023-02-01 19:06 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Applied, thanks!

Sergey Bugaev, le lun. 30 janv. 2023 15:52:14 +0300, a ecrit:
> Instead of __file_name_lookup_at delegating to __file_name_lookup
> in simple cases, make __file_name_lookup_at deal with both cases, and
> have __file_name_lookup simply wrap __file_name_lookup_at.
> 
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
> ---
>  hurd/hurdlookup.c | 10 ++--------
>  hurd/lookup-at.c  | 51 ++++++++++++++++++++++++++++++++---------------
>  2 files changed, 37 insertions(+), 24 deletions(-)
> 
> diff --git a/hurd/hurdlookup.c b/hurd/hurdlookup.c
> index 49eac443..3cfe444f 100644
> --- a/hurd/hurdlookup.c
> +++ b/hurd/hurdlookup.c
> @@ -16,6 +16,7 @@
>     <https://www.gnu.org/licenses/>.  */
>  
>  #include <hurd.h>
> +#include <hurd/fd.h>
>  #include <hurd/lookup.h>
>  #include <string.h>
>  #include <fcntl.h>
> @@ -220,14 +221,7 @@ weak_alias (__hurd_directory_name_split, hurd_directory_name_split)
>  file_t
>  __file_name_lookup (const char *file_name, int flags, mode_t mode)
>  {
> -  error_t err;
> -  file_t result;
> -
> -  err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
> -				 file_name, flags, mode & ~_hurd_umask,
> -				 &result);
> -
> -  return err ? (__hurd_fail (err), MACH_PORT_NULL) : result;
> +  return __file_name_lookup_at (AT_FDCWD, 0, file_name, flags, mode);
>  }
>  weak_alias (__file_name_lookup, file_name_lookup)
>  
> diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
> index 6f30a067..25dab5a1 100644
> --- a/hurd/lookup-at.c
> +++ b/hurd/lookup-at.c
> @@ -36,9 +36,6 @@ __file_name_lookup_at (int fd, int at_flags,
>    if (err)
>      return (__hurd_fail (err), MACH_PORT_NULL);
>  
> -  if (fd == AT_FDCWD || file_name[0] == '/')
> -    return __file_name_lookup (file_name, flags, mode);
> -
>    if (empty != 0 && file_name[0] == '\0')
>      {
>        enum retry_type doretry;
> @@ -56,22 +53,44 @@ __file_name_lookup_at (int fd, int at_flags,
>        return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
>      }
>  
> -  file_t startdir;
> -  error_t use_init_port (int which, error_t (*operate) (mach_port_t))
> +  if (fd == AT_FDCWD || file_name[0] == '/')
>      {
> -      return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
> -	      : _hurd_ports_use (which, operate));
> +      err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
> +                                     file_name, flags, mode & ~_hurd_umask,
> +                                     &result);
> +      if (err)
> +        {
> +          __hurd_fail (err);
> +          return MACH_PORT_NULL;
> +        }
> +    }
> +  else
> +    {
> +      file_t startdir;
> +      /* We need to look the file up relative to the given directory (and
> +         not our cwd).  For this to work, we supply our own wrapper for
> +         _hurd_ports_use, which replaces cwd with our startdir.  */
> +      error_t use_init_port (int which, error_t (*operate) (mach_port_t))
> +        {
> +          return (which == INIT_PORT_CWDIR ? (*operate) (startdir)
> +	          : _hurd_ports_use (which, operate));
> +        }
> +
> +      err = HURD_DPORT_USE (fd, (startdir = port,
> +                                 __hurd_file_name_lookup (&use_init_port,
> +                                                          &__getdport, NULL,
> +                                                          file_name,
> +                                                          flags,
> +                                                          mode & ~_hurd_umask,
> +                                                          &result)));
> +      if (err)
> +        {
> +          __hurd_dfail (fd, err);
> +          return MACH_PORT_NULL;
> +        }
>      }
>  
> -  err = HURD_DPORT_USE (fd, (startdir = port,
> -			     __hurd_file_name_lookup (&use_init_port,
> -						      &__getdport, NULL,
> -						      file_name,
> -						      flags,
> -						      mode & ~_hurd_umask,
> -						      &result)));
> -
> -  return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
> +  return result;
>  }
>  
>  file_t
> -- 
> 2.39.1
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 2/3] hurd: Implement O_TMPFILE
  2023-01-30 12:52         ` [PATCH v2 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
@ 2023-02-01 22:34           ` Samuel Thibault
  0 siblings, 0 replies; 15+ messages in thread
From: Samuel Thibault @ 2023-02-01 22:34 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Applied, thanks!

Sergey Bugaev, le lun. 30 janv. 2023 15:52:15 +0300, a ecrit:
> This is a flag that causes open () to create a new, unnamed file in the
> same filesystem as the given directory. The file descriptor can be
> simply used in the creating process as a temporary file, or shared with
> children processes via fork (), or sent over a Unix socket. The file can
> be left anonymous, in which case it will be deleted from the backing
> file system once all copies of the file descriptor are closed, or given
> a permanent name with a linkat () call, such as the following:
> 
> int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700);
> /* Do something with the file... */
> linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH);
> 
> In between creating the file and linking it to the file system, it is
> possible to set the file content, mode, ownership, author, and other
> attributes, so that the file visibly appears in the file system (perhaps
> replacing another file) atomically, with all of its attributes already
> set up.
> 
> The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to
> user programs. Previously, dir_mkfile was used by glibc internally, in
> particular for implementing tmpfile (), but not exposed to user programs
> through a Unix-level API.
> 
> O_TMPFILE was initially introduced by Linux. This implementation is
> intended to be compatible with the Linux implementation, except that the
> O_EXCL flag is not given the special meaning when used together with
> O_TMPFILE, unlike on Linux.
> 
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
> ---
>  hurd/lookup-at.c               | 21 +++++++++++++++++++++
>  sysdeps/mach/hurd/bits/fcntl.h |  4 ++++
>  2 files changed, 25 insertions(+)
> 
> diff --git a/hurd/lookup-at.c b/hurd/lookup-at.c
> index 25dab5a1..88c83779 100644
> --- a/hurd/lookup-at.c
> +++ b/hurd/lookup-at.c
> @@ -29,6 +29,7 @@ __file_name_lookup_at (int fd, int at_flags,
>    error_t err;
>    file_t result;
>    int empty = at_flags & AT_EMPTY_PATH;
> +  int orig_flags;
>  
>    at_flags &= ~AT_EMPTY_PATH;
>  
> @@ -53,6 +54,10 @@ __file_name_lookup_at (int fd, int at_flags,
>        return err ? (__hurd_dfail (fd, err), MACH_PORT_NULL) : result;
>      }
>  
> +  orig_flags = flags;
> +  if (flags & O_TMPFILE)
> +    flags = O_DIRECTORY;
> +
>    if (fd == AT_FDCWD || file_name[0] == '/')
>      {
>        err = __hurd_file_name_lookup (&_hurd_ports_use, &__getdport, 0,
> @@ -90,6 +95,22 @@ __file_name_lookup_at (int fd, int at_flags,
>          }
>      }
>  
> +  if (orig_flags & O_TMPFILE)
> +    {
> +      /* What we have looked up is not the file itself, but actually
> +         the directory to create the file in.  Do that now.  */
> +      file_t dir = result;
> +
> +      err = __dir_mkfile (dir, orig_flags & ~(O_TMPFILE | O_DIRECTORY),
> +                          mode, &result);
> +      __mach_port_deallocate (__mach_task_self (), dir);
> +      if (err)
> +        {
> +          __hurd_fail (err);
> +          return MACH_PORT_NULL;
> +        }
> +    }
> +
>    return result;
>  }
>  
> diff --git a/sysdeps/mach/hurd/bits/fcntl.h b/sysdeps/mach/hurd/bits/fcntl.h
> index 970f79b8..c24a819e 100644
> --- a/sysdeps/mach/hurd/bits/fcntl.h
> +++ b/sysdeps/mach/hurd/bits/fcntl.h
> @@ -123,6 +123,10 @@
>  # define O_CLOEXEC	0x00400000 /* Set FD_CLOEXEC.  */
>  #endif
>  
> +#ifdef __USE_GNU
> +# define O_TMPFILE	0x00800000 /* Make a new unnamed file.  */
> +#endif
> +
>  
>  /* Controlling terminal flags.  These are understood only by `open',
>     and are not preserved once the file has been opened.  */
> -- 
> 2.39.1
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 3/3] hurd: Implement SHM_ANON
  2023-01-30 12:52         ` [PATCH v2 3/3] hurd: Implement SHM_ANON Sergey Bugaev
@ 2023-02-01 22:36           ` Samuel Thibault
  0 siblings, 0 replies; 15+ messages in thread
From: Samuel Thibault @ 2023-02-01 22:36 UTC (permalink / raw)
  To: Sergey Bugaev; +Cc: bug-hurd, libc-alpha

Applied, thanks!

Sergey Bugaev, le lun. 30 janv. 2023 15:52:16 +0300, a ecrit:
> This adds a special SHM_ANON value that can be passed into shm_open ()
> in place of a name. When called in this way, shm_open () will create a
> new anonymous shared memory file. The file will be created in the same
> way that other shared memory files are created (i.e., under /dev/shm/),
> except that it is not given a name and therefore cannot be reached from
> the file system, nor by other calls to shm_open (). This is accomplished
> by utilizing O_TMPFILE.
> 
> This is intended to be compatible with FreeBSD's API of the same name.
> 
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
> ---
>  posix/shm-directory.c             | 25 +++++++++++++++++++++----
>  rt/shm_open.c                     |  5 +++++
>  sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++++++++++++++++
>  3 files changed, 51 insertions(+), 4 deletions(-)
>  create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h
> 
> diff --git a/posix/shm-directory.c b/posix/shm-directory.c
> index 47e2355d..86d9fd8e 100644
> --- a/posix/shm-directory.c
> +++ b/posix/shm-directory.c
> @@ -23,17 +23,34 @@
>  #include <alloc_buffer.h>
>  #include <shm-directory.h>
>  #include <string.h>
> +#include <sys/mman.h>
> +#include <fcntl.h>
>  
>  int
>  __shm_get_name (struct shmdir_name *result, const char *name, bool sem_prefix)
>  {
> +  struct alloc_buffer buffer;
> +  size_t namelen;
> +
> +  buffer = alloc_buffer_create (result->name, sizeof (result->name));
> +  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
> +
> +#if defined (SHM_ANON) && defined (O_TMPFILE)
> +  if (name == SHM_ANON)
> +    {
> +      /* For SHM_ANON, we want shm_open () to pass O_TMPFILE to open (),
> +         with SHMDIR itself as the path.  So, leave it at that.  */
> +      alloc_buffer_add_byte (&buffer, 0);
> +      if (alloc_buffer_has_failed (&buffer))
> +        return -1;
> +      return 0;
> +    }
> +#endif
> +
>    while (name[0] == '/')
>      ++name;
> -  size_t namelen = strlen (name);
> +  namelen = strlen (name);
>  
> -  struct alloc_buffer buffer
> -    = alloc_buffer_create (result->name, sizeof (result->name));
> -  alloc_buffer_copy_bytes (&buffer, SHMDIR, strlen (SHMDIR));
>    if (sem_prefix)
>      alloc_buffer_copy_bytes (&buffer, "sem.", strlen ("sem."));
>    alloc_buffer_copy_bytes (&buffer, name, namelen + 1);
> diff --git a/rt/shm_open.c b/rt/shm_open.c
> index 48970bbc..6c1f4d60 100644
> --- a/rt/shm_open.c
> +++ b/rt/shm_open.c
> @@ -23,6 +23,7 @@
>  #include <shlib-compat.h>
>  #include <shm-directory.h>
>  #include <unistd.h>
> +#include <sys/mman.h>
>  
>  /* Open shared memory object.  */
>  int
> @@ -36,6 +37,10 @@ __shm_open (const char *name, int oflag, mode_t mode)
>      }
>  
>    oflag |= O_NOFOLLOW | O_CLOEXEC;
> +#if defined (SHM_ANON) && defined (O_TMPFILE)
> +  if (name == SHM_ANON)
> +    oflag |= O_TMPFILE;
> +#endif
>  
>    int fd = __open64_nocancel (dirname.name, oflag, mode);
>    if (fd == -1 && __glibc_unlikely (errno == EISDIR))
> diff --git a/sysdeps/mach/hurd/bits/mman_ext.h b/sysdeps/mach/hurd/bits/mman_ext.h
> new file mode 100644
> index 00000000..f022826e
> --- /dev/null
> +++ b/sysdeps/mach/hurd/bits/mman_ext.h
> @@ -0,0 +1,25 @@
> +/* System-specific extensions of <sys/mman.h>, Hurd version.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#ifndef _SYS_MMAN_H
> +# error "Never include <bits/mman_ext.h> directly; use <sys/mman.h> instead."
> +#endif
> +
> +#ifdef __USE_GNU
> +# define SHM_ANON	((const char *) 1)
> +#endif /* __USE_GNU  */
> -- 
> 2.39.1
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2023-02-01 22:36 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-12 11:46 [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
2022-12-12 11:46 ` [RFC PATCH 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
2022-12-12 11:46 ` [RFC PATCH 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
2023-01-29 23:25   ` Samuel Thibault
2023-01-30  9:53     ` Sergey Bugaev
2023-01-30  9:59       ` Samuel Thibault
2023-01-30 12:52         ` [PATCH v2 0/3] O_TMPFILE and SHM_ANON for the Hurd Sergey Bugaev
2023-01-30 12:52         ` [PATCH v2 1/3] hurd: Consolidate file_name_lookup implementation Sergey Bugaev
2023-02-01 19:06           ` Samuel Thibault
2023-01-30 12:52         ` [PATCH v2 2/3] hurd: Implement O_TMPFILE Sergey Bugaev
2023-02-01 22:34           ` Samuel Thibault
2023-01-30 12:52         ` [PATCH v2 3/3] hurd: Implement SHM_ANON Sergey Bugaev
2023-02-01 22:36           ` Samuel Thibault
2022-12-12 11:46 ` [RFC PATCH " Sergey Bugaev
2023-01-29 23:31 ` [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Samuel Thibault

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).