public inbox for libffi-discuss@sourceware.org
 help / color / mirror / Atom feed
* is fork() supported?
@ 2021-08-04 20:00 DJ Delorie
  2021-08-05  8:17 ` Andrew Haley
  0 siblings, 1 reply; 15+ messages in thread
From: DJ Delorie @ 2021-08-04 20:00 UTC (permalink / raw)
  To: libffi-discuss


I don't mean fork/exec (duh) but a fork() and keep going...

Consider a case where a process is using a file-backed mapping for
closures; a fork() doesn't isolate that backing between parent/child
so there's a chance (for example) the parent could deallocate a
closure that the child needs, etc...

I see four options...

1. Not supported.  Sorry.

2. Supported only if you have fork-able closure backings (mmap or
   maybe ktmpfile, but not file-backed) but this means documenting
   that (at least) selinux settings might affect this.  Caveat
   Programmer.

3. Some API that says "I need this" so it can fail in a more useful
   way than "segfault".

4. Fully supported, and we have some work to do to handle fork events.

Comments?

DJ


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-04 20:00 is fork() supported? DJ Delorie
@ 2021-08-05  8:17 ` Andrew Haley
  2021-08-05  8:27   ` Florian Weimer
  2021-08-05 18:13   ` DJ Delorie
  0 siblings, 2 replies; 15+ messages in thread
From: Andrew Haley @ 2021-08-05  8:17 UTC (permalink / raw)
  To: libffi-discuss

On 8/4/21 9:00 PM, DJ Delorie via Libffi-discuss wrote:
> I don't mean fork/exec (duh) but a fork() and keep going...
> 
> Consider a case where a process is using a file-backed mapping for
> closures; a fork() doesn't isolate that backing between parent/child
> so there's a chance (for example) the parent could deallocate a
> closure that the child needs, etc...
> 
> I see four options...
> 
> 1. Not supported.  Sorry.
> 
> 2. Supported only if you have fork-able closure backings (mmap or
>    maybe ktmpfile, but not file-backed) but this means documenting
>    that (at least) selinux settings might affect this.  Caveat
>    Programmer.
> 
> 3. Some API that says "I need this" so it can fail in a more useful
>    way than "segfault".
> 
> 4. Fully supported, and we have some work to do to handle fork events.

The bug that will never die.

https://bugzilla.redhat.com/show_bug.cgi?id=1249685
https://bugzilla.redhat.com/show_bug.cgi?id=531233
https://bugzilla.redhat.com/show_bug.cgi?id=772657

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-05  8:17 ` Andrew Haley
@ 2021-08-05  8:27   ` Florian Weimer
  2021-08-24 18:15     ` DJ Delorie
  2021-08-05 18:13   ` DJ Delorie
  1 sibling, 1 reply; 15+ messages in thread
From: Florian Weimer @ 2021-08-05  8:27 UTC (permalink / raw)
  To: Andrew Haley via Libffi-discuss

* Andrew Haley via Libffi-discuss:

> On 8/4/21 9:00 PM, DJ Delorie via Libffi-discuss wrote:
>> I don't mean fork/exec (duh) but a fork() and keep going...
>> 
>> Consider a case where a process is using a file-backed mapping for
>> closures; a fork() doesn't isolate that backing between parent/child
>> so there's a chance (for example) the parent could deallocate a
>> closure that the child needs, etc...
>> 
>> I see four options...
>> 
>> 1. Not supported.  Sorry.
>> 
>> 2. Supported only if you have fork-able closure backings (mmap or
>>    maybe ktmpfile, but not file-backed) but this means documenting
>>    that (at least) selinux settings might affect this.  Caveat
>>    Programmer.
>> 
>> 3. Some API that says "I need this" so it can fail in a more useful
>>    way than "segfault".
>> 
>> 4. Fully supported, and we have some work to do to handle fork events.
>
> The bug that will never die.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1249685
> https://bugzilla.redhat.com/show_bug.cgi?id=531233
> https://bugzilla.redhat.com/show_bug.cgi?id=772657

Well, there are two ways to fix it: Avoid the need for alias mappings by
switching to static trampolines, or create new alias mappings upon fork.
(A third option would be kernel support for private alias mappings.  A
fourth, get some form of explicit JIT support for SELinux.)

Thanks,
Florian


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-05  8:17 ` Andrew Haley
  2021-08-05  8:27   ` Florian Weimer
@ 2021-08-05 18:13   ` DJ Delorie
  2021-08-05 21:21     ` Jay K
  1 sibling, 1 reply; 15+ messages in thread
From: DJ Delorie @ 2021-08-05 18:13 UTC (permalink / raw)
  To: Andrew Haley, Florian Weimer; +Cc: libffi-discuss


>> I see four options...
>> 
>> 1. Not supported.  Sorry.
>> 
>> 2. Supported only if you have fork-able closure backings (mmap or
>>    maybe ktmpfile, but not file-backed) but this means documenting
>>    that (at least) selinux settings might affect this.  Caveat
>>    Programmer.
>> 
>> 3. Some API that says "I need this" so it can fail in a more useful
>>    way than "segfault".
>> 
>> 4. Fully supported, and we have some work to do to handle fork events.

Andrew Haley writes:
> The bug that will never die.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1249685
> https://bugzilla.redhat.com/show_bug.cgi?id=531233
> https://bugzilla.redhat.com/show_bug.cgi?id=772657

Sounds like a vote for (1) or (2) ;-)

Florian Weimer writes:
> Well, there are two ways to fix it: Avoid the need for alias mappings by
> switching to static trampolines, or create new alias mappings upon fork.
> (A third option would be kernel support for private alias mappings.  A
> fourth, get some form of explicit JIT support for SELinux.)

Sounds like a vote for (4)


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-05 18:13   ` DJ Delorie
@ 2021-08-05 21:21     ` Jay K
  2021-08-06  8:32       ` Andrew Haley
  0 siblings, 1 reply; 15+ messages in thread
From: Jay K @ 2021-08-05 21:21 UTC (permalink / raw)
  To: Andrew Haley, Florian Weimer, DJ Delorie; +Cc: libffi-discuss

> The bug that will never die.
 > 1249685 – python-cffi should not require execmem when selinux is enabled (redhat.com)<https://bugzilla.redhat.com/show_bug.cgi?id=1249685>

I think that is dying soon actually, in as much as libffi is the contributor (based on the title..)

 - Jay


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-05 21:21     ` Jay K
@ 2021-08-06  8:32       ` Andrew Haley
  0 siblings, 0 replies; 15+ messages in thread
From: Andrew Haley @ 2021-08-06  8:32 UTC (permalink / raw)
  To: Jay K, Florian Weimer, DJ Delorie; +Cc: libffi-discuss

On 8/5/21 10:21 PM, Jay K wrote:
>> The bug that will never die.
>  > 1249685 – python-cffi should not require execmem when selinux is enabled (redhat.com) <https://bugzilla.redhat.com/show_bug.cgi?id=1249685>
> 
> I think that is dying soon actually, in as much as libffi is the contributor (based on the title..)

What do you mean? That Python will give up using fork() and then ffi?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-05  8:27   ` Florian Weimer
@ 2021-08-24 18:15     ` DJ Delorie
  2021-08-24 18:27       ` Jay K
  0 siblings, 1 reply; 15+ messages in thread
From: DJ Delorie @ 2021-08-24 18:15 UTC (permalink / raw)
  To: libffi-discuss

Florian Weimer via Libffi-discuss <libffi-discuss@sourceware.org>
writes:
>> The bug that will never die.
> Well, there are two ways to fix it:

The thread ended without resolution...  we've (RH) got a bug report
against it, and I need to know if I can tell them "don't do that" or if
I'm obligated to pursue some fix.

So my question isn't CAN we fix it, it's MUST we fix it...  I.e. is this
a supported feature, or something that 'happened to work' so got abused?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-24 18:15     ` DJ Delorie
@ 2021-08-24 18:27       ` Jay K
  2021-08-24 18:45         ` DJ Delorie
  0 siblings, 1 reply; 15+ messages in thread
From: Jay K @ 2021-08-24 18:27 UTC (permalink / raw)
  To: libffi-discuss, DJ Delorie

On my point, sorry:


  1.  Sorry, I assumed mail from DJ was about djgpp, and that context was lacking in libffi. My mistake.
  2.  Either way, I was referring to the work that came after "trampfd", with the static trampolines that get mmaped repeatedly.
  3.  So libffi no longer has read/write/execute memory, or turns read/write into execute, correct?
  4.  So the implied assertion or question then, does that work very much sweep away "all such problems", or some remain?

 - Jay

________________________________
From: Libffi-discuss <libffi-discuss-bounces+jay.krell=cornell.edu@sourceware.org> on behalf of DJ Delorie via Libffi-discuss <libffi-discuss@sourceware.org>
Sent: Tuesday, August 24, 2021 6:15 PM
To: libffi-discuss@sourceware.org <libffi-discuss@sourceware.org>
Subject: Re: is fork() supported?

Florian Weimer via Libffi-discuss <libffi-discuss@sourceware.org>
writes:
>> The bug that will never die.
> Well, there are two ways to fix it:

The thread ended without resolution...  we've (RH) got a bug report
against it, and I need to know if I can tell them "don't do that" or if
I'm obligated to pursue some fix.

So my question isn't CAN we fix it, it's MUST we fix it...  I.e. is this
a supported feature, or something that 'happened to work' so got abused?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-24 18:27       ` Jay K
@ 2021-08-24 18:45         ` DJ Delorie
  2021-08-24 21:12           ` Kaz Kylheku (libffi)
  0 siblings, 1 reply; 15+ messages in thread
From: DJ Delorie @ 2021-08-24 18:45 UTC (permalink / raw)
  To: Jay K; +Cc: libffi-discuss

Jay K <jayk123@hotmail.com> writes:
> 1 Sorry, I assumed mail from DJ was about djgpp, and that context was
> lacking in libffi. My mistake.

Ha!  Yeah, not about djgpp, which I think "just works" ;-)

> 3 So libffi no longer has read/write/execute memory, or turns
> read/write into execute, correct?

The problem case is when libffi can't mmap a chunk of write+exec memory,
and resorts to writing to a file and mapping the file read+exec, which
happens in Linux with certain selinux security profiles.  Those mappings
get shared between a parent/child pair.

> 4 So the implied assertion or question then, does that work very much
> sweep away "all such problems", or some remain?

I think the trampfd static templates *should* solve the problem on
platforms that support mmap().

But, as I said, this is besides the point - the question is "is it
supported", not "is it possible".  Given we've added at least one
closure method that violates the fork() premise - was adding that method
a bug, or does it imply that it's not supported?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-24 18:45         ` DJ Delorie
@ 2021-08-24 21:12           ` Kaz Kylheku (libffi)
  2021-08-24 21:58             ` Jay K
  0 siblings, 1 reply; 15+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-08-24 21:12 UTC (permalink / raw)
  To: DJ Delorie; +Cc: Jay K, libffi-discuss, Libffi-discuss

On 2021-08-24 11:45, DJ Delorie via Libffi-discuss wrote:
> Jay K <jayk123@hotmail.com> writes:
>> 1 Sorry, I assumed mail from DJ was about djgpp, and that context was
>> lacking in libffi. My mistake.
> 
> Ha!  Yeah, not about djgpp, which I think "just works" ;-)
> 
>> 3 So libffi no longer has read/write/execute memory, or turns
>> read/write into execute, correct?
> 
> The problem case is when libffi can't mmap a chunk of write+exec 
> memory,
> and resorts to writing to a file and mapping the file read+exec, which
> happens in Linux with certain selinux security profiles.

Isn't there some way way set up two anonymous, virtual regions such that
they alias to the same frames? One can be read+exec, and the other 
write.

I could easily write code in the kernel to take some user space pages
of the calling process and make them also appear somewhere else in
the address space.

I don't see anything like that in the Linux mmap among any of the flags
and whatnot.

It would be a useful extension.

Imagine a MAP_ALIAS flag, which requires the void *addr argument to be
present. addr, normally used with MAP_FIXED, in this case would specify
the virtual address of the target range to be aliased by the returned
mapping.

Being able to obtain a writable alias of executable space is
a security risk though: the risk that an attacker can inject a mmap
call to create an writable alias of executable space.
Some new PROT_* bit could help mitigate: the read+exec mapping would 
have
this bit to indicate that such aliasing of it is permitted.
Mappings like shared libs and executables would not have this bit,
only libffi's closure buffer.

(Attackers who can call mmap to obtain a writable mapping could
arguably perpetrate the same workaround as libffi: write to a file
and then read+exec map it. So why worry about this too much.)

Summary:

   /* new flags + kernel support */
   #define MAP_ALIAS        ...
   #define PROT_ALIAS_OK ...

   void *closures = mmap(NULL, len,
                         PROT_READ | PROT_EXEC | PROT_ALIAS_OK,
                         MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);

   void *write_closures = mmap(closures, len,
                               PROT_WRITE,
                               MAP_ALIAS | MAP_PRIVATE, -1, 0);


Write closures through write_closures pointer, execute via
closures pointer.

The kernel code for MAP_ALIAS has to ensure that fork
observes this linkage. That is tricky. The child process gets
its own copy of this memory, and the two aliased views of it.
The views cannot just be subject to independent COW because
their aliasing relationship will break.

One more detail: mprotect should disallow PROT_ALIAS_OK.
Only the original mmap must be able to specify PROT_ALIAS_OK,
which is then a mapping's immutable flag.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-24 21:12           ` Kaz Kylheku (libffi)
@ 2021-08-24 21:58             ` Jay K
  2021-08-25  9:27               ` Andrew Haley
  0 siblings, 1 reply; 15+ messages in thread
From: Jay K @ 2021-08-24 21:58 UTC (permalink / raw)
  To: Kaz Kylheku (libffi), DJ Delorie; +Cc: libffi-discuss, Libffi-discuss

I have used the multi-mmap of static code in another project.

Can libffi not always do that?

 > The problem case is when libffi can't mmap a chunk of write+exec memory,

Why does it need to, given the multi-mmap of static trampoline approach?

 > Isn't there some way way set up two anonymous, virtual regions such
 > that they alias to the same frames? One can be read+exec, and the other write.

People speak of "arbitrary codegen" (ACG) or unspecififically "JIT", and
"something else", I have never seen a name for, "non-arbitrary codegen"?,
the scenario where there is static code, and it can be mapped multiple times.

In ACG, the two mapping approach is used by some systems.
The writable address may be "difficult" for an attacker to find.
It ups the arms race, some. I cannot quantify that.

The writing might also be done from another process (in Windows
this is "easy", at the API level, given an adequately factored JIT..
I realize the double whammy here, of both likely different addresses
and address spaces, on existing code).

The executing process might even be prohibited from having the writable alias,
which is surely a very nice escalation.

But, all that aside, given the recent work in libffi, the follow-up
to "trampfd", why is a writable mapping every needed?

I believe MacOSX also has this as an extension like this, in that instead of
giving an fd to map, you can give an address, to another function,
I cannot find the name. This can be used, I guess, to avoid remapping the entire .so.

Thank you,
 - Jay

________________________________
From: Kaz Kylheku (libffi) <382-725-6798@kylheku.com>
Sent: Tuesday, August 24, 2021 9:12 PM
To: DJ Delorie <dj@redhat.com>
Cc: Jay K <jayk123@hotmail.com>; libffi-discuss@sourceware.org <libffi-discuss@sourceware.org>; Libffi-discuss <libffi-discuss-bounces+382-725-6798=kylheku.com@sourceware.org>
Subject: Re: is fork() supported?

On 2021-08-24 11:45, DJ Delorie via Libffi-discuss wrote:
> Jay K <jayk123@hotmail.com> writes:
>> 1 Sorry, I assumed mail from DJ was about djgpp, and that context was
>> lacking in libffi. My mistake.
>
> Ha!  Yeah, not about djgpp, which I think "just works" ;-)
>
>> 3 So libffi no longer has read/write/execute memory, or turns
>> read/write into execute, correct?
>
> The problem case is when libffi can't mmap a chunk of write+exec
> memory,
> and resorts to writing to a file and mapping the file read+exec, which
> happens in Linux with certain selinux security profiles.

Isn't there some way way set up two anonymous, virtual regions such that
they alias to the same frames? One can be read+exec, and the other
write.

I could easily write code in the kernel to take some user space pages
of the calling process and make them also appear somewhere else in
the address space.

I don't see anything like that in the Linux mmap among any of the flags
and whatnot.

It would be a useful extension.

Imagine a MAP_ALIAS flag, which requires the void *addr argument to be
present. addr, normally used with MAP_FIXED, in this case would specify
the virtual address of the target range to be aliased by the returned
mapping.

Being able to obtain a writable alias of executable space is
a security risk though: the risk that an attacker can inject a mmap
call to create an writable alias of executable space.
Some new PROT_* bit could help mitigate: the read+exec mapping would
have
this bit to indicate that such aliasing of it is permitted.
Mappings like shared libs and executables would not have this bit,
only libffi's closure buffer.

(Attackers who can call mmap to obtain a writable mapping could
arguably perpetrate the same workaround as libffi: write to a file
and then read+exec map it. So why worry about this too much.)

Summary:

   /* new flags + kernel support */
   #define MAP_ALIAS        ...
   #define PROT_ALIAS_OK ...

   void *closures = mmap(NULL, len,
                         PROT_READ | PROT_EXEC | PROT_ALIAS_OK,
                         MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);

   void *write_closures = mmap(closures, len,
                               PROT_WRITE,
                               MAP_ALIAS | MAP_PRIVATE, -1, 0);


Write closures through write_closures pointer, execute via
closures pointer.

The kernel code for MAP_ALIAS has to ensure that fork
observes this linkage. That is tricky. The child process gets
its own copy of this memory, and the two aliased views of it.
The views cannot just be subject to independent COW because
their aliasing relationship will break.

One more detail: mprotect should disallow PROT_ALIAS_OK.
Only the original mmap must be able to specify PROT_ALIAS_OK,
which is then a mapping's immutable flag.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-24 21:58             ` Jay K
@ 2021-08-25  9:27               ` Andrew Haley
  2021-08-25 15:58                 ` Jay K
  2021-08-25 16:59                 ` Kaz Kylheku (libffi)
  0 siblings, 2 replies; 15+ messages in thread
From: Andrew Haley @ 2021-08-25  9:27 UTC (permalink / raw)
  To: libffi-discuss

On 8/24/21 10:58 PM, Jay K via Libffi-discuss wrote:

> I believe MacOSX also has this as an extension like this, in that
> instead of giving an fd to map, you can give an address, to another
> function, I cannot find the name. This can be used, I guess, to
> avoid remapping the entire .so.

Mac solves the problem in a much nicer way, one that is JIT-friendly
but does not allow pages to be both W and X.

Here's how it works:

    Call mmap with the MAP_JIT option to create a memory region for
    the new machine instructions.

    Call pthread_jit_write_protect_np() with the value false to disable
    JIT write protections for the memory region in the current thread.

    Write the machine instructions to the memory region.

Note that this is *per thread*. other threads will simply continue to
execute code in the JITted region. The JIT can generate code, but can
not execute any JITted code until it calls
pthread_jit_write_protect_np(true). Of course this requires threads to
have differently-mapped regions. It would be very nice to have in
Linux. It is the right way to do it.

https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-25  9:27               ` Andrew Haley
@ 2021-08-25 15:58                 ` Jay K
  2021-08-25 16:59                 ` Kaz Kylheku (libffi)
  1 sibling, 0 replies; 15+ messages in thread
From: Jay K @ 2021-08-25 15:58 UTC (permalink / raw)
  To: libffi-discuss, Andrew Haley

 > pthread_jit_write_protect_np

It is all tradeoffs I think.
Attacker can still try to write to the pages while they are writable,
in any of the approaches (Apple, two mappings, two processes, etc.)

But yes that does of course help.

But I again I think all those methods, including the Apple one,
are needed only for "arbitrary" code gen, not codegen that can be made
data driven (n thunks, array of n to drive them), and the code is
therefore constant, and there "just" needs to be
a way to make multiple copies/mappings of it (to expand the pool of
n by another n).

vm_remap is the Apple function I was thinking of.

 - Jay

________________________________
From: Libffi-discuss <libffi-discuss-bounces+jay.krell=cornell.edu@sourceware.org> on behalf of Andrew Haley via Libffi-discuss <libffi-discuss@sourceware.org>
Sent: Wednesday, August 25, 2021 9:27 AM
To: libffi-discuss@sourceware.org <libffi-discuss@sourceware.org>
Subject: Re: is fork() supported?

On 8/24/21 10:58 PM, Jay K via Libffi-discuss wrote:

> I believe MacOSX also has this as an extension like this, in that
> instead of giving an fd to map, you can give an address, to another
> function, I cannot find the name. This can be used, I guess, to
> avoid remapping the entire .so.

Mac solves the problem in a much nicer way, one that is JIT-friendly
but does not allow pages to be both W and X.

Here's how it works:

    Call mmap with the MAP_JIT option to create a memory region for
    the new machine instructions.

    Call pthread_jit_write_protect_np() with the value false to disable
    JIT write protections for the memory region in the current thread.

    Write the machine instructions to the memory region.

Note that this is *per thread*. other threads will simply continue to
execute code in the JITted region. The JIT can generate code, but can
not execute any JITted code until it calls
pthread_jit_write_protect_np(true). Of course this requires threads to
have differently-mapped regions. It would be very nice to have in
Linux. It is the right way to do it.

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fdocumentation%2Fapple-silicon%2Fporting-just-in-time-compilers-to-apple-silicon&amp;data=04%7C01%7C%7C8e3a0f2a2e084358a7f708d967aa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637654804622490592%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2FCuwk5zssydzfZqBgj9q%2BCt32dJ07Y8DyvEoRwZ2ZhI%3D&amp;reserved=0

--
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&amp;data=04%7C01%7C%7C8e3a0f2a2e084358a7f708d967aa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637654804622490592%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=6hzVYMKexxJS2Rj39mZMOzsuh3F%2FF6fI1RSEYgFjk9Q%3D&amp;reserved=0>
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkeybase.io%2Fandrewhaley&amp;data=04%7C01%7C%7C8e3a0f2a2e084358a7f708d967aa962f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637654804622500585%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=9h6U8eCdUgolrNdQgPDktROXmT2J1JMhchU4DJYAOyg%3D&amp;reserved=0
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-25  9:27               ` Andrew Haley
  2021-08-25 15:58                 ` Jay K
@ 2021-08-25 16:59                 ` Kaz Kylheku (libffi)
  2021-08-25 21:17                   ` Andrew Haley
  1 sibling, 1 reply; 15+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-08-25 16:59 UTC (permalink / raw)
  To: Andrew Haley; +Cc: libffi-discuss

On 2021-08-25 02:27, Andrew Haley via Libffi-discuss wrote:
> Note that this is *per thread*. other threads will simply continue to
> execute code in the JITted region. The JIT can generate code, but can
> not execute any JITted code until it calls
> pthread_jit_write_protect_np(true). Of course this requires threads to
> have differently-mapped regions. It would be very nice to have in
> Linux. It is the right way to do it.

It's *a* way to do it, but tweaking the address space so that it looks
different in different threads seems extremely wrong and repugnant.

The threads of a process should live in exactly the same address space,
in every detail: every page, every protection bit.

There should be no TLB switching/flushing when we task switch from
one thread to another in the same process.

Of course, this is just another requirement, and no requirement
is absolutely non-negotiable otherwise otherwise we have religion
and not engineering.

However, this is a pretty firm, fundamental thing, I would think!

If it was done without actually making different memory maps per
thread that would be great.

For instance, suppose the mapping that is opened up for writing for
one thread is actually write protected to all threads. What happens
is this:

1. The "blessed" thread which owns the JIT mapping performs a write.
2. This traps into the kernel.
3. The kernel sees, aha, this is the blessed thread which is allowed
    to write.
4. The kernel emulates the write.
5. The thread's instruction pointer and other machine state is fixed
    up to look like it had done the write.
6. The thread is restarted.

If the thread is not the blessed one, a fatal signal is generated,
as usual.

One the JITted code is installed, the blessedness goes away; all
threads bomb if they write access it.

Prior art for this, is obviously, things like simulating misaligned
memory accesses on machines that trap on misalignment.

The downside is that it's slow for writing a large amount of JIT
material. Though installation of JIT material is amortized over the
cost of compiling it in the first place, that may still be
unacceptable overhead. (Also, it's not always amortized. Some
ahead-of-time compiling situations would also use this mechanism;
JIT is just a convenient acronym. Not all code loading is done by
mapping directly into memory. In some languages, an code object
might be read containing the executable code as a blob literal,
which is then installed as if it came from JIT).

Really, you need a dedicated write-like syscall for this, which the
blessed thread can use to specify the bytes to be written to the
JIT memory. That is similar to the solution involving writes to
a file, except there is no file.

In Linux, a possible place to dump this this might be oh, let's see,
prctl?

   prctl(PR_WRITE_JIT, (long) dst, (long) src, (long) size);

[dst, dst + size) must be a MAP_JIT mapping indicating the
calling thread as being blessed, otherwise it fails.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: is fork() supported?
  2021-08-25 16:59                 ` Kaz Kylheku (libffi)
@ 2021-08-25 21:17                   ` Andrew Haley
  0 siblings, 0 replies; 15+ messages in thread
From: Andrew Haley @ 2021-08-25 21:17 UTC (permalink / raw)
  To: Kaz Kylheku (libffi); +Cc: libffi-discuss

On 8/25/21 5:59 PM, Kaz Kylheku (libffi) wrote:
> On 2021-08-25 02:27, Andrew Haley via Libffi-discuss wrote:
>> Note that this is *per thread*. other threads will simply continue to
>> execute code in the JITted region. The JIT can generate code, but can
>> not execute any JITted code until it calls
>> pthread_jit_write_protect_np(true). Of course this requires threads to
>> have differently-mapped regions. It would be very nice to have in
>> Linux. It is the right way to do it.
> 
> It's *a* way to do it, but tweaking the address space so that it looks
> different in different threads seems extremely wrong and repugnant.

It works a treat.

> The threads of a process should live in exactly the same address space,
> in every detail: every page, every protection bit.
> 
> There should be no TLB switching/flushing when we task switch from
> one thread to another in the same process.
>
> Of course, this is just another requirement, and no requirement
> is absolutely non-negotiable otherwise otherwise we have religion
> and not engineering.
I tend to treat sentences with a "should" but no "because" as ill-formed.
Only the pope gets to talk like that!  ;-)

Why do you care anyway? It's not as if such things will be at all common.
It'll only happen if a thread's time slice expires while generating code.

> However, this is a pretty firm, fundamental thing, I would think!
>
> If it was done without actually making different memory maps per
> thread that would be great.
> 
> For instance, suppose the mapping that is opened up for writing for
> one thread is actually write protected to all threads. What happens
> is this:
> 
> 1. The "blessed" thread which owns the JIT mapping performs a write.
> 2. This traps into the kernel.
> 3. The kernel sees, aha, this is the blessed thread which is allowed
>     to write.
> 4. The kernel emulates the write.
> 5. The thread's instruction pointer and other machine state is fixed
>     up to look like it had done the write.
> 6. The thread is restarted.

From userspace's point of view, that looks exactly the same, but much
slower. Seems to me that it's going to be more costly than an occasional
TLB flush; and if you have more processes than processors you're doing
that anyway.

> If the thread is not the blessed one, a fatal signal is generated,
> as usual.
> 
> One the JITted code is installed, the blessedness goes away; all
> threads bomb if they write access it.
>
> Prior art for this, is obviously, things like simulating misaligned
> memory accesses on machines that trap on misalignment.
> 
> The downside is that it's slow for writing a large amount of JIT
> material. Though installation of JIT material is amortized over the
> cost of compiling it in the first place, that may still be
> unacceptable overhead. (Also, it's not always amortized. Some
> ahead-of-time compiling situations would also use this mechanism;
> JIT is just a convenient acronym. Not all code loading is done by
> mapping directly into memory. In some languages, an code object
> might be read containing the executable code as a blob literal,
> which is then installed as if it came from JIT).
> 
> Really, you need a dedicated write-like syscall for this, which the
> blessed thread can use to specify the bytes to be written to the
> JIT memory. That is similar to the solution involving writes to
> a file, except there is no file.
> 
> In Linux, a possible place to dump this this might be oh, let's see,
> prctl?
> 
>    prctl(PR_WRITE_JIT, (long) dst, (long) src, (long) size);
> 
> [dst, dst + size) must be a MAP_JIT mapping indicating the
> calling thread as being blessed, otherwise it fails.

Hmm, that might work, but would be a pain. In Java we
copy code around and then fix up the relocs at the destination
site.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-08-25 21:17 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-04 20:00 is fork() supported? DJ Delorie
2021-08-05  8:17 ` Andrew Haley
2021-08-05  8:27   ` Florian Weimer
2021-08-24 18:15     ` DJ Delorie
2021-08-24 18:27       ` Jay K
2021-08-24 18:45         ` DJ Delorie
2021-08-24 21:12           ` Kaz Kylheku (libffi)
2021-08-24 21:58             ` Jay K
2021-08-25  9:27               ` Andrew Haley
2021-08-25 15:58                 ` Jay K
2021-08-25 16:59                 ` Kaz Kylheku (libffi)
2021-08-25 21:17                   ` Andrew Haley
2021-08-05 18:13   ` DJ Delorie
2021-08-05 21:21     ` Jay K
2021-08-06  8:32       ` Andrew Haley

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).