* libffi & fork [not found] <CAP+vr+PX9gw7Ot6Zc7AN8Gks9P+WHVQg8bX5S7YRnTRyRBHp7A@mail.gmail.com> @ 2012-04-24 19:41 ` Gaash Hazan 2012-04-25 9:11 ` Andrew Haley 0 siblings, 1 reply; 12+ messages in thread From: Gaash Hazan @ 2012-04-24 19:41 UTC (permalink / raw) To: libffi-discuss Hi experts, I am new to libffi, so forgive me for my ignorance. I have a python crash involving ctypes and functions callbacks (read closures) and fork. The crash happens when process is forked, some callbacks are modified and a function callback is made. I suspect that the problem could be related to libffi closures and fork. dlmmap() at libffi/closures.c allocates memory block that has to be both read-write and read-execute. To achieve that, libffi creates a temp file and performs two mmap-s to the that file, the first read-write and the 2nd read-execute. 7fe27d64f000-7fe27d650000 rw-s 00000000 08:02 1840563 /tmp/ffiHf53x5 (deleted) 7fe27d650000-7fe27d651000 r-xs 00000000 08:02 1840563 /tmp/ffiHf53x5 (deleted) The thing is that those maps are marked as shared. Hence when the process is forked the allocated(mapped) memory block is *shared* between the processes (and not copied-on-write). So, when father process changes something in that memory block, the child process will be affected as well, creating unplanned shared-memory block between the two processes. I suspect that this is not the intended behavior and that it leads to the crash. snips from /proc/XXX/maps: father PID=7975, child PID=7976 ---------------------------------------------------------------------------------------- $cat /proc/7975/maps | grep /tmp/ 7fc537d49000-7fc537d4a000 rw-s 00000000 08:02 1840370 /tmp/ffiqyLKFR (deleted) 7fc537d4a000-7fc537d4b000 r-xs 00000000 08:02 1840370 /tmp/ffiqyLKFR (deleted) $cat /proc/7976/maps | grep /tmp/ 7fc537d49000-7fc537d4a000 rw-s 00000000 08:02 1840370 /tmp/ffiqyLKFR (deleted) 7fc537d4a000-7fc537d4b000 r-xs 00000000 08:02 1840370 /tmp/ffiqyLKFR (deleted) test .py: ----------- import os,time import ctypes os.fork() print str(os.getpid()) time.sleep( 1000) system info: centos 6.0, pyhton 2.6.5, selinux disabled, x86_64. Also tested with python 2.7.3 Thanks, Gaash ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-24 19:41 ` libffi & fork Gaash Hazan @ 2012-04-25 9:11 ` Andrew Haley 2012-04-25 17:33 ` Gaash Hazan 0 siblings, 1 reply; 12+ messages in thread From: Andrew Haley @ 2012-04-25 9:11 UTC (permalink / raw) To: libffi-discuss On 04/24/2012 08:40 PM, Gaash Hazan wrote: > I am new to libffi, so forgive me for my ignorance. > > I have a python crash involving ctypes and functions callbacks (read > closures) and fork. The crash happens when process is forked, some > callbacks are modified and a function callback is made. I suspect that > the problem could be related to libffi closures and fork. There is, as far as I can see, only one way to remedy this: allow python to map anonymous memory with both write and execute permission. Then it should work fine. Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-25 9:11 ` Andrew Haley @ 2012-04-25 17:33 ` Gaash Hazan 2012-04-26 10:00 ` Andrew Haley 0 siblings, 1 reply; 12+ messages in thread From: Gaash Hazan @ 2012-04-25 17:33 UTC (permalink / raw) To: Andrew Haley; +Cc: libffi-discuss > There is, as far as I can see, only one way to remedy this: allow > python to map anonymous memory with both write and execute permission. > Then it should work fine. > I believe memory allocation is a service provided by the OS and not by the application or interpreter. Python does not provide memory allocation service to libffi. In this case libffi creates read-write-exec memory block using mmap to a tmp file. I guess the problem is common to libff users and it is not unique to python. libffi uses mmap with MAP_SHARED at libffi:closures.c:dlmap(). What was the reason for using MAP_SHARED in the first place? I think MAP_PRIVATE would create the desired behavior of copy-on-write when forked. Would that be a proper fix? Gaash ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-25 17:33 ` Gaash Hazan @ 2012-04-26 10:00 ` Andrew Haley 2012-04-29 15:34 ` Gaash Hazan 0 siblings, 1 reply; 12+ messages in thread From: Andrew Haley @ 2012-04-26 10:00 UTC (permalink / raw) To: libffi-discuss On 04/25/2012 06:32 PM, Gaash Hazan wrote: >> There is, as far as I can see, only one way to remedy this: allow >> python to map anonymous memory with both write and execute permission. >> Then it should work fine. > > I believe memory allocation is a service provided by the OS and not by > the application or interpreter. The usual problem is a mechanism such as SELinux that is forbidding Python from mmap()ing PROT_WRITE|PROT_EXEC . If you label the executable unconfined_execmem_exec_t it may work because then libffi won't bother with all this multiple maps though a shared file descriptor business. > Python does not provide memory allocation service to libffi. In this > case libffi creates read-write-exec memory block using mmap to a tmp > file. I guess the problem is common to libff users and it is not > unique to python. It's special to Python because Python is the only common libffi client that forks and then tries to use the closures. > libffi uses mmap with MAP_SHARED at libffi:closures.c:dlmap(). What > was the reason for using MAP_SHARED in the first place? > > I think MAP_PRIVATE would create the desired behavior of copy-on-write > when forked. Would that be a proper fix? No, memory regions that are backed by a real file are always shared between processes. There is a single file descriptor that references the block of memory that holds the closures. The only way you'll get it to work is to create a new file. Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-26 10:00 ` Andrew Haley @ 2012-04-29 15:34 ` Gaash Hazan 2012-04-29 17:58 ` Anthony Green 0 siblings, 1 reply; 12+ messages in thread From: Gaash Hazan @ 2012-04-29 15:34 UTC (permalink / raw) To: Andrew Haley; +Cc: libffi-discuss Hi Andrew, Thanks for the detailed answer. I think I found the root cause of the problem. The libffi version that I use (3.0.5) contains a bug in closures.c:selinux_enabled_check() that incorrectly detected non-selinux systsem as selinux system. Because system is detected (incorrectly) as selinux dlmap() does not attempts to do mmap() for private anonymous block with write-exec protection. It reverts to the trick with two mmap()s on a tmp file. But trick does not with fork. If the system is detected (correctly) as non-selinux, then dlmp() would use anonymous write-exec mmap() and it would not use the trick. Hence the fork problem would not occurred. The selinux detected problem was fixed in https://github.com/atgreen/libffi/commit/eaf444eabc4c78703c0f98ac0197b1619c1b1bef#src/closures.c (closures.c line 149) Unfortunately Redhat 6.2 and 6.3-bets uses libffi 3.0.5 that has this problem. Regards, Gaash On Thu, Apr 26, 2012 at 1:00 PM, Andrew Haley <aph@redhat.com> wrote: > > On 04/25/2012 06:32 PM, Gaash Hazan wrote: > >> There is, as far as I can see, only one way to remedy this: allow > >> python to map anonymous memory with both write and execute permission. > >> Then it should work fine. > > > > I believe memory allocation is a service provided by the OS and not by > > the application or interpreter. > > The usual problem is a mechanism such as SELinux that is forbidding > Python from mmap()ing PROT_WRITE|PROT_EXEC . If you label the > executable unconfined_execmem_exec_t it may work because then libffi > won't bother with all this multiple maps though a shared file > descriptor business. > > > Python does not provide memory allocation service to libffi. In this > > case libffi creates read-write-exec memory block using mmap to a tmp > > file. I guess the problem is common to libff users and it is not > > unique to python. > > It's special to Python because Python is the only common libffi client > that forks and then tries to use the closures. > > > libffi uses mmap with MAP_SHARED at libffi:closures.c:dlmap(). What > > was the reason for using MAP_SHARED in the first place? > > > > I think MAP_PRIVATE would create the desired behavior of copy-on-write > > when forked. Would that be a proper fix? > > No, memory regions that are backed by a real file are always shared > between processes. There is a single file descriptor that > references the block of memory that holds the closures. The only > way you'll get it to work is to create a new file. > > Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-29 15:34 ` Gaash Hazan @ 2012-04-29 17:58 ` Anthony Green 2012-04-30 8:32 ` Andrew Haley 0 siblings, 1 reply; 12+ messages in thread From: Anthony Green @ 2012-04-29 17:58 UTC (permalink / raw) To: Gaash Hazan; +Cc: Andrew Haley, libffi-discuss On Sun, Apr 29, 2012 at 11:34 AM, Gaash Hazan <gaash@qwilt.com> wrote: > The selinux detected problem was fixed in > https://github.com/atgreen/libffi/commit/eaf444eabc4c78703c0f98ac0197b1619c1b1bef#src/closures.c > (closures.c line 149) > > Unfortunately Redhat 6.2 and 6.3-bets uses libffi 3.0.5 that has this problem. Thanks Gaash. Andrew - it would be nice if we could upgrade RHEL's libffi to a non-ABI breaking 3.0.10. If, however, it makes more sense to cherry-pick patches since 3.0.5, I can see some nice ones from mjw and jakub back in 2010, and there are likely more. If you think we should open an RFE I can propose some specific patches for consideration. There may only be 4 or 5 and they are all pretty obvious. AG ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-29 17:58 ` Anthony Green @ 2012-04-30 8:32 ` Andrew Haley 2012-04-30 19:16 ` Anthony Green 0 siblings, 1 reply; 12+ messages in thread From: Andrew Haley @ 2012-04-30 8:32 UTC (permalink / raw) To: libffi-discuss On 04/29/2012 06:58 PM, Anthony Green wrote: > On Sun, Apr 29, 2012 at 11:34 AM, Gaash Hazan <gaash@qwilt.com> wrote: >> The selinux detected problem was fixed in >> https://github.com/atgreen/libffi/commit/eaf444eabc4c78703c0f98ac0197b1619c1b1bef#src/closures.c >> (closures.c line 149) >> >> Unfortunately Redhat 6.2 and 6.3-bets uses libffi 3.0.5 that has this problem. > > Thanks Gaash. > > Andrew - it would be nice if we could upgrade RHEL's libffi to a > non-ABI breaking 3.0.10. If, however, it makes more sense to > cherry-pick patches since 3.0.5, I can see some nice ones from mjw and > jakub back in 2010, and there are likely more. If you think we should > open an RFE I can propose some specific patches for consideration. > There may only be 4 or 5 and they are all pretty obvious. I think we should do that. Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-30 8:32 ` Andrew Haley @ 2012-04-30 19:16 ` Anthony Green 2012-05-01 7:56 ` Andrew Haley 0 siblings, 1 reply; 12+ messages in thread From: Anthony Green @ 2012-04-30 19:16 UTC (permalink / raw) To: Andrew Haley; +Cc: libffi-discuss On Mon, Apr 30, 2012 at 4:32 AM, Andrew Haley <aph@redhat.com> wrote: >> Andrew - it would be nice if we could upgrade RHEL's libffi to a >> non-ABI breaking 3.0.10. If, however, it makes more sense to >> cherry-pick patches since 3.0.5, I can see some nice ones from mjw and >> jakub back in 2010, and there are likely more. If you think we should >> open an RFE I can propose some specific patches for consideration. >> There may only be 4 or 5 and they are all pretty obvious. > > I think we should do that. Hmm.. check out this bug: https://bugzilla.redhat.com/show_bug.cgi?id=772657 If our selinux state detection code really is this fragile, maybe we should link against libselinux and use is_selinux_enabled() as suggested in comment 4. AG ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-04-30 19:16 ` Anthony Green @ 2012-05-01 7:56 ` Andrew Haley 2012-05-02 20:23 ` Anthony Green 0 siblings, 1 reply; 12+ messages in thread From: Andrew Haley @ 2012-05-01 7:56 UTC (permalink / raw) To: Anthony Green; +Cc: libffi-discuss On 04/30/2012 08:16 PM, Anthony Green wrote: > On Mon, Apr 30, 2012 at 4:32 AM, Andrew Haley <aph@redhat.com> wrote: >>> Andrew - it would be nice if we could upgrade RHEL's libffi to a >>> non-ABI breaking 3.0.10. If, however, it makes more sense to >>> cherry-pick patches since 3.0.5, I can see some nice ones from mjw and >>> jakub back in 2010, and there are likely more. If you think we should >>> open an RFE I can propose some specific patches for consideration. >>> There may only be 4 or 5 and they are all pretty obvious. >> >> I think we should do that. > > Hmm.. check out this bug: > https://bugzilla.redhat.com/show_bug.cgi?id=772657 > > If our selinux state detection code really is this fragile, maybe we > should link against libselinux and use is_selinux_enabled() as > suggested in comment 4. Would it not make far more sense simply to try to map the region and fall back if that fails? Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-05-01 7:56 ` Andrew Haley @ 2012-05-02 20:23 ` Anthony Green 2012-05-04 9:00 ` Andrew Haley 0 siblings, 1 reply; 12+ messages in thread From: Anthony Green @ 2012-05-02 20:23 UTC (permalink / raw) To: Andrew Haley; +Cc: libffi-discuss On Tue, May 1, 2012 at 3:56 AM, Andrew Haley <aph@redhat.com> wrote: >> Hmm.. check out this bug: >> https://bugzilla.redhat.com/show_bug.cgi?id=772657 >> >> If our selinux state detection code really is this fragile, maybe we >> should link against libselinux and use is_selinux_enabled() as >> suggested in comment 4. > > Would it not make far more sense simply to try to map the region > and fall back if that fails? Wouldn't that generate spurious SELinux warnings/errors? I'd rather not have that as expected behaviour. AG ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-05-02 20:23 ` Anthony Green @ 2012-05-04 9:00 ` Andrew Haley 2012-05-05 13:19 ` Anthony Green 0 siblings, 1 reply; 12+ messages in thread From: Andrew Haley @ 2012-05-04 9:00 UTC (permalink / raw) To: libffi-discuss On 05/02/2012 09:23 PM, Anthony Green wrote: > On Tue, May 1, 2012 at 3:56 AM, Andrew Haley <aph@redhat.com> wrote: >>> Hmm.. check out this bug: >>> https://bugzilla.redhat.com/show_bug.cgi?id=772657 >>> >>> If our selinux state detection code really is this fragile, maybe we >>> should link against libselinux and use is_selinux_enabled() as >>> suggested in comment 4. >> >> Would it not make far more sense simply to try to map the region >> and fall back if that fails? > > Wouldn't that generate spurious SELinux warnings/errors? I'd rather > not have that as expected behaviour. I suppose it would. We could ask Dan Walsh what we should do. Detecting the presence of SELinux is incorrect because: 1. SELinux might be configured to allow write/exec regions. 2. SELinux might be configured to allow write/exec regions for Python. 3. Some non-SELinux mechanism might disallow write/exec regions. One more alternative would be to probe but only at install time. That's still wrong because someone could change SELinux settings. Andrew. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libffi & fork 2012-05-04 9:00 ` Andrew Haley @ 2012-05-05 13:19 ` Anthony Green 0 siblings, 0 replies; 12+ messages in thread From: Anthony Green @ 2012-05-05 13:19 UTC (permalink / raw) To: Andrew Haley; +Cc: libffi-discuss On Fri, May 4, 2012 at 4:59 AM, Andrew Haley <aph@redhat.com> wrote: > I suppose it would. We could ask Dan Walsh what we should do. This is exactly what I've done in that bugzilla ticket. Thanks, AG ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-05-05 13:19 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAP+vr+PX9gw7Ot6Zc7AN8Gks9P+WHVQg8bX5S7YRnTRyRBHp7A@mail.gmail.com> 2012-04-24 19:41 ` libffi & fork Gaash Hazan 2012-04-25 9:11 ` Andrew Haley 2012-04-25 17:33 ` Gaash Hazan 2012-04-26 10:00 ` Andrew Haley 2012-04-29 15:34 ` Gaash Hazan 2012-04-29 17:58 ` Anthony Green 2012-04-30 8:32 ` Andrew Haley 2012-04-30 19:16 ` Anthony Green 2012-05-01 7:56 ` Andrew Haley 2012-05-02 20:23 ` Anthony Green 2012-05-04 9:00 ` Andrew Haley 2012-05-05 13:19 ` Anthony Green
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).