[Bug linuxthreads/761] New: vm86 system call makes pthreads segfault

[Bug linuxthreads/761] New: vm86 system call makes pthreads segfault

[mschulkind at gmail dot com]

I first saw this bug when trying to use the vm86 system call with pthreads. I
worked around this by not using pthreads in the same application. The problem
was that I think the vm86 system call would segfault on return whenever pthreads
was explicitly compiled in, but I"m not sure exactly when the segfault occured.
I strongly suspect this is closely related to the following bug report.

Now I have re-encountered this bug in a slightly different way. Even though I
don't use pthreads, it looks like glibc does, and I'm having a similar problem.

This is on gentoo with what they call glibc-2.3.4.20041102, this DOES NOT happen
with glibc-2.3.4.20040808-r1 so it seems like something recent started to cause
this.

After a single call to the vm86 system call, then next subsequent call to printf
causes a segfault. Here is the backtrace:

#0  0x405dfcc3 in _pthread_cleanup_push_defer (buffer=0xbfffedd4, routine=0,
arg=0x0) at cancel.c:181
#1  0x40668b9e in _IO_vfprintf (s=0x407447c0, format=0x804e2a0 "debug> ",
ap=0xbffff204 "\001") at vfprintf.c:1272
#2  0x406714e0 in printf (format=0x0) at printf.c:34
#3  0x08049229 in debug_image () at debugger.cpp:81
#4  0x08048f9e in main (argc=1, argv=0x0) at main.cpp:43

Frames 3 and 4 are my program.

The exact line in _pthreads_cleanup_push_defer that it breaks on is:

"buffer->__canceltype = THREAD_GETMEM(self, p_canceltype);"

I'll try to debug this if I can, but I don't know how much I can figure out. I
can't even see right now how the code execution goes from _IO_vfprintf to
_pthreads_cleanup_push_defer.

I hope this is enough information. Let me know if you want more.

-- 
           Summary: vm86 system call makes pthreads segfault
           Product: glibc
           Version: 2.3.4
            Status: NEW
          Severity: normal
          Priority: P2
         Component: linuxthreads
        AssignedTo: gotom at debian dot or dot jp
        ReportedBy: mschulkind at gmail dot com
                CC: glibc-bugs at sources dot redhat dot com
GCC target triplet: i686-pc-linux-gnu


http://sources.redhat.com/bugzilla/show_bug.cgi?id=761

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug linuxthreads/761] vm86 system call makes pthreads segfault

[jakub at redhat dot com]

------- Additional Comments From jakub at redhat dot com  2005-03-15 08:19 -------
That sounds like the %gs segment base, limit or value got screwed up during the
vm86 system call.  See whether $gs is the same before and after the vm86
syscall in the debugger.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING


http://sources.redhat.com/bugzilla/show_bug.cgi?id=761

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug linuxthreads/761] vm86 system call makes pthreads segfault

[mschulkind at gmail dot com]

------- Additional Comments From mschulkind at gmail dot com  2005-03-18 06:46 -------
Thanks. That's definitely the problem. Preserving %gs fixed the problem in the
bug report and I assume it will also fix the problem in the first paragraph.

Feel free to close this bug, it looks like a kernel bug.

-- 


http://sources.redhat.com/bugzilla/show_bug.cgi?id=761

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug linuxthreads/761] vm86 system call makes pthreads segfault

[gotom at debian dot or dot jp]

------- Additional Comments From gotom at debian dot or dot jp  2005-03-20 15:51 -------
2.4 Linux kernel does not save %gs register during vm86 syscall.
Note that the fix was put in 2.5:
http://seclists.org/lists/linux-kernel/2002/Dec/5053.html

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |INVALID


http://sources.redhat.com/bugzilla/show_bug.cgi?id=761

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.