[ATTN MAINTAINER] python

[ATTN MAINTAINER] python

[Achim Gratz]

While building maxima I've come across a consistent SEGV crash of python
on both architectures. To reproduce, change into the doc/info dirextory
and run

$ sh ./extract_categories.sh maxima

The crash seems to relate to the number of lines in the program (thare
are about 20000 lines and it does not crash with just 5000).  The same
program works without problems on Linux.  It creates and deletes a lot
of variables, so the SEGV might be related to either the GC or memory
allocation in Python.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Achim Gratz writes:
> While building maxima I've come across a consistent SEGV crash of python
> on both architectures. To reproduce, change into the doc/info dirextory
> and run
>
> $ sh ./extract_categories.sh maxima
>
> The crash seems to relate to the number of lines in the program (thare
> are about 20000 lines and it does not crash with just 5000).  The same
> program works without problems on Linux.  It creates and deletes a lot
> of variables, so the SEGV might be related to either the GC or memory
> allocation in Python.

Increasing the stack reserve size of the python executable to 0x400000
on 32bit and 0x800000 on 64bit avoids the SEGV.  I'm not sure if this
indicates an error in Python itself or simply a too restricted
configuration.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [ATTN MAINTAINER] python

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]

On Mar  3 21:51, Achim Gratz wrote:
> Achim Gratz writes:
> > While building maxima I've come across a consistent SEGV crash of python
> > on both architectures. To reproduce, change into the doc/info dirextory
> > and run
> >
> > $ sh ./extract_categories.sh maxima
> >
> > The crash seems to relate to the number of lines in the program (thare
> > are about 20000 lines and it does not crash with just 5000).  The same
> > program works without problems on Linux.  It creates and deletes a lot
> > of variables, so the SEGV might be related to either the GC or memory
> > allocation in Python.
> 
> Increasing the stack reserve size of the python executable to 0x400000
> on 32bit and 0x800000 on 64bit avoids the SEGV.  I'm not sure if this
> indicates an error in Python itself or simply a too restricted
> configuration.

Maybe it just needs a really big stack? 


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Corinna Vinschen writes:
>> Increasing the stack reserve size of the python executable to 0x400000
>> on 32bit and 0x800000 on 64bit avoids the SEGV.  I'm not sure if this
>> indicates an error in Python itself or simply a too restricted
>> configuration.
>
> Maybe it just needs a really big stack? 

Obviously it wants something larger than it is compiled with (what
exactly does "stack reserve" mean, btw?)…  The backtrace had something
like 5000 of the same call inside compile.c in it.  But I would have
expected it to grow the stack when necessary or give some intelligible
error instead of simply segfault.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [ATTN MAINTAINER] python

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

On Mar  4 07:06, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> Increasing the stack reserve size of the python executable to 0x400000
> >> on 32bit and 0x800000 on 64bit avoids the SEGV.  I'm not sure if this
> >> indicates an error in Python itself or simply a too restricted
> >> configuration.
> >
> > Maybe it just needs a really big stack? 
> 
> Obviously it wants something larger than it is compiled with (what
> exactly does "stack reserve" mean, btw?)…  The backtrace had something
> like 5000 of the same call inside compile.c in it.  But I would have
> expected it to grow the stack when necessary or give some intelligible
> error instead of simply segfault.

The problem here is how the Windows stack works.  "Stack reserve" is the
initial size of the reserved virtual memory space used for the stack of
the main thread.  By default that's 2 Megs.

However, the stack for the main thread has a problem.  The way a new
process is created, the main thread stack is jammed between certain
other datastructures, in the address space from 0x30000 and 0x230000.
Since the stack is growing top-down, there's nothing a process can do
when the reserved stack space is exhausted.  Except generating an
exception STATUS_STACK_OVERFLOW.  And then... what?  There's no way to
reserve more space below 0x30000 and even *if* it would be possible, it's
only a mere 192K.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Corinna Vinschen writes:
> The problem here is how the Windows stack works.  "Stack reserve" is the
> initial size of the reserved virtual memory space used for the stack of
> the main thread.  By default that's 2 Megs.

There were in fact two threads at the point of the SEGV, one in the
kernel DLL with a very short stack and the other one with a lot of stack
in compile.c (I don't know which one would be the main thread).

> However, the stack for the main thread has a problem.  The way a new
> process is created, the main thread stack is jammed between certain
> other datastructures, in the address space from 0x30000 and 0x230000.
> Since the stack is growing top-down, there's nothing a process can do
> when the reserved stack space is exhausted.  Except generating an
> exception STATUS_STACK_OVERFLOW.  And then... what?  There's no way to
> reserve more space below 0x30000 and even *if* it would be possible, it's
> only a mere 192K.

I'm just saying that a message like "stack overrun" or something of that
sort would have told me to go look for the stack size a lot earlier than
just a segfault.  As far as building maxima goes the problem is solved
and I just posted here again in case Yaakov thinks that shouldn't have
happened or maybe increase the stack size for the next build.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [ATTN MAINTAINER] python

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1774 bytes --]

On Mar  4 19:22, Achim Gratz wrote:
> Corinna Vinschen writes:
> > The problem here is how the Windows stack works.  "Stack reserve" is the
> > initial size of the reserved virtual memory space used for the stack of
> > the main thread.  By default that's 2 Megs.
> 
> There were in fact two threads at the point of the SEGV, one in the
> kernel DLL with a very short stack and the other one with a lot of stack
> in compile.c (I don't know which one would be the main thread).
> 
> > However, the stack for the main thread has a problem.  The way a new
> > process is created, the main thread stack is jammed between certain
> > other datastructures, in the address space from 0x30000 and 0x230000.
> > Since the stack is growing top-down, there's nothing a process can do
> > when the reserved stack space is exhausted.  Except generating an
> > exception STATUS_STACK_OVERFLOW.  And then... what?  There's no way to
> > reserve more space below 0x30000 and even *if* it would be possible, it's
> > only a mere 192K.
> 
> I'm just saying that a message like "stack overrun" or something of that
> sort would have told me to go look for the stack size a lot earlier than
> just a segfault.

The problem is, a stack overflow is correctly handled as SEGV from the
POSIX perspective.

But there should have been the right info available.  If you run this
under strace, you get an exception 0xc00000fd or 0xc0000228,
STATUS_STACK_OVERFLOW or STATUS_STACK_OVERFLOW_READ.

Can you check?  If it's an 0xc0000228, there is a chance we're missing
to handle it in our exception handler.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Corinna Vinschen writes:
> The problem is, a stack overflow is correctly handled as SEGV from the
> POSIX perspective.
>
> But there should have been the right info available.  If you run this
> under strace, you get an exception 0xc00000fd or 0xc0000228,
> STATUS_STACK_OVERFLOW or STATUS_STACK_OVERFLOW_READ.
>
> Can you check?  If it's an 0xc0000228, there is a chance we're missing
> to handle it in our exception handler.

  829 2952008 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   35 2952043 [main] python2.7 604 mmap64: 0x6FFFB850000 = mmap() 
  703 2952746 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   34 2952780 [main] python2.7 604 mmap64: 0x6FFFB810000 = mmap() 
  596 2953376 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   37 2953413 [main] python2.7 604 mmap64: 0x6FFFB7D0000 = mmap() 
  607 2954020 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   37 2954057 [main] python2.7 604 mmap64: 0x6FFFB790000 = mmap() 
  605 2954662 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   33 2954695 [main] python2.7 604 mmap64: 0x6FFFB750000 = mmap() 
  587 2955282 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
   33 2955315 [main] python2.7 604 mmap64: 0x6FFFB710000 = mmap() 
--- Process 604, exception c00000fd at 00000003E91E9619
--- Process 604, exception c0000005 at 00000001801C2F96
Segmentation fault


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [ATTN MAINTAINER] python

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

On Mar  4 20:55, Achim Gratz wrote:
> Corinna Vinschen writes:
> > The problem is, a stack overflow is correctly handled as SEGV from the
> > POSIX perspective.
> >
> > But there should have been the right info available.  If you run this
> > under strace, you get an exception 0xc00000fd or 0xc0000228,
> > STATUS_STACK_OVERFLOW or STATUS_STACK_OVERFLOW_READ.
> >
> > Can you check?  If it's an 0xc0000228, there is a chance we're missing
> > to handle it in our exception handler.
> 
>   829 2952008 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    35 2952043 [main] python2.7 604 mmap64: 0x6FFFB850000 = mmap() 
>   703 2952746 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    34 2952780 [main] python2.7 604 mmap64: 0x6FFFB810000 = mmap() 
>   596 2953376 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    37 2953413 [main] python2.7 604 mmap64: 0x6FFFB7D0000 = mmap() 
>   607 2954020 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    37 2954057 [main] python2.7 604 mmap64: 0x6FFFB790000 = mmap() 
>   605 2954662 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    33 2954695 [main] python2.7 604 mmap64: 0x6FFFB750000 = mmap() 
>   587 2955282 [main] python2.7 604 mmap64: addr 0x0, len 262144, prot 0x3, flags 0x22, fd -1, off 0x0
>    33 2955315 [main] python2.7 604 mmap64: 0x6FFFB710000 = mmap() 
> --- Process 604, exception c00000fd at 00000003E91E9619
> --- Process 604, exception c0000005 at 00000001801C2F96

DLL version?  Did you call addr2line to see where 00000001801C2F96 is
(hint: requires cygwin-debuginfo) in the DLL?  This might give a clue.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Corinna Vinschen writes:
> DLL version?  Did you call addr2line to see where 00000001801C2F96 is
> (hint: requires cygwin-debuginfo) in the DLL?  This might give a clue.

/mnt/share/maint (1962) uname -a
CYGWIN_NT-6.3 Cygwin 1.7.35(0.286/5/3) 2015-02-27 17:19 x86_64 Cygwin
/mnt/share/maint (1963) addr2line -e /usr/bin/python2.7.exe 00000003E91E9619 00000001801C2F96
??:0
??:0


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [ATTN MAINTAINER] python

[Marco Atzeri]

On 3/4/2015 10:23 PM, Achim Gratz wrote:
> Corinna Vinschen writes:
>> DLL version?  Did you call addr2line to see where 00000001801C2F96 is
>> (hint: requires cygwin-debuginfo) in the DLL?  This might give a clue.
>
> /mnt/share/maint (1962) uname -a
> CYGWIN_NT-6.3 Cygwin 1.7.35(0.286/5/3) 2015-02-27 17:19 x86_64 Cygwin
> /mnt/share/maint (1963) addr2line -e /usr/bin/python2.7.exe 00000003E91E9619 00000001801C2F96
> ??:0
> ??:0
>
>
> Regards,
> Achim.
>

I presume Corinna asked for this:

  $ addr2line.exe -a 00000001801C2F96 -e /usr/bin/cygwin1.dll

Re: [ATTN MAINTAINER] python

[Achim Gratz]

Marco Atzeri writes:
> I presume Corinna asked for this:
>
>  $ addr2line.exe -a 00000001801C2F96 -e /usr/bin/cygwin1.dll

Then that's giving us this:

/mnt/share/maint (1964) addr2line.exe -a 00000001801C2F96 -e /usr/bin/cygwin1.dll
0x00000001801c2f96
/builddir/build/BUILD/gcc-4.8.3/build_cyg64/x86_64-pc-cygwin/libgcc/../../../libgcc/config/i386/cygwin.S:146


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [ATTN MAINTAINER] python

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

On Mar  4 22:39, Achim Gratz wrote:
> Marco Atzeri writes:
> > I presume Corinna asked for this:
> >
> >  $ addr2line.exe -a 00000001801C2F96 -e /usr/bin/cygwin1.dll
> 
> Then that's giving us this:
> 
> /mnt/share/maint (1964) addr2line.exe -a 00000001801C2F96 -e /usr/bin/cygwin1.dll
> 0x00000001801c2f96
> /builddir/build/BUILD/gcc-4.8.3/build_cyg64/x86_64-pc-cygwin/libgcc/../../../libgcc/config/i386/cygwin.S:146

Nothing to see here.  That's just gcc's alloca implementation
crashing due to a stack overflow.  No hint as for the cause of
the excessive stack usage, unfortunately.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]