* Re: My C arrays are too large
2019-09-18 20:40 ` Jose Isaias Cabrera
@ 2019-09-18 20:46 ` Eliot Moss
2019-09-19 1:19 ` Joel Rees
2019-09-19 21:58 ` Brian Inglis
2 siblings, 0 replies; 14+ messages in thread
From: Eliot Moss @ 2019-09-18 20:46 UTC (permalink / raw)
To: cygwin
On 9/18/2019 4:35 PM, Jose Isaias Cabrera wrote:
>
> Joel Rees, on Wednesday, September 18, 2019 02:38 PM, wrote...
>>
>> 2019年9月14日(土) 3:50 Jose Isaias Cabrera, on
>>Moss
>>> Achim Gratz, on Friday, September 13, 2019 02:39 PM, wrote...
>>>>
>>>> Blair, Charles E III writes:
>>>>> My apologies for failing to reply on-list. I don't know how :(
>>>>>
>>>>> My machine is 64 bit, and I hope I installed the correct version of
>>> cygwin.
>>>>>
>>>>> This program:
>>>>>
>>>>> #include<stdio.h>
>>>>> int main(){char *a[50][8192];
>>>>> return 0;}
>>>>>
>>>>> compiles with gcc (no special options) but gives "Segmentation fault".
>>>>
>>>> You are creating an automatic variable that's larger than the default
>>>> stack. You need to enlarge the stack, either during link time or later
>>>> e.g. via
>>>>
>>>> peflags -x0x800000 a.out
>>>
>>> This is great! Thanks.
>>>
>>> But, let's talk about this a bit... Shouldn't the compiler provide some
>>> warning, and also, it should never blow up with a "Segmentation fault". I
>>> believe there should be some type of Out Of Memory error, or something like
>>> it. But now just blow up. Anyone thinks like me? Just my 102 Dominican
>>> cents ($1 = $51 Dominican). :-)
>>>
>>
>> Well, the behavior of the compiler itself is better discussed on the
>> compiler's forums, although you may need your asbestos suit when you do so.
>>
>> That said, why do you want this variable to be automatic? Why do you want
>> it allocated on the stack?
>
> I did not say automatically. I said that the compiler should provide some warning about the allocation being larger than the default stack. And, it should not result in a segmentation fault, but instead, the program should error out with out of memory, or, at least, "memory allocation is larger than default stack." Not just blow up.
Automatic here means "stack allocated" (the "auto" keyword in C, which is almost
always omitted because it is the default). Cheers - Eliot Moss
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: My C arrays are too large
2019-09-18 20:40 ` Jose Isaias Cabrera
2019-09-18 20:46 ` Eliot Moss
@ 2019-09-19 1:19 ` Joel Rees
2019-09-19 18:04 ` Jose Isaias Cabrera
2019-09-19 21:58 ` Brian Inglis
2 siblings, 1 reply; 14+ messages in thread
From: Joel Rees @ 2019-09-19 1:19 UTC (permalink / raw)
To: Jose Isaias Cabrera; +Cc: cygwin
I guess I should unpack things a bit, and I may sound argumentative, but my
argument is not really with anyone on this list.
2019年9月19日(木) 5:35 Jose Isaias Cabrera <jicman@outlook.com>:
>
> Joel Rees, on Wednesday, September 18, 2019 02:38 PM, wrote...
> >
> > 2019年9月14日(土) 3:50 Jose Isaias Cabrera, on
> >
> > >
> > > Achim Gratz, on Friday, September 13, 2019 02:39 PM, wrote...
> > > >
> > > > Blair, Charles E III writes:
> > > > > My apologies for failing to reply on-list. I don't know how :(
> > > > >
> > > > > My machine is 64 bit, and I hope I installed the correct version of
> > > cygwin.
> > > > >
> > > > > This program:
>
> > > >
> > > > > #include<stdio.h>
> > > > > int main(){char *a[50][8192];
> > > > > return 0;}
> > > > >
>
/* programmatic example by Jose Isaias Cabrera
// reformatting and annotation by Joel Rees
*/
#include <stdio.h>
int main( void )
{
char *a[50][8192];
/* Note that variables declared here are default "auto" allocated. */
return 0;
/* Note that some optimization settings might
// entirely optimize the allocation out in recent compilers,
// or optimize main() to just return 0 to the calling environment,
// completely removing the allocation.
*/
}
> > > > compiles with gcc (no special options) but gives "Segmentation
> fault".
> > > >
> > > > You are creating an automatic variable that's larger than the default
> > > > stack. You need to enlarge the stack, either during link time or
> later
> > > > e.g. via
> > > >
> > > > peflags -x0x800000 a.out
> > >
> > > This is great! Thanks.
> > >
> > > But, let's talk about this a bit... Shouldn't the compiler provide some
> > > warning,
>
It would be nice, yes.
> > and also, it should never blow up with a "Segmentation fault".
>
But we are discussing C, and, frankly, I'd prefer segementation faults to
some of the possibilities that have been discussed by the engineering teams
that work on C compilers.
Ada, yes, the compiler is supposed to (mostly) prevent you from writing
things that give segmentation faults. C, no. It's within the nature of the
language itself.
C is specifically designed to allow the programmer to shoot him/herself in
the foot, and with good reason.
Lately, some reasonable protection is provided, according to the
engineering teams' consideration of what is reasonable, and there is a lot
of argument about what level of protection is too much.
> > I
> > > believe there should be some type of Out Of Memory error, or something
> like
> > > it. But now just blow up. Anyone thinks like me? Just my 102
> Dominican
> > > cents ($1 = $51 Dominican). :-)
> > >
> >
> > Well, the behavior of the compiler itself is better discussed on the
> > compiler's forums, although you may need your asbestos suit when you do
> so.
> >
> > That said, why do you want this variable to be automatic?
>
As Eliot says, it might have been more clear if I had said "auto".
Auto in C means local to the function in both visibility and duration,
which essentially translates to allocation that looks like it's on a stack.
> Why do you want
> > it allocated on the stack?
>
Now, a good introductory software engineering course will usually encourage
you to use local visibility and duration unless you have a good reason not
to. But the larger a variable is, the more likely there is to be a good
reason for declaring the variable to have different scope.
Buffers, for instance, should persist between calls (static duration), and
symbol tables should usually be visible to all functions that refer to them
(extern visibility).
All of that said, you are right to wonder at this behavior.
Modern compilers inherit a lot of anti-optimal best worst practices from
the days of sixteen-bit addresses.
One of those is combining the flow-of-control stack with the parameter
stack in an interleaved linked list of local allocations, enabling simple
stack-smash attacks.
Locating that overloaded stack in as small a corner of address space as
possible is another, and it results in trouble anytime you allocate large
things on the stack, with local visibility, local duration, thus, auto.
(This practice is more in the operating system scope than in the compiler
scope, but the libraries the compiler links in define the behavior and give
the error messages.)
Yeah, this is a favorite topic of mine, occupying a bit of space in my
programming and computing related blogs. Non-optimal, but it is what you
get:
Either don't allocate large variables and structures on the stack, or use
compiler or linker flags, or command line tools, to set the object to
allocate a large stack.
Or hope you get segment violations instead of silent errors.
Joel Rees
(Maybe I should blog this.)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: My C arrays are too large
2019-09-19 1:19 ` Joel Rees
@ 2019-09-19 18:04 ` Jose Isaias Cabrera
0 siblings, 0 replies; 14+ messages in thread
From: Jose Isaias Cabrera @ 2019-09-19 18:04 UTC (permalink / raw)
To: Joel Rees; +Cc: cygwin
>
>
> From: Joel Rees, on Wednesday, September 18, 2019 07:09 PM, wrote...
> 2019年9月19日(木) 5:35 Jose Isaias Cabrera, on
>
>
> Joel Rees, on Wednesday, September 18, 2019 02:38 PM, wrote...
> >
> > 2019年9月14日(土) 3:50 Jose Isaias Cabrera, on
> >
> > >
> > > Achim Gratz, on Friday, September 13, 2019 02:39 PM, wrote...
> > > >
> > > > Blair, Charles E III writes:
> > > > > My apologies for failing to reply on-list. I don't know how :(
> > > > >
> > > > > My machine is 64 bit, and I hope I installed the correct version of
> > > cygwin.
> > > > >
> > > > > This program:
>
> > > > >
> > > > > #include<stdio.h>
> > > > > int main(){char *a[50][8192];
> > > > > return 0;}
> > > > >
>
>
>
> /* programmatic example by Jose Isaias Cabrera
> // reformatting and annotation by Joel Rees
> */
>
> #include, on
>
>
> int main( void )
> {
> char *a[50][8192];
> /* Note that variables declared here are default "auto" allocated. */
>
> return 0;
> /* Note that some optimization settings might
> // entirely optimize the allocation out in recent compilers,
> // or optimize main() to just return 0 to the calling environment,
> // completely removing the allocation.
> */
> }
>
> > > > > compiles with gcc (no special options) but gives "Segmentation fault".
> > > >
> > > > You are creating an automatic variable that's larger than the default
> > > > stack. You need to enlarge the stack, either during link time or later
> > > > e.g. via
> > > >
> > > > peflags -x0x800000 a.out
> > >
> > > This is great! Thanks.
> > >
> > > But, let's talk about this a bit... Shouldn't the compiler provide some
> > > warning,
>
>
> It would be nice, yes.
>
> > > and also, it should never blow up with a "Segmentation fault".
>
>
> But we are discussing C, and, frankly, I'd prefer segementation faults to some of
> the possibilities that have been discussed by the engineering teams that work on C
> compilers.
>
> Ada, yes, the compiler is supposed to (mostly) prevent you from writing things that
> give segmentation faults. C, no. It's within the nature of the language itself.
>
> C is specifically designed to allow the programmer to shoot him/herself in the foot,
> and with good reason.
>
> Lately, some reasonable protection is provided, according to the engineering teams'
> consideration of what is reasonable, and there is a lot of argument about what level
> of protection is too much.
>
> > > I
> > > believe there should be some type of Out Of Memory error, or something like
> > > it. But now just blow up. Anyone thinks like me? Just my 102 Dominican
> > > cents ($1 = $51 Dominican). :-)
> > >
> >
> > Well, the behavior of the compiler itself is better discussed on the
> > compiler's forums, although you may need your asbestos suit when you do so.
> >
> > That said, why do you want this variable to be automatic?
>
>
> As Eliot says, it might have been more clear if I had said "auto".
>
> Auto in C means local to the function in both visibility and duration, which
> essentially translates to allocation that looks like it's on a stack.
>
> > Why do you want
> > it allocated on the stack?
>
>
> Now, a good introductory software engineering course will usually encourage you to use
> local visibility and duration unless you have a good reason not to. But the larger a
> variable is, the more likely there is to be a good reason for declaring the variable to
> have different scope.
>
> Buffers, for instance, should persist between calls (static duration), and symbol tables
> should usually be visible to all functions that refer to them (extern visibility).
>
> All of that said, you are right to wonder at this behavior.
>
> Modern compilers inherit a lot of anti-optimal best worst practices from the days of
> sixteen-bit addresses.
>
> One of those is combining the flow-of-control stack with the parameter stack in an
> interleaved linked list of local allocations, enabling simple stack-smash attacks.
>
> Locating that overloaded stack in as small a corner of address space as possible is
> another, and it results in trouble anytime you allocate large things on the stack, with
> local visibility, local duration, thus, auto. (This practice is more in the operating
> system scope than in the compiler scope, but the libraries the compiler links in define
> the behavior and give the error messages.)
>
> Yeah, this is a favorite topic of mine, occupying a bit of space in my programming and
> computing related blogs. Non-optimal, but it is what you get:
>
> Either don't allocate large variables and structures on the stack, or use compiler or
> linker flags, or command line tools, to set the object to allocate a large stack.
>
> Or hope you get segment violations instead of silent errors.
>
> Joel Rees
Touché, Joel. Touché! As a friend of mine in the inner city used to say to me when
someone did a move on me playing basketball, "Man, he taught YOU!"
> (Maybe I should blog this.)
Nah, you'll have to make it longer. ;-)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: My C arrays are too large
2019-09-18 20:40 ` Jose Isaias Cabrera
2019-09-18 20:46 ` Eliot Moss
2019-09-19 1:19 ` Joel Rees
@ 2019-09-19 21:58 ` Brian Inglis
2 siblings, 0 replies; 14+ messages in thread
From: Brian Inglis @ 2019-09-19 21:58 UTC (permalink / raw)
To: cygwin
On 2019-09-18 14:35, Jose Isaias Cabrera wrote:
> Joel Rees, on Wednesday, September 18, 2019 02:38 PM, wrote...
>> 2019年9月14日(土) 3:50 Jose Isaias Cabrera, on
>>> Achim Gratz, on Friday, September 13, 2019 02:39 PM, wrote...
>>>> Blair, Charles E III writes:
>>>>> My apologies for failing to reply on-list. I don't know how :(
>>>>> My machine is 64 bit, and I hope I installed the correct version of
>>>>> cygwin.
>>>>> This program:
>>>>> #include<stdio.h>
>>>>> int main(){char *a[50][8192];
>>>>> return 0;}
>>>>> compiles with gcc (no special options) but gives "Segmentation fault".
>>>> You are creating an automatic variable that's larger than the default
>>>> stack. You need to enlarge the stack, either during link time or later
>>>> e.g. via
>>>> peflags -x0x800000 a.out
>>> This is great! Thanks.
>>> But, let's talk about this a bit... Shouldn't the compiler provide some
>>> warning, and also, it should never blow up with a "Segmentation fault". I
>>> believe there should be some type of Out Of Memory error, or something like
>>> it. But now just blow up. Anyone thinks like me? Just my 102 Dominican
>>> cents ($1 = $51 Dominican). :-)
>>
>> Well, the behavior of the compiler itself is better discussed on the
>> compiler's forums, although you may need your asbestos suit when you do so.
>>
>> That said, why do you want this variable to be automatic? Why do you want
>> it allocated on the stack?
>
> I did not say automatically. I said that the compiler should provide some
> warning about the allocation being larger than the default stack. And, it
> should not result in a segmentation fault, but instead, the program should error
> out with out of memory, or, at least, "memory allocation is larger than default
> stack." Not just blow up.
Arrays over about 4KB on the stack are not portable: developers either make them
static local or global arrays for singletons, or malloc(3) dynamic arrays to
allow multiple instances.
As usual with C, gcc makes few default decisions about how compilations should
be performed, as that is very dependent on the target runtime environment,
leaving it to the developer to set the appropriate options for their application
and target environment.
Most development environments are heavy weight systems with GB of memory and
disk, where the final target could be an embedded POS terminal, running some
embedded Windows or Linux, or maybe just QNX or RTEMS.
If you look at Cygwin gcc -v or -### you will see it was configured with
--disable-libssp as default, leaving use of it to the developer's choice.
If you want stack smashing protection (ssp) -fstack-protector look at all the
gcc -fstack-... and -Wstack-... options: you may have to disable optimizations
or selection of some options may be incompatible with some optimizations, set
some warning levels depending on your environment, and add some libraries -lssp.
Cygwin is a popular environment supporting development on Windows for all kinds
of hardware e.g. FPGAs and SoCs, and systems e.g. BSD and Linux, and shares the
same newlib libc/libm libraries used by RTEMS and other embedded systems, except
OS-dependent functions which have to be emulated under Windows.
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 14+ messages in thread