* maximum number of variables? @ 2005-08-30 21:01 Nicolas Ouellette 2005-08-30 21:08 ` Bob Rossi 2005-08-30 23:01 ` corey taylor 0 siblings, 2 replies; 11+ messages in thread From: Nicolas Ouellette @ 2005-08-30 21:01 UTC (permalink / raw) To: gcc-help Hello I am presently running a huge code following the hydrodynamical evolution of a system (in 2 dimentions). I store the quantities evolving (density, velocity, pressure) into large 2-D arrays. Recently, I tried to make my 2-D arrays bigger (to study a bigger system) and eventually, I reached a certain point where I was getting a "segmentation error". The only thing I changed was the size of my arrays. I deleted a few arrays that were redundant (making my code less user friendly), and I found that I could slightly increase the size of my arrays until I reached the "segmentation error". Here are my questions: 1) Is there a maximum number of variables allowed when using gcc? (from my experience I am guessing yes) 2) Is there a way to increase the number of variables allowed when using gcc? Thanking you for your help Nicolas Ouellette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-30 21:01 maximum number of variables? Nicolas Ouellette @ 2005-08-30 21:08 ` Bob Rossi 2005-08-30 21:41 ` Nicolas Ouellette 2005-08-30 23:01 ` corey taylor 1 sibling, 1 reply; 11+ messages in thread From: Bob Rossi @ 2005-08-30 21:08 UTC (permalink / raw) To: Nicolas Ouellette; +Cc: gcc-help On Tue, Aug 30, 2005 at 05:00:57PM -0400, Nicolas Ouellette wrote: > Hello > > I am presently running a huge code following the hydrodynamical evolution > of a system (in 2 dimentions). I store the quantities evolving (density, > velocity, pressure) into large 2-D arrays. Recently, I tried to make my > 2-D arrays bigger (to study a bigger system) and eventually, I reached a > certain point where I was getting a "segmentation error". The only thing I > changed was the size of my arrays. > > I deleted a few arrays that were redundant (making my code less user > friendly), and I found that I could slightly increase the size of my arrays > until I reached the "segmentation error". > > Here are my questions: 1) Is there a maximum number of variables allowed > when using gcc? (from my experience I am guessing yes) 2) Is there a way to > increase the number of variables allowed when using gcc? It sounds like you may have overflowed the stack. Could you try increasing the stack size? I don't know how to do that though with gcc. Bob Rossi ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-30 21:08 ` Bob Rossi @ 2005-08-30 21:41 ` Nicolas Ouellette 2005-08-30 22:13 ` Lexington Luthor 0 siblings, 1 reply; 11+ messages in thread From: Nicolas Ouellette @ 2005-08-30 21:41 UTC (permalink / raw) To: bob; +Cc: gcc-help >>Hello >> >>I am presently running a huge code following the hydrodynamical evolution >>of a system (in 2 dimentions). I store the quantities evolving (density, >>velocity, pressure) into large 2-D arrays. Recently, I tried to make my >>2-D arrays bigger (to study a bigger system) and eventually, I reached a >>certain point where I was getting a "segmentation error". The only thing >>I changed was the size of my arrays. >> >>I deleted a few arrays that were redundant (making my code less user >>friendly), and I found that I could slightly increase the size of my >>arrays until I reached the "segmentation error". >> >>Here are my questions: 1) Is there a maximum number of variables allowed >>when using gcc? (from my experience I am guessing yes) 2) Is there a way >>to increase the number of variables allowed when using gcc? >It sounds like you may have overflowed the stack. Could you try >increasing the stack size? I don't know how to do that though with gcc. I had to check on the web real to see what you meant. I am a physicist trying to program, and not a programmer trying to do physics :) According to a web site, the default stack size on OS X is 8 MB. it is possible that I have exceeded that. I am looking into ways of trying to increase this. Nicolas Ouellette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-30 21:41 ` Nicolas Ouellette @ 2005-08-30 22:13 ` Lexington Luthor 2005-08-31 1:01 ` Bob Rossi 0 siblings, 1 reply; 11+ messages in thread From: Lexington Luthor @ 2005-08-30 22:13 UTC (permalink / raw) To: gcc-help Nicolas Ouellette wrote: > I had to check on the web real to see what you meant. I am a physicist > trying to program, and not a programmer trying to do physics :) > > According to a web site, the default stack size on OS X is 8 MB. it is > possible that I have exceeded that. I am looking into ways of trying to > increase this. > > Nicolas Ouellette > > > I think for large arrays, its best to allocate them from the heap. Use malloc() or new (if using C++). LL ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-30 22:13 ` Lexington Luthor @ 2005-08-31 1:01 ` Bob Rossi 2005-08-31 4:20 ` Ian Lance Taylor 0 siblings, 1 reply; 11+ messages in thread From: Bob Rossi @ 2005-08-31 1:01 UTC (permalink / raw) To: Lexington Luthor; +Cc: gcc-help On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: > Nicolas Ouellette wrote: > >I had to check on the web real to see what you meant. I am a physicist > >trying to program, and not a programmer trying to do physics :) > > > >According to a web site, the default stack size on OS X is 8 MB. it is > >possible that I have exceeded that. I am looking into ways of trying to > >increase this. > > > >Nicolas Ouellette > > > > > > > > I think for large arrays, its best to allocate them from the heap. Use > malloc() or new (if using C++). I've always been told that the heap and stack grow towards each other. If this is true, why would it be OK to create the item on the heap, vs on the stack? If it's not true, could someone simply explain how this works? Bob Rossi ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-31 1:01 ` Bob Rossi @ 2005-08-31 4:20 ` Ian Lance Taylor 2005-08-31 9:13 ` Re[2]: " Miguel A. Nuñez 2005-08-31 11:19 ` Bob Rossi 0 siblings, 2 replies; 11+ messages in thread From: Ian Lance Taylor @ 2005-08-31 4:20 UTC (permalink / raw) To: Bob Rossi; +Cc: Lexington Luthor, gcc-help Bob Rossi <bob@brasko.net> writes: > On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: > > > I think for large arrays, its best to allocate them from the heap. Use > > malloc() or new (if using C++). > > I've always been told that the heap and stack grow towards each other. > If this is true, why would it be OK to create the item on the heap, vs > on the stack? If it's not true, could someone simply explain how this > works? You are correct in theory. In practice the heap and stack have different limits, and the limit on the heap is much larger than the stack (if running bash, compare ulimit -s and ulimit -v). And if you worry about portability, on some platforms allocating a large stack frame will simply fail, and on some other platforms it will require extra work to emit stack probes to tell the OS that you are intentionally extending the stack rather than just referencing a random memory address. Ian ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re[2]: maximum number of variables? 2005-08-31 4:20 ` Ian Lance Taylor @ 2005-08-31 9:13 ` Miguel A. Nuñez 2005-08-31 11:17 ` Bob Rossi 2005-08-31 11:19 ` Bob Rossi 1 sibling, 1 reply; 11+ messages in thread From: Miguel A. Nuñez @ 2005-08-31 9:13 UTC (permalink / raw) To: Ian Lance Taylor; +Cc: Bob Rossi, Lexington Luthor, gcc-help ILT> Bob Rossi <bob@brasko.net> writes: >> On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: >> >> > I think for large arrays, its best to allocate them from the heap. Use >> > malloc() or new (if using C++). >> >> I've always been told that the heap and stack grow towards each other. >> If this is true, why would it be OK to create the item on the heap, vs >> on the stack? If it's not true, could someone simply explain how this >> works? ILT> You are correct in theory. In practice the heap and stack have ILT> different limits, and the limit on the heap is much larger than the ILT> stack (if running bash, compare ulimit -s and ulimit -v). And if you ILT> worry about portability, on some platforms allocating a large stack ILT> frame will simply fail, and on some other platforms it will require ILT> extra work to emit stack probes to tell the OS that you are ILT> intentionally extending the stack rather than just referencing a ILT> random memory address. ILT> Ian Ian, I did ulimit -s and ulimit -v in a solaris 9 sun machine. ulimit -s returns 8192, ok, but ulimit -s returns unlimited. Does this make sense? Miguel Angel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-31 9:13 ` Re[2]: " Miguel A. Nuñez @ 2005-08-31 11:17 ` Bob Rossi 2005-09-01 8:20 ` Re[2]: " Miguel A. Nuñez 0 siblings, 1 reply; 11+ messages in thread From: Bob Rossi @ 2005-08-31 11:17 UTC (permalink / raw) To: Miguel A. Nu?ez; +Cc: Ian Lance Taylor, Lexington Luthor, gcc-help On Wed, Aug 31, 2005 at 11:13:00AM +0200, Miguel A. Nu?ez wrote: > > > ILT> Bob Rossi <bob@brasko.net> writes: > > >> On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: > >> > >> > I think for large arrays, its best to allocate them from the heap. Use > >> > malloc() or new (if using C++). > >> > >> I've always been told that the heap and stack grow towards each other. > >> If this is true, why would it be OK to create the item on the heap, vs > >> on the stack? If it's not true, could someone simply explain how this > >> works? > > ILT> You are correct in theory. In practice the heap and stack have > ILT> different limits, and the limit on the heap is much larger than the > ILT> stack (if running bash, compare ulimit -s and ulimit -v). And if you > ILT> worry about portability, on some platforms allocating a large stack > ILT> frame will simply fail, and on some other platforms it will require > ILT> extra work to emit stack probes to tell the OS that you are > ILT> intentionally extending the stack rather than just referencing a > ILT> random memory address. > > ILT> Ian > > > Ian, > > I did ulimit -s and ulimit -v in a solaris 9 sun machine. ulimit -s returns 8192, ok, but ulimit -s returns unlimited. Does this make sense? Assumming you mean ulimit -s = 8192 and ulimit -v = unlimited, then yes, I suppose that makes sense. On my linux machine, ulimit -s and ulimit -v = unlimited. In this case, I suppose it's possible that if the array was on the stack and things borked, it would also bork when on the heap. However, in your case, I know understand why things would work for you if you moved from the stack to the heap. Bob Rossi ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re[2]: maximum number of variables? 2005-08-31 11:17 ` Bob Rossi @ 2005-09-01 8:20 ` Miguel A. Nuñez 0 siblings, 0 replies; 11+ messages in thread From: Miguel A. Nuñez @ 2005-09-01 8:20 UTC (permalink / raw) To: Bob Rossi; +Cc: Ian Lance Taylor, Lexington Luthor, gcc-help >> >> >> ILT> Bob Rossi <bob@brasko.net> writes: >> >> >> On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: >> >> >> >> > I think for large arrays, its best to allocate them from the heap. Use >> >> > malloc() or new (if using C++). >> >> >> >> I've always been told that the heap and stack grow towards each other. >> >> If this is true, why would it be OK to create the item on the heap, vs >> >> on the stack? If it's not true, could someone simply explain how this >> >> works? >> >> ILT> You are correct in theory. In practice the heap and stack have >> ILT> different limits, and the limit on the heap is much larger than the >> ILT> stack (if running bash, compare ulimit -s and ulimit -v). And if you >> ILT> worry about portability, on some platforms allocating a large stack >> ILT> frame will simply fail, and on some other platforms it will require >> ILT> extra work to emit stack probes to tell the OS that you are >> ILT> intentionally extending the stack rather than just referencing a >> ILT> random memory address. >> >> ILT> Ian >> >> >> Ian, >> >> I did ulimit -s and ulimit -v in a solaris 9 sun machine. >> ulimit -s returns 8192, ok, but ulimit -s returns unlimited. Does >> this make sense? BR> Assumming you mean ulimit -s = 8192 and ulimit -v = unlimited, then BR> yes, I suppose that makes sense. On my linux machine, ulimit -s and BR> ulimit -v = unlimited. In this case, I suppose it's possible that if the BR> array was on the stack and things borked, it would also bork when on the BR> heap. However, in your case, I know understand why things would work for BR> you if you moved from the stack to the heap. BR> Bob Rossi Yes, I meant that Bob. Anyway, remember it was Nicolas who originally had this problem. I'm running into a similar problem, so I tried the commands Ian suggested, and the value 'unlimited' was strange to me. Let's see if anyone can help me with my problem. My application creates a new thread for a receiving infite loop. pthread_create runs the function containing the loop, and this functions defines a local variable that is a huge struct (sizeof returns 984KB). I initialize it using 'mystruct_t mystruct={0}'. Up to this point everything is ok. When something is received, the loop calls a new function. This function creates a new local variable, the same struct. Here the program crashes. I have read the default stack size when creating a thread in solaris 32 bits is 1MB, so I thought the crash would be due to allocating more memory than the stack size. I put the initialization in a separate line, with a for loop giving value 0 byte per byte, from 0 to sizeof(mystruct_t)-1. Strangely (at least to me) the second struct is allocated, and the crash is always produced when writing to the bytes of the struct. Is it possible that the local variable is allocated partly into the stack and partly beyond the limits of the stack, so that when I write to it I get my segmentation fault?? How could I know where the stack begins and ends? I will allocate the structs at the heap, instead of the stack, but I would like to understand this situation. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-31 4:20 ` Ian Lance Taylor 2005-08-31 9:13 ` Re[2]: " Miguel A. Nuñez @ 2005-08-31 11:19 ` Bob Rossi 1 sibling, 0 replies; 11+ messages in thread From: Bob Rossi @ 2005-08-31 11:19 UTC (permalink / raw) To: lexington.luthor, gcc-help On Tue, Aug 30, 2005 at 09:19:19PM -0700, Ian Lance Taylor wrote: > Bob Rossi <bob@brasko.net> writes: > > > On Tue, Aug 30, 2005 at 10:54:02PM +0100, Lexington Luthor wrote: > > > > > I think for large arrays, its best to allocate them from the heap. Use > > > malloc() or new (if using C++). > > > > I've always been told that the heap and stack grow towards each other. > > If this is true, why would it be OK to create the item on the heap, vs > > on the stack? If it's not true, could someone simply explain how this > > works? > > You are correct in theory. In practice the heap and stack have > different limits, and the limit on the heap is much larger than the > stack (if running bash, compare ulimit -s and ulimit -v). And if you > worry about portability, on some platforms allocating a large stack > frame will simply fail, and on some other platforms it will require > extra work to emit stack probes to tell the OS that you are > intentionally extending the stack rather than just referencing a > random memory address. Thanks for the great explanation! Also, please correct me if I'm wrong. Most likely if ulimit -s == ulimit -v, then it's possible that moving the array from the stack to the heap would still cause the same problem. However, if the limit's are different (stack being smaller), then moving the array could be a successful choice. Does this sound correct to you? Thanks, Bob Rossi ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: maximum number of variables? 2005-08-30 21:01 maximum number of variables? Nicolas Ouellette 2005-08-30 21:08 ` Bob Rossi @ 2005-08-30 23:01 ` corey taylor 1 sibling, 0 replies; 11+ messages in thread From: corey taylor @ 2005-08-30 23:01 UTC (permalink / raw) To: Nicolas Ouellette; +Cc: gcc-help Nicolas, The stack settings are fairly common, but I found a decent FAQ answer for you pertaining directly to OSX and even an XCode option which is nice. http://developer.apple.com/qa/qa2005/qa1419.html corey On 8/30/05, Nicolas Ouellette <sideshownic@hotmail.com> wrote: > Hello > > I am presently running a huge code following the hydrodynamical evolution of > a system (in 2 dimentions). I store the quantities evolving (density, > velocity, pressure) into large 2-D arrays. Recently, I tried to make my 2-D > arrays bigger (to study a bigger system) and eventually, I reached a certain > point where I was getting a "segmentation error". The only thing I changed > was the size of my arrays. > > I deleted a few arrays that were redundant (making my code less user > friendly), and I found that I could slightly increase the size of my arrays > until I reached the "segmentation error". > > Here are my questions: 1) Is there a maximum number of variables allowed > when using gcc? (from my experience I am guessing yes) 2) Is there a way to > increase the number of variables allowed when using gcc? > > Thanking you for your help > > Nicolas Ouellette > > > ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2005-09-01 8:20 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-08-30 21:01 maximum number of variables? Nicolas Ouellette 2005-08-30 21:08 ` Bob Rossi 2005-08-30 21:41 ` Nicolas Ouellette 2005-08-30 22:13 ` Lexington Luthor 2005-08-31 1:01 ` Bob Rossi 2005-08-31 4:20 ` Ian Lance Taylor 2005-08-31 9:13 ` Re[2]: " Miguel A. Nuñez 2005-08-31 11:17 ` Bob Rossi 2005-09-01 8:20 ` Re[2]: " Miguel A. Nuñez 2005-08-31 11:19 ` Bob Rossi 2005-08-30 23:01 ` corey taylor
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).