* maximum array size?
@ 2010-04-15 9:17 Anna Sidera
2010-04-15 10:17 ` Brian Budge
2010-04-15 15:31 ` John S. Fine
0 siblings, 2 replies; 7+ messages in thread
From: Anna Sidera @ 2010-04-15 9:17 UTC (permalink / raw)
To: gcc-help
Hello,
I wrote a program in gcc. I compile it with the option -m64. Is there a limit in how much memory malloc can allocate? I run the program for different values of some input parameters. The program defines some scalar variables and arrays. Then it defines a large array and tries to set a value in an element of the array.
int **bufferc = malloc((inp_par_a*inp_par_b+2)*sizeof(int *));
int *bufferc_aux = malloc((inp_par_a*inp_par_b+2)*(inp_par_c+1)*sizeof(int));
int bfc=1;
bufferc[bfc]=bufferc_aux+bfc*(inp_par_c+1);
bufferc[bfc][0]=0;
At this point sometimes there is no problem but sometimes I get segmentation fault. There is a lot of unused ram in the server when the problem occurs. Do you think the problem is due to a bug that I have before defining the array?
Thanks,
Anna
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
2010-04-15 9:17 maximum array size? Anna Sidera
@ 2010-04-15 10:17 ` Brian Budge
2010-04-15 12:43 ` Brian Budge
2010-04-15 15:31 ` John S. Fine
1 sibling, 1 reply; 7+ messages in thread
From: Brian Budge @ 2010-04-15 10:17 UTC (permalink / raw)
To: Anna Sidera; +Cc: gcc-help
Is this running in Linux? Linux usually behaves optimistically,
actually returning a valid pointer from malloc, even when there is not
enough memory left in the system to satisfy the request. A good
explanation can be found here:
http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
On Thu, Apr 15, 2010 at 1:26 AM, Anna Sidera <sidera.anan@ucy.ac.cy> wrote:
> Hello,
>
> I wrote a program in gcc. I compile it with the option -m64. Is there a limit in how much memory malloc can allocate? I run the program for different values of some input parameters. The program defines some scalar variables and arrays. Then it defines a large array and tries to set a value in an element of the array.
> int **bufferc = malloc((inp_par_a*inp_par_b+2)*sizeof(int *));
> int *bufferc_aux = malloc((inp_par_a*inp_par_b+2)*(inp_par_c+1)*sizeof(int));
> int bfc=1;
> bufferc[bfc]=bufferc_aux+bfc*(inp_par_c+1);
> bufferc[bfc][0]=0;
> At this point sometimes there is no problem but sometimes I get segmentation fault. There is a lot of unused ram in the server when the problem occurs. Do you think the problem is due to a bug that I have before defining the array?
>
> Thanks,
> Anna
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
2010-04-15 10:17 ` Brian Budge
@ 2010-04-15 12:43 ` Brian Budge
0 siblings, 0 replies; 7+ messages in thread
From: Brian Budge @ 2010-04-15 12:43 UTC (permalink / raw)
To: Anna Sidera; +Cc: gcc-help
Oh, you said that there is a lot of *unused* memory... sorry, I
misread. This is almost certainly a bug in your program. You should
compile with -g and run the program in gdb to see where the program
crashes.
Brian
On Thu, Apr 15, 2010 at 1:36 AM, Brian Budge <brian.budge@gmail.com> wrote:
> Is this running in Linux? Linux usually behaves optimistically,
> actually returning a valid pointer from malloc, even when there is not
> enough memory left in the system to satisfy the request. A good
> explanation can be found here:
> http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
>
> On Thu, Apr 15, 2010 at 1:26 AM, Anna Sidera <sidera.anan@ucy.ac.cy> wrote:
>> Hello,
>>
>> I wrote a program in gcc. I compile it with the option -m64. Is there a limit in how much memory malloc can allocate? I run the program for different values of some input parameters. The program defines some scalar variables and arrays. Then it defines a large array and tries to set a value in an element of the array.
>> int **bufferc = malloc((inp_par_a*inp_par_b+2)*sizeof(int *));
>> int *bufferc_aux = malloc((inp_par_a*inp_par_b+2)*(inp_par_c+1)*sizeof(int));
>> int bfc=1;
>> bufferc[bfc]=bufferc_aux+bfc*(inp_par_c+1);
>> bufferc[bfc][0]=0;
>> At this point sometimes there is no problem but sometimes I get segmentation fault. There is a lot of unused ram in the server when the problem occurs. Do you think the problem is due to a bug that I have before defining the array?
>>
>> Thanks,
>> Anna
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
2010-04-15 9:17 maximum array size? Anna Sidera
2010-04-15 10:17 ` Brian Budge
@ 2010-04-15 15:31 ` John S. Fine
[not found] ` <f61380541765a.4bc757a9@ucy.ac.cy>
1 sibling, 1 reply; 7+ messages in thread
From: John S. Fine @ 2010-04-15 15:31 UTC (permalink / raw)
To: Anna Sidera; +Cc: gcc-help
Anna Sidera wrote:
> int **bufferc = malloc((inp_par_a*inp_par_b+2)*sizeof(int *));
> int *bufferc_aux = malloc((inp_par_a*inp_par_b+2)*(inp_par_c+1)*sizeof(int));
> int bfc=1;
> bufferc[bfc]=bufferc_aux+bfc*(inp_par_c+1);
> bufferc[bfc][0]=0;
>
How big are you talking about? What are the values of inp_par_a,
inp_par_b, and inp_par_c
You ought to be able to allocate a chunk of memory as large as all your
unused swap space plus a large part of your unused ram. Even if Linux
has already "committed" all that unused memory to other tasks that have
reserved it without yet using it, Linux will commit it again to your task.
So if you aren't talking about a chunk big enough to exceed all unused
swap space plus a big part of unused ram, then you probably have some
bug other than trying to commit too much memory.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
[not found] ` <f61380541765a.4bc757a9@ucy.ac.cy>
@ 2010-04-15 16:00 ` John S. Fine
2010-04-16 13:48 ` Anna Sidera
2010-04-16 13:51 ` Anna Sidera
0 siblings, 2 replies; 7+ messages in thread
From: John S. Fine @ 2010-04-15 16:00 UTC (permalink / raw)
To: Anna Sidera, GCC Help Mailing List
Integers in x86_64 are 32 bit. So in your chain of multiplying you need
to multiply by something of type std::size_t before the running value
goes over two billion.
As it is now, you get overflow in the integer multiply so the case that
seems to work is actually reserving far less memory than you think you
are requesting.
Once you get rid of the integer overflow, you should see that the amount
you can reserve is limited by swap size plus large part of ram, so if
you really want something that big, you'll need a lot more swap space.
Anna Sidera wrote:
> inp_par_a=10000 and inp_par_b=10000. For inp_par_c=60 the program works. For inp_par_c=40 or inp_par_c=80 I get segmentation fault.
>
> I just calculated the memory the array needs and it is more than the available 10GB ram + 3GB swap in the system even for inp_par_c=40.
>
>
>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
2010-04-15 16:00 ` John S. Fine
@ 2010-04-16 13:48 ` Anna Sidera
2010-04-16 13:51 ` Anna Sidera
1 sibling, 0 replies; 7+ messages in thread
From: Anna Sidera @ 2010-04-16 13:48 UTC (permalink / raw)
To: johnsfine, gcc-help
Ok, thanks a lot for your help.
Anna
----- Original Message -----
From: "John S. Fine" <johnsfine@verizon.net>
Date: Thursday, April 15, 2010 6:30 pm
Subject: Re: maximum array size?
> Integers in x86_64 are 32 bit. So in your chain of multiplying
> you need
> to multiply by something of type std::size_t before the running
> value
> goes over two billion.
>
> As it is now, you get overflow in the integer multiply so the case
> that
> seems to work is actually reserving far less memory than you think
> you
> are requesting.
>
> Once you get rid of the integer overflow, you should see that the
> amount
> you can reserve is limited by swap size plus large part of ram, so
> if
> you really want something that big, you'll need a lot more swap space.
>
> Anna Sidera wrote:
> > inp_par_a=10000 and inp_par_b=10000. For inp_par_c=60 the
> program works. For inp_par_c=40 or inp_par_c=80 I get segmentation
> fault.>
> > I just calculated the memory the array needs and it is more than
> the available 10GB ram + 3GB swap in the system even for inp_par_c=40.
> >
> >
> >
> >
> >
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: maximum array size?
2010-04-15 16:00 ` John S. Fine
2010-04-16 13:48 ` Anna Sidera
@ 2010-04-16 13:51 ` Anna Sidera
1 sibling, 0 replies; 7+ messages in thread
From: Anna Sidera @ 2010-04-16 13:51 UTC (permalink / raw)
To: John S. Fine; +Cc: gcc-help
Ok, thanks a lot for your help.
Anna
----- Original Message -----
From: "John S. Fine" <johnsfine@verizon.net>
Date: Thursday, April 15, 2010 6:30 pm
Subject: Re: maximum array size?
> Integers in x86_64 are 32 bit. So in your chain of multiplying
> you need
> to multiply by something of type std::size_t before the running
> value
> goes over two billion.
>
> As it is now, you get overflow in the integer multiply so the case
> that
> seems to work is actually reserving far less memory than you think
> you
> are requesting.
>
> Once you get rid of the integer overflow, you should see that the
> amount
> you can reserve is limited by swap size plus large part of ram, so
> if
> you really want something that big, you'll need a lot more swap space.
>
> Anna Sidera wrote:
> > inp_par_a=10000 and inp_par_b=10000. For inp_par_c=60 the
> program works. For inp_par_c=40 or inp_par_c=80 I get segmentation
> fault.>
> > I just calculated the memory the array needs and it is more than
> the available 10GB ram + 3GB swap in the system even for inp_par_c=40.
> >
> >
> >
> >
> >
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2010-04-16 12:04 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-04-15 9:17 maximum array size? Anna Sidera
2010-04-15 10:17 ` Brian Budge
2010-04-15 12:43 ` Brian Budge
2010-04-15 15:31 ` John S. Fine
[not found] ` <f61380541765a.4bc757a9@ucy.ac.cy>
2010-04-15 16:00 ` John S. Fine
2010-04-16 13:48 ` Anna Sidera
2010-04-16 13:51 ` Anna Sidera
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).