malloc/free & new/delete balance

malloc/free & new/delete balance

[Carlo Wood]

In gcc/cp/new2.cc,

WEAK(void * operator new[] (size_t sz) throw (std::bad_alloc))
{
  return ::operator new(sz);
}

...

WEAK (void operator delete[] (void *ptr) throw ())
{
  if (ptr)
    free (ptr);
}

This unbalance between calling `::operator new(size_t)' and `free(void *)'
is causing troubles when `operator new(size_t)' and `operator delete(void *)'
are overloaded [ for example to call special memory allocation/deallocation
functions called malloc_with_prepended_magic_number(size_t) and
free_with_prepended_magic_number(void *). Then calling 'operator
new[](size_t)' uses malloc_with_prepended_magic_number(size_t) and
'operator delete[](void *)' uses just free(void *) ].

The way it is now you force people to also overload operator new[] and
operator delete[] when they overloaded operator new and operator delete.

Imho, the only correct way is to balance this and use:

WEAK (void operator delete[] (void *ptr) throw ())
{
  ::operator delete(ptr);
}

Alternatively, and probably a little faster, is to duplicate the
code from operator new(size_t) and use:

WEAK(void * operator new[] (size_t sz) throw (std::bad_alloc))
{
  /*
   * This is an exact copy of
   * void * operator new (size_t sz) throw (std::bad_alloc)
   */

  void *p;

  /* malloc (0) is unpredictable; avoid it.  */
  if (sz == 0)
    sz = 1;
  p = (void *) malloc (sz);
  while (p == 0)
    {
      new_handler handler = __new_handler;
      if (! handler)
        throw bad_alloc ();
      handler ();
      p = (void *) malloc (sz);
    }

  return p;
}

Comments?

-- 
 Carlo Wood  <carlo@runaway.xs4all.nl>

Re: malloc/free & new/delete balance

[Alexandre Oliva]

Carlo Wood <carlo@runaway.xs4all.nl> writes:

> WEAK (void operator delete[] (void *ptr) throw ())
> {
>   if (ptr)
>     free (ptr);
> }

> This unbalance between calling `::operator new(size_t)' and `free(void *)'
> is causing troubles when `operator new(size_t)' and `operator delete(void *)'
> are overloaded

Agreed, but, for some reason, the FDIS [lib.new.delete.array] defines
operator new[](size_t) as returning operator new(size_t) by default,
while it states that operator delete[](void*) frees the pointer it is
given.  So, I'd say the implementation is correct, and calling
operator delete(void*) is non-standard.  Too bad. :-(

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:aoliva@acm.org
http://www.dcc.unicamp.br/~oliva
Universidade Estadual de Campinas, SP, Brasil

Re: malloc/free & new/delete balance

[Carlo Wood]

| Carlo Wood <carlo@runaway.xs4all.nl> writes:
| 
| > WEAK (void operator delete[] (void *ptr) throw ())
| > {
| >   if (ptr)
| >     free (ptr);
| > }
| 
| > This unbalance between calling `::operator new(size_t)' and `free(void *)'
| > is causing troubles when `operator new(size_t)' and `operator delete(void *)'
| > are overloaded
| 
| Agreed, but, for some reason, the FDIS [lib.new.delete.array] defines
| operator new[](size_t) as returning operator new(size_t) by default,

True

| while it states that operator delete[](void*) frees the pointer it is
| given.  So, I'd say the implementation is correct, and calling
| operator delete(void*) is non-standard.  Too bad. :-(

CD2 reads:

  void operator delete[](void* ptr) throw();
  void operator delete[](void* ptr, const std::nothrow_t&) throw();

...

  Default behavior:

  --For a null value of ptr, does nothing.

  --Any  other  value of ptr shall be a value returned earlier by a call
    to  the  default operator new[](std::size_t).33) For such a non-null
    value of ptr, reclaims storage allocated by the earlier call to  the
    default operator new[].

It seems to say that it should undo the allocation that was done by the
call to the default operator new[](size_t) that returned this non-null pointer.

If the default operator new[] calls operator new(size_t), and the
only correct way to undo a call to operator new(size_t) is calling
operator delete(void *), then this means that the default
operator delete[](void*) should call ::operator delete(ptr) imho.

If CD2 is not up to date concerning this, then where can I find the
most up to date document?

-- 
 Carlo Wood  <carlo@runaway.xs4all.nl>

Re: malloc/free & new/delete balance

[Alexandre Oliva]

Carlo Wood <carlo@runaway.xs4all.nl> writes:

> If the default operator new[] calls operator new(size_t), and the
> only correct way to undo a call to operator new(size_t) is calling
> operator delete(void *), then this means that the default
> operator delete[](void*) should call ::operator delete(ptr) imho.

Souds reasonable, but this is ``calling'' for confirmation in
comp.std.c++ :-)

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:aoliva@acm.org
http://www.dcc.unicamp.br/~oliva
Universidade Estadual de Campinas, SP, Brasil

Re: malloc/free & new/delete balance

[Sol Foster]

Alexandre Oliva <oliva@dcc.unicamp.br> wrote:
> 
> Carlo Wood <carlo@runaway.xs4all.nl> writes:
> 
> > WEAK (void operator delete[] (void *ptr) throw ())
> > {
> >   if (ptr)
> >     free (ptr);
> > }
> 
> > This unbalance between calling `::operator new(size_t)' and `free(void *)'
> > is causing troubles when `operator new(size_t)' and `operator delete(void *)'
> > are overloaded
> 
> Agreed, but, for some reason, the FDIS [lib.new.delete.array] defines
> operator new[](size_t) as returning operator new(size_t) by default,
> while it states that operator delete[](void*) frees the pointer it is
> given.  

Huh?  It's correct in the Nov'96 paper.  "For such a non-null value of
ptr, reclaims storage allocated by the earlier call to the default
operator new[]."  Nothing about free in there.  You mean they changed it
to something obviously incorrect?  *None* of the deletes should be defined
in terms of free...  (Implemented, sure, but not defined.)

> So, I'd say the implementation is correct, and calling operator
> delete(void*) is non-standard.  Too bad. :-(

While it isn't that hard to get around (just replace the all operators 
with your own) it seems a shame to allow an obvious mistake like this into
EGCS.


-- 
Sol Foster: colomon@ralf.org

A genius is someone who travels to truth by an unexpected path.
Unfortunately, unexpected paths lead to disaster in everyday life.  
                                        -Alfred Bester

Re: malloc/free & new/delete balance

[Mike Stump]

> To: Carlo Wood <carlo@runaway.xs4all.nl>
> Cc: egcs@cygnus.com (egcs@cygnus.com)
> From: Alexandre Oliva <oliva@dcc.unicamp.br>
> Date: 29 Jun 1998 21:54:36 -0300

> > This unbalance between calling `::operator new(size_t)' and
> > `free(void *)' is causing troubles when `operator new(size_t)' and
> > `operator delete(void *)' are overloaded

> Agreed, but, for some reason, the FDIS [lib.new.delete.array]
> defines operator new[](size_t) as returning operator new(size_t) by
> default, while it states that operator delete[](void*) frees

Can you provide an exact quote of this?  I can't find it.  I'll quote
from the FDIS below.

> the pointer it is given.  So, I'd say the implementation is correct,
> and calling operator delete(void*) is non-standard.  Too bad. :-(

The next stage of your learning is to be able to say when the standard
is wrong.  :-)

Also, I think we are staring at different papers.  Does yours have the
number 14882 or X3J16/97-0079 on it?  If not, that isn't the FDIS, and
you should not call it that.

From the real FDIS:

void operator delete[](void* ptr) throw();
void operator delete[](void* ptr, const std::nothrow_t&) throw();

-9- Effects: The deallocation function
 (basic.stc.dynamic.deallocation) called by the array form of a
 delete-expression to render the value of ptr invalid.

-10- Replaceable: a C++ program can define a function with this
 function signature that displaces the default version defined by the
 C++ Standard library.

-11- Required behavior: accept a value of ptr that is null or that was
returned by an earlier call to operator new[](std::size_t) or operator
new[](std::size_t,const std::nothrow_t&).

-12- Default behavior: 

       For a null value of ptr , does nothing. 

       Any other value of ptr shall be a value returned earlier by a
       call to the default operator new[](std::size_t).*

              [Footnote: The value must not have been invalidated by
              an intervening call to operator delete[](void*)
              (lib.res.on.arguments). --- end foonote]

       For such a non-null value of ptr , reclaims storage allocated
       by the earlier call to the default operator new[].

-13- It is unspecified under what conditions part or all of such
reclaimed storage is allocated by a subsequent call to operator new or
any of calloc, malloc, or realloc, declared in <cstdlib>.


Note the word above, ``reclaims.''  The calls should be symmetric as
one would guess without reading the standard.  In fact, it is a bug
that the standard doesn't refine the language and say exactly that it
must call delete.

Re: malloc/free & new/delete balance

[Alexandre Oliva]

Mike Stump <mrs@wrs.com> writes:

>> Agreed, but, for some reason, the FDIS [lib.new.delete.array]
>> defines operator new[](size_t) as returning operator new(size_t) by
>> default, while it states that operator delete[](void*) frees

> Can you provide an exact quote of this?  I can't find it.  I'll quote
> from the FDIS below.

I didn't write delete[] calls free(), I just said it frees
(deallocates) the memory.  No where it says delete[] calls delete by
default, and, since it does for new[] and new, I took it as a
difference, rather than as an omission.

> The next stage of your learning is to be able to say when the standard
> is wrong.  :-)

:-)

> Also, I think we are staring at different papers.  Does yours have the
> number 14882 or X3J16/97-0079 on it?  If not, that isn't the FDIS, and
> you should not call it that.

Actually, mine is an html-ized version that has no such numbers, it
just says `November 1997 Draft Standard'.  AFAIK, it contains all the
text from the FDIS.

>> From the real FDIS:

> -12- Default behavior: 
[snip]
>        For such a non-null value of ptr , reclaims storage allocated
>        by the earlier call to the default operator new[].

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:aoliva@acm.org
http://www.dcc.unicamp.br/~oliva
Universidade Estadual de Campinas, SP, Brasil

Re: malloc/free & new/delete balance

[Sol Foster]

Alexandre Oliva <oliva@dcc.unicamp.br> wrote:
> I didn't write delete[] calls free(), I just said it frees
> (deallocates) the memory.  No where it says delete[] calls delete by
> default, and, since it does for new[] and new, I took it as a
> difference, rather than as an omission.

I'd argue it's strongly implied that, if the memory is allocated with new,
then it is deallocated with delete.  It's disappointing that the standard
doesn't specify this, but then, it doesn't even specify that the memory is
actually given back to the heap.  (At least, that's my reading, and
Plauger seems to agree: "It is unspecified under what conditions part or
all of such reclaimed storage is allocated by a subsequent call to
operator new(size_t), or to any of calloc(size_t), malloc(size_t),
or realloc(void*, size_t).")

Can anyone think of any reasons other than these to use free?

1) delete just calls free anyway, so this saves a function call.

2) The standard doesn't absolutely specify using delete, so standard
conforming programs must replace new[] and delete[] if they replace new
and delete.  Therefore we should introduce hideous bugs into their code if
they don't comply.

Is the extra function call that much more expensive than what free does
internally?  



-- 
Sol Foster: colomon@ralf.org

If you pick up a starving dog and make him prosperous, he will not
bite you. This is the principal difference between a dog and a man. 
                                        -Mark Twain

Re: malloc/free & new/delete balance

[Carlo Wood]

| Can anyone think of any reasons other than these to use free?
| 
| 1) delete just calls free anyway, so this saves a function call.
| 
| 2) The standard doesn't absolutely specify using delete, so standard
| conforming programs must replace new[] and delete[] if they replace new
| and delete.  Therefore we should introduce hideous bugs into their code if
| they don't comply.

These are the two reasons I came up with too; and they would be
enough reason to keep the current call to free() if that wasn't
a bug:

1) The standards demands that the default delete[] reclaims storage
   allocated by a previous call to default new[].
2) The default new[] returns new, but NOT necessarily the default new.

While calling free() works for the egcs implementation of the
*default* `operator new(size_t)', it is no more then 'works'.
A coincident, because of how the default new of egcs is implemented.
In general it is not correct to call free() for a pointer allocated
with new.  Calling free() does NOT garantee that the call to delete[]
will reclaim the storage allocated by a previous call to default new[]
(it might just crash).

I think that the following code is valid C++, not invalidated by the
standard (ie: the standard does not demand to overload new[] and delete[]
when you overload new and delete):

void *operator new(size_t size)
{
  void *ptr = malloc(size + 2 * sizeof(size_t));
  if (ptr == NULL)
    return NULL;
  size_t *info = static_cast<size_t *>(ptr);
  info[0] = MAGIC_NUMBER;
  info[1] = size;
  return &info[2];
}

void operator delete(void *ptr)
{
  if (ptr == NULL)
    return;
  size_t *info = static_cast<size_t *>(ptr) - 2;
  free(info);
}

int main(void)
{
  char *p = new char[10];	// Calls default new[]
  delete [] p;			// Calls default delete[]
}

Which proves that delete[] calling free is not conforming to the
standard: It doesn't reclaim the storage allocated by the call
to new[], it crashes.

-- 
 Carlo Wood  <carlo@runaway.xs4all.nl>

Re: malloc/free & new/delete balance

[Nathan Myers]

Sol Foster wrote:
> 
> Alexandre Oliva <oliva@dcc.unicamp.br> wrote:
> > I didn't write delete[] calls free(), I just said it frees
> > (deallocates) the memory.  No where it says delete[] calls delete by
> > default, and, since it does for new[] and new, I took it as a
> > difference, rather than as an omission.
> 
> I'd argue it's strongly implied that, if the memory is allocated with new,
> then it is deallocated with delete.  It's disappointing that the standard
> doesn't specify this, but then, it doesn't even specify that the memory is
> actually given back to the heap. 

*You* try to write a requirement that the storage is re-used without
actually specifying an algorithm.  The C committee gave up, and the
C++ committee doesn't claim to be (much :) smarter.

> Is the extra function call that much more expensive than what free does
> internally?

In a *good* implementation you can expect the whole new/delete cycle
to take fewer than 40 instructions, on average.  Hence, an extra 
function call might be considered expensive.  (Now, *good* implementations
are not as common as I would like, but that ought to be fixable.)

Nathan

Re: malloc/free & new/delete balance

[Joern Rennecke]

> *You* try to write a requirement that the storage is re-used without
> actually specifying an algorithm.  The C committee gave up, and the
> C++ committee doesn't claim to be (much :) smarter.

How about listing some minimum requirement, like:

if an object is freed, and the next allocation is for an object of the
same size, that allocation will succeed.

Re: malloc/free & new/delete balance

[John Carr]

> How about listing some minimum requirement, like:
> 
> if an object is freed, and the next allocation is for an object of the
> same size, that allocation will succeed.

That rule prevents the runtime from releasing memory to the operating
system and also forces the runtime to use a different method to
allocate memory for its internal use.

Re: malloc/free & new/delete balance

[Joern Rennecke]

> > How about listing some minimum requirement, like:
> > 
> > if an object is freed, and the next allocation is for an object of the
> > same size, that allocation will succeed.
> 
> That rule prevents the runtime from releasing memory to the operating
> system and also forces the runtime to use a different method to
> allocate memory for its internal use.

Well, to address the internal use problem, you could define a set of
operations that won't cause internal use, and only make a statement
what happens as long as only these operations are used.  I hope no
implementation woul need to internally allocate memory for a sucession
of calls to new / delete after such a call has been made before?

Forbidding to release memory to the operating system is a different
matter, though.
Maybe we can work around this problem by limiting the statement to situations
where the operating system exhibits the same behaviour for allocating/freeing
memory. (I.e. you can re-allocate what you have freed - that should be
the case when no other process allocates memory, or if allocation is
limited by a software per-process limit first.
Of course a silly operating system might not even guarantee re-allocation
even for a single process running and no I/O going on; but that would
be not object of the definition.

We'd only that that *if* / while the OS allows re-allocation, and
while the program performs only operations of a limited set that need
no dynamic internal storage growth in the runtime, then the program
can also perform re-allocation.