[ECOS] eCos Kernel: ALLOC callbacks

[ECOS] eCos Kernel: ALLOC callbacks

[NavEcos]

About a month ago, I suggested making some changes to eCos that would allow:

a) a callback for every heap creation
b) a callback for every heap destroy
c) a callback for post allocation/decallocation
d) a callback for pre allocation/decallocation

This was so that you could add additional functionality, like putting in 
padding in the alloced memory blocks to make certain memory isn't being 
corrupted, and being certain that you're not allocated from heaps that have 
been destroyed, etc.

Here are my questions:

1)

I think the changes need to be made to

services/memalloc/common/current/src/memvar.cxx
services/memalloc/common/current/src/memfixed.cxx

And the definitions of the classes Cyg_Mempool_Fixed and Cyg_Mempool_Variable

And that should cover doug lea's memory allocation as well.  Is this correct 
or am I missing other classes that need to be modified

2)

It appears that heaps are being created before cyg_user_start.  I want to 
install the callbacks into the system before any heaps are created - where 
should I install the callbacks?

3)

I would like to make the callbacks global that that installing the callbacks 
will do it for all heaps, not just an individual heap.  Are there any 
objections to this?


Thanks,
-Rich

--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

[ECOS] How to add a source file into a configuration

[Yao-Lu Tsai]

Hi,

If I choose Samsung SNDS platform with new_net template, everything
when compiling & linking is OK. However, if I add "Samsung KS32C5000
ethernet driver" package into this configuration, linking is failed due to
"setdomainname" undefined. This function is declared in dns.c. I don't know
why this file is not compiled after I add an additional package. My question
is,
how to add dns.c into the configuration to solve the linking error ?

Thanks.
=================
Yao-Lu Tsai
KCodes Corporation



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] How to add a source file into a configuration

[Andrew Lunn]

On Wed, Oct 23, 2002 at 04:06:43PM +0800, Yao-Lu Tsai wrote:
> Hi,
> 
> If I choose Samsung SNDS platform with new_net template, everything
> when compiling & linking is OK. However, if I add "Samsung KS32C5000
> ethernet driver" package into this configuration, linking is failed due to
> "setdomainname" undefined. This function is declared in dns.c. I don't know
> why this file is not compiled after I add an additional package. My question
> is,
> how to add dns.c into the configuration to solve the linking error ?

The DNS package is automatically added as part of the new_net
template. 

Here is what i did:

$ ecosconfig new snds new_net
U CYGPKG_KERNEL_THREADS_DESTRUCTORS, new inferred value 1
U CYGBLD_ISO_POSIX_CLOCK_TYPES_HEADER, new inferred value <cyg/posix/time.h>
U CYGBLD_ISO_POSIX_CLOCKS_HEADER, new inferred value <cyg/posix/time.h>
U CYGBLD_ISO_PTHREAD_MUTEX_HEADER, new inferred value <cyg/posix/mutex.h>
U CYGBLD_ISO_NAME_MAX_HEADER, new inferred value <cyg/fileio/limits.h>
U CYGBLD_ISO_NETDB_PROTO_HEADER, new inferred value <net/netdb.h>
U CYGBLD_ISO_NETDB_SERV_HEADER, new inferred value <net/netdb.h>
$ ecosconfig add eth_ks32c5000
$ ecosconfig tree
$ make

and then dns.c is getting compiled.

Try doing it again and make sure you are start in an empty directory.

    Andrew

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] eCos Kernel: ALLOC callbacks

[Jonathan Larmour]

NavEcos wrote:
> About a month ago, I suggested making some changes to eCos that would allow:
> 
> a) a callback for every heap creation
> b) a callback for every heap destroy
> c) a callback for post allocation/decallocation
> d) a callback for pre allocation/decallocation
> 
> This was so that you could add additional functionality, like putting in 
> padding in the alloced memory blocks to make certain memory isn't being 
> corrupted, and being certain that you're not allocated from heaps that have 
> been destroyed, etc.

I should comment on your original mail better, but in brief this is 
reasonable, as long as it is optional!

> Here are my questions:
> 
> 1)
> 
> I think the changes need to be made to
> 
> services/memalloc/common/current/src/memvar.cxx
> services/memalloc/common/current/src/memfixed.cxx
> 
> And the definitions of the classes Cyg_Mempool_Fixed and Cyg_Mempool_Variable
> 
> And that should cover doug lea's memory allocation as well.  Is this correct 
> or am I missing other classes that need to be modified

No, it wouldn't cover dlmalloc. Just the bog standard fixed and variable 
block allocators. You'd need to edit dlmalloc.cxx

> 2)
> 
> It appears that heaps are being created before cyg_user_start.  I want to 
> install the callbacks into the system before any heaps are created - where 
> should I install the callbacks?

They are created by global constructors. "Simply" ( :-) ) create a  global 
constructor of higher priority than any constructor creating a heap.

> 3)
> 
> I would like to make the callbacks global that that installing the callbacks 
> will do it for all heaps, not just an individual heap.  Are there any 
> objections to this?

Configurability is the key. With configurability you could maybe do 
either, both[1] or neither.

Jifl
[1] Use a specific callback if called, or else the global one
-- 
eCosCentric       http://www.eCosCentric.com/       <info@eCosCentric.com>
--[ "You can complain because roses have thorns, or you ]--
--[  can rejoice because thorns have roses." -Lincoln   ]-- Opinions==mine


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] How to add a source file into a configuration

[Jonathan Larmour]

Andrew Lunn wrote:
> On Wed, Oct 23, 2002 at 04:06:43PM +0800, Yao-Lu Tsai wrote:
>>
>>If I choose Samsung SNDS platform with new_net template, everything
>>when compiling & linking is OK. However, if I add "Samsung KS32C5000
>>ethernet driver" package into this configuration, linking is failed due to
>>"setdomainname" undefined. This function is declared in dns.c. I don't know
>>why this file is not compiled after I add an additional package. My question
>>is,
>>how to add dns.c into the configuration to solve the linking error ?
> 
> 
> The DNS package is automatically added as part of the new_net
> template. 
> 
> Here is what i did: [snip] 
> and then dns.c is getting compiled.
> 
> Try doing it again and make sure you are start in an empty directory.

It could also happen if one of the requirements of CYGPKG_NS_DNS_BUILD 
isn't being fulfilled and it got inferred off. Check the value of that CDL 
option in particular.

Jifl
-- 
eCosCentric       http://www.eCosCentric.com/       <info@eCosCentric.com>
--[ "You can complain because roses have thorns, or you ]--
--[  can rejoice because thorns have roses." -Lincoln   ]-- Opinions==mine


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] eCos Kernel: ALLOC callbacks

[NavEcos]

On Wednesday 23 October 2002 08:24 pm, Jonathan Larmour wrote:
> NavEcos wrote:
> > About a month ago, I suggested making some changes to eCos that would
> > allow:
> >
> > a) a callback for every heap creation
> > b) a callback for every heap destroy
> > c) a callback for post allocation/decallocation
> > d) a callback for pre allocation/decallocation
> >
> > This was so that you could add additional functionality, like putting in
> > padding in the alloced memory blocks to make certain memory isn't being
> > corrupted, and being certain that you're not allocated from heaps that
> > have been destroyed, etc.
>
> I should comment on your original mail better, but in brief this is
> reasonable, as long as it is optional!

Yes, it will be an option in CDL of course.

> > Here are my questions:
> >
> > 1)
> >
> > I think the changes need to be made to
> >
> > services/memalloc/common/current/src/memvar.cxx
> > services/memalloc/common/current/src/memfixed.cxx
> >
> > And the definitions of the classes Cyg_Mempool_Fixed and
> > Cyg_Mempool_Variable
> >
> > And that should cover doug lea's memory allocation as well.  Is this
> > correct or am I missing other classes that need to be modified
>
> No, it wouldn't cover dlmalloc. Just the bog standard fixed and variable
> block allocators. You'd need to edit dlmalloc.cxx

OK.

Let me make it clear that I want to cover *every* dyanmic memory allocation 
that can be made by the application layer or the system.  My goal it to be 
able to PROVE there are no memory leaks anywhere in the sytem, including the 
OS itself.  This should give eCos a stronger selling point to potential 
customers.  The same functionality can be used to prove there are no memory 
leaks in an application on top of eCos.

I was putting the hooks into the Cyg_Mempool_Fixed and Cyg_Mempool_Variable 
classes, would putting in the additional functionality into 
Cyg_Mempool_dlmalloc cover all the bases then?  Should I instead put the 
hooks into the *impl.hxx files instead?

Essentially what I'm asking, is where should I put in the hooks to make it the 
LEAST amount of work to maintain and yet cover all dynamic memory allocations 
and frees?  So far, the list I have is:

memfixed.hxx
memvar.hxx
dlmalloc.hxx

And their associated .cxx files.

Am I missing any files?  If I am, please tell me which ones.

> > 2)
> >
> > It appears that heaps are being created before cyg_user_start.  I want to
> > install the callbacks into the system before any heaps are created -
> > where should I install the callbacks?
>
> They are created by global constructors. "Simply" ( :-) ) create a  global
> constructor of higher priority than any constructor creating a heap.

Ow - I think I got hooked on that catch.

Can you provide pointers as to how to do this?  Is it ANSI C++ to do this 
(i.e. portable across compilers) ?

> > 3)
> >
> > I would like to make the callbacks global that that installing the
> > callbacks will do it for all heaps, not just an individual heap.  Are
> > there any objections to this?
>
> Configurability is the key. With configurability you could maybe do
> either, both[1] or neither.
>
> Jifl
> [1] Use a specific callback if called, or else the global one

Let me do the global callback support first since that is what *I* need, I'll 
be happy to implement the individual ones next :)  That sound good?

I appreciate your input very much and your help.

-Rich

--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] How to add a source file into a configuration

[Yao-Lu Tsai]

I eventually solved this problem by turning on the "DNS client -> Build DNS
NS client
package" option in the GUI configuration tool.

Regards,
=================
Yao-Lu Tsai
KCodes Corporation

----- Original Message -----
From: "Jonathan Larmour" <jifl@eCosCentric.com>
To: "Andrew Lunn" <andrew.lunn@ascom.ch>
Cc: "Yao-Lu Tsai" <ytsai@kcodes.com>; "eCos Discussion"
<ecos-discuss@sources.redhat.com>
Sent: Thursday, October 24, 2002 11:26 AM
Subject: Re: [ECOS] How to add a source file into a configuration


> Andrew Lunn wrote:
> > On Wed, Oct 23, 2002 at 04:06:43PM +0800, Yao-Lu Tsai wrote:
> >>
> >>If I choose Samsung SNDS platform with new_net template, everything
> >>when compiling & linking is OK. However, if I add "Samsung KS32C5000
> >>ethernet driver" package into this configuration, linking is failed due
to
> >>"setdomainname" undefined. This function is declared in dns.c. I don't
know
> >>why this file is not compiled after I add an additional package. My
question
> >>is,
> >>how to add dns.c into the configuration to solve the linking error ?
> >
> >
> > The DNS package is automatically added as part of the new_net
> > template.
> >
> > Here is what i did: [snip]
> > and then dns.c is getting compiled.
> >
> > Try doing it again and make sure you are start in an empty directory.
>
> It could also happen if one of the requirements of CYGPKG_NS_DNS_BUILD
> isn't being fulfilled and it got inferred off. Check the value of that CDL
> option in particular.
>
> Jifl
> --
> eCosCentric       http://www.eCosCentric.com/       <info@eCosCentric.com>
> --[ "You can complain because roses have thorns, or you ]--
> --[  can rejoice because thorns have roses." -Lincoln   ]-- Opinions==mine
>
>
> --
> Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
> and search the list archive: http://sources.redhat.com/ml/ecos-discuss


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] eCos Kernel: ALLOC callbacks

[Jonathan Larmour]

NavEcos wrote:
> [snip]
>>>Here are my questions:
>>>
>>>1)
>>>
>>>I think the changes need to be made to
>>>
>>>services/memalloc/common/current/src/memvar.cxx
>>>services/memalloc/common/current/src/memfixed.cxx
>>>
>>>And the definitions of the classes Cyg_Mempool_Fixed and
>>>Cyg_Mempool_Variable
>>>
>>>And that should cover doug lea's memory allocation as well.  Is this
>>>correct or am I missing other classes that need to be modified
>>
>>No, it wouldn't cover dlmalloc. Just the bog standard fixed and variable
>>block allocators. You'd need to edit dlmalloc.cxx
> 
> 
> OK.
> 
> Let me make it clear that I want to cover *every* dyanmic memory allocation 
> that can be made by the application layer or the system.  My goal it to be 
> able to PROVE there are no memory leaks anywhere in the sytem, including the 
> OS itself.  This should give eCos a stronger selling point to potential 
> customers.  The same functionality can be used to prove there are no memory 
> leaks in an application on top of eCos.
> 
> I was putting the hooks into the Cyg_Mempool_Fixed and Cyg_Mempool_Variable 
> classes, would putting in the additional functionality into 
> Cyg_Mempool_dlmalloc cover all the bases then?

Should do. The sepmeta allocator isn't instantiated for anything.

 > Should I instead put the
> hooks into the *impl.hxx files instead?

Well, there's no sensible way to add them there without an unnecessary 
extra level of indirection. So I'd say no.

> Essentially what I'm asking, is where should I put in the hooks to make it the 
> LEAST amount of work to maintain and yet cover all dynamic memory allocations 
> and frees?  So far, the list I have is:
> 
> memfixed.hxx
> memvar.hxx
> dlmalloc.hxx
> 
> And their associated .cxx files.
> 
> Am I missing any files?  If I am, please tell me which ones.

Well, one could argue that the system and the user always use separate 
pools, the only exception being malloc, so malloc.cxx may be sufficient. 
In other words, are you sure you want to track the times the kernel uses 
memory pools (and remember they have a fixed size and are segregated from 
other dynamic allocations) for *any* reason, or only when you want to 
track those allocations that could overlap with a user's.

>>>2)
>>>
>>>It appears that heaps are being created before cyg_user_start.  I want to
>>>install the callbacks into the system before any heaps are created -
>>>where should I install the callbacks?
>>
>>They are created by global constructors. "Simply" ( :-) ) create a  global
>>constructor of higher priority than any constructor creating a heap.
> 
> 
> Ow - I think I got hooked on that catch.
> 
> Can you provide pointers as to how to do this?  Is it ANSI C++ to do this 
> (i.e. portable across compilers) ?

It's not ISO C++ or portable no. There's no way in ISO C++ to do this - 
global constructor initialization order is undefined. This is a G++ 
extension. The info files document it, but the raw syntax is:

Objtype obj __attribute__((init_priority(50000)));

or some other number from 0 to 65535, the latter being the default init 
priority anyway. But the preferred way to do this is with a macro 
CYGBLD_ATTRIB_INIT_PRI. See infra/current/include/cyg_type.h. Search for 
the uses of this macro (grep -r) for usage examples. There's plenty 
around. Some code still uses the CYG_INIT_PRIORITY macro, but that's 
deprecated really.

Jifl
-- 
eCosCentric       http://www.eCosCentric.com/       <info@eCosCentric.com>
--[ "You can complain because roses have thorns, or you ]--
--[  can rejoice because thorns have roses." -Lincoln   ]-- Opinions==mine


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] eCos Kernel: ALLOC callbacks

[NavEcos]

On Thursday 24 October 2002 08:24 pm, Jonathan Larmour wrote:

This was getting pretty long so:

[snip]

> Well, one could argue that the system and the user always use separate
> pools, the only exception being malloc, so malloc.cxx may be sufficient.
> In other words, are you sure you want to track the times the kernel uses
> memory pools (and remember they have a fixed size and are segregated from
> other dynamic allocations) for *any* reason, or only when you want to
> track those allocations that could overlap with a user's.

I want to track all allocations, or at least have the ability to in order to 
prove there are no memory leaks anywhere in the system.  Doing it for the 
system as well as the user level is equally easy, so I may as well cover all 
of them is my thinking.

I will submit a preliminary path pretty soon to the changes I make, and try to 
include CDL changes as well.

> >>>2)
> >>>
> >>>It appears that heaps are being created before cyg_user_start.  I want
> >>> to install the callbacks into the system before any heaps are created -
> >>> where should I install the callbacks?
> >>
> >>They are created by global constructors. "Simply" ( :-) ) create a 
> >> global constructor of higher priority than any constructor creating a
> >> heap.
> >
> > Ow - I think I got hooked on that catch.
> >
> > Can you provide pointers as to how to do this?  Is it ANSI C++ to do this
> > (i.e. portable across compilers) ?
>
> It's not ISO C++ or portable no. There's no way in ISO C++ to do this -
> global constructor initialization order is undefined. This is a G++
> extension. The info files document it, but the raw syntax is:
>
> Objtype obj __attribute__((init_priority(50000)));
>
> or some other number from 0 to 65535, the latter being the default init
> priority anyway. But the preferred way to do this is with a macro
> CYGBLD_ATTRIB_INIT_PRI. See infra/current/include/cyg_type.h. Search for
> the uses of this macro (grep -r) for usage examples. There's plenty
> around. Some code still uses the CYG_INIT_PRIORITY macro, but that's
> deprecated really.
>
> Jifl

OK - I'll look into doing that.

--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

[ECOS] Quick and stupid question RE: dlmalloc

[NavEcos]

I've been tracing the code, and I don't see who CAN use dlmalloc.  It doesn't 
appear to be part of the eCos kapi.cxx files so the cyg_mempool_var_* don't 
use the dlmalloc implementation.  How do you access the dlmalloc from user 
level code or is it reserved for system level allocation/deallocation?

Thanks
-Rich

--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

[ECOS] Re: Quick and stupid question RE: dlmalloc

[Jonathan Larmour]

NavEcos wrote:
> I've been tracing the code, and I don't see who CAN use dlmalloc.  It doesn't 
> appear to be part of the eCos kapi.cxx files so the cyg_mempool_var_* don't 
> use the dlmalloc implementation.  How do you access the dlmalloc from user 
> level code or is it reserved for system level allocation/deallocation?

malloc uses it by default. Or users can use it directly using 
<cyg/memalloc/dlmalloc.hxx>. See the memalloc package's CDL.

Jifl
-- 
eCosCentric       http://www.eCosCentric.com/       <info@eCosCentric.com>
--[ "You can complain because roses have thorns, or you ]--
--[  can rejoice because thorns have roses." -Lincoln   ]-- Opinions==mine


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss