section placement q

section placement q

[Andrew Morton]

The linux kernel likes to put lots of things into special sections so
those sections can be unloaded when not neeed.  However this is less
effective than it might be because strings are still placed in .rodata. 
Example:

struct foo
{
        int i;
        char *s;
};

struct foo foo_array[] __attribute__ ((__section__(".foosect"))) =
{
        { 12, "foo1" }
};

Here, the integer and the char * are placed in .foosect, but the actual
string "foo1\0" is placed in .rodata.

I believe this behaviour is correct and logical.  One probably unpopular
workaround is to change 'char *s' into 'char s[20]'.

Can anyone suggest a construct which will allow the string to be placed
in a different section while retaining the current 'char *' semantics?


Thanks.

-- 
-akpm-

Re: section placement q

[Linus Torvalds]

In article < 391B5AF7.C26E47E4@uow.edu.au > you write:
>The linux kernel likes to put lots of things into special sections so
>those sections can be unloaded when not neeed.  However this is less
>effective than it might be because strings are still placed in .rodata. 
>Example:
>
>struct foo
>{
>        int i;
>        char *s;
>};
>
>struct foo foo_array[] __attribute__ ((__section__(".foosect"))) =
>{
>        { 12, "foo1" }
>};
>
>Here, the integer and the char * are placed in .foosect, but the actual
>string "foo1\0" is placed in .rodata.
>
>I believe this behaviour is correct and logical.  One probably unpopular
>workaround is to change 'char *s' into 'char s[20]'.

I don't think this is necessarily logical, although "correct" is
obviously simply a definition question. It depends on what you want. In
many cases it might be advantageous to be able to specify where the
initialiser goes regardless of where the actual data is. I can imagine
cases where you might have

	struct char * name __attribute__((__section(".namesection"))) =
		(char [] __attribute__((__section__(".stringsection"))) "foo";

Ugly as hell, but certainly flexible.  And anybody sane would hide this
behind macros anyway, if for no other reason than portability. 

(I actually think that the "unused inline function that has a string"
case is more important, but I think that got fixed already, no?)

>Can anyone suggest a construct which will allow the string to be placed
>in a different section while retaining the current 'char *' semantics?

Have you looked at how the kernel build handles the PCI "names" issue?
This was one case where the string list is _huge_, and we obviously want
to drop it after booting and naming only the devices we have (as opposed
to every single PCI name in existence).  So it uses generated code to do
what we want - ugly, but works, and causes all strings to be in the
".init" section. 

Of course, as the PCI name database is generated anyway, this solution
works fine for that case. It certainly doesn't handle the generic case
for the automatic link-time garbage-collection, which is what I assume
you're working on..

		Linus

Re: section placement q

[Andrew Morton]

Linus Torvalds wrote:
> 
> In article < 391B5AF7.C26E47E4@uow.edu.au > you write:
> >The linux kernel likes to put lots of things into special sections so
> >those sections can be unloaded when not neeed.  However this is less
> >effective than it might be because strings are still placed in .rodata.
> >Example:
> >
> >struct foo
> >{
> >        int i;
> >        char *s;
> >};
> >
> >struct foo foo_array[] __attribute__ ((__section__(".foosect"))) =
> >{
> >        { 12, "foo1" }
> >};
> >
> >Here, the integer and the char * are placed in .foosect, but the actual
> >string "foo1\0" is placed in .rodata.
> >
> >I believe this behaviour is correct and logical.  One probably unpopular
> >workaround is to change 'char *s' into 'char s[20]'.
> 
> I don't think this is necessarily logical, although "correct" is
> obviously simply a definition question. It depends on what you want.

yes.  We've asked the compiler to place the struct in .foosect but not
the string.  It does this, but it is not what we want.

> ...
> (I actually think that the "unused inline function that has a string"
> case is more important, but I think that got fixed already, no?)

There are several similar instances where the behaviour we want is
different.

I reported one of these - it was the

	if (0)
	{
		foo("some_string");
	}

problem.

I'll detail them below.

> ...
> It certainly doesn't handle the generic case
> for the automatic link-time garbage-collection, which is what I assume
> you're working on..

Actually, no.  My itch is that fact that marking an array of structs
(3c59x.c:vortex_info_tbl[]) as __devinitdata puts the array into section
.data.init, but the strings remain in .rodata.  Not what we want.


Let's itemise the things we _do_ want:

-ffunction-sections
-------------------

foo()
{
        bar("hello, world\n");
}
gcc-experimental -ffunction-sections -fdata-sections -O -S fs.c

In this case, the function goes in ".text.foo", but the string goes into
.rodata.  We (Graham) would like the string in ".data.foo", I guess.

__attribute__ (__section__)
---------------------------

void __attribute__ ((__section__ (".some_section"))) foo(void)
{
        bar("hello, world\n");
}

The string goes into .rodata.  We'd like it in <where?>

inline
------

extern inline foo(char *s)
{
}

bar()
{
        foo("hello\n");
}

The string hangs around in .data, unreferenced.  This is a dead-code
(dead data?) elimination issue, rather than a section-fiddling issue. 
Martin Loewis has entered this in gnats.

anon strings in static defns
----------------------------

struct thing
{
        int i;
        char *s;
};

struct thing __attribute__ ((__section__ (".text.init"))) things[]  =
{
        { 12, "a string" }
};

The string goes in .rodata.  We'd like to be able to control its
section. In this case ".data.init".




We either need a way of telling the compiler to divert '.rodata', or a
construct like:

foo()
{
        char *p;

        p = __attribute__ ((__section__ (".some_section"))) "hello";
}

Re: section placement q

[Graham Stoney]

I might just comment on the items I'm particularly interested in...

Andrew Morton writes:
> -ffunction-sections
> -------------------
>
> foo()
> {
>         bar("hello, world\n");
> }
> gcc-experimental -ffunction-sections -fdata-sections -O -S fs.c
>
> In this case, the function goes in ".text.foo", but the string goes into
> .rodata.  We (Graham) would like the string in ".data.foo", I guess.

Only if -fwritable-strings is also specified. Otherwise, I think it should go
in ".rodata.foo", where "foo" is the first function that uses the string given
that read-only strings can be shared. This will Do The Right Thing(tm) in the
face of --gc-sections even if bar gets optimised away and the string is also
used by another function.

> anon strings in static defns
> ----------------------------
> 
> struct thing
> {
>         int i;
>         char *s;
> };
> 
> struct thing __attribute__ ((__section__ (".text.init"))) things[]  =
> {
>         { 12, "a string" }
> };
> 
> The string goes in .rodata.  We'd like to be able to control its
> section. In this case ".data.init".

Shouldn't this be using section ".data.init"?. I think you want all the
strings referenced in "things" go in this section too. In other words, string
literals referenced in static structs could inherit the same section as their
parent object. The compiler might need to recognise that strings in such
circumstances aren't necessarily shareable if their sections differ, so that
non-.init code which also uses "a string" doesn't break.

Regards,
Graham
-- 
Graham Stoney
Principal Hardware/Software Engineer
Canon Information Systems Research Australia
Ph: +61 2 9805 2909  Fax: +61 2 9805 2929

Re: section placement q

[Linus Torvalds]

On Mon, 15 May 2000, Graham Stoney wrote:

> I might just comment on the items I'm particularly interested in...
> 
> Andrew Morton writes:
> > -ffunction-sections
> > -------------------
> >
> > foo()
> > {
> >         bar("hello, world\n");
> > }
> > gcc-experimental -ffunction-sections -fdata-sections -O -S fs.c
> >
> > In this case, the function goes in ".text.foo", but the string goes into
> > .rodata.  We (Graham) would like the string in ".data.foo", I guess.
> 
> Only if -fwritable-strings is also specified. Otherwise, I think it should go
> in ".rodata.foo", where "foo" is the first function that uses the string given
> that read-only strings can be shared. This will Do The Right Thing(tm) in the
> face of --gc-sections even if bar gets optimised away and the string is also
> used by another function.

The problem (and I bet the reason why gcc does what it currently does), is
that there are no guarantees that the section will be entirely removed. It
might be only partially used, as with the Linux use of an "initdata"
section that is used for initializations, but later dropped.

To see why this is a problem, imagine that we have such an "init" section,
and that "foo" foes into ".text.init", and the string "hello, world\n"
goes into the section ".rodata.init". Is that the right thing to do?

Not necessarily: the initialization function may set up data structures
etc, and save away the pointer to the string. Which is not all that
obviously a bug. So when (later) the init sections are free'd, you get
faulty behaviour.

This is true of function-sections as well. As long as the
function-sections are used _only_ for garbage collection at link-time, the
".rodata.foo" behaviour is the obviously correct one. But the ".rodata"
approach is at least the one that is the least likely to result in some
rather subtle bugs - there might be other, quite valid, uses of
per-function sections.

So a constant string data item is definitely different from a static
variable. Making a static variable follow the function (-fdata-sections)
makes sense pretty much all of the time. For constant strings this is not
necessarily as obviously the case if only because they are sometimes taken
for granted.

		Linus

Re: section placement q

[Artur Skawina]

Andrew Morton wrote:
> 
> void __attribute__ ((__section__ (".some_section"))) foo(void)
> {
>         bar("hello, world\n");
> }
> 
> The string goes into .rodata.  We'd like it in <where?>

i don't know where you'd like to have it, but it _must_ default to
.rodata (for obvious reasons -- the pointer might stick around) 


> We either need a way of telling the compiler to divert '.rodata', or a
> construct like:
> 
> foo()
> {
>         char *p;
> 
>         p = __attribute__ ((__section__ (".some_section"))) "hello";
> }

fwiw what i once did was:

--- /img/linux-2.3.40pre1/include/linux/init.h  Thu Jan 13 18:13:33 2000
+++ linux-2.3.40pre1as/include/linux/init.h     Thu Feb 17 03:46:25 2000
@@ -73,6 +73,7 @@ extern struct kernel_param __setup_start
 #define __exitdata     __attribute__ ((unused, __section__ (".data.exit")))
 #define __initsetup    __attribute__ ((unused,__section__ (".setup.init")))
 #define __init_call    __attribute__ ((unused,__section__ (".initcall.init")))
+#define __initstr(s) ({static const char __attribute__((__section__(".rodata.init"))) __is[] = s; __is; })
 
 /* For assembly routines */
 #define __INIT         .section        ".text.init","ax"

and marked all strings as necessary. Saved a few dozen kb, but was
ugly enough that i decided it wasn't worth maintaining.
i can't see much that the compiler could help with here...

Re: section placement q

[Andrew Morton]

Artur Skawina wrote:
> 
> Andrew Morton wrote:
> >
> > void __attribute__ ((__section__ (".some_section"))) foo(void)
> > {
> >         bar("hello, world\n");
> > }
> >
> > The string goes into .rodata.  We'd like it in <where?>
> 
> i don't know where you'd like to have it, but it _must_ default to
> .rodata (for obvious reasons -- the pointer might stick around)

Sure, the default shouldn't be altered.

What would be ideal is the ability to alter the default for a particular
scope:

void __attribute__ ( (__section__ (".some_section")),
                     (__strings__(".some_string_section")))
foo(void)
{
	bar("hello");
}

So during the scope of the __strings__ attribute, the old (".rodata")
section identifier is stacked and replaced with ".some_string_section".

Guess I should shut up and look at the source :)

> ...
> Saved a few dozen kb, but was
> ugly enough that i decided it wasn't worth maintaining.

mm..  A "few dozen kb" is fairly attractive, actually.

-- 
-akpm-

Re: section placement q

[Artur Skawina]

Andrew Morton wrote:
> 
> > > void __attribute__ ((__section__ (".some_section"))) foo(void)
> > > {
> > >         bar("hello, world\n");
> > > }
> > >
> > > The string goes into .rodata.  We'd like it in <where?>
> >
> > i don't know where you'd like to have it, but it _must_ default to
> > .rodata (for obvious reasons -- the pointer might stick around)
> 
> Sure, the default shouldn't be altered.
> 
> What would be ideal is the ability to alter the default for a particular
> scope:
> 
> void __attribute__ ( (__section__ (".some_section")),
>                      (__strings__(".some_string_section")))
> foo(void)
> {
>         bar("hello");
> }
> 
> So during the scope of the __strings__ attribute, the old (".rodata")
> section identifier is stacked and replaced with ".some_string_section".

But would this really solve the problem? Yes, it could reduce the
ugliness a bit, but is potentialy dangerous. As you still have to
make sure no "local" strings are used later, i'm not convinced this
is actually an improvement over the conservative approach.


> > Saved a few dozen kb, but was
> > ugly enough that i decided it wasn't worth maintaining.
> 
> mm..  A "few dozen kb" is fairly attractive, actually.

yep, that's why i did it. Most of it are strings that never get
accessed (fatal err msgs etc) or are only used once, but aren't
freed wasting precious unswappable memory.
I've put up the patch vs 2.3.40pre1 at 
 http://www.geocities.com/SiliconValley/Heights/6494/sw/patch-initstr.gz
It really is ugly as hell, but i'm not sure it can be made cleaner by
any compiler extension (flipping the default could be too dangerous)

Re: section placement q

[Andrew Morton]

Artur Skawina wrote:
> 
> ...
> > void __attribute__ ( (__section__ (".some_section")),
> >                      (__strings__(".some_string_section")))
> > foo(void)
> > {
> >         bar("hello");
> > }
> >
> > So during the scope of the __strings__ attribute, the old (".rodata")
> > section identifier is stacked and replaced with ".some_string_section".
> 
> But would this really solve the problem? Yes, it could reduce the
> ugliness a bit, but is potentialy dangerous. As you still have to
> make sure no "local" strings are used later, i'm not convinced this
> is actually an improvement over the conservative approach.

I agree that it's a bit dangerous to use on a function, but in practise,
this 'default string diversion' would mainly be used within
initialisers, where accidents are less likely to occur.  Caveat emptor.

> ...
> I've put up the patch vs 2.3.40pre1 at
>  http://www.geocities.com/SiliconValley/Heights/6494/sw/patch-initstr.gz
> ...

Wow.  Did that save ~30k?  It only touches a fraction of the files which
it _could_ be used on.

-- 
-akpm-

Re: section placement q

[Artur Skawina]

Andrew Morton wrote:
> 
> >  http://www.geocities.com/SiliconValley/Heights/6494/sw/patch-initstr.gz
> 
> Wow.  Did that save ~30k? 

I don't remember the actual numbers anymore (iirc for UP kernels
the saving was ~12k, for SMP ones it was significantly more).
Something that the compiler could help with is string alignment
-- aligning to 32 bytes etc is not really necessary for those
strings...

Re: section placement q

[Jamie Lokier]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Artur Skawina wrote:
> > So during the scope of the __strings__ attribute, the old (".rodata")
> > section identifier is stacked and replaced with ".some_string_section".
> 
> But would this really solve the problem? Yes, it could reduce the
> ugliness a bit, but is potentialy dangerous. As you still have to
> make sure no "local" strings are used later, i'm not convinced this
> is actually an improvement over the conservative approach.

A much more thorough approach, without the danger and still with
sharing, is to have the linker optimise, à la -ffunction-sections
-fdata-sections.  But extend the linker's gc-sections to do something
like: put section .S in .init.S unless anybody from outside .init*
references the section.

There's probably a way to do that with the existing linker and multiple
link stages.

That would even mean you wouldn't have to type "__init" before each
function of the appropriate kind.  You'd only use it for the few things
that are referenced by non-init but should be discarded after anyway.

-- Jamie

Re: section placement q

[Richard Henderson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

On Thu, May 25, 2000 at 07:09:18PM +0200, Jamie Lokier wrote:
> A much more thorough approach, without the danger and still with
> sharing, is to have the linker optimise, à la -ffunction-sections
> -fdata-sections.  But extend the linker's gc-sections to do something
> like: put section .S in .init.S unless anybody from outside .init*
> references the section.

Putting this kind of policy in the linker proper would be silly.
You're talking about exactly one application that wants this sort
of .init behaviour.

It wouldn't be too hard, however, to build a tool that examined
object files and generated a link script to do exactly what you want.


r~

Re: section placement q

[Artur Skawina]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 425 bytes --]

Jamie Lokier wrote:
> 
> A much more thorough approach, without the danger and still with
> sharing, is to have the linker optimise, à la -ffunction-sections
> -fdata-sections.  But extend the linker's gc-sections to do something
> like: put section .S in .init.S unless anybody from outside .init*
> references the section.

void foo() __attribute__ ((section("init")) { bar("hello"); }
char *p;
void bar(char *s) {p=s;}

Re: section placement q

[Jamie Lokier]

Artur Skawina wrote:
> > A much more thorough approach, without the danger and still with
> > sharing, is to have the linker optimise,  la -ffunction-sections
> > -fdata-sections.  But extend the linker's gc-sections to do something
> > like: put section .S in .init.S unless anybody from outside .init*
> > references the section.
> 
> void foo() __attribute__ ((section("init")) { bar("hello"); }
> char *p;
> void bar(char *s) {p=s;}

Ew.  Is there anything short of global data flow/type analysis that can
at least warn about this sort of thing?

-- Jamie

Re: section placement q

[Andrew Morton]

Jamie Lokier wrote:
> 
> Artur Skawina wrote:
> > > A much more thorough approach, without the danger and still with
> > > sharing, is to have the linker optimise,  la -ffunction-sections
> > > -fdata-sections.  But extend the linker's gc-sections to do something
> > > like: put section .S in .init.S unless anybody from outside .init*
> > > references the section.
> >
> > void foo() __attribute__ ((section("init")) { bar("hello"); }
> > char *p;
> > void bar(char *s) {p=s;}
> 

I really don't see what we're fussed about.

If you do the above, you have written a bug and you get to enjoy a core
file.  There are plenty of ways of doing that!  You can create the same
bug with the _existing_ section control code.

There's nothing inherently bad about being to divert the section for
strings.  You just need to know what you're doing, no?


-- 
-akpm-

Re: section placement q

[Jamie Lokier]

Andrew Morton wrote:
> There's nothing inherently bad about being to divert the section for
> strings.  You just need to know what you're doing, no?

That's the problem.  Not only can it be tricky -- do you think every
device driver author knows what __init etc. really mean?  But also even
when it's wrong, it will often appear to work.  A warning would be nice.
Perhaps this is out of GCC's scope though, along with lock checking.

-- Jamie