Multiple ABIs for a single compilation unit?

Multiple ABIs for a single compilation unit?

[Lauri Alanko]

Hello.

With x86, the -freg-struct-return and -mregparm=x parameters are fun,
and create (to my untrained eye) shorter and more efficient
code. However, these parameters aren't used in the standard ABIs for
Linux, at least not in Debian or Redhat. So one cannot use them in
practice, unless one is to recompile all system libraries.

So what I'd like is a way to compile programs so that _some_ external
functions are assumed to use one (the "conventional") ABI, while other
stuff, basically what I've compiled, would use -freg-struct-return and
-mregparm, or other exotic options. Currently there seems to be no way
to do this.

All right, there is the __attribute__ ((regparm (x))), as well as cdecl
and stdcall, but that is, IMO, the wrong way to do it. Firstly, a header
file should only describe the API, not the ABI. (Or otherwise
maintaining headers will be horrible). Second, the regparm attribute
doesn't work too well (it isn't inherited by virtual functions in C++,
for example, and there's no warning of it either!).

In practice, I think the way to do it would be to support some sort of
list of directories and headers with their ABI specifications, and then
the compiler would use the preprocessor filename information to decide
what ABI a function is supposed to have. So I could say something like:

/usr/include/*: -fpcc-struct-return -mregparm=0
/usr/local/include/memtight-lib/*: -fpack-struct
/home/me/devel/*: -freg-struct-return -mregparm=3
/usr/local/include/experimental/*: -mrtd

And a single source file could include headers from all of these places,
and the function calls and struct accesses would be generated according
to each header's abi specification.

Has anyone given these things a thought, or am I the only one who thinks 
this is an issue? I think in the future, multiple ABIs will be more of a 
reality, and it would be more important to get a single program to
utilize many of them, if at all possible..

Would appreciate some comments,


Lauri Alanko
la@iki.fi

Re: Multiple ABIs for a single compilation unit?

[Martin v. Loewis]

> Has anyone given these things a thought, or am I the only one who thinks 
> this is an issue? I think in the future, multiple ABIs will be more of a 
> reality, and it would be more important to get a single program to
> utilize many of them, if at all possible..

Some of the issues you mention are serious, but I don't like your
proposed solutions.

The header file *is* the place to declare the ABI. This has a long
tradition:
- In C++, external function may get a linkage declaration (extern "C")
  which affects the ABI.
- Microsoft has been putting ABI aspects into prototypes for ages
  (remember: far pointers, stdcall attributes, declspec(dllimport))
- packed structures are declared together with the structure in a number
  of compilers.

I don't think header maintainance would be too terrible: the examples
above show what common strategies for these problems are.

That __attribute__((regparm)) doesn't work for inherited methods is a
clearly a bug; the right solution is to fix the bug, not to avoid the
feature.

You didn't elaborate where exactly your command line patterns are
specified. Whatever the approach is, it seems unmaintainable: When I
install a library, how do I know what magic patterns I have to use?

So, I'd rather prefer to use the existing techniques. If you
absolutely want to have short-cuts, feel free to write a tool that
reads the directory lists and then automatically puts the right
__attribute__s into all function declarations.

Regards,
Martin

Re: Multiple ABIs for a single compilation unit?

[Jamie Lokier]

Lauri Alanko wrote:
> With x86, the -freg-struct-return and -mregparm=x parameters are fun,
> and create (to my untrained eye) shorter and more efficient
> code.

Consider -mrtd too.

I once did some measuring and found -mregparm=1 to be best for C++.

Years ago I found that __attribute__ ((regparm (0), cdecl)) didn't work:
only one of the two attributes was used.  __attribute__ ((cdecl, regparm
(0))) used the other.  I forgot to report the bug.

So there was no way to add attributes system header files such that
`-mregparm=1 -mrtd' could be used while still calling system libraries
with the default ABI.

Question for developers:
Hopefully the two attributes work together now.  Do they?

-- Jamie

Re: Multiple ABIs for a single compilation unit?

[Lauri Alanko]

On Thu, Apr 22, 1999, it occurred to Martin v. Loewis to write:
> > Has anyone given these things a thought, or am I the only one who thinks 
> > this is an issue? I think in the future, multiple ABIs will be more of a 
> > reality, and it would be more important to get a single program to
> > utilize many of them, if at all possible..
> 
> Some of the issues you mention are serious, but I don't like your
> proposed solutions.
> 
> The header file *is* the place to declare the ABI. This has a long
> tradition:
> - In C++, external function may get a linkage declaration (extern "C")
>   which affects the ABI.

This is indeed a good example. But the difference is that you can do
extern "C" {
#include <c_header.h>
}

And thus the header itself need not know about ABI issues.

> - Microsoft has been putting ABI aspects into prototypes for ages
>   (remember: far pointers, stdcall attributes, declspec(dllimport))

Then again, in M$ world there isn't a very wide variety of ABIs, at
least you can't compile the system libraries with arbitrary options
yourself.. And what little I've seen of multi-platform library headers
that support M$ systems, their headers usually do look like a horrible
mess..

> - packed structures are declared together with the structure in a number
>   of compilers.

All right, but these are (to my understanding) mostly used in very
architecture-specific situations, and then worrying about ABIs isn't
your biggest portability problem.

> I don't think header maintainance would be too terrible: the examples
> above show what common strategies for these problems are.

Usually, the common strategy is defining macros for all possible
nonportabilities, and filling your headers with these. It works,
granted, but it's not pretty, nor very fun. And messing with system
headers is not an option under in any situation.

> That __attribute__((regparm)) doesn't work for inherited methods is a
> clearly a bug; the right solution is to fix the bug, not to avoid the
> feature.

That is true, of course. It was just an excuse for my dislike of
function-specific attributes for generic ABI matters.

> You didn't elaborate where exactly your command line patterns are
> specified. Whatever the approach is, it seems unmaintainable: When I
> install a library, how do I know what magic patterns I have to use?

You just have to know, just like you have to know where your library's
headers are located, and what other dependency libraries you need to
link with (unless you have a smart enough linker). What would be the
alternative? That all libraries declare their ABI in header files? That
would in theory be possible, but obviously it's not a common practice,
and one can't change the world.. Since this is a pretty marginal issue
(yet, anyway), it seems much more sensible that the user of the libraries

> So, I'd rather prefer to use the existing techniques. If you
> absolutely want to have short-cuts, feel free to write a tool that
> reads the directory lists and then automatically puts the right
> __attribute__s into all function declarations.

Now that is ugly. It is my opinion that when you need compatibility with
something, you should never write something that translates old things
to new things (since there is an indefinite amount of the old things),
but rather change the (finite number of) users of the things to
understand both old and new things. That is, better to modify the
one compiler than all the code in the world.

Actually, upon thinking a little, I do agree that my original idea
wasn't very practical or clean. But your example of C++'s extern "C"
does have the right idea. You can use it to specify the linkage (ABI, if
you will), of a _block_ of code, and this is a very good thing. C
headers don't declare their linkage in their headers (usually), C++
programs that include them do it. So how about extending the same syntax
for some gcc-specific abi definitions?  Like:

extern "regparm=2" "rtd" {
#include "MyLibHeader.H"
}	

This would to me seem quite sufficient, for then it would be easy for
one to do wrapper headers as one needs. Of course the syntax doesn't
really matter, even something like

__attribute__ ((regparm(2)) {
...
}

would be okay. The important thing is that you can somehow declare any
ABI options for a block of code without touching the actual headers.

The other way would of course be some #pragma directives, but since you
can't make macros out of those, that doesn't seem very attractive.

Even the current __attribute__s for function call conventions are a bit
lacking. To my knowledge, there are no attributes for
-freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
definable on a per-block basis would, I think, help in migrating to
newer name mangling versions etc..

So is there any idea in this? I might even be able to help in
implementing it, although currently (having occasionally glanced at the
source), I have no clue of the compiler's internals...

Regards,


Lauri Alanko
la@iki.fi

Re: Multiple ABIs for a single compilation unit?

[Jamie Lokier]

Lauri Alanko wrote:
> extern "regparm=2" "rtd" {
> #include "MyLibHeader.H"
> }	
> 
> This would to me seem quite sufficient, for then it would be easy for
> one to do wrapper headers as one needs. Of course the syntax doesn't
> really matter, even something like
> 
> __attribute__ ((regparm(2)) {
> ...
> }
> 
> would be okay. The important thing is that you can somehow declare any
> ABI options for a block of code without touching the actual headers.

Hey that was my idea too!  And what a brilliant one it is :-)

[Meaning: I second the suggestion]

It could go in Glibc's __BEGIN_DECLS/__END_DECLS macros.

If you want to use other libraries, I guess you need to know what you're
doing.  Having the linker be able to catch or even resolve ABI
differences would be a great time saver.

-- Jamie

Re: Multiple ABIs for a single compilation unit?

[Martin v. Loewis]

> Even the current __attribute__s for function call conventions are a bit
> lacking. To my knowledge, there are no attributes for
> -freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
> definable on a per-block basis would, I think, help in migrating to
> newer name mangling versions etc..
> 
> So is there any idea in this? I might even be able to help in
> implementing it, although currently (having occasionally glanced at the
> source), I have no clue of the compiler's internals...

If there are no attributes now for that, it is an indication that the
compiler isn't capable of remembering that on a per-declaration basis,
but that these flags are truly global.

So you would need first to identify what features you want to select
on a per-declaration basis, and then modifiy the compiler to support
that. In some cases, it might not be trivial: If you use
-fname-mangling-version on a block, and in the block is a function
with a parameter type for which explicitly a different mangling
version was requested - which one should you use?

Regards,
Martin

Re: Multiple ABIs for a single compilation unit?

[Lauri Alanko]

On Sat, Apr 24, 1999, it occurred to Martin v. Loewis to write:
> > Even the current __attribute__s for function call conventions are a bit
> > lacking. To my knowledge, there are no attributes for
> > -freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
> > definable on a per-block basis would, I think, help in migrating to
> > newer name mangling versions etc..
> 
> If there are no attributes now for that, it is an indication that the
> compiler isn't capable of remembering that on a per-declaration basis,
> but that these flags are truly global.

Well, yes, obviously. As I said, I'm not acquainted with the internals
of egcs, but I'd suspect adding a new piece of information for all
declarations shouldn't require too drastic modifications.

> So you would need first to identify what features you want to select
> on a per-declaration basis,

What I want? Everything, of course. Customizability is always good. An
altogether another question is what features would be worth the
trouble. For example, it wouldn't be _too_ useful to be able affect the
debugging options for each function separately. I think any ABI options
that would ease the simultaneous usage of different libraries of
different ages (and performance/debugging characteristics) would be
useful.

> and then modifiy the compiler to support that. In some cases, it might
> not be trivial: If you use -fname-mangling-version on a block, and in
> the block is a function with a parameter type for which explicitly a
> different mangling version was requested - which one should you use?

Hmm? This is just a policy question. The most obvious is that more
specific always overrides more generic, though some kind of a warning
might be appropriate in this case. Of course, the exact behavior could
be made a compile-time option..


Lauri Alanko
la@iki.fi

Re: Multiple ABIs for a single compilation unit?

[Alexandre Oliva]

On Apr 23, 1999, Lauri Alanko <la@iki.fi> wrote:

> So how about extending the same syntax for some gcc-specific abi
> definitions?  Like:

> extern "regparm=2" "rtd" {
> #include "MyLibHeader.H"
> }	

I'd rather have:

extern "C" __attribute__((regparm(2))) {
...
}

-- 
Alexandre Oliva http://www.dcc.unicamp.br/~oliva IC-Unicamp, Bra[sz]il
{oliva,Alexandre.Oliva}@dcc.unicamp.br  aoliva@{acm.org,computer.org}
oliva@{gnu.org,kaffe.org,{egcs,sourceware}.cygnus.com,samba.org}
*** E-mail about software projects will be forwarded to mailing lists

Re: Multiple ABIs for a single compilation unit?

[Alexandre Oliva]

On Apr 23, 1999, Lauri Alanko <la@iki.fi> wrote:

> So how about extending the same syntax for some gcc-specific abi
> definitions?  Like:

> extern "regparm=2" "rtd" {
> #include "MyLibHeader.H"
> }	

I'd rather have:

extern "C" __attribute__((regparm(2))) {
...
}

-- 
Alexandre Oliva http://www.dcc.unicamp.br/~oliva IC-Unicamp, Bra[sz]il
{oliva,Alexandre.Oliva}@dcc.unicamp.br  aoliva@{acm.org,computer.org}
oliva@{gnu.org,kaffe.org,{egcs,sourceware}.cygnus.com,samba.org}
*** E-mail about software projects will be forwarded to mailing lists

Re: Multiple ABIs for a single compilation unit?

[Martin v. Loewis]

> Even the current __attribute__s for function call conventions are a bit
> lacking. To my knowledge, there are no attributes for
> -freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
> definable on a per-block basis would, I think, help in migrating to
> newer name mangling versions etc..
> 
> So is there any idea in this? I might even be able to help in
> implementing it, although currently (having occasionally glanced at the
> source), I have no clue of the compiler's internals...

If there are no attributes now for that, it is an indication that the
compiler isn't capable of remembering that on a per-declaration basis,
but that these flags are truly global.

So you would need first to identify what features you want to select
on a per-declaration basis, and then modifiy the compiler to support
that. In some cases, it might not be trivial: If you use
-fname-mangling-version on a block, and in the block is a function
with a parameter type for which explicitly a different mangling
version was requested - which one should you use?

Regards,
Martin

Re: Multiple ABIs for a single compilation unit?

[Jamie Lokier]

Lauri Alanko wrote:
> With x86, the -freg-struct-return and -mregparm=x parameters are fun,
> and create (to my untrained eye) shorter and more efficient
> code.

Consider -mrtd too.

I once did some measuring and found -mregparm=1 to be best for C++.

Years ago I found that __attribute__ ((regparm (0), cdecl)) didn't work:
only one of the two attributes was used.  __attribute__ ((cdecl, regparm
(0))) used the other.  I forgot to report the bug.

So there was no way to add attributes system header files such that
`-mregparm=1 -mrtd' could be used while still calling system libraries
with the default ABI.

Question for developers:
Hopefully the two attributes work together now.  Do they?

-- Jamie

Re: Multiple ABIs for a single compilation unit?

[Lauri Alanko]

On Sat, Apr 24, 1999, it occurred to Martin v. Loewis to write:
> > Even the current __attribute__s for function call conventions are a bit
> > lacking. To my knowledge, there are no attributes for
> > -freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
> > definable on a per-block basis would, I think, help in migrating to
> > newer name mangling versions etc..
> 
> If there are no attributes now for that, it is an indication that the
> compiler isn't capable of remembering that on a per-declaration basis,
> but that these flags are truly global.

Well, yes, obviously. As I said, I'm not acquainted with the internals
of egcs, but I'd suspect adding a new piece of information for all
declarations shouldn't require too drastic modifications.

> So you would need first to identify what features you want to select
> on a per-declaration basis,

What I want? Everything, of course. Customizability is always good. An
altogether another question is what features would be worth the
trouble. For example, it wouldn't be _too_ useful to be able affect the
debugging options for each function separately. I think any ABI options
that would ease the simultaneous usage of different libraries of
different ages (and performance/debugging characteristics) would be
useful.

> and then modifiy the compiler to support that. In some cases, it might
> not be trivial: If you use -fname-mangling-version on a block, and in
> the block is a function with a parameter type for which explicitly a
> different mangling version was requested - which one should you use?

Hmm? This is just a policy question. The most obvious is that more
specific always overrides more generic, though some kind of a warning
might be appropriate in this case. Of course, the exact behavior could
be made a compile-time option..


Lauri Alanko
la@iki.fi

Re: Multiple ABIs for a single compilation unit?

[Jamie Lokier]

Lauri Alanko wrote:
> extern "regparm=2" "rtd" {
> #include "MyLibHeader.H"
> }	
> 
> This would to me seem quite sufficient, for then it would be easy for
> one to do wrapper headers as one needs. Of course the syntax doesn't
> really matter, even something like
> 
> __attribute__ ((regparm(2)) {
> ...
> }
> 
> would be okay. The important thing is that you can somehow declare any
> ABI options for a block of code without touching the actual headers.

Hey that was my idea too!  And what a brilliant one it is :-)

[Meaning: I second the suggestion]

It could go in Glibc's __BEGIN_DECLS/__END_DECLS macros.

If you want to use other libraries, I guess you need to know what you're
doing.  Having the linker be able to catch or even resolve ABI
differences would be a great time saver.

-- Jamie

Re: Multiple ABIs for a single compilation unit?

[Lauri Alanko]

On Thu, Apr 22, 1999, it occurred to Martin v. Loewis to write:
> > Has anyone given these things a thought, or am I the only one who thinks 
> > this is an issue? I think in the future, multiple ABIs will be more of a 
> > reality, and it would be more important to get a single program to
> > utilize many of them, if at all possible..
> 
> Some of the issues you mention are serious, but I don't like your
> proposed solutions.
> 
> The header file *is* the place to declare the ABI. This has a long
> tradition:
> - In C++, external function may get a linkage declaration (extern "C")
>   which affects the ABI.

This is indeed a good example. But the difference is that you can do
extern "C" {
#include <c_header.h>
}

And thus the header itself need not know about ABI issues.

> - Microsoft has been putting ABI aspects into prototypes for ages
>   (remember: far pointers, stdcall attributes, declspec(dllimport))

Then again, in M$ world there isn't a very wide variety of ABIs, at
least you can't compile the system libraries with arbitrary options
yourself.. And what little I've seen of multi-platform library headers
that support M$ systems, their headers usually do look like a horrible
mess..

> - packed structures are declared together with the structure in a number
>   of compilers.

All right, but these are (to my understanding) mostly used in very
architecture-specific situations, and then worrying about ABIs isn't
your biggest portability problem.

> I don't think header maintainance would be too terrible: the examples
> above show what common strategies for these problems are.

Usually, the common strategy is defining macros for all possible
nonportabilities, and filling your headers with these. It works,
granted, but it's not pretty, nor very fun. And messing with system
headers is not an option under in any situation.

> That __attribute__((regparm)) doesn't work for inherited methods is a
> clearly a bug; the right solution is to fix the bug, not to avoid the
> feature.

That is true, of course. It was just an excuse for my dislike of
function-specific attributes for generic ABI matters.

> You didn't elaborate where exactly your command line patterns are
> specified. Whatever the approach is, it seems unmaintainable: When I
> install a library, how do I know what magic patterns I have to use?

You just have to know, just like you have to know where your library's
headers are located, and what other dependency libraries you need to
link with (unless you have a smart enough linker). What would be the
alternative? That all libraries declare their ABI in header files? That
would in theory be possible, but obviously it's not a common practice,
and one can't change the world.. Since this is a pretty marginal issue
(yet, anyway), it seems much more sensible that the user of the libraries

> So, I'd rather prefer to use the existing techniques. If you
> absolutely want to have short-cuts, feel free to write a tool that
> reads the directory lists and then automatically puts the right
> __attribute__s into all function declarations.

Now that is ugly. It is my opinion that when you need compatibility with
something, you should never write something that translates old things
to new things (since there is an indefinite amount of the old things),
but rather change the (finite number of) users of the things to
understand both old and new things. That is, better to modify the
one compiler than all the code in the world.

Actually, upon thinking a little, I do agree that my original idea
wasn't very practical or clean. But your example of C++'s extern "C"
does have the right idea. You can use it to specify the linkage (ABI, if
you will), of a _block_ of code, and this is a very good thing. C
headers don't declare their linkage in their headers (usually), C++
programs that include them do it. So how about extending the same syntax
for some gcc-specific abi definitions?  Like:

extern "regparm=2" "rtd" {
#include "MyLibHeader.H"
}	

This would to me seem quite sufficient, for then it would be easy for
one to do wrapper headers as one needs. Of course the syntax doesn't
really matter, even something like

__attribute__ ((regparm(2)) {
...
}

would be okay. The important thing is that you can somehow declare any
ABI options for a block of code without touching the actual headers.

The other way would of course be some #pragma directives, but since you
can't make macros out of those, that doesn't seem very attractive.

Even the current __attribute__s for function call conventions are a bit
lacking. To my knowledge, there are no attributes for
-freg-struct-return, -fname-mangling-version-x, etc. Having all of these 
definable on a per-block basis would, I think, help in migrating to
newer name mangling versions etc..

So is there any idea in this? I might even be able to help in
implementing it, although currently (having occasionally glanced at the
source), I have no clue of the compiler's internals...

Regards,


Lauri Alanko
la@iki.fi

Multiple ABIs for a single compilation unit?

[Lauri Alanko]

Hello.

With x86, the -freg-struct-return and -mregparm=x parameters are fun,
and create (to my untrained eye) shorter and more efficient
code. However, these parameters aren't used in the standard ABIs for
Linux, at least not in Debian or Redhat. So one cannot use them in
practice, unless one is to recompile all system libraries.

So what I'd like is a way to compile programs so that _some_ external
functions are assumed to use one (the "conventional") ABI, while other
stuff, basically what I've compiled, would use -freg-struct-return and
-mregparm, or other exotic options. Currently there seems to be no way
to do this.

All right, there is the __attribute__ ((regparm (x))), as well as cdecl
and stdcall, but that is, IMO, the wrong way to do it. Firstly, a header
file should only describe the API, not the ABI. (Or otherwise
maintaining headers will be horrible). Second, the regparm attribute
doesn't work too well (it isn't inherited by virtual functions in C++,
for example, and there's no warning of it either!).

In practice, I think the way to do it would be to support some sort of
list of directories and headers with their ABI specifications, and then
the compiler would use the preprocessor filename information to decide
what ABI a function is supposed to have. So I could say something like:

/usr/include/*: -fpcc-struct-return -mregparm=0
/usr/local/include/memtight-lib/*: -fpack-struct
/home/me/devel/*: -freg-struct-return -mregparm=3
/usr/local/include/experimental/*: -mrtd

And a single source file could include headers from all of these places,
and the function calls and struct accesses would be generated according
to each header's abi specification.

Has anyone given these things a thought, or am I the only one who thinks 
this is an issue? I think in the future, multiple ABIs will be more of a 
reality, and it would be more important to get a single program to
utilize many of them, if at all possible..

Would appreciate some comments,


Lauri Alanko
la@iki.fi

Re: Multiple ABIs for a single compilation unit?

[Martin v. Loewis]

> Has anyone given these things a thought, or am I the only one who thinks 
> this is an issue? I think in the future, multiple ABIs will be more of a 
> reality, and it would be more important to get a single program to
> utilize many of them, if at all possible..

Some of the issues you mention are serious, but I don't like your
proposed solutions.

The header file *is* the place to declare the ABI. This has a long
tradition:
- In C++, external function may get a linkage declaration (extern "C")
  which affects the ABI.
- Microsoft has been putting ABI aspects into prototypes for ages
  (remember: far pointers, stdcall attributes, declspec(dllimport))
- packed structures are declared together with the structure in a number
  of compilers.

I don't think header maintainance would be too terrible: the examples
above show what common strategies for these problems are.

That __attribute__((regparm)) doesn't work for inherited methods is a
clearly a bug; the right solution is to fix the bug, not to avoid the
feature.

You didn't elaborate where exactly your command line patterns are
specified. Whatever the approach is, it seems unmaintainable: When I
install a library, how do I know what magic patterns I have to use?

So, I'd rather prefer to use the existing techniques. If you
absolutely want to have short-cuts, feel free to write a tool that
reads the directory lists and then automatically puts the right
__attribute__s into all function declarations.

Regards,
Martin