__REDIRECT macro

__REDIRECT macro

[Martin Uecker]

Hi all,

I have a question about the __REDIRECT macro which was
added by Zack Weinberg in 1998 for large file support
and that makes use of __asm.  It comes with a comment.

+/* __asm__ ("xyz") is used throughout the headers to rename functions
+   at the assembly language level.  This is wrapped by the __REDIRECT
+   macro, in order to support compilers that can do this some other
+   way.  When compilers don't support asm-names at all, we have to do
+   preprocessor tricks instead (which don't have exactly the right
+   semantics, but it's the best we can do).

https://sourceware.org/git/?p=glibc.git;a=commit;h=01cad722667c7b25535b2d248598f3d535e7caa9

If __asm is not available there is a fall back to a simple #define
(does preprocessor tricks refer to this?).  

My main question is: What semantics are not exactly right with a
#define?

From the C standard' side, we have

"The use of #undef to remove any macro definition will also ensure that
an actual function is referred to."

which would not work for a macro. But apart from this, are there any
other semantic semantic differences this refers to?

We also have

"Provided that a library function can be declared without reference to
any type defined in a header, it is also permissible to declare the function
and use it without including its associated header."

which would seem to work neither with __REDIRECT nor with a macro.
Is it correct that for redirected functions this does not work?


Martin

Re: __REDIRECT macro

[Andrew Pinski]

[-- Attachment #1: Type: text/plain, Size: 1903 bytes --]

On Sun, Oct 27, 2024, 11:39 PM Martin Uecker <ma.uecker@gmail.com> wrote:

>
>
> Hi all,
>
> I have a question about the __REDIRECT macro which was
> added by Zack Weinberg in 1998 for large file support
> and that makes use of __asm.  It comes with a comment.
>
> +/* __asm__ ("xyz") is used throughout the headers to rename functions
> +   at the assembly language level.  This is wrapped by the __REDIRECT
> +   macro, in order to support compilers that can do this some other
> +   way.  When compilers don't support asm-names at all, we have to do
> +   preprocessor tricks instead (which don't have exactly the right
> +   semantics, but it's the best we can do).
>
>
> https://sourceware.org/git/?p=glibc.git;a=commit;h=01cad722667c7b25535b2d248598f3d535e7caa9
>
> If __asm is not available there is a fall back to a simple #define
> (does preprocessor tricks refer to this?).
>
> My main question is: What semantics are not exactly right with a
> #define?
>


The gcc builtin for the function does not change when using the
preprocessor tricks.

The second is taking the function address does not always work. Especially
if you do &(function).

Thanks,
Andrew



> From the C standard' side, we have
>
> "The use of #undef to remove any macro definition will also ensure that
> an actual function is referred to."
>
> which would not work for a macro. But apart from this, are there any
> other semantic semantic differences this refers to?
>
> We also have
>
> "Provided that a library function can be declared without reference to
> any type defined in a header, it is also permissible to declare the
> function
> and use it without including its associated header."
>
> which would seem to work neither with __REDIRECT nor with a macro.
> Is it correct that for redirected functions this does not work?
>
>
> Martin
>
>
>
>
>

Re: __REDIRECT macro

[Martin Uecker]

Am Sonntag, dem 27.10.2024 um 23:44 -0700 schrieb Andrew Pinski:
> 
> 
> On Sun, Oct 27, 2024, 11:39 PM Martin Uecker <ma.uecker@gmail.com> wrote:
> > 
> > 
> > Hi all,
> > 
> > I have a question about the __REDIRECT macro which was
> > added by Zack Weinberg in 1998 for large file support
> > and that makes use of __asm.  It comes with a comment.
> > 
> > +/* __asm__ ("xyz") is used throughout the headers to rename functions
> > +   at the assembly language level.  This is wrapped by the __REDIRECT
> > +   macro, in order to support compilers that can do this some other
> > +   way.  When compilers don't support asm-names at all, we have to do
> > +   preprocessor tricks instead (which don't have exactly the right
> > +   semantics, but it's the best we can do).
> > 
> > https://sourceware.org/git/?p=glibc.git;a=commit;h=01cad722667c7b25535b2d248598f3d535e7caa9
> > 
> > If __asm is not available there is a fall back to a simple #define
> > (does preprocessor tricks refer to this?).  
> > 
> > My main question is: What semantics are not exactly right with a
> > #define?
> 
> 
> The gcc builtin for the function does not change when using the preprocessor tricks.

Thanks! This is the information I am looking for. 

When is this desired?  Assuming something like

time
time64

should it get the same builtin?

> The second is taking the function address does not always work. Especially if you do &(function).

That works with object-like macros.

void bar(void);
#define foo bar
&foo

Martin

> 
> Thanks,
> Andrew
> 
> 
> > 
> > From the C standard' side, we have
> > 
> > "The use of #undef to remove any macro definition will also ensure that
> > an actual function is referred to."
> > 
> > which would not work for a macro. But apart from this, are there any
> > other semantic semantic differences this refers to?
> > 
> > We also have
> > 
> > "Provided that a library function can be declared without reference to
> > any type defined in a header, it is also permissible to declare the function
> > and use it without including its associated header."
> > 
> > which would seem to work neither with __REDIRECT nor with a macro.
> > Is it correct that for redirected functions this does not work?
> > 
> > 
> > Martin
> > 
> > 
> > 
> > 

Re: __REDIRECT macro

[Florian Weimer]

* Martin Uecker:

> Am Sonntag, dem 27.10.2024 um 23:44 -0700 schrieb Andrew Pinski:
>> The gcc builtin for the function does not change when using the
>> preprocessor tricks.
>
> Thanks! This is the information I am looking for. 
>
> When is this desired?  Assuming something like
>
> time
> time64
>
> should it get the same builtin?

I don't think the 64-bit file system and time interfaces benefit much
from this aspect.  It's more about ensuring that math functions call the
implementations for the right long double type.  For that, GCC may
actually have to synthesize a call.  Here's an example:

#include <math.h>

void
f (long double x, long double *y, long double *z)
{
  *y = sinl (x);
  *z = cosl (x);
}

On powerpc64le, the resulting sincos call needs to use __sincosieee128
with -mabi=ieeelongdouble, sincosl with -mabi=ibmlongdouble.  Some
targets also support -mlong-double-64, and then the call should go to
sincos.

This is also allegedly one of the cases where it's supported to declare
those functions yourself, but evidently this can't be true because the
target symbol does not follow the compilation-time ABI.  That rule
mostly works for off_t/time_t variance because header-defined types tend
to show up in their prototypes.  Not so much for functions in <math.h>.

(Redirection of built-ins historically hasn't been implemented by Clang,
I think.  Not sure if this has changed.)

The macro approach has the problem that it changes the mangling of C++
function names, particular member function names.  This is particularly
problematic if the libc header containing the macro is not consistenly
included everywhere the C++ function is declared, so you get symbol
inconsistencies within a single program.  I don't know if that's what
prompted Zack to make his 1998 change, you would have to ask him that.

Thanks,
Florian

Re: __REDIRECT macro

[Martin Uecker]

Am Montag, dem 28.10.2024 um 08:33 +0100 schrieb Florian Weimer:
> * Martin Uecker:
> 
> > Am Sonntag, dem 27.10.2024 um 23:44 -0700 schrieb Andrew Pinski:
> > > The gcc builtin for the function does not change when using the
> > > preprocessor tricks.
> > 
> > Thanks! This is the information I am looking for. 
> > 
> > When is this desired?  Assuming something like
> > 
> > time
> > time64
> > 
> > should it get the same builtin?
> 
> I don't think the 64-bit file system and time interfaces benefit much
> from this aspect.  It's more about ensuring that math functions call the
> implementations for the right long double type.  For that, GCC may
> actually have to synthesize a call.  Here's an example:
> 
> #include <math.h>
> 
> void
> f (long double x, long double *y, long double *z)
> {
>   *y = sinl (x);
>   *z = cosl (x);
> }
> 
> On powerpc64le, the resulting sincos call needs to use __sincosieee128
> with -mabi=ieeelongdouble, sincosl with -mabi=ibmlongdouble.  Some
> targets also support -mlong-double-64, and then the call should go to
> sincos.
> 
> This is also allegedly one of the cases where it's supported to declare
> those functions yourself, but evidently this can't be true because the
> target symbol does not follow the compilation-time ABI.  That rule
> mostly works for off_t/time_t variance because header-defined types tend
> to show up in their prototypes.  Not so much for functions in <math.h>.

So for "sinl" I can not declare it myself because it might be mapped
to something else? 

One option which was discussed is to strike this requirement from ISO C
and always require that a standard header needs to be included.  But
in general it also seems useful that there is some guaranteed
correspondence between source-level names and names relevant for linkage. 

> 
> (Redirection of built-ins historically hasn't been implemented by Clang,
> I think.  Not sure if this has changed.)
> 
> The macro approach has the problem that it changes the mangling of C++
> function names, particular member function names.  This is particularly
> problematic if the libc header containing the macro is not consistenly
> included everywhere the C++ function is declared, so you get symbol
> inconsistencies within a single program. 
> 
Is the issue that the macro would change the name but a independently
declared function would then have a different name?  But this would be
the same in C, o.e. the issue above. I do not quite understand how this
relates to name mangling in C++. Can you explain this a bit more? 


> I don't know if that's what
> prompted Zack to make his 1998 change, you would have to ask him that.

Now CC-ed.

The background for my inquiry is a proposal ("Transparent Aliases" *)
for WG14 that would allow

void bar(void);
_Alias foo = bar;

which - according to my understanding - would correspond to

void foo(void) __asm("bar");


This seems to be sufficient for __REDIRECT, but would not support
other use cases of __asm (e.g. assembler names that are not
C identifiers or mapping to functions not declared before in the same
TU etc.)

So I am a bit skeptical because this feature, because this is essentially
what you also already get with object-like macros I would be more
in favor to either a have feature which supports more of __asm or 
more comprehensively improves versioning or name handling in C. But
if this feature is exactly what is needed that might convince me.

Martin


*) I would link to the proposal, but the server is down atm.

Re: __REDIRECT macro

[Florian Weimer]

* Martin Uecker:

> So for "sinl" I can not declare it myself because it might be mapped
> to something else?

Correct.  For the historic case, you could link with -lnldbl, but we
didn't provide something like that for later transitions to binary128
long double.

> One option which was discussed is to strike this requirement from ISO C
> and always require that a standard header needs to be included.  But
> in general it also seems useful that there is some guaranteed
> correspondence between source-level names and names relevant for linkage. 

That doesn't really exist because there will always be cases where some
existing function is standardized with differnet semantics, and then we
need redirects.

>> The macro approach has the problem that it changes the mangling of C++
>> function names, particular member function names.  This is particularly
>> problematic if the libc header containing the macro is not consistenly
>> included everywhere the C++ function is declared, so you get symbol
>> inconsistencies within a single program. 
>> 
> Is the issue that the macro would change the name but a independently
> declared function would then have a different name?  But this would be
> the same in C, o.e. the issue above. I do not quite understand how this
> relates to name mangling in C++. Can you explain this a bit more?

With C, you cannot define multiple different functions called “open”
with external linkage.  Instead, libraries tend to use prefixes to
separate the namespace, so it's unlikely that an “open” identifier
occurs on its own that gets macro-expanded.  If it happens in a struct
field or a parameter name, it does not matter for linkage purposes.

With C++, “open” could be a member function, or a function nested in a
namespace, or even an overload in the global namespace.  Here, the
“open” token actually appears in the source, and is subject to macro
expansion.

I hope this explains the difference in impact.

> The background for my inquiry is a proposal ("Transparent Aliases" *)
> for WG14 that would allow
>
> void bar(void);
> _Alias foo = bar;
>
> which - according to my understanding - would correspond to
>
> void foo(void) __asm("bar");

It looks more like GCC's alias attribute because there is a requirement
for matching types (which curiously aren't declared in the proposed
syntax).

I'm not sure if __asm is a good target for standardization because it
introduces a new source of undefined behavior.  (But aliases may as
well.)

Thanks,
Florian

Re: __REDIRECT macro

[Martin Uecker]

Am Montag, dem 28.10.2024 um 10:35 +0100 schrieb Florian Weimer:
> * Martin Uecker:
> 
...

> > > The macro approach has the problem that it changes the mangling of C++
> > > function names, particular member function names.  This is particularly
> > > problematic if the libc header containing the macro is not consistenly
> > > included everywhere the C++ function is declared, so you get symbol
> > > inconsistencies within a single program. 
> > > 
> > Is the issue that the macro would change the name but a independently
> > declared function would then have a different name?  But this would be
> > the same in C, o.e. the issue above. I do not quite understand how this
> > relates to name mangling in C++. Can you explain this a bit more?
> 
> With C, you cannot define multiple different functions called “open”
> with external linkage.  Instead, libraries tend to use prefixes to
> separate the namespace, so it's unlikely that an “open” identifier
> occurs on its own that gets macro-expanded.  If it happens in a struct
> field or a parameter name, it does not matter for linkage purposes.
> 
> With C++, “open” could be a member function, or a function nested in a
> namespace, or even an overload in the global namespace.  Here, the
> “open” token actually appears in the source, and is subject to macro
> expansion.
> 
> I hope this explains the difference in impact.

Yes, thanks!

> 
> > The background for my inquiry is a proposal ("Transparent Aliases" *)
> > for WG14 that would allow
> > 
> > void bar(void);
> > _Alias foo = bar;
> > 
> > which - according to my understanding - would correspond to
> > 
> > void foo(void) __asm("bar");
> 
> It looks more like GCC's alias attribute because there is a requirement
> for matching types (which curiously aren't declared in the proposed
> syntax).

GCC's alias seems to require a definition in the same TU, so I think
this is different.

> I'm not sure if __asm is a good target for standardization because it
> introduces a new source of undefined behavior.  (But aliases may as
> well.)


Martin

> 

Re: __REDIRECT macro

[Zack Weinberg]

On Mon, Oct 28, 2024, at 5:19 AM, Martin Uecker wrote:
> Am Montag, dem 28.10.2024 um 08:33 +0100 schrieb Florian Weimer:
>> I don't know if that's what prompted Zack to make his 1998 change,
>> you would have to ask him that.
>
> Now CC-ed.

It has been a very long time but I believe the primary motivation was
C (not C++) standard compliance.  The C standard permits any standard
library function to be defined as a *function-like* macro, but not as an
object-like macro (7.1.4p1).  But a function-like macro cannot be used
to intercept all uses of the function name, so some other mechanism was
needed.  There may have been other concerns at the time, this is just
the one I remember.

> The background for my inquiry is a proposal ("Transparent Aliases" *)
> for WG14 that would allow
>
> void bar(void); _Alias foo = bar;
>
> which - according to my understanding - would correspond to
>
> void foo(void) __asm("bar");
>
> This seems to be sufficient for __REDIRECT, but would not support
> other use cases of __asm (e.g. assembler names that are not C
> identifiers or mapping to functions not declared before in the same
> TU etc.)

Look carefully at how glibc does symbol versioning; IIRC that does
involve use of __asm with assembler names that are not C identifiers.
Also, there may be cases where glibc uses __REDIRECT to map onto a
function with a different prototype, although I wouldn't blame the C
committee for refusing to support that.

zw

Re: __REDIRECT macro

[Joseph Myers]

On Mon, 28 Oct 2024, Martin Uecker wrote:

> One option which was discussed is to strike this requirement from ISO C
> and always require that a standard header needs to be included.  But

I think that would make sense; the requirement is of questionable value 
and problematic in practice.  (In principle stdc-predef.h could use 
#pragma redefine_extname for redirection, but wouldn't have access to any 
macros from features.h when doing so, and replicating the feature test 
logic in multiple places would be complicated.)

Note that in C23 typeof means that more types are now accessible without 
including a header (e.g. size_t and ptrdiff_t via typeof applied to 
appropriate expressions), so the rule about being able to declare without 
using any type from a header ends up covering more functions.

In the <math.h> case, GCC needs to know remappings such as __*ieee128 
anyway to support use of the functions from Fortran code (or other non-C 
languages) that can't include C headers at all.

I think that compilers without asm redirection support for __REDIRECT 
probably don't work well with glibc headers in practice even if the 
headers theoretically attempt to support them.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: __REDIRECT macro

[Zack Weinberg]

On Mon, Oct 28, 2024, at 6:15 PM, Joseph Myers wrote:
> On Mon, 28 Oct 2024, Martin Uecker wrote:
>
>> One option which was discussed is to strike this requirement from ISO C
>> and always require that a standard header needs to be included.  But
>
> I think that would make sense; the requirement is of questionable value 
> and problematic in practice.

I would support this change as well.  It's already the case in C++ and it
would be consistent with the general trend of de-supporting pre-standardization
idioms.

zw

Re: __REDIRECT macro

[Martin Uecker]

Am Mittwoch, dem 30.10.2024 um 10:57 -0400 schrieb Zack Weinberg:
> On Mon, Oct 28, 2024, at 6:15 PM, Joseph Myers wrote:
> > On Mon, 28 Oct 2024, Martin Uecker wrote:
> > 
> > > One option which was discussed is to strike this requirement from ISO C
> > > and always require that a standard header needs to be included.  But
> > 
> > I think that would make sense; the requirement is of questionable value 
> > and problematic in practice.
> 
> I would support this change as well.  It's already the case in C++ and it
> would be consistent with the general trend of de-supporting pre-standardization
> idioms.

On the other hand, we lose the direct correspondance of C library
functions to symbols. I worry a bit about language interoperability.

So this may be a stupid question, but wouldn't a linker mechanism
for selecting symbols based on a specific ABI version not be a more
robust solution?   Somehow it seems strange that this is done in
the preprocessor.

Martin

Re: __REDIRECT macro

[Zack Weinberg]

On Wed, Oct 30, 2024, at 4:28 PM, Martin Uecker wrote:
> On the other hand, we lose the direct correspondance of C library
> functions to symbols. I worry a bit about language interoperability.

This is a real concern ... but the __REDIRECT mechanism has been around,
well, since 1998, and other C libraries do similar things for similar
reasons.

An idea I had in about 2002 and never did anything with, but I kinda
wish I had, was to import the `extern "language" { ... }` syntax to C.
In particular, writing extern function declarations inside an
`extern "C++"` block would cause them to receive C++ mangled names.
(There would be no language-level semantic changes; it would still be
an error to try to overload such functions, for instance.)  The original
motivation for this idea was that it would let library authors opt in to
link-time detection of mismatched declarations.  However, what glibc
currently does with constructs like

#ifdef __USE_FILE_OFFSET_64
extern ssize_t __REDIRECT (pread, (int __fd, void *__buf, size_t __nbytes,
				   __off64_t __offset),
			   pread64);
#else
extern ssize_t pread (int __fd, void *__buf, size_t __nbytes,
		      __off_t __offset);
#endif

could instead be handled with

extern "C++" {
#ifdef __USE_FILE_OFFSET_64
extern ssize_t pread (int __fd, void *__buf, size_t __nbytes,
                      __off64_t __offset);
#else
extern ssize_t pread (int __fd, void *__buf, size_t __nbytes,
		      __off_t __offset);
#endif
}

Programs compiled with -D_FILE_OFFSET_BITS=64 would call _Z5preadiPvml
and those compiled with -D_FILE_OFFSET_BITS=32 would call _Z5preadiPvmi.
And a not-C language would have a principled way to figure out how to
call the variant it wanted ("just" mangle the function's arguments the
way C++ would do it), instead of needing an ad-hoc list.

> So this may be a stupid question, but wouldn't a linker mechanism
> for selecting symbols based on a specific ABI version not be a more
> robust solution?   Somehow it seems strange that this is done in
> the preprocessor.

As you can see in the above example, use of __REDIRECT by glibc's
headers is usually keyed off feature selection macros, not ABI versions.

zw