public inbox for libffi-discuss@sourceware.org
 help / color / mirror / Atom feed
* wide function pointer type
@ 2021-10-10 11:32 Martin Uecker
  2021-10-17 23:35 ` Anthony Green
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-10 11:32 UTC (permalink / raw)
  To: libffi-discuss



Hi all,

I will propose a wide function pointer type (actually
a wide function type) to WG14 for C23 as a common 
type for callbacks, closures, which now require an
additional void pointer argument in C APIs. This
is intended to be compatible with ABIs with now
use a static chain register.

An early draft can be found here:

http://wwwuser.gwdg.de/~muecker1/wide_v4.pdf


I thought you might be interested and I would love
to hear your feedback.

Martin




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 11:32 wide function pointer type Martin Uecker
@ 2021-10-17 23:35 ` Anthony Green
  2021-10-18  5:33   ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Anthony Green @ 2021-10-17 23:35 UTC (permalink / raw)
  To: Martin Uecker; +Cc: libffi-discuss

Hi Martin,

  I haven't read the whole proposal yet, but I'll try to this week.  I
did, however, read the first paragraph, and it says:  "Trying to use
regular function pointers for callbacks with data requires run-time
code generation (e.g. nested functions in GCC or XL C, and closures in
libffi [1]) but this is complex, inefficient, and problematic from a
security point of view."

  This is no longer true for libffi on the important Linux platforms.
We use static code templates that are placed in memory at a fixed,
predetermined distance from the data they reference.  There is no
runtime code generation.  Well, to be fair, perhaps it is still
complex, but it's far more secure than before.

  Thanks for sharing.  I'm not a language expert, but I know others here are.

AG

On Sun, Oct 10, 2021 at 7:32 AM Martin Uecker via Libffi-discuss
<libffi-discuss@sourceware.org> wrote:
>
>
>
> Hi all,
>
> I will propose a wide function pointer type (actually
> a wide function type) to WG14 for C23 as a common
> type for callbacks, closures, which now require an
> additional void pointer argument in C APIs. This
> is intended to be compatible with ABIs with now
> use a static chain register.
>
> An early draft can be found here:
>
> http://wwwuser.gwdg.de/~muecker1/wide_v4.pdf
>
>
> I thought you might be interested and I would love
> to hear your feedback.
>
> Martin
>
>
>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-17 23:35 ` Anthony Green
@ 2021-10-18  5:33   ` Martin Uecker
  2021-10-18  5:58     ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-18  5:33 UTC (permalink / raw)
  To: Anthony Green; +Cc: libffi-discuss


Hi Anthony,

Am Sonntag, den 17.10.2021, 19:35 -0400 schrieb Anthony Green:
> Hi Martin,
> 
>   I haven't read the whole proposal yet, but I'll try to this week.  I
> did, however, read the first paragraph, and it says:  "Trying to use
> regular function pointers for callbacks with data requires run-time
> code generation (e.g. nested functions in GCC or XL C, and closures in
> libffi [1]) but this is complex, inefficient, and problematic from a
> security point of view."
> 
>   This is no longer true for libffi on the important Linux platforms.
> We use static code templates that are placed in memory at a fixed,
> predetermined distance from the data they reference.  There is no
> runtime code generation.  Well, to be fair, perhaps it is still
> complex, but it's far more secure than before.
> 
>   Thanks for sharing.  I'm not a language expert, but I know others here are.

Thank you for your comments! I had to submit this to WG14 to meet
the deadline for C23. So the final version (so far) is here:

 http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2787.pdf

Of course, it is still possible to modify the proposal later.

It is great to hear that libffi can create trampolines
without run-time code generation! We previously discussed
such an approach also for GCC, but nothing was implemented.


Martin


> AG
> 
> On Sun, Oct 10, 2021 at 7:32 AM Martin Uecker via Libffi-discuss
> <libffi-discuss@sourceware.org> wrote:
> > 
> > 
> > Hi all,
> > 
> > I will propose a wide function pointer type (actually
> > a wide function type) to WG14 for C23 as a common
> > type for callbacks, closures, which now require an
> > additional void pointer argument in C APIs. This
> > is intended to be compatible with ABIs with now
> > use a static chain register.
> > 
> > An early draft can be found here:
> > 
> > http://wwwuser.gwdg.de/~muecker1/wide_v4.pdf
> > 
> > 
> > I thought you might be interested and I would love
> > to hear your feedback.
> > 
> > Martin
> > 
> > 
> > 


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-18  5:33   ` Martin Uecker
@ 2021-10-18  5:58     ` Martin Uecker
  2021-10-18  7:36       ` Florian Weimer
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-18  5:58 UTC (permalink / raw)
  To: Anthony Green; +Cc: libffi-discuss


I would be interested to learn about ABI issues
I might not be aware of. 

In particular, I would be good to know whether
implementing a perfectly forwarding stub for
a variadic functions that loads the static
is possible on all architectures. I assume so,
but I am not entirely sure.

If not, we might need to restrict conversion for
regular functions pointers to wide pointers to only
allow functions without variadic args, or define
a new ABI for wide pointers, but this then
partially defeats the point (on the affected
architecture).

Martin


Am Montag, den 18.10.2021, 07:33 +0200 schrieb Martin Uecker:
> Hi Anthony,
> 
> Am Sonntag, den 17.10.2021, 19:35 -0400 schrieb Anthony Green:
> > Hi Martin,
> > 
> >   I haven't read the whole proposal yet, but I'll try to this week.  I
> > did, however, read the first paragraph, and it says:  "Trying to use
> > regular function pointers for callbacks with data requires run-time
> > code generation (e.g. nested functions in GCC or XL C, and closures in
> > libffi [1]) but this is complex, inefficient, and problematic from a
> > security point of view."
> > 
> >   This is no longer true for libffi on the important Linux platforms.
> > We use static code templates that are placed in memory at a fixed,
> > predetermined distance from the data they reference.  There is no
> > runtime code generation.  Well, to be fair, perhaps it is still
> > complex, but it's far more secure than before.
> > 
> >   Thanks for sharing.  I'm not a language expert, but I know others here are.
> 
> Thank you for your comments! I had to submit this to WG14 to meet
> the deadline for C23. So the final version (so far) is here:
> 
>  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2787.pdf
> 
> Of course, it is still possible to modify the proposal later.
> 
> It is great to hear that libffi can create trampolines
> without run-time code generation! We previously discussed
> such an approach also for GCC, but nothing was implemented.
> 
> 
> Martin
> 
> 
> > AG
> > 
> > On Sun, Oct 10, 2021 at 7:32 AM Martin Uecker via Libffi-discuss
> > <libffi-discuss@sourceware.org> wrote:
> > > 
> > > Hi all,
> > > 
> > > I will propose a wide function pointer type (actually
> > > a wide function type) to WG14 for C23 as a common
> > > type for callbacks, closures, which now require an
> > > additional void pointer argument in C APIs. This
> > > is intended to be compatible with ABIs with now
> > > use a static chain register.
> > > 
> > > An early draft can be found here:
> > > 
> > > http://wwwuser.gwdg.de/~muecker1/wide_v4.pdf
> > > 
> > > 
> > > I thought you might be interested and I would love
> > > to hear your feedback.
> > > 
> > > Martin
> > > 
> > > 
> > > 


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-18  5:58     ` Martin Uecker
@ 2021-10-18  7:36       ` Florian Weimer
  2021-10-18  7:56         ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Florian Weimer @ 2021-10-18  7:36 UTC (permalink / raw)
  To: Martin Uecker via Libffi-discuss

* Martin Uecker via Libffi-discuss:

> In particular, I would be good to know whether
> implementing a perfectly forwarding stub for
> a variadic functions that loads the static
> is possible on all architectures. I assume so,
> but I am not entirely sure.

Are you asking if it is possible to consume an argument in a variadic
wrapper and forward the remaining arguments to a variadic function?  The
answer to that is no.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-18  7:36       ` Florian Weimer
@ 2021-10-18  7:56         ` Martin Uecker
  2021-10-19  9:22           ` Florian Weimer
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-18  7:56 UTC (permalink / raw)
  To: Florian Weimer, Martin Uecker via Libffi-discuss

Am Montag, den 18.10.2021, 09:36 +0200 schrieb Florian Weimer:
> * Martin Uecker via Libffi-discuss:
> 
> > In particular, I would be good to know whether
> > implementing a perfectly forwarding stub for
> > a variadic functions that loads the static
> > is possible on all architectures. I assume so,
> > but I am not entirely sure.
> 
> Are you asking if it is possible to consume an argument in a variadic
> wrapper and forward the remaining arguments to a variadic function?  The
> answer to that is no.

Yes, I assumed this does not work. The question 
is more whether architectures exist where the
existing ABI states that the static chain has to be
passed as a first argument (which would then need to be
removed when calling a regular function).

The implementation most architectures
would use is that 1) the static chain is loaded from
the wide pointer 2) stored in the designated register
or stack slot and then 3) a call is performed
using the same calling convention as used for
regular functions.

If a regular function is converted to a wide pointer,
then the static chain would simply be NULL which
should not affect the function.


Martin




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-18  7:56         ` Martin Uecker
@ 2021-10-19  9:22           ` Florian Weimer
  2021-10-19  9:43             ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Florian Weimer @ 2021-10-19  9:22 UTC (permalink / raw)
  To: Martin Uecker; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

* Martin Uecker:

> Am Montag, den 18.10.2021, 09:36 +0200 schrieb Florian Weimer:
>> * Martin Uecker via Libffi-discuss:
>> 
>> > In particular, I would be good to know whether
>> > implementing a perfectly forwarding stub for
>> > a variadic functions that loads the static
>> > is possible on all architectures. I assume so,
>> > but I am not entirely sure.
>> 
>> Are you asking if it is possible to consume an argument in a variadic
>> wrapper and forward the remaining arguments to a variadic function?  The
>> answer to that is no.
>
> Yes, I assumed this does not work. The question 
> is more whether architectures exist where the
> existing ABI states that the static chain has to be
> passed as a first argument (which would then need to be
> removed when calling a regular function).
>
> The implementation most architectures
> would use is that 1) the static chain is loaded from
> the wide pointer 2) stored in the designated register
> or stack slot and then 3) a call is performed
> using the same calling convention as used for
> regular functions.

Many ABIs do not actually specify a register for the chain pointer.  For
GCC's nested function extension, there are no ABI concerns because the
nested function definition and the thunk generation are always built
from the same translation unit.  So GCC can just pick a custom calling
convention, similar to what it does for other local, non-escaping
functions.

For obvious reasons, the proposal does not achieve interoperability with
std::function from C++, so there is no ABI concern, either.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-19  9:22           ` Florian Weimer
@ 2021-10-19  9:43             ` Martin Uecker
  2021-10-19 10:15               ` Florian Weimer
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-19  9:43 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

Am Dienstag, den 19.10.2021, 11:22 +0200 schrieb Florian Weimer:
> * Martin Uecker:
> 
> > Am Montag, den 18.10.2021, 09:36 +0200 schrieb Florian Weimer:
> > > * Martin Uecker via Libffi-discuss:
> > > 
> > > > In particular, I would be good to know whether
> > > > implementing a perfectly forwarding stub for
> > > > a variadic functions that loads the static
> > > > is possible on all architectures. I assume so,
> > > > but I am not entirely sure.
> > > 
> > > Are you asking if it is possible to consume an argument in a variadic
> > > wrapper and forward the remaining arguments to a variadic function?  The
> > > answer to that is no.
> > 
> > Yes, I assumed this does not work. The question 
> > is more whether architectures exist where the
> > existing ABI states that the static chain has to be
> > passed as a first argument (which would then need to be
> > removed when calling a regular function).
> > 
> > The implementation most architectures
> > would use is that 1) the static chain is loaded from
> > the wide pointer 2) stored in the designated register
> > or stack slot and then 3) a call is performed
> > using the same calling convention as used for
> > regular functions.
> 
> Many ABIs do not actually specify a register for the chain pointer.

Ok. The ones I looked at do, but I looked only at a few.

>   For
> GCC's nested function extension, there are no ABI concerns because the
> nested function definition and the thunk generation are always built
> from the same translation unit.  So GCC can just pick a custom calling
> convention, similar to what it does for other local, non-escaping
> functions.

Yes, but there are many other languages that make use of the
static chain.  So it would be good to have a standardized ABI
for this.

> For obvious reasons, the proposal does not achieve interoperability with
> std::function from C++, so there is no ABI concern, either.

Not directly, but you could automatically create a wide
pointer that points to a compile-time stub that calls 
a std::function.

Martin




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-19  9:43             ` Martin Uecker
@ 2021-10-19 10:15               ` Florian Weimer
  2021-10-19 12:13                 ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Florian Weimer @ 2021-10-19 10:15 UTC (permalink / raw)
  To: Martin Uecker; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

* Martin Uecker:

>> For GCC's nested function extension, there are no ABI concerns
>> because the nested function definition and the thunk generation are
>> always built from the same translation unit.  So GCC can just pick a
>> custom calling convention, similar to what it does for other local,
>> non-escaping functions.
>
> Yes, but there are many other languages that make use of the
> static chain.  So it would be good to have a standardized ABI
> for this.

What's the exact use case for this?  These other languages do not
usually have stable ABIs that inter-operate across different
implementations.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-19 10:15               ` Florian Weimer
@ 2021-10-19 12:13                 ` Martin Uecker
  2021-10-20  8:24                   ` Kaz Kylheku (libffi)
  2021-10-20  9:10                   ` Florian Weimer
  0 siblings, 2 replies; 28+ messages in thread
From: Martin Uecker @ 2021-10-19 12:13 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

Am Dienstag, den 19.10.2021, 12:15 +0200 schrieb Florian Weimer:
> * Martin Uecker:
> 
> > > For GCC's nested function extension, there are no ABI concerns
> > > because the nested function definition and the thunk generation are
> > > always built from the same translation unit.  So GCC can just pick a
> > > custom calling convention, similar to what it does for other local,
> > > non-escaping functions.
> > 
> > Yes, but there are many other languages that make use of the
> > static chain.  So it would be good to have a standardized ABI
> > for this.
> 
> What's the exact use case for this?  These other languages do not
> usually have stable ABIs that inter-operate across different
> implementations.

One (of several) use cases is for language interoperability.

A common problem is to pass a function of a high-level
language as a callback to an C API.  This now often
requires special boiler plate code for each case and
there is no automatic way to do this. The fundamental
problem is that the C type can not express that a data
pointer belongs to a function pointer.

void foo(
  void (cb1)(void* data, int a), void* data1,
  void* other_data);  

Here a human (and maybe also a machine) could guess
that data1 belongs to cb1 but not other_data.  But
it is not clear and in more complicated cases
even less so.

void foo(void (_Wide cb1)(int a), void* other_data);


With the new type that would be unambiguous (and 
wrappers could then often be created automatically).
At least for the C API one would also expect ABI
stability.


Martin



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-19 12:13                 ` Martin Uecker
@ 2021-10-20  8:24                   ` Kaz Kylheku (libffi)
  2021-10-20 18:52                     ` Martin Uecker
  2021-10-20  9:10                   ` Florian Weimer
  1 sibling, 1 reply; 28+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-10-20  8:24 UTC (permalink / raw)
  To: Martin Uecker
  Cc: Florian Weimer, Martin Uecker via Libffi-discuss, Libffi-discuss

On 2021-10-19 05:13, Martin Uecker via Libffi-discuss wrote:
> void foo(
>   void (cb1)(void* data, int a), void* data1,
>   void* other_data);
> 
> Here a human (and maybe also a machine) could guess
> that data1 belongs to cb1 but not other_data.  But
> it is not clear and in more complicated cases
> even less so.

That's a property of almost every C API. You cannot
guess the properties from the declaration alone.

If memcpy didn't have the const on the source
argument, you couldn't guess which argument is
the destination and which is the source.

                                 y
Argument names and analogy to x    tell you
which argument of pow is the base and which the
exponent.

That's the job of documentation.

A declarative mechanism which indicates that a given
context parameter goes with a given function could
exist instead of _Wide.

Such a thing was introduced 22 years ago for VLA's:

   void fun(int m, double a[m]);

An obvious convention is possible, and often occurs
in practice: namely, the context parameter in the API
function is given a name which exactly matches the
one in the function pointer:

   void foo(void (cb1)(void *the_context, int a),
            void *unrelated_data,
            void *the_context);

Your proposal also assumes that closures must always
be specified as two argument words, which is an
inappropriate choice to foist onto programmers.

So that is to say, you're taking it for granted that
you want to keep a distinct disadvantage of the
current approach and just wrap it in some syntactic
sugar machinery so that the two arguments appear
encapsulated as some _Wide thing passed by value.

But callback interfaces can easily use just a single
argument word to point to a object that is passed
with reference semantics; that object has a function
and context data.

In high level languages with closures, like Lisp
dialects, closures are of the same "size" as other
values; e.g. 32 or 64 bit pointer-sized word or
what have you.

C++ callback objects are usually one pointer also.

    void f(CallBackBase *cb)
    {
       (*cb)(42); // via operator ()
       // or
       cb->callBack(42); // regular member function
    }

C programs do this too:

    void f(struct obj *o)
    {
       o->ops->callback(o, 42);
    }

Or with the ops table in the object instance:

    void f(struct obj *o)
    {
       o->ops.callback(o, 42);
    }

The idea that all callback interfaces in C
use a function and a void * argument is a false; you
have to survey the entire field of practice in this general
area.

> void foo(void (_Wide cb1)(int a), void* other_data);

Without _Wide, we have

   void (cb1)(int a)

That (cb1)(int a) is a function declarator (with superfluous
parentheses); but when that occurs as a parameter, it
declares a function pointer: it is equivalent to:

   void (*cb1)(int a)

But it looks like

    void (_Wide cb1)(int)

might not be similarly equivalent to

    void (_Wide *cb1)(int)

and the difference could be useful.

Furthermore, given that you can have a _Wide value W, or
a pointer to a _Wide value PW, do you call them using different
syntax?

Is it just W(arg, ...) and PW(arg, ....)?

Or do we require (*PW)(arg, ...)?

If we don't require it, is it because of a decay rule, so that
we can write PW(arg, ...) or (*PW)(arg, ...) or (**PW)(arg, ...)?
and same with W? Or is it due to separately articulated semantics:
that the function call operator works with a _Wide and a
pointer-to_Wide, without any "decay" conversion going on?







^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-19 12:13                 ` Martin Uecker
  2021-10-20  8:24                   ` Kaz Kylheku (libffi)
@ 2021-10-20  9:10                   ` Florian Weimer
  2021-10-20  9:21                     ` Martin Uecker
  2021-10-20 17:27                     ` Kaz Kylheku (libffi)
  1 sibling, 2 replies; 28+ messages in thread
From: Florian Weimer @ 2021-10-20  9:10 UTC (permalink / raw)
  To: Martin Uecker; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

* Martin Uecker:

> One (of several) use cases is for language interoperability.
>
> A common problem is to pass a function of a high-level
> language as a callback to an C API.  This now often
> requires special boiler plate code for each case and
> there is no automatic way to do this. The fundamental
> problem is that the C type can not express that a data
> pointer belongs to a function pointer.
>
> void foo(
>   void (cb1)(void* data, int a), void* data1,
>   void* other_data);  
>
> Here a human (and maybe also a machine) could guess
> that data1 belongs to cb1 but not other_data.  But
> it is not clear and in more complicated cases
> even less so.
>
> void foo(void (_Wide cb1)(int a), void* other_data);
>
>
> With the new type that would be unambiguous (and 
> wrappers could then often be created automatically).
> At least for the C API one would also expect ABI
> stability.

I don't expect a lot of uptake for this.  One problem is the suggested
aggregate representation of wide function pointers.  It's not exactly
common for non-C implementations to implement those aspects of the
platform C ABI.  For many targets, even C and C++ compilers do not
always agree completely on the finer points.  Passing code and data
pointer in separate arguments is much more robust, and safely within the
interoperable ABI subset.

Perhaps a set of standard attributes to describe the explicit closure
argument would be useful, though.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-20  9:10                   ` Florian Weimer
@ 2021-10-20  9:21                     ` Martin Uecker
  2021-10-20  9:27                       ` Florian Weimer
  2021-10-20 17:27                     ` Kaz Kylheku (libffi)
  1 sibling, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-20  9:21 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

Am Mittwoch, den 20.10.2021, 11:10 +0200 schrieb Florian Weimer:
> * Martin Uecker:
> 
> > One (of several) use cases is for language interoperability.
> > 
> > A common problem is to pass a function of a high-level
> > language as a callback to an C API.  This now often
> > requires special boiler plate code for each case and
> > there is no automatic way to do this. The fundamental
> > problem is that the C type can not express that a data
> > pointer belongs to a function pointer.
> > 
> > void foo(
> >   void (cb1)(void* data, int a), void* data1,
> >   void* other_data);  
> > 
> > Here a human (and maybe also a machine) could guess
> > that data1 belongs to cb1 but not other_data.  But
> > it is not clear and in more complicated cases
> > even less so.
> > 
> > void foo(void (_Wide cb1)(int a), void* other_data);
> > 
> > 
> > With the new type that would be unambiguous (and 
> > wrappers could then often be created automatically).
> > At least for the C API one would also expect ABI
> > stability.
> 
> I don't expect a lot of uptake for this.  One problem is the suggested
> aggregate representation of wide function pointers.  It's not exactly
> common for non-C implementations to implement those aspects of the
> platform C ABI.  

Can you explain what you mean by this?  Usually non-C
implementations build on C ABI, so if the C ABI is
extended the extension is then available to them.

I would expect a lot of update (in the long run) because
there is now a 1:1 mapping for a function pointer
type in other languages to the function type in C.

> For many targets, even C and C++ compilers do not
> always agree completely on the finer points. 

Unfortunately, e.g. atomics. But the goal and intention
is interoperability. 

> Passing code and data pointer in separate arguments is
> much more robust, and safely within the
> interoperable ABI subset.

It is not my experience that the void pointer based
APIs are robust and safe.

> Perhaps a set of standard attributes to describe the explicit closure
> argument would be useful, though.

I also thought about this (and this could be added
too) but this does not solve all problems.

Martin



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-20  9:21                     ` Martin Uecker
@ 2021-10-20  9:27                       ` Florian Weimer
  0 siblings, 0 replies; 28+ messages in thread
From: Florian Weimer @ 2021-10-20  9:27 UTC (permalink / raw)
  To: Martin Uecker; +Cc: Martin Uecker via Libffi-discuss, Anthony Green

* Martin Uecker:

>> I don't expect a lot of uptake for this.  One problem is the suggested
>> aggregate representation of wide function pointers.  It's not exactly
>> common for non-C implementations to implement those aspects of the
>> platform C ABI.  
>
> Can you explain what you mean by this?  Usually non-C
> implementations build on C ABI, so if the C ABI is
> extended the extension is then available to them.

In practice, the shared ABI is just a subset of what is required for
full C.  Often, support for pass-by-value struct and union arguments and
return values are missing or implemented incorrectly (not according to
the platform ABI).  Other traditionally incomplete parts (relative to
C90) are variadic functions and bitfields.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-20  9:10                   ` Florian Weimer
  2021-10-20  9:21                     ` Martin Uecker
@ 2021-10-20 17:27                     ` Kaz Kylheku (libffi)
  2021-10-21  9:48                       ` Florian Weimer
  1 sibling, 1 reply; 28+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-10-20 17:27 UTC (permalink / raw)
  To: Florian Weimer; +Cc: Martin Uecker, Martin Uecker via Libffi-discuss

On 2021-10-20 02:10, Florian Weimer via Libffi-discuss wrote:
> Perhaps a set of standard attributes to describe the explicit closure
> argument would be useful, though.

For what purpose though? If there is some attribute mechanism in the 
syntax,
what translation decision does it drive? Or else, what diagnostic 
purpose?

Variable length arrays use matching names:

   void f(int x, double array[x])

the size attribute of the array is linked to the x parameter by name.

This is semantically important, because inside the function, the
run-time size of the array is computable with "sizeof array",
and that is derived from the value of x.

If the compiler knows that some closure argument c is linked with a
function f, what special treatment can it perform?

The function could be callable with fewer arguments, such that
c is implicitly inserted into the call.

Or, perhaps more usefully, we might be interested in diagnosing the
situation when the program neglects to pass the context parameter
to the function.

But if all we have is some parameter attributes, that is of limited
use; what if the function want to store those
two pieces into some structure, so that some other function
will later call f(c, ...).

The annotation mechanism would have to be propagated to all the
places in the program where the f value can reach; which means
additional declaration material in multiple places. If a structure
stores a function pointer and context value, we need to be able
to relate them in a structure, and so forth.

Imagine we had:

    struct callback {
      void *c __attribute__((context (f)));
      void (*f)(void *context);
    };

Let's use GCC-like attribute syntax with the understanding
that the actual syntax could be something else.

    void fun(void *context _attribute__ ((context (f))),
             void *other,
             void (*f)(void *))
    {
       struct callback cb;
       cb.f = f;
       cb.c = other; // ERROR
    }

This function could be diagnosable; however, depending on the exact
rules, it could requires data flow techniques, difficult to implement
without optimization.

The semantic constraint rules could be as simple as:

1. If either the members *f* or *c* are assigned, the other one
    must be assigned in the same expression, or adjacent statements
    of a compound statement, such that no other operation intervenes
    between the assignments.

2. Both members must be assigned from a linked pair. The assigned
    values are a linked function-context argument pair, or
    linked function-context members of the same structure.

3. In a struct definition with an initializer, if one linked member
    has an explicit initializer, the other must have one also, and
    the values must be a linked pair.

The only ingredient missing is the problem to go with this wonderful 
solution!



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-20  8:24                   ` Kaz Kylheku (libffi)
@ 2021-10-20 18:52                     ` Martin Uecker
  0 siblings, 0 replies; 28+ messages in thread
From: Martin Uecker @ 2021-10-20 18:52 UTC (permalink / raw)
  To: Kaz Kylheku (libffi)
  Cc: Florian Weimer, Martin Uecker via Libffi-discuss, Libffi-discuss

Am Mittwoch, den 20.10.2021, 01:24 -0700 schrieb Kaz Kylheku (libffi):
> On 2021-10-19 05:13, Martin Uecker via Libffi-discuss wrote:
> > void foo(
> >   void (cb1)(void* data, int a), void* data1,
> >   void* other_data);
> > 
> > Here a human (and maybe also a machine) could guess
> > that data1 belongs to cb1 but not other_data.  But
> > it is not clear and in more complicated cases
> > even less so.
> 
> That's a property of almost every C API. You cannot
> guess the properties from the declaration alone.
> 
> If memcpy didn't have the const on the source
> argument, you couldn't guess which argument is
> the destination and which is the source.

The difference here is that something which is a fundamental
type in most other languages is split up into two things
in C.

>                                  y
> Argument names and analogy to x    tell you
> which argument of pow is the base and which the
> exponent.
> 
> That's the job of documentation.

Generators for language wrappers do
not read documentation ;-)

> A declarative mechanism which indicates that a given
> context parameter goes with a given function could
> exist instead of _Wide.

Yes, I had considered this too.

> Such a thing was introduced 22 years ago for VLA's:
> 
>    void fun(int m, double a[m]);

Sure, but VLA types are also underdeveloped
The limitations are:

- You can not store a pointer to a VLA in the struct.
- You can not put the size later.
- You can not
always automatically verify that
the size is correct at the caller side because
the semantic type of a
declaration is:

void fun(int m, double a[*]);

All this would be avoided with a wide
pointer type that integrates the size into the
pointer (as in FORTRAN). This was in fact proposed
by Dennis Ritchie himself a long time ago:

Ritchie DM. Variable-size arrays in C.
The Journal of C Language Translation 1990;2:81-86.

> An obvious convention is possible, and often occurs
> in practice: namely, the context parameter in the API
> function is given a name which exactly matches the
> one in the function pointer:
> 
>    void foo(void (cb1)(void *the_context, int a),
>             void *unrelated_data,
>             void *the_context);
> 
> Your proposal also assumes that closures must always
> be specified as two argument words, which is an
> inappropriate choice to foist onto programmers.

No it does not assume this. You can always keep
doing this as before.

> So that is to say, you're taking it for granted that
> you want to keep a distinct disadvantage of the
> current approach and just wrap it in some syntactic
> sugar machinery so that the two arguments appear
> encapsulated as some _Wide thing passed by value.
> 
> But callback interfaces can easily use just a single
> argument word to point to a object that is passed
> with reference semantics; that object has a function
> and context data.

I am not sure what you are talking about..

> In high level languages with closures, like Lisp
> dialects, closures are of the same "size" as other
> values; e.g. 32 or 64 bit pointer-sized word or
> what have you.
> 
> C++ callback objects are usually one pointer also.
> 
>     void f(CallBackBase *cb)
>     {
>        (*cb)(42); // via operator ()
>        // or
>        cb->callBack(42); // regular member function
>     }

Not at all. C++ has std::function for this purpose
which is usually even bigger than two words because
it does an optimization for small objects.

> C programs do this too:
> 
>     void f(struct obj *o)
>     {
>        o->ops->callback(o, 42);
>     }
> 
> Or with the ops table in the object instance:
> 
>     void f(struct obj *o)
>     {
>        o->ops.callback(o, 42);
>     }
> 
> The idea that all callback interfaces in C
> use a function and a void * argument is a false; you
> have to survey the entire field of practice in this general
> area.

I never claimed that "all callbacks use void*". So please
do not use strawman arguments.

> > void foo(void (_Wide cb1)(int a), void* other_data);
> 
> Without _Wide, we have
> 
>    void (cb1)(int a)
> 
> That (cb1)(int a) is a function declarator (with superfluous
> parentheses); but when that occurs as a parameter, it
> declares a function pointer: it is equivalent to:
> 
>    void (*cb1)(int a)
> 
> But it looks like
> 
>     void (_Wide cb1)(int)
> 
> might not be similarly equivalent to
> 
>     void (_Wide *cb1)(int)
> 
> and the difference could be useful.
>
> Furthermore, given that you can have a _Wide value W, or
> a pointer to a _Wide value PW, do you call them using different
> syntax?
> 
> Is it just W(arg, ...) and PW(arg, ....)?
> 
> Or do we require (*PW)(arg, ...)?
>
> If we don't require it, is it because of a decay rule, so that
> we can write PW(arg, ...) or (*PW)(arg, ...) or (**PW)(arg, ...)?
> and same with W? Or is it due to separately articulated semantics:
> that the function call operator works with a _Wide and a
> pointer-to_Wide, without any "decay" conversion going on?

If _Wide is a qualifier as in the proposal
Function types nothing changes.

Martin





^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-20 17:27                     ` Kaz Kylheku (libffi)
@ 2021-10-21  9:48                       ` Florian Weimer
  0 siblings, 0 replies; 28+ messages in thread
From: Florian Weimer @ 2021-10-21  9:48 UTC (permalink / raw)
  To: Kaz Kylheku (libffi); +Cc: Martin Uecker, Martin Uecker via Libffi-discuss

* Kaz Kylheku:

> On 2021-10-20 02:10, Florian Weimer via Libffi-discuss wrote:
>> Perhaps a set of standard attributes to describe the explicit closure
>> argument would be useful, though.
>
> For what purpose though? If there is some attribute mechanism in the
> syntax, what translation decision does it drive? Or else, what
> diagnostic purpose?

Auto-generation of bindings for languages that have functions with
environments.

> If the compiler knows that some closure argument c is linked with a
> function f, what special treatment can it perform?

It can automatically separate the environment/chain pointer from the
code pointer and pass them in the right arguments.  And a thunk can be
create if necessary.

> The annotation mechanism would have to be propagated to all the
> places in the program where the f value can reach; which means
> additional declaration material in multiple places. If a structure
> stores a function pointer and context value, we need to be able
> to relate them in a structure, and so forth.

Environment/chain arguments in structs would only need annotating if
they are publicly exposed on API boundaries.  I am not convinced this is
a common occurrence.

With the new function pointer type, all these types have to be
duplicated, to preserve compatibility with older C versions.  That seems
rather unlikely.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-16  8:08               ` Jarkko Hietaniemi
@ 2021-10-16  9:35                 ` Jarkko Hietaniemi
  0 siblings, 0 replies; 28+ messages in thread
From: Jarkko Hietaniemi @ 2021-10-16  9:35 UTC (permalink / raw)
  To: Martin Uecker
  Cc: Daniel Colascione, Kaz Kylheku (libffi),
	Martin Uecker via Libffi-discuss

Related:

http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2021/p2303r0.pdf

la 16. lokak. 2021 klo 11.08 Jarkko Hietaniemi (jhi@iki.fi) kirjoitti:

> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2787.pdf
>
> su 10. lokak. 2021 klo 22.25 Martin Uecker via Libffi-discuss (
> libffi-discuss@sourceware.org) kirjoitti:
>
>> Am Sonntag, den 10.10.2021, 11:57 -0700 schrieb Daniel Colascione:
>> >
>> > On October 10, 2021 11:47:18 Martin Uecker <ma.uecker@gmail.com> wrote:
>> >
>> > > Am Sonntag, den 10.10.2021, 11:17 -0700 schrieb Daniel Colascione:
>> > > > On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com>
>> wrote:
>> > > >
>> > > > > Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
>> > > > > > On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
>> > > > > > <libffi-discuss@sourceware.org> wrote:
>> > > > > >
>> > > > > > > Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku
>> (libffi):
>> > > > > > > > On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
>> > > > > > > > > Hi all,
>> > > > > > > > >
>> > > > > > > > > I will propose a wide function pointer type (actually
>> > > > > > > > > a wide function type) to WG14 for C23 as a common
>> > > > > > > > > type for callbacks, closures, which now require an
>> > > > > > > > > additional void pointer argument in C APIs. This
>> > > > > > > > > is intended to be compatible with ABIs with now
>> > > > > > > > > use a static chain register.
>> > > > > > > >
>> > > > > > > > Opposed. There is nothing wrong with separate arguments
>> > > > > > > > for function pointer and context.
>> > > > > > >
>> > > > > > > Noted.  Your argument sbelow all boil down to the point
>> > > > > > > that there are cases where it might not be the ideal
>> > > > > > > choice. But nobody forces anyone to use it.
>> > > > > > "I like it" is not by itself a rationale for including
>> something in a core
>> > > > > > language standard. This is not something that should be in C.
>> Why,
>> > > > > > precisely, can't you just define a struct with two fields and
>> make that
>> > > > > > your closure type?
>> > > > >
>> > > > > Sure. There are at least five reasons:
>> > > > >
>> > > > > - one now does has to do it manually which is
>> > > > > inconvenient because it has to be done for each type,
>> > > > > although this a common pattern which is trivial
>> > > > > to automate.
>> > > > >
>> > > > > - APIs are often unsafe because they have to use
>> > > > > void pointer to be generic. When the void pointer
>> > > > > can be packaged into the wide pointer this problem
>> > > > > goes away.
>> > > > >
>> > > > > - APIs are often different for no good reason.
>> > > > > Having a standard approach would make this more
>> > > > > canonical, hence simpler and less error prone,
>> > > > > and also more compatible - requiring less adapter
>> > > > > code.
>> > > > >
>> > > > > - There are APIs where a pointer to a
>> > > > > function is expected but there is no data pointer.
>> > > > > Then other languages have a problem interfacing
>> > > > > with such APIs.   (or why does libffi generate
>> > > > > trampolines?)
>> > > > >
>> > > > > - Finally, C might get lambda expressions
>> > > > > which are far more useful with such a pointer type.
>> > > > > (for nested function you could avoid executable
>> > > > > stack).
>> > > > >
>> > > > >
>> > > > > All five are good reasons to put a standard
>> > > > > approach in the core language in my opinion.
>> > > > >
>> > > > >
>> > > > > But I am actually not interested in this discussion
>> > > > > and do not have time for it.
>> > > > >
>> > > > > I am more interested in constructive technical
>> > > > > feedback.
>> > > >
>> > > > Your proposal doesn't actually solve any of these problems though,
>> and as
>> > > > we agree, users can already trivially make their own closure types.
>> > >
>> > > I wonder why you think it does not address
>> > > these problems?
>> >
>> > Because there's no difference between your proposal and a simple user
>> > struct that does the same thing.
>>
>> I see at least the following differences:
>>
>> - Simple user code does not know how to load the static chain
>> pointer into its ABI-specified register in a portable way.
>>
>> - A simple user struct works exactly for one function type,
>> so such a struct has to be created for all APIs separately.
>>
>> - One can not easily convert a regular pointer to a
>> simple user struct (this would require at least inserting
>> a branch at it each call site)
>>
>> - A simple user struct can be defined in different ways,
>> so there is no standard.
>>
>> - If a library defines a simple struct and another library
>> another simple struct, then these two structs are not
>> compatible even if defined identically.
>>
>> - Other languages or interface generating tools will not
>> know about the simple user struct, but will certainly
>> know about a standardized type.
>>
>>
>> >  The language could get lambdas one day
>> > (though I doubt it), but if it does, each lambda will likely have a
>> unique
>> > and anonymous type (like in C++) and won't fit in your wide function
>> > pointer struct anyway.
>>
>> I think the lambda proposal has a good chance, although
>> I am not sure it will make it already into C23.
>> There would be a conversion rule (as mentioned in
>> the proposal).
>>
>> C++ has std::function for a similar purpose.
>>
>> > > > What *would* be interesting is a new standard library facility for
>> making
>> > > > libffi-style closures and a standard struct type for packaging the
>> things.
>> > > > *That's* something that could be justified for inclusion in a
>> standard
>> > > > library, since it's not something that programs can make on their
>> own
>> > > > without the kind of platform specific glue that libffi provides.
>> > >
>> > > I also would love to have this, but see below...
>> > >
>> > > > Basically,
>> > > > the interface would look something like this:
>> > > >
>> > > > typedef opaque_mumble stdclosure_t;
>> > > >
>> > > > // Returns a callable function pointer or NULL on error with errno
>> set
>> > > > void* stdclosure_init(stdclosure_t* closure, void* fnptr, void*
>> data);
>> > >
>> > > A void* might not be big enough for a callable function
>> > > pointer an all architectures supported by C so this
>> > > needs to return something like
>> > >
>> > > void (*)();
>> > >
>> > >
>> > > (for POSIX this is not a problem)
>> >
>> > Yeah, although at this point, I'm in favor of standardizing things that
>> are
>> > de facto universal anyway. POSIX for example mandates that NULL is all
>> zero
>> > bits and that function pointers fit in void*, and I think it's time for
>> C
>> > to just acknowledge the same.
>>
>> I tend to agree, but I do not see this happening.
>>
>> > >
>> > > > void stsclosure_destroy(stdclosure_t* closure);
>> > > >
>> > > > The function pointer returned by stdclosure_init would, when
>> called, call
>> > > > FNPTR with an extra initial parameter (or trailing parameter or
>> something)
>> > > > of type void* and value DATA and all the remaining parameters with
>> which it
>> > > > was called.
>> > > >
>> > > > This facility would actually probably have to be more complicated
>> in real
>> > > > life due to varying platform ABIs, but you get the idea. This is
>> something
>> > > > that could belong in the standard because it'd provide a generic
>> > > > abstraction for something every platform can do but that every
>> platform has
>> > > > to do in a slightly different way.
>> > >
>> > > One problem could be that this requires generating
>> > > trampolines at run-time (and it is not clear we
>> > > can do this everywhere where C is supported) or
>> > > having a fixed set of preallocated trampolines.
>> >
>> > Or remapping a fixed set of trampolines, like libffi does on some
>> > platforms.
>>
>> This should be quite expensive though, if you use lots
>> of closures.
>>
>> >  The new stdclosure feature would have to be optional, yes, but
>> > in practice it would be supported everywhere people write programs
>> except
>> > maybe a few microcontrollers and DSPs.
>>
>> Yes, you should try to propose this.
>>
>> > > The other limitation (which the new type would
>> > > avoid) is the need for dynamic resource allocation,
>> > > which is sometimes not desirable.
>> > >
>> > > But I would love to see such a proposal
>> > > brought forward.
>>
>> Martin
>>
>>
>>
>
> --
> There is this special biologist word we use for 'stable'. It is 'dead'. --
> Jack Cohen
>


-- 
There is this special biologist word we use for 'stable'. It is 'dead'. --
Jack Cohen

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 19:24             ` Martin Uecker
@ 2021-10-16  8:08               ` Jarkko Hietaniemi
  2021-10-16  9:35                 ` Jarkko Hietaniemi
  0 siblings, 1 reply; 28+ messages in thread
From: Jarkko Hietaniemi @ 2021-10-16  8:08 UTC (permalink / raw)
  To: Martin Uecker
  Cc: Daniel Colascione, Kaz Kylheku (libffi),
	Martin Uecker via Libffi-discuss

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2787.pdf

su 10. lokak. 2021 klo 22.25 Martin Uecker via Libffi-discuss (
libffi-discuss@sourceware.org) kirjoitti:

> Am Sonntag, den 10.10.2021, 11:57 -0700 schrieb Daniel Colascione:
> >
> > On October 10, 2021 11:47:18 Martin Uecker <ma.uecker@gmail.com> wrote:
> >
> > > Am Sonntag, den 10.10.2021, 11:17 -0700 schrieb Daniel Colascione:
> > > > On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com>
> wrote:
> > > >
> > > > > Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
> > > > > > On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
> > > > > > <libffi-discuss@sourceware.org> wrote:
> > > > > >
> > > > > > > Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku
> (libffi):
> > > > > > > > On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I will propose a wide function pointer type (actually
> > > > > > > > > a wide function type) to WG14 for C23 as a common
> > > > > > > > > type for callbacks, closures, which now require an
> > > > > > > > > additional void pointer argument in C APIs. This
> > > > > > > > > is intended to be compatible with ABIs with now
> > > > > > > > > use a static chain register.
> > > > > > > >
> > > > > > > > Opposed. There is nothing wrong with separate arguments
> > > > > > > > for function pointer and context.
> > > > > > >
> > > > > > > Noted.  Your argument sbelow all boil down to the point
> > > > > > > that there are cases where it might not be the ideal
> > > > > > > choice. But nobody forces anyone to use it.
> > > > > > "I like it" is not by itself a rationale for including something
> in a core
> > > > > > language standard. This is not something that should be in C.
> Why,
> > > > > > precisely, can't you just define a struct with two fields and
> make that
> > > > > > your closure type?
> > > > >
> > > > > Sure. There are at least five reasons:
> > > > >
> > > > > - one now does has to do it manually which is
> > > > > inconvenient because it has to be done for each type,
> > > > > although this a common pattern which is trivial
> > > > > to automate.
> > > > >
> > > > > - APIs are often unsafe because they have to use
> > > > > void pointer to be generic. When the void pointer
> > > > > can be packaged into the wide pointer this problem
> > > > > goes away.
> > > > >
> > > > > - APIs are often different for no good reason.
> > > > > Having a standard approach would make this more
> > > > > canonical, hence simpler and less error prone,
> > > > > and also more compatible - requiring less adapter
> > > > > code.
> > > > >
> > > > > - There are APIs where a pointer to a
> > > > > function is expected but there is no data pointer.
> > > > > Then other languages have a problem interfacing
> > > > > with such APIs.   (or why does libffi generate
> > > > > trampolines?)
> > > > >
> > > > > - Finally, C might get lambda expressions
> > > > > which are far more useful with such a pointer type.
> > > > > (for nested function you could avoid executable
> > > > > stack).
> > > > >
> > > > >
> > > > > All five are good reasons to put a standard
> > > > > approach in the core language in my opinion.
> > > > >
> > > > >
> > > > > But I am actually not interested in this discussion
> > > > > and do not have time for it.
> > > > >
> > > > > I am more interested in constructive technical
> > > > > feedback.
> > > >
> > > > Your proposal doesn't actually solve any of these problems though,
> and as
> > > > we agree, users can already trivially make their own closure types.
> > >
> > > I wonder why you think it does not address
> > > these problems?
> >
> > Because there's no difference between your proposal and a simple user
> > struct that does the same thing.
>
> I see at least the following differences:
>
> - Simple user code does not know how to load the static chain
> pointer into its ABI-specified register in a portable way.
>
> - A simple user struct works exactly for one function type,
> so such a struct has to be created for all APIs separately.
>
> - One can not easily convert a regular pointer to a
> simple user struct (this would require at least inserting
> a branch at it each call site)
>
> - A simple user struct can be defined in different ways,
> so there is no standard.
>
> - If a library defines a simple struct and another library
> another simple struct, then these two structs are not
> compatible even if defined identically.
>
> - Other languages or interface generating tools will not
> know about the simple user struct, but will certainly
> know about a standardized type.
>
>
> >  The language could get lambdas one day
> > (though I doubt it), but if it does, each lambda will likely have a
> unique
> > and anonymous type (like in C++) and won't fit in your wide function
> > pointer struct anyway.
>
> I think the lambda proposal has a good chance, although
> I am not sure it will make it already into C23.
> There would be a conversion rule (as mentioned in
> the proposal).
>
> C++ has std::function for a similar purpose.
>
> > > > What *would* be interesting is a new standard library facility for
> making
> > > > libffi-style closures and a standard struct type for packaging the
> things.
> > > > *That's* something that could be justified for inclusion in a
> standard
> > > > library, since it's not something that programs can make on their own
> > > > without the kind of platform specific glue that libffi provides.
> > >
> > > I also would love to have this, but see below...
> > >
> > > > Basically,
> > > > the interface would look something like this:
> > > >
> > > > typedef opaque_mumble stdclosure_t;
> > > >
> > > > // Returns a callable function pointer or NULL on error with errno
> set
> > > > void* stdclosure_init(stdclosure_t* closure, void* fnptr, void*
> data);
> > >
> > > A void* might not be big enough for a callable function
> > > pointer an all architectures supported by C so this
> > > needs to return something like
> > >
> > > void (*)();
> > >
> > >
> > > (for POSIX this is not a problem)
> >
> > Yeah, although at this point, I'm in favor of standardizing things that
> are
> > de facto universal anyway. POSIX for example mandates that NULL is all
> zero
> > bits and that function pointers fit in void*, and I think it's time for
> C
> > to just acknowledge the same.
>
> I tend to agree, but I do not see this happening.
>
> > >
> > > > void stsclosure_destroy(stdclosure_t* closure);
> > > >
> > > > The function pointer returned by stdclosure_init would, when called,
> call
> > > > FNPTR with an extra initial parameter (or trailing parameter or
> something)
> > > > of type void* and value DATA and all the remaining parameters with
> which it
> > > > was called.
> > > >
> > > > This facility would actually probably have to be more complicated in
> real
> > > > life due to varying platform ABIs, but you get the idea. This is
> something
> > > > that could belong in the standard because it'd provide a generic
> > > > abstraction for something every platform can do but that every
> platform has
> > > > to do in a slightly different way.
> > >
> > > One problem could be that this requires generating
> > > trampolines at run-time (and it is not clear we
> > > can do this everywhere where C is supported) or
> > > having a fixed set of preallocated trampolines.
> >
> > Or remapping a fixed set of trampolines, like libffi does on some
> > platforms.
>
> This should be quite expensive though, if you use lots
> of closures.
>
> >  The new stdclosure feature would have to be optional, yes, but
> > in practice it would be supported everywhere people write programs
> except
> > maybe a few microcontrollers and DSPs.
>
> Yes, you should try to propose this.
>
> > > The other limitation (which the new type would
> > > avoid) is the need for dynamic resource allocation,
> > > which is sometimes not desirable.
> > >
> > > But I would love to see such a proposal
> > > brought forward.
>
> Martin
>
>
>

-- 
There is this special biologist word we use for 'stable'. It is 'dead'. --
Jack Cohen

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 18:57           ` Daniel Colascione
@ 2021-10-10 19:24             ` Martin Uecker
  2021-10-16  8:08               ` Jarkko Hietaniemi
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-10 19:24 UTC (permalink / raw)
  To: Daniel Colascione, Kaz Kylheku (libffi),
	Martin Uecker via Libffi-discuss

Am Sonntag, den 10.10.2021, 11:57 -0700 schrieb Daniel Colascione:
> 
> On October 10, 2021 11:47:18 Martin Uecker <ma.uecker@gmail.com> wrote:
> 
> > Am Sonntag, den 10.10.2021, 11:17 -0700 schrieb Daniel Colascione:
> > > On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com> wrote:
> > > 
> > > > Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
> > > > > On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
> > > > > <libffi-discuss@sourceware.org> wrote:
> > > > > 
> > > > > > Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
> > > > > > > On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > I will propose a wide function pointer type (actually
> > > > > > > > a wide function type) to WG14 for C23 as a common
> > > > > > > > type for callbacks, closures, which now require an
> > > > > > > > additional void pointer argument in C APIs. This
> > > > > > > > is intended to be compatible with ABIs with now
> > > > > > > > use a static chain register.
> > > > > > > 
> > > > > > > Opposed. There is nothing wrong with separate arguments
> > > > > > > for function pointer and context.
> > > > > > 
> > > > > > Noted.  Your argument sbelow all boil down to the point
> > > > > > that there are cases where it might not be the ideal
> > > > > > choice. But nobody forces anyone to use it.
> > > > > "I like it" is not by itself a rationale for including something in a core
> > > > > language standard. This is not something that should be in C. Why,
> > > > > precisely, can't you just define a struct with two fields and make that
> > > > > your closure type?
> > > > 
> > > > Sure. There are at least five reasons:
> > > > 
> > > > - one now does has to do it manually which is
> > > > inconvenient because it has to be done for each type,
> > > > although this a common pattern which is trivial
> > > > to automate.
> > > > 
> > > > - APIs are often unsafe because they have to use
> > > > void pointer to be generic. When the void pointer
> > > > can be packaged into the wide pointer this problem
> > > > goes away.
> > > > 
> > > > - APIs are often different for no good reason.
> > > > Having a standard approach would make this more
> > > > canonical, hence simpler and less error prone,
> > > > and also more compatible - requiring less adapter
> > > > code.
> > > > 
> > > > - There are APIs where a pointer to a
> > > > function is expected but there is no data pointer.
> > > > Then other languages have a problem interfacing
> > > > with such APIs.   (or why does libffi generate
> > > > trampolines?)
> > > > 
> > > > - Finally, C might get lambda expressions
> > > > which are far more useful with such a pointer type.
> > > > (for nested function you could avoid executable
> > > > stack).
> > > > 
> > > > 
> > > > All five are good reasons to put a standard
> > > > approach in the core language in my opinion.
> > > > 
> > > > 
> > > > But I am actually not interested in this discussion
> > > > and do not have time for it.
> > > > 
> > > > I am more interested in constructive technical
> > > > feedback.
> > > 
> > > Your proposal doesn't actually solve any of these problems though, and as
> > > we agree, users can already trivially make their own closure types.
> > 
> > I wonder why you think it does not address
> > these problems?
> 
> Because there's no difference between your proposal and a simple user 
> struct that does the same thing.

I see at least the following differences:

- Simple user code does not know how to load the static chain
pointer into its ABI-specified register in a portable way.

- A simple user struct works exactly for one function type,
so such a struct has to be created for all APIs separately.

- One can not easily convert a regular pointer to a
simple user struct (this would require at least inserting
a branch at it each call site) 

- A simple user struct can be defined in different ways,
so there is no standard.

- If a library defines a simple struct and another library
another simple struct, then these two structs are not
compatible even if defined identically.

- Other languages or interface generating tools will not
know about the simple user struct, but will certainly
know about a standardized type.


>  The language could get lambdas one day 
> (though I doubt it), but if it does, each lambda will likely have a unique 
> and anonymous type (like in C++) and won't fit in your wide function 
> pointer struct anyway.

I think the lambda proposal has a good chance, although
I am not sure it will make it already into C23.
There would be a conversion rule (as mentioned in
the proposal).

C++ has std::function for a similar purpose.

> > > What *would* be interesting is a new standard library facility for making
> > > libffi-style closures and a standard struct type for packaging the things.
> > > *That's* something that could be justified for inclusion in a standard
> > > library, since it's not something that programs can make on their own
> > > without the kind of platform specific glue that libffi provides.
> > 
> > I also would love to have this, but see below...
> > 
> > > Basically,
> > > the interface would look something like this:
> > > 
> > > typedef opaque_mumble stdclosure_t;
> > > 
> > > // Returns a callable function pointer or NULL on error with errno set
> > > void* stdclosure_init(stdclosure_t* closure, void* fnptr, void* data);
> > 
> > A void* might not be big enough for a callable function
> > pointer an all architectures supported by C so this
> > needs to return something like
> > 
> > void (*)();
> > 
> > 
> > (for POSIX this is not a problem)
> 
> Yeah, although at this point, I'm in favor of standardizing things that are 
> de facto universal anyway. POSIX for example mandates that NULL is all zero 
> bits and that function pointers fit in void*, and I think it's time for C 
> to just acknowledge the same.

I tend to agree, but I do not see this happening.

> > 
> > > void stsclosure_destroy(stdclosure_t* closure);
> > > 
> > > The function pointer returned by stdclosure_init would, when called, call
> > > FNPTR with an extra initial parameter (or trailing parameter or something)
> > > of type void* and value DATA and all the remaining parameters with which it
> > > was called.
> > > 
> > > This facility would actually probably have to be more complicated in real
> > > life due to varying platform ABIs, but you get the idea. This is something
> > > that could belong in the standard because it'd provide a generic
> > > abstraction for something every platform can do but that every platform has
> > > to do in a slightly different way.
> > 
> > One problem could be that this requires generating
> > trampolines at run-time (and it is not clear we
> > can do this everywhere where C is supported) or
> > having a fixed set of preallocated trampolines.
> 
> Or remapping a fixed set of trampolines, like libffi does on some 
> platforms.

This should be quite expensive though, if you use lots
of closures.

>  The new stdclosure feature would have to be optional, yes, but 
> in practice it would be supported everywhere people write programs except 
> maybe a few microcontrollers and DSPs.

Yes, you should try to propose this.

> > The other limitation (which the new type would
> > avoid) is the need for dynamic resource allocation,
> > which is sometimes not desirable.
> > 
> > But I would love to see such a proposal
> > brought forward.

Martin



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 18:47         ` Martin Uecker
@ 2021-10-10 18:57           ` Daniel Colascione
  2021-10-10 19:24             ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Daniel Colascione @ 2021-10-10 18:57 UTC (permalink / raw)
  To: Martin Uecker, Kaz Kylheku (libffi), Martin Uecker via Libffi-discuss



On October 10, 2021 11:47:18 Martin Uecker <ma.uecker@gmail.com> wrote:

> Am Sonntag, den 10.10.2021, 11:17 -0700 schrieb Daniel Colascione:
>> On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com> wrote:
>>
>>> Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
>>>> On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
>>>> <libffi-discuss@sourceware.org> wrote:
>>>>
>>>>> Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
>>>>>> On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I will propose a wide function pointer type (actually
>>>>>>> a wide function type) to WG14 for C23 as a common
>>>>>>> type for callbacks, closures, which now require an
>>>>>>> additional void pointer argument in C APIs. This
>>>>>>> is intended to be compatible with ABIs with now
>>>>>>> use a static chain register.
>>>>>>
>>>>>> Opposed. There is nothing wrong with separate arguments
>>>>>> for function pointer and context.
>>>>>
>>>>> Noted.  Your argument sbelow all boil down to the point
>>>>> that there are cases where it might not be the ideal
>>>>> choice. But nobody forces anyone to use it.
>>>> "I like it" is not by itself a rationale for including something in a core
>>>> language standard. This is not something that should be in C. Why,
>>>> precisely, can't you just define a struct with two fields and make that
>>>> your closure type?
>>>
>>> Sure. There are at least five reasons:
>>>
>>> - one now does has to do it manually which is
>>> inconvenient because it has to be done for each type,
>>> although this a common pattern which is trivial
>>> to automate.
>>>
>>> - APIs are often unsafe because they have to use
>>> void pointer to be generic. When the void pointer
>>> can be packaged into the wide pointer this problem
>>> goes away.
>>>
>>> - APIs are often different for no good reason.
>>> Having a standard approach would make this more
>>> canonical, hence simpler and less error prone,
>>> and also more compatible - requiring less adapter
>>> code.
>>>
>>> - There are APIs where a pointer to a
>>> function is expected but there is no data pointer.
>>> Then other languages have a problem interfacing
>>> with such APIs.   (or why does libffi generate
>>> trampolines?)
>>>
>>> - Finally, C might get lambda expressions
>>> which are far more useful with such a pointer type.
>>> (for nested function you could avoid executable
>>> stack).
>>>
>>>
>>> All five are good reasons to put a standard
>>> approach in the core language in my opinion.
>>>
>>>
>>> But I am actually not interested in this discussion
>>> and do not have time for it.
>>>
>>> I am more interested in constructive technical
>>> feedback.
>>
>> Your proposal doesn't actually solve any of these problems though, and as
>> we agree, users can already trivially make their own closure types.
>
> I wonder why you think it does not address
> these problems?

Because there's no difference between your proposal and a simple user 
struct that does the same thing. The language could get lambdas one day 
(though I doubt it), but if it does, each lambda will likely have a unique 
and anonymous type (like in C++) and won't fit in your wide function 
pointer struct anyway.

>> What *would* be interesting is a new standard library facility for making
>> libffi-style closures and a standard struct type for packaging the things.
>
>> *That's* something that could be justified for inclusion in a standard
>> library, since it's not something that programs can make on their own
>> without the kind of platform specific glue that libffi provides.
>
> I also would love to have this, but see below...
>
>> Basically,
>> the interface would look something like this:
>>
>> typedef opaque_mumble stdclosure_t;
>>
>> // Returns a callable function pointer or NULL on error with errno set
>> void* stdclosure_init(stdclosure_t* closure, void* fnptr, void* data);
>
> A void* might not be big enough for a callable function
> pointer an all architectures supported by C so this
> needs to return something like
>
> void (*)();
>
>
> (for POSIX this is not a problem)

Yeah, although at this point, I'm in favor of standardizing things that are 
de facto universal anyway. POSIX for example mandates that NULL is all zero 
bits and that function pointers fit in void*, and I think it's time for C 
to just acknowledge the same.


>
>
>> void stsclosure_destroy(stdclosure_t* closure);
>>
>> The function pointer returned by stdclosure_init would, when called, call
>> FNPTR with an extra initial parameter (or trailing parameter or something)
>> of type void* and value DATA and all the remaining parameters with which it
>> was called.
>>
>> This facility would actually probably have to be more complicated in real
>> life due to varying platform ABIs, but you get the idea. This is something
>> that could belong in the standard because it'd provide a generic
>> abstraction for something every platform can do but that every platform has
>> to do in a slightly different way.
>
> One problem could be that this requires generating
> trampolines at run-time (and it is not clear we
> can do this everywhere where C is supported) or
> having a fixed set of preallocated trampolines.

Or remapping a fixed set of trampolines, like libffi does on some 
platforms. The new stdclosure feature would have to be optional, yes, but 
in practice it would be supported everywhere people write programs except 
maybe a few microcontrollers and DSPs.

> The other limitation (which the new type would
> avoid) is the need for dynamic resource allocation,
> which is sometimes not desirable.
>
> But I would love to see such a proposal
> brought forward.
>
> Martin


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 18:17       ` Daniel Colascione
@ 2021-10-10 18:47         ` Martin Uecker
  2021-10-10 18:57           ` Daniel Colascione
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-10 18:47 UTC (permalink / raw)
  To: Daniel Colascione, Kaz Kylheku (libffi),
	Martin Uecker via Libffi-discuss

Am Sonntag, den 10.10.2021, 11:17 -0700 schrieb Daniel Colascione:
> On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com> wrote:
> 
> > Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
> > > On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
> > > <libffi-discuss@sourceware.org> wrote:
> > > 
> > > > Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
> > > > > On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> > > > > > Hi all,
> > > > > > 
> > > > > > I will propose a wide function pointer type (actually
> > > > > > a wide function type) to WG14 for C23 as a common
> > > > > > type for callbacks, closures, which now require an
> > > > > > additional void pointer argument in C APIs. This
> > > > > > is intended to be compatible with ABIs with now
> > > > > > use a static chain register.
> > > > > 
> > > > > Opposed. There is nothing wrong with separate arguments
> > > > > for function pointer and context.
> > > > 
> > > > Noted.  Your argument sbelow all boil down to the point
> > > > that there are cases where it might not be the ideal
> > > > choice. But nobody forces anyone to use it.
> > > "I like it" is not by itself a rationale for including something in a core
> > > language standard. This is not something that should be in C. Why,
> > > precisely, can't you just define a struct with two fields and make that
> > > your closure type?
> > 
> > Sure. There are at least five reasons:
> > 
> > - one now does has to do it manually which is
> > inconvenient because it has to be done for each type,
> > although this a common pattern which is trivial
> > to automate.
> > 
> > - APIs are often unsafe because they have to use
> > void pointer to be generic. When the void pointer
> > can be packaged into the wide pointer this problem
> > goes away.
> > 
> > - APIs are often different for no good reason.
> > Having a standard approach would make this more
> > canonical, hence simpler and less error prone,
> > and also more compatible - requiring less adapter
> > code.
> > 
> > - There are APIs where a pointer to a
> > function is expected but there is no data pointer.
> > Then other languages have a problem interfacing
> > with such APIs.   (or why does libffi generate
> > trampolines?)
> > 
> > - Finally, C might get lambda expressions
> > which are far more useful with such a pointer type.
> > (for nested function you could avoid executable
> > stack).
> > 
> > 
> > All five are good reasons to put a standard
> > approach in the core language in my opinion.
> > 
> > 
> > But I am actually not interested in this discussion
> > and do not have time for it.
> > 
> > I am more interested in constructive technical
> > feedback.
> 
> Your proposal doesn't actually solve any of these problems though, and as 
> we agree, users can already trivially make their own closure types.

I wonder why you think it does not address
these problems?

> What *would* be interesting is a new standard library facility for making 
> libffi-style closures and a standard struct type for packaging the things.

> *That's* something that could be justified for inclusion in a standard 
> library, since it's not something that programs can make on their own 
> without the kind of platform specific glue that libffi provides. 

I also would love to have this, but see below...

> Basically, 
> the interface would look something like this:
> 
> typedef opaque_mumble stdclosure_t;
> 
> // Returns a callable function pointer or NULL on error with errno set
> void* stdclosure_init(stdclosure_t* closure, void* fnptr, void* data);

A void* might not be big enough for a callable function
pointer an all architectures supported by C so this
needs to return something like

void (*)();


(for POSIX this is not a problem)

> void stsclosure_destroy(stdclosure_t* closure);
> 
> The function pointer returned by stdclosure_init would, when called, call 
> FNPTR with an extra initial parameter (or trailing parameter or something) 
> of type void* and value DATA and all the remaining parameters with which it 
> was called.
> 
> This facility would actually probably have to be more complicated in real 
> life due to varying platform ABIs, but you get the idea. This is something 
> that could belong in the standard because it'd provide a generic 
> abstraction for something every platform can do but that every platform has 
> to do in a slightly different way.

One problem could be that this requires generating
trampolines at run-time (and it is not clear we
can do this everywhere where C is supported) or
having a fixed set of preallocated trampolines.

The other limitation (which the new type would
avoid) is the need for dynamic resource allocation,
which is sometimes not desirable.

But I would love to see such a proposal
brought forward.

Martin




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 17:44 ` Martin Uecker
  2021-10-10 17:49   ` Daniel Colascione
@ 2021-10-10 18:31   ` Kaz Kylheku (libffi)
  1 sibling, 0 replies; 28+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-10-10 18:31 UTC (permalink / raw)
  To: Martin Uecker; +Cc: libffi-discuss

On 2021-10-10 10:44, Martin Uecker wrote:
> Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
>> On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
>> > Hi all,
>> >
>> > I will propose a wide function pointer type (actually
>> > a wide function type) to WG14 for C23 as a common
>> > type for callbacks, closures, which now require an
>> > additional void pointer argument in C APIs. This
>> > is intended to be compatible with ABIs with now
>> > use a static chain register.
>> 
>> Opposed. There is nothing wrong with separate arguments
>> for function pointer and context.
> 
> Noted.  Your argument sbelow all boil down to the point
> that there are cases where it might not be the ideal
> choice. But nobody forces anyone to use it.
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Marin, you should know very well that this argument holds
only for the lone hacker working by herself or himself.

I don't fault you; It has been trotted out by some great minds.

For instance, in some interview in the early 1990's, Knuth
made some nice remarks about the C language, and noted
that you can ignore the parts you don't like, or something
to that effect.(*)

If you're Knuth working by yourself on some great
all-American program, you can, for instance, declare that
the do/while statement is evil, and never use it.

Outside of that, there are multiple somebodies who effectively
force you to use something. You have to use something if
you work with other people's code where it is prevalent.

For instance it appears in legacy code you're working with
and is not easy to eliminate. Or in third party API's.

Moreover, if a feature is a standard part of the language, you
will face standard-backed, authoritarian arguments against
your preference of eschewing it.

Sometimes, it's really better to just nip it at the bud.

---
* http://softpanorama.org/People/Knuth/knuth_interview.html

   "Knuth: I think C has a lot of features that are very important.
    The way C handles pointers, for example, was a brilliant
    innovation; it solved a lot of problems that we had before
    in data structuring and made the programs look good afterwards.
    C isn't the perfect language, no language is, but I think it has
    a lot of virtues, and you can avoid the parts you don't like."

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 18:05     ` Martin Uecker
@ 2021-10-10 18:17       ` Daniel Colascione
  2021-10-10 18:47         ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Daniel Colascione @ 2021-10-10 18:17 UTC (permalink / raw)
  To: Martin Uecker, Kaz Kylheku (libffi), Martin Uecker via Libffi-discuss

On October 10, 2021 11:05:07 Martin Uecker <ma.uecker@gmail.com> wrote:

> Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
>> On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss
>> <libffi-discuss@sourceware.org> wrote:
>>
>>> Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
>>>> On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
>>>>> Hi all,
>>>>>
>>>>> I will propose a wide function pointer type (actually
>>>>> a wide function type) to WG14 for C23 as a common
>>>>> type for callbacks, closures, which now require an
>>>>> additional void pointer argument in C APIs. This
>>>>> is intended to be compatible with ABIs with now
>>>>> use a static chain register.
>>>>
>>>> Opposed. There is nothing wrong with separate arguments
>>>> for function pointer and context.
>>>
>>> Noted.  Your argument sbelow all boil down to the point
>>> that there are cases where it might not be the ideal
>>> choice. But nobody forces anyone to use it.
>> "I like it" is not by itself a rationale for including something in a core
>> language standard. This is not something that should be in C. Why,
>> precisely, can't you just define a struct with two fields and make that
>> your closure type?
>
> Sure. There are at least five reasons:
>
> - one now does has to do it manually which is
> inconvenient because it has to be done for each type,
> although this a common pattern which is trivial
> to automate.
>
> - APIs are often unsafe because they have to use
> void pointer to be generic. When the void pointer
> can be packaged into the wide pointer this problem
> goes away.
>
> - APIs are often different for no good reason.
> Having a standard approach would make this more
> canonical, hence simpler and less error prone,
> and also more compatible - requiring less adapter
> code.
>
> - There are APIs where a pointer to a
> function is expected but there is no data pointer.
> Then other languages have a problem interfacing
> with such APIs.   (or why does libffi generate
> trampolines?)
>
> - Finally, C might get lambda expressions
> which are far more useful with such a pointer type.
> (for nested function you could avoid executable
> stack).
>
>
> All five are good reasons to put a standard
> approach in the core language in my opinion.
>
>
> But I am actually not interested in this discussion
> and do not have time for it.
>
> I am more interested in constructive technical
> feedback.

Your proposal doesn't actually solve any of these problems though, and as 
we agree, users can already trivially make their own closure types.

What *would* be interesting is a new standard library facility for making 
libffi-style closures and a standard struct type for packaging the things. 
*That's* something that could be justified for inclusion in a standard 
library, since it's not something that programs can make on their own 
without the kind of platform specific glue that libffi provides. Basically, 
the interface would look something like this:

typedef opaque_mumble stdclosure_t;

// Returns a callable function pointer or NULL on error with errno set
void* stdclosure_init(stdclosure_t* closure, void* fnptr, void* data);

void stsclosure_destroy(stdclosure_t* closure);

The function pointer returned by stdclosure_init would, when called, call 
FNPTR with an extra initial parameter (or trailing parameter or something) 
of type void* and value DATA and all the remaining parameters with which it 
was called.

This facility would actually probably have to be more complicated in real 
life due to varying platform ABIs, but you get the idea. This is something 
that could belong in the standard because it'd provide a generic 
abstraction for something every platform can do but that every platform has 
to do in a slightly different way.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 17:49   ` Daniel Colascione
@ 2021-10-10 18:05     ` Martin Uecker
  2021-10-10 18:17       ` Daniel Colascione
  0 siblings, 1 reply; 28+ messages in thread
From: Martin Uecker @ 2021-10-10 18:05 UTC (permalink / raw)
  To: Daniel Colascione, Kaz Kylheku (libffi),
	Martin Uecker via Libffi-discuss

Am Sonntag, den 10.10.2021, 10:49 -0700 schrieb Daniel Colascione:
> On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss 
> <libffi-discuss@sourceware.org> wrote:
> 
> > Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
> > > On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> > > > Hi all,
> > > > 
> > > > I will propose a wide function pointer type (actually
> > > > a wide function type) to WG14 for C23 as a common
> > > > type for callbacks, closures, which now require an
> > > > additional void pointer argument in C APIs. This
> > > > is intended to be compatible with ABIs with now
> > > > use a static chain register.
> > > 
> > > Opposed. There is nothing wrong with separate arguments
> > > for function pointer and context.
> > 
> > Noted.  Your argument sbelow all boil down to the point
> > that there are cases where it might not be the ideal
> > choice. But nobody forces anyone to use it.
> "I like it" is not by itself a rationale for including something in a core 
> language standard. This is not something that should be in C. Why, 
> precisely, can't you just define a struct with two fields and make that 
> your closure type?

Sure. There are at least five reasons:

 - one now does has to do it manually which is
inconvenient because it has to be done for each type,
although this a common pattern which is trivial
to automate. 

- APIs are often unsafe because they have to use
void pointer to be generic. When the void pointer
can be packaged into the wide pointer this problem
goes away.

- APIs are often different for no good reason.
Having a standard approach would make this more
canonical, hence simpler and less error prone,
and also more compatible - requiring less adapter
code.

- There are APIs where a pointer to a
function is expected but there is no data pointer.
Then other languages have a problem interfacing
with such APIs.   (or why does libffi generate
trampolines?)

- Finally, C might get lambda expressions
which are far more useful with such a pointer type.
(for nested function you could avoid executable
stack).


All five are good reasons to put a standard
approach in the core language in my opinion.


But I am actually not interested in this discussion
and do not have time for it.

I am more interested in constructive technical
feedback. 

Martin



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 17:44 ` Martin Uecker
@ 2021-10-10 17:49   ` Daniel Colascione
  2021-10-10 18:05     ` Martin Uecker
  2021-10-10 18:31   ` Kaz Kylheku (libffi)
  1 sibling, 1 reply; 28+ messages in thread
From: Daniel Colascione @ 2021-10-10 17:49 UTC (permalink / raw)
  To: Martin Uecker, Kaz Kylheku (libffi), Martin Uecker via Libffi-discuss
  Cc: libffi-discuss

On October 10, 2021 10:44:32 Martin Uecker via Libffi-discuss 
<libffi-discuss@sourceware.org> wrote:

> Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
>> On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
>>> Hi all,
>>>
>>> I will propose a wide function pointer type (actually
>>> a wide function type) to WG14 for C23 as a common
>>> type for callbacks, closures, which now require an
>>> additional void pointer argument in C APIs. This
>>> is intended to be compatible with ABIs with now
>>> use a static chain register.
>>
>> Opposed. There is nothing wrong with separate arguments
>> for function pointer and context.
>
> Noted.  Your argument sbelow all boil down to the point
> that there are cases where it might not be the ideal
> choice. But nobody forces anyone to use it.


>
"I like it" is not by itself a rationale for including something in a core 
language standard. This is not something that should be in C. Why, 
precisely, can't you just define a struct with two fields and make that 
your closure type?

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
  2021-10-10 17:01 Kaz Kylheku (libffi)
@ 2021-10-10 17:44 ` Martin Uecker
  2021-10-10 17:49   ` Daniel Colascione
  2021-10-10 18:31   ` Kaz Kylheku (libffi)
  0 siblings, 2 replies; 28+ messages in thread
From: Martin Uecker @ 2021-10-10 17:44 UTC (permalink / raw)
  To: Kaz Kylheku (libffi); +Cc: libffi-discuss

Am Sonntag, den 10.10.2021, 10:01 -0700 schrieb Kaz Kylheku (libffi):
> On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> > Hi all,
> > 
> > I will propose a wide function pointer type (actually
> > a wide function type) to WG14 for C23 as a common
> > type for callbacks, closures, which now require an
> > additional void pointer argument in C APIs. This
> > is intended to be compatible with ABIs with now
> > use a static chain register.
> 
> Opposed. There is nothing wrong with separate arguments
> for function pointer and context.

Noted.  Your argument sbelow all boil down to the point
that there are cases where it might not be the ideal
choice. But nobody forces anyone to use it.

Martin


> Higher level languages that interface C or are built on C
> do not agree on an exact representation for closures;
> it's not a good idea to second guess them. Many languages
> are perfectly able, in their implementations, to represent
> function callable objects of all sorts using ordinary
> pointer-sized reference values. What those values refer to
> varies quite a bit.
> 
> Callback mechanisms are often bound to OOP-style objects
> rather than closures. The same context pointer is
> associated with more than one callback.
> 
> The context argument has the right flexibility to cover
> all the situations that arise.
> 
> In some library API's, the object pointer is registered
> separately, and then other functions implicitly use it
> when invoking callbacks, so it doesn't have to be passed
> any more.
> 
> Sometimes there are multiple callbacks, but one context
> pointer, passed in one call:
> 
>    initiate_operation(arg, callback_context,
>                       get_data_fn, progress_fn);
> 
> Should there be a triple wide pointer just to roll three
> arguments into one?
> 


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: wide function pointer type
@ 2021-10-10 17:01 Kaz Kylheku (libffi)
  2021-10-10 17:44 ` Martin Uecker
  0 siblings, 1 reply; 28+ messages in thread
From: Kaz Kylheku (libffi) @ 2021-10-10 17:01 UTC (permalink / raw)
  To: Martin Uecker; +Cc: libffi-discuss

On 2021-10-10 04:32, Martin Uecker via Libffi-discuss wrote:
> Hi all,
> 
> I will propose a wide function pointer type (actually
> a wide function type) to WG14 for C23 as a common
> type for callbacks, closures, which now require an
> additional void pointer argument in C APIs. This
> is intended to be compatible with ABIs with now
> use a static chain register.

Opposed. There is nothing wrong with separate arguments
for function pointer and context.

Higher level languages that interface C or are built on C
do not agree on an exact representation for closures;
it's not a good idea to second guess them. Many languages
are perfectly able, in their implementations, to represent
function callable objects of all sorts using ordinary
pointer-sized reference values. What those values refer to
varies quite a bit.

Callback mechanisms are often bound to OOP-style objects
rather than closures. The same context pointer is
associated with more than one callback.

The context argument has the right flexibility to cover
all the situations that arise.

In some library API's, the object pointer is registered
separately, and then other functions implicitly use it
when invoking callbacks, so it doesn't have to be passed
any more.

Sometimes there are multiple callbacks, but one context
pointer, passed in one call:

   initiate_operation(arg, callback_context,
                      get_data_fn, progress_fn);

Should there be a triple wide pointer just to roll three
arguments into one?


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2021-10-21  9:48 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-10 11:32 wide function pointer type Martin Uecker
2021-10-17 23:35 ` Anthony Green
2021-10-18  5:33   ` Martin Uecker
2021-10-18  5:58     ` Martin Uecker
2021-10-18  7:36       ` Florian Weimer
2021-10-18  7:56         ` Martin Uecker
2021-10-19  9:22           ` Florian Weimer
2021-10-19  9:43             ` Martin Uecker
2021-10-19 10:15               ` Florian Weimer
2021-10-19 12:13                 ` Martin Uecker
2021-10-20  8:24                   ` Kaz Kylheku (libffi)
2021-10-20 18:52                     ` Martin Uecker
2021-10-20  9:10                   ` Florian Weimer
2021-10-20  9:21                     ` Martin Uecker
2021-10-20  9:27                       ` Florian Weimer
2021-10-20 17:27                     ` Kaz Kylheku (libffi)
2021-10-21  9:48                       ` Florian Weimer
2021-10-10 17:01 Kaz Kylheku (libffi)
2021-10-10 17:44 ` Martin Uecker
2021-10-10 17:49   ` Daniel Colascione
2021-10-10 18:05     ` Martin Uecker
2021-10-10 18:17       ` Daniel Colascione
2021-10-10 18:47         ` Martin Uecker
2021-10-10 18:57           ` Daniel Colascione
2021-10-10 19:24             ` Martin Uecker
2021-10-16  8:08               ` Jarkko Hietaniemi
2021-10-16  9:35                 ` Jarkko Hietaniemi
2021-10-10 18:31   ` Kaz Kylheku (libffi)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).