public inbox for libc-help@sourceware.org
 help / color / mirror / Atom feed
* Re-port Intent for HPE NonStop (a.k.a. Tandem)
@ 2022-03-13 12:43 rsbecker
  2022-03-14  7:54 ` Florian Weimer
  0 siblings, 1 reply; 9+ messages in thread
From: rsbecker @ 2022-03-13 12:43 UTC (permalink / raw)
  To: libc-help; +Cc: Bill Honaker, jojo, Shiva Subramanian, mike_kilpatrick

Hi Glibc Team,

I have been asked by members of the HPE NonStop (formerly Tandem) community
to consider re-porting Glibc to the platform. There was a port back around
2.12 but that never made it to x86. The current situation is that the
platform is not on the list of known ports. Glibc is a dependency of many
projects that we want to bring to the platform. How can this be pursued? My
team is willing to officially support the port once done. Although POSIX
compliant, we are limited to using a c99 compiler. gcc does not port, making
this interesting.

Thanks for any pointers.

Note: The link http://www.gnu.org/software/libc/development.html on your
wiki page https://sourceware.org/glibc/wiki/#Development gets a 404.

Regards,
Randall

--
Brief whoami: NonStop&UNIX developer since approximately
UNIX(421664400)
NonStop(211288444200000000)
-- In real life, I talk too much.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-13 12:43 Re-port Intent for HPE NonStop (a.k.a. Tandem) rsbecker
@ 2022-03-14  7:54 ` Florian Weimer
  2022-03-14  8:12   ` AW: " Joachim Schmitz
  2022-03-14 12:48   ` rsbecker
  0 siblings, 2 replies; 9+ messages in thread
From: Florian Weimer @ 2022-03-14  7:54 UTC (permalink / raw)
  To: rsbecker
  Cc: libc-help, mike_kilpatrick, Shiva Subramanian, Bill Honaker, jojo

> I have been asked by members of the HPE NonStop (formerly Tandem) community
> to consider re-porting Glibc to the platform. There was a port back around
> 2.12 but that never made it to x86.

It must have been a private port, not released to the general public.

Does HPE NonStop provide a stable kernel interface?  Linux is somewhat
unique in that.  Other systems require some level of synchronization
between libc and kernel updates.

> The current situation is that the platform is not on the list of known
> ports. Glibc is a dependency of many projects that we want to bring to
> the platform. How can this be pursued? My team is willing to
> officially support the port once done. Although POSIX compliant, we
> are limited to using a c99 compiler. gcc does not port, making this
> interesting.

Can you use a GCC cross-compiler?

We need a publicly available, free (libre) toolchain for any port.  This
is both a matter of policy, and also to prevent the port from
bit-rotting.

> Note: The link http://www.gnu.org/software/libc/development.html on your
> wiki page https://sourceware.org/glibc/wiki/#Development gets a 404.

I've removed this section because it is outdated.  I'm not sure if we
would accept ports to new kernels at this point.  (New targets is a
different matter.)

Thanks,
Florian


^ permalink raw reply	[flat|nested] 9+ messages in thread

* AW: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14  7:54 ` Florian Weimer
@ 2022-03-14  8:12   ` Joachim Schmitz
  2022-03-14  8:32     ` Florian Weimer
  2022-03-14 12:48   ` rsbecker
  1 sibling, 1 reply; 9+ messages in thread
From: Joachim Schmitz @ 2022-03-14  8:12 UTC (permalink / raw)
  To: 'Florian Weimer', rsbecker
  Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker'

Hi Florian

No idea whether that port had ever been send back upstream, it must have
been done in 2007 or before.
Not sure what you mean by "stable kernel interface"?
No, sorry, unfortunately no gcc available nor possible to cross-compile, it
is actually the assembler part that is missing IIRC
While gcc may be free (libre) it is also provides quite a few non-standard
features. Our compilers do have some extensions too, but generally are
pretty strictly C89/90, C99 compliant.

Bye, Jojo

-----Ursprüngliche Nachricht-----
Von: Florian Weimer <fweimer@redhat.com> 
Gesendet: Montag, 14. März 2022 08:55
An: rsbecker@nexbridge.com
Cc: libc-help@sourceware.org; mike_kilpatrick@nskopensource.com; Shiva
Subramanian <shiva.subramanian@tcm.uk.com>; Bill Honaker <bhonaker@xid.com>;
jojo@schmitz-digital.de
Betreff: Re: Re-port Intent for HPE NonStop (a.k.a. Tandem)

> I have been asked by members of the HPE NonStop (formerly Tandem) 
> community to consider re-porting Glibc to the platform. There was a 
> port back around
> 2.12 but that never made it to x86.

It must have been a private port, not released to the general public.

Does HPE NonStop provide a stable kernel interface?  Linux is somewhat
unique in that.  Other systems require some level of synchronization between
libc and kernel updates.

> The current situation is that the platform is not on the list of known 
> ports. Glibc is a dependency of many projects that we want to bring to 
> the platform. How can this be pursued? My team is willing to 
> officially support the port once done. Although POSIX compliant, we 
> are limited to using a c99 compiler. gcc does not port, making this 
> interesting.

Can you use a GCC cross-compiler?

We need a publicly available, free (libre) toolchain for any port.  This is
both a matter of policy, and also to prevent the port from bit-rotting.

> Note: The link http://www.gnu.org/software/libc/development.html on 
> your wiki page https://sourceware.org/glibc/wiki/#Development gets a 404.

I've removed this section because it is outdated.  I'm not sure if we would
accept ports to new kernels at this point.  (New targets is a different
matter.)

Thanks,
Florian



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14  8:12   ` AW: " Joachim Schmitz
@ 2022-03-14  8:32     ` Florian Weimer
  2022-03-14  8:49       ` AW: " Joachim Schmitz
  0 siblings, 1 reply; 9+ messages in thread
From: Florian Weimer @ 2022-03-14  8:32 UTC (permalink / raw)
  To: Joachim Schmitz
  Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker'

* Joachim Schmitz:

> No idea whether that port had ever been send back upstream, it must have
> been done in 2007 or before.
> Not sure what you mean by "stable kernel interface"?

In general, Linux does not remove any interfaces for entering the kernel
once they have been added.  The system call numbers that identify the
entry points also does not change over time.  For example, on x86-64,
there is still a fstat system call, even though it has been superseded
by newfstatat first, and then by statx.

Other operating systems treat libc has the stable binary interface (if
they have one at all), and remove the old system calls from their
kernels once libc has been upgraded.  This means it is not feasible to
maintain libc as a separate project.

> No, sorry, unfortunately no gcc available nor possible to cross-compile, it
> is actually the assembler part that is missing IIRC

You will have to release that first, preferably as a new target for
binutils or LLVM, and then a compiler.  But again, even if you do all
that, it is unlikely that we would accept a glibc port.  But without the
toolchain, there is simply no way that it can happen.  I think glibc
stopped supporting proprietary toolchains some time in the 90s, before
the glibc 2.0 release, and we are definitely not going back.  Sorry.

If you need a glibc compatibility layer, maybe you should look at gnulib
instead.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 9+ messages in thread

* AW: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14  8:32     ` Florian Weimer
@ 2022-03-14  8:49       ` Joachim Schmitz
  2022-03-14  8:55         ` Florian Weimer
  0 siblings, 1 reply; 9+ messages in thread
From: Joachim Schmitz @ 2022-03-14  8:49 UTC (permalink / raw)
  To: 'Florian Weimer'
  Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker'

Hi Florian


OK, then in that sense NonStop Kernel is stable too

Not sure what you'd expect us to "release first", out compiler?
If so: sorry, that won't happen. We don't even own it ourself.
If we were able to port gcc, I believe we'd not be allowed to share it, due
to proprietary code needed inside, needed to support the extension out
platform needs.

If glibc can't be made POSIX complaint, then that's where everything is lost
for us, as far as I can tell.

But yes, we're using gnulib in places, and on top have our own compatibility
layer at some places.

Bye, Jojo

-----Ursprüngliche Nachricht-----
Von: Florian Weimer <fweimer@redhat.com> 
Gesendet: Montag, 14. März 2022 09:33
An: Joachim Schmitz <jojo@schmitz-digital.de>
Cc: rsbecker@nexbridge.com; libc-help@sourceware.org;
mike_kilpatrick@nskopensource.com; 'Shiva Subramanian'
<shiva.subramanian@tcm.uk.com>; 'Bill Honaker' <bhonaker@xid.com>
Betreff: Re: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem)

* Joachim Schmitz:

> No idea whether that port had ever been send back upstream, it must 
> have been done in 2007 or before.
> Not sure what you mean by "stable kernel interface"?

In general, Linux does not remove any interfaces for entering the kernel
once they have been added.  The system call numbers that identify the entry
points also does not change over time.  For example, on x86-64, there is
still a fstat system call, even though it has been superseded by newfstatat
first, and then by statx.

Other operating systems treat libc has the stable binary interface (if they
have one at all), and remove the old system calls from their kernels once
libc has been upgraded.  This means it is not feasible to maintain libc as a
separate project.

> No, sorry, unfortunately no gcc available nor possible to 
> cross-compile, it is actually the assembler part that is missing IIRC

You will have to release that first, preferably as a new target for binutils
or LLVM, and then a compiler.  But again, even if you do all that, it is
unlikely that we would accept a glibc port.  But without the toolchain,
there is simply no way that it can happen.  I think glibc stopped supporting
proprietary toolchains some time in the 90s, before the glibc 2.0 release,
and we are definitely not going back.  Sorry.

If you need a glibc compatibility layer, maybe you should look at gnulib
instead.

Thanks,
Florian



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: AW: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14  8:49       ` AW: " Joachim Schmitz
@ 2022-03-14  8:55         ` Florian Weimer
  0 siblings, 0 replies; 9+ messages in thread
From: Florian Weimer @ 2022-03-14  8:55 UTC (permalink / raw)
  To: Joachim Schmitz
  Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker'

* Joachim Schmitz:

> Not sure what you'd expect us to "release first", out compiler?
> If so: sorry, that won't happen. We don't even own it ourself.
> If we were able to port gcc, I believe we'd not be allowed to share it, due
> to proprietary code needed inside, needed to support the extension out
> platform needs.

Understood, but if you want to even have to a chance at an official
glibc port, you'd have to address these issues somehow.

> If glibc can't be made POSIX complaint, then that's where everything is lost
> for us, as far as I can tell.

There might be a misunderstanding here.  glibc is not layered on top of
a POSIX interface, it itself is the POSIX interface for GNU systems.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 9+ messages in thread

* RE: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14  7:54 ` Florian Weimer
  2022-03-14  8:12   ` AW: " Joachim Schmitz
@ 2022-03-14 12:48   ` rsbecker
  2022-03-14 13:12     ` Florian Weimer
  1 sibling, 1 reply; 9+ messages in thread
From: rsbecker @ 2022-03-14 12:48 UTC (permalink / raw)
  To: 'Florian Weimer'
  Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker',
	jojo

On March 14, 2022 3:55 AM, Florian Weimer wrote:
>> I have been asked by members of the HPE NonStop (formerly Tandem)
>> community to consider re-porting Glibc to the platform. There was a
>> port back around
>> 2.12 but that never made it to x86.
>
>It must have been a private port, not released to the general public.

The port was released to the public, but not through standard glibc
mechanisms. Both binaries and sources were available to the NonStop
community.

>Does HPE NonStop provide a stable kernel interface?  Linux is somewhat
unique in
>that.  Other systems require some level of synchronization between libc and
>kernel updates.

Yes. The kernel is highly stable, with HPE providing long-term compatibility
statements. The POSIX components date back to 1995 and continuously
upgraded. To date, source compatibility has been maintained even across
chipset changes.

>> The current situation is that the platform is not on the list of known
>> ports. Glibc is a dependency of many projects that we want to bring to
>> the platform. How can this be pursued? My team is willing to
>> officially support the port once done. Although POSIX compliant, we
>> are limited to using a c99 compiler. gcc does not port, making this
>> interesting.
>
>Can you use a GCC cross-compiler?

No. GCC does not generate code that will run on the platform. While it is
x86-based, there are complex endian concerns in the code generator,
significant ELF header differences, and major ELF debug section differences
that cannot be emitted by GCC. There have been at least 3 unsuccessful
attempts I know of to port gcc to the platform and/or to generate cross
compile code for it.

>We need a publicly available, free (libre) toolchain for any port.  This is
both a
>matter of policy, and also to prevent the port from bit-rotting.

I have successfully done ports of other products with the same constraint
(git, openssh, openssl). Usually ports are possible with minimal changes. I
would like to do a bit of an impact analysis to see how difficult this one
would be. Glibc is an ongoing concern because it is a dependency for many
Open-Source projects.

>> Note: The link http://www.gnu.org/software/libc/development.html on
>> your wiki page https://sourceware.org/glibc/wiki/#Development gets a 404.
>
>I've removed this section because it is outdated.  I'm not sure if we would
accept
>ports to new kernels at this point.  (New targets is a different matter.)

How should I proceed, if I need glibc as a dependency?

Thanks,
Randall


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14 12:48   ` rsbecker
@ 2022-03-14 13:12     ` Florian Weimer
  2022-03-14 14:20       ` Adhemerval Zanella
  0 siblings, 1 reply; 9+ messages in thread
From: Florian Weimer @ 2022-03-14 13:12 UTC (permalink / raw)
  To: rsbecker
  Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian',
	'Bill Honaker',
	jojo

>>Can you use a GCC cross-compiler?
>
> No. GCC does not generate code that will run on the platform. While it is
> x86-based, there are complex endian concerns in the code generator,
> significant ELF header differences, and major ELF debug section differences
> that cannot be emitted by GCC. There have been at least 3 unsuccessful
> attempts I know of to port gcc to the platform and/or to generate cross
> compile code for it.

In this case, we would really need an open toolchain, otherwise we can't
maintain the ELF parts.

Keep in mind that glibc contains the dynamic linker, so it really has to
know about the details of your architecture.

> I have successfully done ports of other products with the same constraint
> (git, openssh, openssl). Usually ports are possible with minimal
> changes.

That's because you rely on the C run-time library to paper over the
differences in kernel interfaces.  With glibc itself, that's different.

> I would like to do a bit of an impact analysis to see how difficult
> this one would be.

You need to provide the rest of the toolchain first, and that seems to
require a major effort (and it's not just about writing code).

I also doubt that we would accept patches which retarget to a
proprietary kernel, but that depends on how invasive those patches are.

> How should I proceed, if I need glibc as a dependency?

You need to port GCC and binutils first, and then you can proceed with
glibc.  It's how new ports are brought up.  I'm not aware of any other
port in recent times that has been done in a different way (not using
GCC).  This is not me being snarky, it's just that the GNU toolchain
(binutils/GCC/GDB/glibc) is an integrated whole and works best in
conjunction.

It might be possible to get away with an LLVM-based toolchain, but the
porting of glibc istelf to LLVM is not yet complete, so you would
struggle on this front as well.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem)
  2022-03-14 13:12     ` Florian Weimer
@ 2022-03-14 14:20       ` Adhemerval Zanella
  0 siblings, 0 replies; 9+ messages in thread
From: Adhemerval Zanella @ 2022-03-14 14:20 UTC (permalink / raw)
  To: Florian Weimer, rsbecker
  Cc: libc-help, mike_kilpatrick, 'Bill Honaker',
	jojo, 'Shiva Subramanian'



On 14/03/2022 10:12, Florian Weimer via Libc-help wrote:
>>> Can you use a GCC cross-compiler?
>>
>> No. GCC does not generate code that will run on the platform. While it is
>> x86-based, there are complex endian concerns in the code generator,
>> significant ELF header differences, and major ELF debug section differences
>> that cannot be emitted by GCC. There have been at least 3 unsuccessful
>> attempts I know of to port gcc to the platform and/or to generate cross
>> compile code for it.
> 
> In this case, we would really need an open toolchain, otherwise we can't
> maintain the ELF parts.
> 
> Keep in mind that glibc contains the dynamic linker, so it really has to
> know about the details of your architecture.
> 
>> I have successfully done ports of other products with the same constraint
>> (git, openssh, openssl). Usually ports are possible with minimal
>> changes.
> 
> That's because you rely on the C run-time library to paper over the
> differences in kernel interfaces.  With glibc itself, that's different.
> 
>> I would like to do a bit of an impact analysis to see how difficult
>> this one would be.
> 
> You need to provide the rest of the toolchain first, and that seems to
> require a major effort (and it's not just about writing code).
> 
> I also doubt that we would accept patches which retarget to a
> proprietary kernel, but that depends on how invasive those patches are.
> 
>> How should I proceed, if I need glibc as a dependency?
> 
> You need to port GCC and binutils first, and then you can proceed with
> glibc.  It's how new ports are brought up.  I'm not aware of any other
> port in recent times that has been done in a different way (not using
> GCC).  This is not me being snarky, it's just that the GNU toolchain
> (binutils/GCC/GDB/glibc) is an integrated whole and works best in
> conjunction.

Besides all Florian has pointed out, I am inclined to say that porting 
glibc to a non opensource architecture and also without aiming to be the 
'de facto' loader/libc of the system is a dead-end. Without open-source
support for at least the toolchain pieces, it is really a no-go.

Also, besides probably requiring a lot of work to support any ELF and/or 
ABI extension, the resulting loader won't easily support the system shared
libraries without a compat layer (which imposes another set of issues).
Although there are some examples where it has been done with some relative
success (such as gcompat for musl, or linuxabi for FreeBSD) it a relative
time consuming effort where maintaining this out of tree usually ends up
as a bitrotten project (as we saw for prelink, which was removed recently).

Also, most of the generic interfaces glibc provide is already provided 
by gnulib in a extent. There are other interface that are reliant on proper
kernel support, so using a different kernel might ended required *a lot* of
glue code and, worse, some interfaces might not be actually be proper
implemented.

The truth is glibc is currently Linux oriented projected, there are some
Hurd work but it is moving to nowhere (it barely supports one legacy
architecture).

> 
> It might be possible to get away with an LLVM-based toolchain, but the
> porting of glibc istelf to LLVM is not yet complete, so you would
> struggle on this front as well.
> 

I am working to get glibc in a somewhat usable state with clang [1].
It still requires a *lot* of patches and I still struggling in the
loader bootstrap, but at least it *builds*.

[1] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/clang

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-03-14 14:20 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-13 12:43 Re-port Intent for HPE NonStop (a.k.a. Tandem) rsbecker
2022-03-14  7:54 ` Florian Weimer
2022-03-14  8:12   ` AW: " Joachim Schmitz
2022-03-14  8:32     ` Florian Weimer
2022-03-14  8:49       ` AW: " Joachim Schmitz
2022-03-14  8:55         ` Florian Weimer
2022-03-14 12:48   ` rsbecker
2022-03-14 13:12     ` Florian Weimer
2022-03-14 14:20       ` Adhemerval Zanella

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).