* Re-port Intent for HPE NonStop (a.k.a. Tandem) @ 2022-03-13 12:43 rsbecker 2022-03-14 7:54 ` Florian Weimer 0 siblings, 1 reply; 9+ messages in thread From: rsbecker @ 2022-03-13 12:43 UTC (permalink / raw) To: libc-help; +Cc: Bill Honaker, jojo, Shiva Subramanian, mike_kilpatrick Hi Glibc Team, I have been asked by members of the HPE NonStop (formerly Tandem) community to consider re-porting Glibc to the platform. There was a port back around 2.12 but that never made it to x86. The current situation is that the platform is not on the list of known ports. Glibc is a dependency of many projects that we want to bring to the platform. How can this be pursued? My team is willing to officially support the port once done. Although POSIX compliant, we are limited to using a c99 compiler. gcc does not port, making this interesting. Thanks for any pointers. Note: The link http://www.gnu.org/software/libc/development.html on your wiki page https://sourceware.org/glibc/wiki/#Development gets a 404. Regards, Randall -- Brief whoami: NonStop&UNIX developer since approximately UNIX(421664400) NonStop(211288444200000000) -- In real life, I talk too much. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-13 12:43 Re-port Intent for HPE NonStop (a.k.a. Tandem) rsbecker @ 2022-03-14 7:54 ` Florian Weimer 2022-03-14 8:12 ` AW: " Joachim Schmitz 2022-03-14 12:48 ` rsbecker 0 siblings, 2 replies; 9+ messages in thread From: Florian Weimer @ 2022-03-14 7:54 UTC (permalink / raw) To: rsbecker Cc: libc-help, mike_kilpatrick, Shiva Subramanian, Bill Honaker, jojo > I have been asked by members of the HPE NonStop (formerly Tandem) community > to consider re-porting Glibc to the platform. There was a port back around > 2.12 but that never made it to x86. It must have been a private port, not released to the general public. Does HPE NonStop provide a stable kernel interface? Linux is somewhat unique in that. Other systems require some level of synchronization between libc and kernel updates. > The current situation is that the platform is not on the list of known > ports. Glibc is a dependency of many projects that we want to bring to > the platform. How can this be pursued? My team is willing to > officially support the port once done. Although POSIX compliant, we > are limited to using a c99 compiler. gcc does not port, making this > interesting. Can you use a GCC cross-compiler? We need a publicly available, free (libre) toolchain for any port. This is both a matter of policy, and also to prevent the port from bit-rotting. > Note: The link http://www.gnu.org/software/libc/development.html on your > wiki page https://sourceware.org/glibc/wiki/#Development gets a 404. I've removed this section because it is outdated. I'm not sure if we would accept ports to new kernels at this point. (New targets is a different matter.) Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* AW: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 7:54 ` Florian Weimer @ 2022-03-14 8:12 ` Joachim Schmitz 2022-03-14 8:32 ` Florian Weimer 2022-03-14 12:48 ` rsbecker 1 sibling, 1 reply; 9+ messages in thread From: Joachim Schmitz @ 2022-03-14 8:12 UTC (permalink / raw) To: 'Florian Weimer', rsbecker Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker' Hi Florian No idea whether that port had ever been send back upstream, it must have been done in 2007 or before. Not sure what you mean by "stable kernel interface"? No, sorry, unfortunately no gcc available nor possible to cross-compile, it is actually the assembler part that is missing IIRC While gcc may be free (libre) it is also provides quite a few non-standard features. Our compilers do have some extensions too, but generally are pretty strictly C89/90, C99 compliant. Bye, Jojo -----Ursprüngliche Nachricht----- Von: Florian Weimer <fweimer@redhat.com> Gesendet: Montag, 14. März 2022 08:55 An: rsbecker@nexbridge.com Cc: libc-help@sourceware.org; mike_kilpatrick@nskopensource.com; Shiva Subramanian <shiva.subramanian@tcm.uk.com>; Bill Honaker <bhonaker@xid.com>; jojo@schmitz-digital.de Betreff: Re: Re-port Intent for HPE NonStop (a.k.a. Tandem) > I have been asked by members of the HPE NonStop (formerly Tandem) > community to consider re-porting Glibc to the platform. There was a > port back around > 2.12 but that never made it to x86. It must have been a private port, not released to the general public. Does HPE NonStop provide a stable kernel interface? Linux is somewhat unique in that. Other systems require some level of synchronization between libc and kernel updates. > The current situation is that the platform is not on the list of known > ports. Glibc is a dependency of many projects that we want to bring to > the platform. How can this be pursued? My team is willing to > officially support the port once done. Although POSIX compliant, we > are limited to using a c99 compiler. gcc does not port, making this > interesting. Can you use a GCC cross-compiler? We need a publicly available, free (libre) toolchain for any port. This is both a matter of policy, and also to prevent the port from bit-rotting. > Note: The link http://www.gnu.org/software/libc/development.html on > your wiki page https://sourceware.org/glibc/wiki/#Development gets a 404. I've removed this section because it is outdated. I'm not sure if we would accept ports to new kernels at this point. (New targets is a different matter.) Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 8:12 ` AW: " Joachim Schmitz @ 2022-03-14 8:32 ` Florian Weimer 2022-03-14 8:49 ` AW: " Joachim Schmitz 0 siblings, 1 reply; 9+ messages in thread From: Florian Weimer @ 2022-03-14 8:32 UTC (permalink / raw) To: Joachim Schmitz Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker' * Joachim Schmitz: > No idea whether that port had ever been send back upstream, it must have > been done in 2007 or before. > Not sure what you mean by "stable kernel interface"? In general, Linux does not remove any interfaces for entering the kernel once they have been added. The system call numbers that identify the entry points also does not change over time. For example, on x86-64, there is still a fstat system call, even though it has been superseded by newfstatat first, and then by statx. Other operating systems treat libc has the stable binary interface (if they have one at all), and remove the old system calls from their kernels once libc has been upgraded. This means it is not feasible to maintain libc as a separate project. > No, sorry, unfortunately no gcc available nor possible to cross-compile, it > is actually the assembler part that is missing IIRC You will have to release that first, preferably as a new target for binutils or LLVM, and then a compiler. But again, even if you do all that, it is unlikely that we would accept a glibc port. But without the toolchain, there is simply no way that it can happen. I think glibc stopped supporting proprietary toolchains some time in the 90s, before the glibc 2.0 release, and we are definitely not going back. Sorry. If you need a glibc compatibility layer, maybe you should look at gnulib instead. Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* AW: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 8:32 ` Florian Weimer @ 2022-03-14 8:49 ` Joachim Schmitz 2022-03-14 8:55 ` Florian Weimer 0 siblings, 1 reply; 9+ messages in thread From: Joachim Schmitz @ 2022-03-14 8:49 UTC (permalink / raw) To: 'Florian Weimer' Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker' Hi Florian OK, then in that sense NonStop Kernel is stable too Not sure what you'd expect us to "release first", out compiler? If so: sorry, that won't happen. We don't even own it ourself. If we were able to port gcc, I believe we'd not be allowed to share it, due to proprietary code needed inside, needed to support the extension out platform needs. If glibc can't be made POSIX complaint, then that's where everything is lost for us, as far as I can tell. But yes, we're using gnulib in places, and on top have our own compatibility layer at some places. Bye, Jojo -----Ursprüngliche Nachricht----- Von: Florian Weimer <fweimer@redhat.com> Gesendet: Montag, 14. März 2022 09:33 An: Joachim Schmitz <jojo@schmitz-digital.de> Cc: rsbecker@nexbridge.com; libc-help@sourceware.org; mike_kilpatrick@nskopensource.com; 'Shiva Subramanian' <shiva.subramanian@tcm.uk.com>; 'Bill Honaker' <bhonaker@xid.com> Betreff: Re: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem) * Joachim Schmitz: > No idea whether that port had ever been send back upstream, it must > have been done in 2007 or before. > Not sure what you mean by "stable kernel interface"? In general, Linux does not remove any interfaces for entering the kernel once they have been added. The system call numbers that identify the entry points also does not change over time. For example, on x86-64, there is still a fstat system call, even though it has been superseded by newfstatat first, and then by statx. Other operating systems treat libc has the stable binary interface (if they have one at all), and remove the old system calls from their kernels once libc has been upgraded. This means it is not feasible to maintain libc as a separate project. > No, sorry, unfortunately no gcc available nor possible to > cross-compile, it is actually the assembler part that is missing IIRC You will have to release that first, preferably as a new target for binutils or LLVM, and then a compiler. But again, even if you do all that, it is unlikely that we would accept a glibc port. But without the toolchain, there is simply no way that it can happen. I think glibc stopped supporting proprietary toolchains some time in the 90s, before the glibc 2.0 release, and we are definitely not going back. Sorry. If you need a glibc compatibility layer, maybe you should look at gnulib instead. Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AW: AW: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 8:49 ` AW: " Joachim Schmitz @ 2022-03-14 8:55 ` Florian Weimer 0 siblings, 0 replies; 9+ messages in thread From: Florian Weimer @ 2022-03-14 8:55 UTC (permalink / raw) To: Joachim Schmitz Cc: rsbecker, libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker' * Joachim Schmitz: > Not sure what you'd expect us to "release first", out compiler? > If so: sorry, that won't happen. We don't even own it ourself. > If we were able to port gcc, I believe we'd not be allowed to share it, due > to proprietary code needed inside, needed to support the extension out > platform needs. Understood, but if you want to even have to a chance at an official glibc port, you'd have to address these issues somehow. > If glibc can't be made POSIX complaint, then that's where everything is lost > for us, as far as I can tell. There might be a misunderstanding here. glibc is not layered on top of a POSIX interface, it itself is the POSIX interface for GNU systems. Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 7:54 ` Florian Weimer 2022-03-14 8:12 ` AW: " Joachim Schmitz @ 2022-03-14 12:48 ` rsbecker 2022-03-14 13:12 ` Florian Weimer 1 sibling, 1 reply; 9+ messages in thread From: rsbecker @ 2022-03-14 12:48 UTC (permalink / raw) To: 'Florian Weimer' Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker', jojo On March 14, 2022 3:55 AM, Florian Weimer wrote: >> I have been asked by members of the HPE NonStop (formerly Tandem) >> community to consider re-porting Glibc to the platform. There was a >> port back around >> 2.12 but that never made it to x86. > >It must have been a private port, not released to the general public. The port was released to the public, but not through standard glibc mechanisms. Both binaries and sources were available to the NonStop community. >Does HPE NonStop provide a stable kernel interface? Linux is somewhat unique in >that. Other systems require some level of synchronization between libc and >kernel updates. Yes. The kernel is highly stable, with HPE providing long-term compatibility statements. The POSIX components date back to 1995 and continuously upgraded. To date, source compatibility has been maintained even across chipset changes. >> The current situation is that the platform is not on the list of known >> ports. Glibc is a dependency of many projects that we want to bring to >> the platform. How can this be pursued? My team is willing to >> officially support the port once done. Although POSIX compliant, we >> are limited to using a c99 compiler. gcc does not port, making this >> interesting. > >Can you use a GCC cross-compiler? No. GCC does not generate code that will run on the platform. While it is x86-based, there are complex endian concerns in the code generator, significant ELF header differences, and major ELF debug section differences that cannot be emitted by GCC. There have been at least 3 unsuccessful attempts I know of to port gcc to the platform and/or to generate cross compile code for it. >We need a publicly available, free (libre) toolchain for any port. This is both a >matter of policy, and also to prevent the port from bit-rotting. I have successfully done ports of other products with the same constraint (git, openssh, openssl). Usually ports are possible with minimal changes. I would like to do a bit of an impact analysis to see how difficult this one would be. Glibc is an ongoing concern because it is a dependency for many Open-Source projects. >> Note: The link http://www.gnu.org/software/libc/development.html on >> your wiki page https://sourceware.org/glibc/wiki/#Development gets a 404. > >I've removed this section because it is outdated. I'm not sure if we would accept >ports to new kernels at this point. (New targets is a different matter.) How should I proceed, if I need glibc as a dependency? Thanks, Randall ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 12:48 ` rsbecker @ 2022-03-14 13:12 ` Florian Weimer 2022-03-14 14:20 ` Adhemerval Zanella 0 siblings, 1 reply; 9+ messages in thread From: Florian Weimer @ 2022-03-14 13:12 UTC (permalink / raw) To: rsbecker Cc: libc-help, mike_kilpatrick, 'Shiva Subramanian', 'Bill Honaker', jojo >>Can you use a GCC cross-compiler? > > No. GCC does not generate code that will run on the platform. While it is > x86-based, there are complex endian concerns in the code generator, > significant ELF header differences, and major ELF debug section differences > that cannot be emitted by GCC. There have been at least 3 unsuccessful > attempts I know of to port gcc to the platform and/or to generate cross > compile code for it. In this case, we would really need an open toolchain, otherwise we can't maintain the ELF parts. Keep in mind that glibc contains the dynamic linker, so it really has to know about the details of your architecture. > I have successfully done ports of other products with the same constraint > (git, openssh, openssl). Usually ports are possible with minimal > changes. That's because you rely on the C run-time library to paper over the differences in kernel interfaces. With glibc itself, that's different. > I would like to do a bit of an impact analysis to see how difficult > this one would be. You need to provide the rest of the toolchain first, and that seems to require a major effort (and it's not just about writing code). I also doubt that we would accept patches which retarget to a proprietary kernel, but that depends on how invasive those patches are. > How should I proceed, if I need glibc as a dependency? You need to port GCC and binutils first, and then you can proceed with glibc. It's how new ports are brought up. I'm not aware of any other port in recent times that has been done in a different way (not using GCC). This is not me being snarky, it's just that the GNU toolchain (binutils/GCC/GDB/glibc) is an integrated whole and works best in conjunction. It might be possible to get away with an LLVM-based toolchain, but the porting of glibc istelf to LLVM is not yet complete, so you would struggle on this front as well. Thanks, Florian ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re-port Intent for HPE NonStop (a.k.a. Tandem) 2022-03-14 13:12 ` Florian Weimer @ 2022-03-14 14:20 ` Adhemerval Zanella 0 siblings, 0 replies; 9+ messages in thread From: Adhemerval Zanella @ 2022-03-14 14:20 UTC (permalink / raw) To: Florian Weimer, rsbecker Cc: libc-help, mike_kilpatrick, 'Bill Honaker', jojo, 'Shiva Subramanian' On 14/03/2022 10:12, Florian Weimer via Libc-help wrote: >>> Can you use a GCC cross-compiler? >> >> No. GCC does not generate code that will run on the platform. While it is >> x86-based, there are complex endian concerns in the code generator, >> significant ELF header differences, and major ELF debug section differences >> that cannot be emitted by GCC. There have been at least 3 unsuccessful >> attempts I know of to port gcc to the platform and/or to generate cross >> compile code for it. > > In this case, we would really need an open toolchain, otherwise we can't > maintain the ELF parts. > > Keep in mind that glibc contains the dynamic linker, so it really has to > know about the details of your architecture. > >> I have successfully done ports of other products with the same constraint >> (git, openssh, openssl). Usually ports are possible with minimal >> changes. > > That's because you rely on the C run-time library to paper over the > differences in kernel interfaces. With glibc itself, that's different. > >> I would like to do a bit of an impact analysis to see how difficult >> this one would be. > > You need to provide the rest of the toolchain first, and that seems to > require a major effort (and it's not just about writing code). > > I also doubt that we would accept patches which retarget to a > proprietary kernel, but that depends on how invasive those patches are. > >> How should I proceed, if I need glibc as a dependency? > > You need to port GCC and binutils first, and then you can proceed with > glibc. It's how new ports are brought up. I'm not aware of any other > port in recent times that has been done in a different way (not using > GCC). This is not me being snarky, it's just that the GNU toolchain > (binutils/GCC/GDB/glibc) is an integrated whole and works best in > conjunction. Besides all Florian has pointed out, I am inclined to say that porting glibc to a non opensource architecture and also without aiming to be the 'de facto' loader/libc of the system is a dead-end. Without open-source support for at least the toolchain pieces, it is really a no-go. Also, besides probably requiring a lot of work to support any ELF and/or ABI extension, the resulting loader won't easily support the system shared libraries without a compat layer (which imposes another set of issues). Although there are some examples where it has been done with some relative success (such as gcompat for musl, or linuxabi for FreeBSD) it a relative time consuming effort where maintaining this out of tree usually ends up as a bitrotten project (as we saw for prelink, which was removed recently). Also, most of the generic interfaces glibc provide is already provided by gnulib in a extent. There are other interface that are reliant on proper kernel support, so using a different kernel might ended required *a lot* of glue code and, worse, some interfaces might not be actually be proper implemented. The truth is glibc is currently Linux oriented projected, there are some Hurd work but it is moving to nowhere (it barely supports one legacy architecture). > > It might be possible to get away with an LLVM-based toolchain, but the > porting of glibc istelf to LLVM is not yet complete, so you would > struggle on this front as well. > I am working to get glibc in a somewhat usable state with clang [1]. It still requires a *lot* of patches and I still struggling in the loader bootstrap, but at least it *builds*. [1] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/clang ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-03-14 14:20 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-13 12:43 Re-port Intent for HPE NonStop (a.k.a. Tandem) rsbecker 2022-03-14 7:54 ` Florian Weimer 2022-03-14 8:12 ` AW: " Joachim Schmitz 2022-03-14 8:32 ` Florian Weimer 2022-03-14 8:49 ` AW: " Joachim Schmitz 2022-03-14 8:55 ` Florian Weimer 2022-03-14 12:48 ` rsbecker 2022-03-14 13:12 ` Florian Weimer 2022-03-14 14:20 ` Adhemerval Zanella
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).