* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] <19865.1101395592@redhat.com> @ 2004-11-25 18:20 ` Alexandre Oliva [not found] ` <20041125165433.GA2849@parcelfarce.linux.theplanet.co.uk> [not found] ` <Pine.LNX.4.58.0411301243000.22796@ppc970.osdl.org> 2 siblings, 0 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-25 18:20 UTC (permalink / raw) To: David Howells; +Cc: torvalds, hch, matthew, dwmw2, linux-kernel, libc-hacker On Nov 25, 2004, David Howells <dhowells@redhat.com> wrote: > SOURCE INSTALLED AS > ====================== ============ > include/user/ /usr/include/user/ > include/user-i386/ /usr/include/user-i386/ > /usr/include/linux -> user > /usr/include/asm -> user-i386 Although user/ and user-* make a lot of sense within the kernel source tree, I don't think these names would be very clear in /usr/include. I'd rather use names in /usr/include that more clearly associate them with the kernel. Heck, even /usr/include/asm is inappropriate, but it's been there for so long that we really shouldn't try to get rid of it. If I had it my way, we'd have, in the kernel tree, userland-aimed headers in include/linux/user and include/asm-<machine>/user, and have them installed in /usr/include/linux and /usr/include/asm-<machine>. This means these headers shouldn't reference each other as linux/user/something.h, but rather as linux/something.h, such that they still work when installed in /usr/include/linux. This may require headers include/linux/something.h to include linux/user/something.h, but that's already part of the proposal. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <20041125165433.GA2849@parcelfarce.linux.theplanet.co.uk>]
[parent not found: <1101406661.8191.9390.camel@hades.cambridge.redhat.com>]
[parent not found: <20041127032403.GB10536@kroah.com>]
[parent not found: <16810.24893.747522.656073@cargo.ozlabs.ibm.com>]
[parent not found: <Pine.LNX.4.58.0411281710490.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411281710490.22796@ppc970.osdl.org> @ 2004-11-29 17:10 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411290926160.22796@ppc970.osdl.org> [not found] ` <8219.1101828816@redhat.com> [not found] ` <1101721336.21273.6138.camel@baythorne.infradead.org> [not found] ` <20041214025110.A28617@almesberger.net> 2 siblings, 2 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-29 17:10 UTC (permalink / raw) To: Linus Torvalds Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 28, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On the bigger question of what to do with kernel headers in general, let's > just make one thing clear: the kernel headers are for the kernel, and big > and painful re-organizations that don't help _existing_ user space are not > going to happen. So, of the following two options, which one you think everybody should pick? - Linux gets to define the ABI between kernel and userland, and userland must duplicate the contents of headers in which the kernel defines the kernel<->userland ABI, tracking changes in them in the hope that nothing falls through the cracks - we move the kernel<->userland to a separate package, where it's maintained such that it can be used by both kernel and userland, and Linux will only build when given a pointer to the location of this package. I don't think you can have it both ways. As long as the kernel wants freedom to set the ABI, either let people use the headers where you define it, or maintain the ABI separately and let people use them. Forcing others to maintain duplicates of your headers will just get most people to use your headers for purposes they weren't meant to be used; maintaining two similar but far from identical sets of headers in sync is a major pain, that's why few people do it, and even fewer people do it properly. Refusing to help in the effort to maintain them for wishes of freedom to tinker or allergy to external dependencies haven't worked as excuses in the past, and I think they never will. Duplicating the effort is just stupid, and requiring others to do so is arrogant. Why shouldn't we just get together and have something that works for both? Or how about we turn the table around and define an ABI that the kernel must implement, and then have the kernel have to keep track of the changes in such external ABI definition, instead of the way things (fail to) work now? How comfortable would you feel with this arrangement? My usual reply when I hear this argument that the kernel headers that define the ABI with userland are to be used by the kernel only, and someone else is supposed to create the headers for the same purpose to be used by userland, is that you generally won't find the headers that expose the interface of a library in a separate package (think libz.{a,so} and zlib.h). They're part of the same package, built out of the same sources, and there's a reason for that: if they get out of sync, it won't serve its purpose. It makes far more sense to maintain them together. You won't find zlib.h in packages that just want to use zlib, unless they contain zlib in its entirety, just in case it's not pre-installed. I think this makes perfect sense. Requiring libc or a separate package used by libc to contain copies of headers that define the API/ABI exported by the kernel to userland is just the same sort of nonsense, with the difference that it's a way bigger set of headers we're talking about, which renders the problem even worse. Sure you can take the easy way out and claim you told people they shouldn't do what they're doing when they run into problems because of including kernel headers from userland. But as soon as you reject proposals to clean things up such that we would have a set of headers that both kernel and userland will be able to use, in a way that would have the headers still conveniently (for you) sitting in the kernel source tree, I can't see that you could say it's not your problem with a straight face any more. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411290926160.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411290926160.22796@ppc970.osdl.org> @ 2004-11-29 22:28 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411291458040.22796@ppc970.osdl.org> 0 siblings, 1 reply; 14+ messages in thread From: Alexandre Oliva @ 2004-11-29 22:28 UTC (permalink / raw) To: Linus Torvalds Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 29, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On Mon, 29 Nov 2004, Alexandre Oliva wrote: >> - Linux gets to define the ABI between kernel and userland, and >> userland must duplicate the contents of headers in which the kernel >> defines the kernel<->userland ABI, tracking changes in them in the >> hope that nothing falls through the cracks > This is unquestionably true. The kernel obviously _does_ define the > ABI, and userland just lives with it. I don't see it as obvious at all. The need for an agreement between two parties on an ABI doesn't imply that one party gets to define it and the other gets to follow it. An ABI has to be set, and then kernel and userland get to follow it, but I don't see why it should be the kernel that determines it. It might very well be userland that sets the ABI, and the kernel is implemented so as to comply. Stating it the way you did is what I alluded to as being an arrogant attitude in my previous e-mail. It sounds as if the kernel was the center of the universe, and everything else gravitated around it. Just like userland needs a kernel to operate, a kernel without any userland programs to run is pretty much useless. Yes, I know one can point out examples of exceptions to both statements, but those are just that, exceptions. > At some point you have to track things, just because new features > etc just can't be sanely handled any other way. This applies to both sides of the bridge. >> - we move the kernel<->userland to a separate package, where it's >> maintained such that it can be used by both kernel and userland, and >> Linux will only build when given a pointer to the location of this >> package. > No. Quite frankly, I absolutely hate projects that do that. I'm glad to see we agree on this. I happen to find the current proposition of duplicating contents of kernel headers in some other package just because `kernel headers are just that' just as disgusting. If you want to get to set the ABI, which you appear to take as a given (and, in spite of my argumentation above, I'm fine with that), you should do so in a way in which others can sanely comply with the ABI. Duplicating header contents is not a sane way to do it. Just consider that the problem of tracking the dependencies needed to build Linux, should the ukabi headers be moved out of it, affects any project that purports to maintain this set of headers for userland use outside the kernel, just in a more subtle way: you need not only to make sure that you have the package that offers the interfaces you want to use, but also you have to make sure you have a specific version of that package that matches the kernel baseline your userland is intended to work with. Although I can see that, for established ports, the rate of change in the ukabi is quite small (addition of syscalls, mostly), during the development of a new port, the rate of change is very high, and that's the point in which you need the most to make sure that the ABIs are in sync, and maintaining them separately is exactly the right way to make sure they aren't. > Now, on trying to make the tracking _easier_, I would not mind at all to > move (well-defined) things around a bit to make it clearer what is > actually exported to user space. But on the other hand, I don't think it's > actively wrong either to just "mark" them in the headers some way, and > have tools to extract it automatically _without_ having to separate them > into some magic location. As long as the marks are not ones that actually require C preprocessing, which would render them useless to extract bits in a form that makes them usable for a multi-ABI userland targets, I happen to like this better as well. #ifdef __KERNEL__ can be easily machine-extracted (I know dwmw2 has a script for that, dunno whether he wrote it himself or got it from someone else), and the extraction of bits for userland could even be done as part of say a make install-ukabi-headers. There are problems, though: sometimes, someone will make a change such as adding one #include to a header file that is needed because of kernel-only bits, and if this include is nor protected by the same guards and a possibly empty header file with that name is not installed as part of the ukabi, userland builds fail. So, again, we agree that the marking should be more explicit than implicit export. The advantage of __KERNEL__, however, is that it makes the very same header files (presumably) usable for userland and the kernel, which could save some space and speed up the installation of such headers. I'm not sure this advantage is worth the trouble. Another important problem about sharing headers between kernel and userland, that I don't recall anyone having brought up before, is that of licensing. If the kernel headers are licensed under the GNU GPL, and say a libc that is GNU LGPL in the source form uses them, it might inherit the full GPL protection and start imposing the GPL requirements on binaries linked with it. Since the intent of the ukabi headers is to set the interface between kernel and userland and AFAIK you copyright protection doesn't apply to interfaces (IANAL so this is probably quite inaccurate), one might get away with that, but since there's often more than just interfaces in headers that are used by both kernel and userland, the argument might fail. Some (myself included) would probably consider this trap against proprietary software a good thing, but others might consider it a huge problem. If not for any other reason, at least for the sake of openness, it would probably be a good thing to separate the ukabi contents from headers that define other kernel-internal macros, inline functions, data structures, etc, and perhaps use different licensing terms in these split-out headers, explaining to what extent using them limits the licensing choices available to programs that include them. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411291458040.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411291458040.22796@ppc970.osdl.org> @ 2004-11-30 21:22 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301413260.22796@ppc970.osdl.org> 0 siblings, 1 reply; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 21:22 UTC (permalink / raw) To: Linus Torvalds Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 29, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On Mon, 29 Nov 2004, Alexandre Oliva wrote: >> >> I don't see it as obvious at all. The need for an agreement between >> two parties on an ABI doesn't imply that one party gets to define it >> and the other gets to follow it. > Sorry, but that's not how it works. Then maybe this is the fundamental problem. As long as the kernel doesn't recognize that an ABI is a contract, rather than an imposition, kernel developers won't care. But guess what? Kernel developers *do* care about maintaining the userland ABI. They don't just go wild changing data types, structures, renumbering system calls and all that sort of nasty stuff because there's a contract with userland that they don't want to break. Because breaking this contract would be a stupid thing, that would require all affected userland to be modified to cope with the change, even if that's just a recompile using the new kernel headers... erhm... using the result of applying the kernel ABI change in the copy/extract/whatever of kernel headers that userland is entitled to use. As long as we recognize that the notion that the kernel<->userland ABI is a separate entity, that enables kernel and userland to exchange information, rather than something that the kernel defines and userland must follow, we give strength to the argument that userland is not a derived work from the kernel. If this argument proves to be false, then only works released under the GNU GPL or a compatible license will be allowed to run atop of Linux, because libc would be a derived work, its license would become full GPL, and any program linked with it could only be distributed under the full GPL. Oh, and just in case, IANAL :-) Again, I wouldn't mind that at all, but I'm sure others feel otherwise. Besides, as I wrote before, if this is the intent, I'd rather have it exposed in clear, rather than seeing people being misled to believe it's fair game to run non-GPLed software on Linux-based operating systems. > He who writes the code decides what it is. I've been in projects in which I wrote ABI headers before the corresponding kernel port had even started. Sometimes, quite unfortunately, the kernel engineer went ahead and wrote headers with slightly different assumptions, and then we had to get together to define what the ABI was going to be. In fact, the way you seem to want things to be, you want both kernel and userland to write the code, and somehow hope they magically match each other. That's not how things work. The way things work is that either you get agreement first, or each end goes on its way and then at some point they figure out where they diverged and fix things up, or one end goes first and the other takes the definitions and uses them. It's in no way a kernel->userland one-way street. It's a language (for lack of a better word) that kernel and userland choose to exchange information, not one that the kernel invents and then requires others interested in talking to it to learn (you know, that ``Do you speak English/my language?´´ phrase so common in Hollywood movies in which an American character needs to talk to someone abroad :-) Sometimes you'll even find formal ABI specifications, written down before either userland or kernel start being developed. Claiming that the kernel gets to set the ABI as it wishes, without worrying about userland because it never calls userland is a very narrow view, and in some cases it's even a lie. Think signal handling and process start-up, for example: if you don't follow the userland ABI to call the function pointer given to the signal function, or if you don't follow the userland ABI to start the program's entry point, it's not userland that is broken, it's the kernel. > In this case, if the kernel > does a new extension, it's the kernel that gets to decide what it is. As in, say, NPTL? As if a number of userland considerations hadn't driven most of the design decisions behind the then-defined ABI extensions? > If glibc wants to do something new, go wild. The kernel won't care. This is not about glibc. That's not the only userland component. There are several different alternatives one can use. And then, the kernel will care, because it has to. A kernel without userland is nothing. A kernel that kept introducing incompatible ABI changes would die a very quick death. A kernel has to be very careful about complying with the ABI it shares with userland, and you know that, because you do care about it. > And that's really the fundamental issue. The kernel does not care what > user land does. The kernel exports functionality, the kernel does _not_ > ask user land to help. I'm not so much talking about the hundreds of syscalls, the syscall conventions and the number and type of arguments given to each syscall. This is not even well defined in kernel headers. All that's exported regarding syscalls is a mapping from syscall names to numbers. What I'm talking about is the data structures. Sure, the kernel can define these however it likes. So can userland. The point is that, if they don't match, when you get userland and kernel together, in the hopes of getting something useful done, it doesn't work. Both ends can come up with extensions, and do things the old way until the other end picks up the extension. I'll give you that many extensions make to the kernel before say glibc, but that's mostly just like, when we create a new toolchain port, assembler and linker are generally implemented before the compiler. I guess it just makes more sense. It wastes less effort to make sure you got something right in a lower layer before you build something atop of it. It's unlikely that a change would make to glibc if it used a feature that isn't available in the kernel, just because most often features undergo (minor or sometimes major) changes when they're integrated, and it's easier to fix things up if you haven't committed to the way that didn't make it there yet. > That _does_ make it a one-way street. Sorry. Good, we're getting somewhere. This is surely better than a zero-way street, i.e., one in which headers that specify the ABI don't flow in any direction. If we're agreeing to make it one way, let's make it so: have headers that are maintained as part of the kernel, and that are imposed, if you like, on userland. Saying it's a one-way street but denying others from taking what comes their way through this one-way street hasn't been very effective :-) -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411301413260.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411301413260.22796@ppc970.osdl.org> @ 2004-11-30 22:57 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301505580.22796@ppc970.osdl.org> 0 siblings, 1 reply; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 22:57 UTC (permalink / raw) To: Linus Torvalds Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On Tue, 30 Nov 2004, Alexandre Oliva wrote: >> Then maybe this is the fundamental problem. As long as the kernel >> doesn't recognize that an ABI is a contract, rather than an >> imposition, kernel developers won't care. > That's a silly analogy. Worse, it's a very flawed analogy. > If you want to use a legal analogy, the ABI is not a contract, it's a > public _license_. I didn't mean to use a legal analogy. I meant contract in the software engineering sense. Sorry if that wasn't clear. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411301505580.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411301505580.22796@ppc970.osdl.org> @ 2004-12-01 19:41 ` Alexandre Oliva 0 siblings, 0 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-12-01 19:41 UTC (permalink / raw) To: Linus Torvalds Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On Tue, 30 Nov 2004, Alexandre Oliva wrote: >> On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: >> > If you want to use a legal analogy, the ABI is not a contract, it's a >> > public _license_. >> I didn't mean to use a legal analogy. I meant contract in the >> software engineering sense. Sorry if that wasn't clear. > Then your definition of a "contract" is flawed or your world-view has > nothing to do with reality. It's not my definition, it's a definition used in software engineering. Sure, if you ask a legal mind, you'll get a different answer. If you ask a bridge player, you'll get yet another definition. That's why we have jargons in which common-use words have their meanings narrowed to the specifics of certain fields. And that's why it's so hard for specialists in different fields to talk to each other at times when there's disagreement about the precise meaning of certain words. > An ABI is not a contract. Not a legal contract, for sure. An ABI is a definition of an interface, including operations with pre- and post-conditions, data structures with their invariants, constants, file formats, etc. Most of that is covered by the software engineering term `contract'. Sure enough, the headers we're talking about describe only a very small portion on that; pretty much only data structures and constants. It certainly doesn't *specify* the entire ABI, it only highlights portions of it. Anyhow, all of this is beyond the point. I see you've decreed that people can introduce `user' directories in the kernel now. Would you please reconsider and choose a dir name that would enable the same ABI headers to be used by kernel and userland, without adding a directory to /usr/include that has no indication that it comes from the kernel? Thanks, -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <8219.1101828816@redhat.com>]
[parent not found: <Pine.LNX.4.58.0411300744120.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411300744120.22796@ppc970.osdl.org> @ 2004-11-30 20:47 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301249590.22796@ppc970.osdl.org> 0 siblings, 1 reply; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 20:47 UTC (permalink / raw) To: Linus Torvalds Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > I object sternuously to your "the header files". If you can't even say > _which_ header file, I'm not interested. How about this formulation then: - move anything that is not protected by #ifdef __KERNEL__ to the ukabi header tree, adding an include in the beginning of the original header that includes the ukabi header. Since ukabi stuff can't depend on kernel-internal stuff, having this include as the first thing in the existing header should work. - figure out what should have been protected by #ifdef __KERNEL__ but wasn't, and move it back - wait for bug reports from the world and deal with them - repeat until you feel you got it right > See what I'm saying? Whole-sale "move things around because we want to" > I'm not interested in. Specific problems, yes. The specific problem we're trying to address is the creation of header files that define the ABI between userland and kernel, that both kernel and userland can use. >> Personally, I'd prefer us to move to using standard C99 types in lieu of u32 >> and co at least for the interface to userspace because they are just that: >> standard. > No. I told you why it cannot and MUST NOT be done. Repeat after me: > WE MUST NOT POLLUTE THE NAMESPACE! David, Linus is right. Using standard types is not an option because they're not built in type names; you have to include a header to bring them in, and ideally you shouldn't gratuitously get the names defined just because you include some unrelated header, although the C Standard grants standard headers freedom to include other headers or something along these lines; I don't recall the exact passage of the Standard that says so, but we should strive to not pollute the user namespace anyway. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411301249590.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411301249590.22796@ppc970.osdl.org> @ 2004-11-30 21:30 ` Alexandre Oliva 2004-11-30 21:40 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301423030.22796@ppc970.osdl.org> 0 siblings, 2 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 21:30 UTC (permalink / raw) To: Linus Torvalds Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > On Tue, 30 Nov 2004, Alexandre Oliva wrote: >> >> - move anything that is not protected by #ifdef __KERNEL__ to the >> ukabi header tree, adding an include in the beginning of the original >> header that includes the ukabi header. > No. I want stuff that goes into the ABI tree to be clearly _defined_ to be > user-visible. Not a "let's move it all there and then prune it". Right. Which brings us to my second bullet. I *knew* I should have made it the first :-/ > Also, "ukabi" just isn't going to fly as a name. It's also not as simple > as you seem to think, since a lot of these ABI things are architecture- > dependent, which apparently all you guys have totally ignored. Looks like you missed the post that started this thread. It *did* mention machine-independent and machine-specific user headers. > I've suggested "include/user/" and "include/asm-xxx/user", which handles > architecture-specific parts too. I'm ok with doing it the other way > around, ie "include/user/" and "include/user/arch-xxxx". As I pointed out, `user' is a very bad name. As you said yourself, we're talking about the *kernel* ABI. So what's `user' supposed to mean? Was I so successful in my arguments that you now see it as the userland ABI? :-) > (a) it can't break anything (ie the old location still includes the new > one, exactly the same way) You mean it can't break anything in a kernel build, or it can't break anything except for userland apps that abused kernel headers and used to get away with that? > (b) there are people who will actually take _advantage_ of that > particular file (ie "just because I think so" doesn't fly). People who currently get to maintain duplicates of these header contents will take immediate advantage of these changes, since they will no longer have to maintain the duplicates. This is a major step forward in terms of maintainability for everybody else, at no expense to the kernel, that now gets to explicitly (as opposed to implicitly) document what is the ABI it's willing to comply with. It's a win for both ends. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 2004-11-30 21:30 ` Alexandre Oliva @ 2004-11-30 21:40 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301423030.22796@ppc970.osdl.org> 1 sibling, 0 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 21:40 UTC (permalink / raw) To: Linus Torvalds Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker On Nov 30, 2004, Alexandre Oliva <aoliva@redhat.com> wrote: >> I've suggested "include/user/" and "include/asm-xxx/user", which handles >> architecture-specific parts too. I'm ok with doing it the other way >> around, ie "include/user/" and "include/user/arch-xxxx". > As I pointed out, `user' is a very bad name. As you said yourself, > we're talking about the *kernel* ABI. So what's `user' supposed to > mean? Was I so successful in my arguments that you now see it as the > userland ABI? :-) I got carried away with joking and failed to repeat why I consider it a bad name (assuming that, since you missed the beginning of the thread, you probably missed the first reply I posted to the message that started it): since we're going to install these headers in /usr/include (where headers for userland live), /usr/include/user is quite misleading. /usr/include/kernel would be far more appropriate for this purpose. Sure, we could take headers from linux-*/include/user and install them in /usr/include/kernel, but then includes in there that reference other headers in user/ or in asm-<arch>/ will cease to work. So we should come up with a name that makes sense for both users of these headers, which is why I suggested ukabi. linux/abi and asm-<mach>/abi work just as well, and then we can soft-link `abi -> .' in /usr/include/{linux,asm-<mach>} if needed. Ideally, we wouldn't have to. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411301423030.22796@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0411301423030.22796@ppc970.osdl.org> @ 2004-11-30 23:05 ` Alexandre Oliva 0 siblings, 0 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 23:05 UTC (permalink / raw) To: Linus Torvalds Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote: > Wrong. They'll _still_ have to maintain duplicates, since they can't rely > ont he end-user to have a recent enough kernel. No. They just won't have the headers for the newer kernel installed on their system. They stick with the kernel ABI headers provided by a baseline kernel, and that's probably from an older kernel tarball. > It's just that they can hopefully start _copying_ their dupliates more > easily. It's still the headers from the kernel, copied or not. From a different, baseline version, maybe. > At RH you may see only the case where people do a whole-system upgrade. I personally don't care about the case you perceive as the Red Hat case, because I work on embedded systems development, in a different organization within Red Hat than the people who develop Red Hat Enterprise Linux and Fedora. So yes, in the case that actually benefits me, it's even more of a whole-system thing, since for our group it's *the* kernel, not *a* random kernel version. It's the kernel we port and ship to the customer, and the corresponding kernel<->userland ABI. That said, I do understand the issues of kernel baseline ABI settings, or at least part of them. I know you don't want to assume the presence of a certain system call just because you're running a kernel that implements it. In fact, I know you don't want to assume the presence of a system call just because it's defined in linux/unistd.h. As long as the userland software keeps this distinction in mind, I don't see why taking the latest kernel<->userland ABI headers, that describe the newest system calls and any corresponding data structures, could hurt anything. In fact, if you don't have the corresponding system call numbers and data structures, you can't possibly take advantage of the new syscalls even if they happen to be available, unless you duplicate the syscall numbers and data structure definitions in your own code, which, from a software engineering standpoint, is bad. Of course, badly written software can assume the syscalls are present just because their numbers are given, and then people who rely on such bad software lose, but such is the nature of bad software. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <1101721336.21273.6138.camel@baythorne.infradead.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <1101721336.21273.6138.camel@baythorne.infradead.org> @ 2004-12-01 11:48 ` Ralf Baechle 0 siblings, 0 replies; 14+ messages in thread From: Ralf Baechle @ 2004-12-01 11:48 UTC (permalink / raw) To: David Woodhouse Cc: Linus Torvalds, Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch, aoliva, linux-kernel, libc-hacker On Mon, Nov 29, 2004 at 09:42:16AM +0000, David Woodhouse wrote: > I've lost track of the number of times things have broken because of > incorrect use of kernel headers from userspace. That's what we're trying > to fix -- by putting only the bits which are _supposed_ to be visible > into files which userspace sees, where we know they define part of the > userspace API and hence we can be extremely careful when editing them. > > I don't think it makes sense at this point for us to bury our collective > heads in the sand and pretend there isn't a problem here that's worth > fixing. > > I agree that it should be obviously correct though -- and that's why > we're trying to end up with a structure that in the first pass would > give us in userspace essentially what we already have in the various > glibc-kernheaders packages, but without the constant and unnecessary > need for some poor sod to keep those up to date by hand. The concept of copying kernel headers into applications is even worse when arch portability is affected. I stopped counting how often I had to fix that kind of crap - and the state of kernel headers and userspace kernel header packages is really provoking that kind of mess. Ralf ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <20041214025110.A28617@almesberger.net>]
[parent not found: <Pine.LNX.4.58.0412140734340.3279@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0412140734340.3279@ppc970.osdl.org> @ 2004-12-14 16:55 ` Andreas Schwab 0 siblings, 0 replies; 14+ messages in thread From: Andreas Schwab @ 2004-12-14 16:55 UTC (permalink / raw) To: Linus Torvalds Cc: Werner Almesberger, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva, linux-kernel, libc-hacker Linus Torvalds <torvalds@osdl.org> writes: > This is a common issue with namespace pollution. For example, this program > is perfectly valid afaik (well, except for being _stupid_, but that's > another issue): > > #include <stdio.h> > > const char *int32_t(int i) > { > return i ? "non-zero" : "zero"; > } Actually this is not allowed in POSIX. _Any_ header may define any identifier ending with "_t". Andreas. -- Andreas Schwab, SuSE Labs, schwab@suse.de SuSE Linux Products GmbH, MaxfeldstraÃe 5, 90409 Nürnberg, Germany Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <Pine.LNX.4.58.0411301243000.22796@ppc970.osdl.org>]
[parent not found: <20041130223359.GA15443@mars.ravnborg.org>]
[parent not found: <200411302344.21907.mmazur@kernel.pl>]
[parent not found: <20041130230325.GY26051@parcelfarce.linux.theplanet.co.uk>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <20041130230325.GY26051@parcelfarce.linux.theplanet.co.uk> @ 2004-11-30 23:14 ` Alexandre Oliva 0 siblings, 0 replies; 14+ messages in thread From: Alexandre Oliva @ 2004-11-30 23:14 UTC (permalink / raw) To: Al Viro Cc: Mariusz Mazur, Sam Ravnborg, Linus Torvalds, David Woodhouse, Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker On Nov 30, 2004, Al Viro <viro@parcelfarce.linux.theplanet.co.uk> wrote: > WTF? I've got a dozen kernel trees hanging around, which one (and WTF any, > while we are at it) should be "linked to"? Whichever you chose to install in your /usr/include, and use as the kernel ABI definition as far as userland is concerned. I don't think `make install' should touch /usr/include at all. It should be a separate step, such that one can build a kernel abi headers package out of the kernel source tree, ideally without even having to configure it first, and use that as the kernel ABI definition. You should only have to do this again *if* the ABI changes (ideally no removals, only additions) *and* you want to take advantage of them, *and* you're willing to rebuild userland pieces that could take advantage of them. On most systems, people won't do that, they'll just use whatever kernel headers and glibc binaries their distro ships. If they want to change any of these, they have to know what they're doing. -- Alexandre Oliva http://www.ic.unicamp.br/~oliva/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org} ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <20041214135029.A1271@almesberger.net>]
[parent not found: <200412141923.iBEJNCY9011317@laptop11.inf.utfsm.cl>]
[parent not found: <20041214194531.GB13811@mars.ravnborg.org>]
[parent not found: <Pine.LNX.4.58.0412141150460.3279@ppc970.osdl.org>]
* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ [not found] ` <Pine.LNX.4.58.0412141150460.3279@ppc970.osdl.org> @ 2004-12-14 20:26 ` Andreas Schwab 0 siblings, 0 replies; 14+ messages in thread From: Andreas Schwab @ 2004-12-14 20:26 UTC (permalink / raw) To: Linus Torvalds Cc: Sam Ravnborg, Horst von Brand, Werner Almesberger, Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva, linux-kernel, libc-hacker Linus Torvalds <torvalds@osdl.org> writes: > We used to compile with "-nostdinc" to make sure that you couldn't include > user files even by mistake, but that was removed for some reason I can't > for the life of me remember any more. ??? We still do that, see NOSTDINC_FLAGS. Andreas. -- Andreas Schwab, SuSE Labs, schwab@suse.de SuSE Linux Products GmbH, MaxfeldstraÃe 5, 90409 Nürnberg, Germany Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2004-12-14 20:26 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <19865.1101395592@redhat.com> 2004-11-25 18:20 ` [RFC] Splitting kernel headers and deprecating __KERNEL__ Alexandre Oliva [not found] ` <20041125165433.GA2849@parcelfarce.linux.theplanet.co.uk> [not found] ` <1101406661.8191.9390.camel@hades.cambridge.redhat.com> [not found] ` <20041127032403.GB10536@kroah.com> [not found] ` <16810.24893.747522.656073@cargo.ozlabs.ibm.com> [not found] ` <Pine.LNX.4.58.0411281710490.22796@ppc970.osdl.org> 2004-11-29 17:10 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411290926160.22796@ppc970.osdl.org> 2004-11-29 22:28 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411291458040.22796@ppc970.osdl.org> 2004-11-30 21:22 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301413260.22796@ppc970.osdl.org> 2004-11-30 22:57 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301505580.22796@ppc970.osdl.org> 2004-12-01 19:41 ` Alexandre Oliva [not found] ` <8219.1101828816@redhat.com> [not found] ` <Pine.LNX.4.58.0411300744120.22796@ppc970.osdl.org> 2004-11-30 20:47 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301249590.22796@ppc970.osdl.org> 2004-11-30 21:30 ` Alexandre Oliva 2004-11-30 21:40 ` Alexandre Oliva [not found] ` <Pine.LNX.4.58.0411301423030.22796@ppc970.osdl.org> 2004-11-30 23:05 ` Alexandre Oliva [not found] ` <1101721336.21273.6138.camel@baythorne.infradead.org> 2004-12-01 11:48 ` Ralf Baechle [not found] ` <20041214025110.A28617@almesberger.net> [not found] ` <Pine.LNX.4.58.0412140734340.3279@ppc970.osdl.org> 2004-12-14 16:55 ` Andreas Schwab [not found] ` <Pine.LNX.4.58.0411301243000.22796@ppc970.osdl.org> [not found] ` <20041130223359.GA15443@mars.ravnborg.org> [not found] ` <200411302344.21907.mmazur@kernel.pl> [not found] ` <20041130230325.GY26051@parcelfarce.linux.theplanet.co.uk> 2004-11-30 23:14 ` Alexandre Oliva [not found] <20041214135029.A1271@almesberger.net> [not found] ` <200412141923.iBEJNCY9011317@laptop11.inf.utfsm.cl> [not found] ` <20041214194531.GB13811@mars.ravnborg.org> [not found] ` <Pine.LNX.4.58.0412141150460.3279@ppc970.osdl.org> 2004-12-14 20:26 ` Andreas Schwab
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).