Cross-compiling Linux kernel

Cross-compiling Linux kernel

[Fedin Pavel]

 Hello!

 In order to cross-compile Linux kernel i have to patch host-side tools a
little bit. Current problems are:
 1. Cygwin misses linux/types.h
 2. Cygwin defines the following ELF macros according to host machine:
ELF_ST_BIND, ELF_ST_TYPE, ELF_R_SYM, ELF_R_TYPE

 What about improving Linux compatibility ? We could have own version of
linux/types.h (for __u8. __u16, etc), and omit ELF definitions. AFAIK they
are not used anywhere, they are just inherited from BSD.
 At the other hand, i believe the same problem will happen if i try to
cross-compile Linux kernel under BSD. And Cygwin is BSD-style system...

 Just decided to drop this in as one more "what if...".

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Corinna Vinschen]

On Jun 17 17:24, Fedin Pavel wrote:
>  Hello!
> 
>  In order to cross-compile Linux kernel i have to patch host-side tools a
> little bit. Current problems are:
>  1. Cygwin misses linux/types.h
>  2. Cygwin defines the following ELF macros according to host machine:
> ELF_ST_BIND, ELF_ST_TYPE, ELF_R_SYM, ELF_R_TYPE
> 
>  What about improving Linux compatibility ? We could have own version of
> linux/types.h (for __u8. __u16, etc), and omit ELF definitions. AFAIK they
> are not used anywhere, they are just inherited from BSD.
>  At the other hand, i believe the same problem will happen if i try to
> cross-compile Linux kernel under BSD. And Cygwin is BSD-style system...

I have no opinion on the ELF definition issue, but just adding a
linux system header looks pretty wrong to me.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Chris J. Breisch]

On 6/17/2013 11:18 AM, Corinna Vinschen wrote:
> On Jun 17 17:24, Fedin Pavel wrote:
>>   Hello!
>>
>>   In order to cross-compile Linux kernel i have to patch host-side tools a
>> little bit. Current problems are:
>>   1. Cygwin misses linux/types.h
>>   2. Cygwin defines the following ELF macros according to host machine:
>> ELF_ST_BIND, ELF_ST_TYPE, ELF_R_SYM, ELF_R_TYPE
>>
>>   What about improving Linux compatibility ? We could have own version of
>> linux/types.h (for __u8. __u16, etc), and omit ELF definitions. AFAIK they
>> are not used anywhere, they are just inherited from BSD.
>>   At the other hand, i believe the same problem will happen if i try to
>> cross-compile Linux kernel under BSD. And Cygwin is BSD-style system...
> I have no opinion on the ELF definition issue, but just adding a
> linux system header looks pretty wrong to me.
>
>
> Corinna
>
There are far better ways to cross-compile the Linux kernel. I built a 
Cross Linux From Scratch system under Cygwin earlier this year. I just 
got back from vacation, though. Give me a day or so, and I'll find my 
notes on it. I believe most of them have already been posted here in the 
past, however.

Chris J. Breisch

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Yaakov (Cygwin/X)]

On 2013-06-17 08:24, Fedin Pavel wrote:
>   In order to cross-compile Linux kernel i have to patch host-side tools a
> little bit. Current problems are:
>   1. Cygwin misses linux/types.h

I have been trying to get this fixed upstream for quite some time 
without success:

https://lkml.org/lkml/2012/6/11/604
https://lkml.org/lkml/2012/10/15/608
https://lkml.org/lkml/2013/4/28/84

>   2. Cygwin defines the following ELF macros according to host machine:
> ELF_ST_BIND, ELF_ST_TYPE, ELF_R_SYM, ELF_R_TYPE

Those are indeed copied straight from BSD's headers; I'll have to look 
into this further.


Yaakov


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Cross-compiling Linux kernel

[Fedin Pavel]

 Hello!

> >   1. Cygwin misses linux/types.h
> 
> I have been trying to get this fixed upstream for quite some time
> without success:
> 
> https://lkml.org/lkml/2012/6/11/604
> https://lkml.org/lkml/2012/10/15/608
> https://lkml.org/lkml/2013/4/28/84

 Looks like Linux developers have taken over Olympus from greek gods, and
totally don't care about anyone except themselves any mote. I have already
met this problem when tried to push Amiga SFS filesystem implementation to
LKML. They told me something like: "We are *L*I*N*U*X*, we have own cool
filesystems and we don't care about one more obscure filesystem". I was
extremely angry and barely stopped myself from replying that whole their
Linux is not less obscure, because 99% of the world actually use NTFS. :(

 Again, from the practical point of view... May be make some package like
linux-compat ?

 P.S. I have got even more crazy idea, perhaps deserving a separate topic...
BSD systems have Linux binary compatibility layer. Could we have one ?
Technically this depends on ability to construct process image manually in
Windows (*). Is it possible (using NT API of course) ? And, of course,
someone needs lots of spare time to code this. :) OTOH, CoLinux already does
this (yep, they are not 64-bit for now...), so perhaps there's no need to
duplicate the job done.

 P.P.S. Perhaps the answer to (*) is NO, otherwise we would have fast
fork()...

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Corinna Vinschen]

On Jun 18 10:56, Fedin Pavel wrote:
> [...]
>  P.S. I have got even more crazy idea, perhaps deserving a separate topic...
> BSD systems have Linux binary compatibility layer. Could we have one ?
> Technically this depends on ability to construct process image manually in
> Windows (*). Is it possible (using NT API of course) ? And, of course,
> someone needs lots of spare time to code this. :) OTOH, CoLinux already does
> this (yep, they are not 64-bit for now...), so perhaps there's no need to
> duplicate the job done.

Not in the Cygwin DLL.  As an extra DLL layer I don't have a problem
with that.

>  P.P.S. Perhaps the answer to (*) is NO, otherwise we would have fast
> fork()...

That's not quite correct.  The problem is not utilizing the native NT
functions to create a process image, the problem is that the Win32
libraries like advapi32, msvcrt, etc, which are directly or indirectly
loaded into a Cygwin process, are not capable to deal with a process
clone situation.  They were never designed this way and there's not much
chance they will ever get a revamp, just to allow Cygwin to create a
fast fork.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Cross-compiling Linux kernel

[Fedin Pavel]

 Hello!

> >  P.P.S. Perhaps the answer to (*) is NO, otherwise we would have fast
> > fork()...
> 
> That's not quite correct.  The problem is not utilizing the native NT
> functions to create a process image,

 Wow, interesting...
 I wonder if i could get a ELF with some plain hardcoded Windows syscall (an equivalent of OutputDebugString("Hello ELF world!");) running, just as a proof of concept.
 However, perhaps fully functional 32->64 bit bridges in /lib32 (or /bin32 ?) would be more useful for start. There is some prebuilt Cygwin software around, which is of course 32 bit and closed source (Perforce client is an example).

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Cross-compiling Linux kernel

[Fedin Pavel]

 Hello!

 While waiting for the Big Thing to finish compiling, another crazy idea visited my damaged brain. ;-) I wonder if it has some practical value...

> That's not quite correct.  The problem is not utilizing the native NT
> functions to create a process image, the problem is that the Win32
> libraries like advapi32, msvcrt, etc, which are directly or indirectly
> loaded into a Cygwin process, are not capable to deal with a process
> clone situation.

 So, in another words, we can clone everything except several known libraries. What if we use this fact ?
 Since we can create process image manually, what if we clone everything except these libraries ? Then we do a thing similar to what we do now, we start the program from the beginning but tell it to follow "short path", attach missing libraries and jump to our fork().
 Potential advantages:
 1. clone'able DLLs (i assume that all Cygwin DLLs are cloneable) are guaranteed to get the same addresses.
 2. (1) creates smaller number of variants for Windows native libraries, so we have smaller chances to get different addresses for them.

 Even more, what stops us from completely manual process image layout ? This way we could guarantee the same addresses for all libraries.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Corinna Vinschen]

On Jun 18 16:06, Fedin Pavel wrote:
>  Hello!
> 
>  While waiting for the Big Thing to finish compiling, another crazy idea visited my damaged brain. ;-) I wonder if it has some practical value...
> 
> > That's not quite correct.  The problem is not utilizing the native NT
> > functions to create a process image, the problem is that the Win32
> > libraries like advapi32, msvcrt, etc, which are directly or indirectly
> > loaded into a Cygwin process, are not capable to deal with a process
> > clone situation.
> 
>  So, in another words, we can clone everything except several known
>  libraries. What if we use this fact ?
>  Since we can create process image manually, what if we clone
>  everything except these libraries ? Then we do a thing similar to

How would you do that?  Either you use the native call to clone the
processes address space or you don't.  If you clone, you get the
entire address space layout, including all DLL images and heaps
reserved by OS DLLs.  These memory regions have to be free'd to be
able to reload the OS DLLs.  How do you know which memory region
has been reserved by which OS DLL?  Good luck implementing ;)

Btw., for reference:
http://social.msdn.microsoft.com/Forums/en-US/windowsgeneraldevelopmentissues/thread/afdf1b68-1f3e-47f5-94cf-51e397afe073

>  what we do now, we start the program from the beginning but tell it
>  to follow "short path", attach missing libraries and jump to our
>  fork().
>  Potential advantages:
>  1. clone'able DLLs (i assume that all Cygwin DLLs are cloneable) are
>  guaranteed to get the same addresses.
>  2. (1) creates smaller number of variants for Windows native
>  libraries, so we have smaller chances to get different addresses for
>  them.
> 
>  Even more, what stops us from completely manual process image layout
>  ? This way we could guarantee the same addresses for all libraries.

If you can get this working...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Christopher Faylor]

On Tue, Jun 18, 2013 at 12:17:06PM +0200, Corinna Vinschen wrote:
>On Jun 18 10:56, Fedin Pavel wrote:
>> [...]
>>  P.S. I have got even more crazy idea, perhaps deserving a separate topic...
>> BSD systems have Linux binary compatibility layer. Could we have one ?
>> Technically this depends on ability to construct process image manually in
>> Windows (*). Is it possible (using NT API of course) ? And, of course,
>> someone needs lots of spare time to code this. :) OTOH, CoLinux already does
>> this (yep, they are not 64-bit for now...), so perhaps there's no need to
>> duplicate the job done.
>
>Not in the Cygwin DLL.  As an extra DLL layer I don't have a problem
>with that.

There was a project out there many years ago which did this.  You could
run simple linux binaries on Windows.  I offered to host the development
on sourceware.org (aka cygwin.com) but, IIRC, the developer never
responded.

I don't remember what the name of the project was and I'm sure that it
won't work with more modern versions of Windows but it is a cool idea.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Christopher Faylor]

On Tue, Jun 18, 2013 at 10:33:08AM -0400, Christopher Faylor wrote:
>On Tue, Jun 18, 2013 at 12:17:06PM +0200, Corinna Vinschen wrote:
>>On Jun 18 10:56, Fedin Pavel wrote:
>>> [...]
>>>  P.S. I have got even more crazy idea, perhaps deserving a separate topic...
>>> BSD systems have Linux binary compatibility layer. Could we have one ?
>>> Technically this depends on ability to construct process image manually in
>>> Windows (*). Is it possible (using NT API of course) ? And, of course,
>>> someone needs lots of spare time to code this. :) OTOH, CoLinux already does
>>> this (yep, they are not 64-bit for now...), so perhaps there's no need to
>>> duplicate the job done.
>>
>>Not in the Cygwin DLL.  As an extra DLL layer I don't have a problem
>>with that.
>
>There was a project out there many years ago which did this.  You could
>run simple linux binaries on Windows.  I offered to host the development
>on sourceware.org (aka cygwin.com) but, IIRC, the developer never
>responded.
>
>I don't remember what the name of the project was and I'm sure that it
>won't work with more modern versions of Windows but it is a cool idea.

Here's one project: http://lbw.sourceforge.net/ but I don't remember if
this is the one I was thinking about or not.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[René Berber]

On 6/18/2013 9:37 AM, Christopher Faylor wrote:

>> There was a project out there many years ago which did this.  You could
>> run simple linux binaries on Windows.  I offered to host the development
>> on sourceware.org (aka cygwin.com) but, IIRC, the developer never
>> responded.
>>
>> I don't remember what the name of the project was and I'm sure that it
>> won't work with more modern versions of Windows but it is a cool idea.
>
> Here's one project: http://lbw.sourceforge.net/ but I don't remember if
> this is the one I was thinking about or not.

Here's another: http://atratus.org/

Quoting their page "Atratus is a Windows program that can run unmodified 
Linux binaries."  ... "Atratus can load ELF format executables created 
with gcc under Linux run them on a Windows system without a CPU emulator 
or virtual machine".

Haven't used it, but it does sound interesting, and its currently being 
developed (version 0.10).
-- 
René Berber

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Christopher Faylor]

On Tue, Jun 18, 2013 at 01:07:56PM -0500, Ren? Berber wrote:
>On 6/18/2013 9:37 AM, Christopher Faylor wrote:
>
>>> There was a project out there many years ago which did this.  You could
>>> run simple linux binaries on Windows.  I offered to host the development
>>> on sourceware.org (aka cygwin.com) but, IIRC, the developer never
>>> responded.
>>>
>>> I don't remember what the name of the project was and I'm sure that it
>>> won't work with more modern versions of Windows but it is a cool idea.
>>
>> Here's one project: http://lbw.sourceforge.net/ but I don't remember if
>> this is the one I was thinking about or not.
>
>Here's another: http://atratus.org/
>
>Quoting their page "Atratus is a Windows program that can run unmodified 
>Linux binaries."  ... "Atratus can load ELF format executables created 
>with gcc under Linux run them on a Windows system without a CPU emulator 
>or virtual machine".
>
>Haven't used it, but it does sound interesting, and its currently being 
>developed (version 0.10).

Interesting.  That website mentions LINE which, I think is the project I
was struggling to recall.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Larry Hall (Cygwin)]

On 6/18/2013 2:23 PM, Christopher Faylor wrote:
> On Tue, Jun 18, 2013 at 01:07:56PM -0500, Ren? Berber wrote:
>> On 6/18/2013 9:37 AM, Christopher Faylor wrote:
>>
>>>> There was a project out there many years ago which did this.  You could
>>>> run simple linux binaries on Windows.  I offered to host the development
>>>> on sourceware.org (aka cygwin.com) but, IIRC, the developer never
>>>> responded.
>>>>
>>>> I don't remember what the name of the project was and I'm sure that it
>>>> won't work with more modern versions of Windows but it is a cool idea.
>>>
>>> Here's one project: http://lbw.sourceforge.net/ but I don't remember if
>>> this is the one I was thinking about or not.
>>
>> Here's another: http://atratus.org/
>>
>> Quoting their page "Atratus is a Windows program that can run unmodified
>> Linux binaries."  ... "Atratus can load ELF format executables created
>> with gcc under Linux run them on a Windows system without a CPU emulator
>> or virtual machine".
>>
>> Haven't used it, but it does sound interesting, and its currently being
>> developed (version 0.10).
>
> Interesting.  That website mentions LINE which, I think is the project I
> was struggling to recall.

Yeah, I was just going to mention LINE.  I don't know why I can remember
the name of such an obscure (although very interesting!) project and not
the name of people I just met. ;-)

-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Dan Kegel]

On Tue, Jun 18, 2013 at 11:07 AM, René Berber <r.berber@computer.org> wrote:
> Here's another: http://atratus.org/

Aha.  That's by Mike McCormack, a Codeweavers/Wine alum, who also did
http://ring3k.org/

I wonder how far atratus is from running wine.
At which point one could try running cygwin on wine on atratus on windows :-)

Which reminds me: any interest in fixing an alleged cygwin stack
problem that bothers wine?
http://bugs.winehq.org/show_bug.cgi?id=24018#c9
- Dan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cross-compiling Linux kernel

[Christopher Faylor]

On Tue, Jun 18, 2013 at 07:05:56PM -0400, Larry Hall (Cygwin) wrote:
>On 6/18/2013 2:23 PM, Christopher Faylor wrote:
>> On Tue, Jun 18, 2013 at 01:07:56PM -0500, Ren? Berber wrote:
>>> On 6/18/2013 9:37 AM, Christopher Faylor wrote:
>>>
>>>>> There was a project out there many years ago which did this.  You could
>>>>> run simple linux binaries on Windows.  I offered to host the development
>>>>> on sourceware.org (aka cygwin.com) but, IIRC, the developer never
>>>>> responded.
>>>>>
>>>>> I don't remember what the name of the project was and I'm sure that it
>>>>> won't work with more modern versions of Windows but it is a cool idea.
>>>>
>>>> Here's one project: http://lbw.sourceforge.net/ but I don't remember if
>>>> this is the one I was thinking about or not.
>>>
>>> Here's another: http://atratus.org/
>>>
>>> Quoting their page "Atratus is a Windows program that can run unmodified
>>> Linux binaries."  ... "Atratus can load ELF format executables created
>>> with gcc under Linux run them on a Windows system without a CPU emulator
>>> or virtual machine".
>>>
>>> Haven't used it, but it does sound interesting, and its currently being
>>> developed (version 0.10).
>>
>> Interesting.  That website mentions LINE which, I think is the project I
>> was struggling to recall.
>
>Yeah, I was just going to mention LINE.  I don't know why I can remember
>the name of such an obscure (although very interesting!) project and not
>the name of people I just met. ;-)

Don't feel bad about it Harry.  It happens to all of us.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple