Re: GCC changes for Fedora + riscv64

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
> Hi GCC maintainers!
> 
> I'd like to talk about what changes we (may) need to GCC in
> Fedora to get it working on 64-bit RISC-V, and also (more
> importantly) to ask your advice on things we don't fully
> understand yet.  However, I don't know even what venue you'd
> prefer to discuss this in.
> 
> So if you could let me know that I'll try to get something together
> and post it there.
Not really sure what you're asking here Richard.  In general patches
should be posted to gcc-patches@gcc.gnu.org.

Jim Wilson is probably a good person to know.  He's a long time GCC
contributor and has recently joined sifive.

jimw@sifive.com

Jeff

Re: GCC changes for Fedora + riscv64

[Richard W.M. Jones]

On Sun, Apr 08, 2018 at 09:22:07AM -0600, Jeff Law wrote:
> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
> > Hi GCC maintainers!
> > 
> > I'd like to talk about what changes we (may) need to GCC in
> > Fedora to get it working on 64-bit RISC-V, and also (more
> > importantly) to ask your advice on things we don't fully
> > understand yet.  However, I don't know even what venue you'd
> > prefer to discuss this in.
> > 
> > So if you could let me know that I'll try to get something together
> > and post it there.
> Not really sure what you're asking here Richard.  In general patches
> should be posted to gcc-patches@gcc.gnu.org.

I mean patches to the Fedora GCC package, ie. changes to gcc.spec etc
and (potentially) downstream only changes.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Re: GCC changes for Fedora + riscv64

[Jim Wilson]

On 04/08/2018 08:22 AM, Jeff Law wrote:
> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
>> I'd like to talk about what changes we (may) need to GCC in
>> Fedora to get it working on 64-bit RISC-V, and also (more
>> importantly) to ask your advice on things we don't fully
>> understand yet.  However, I don't know even what venue you'd
>> prefer to discuss this in.

A discussion here is fine with me.  I know of a few issues.

I have a work-in-progress --with-multilib-list patch in PR 84797 but it 
isn't quite right yet, and needs to work more like the patch in PR 
85142, which isn't OK to check in.

There is a problem with atomics.  We only have builtins for the ones 
that can be implemented with a single instruction.  Adding -latomic 
unconditionally might fix it, but won't work for gcc builds and the gcc 
testsuite unless we also add paths pointing into the libatomic build 
dir.  I'm also concerned that this might cause build problems, if we end 
up trying to link with libatomic before we have built it.  The simplest 
solution might be to just add expanders for all of the missing atomics, 
even if they require multiple instructions, just like how all of the 
mainstream linux targets currently work.

There is a problem with the linker not searching the right set of dirs 
by default.  That is more a binutils problem than a gcc problem, but the 
linker might need some help from gcc to fix it, as the linker doesn't 
normally take -march and -mabi options.

There is a problem with libffi, which has RISC-V support upstream, but 
not in the FSF GCC copy.  This is needed for go language support.  There 
was also a dispute about go something naming, as to whether it should be 
riscv64 or riscv, with one person doing a port choosing the former and 
another person doing another port choosing the latter.

Those are all of the Linux specific ones I can remember at the moment. 
I might have missed some.

Jim

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 04/09/2018 12:04 PM, Jim Wilson wrote:
> On 04/08/2018 08:22 AM, Jeff Law wrote:
>> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
>>> I'd like to talk about what changes we (may) need to GCC in
>>> Fedora to get it working on 64-bit RISC-V, and also (more
>>> importantly) to ask your advice on things we don't fully
>>> understand yet.  However, I don't know even what venue you'd
>>> prefer to discuss this in.
> 
> A discussion here is fine with me.  I know of a few issues.
> 
> I have a work-in-progress --with-multilib-list patch in PR 84797 but it
> isn't quite right yet, and needs to work more like the patch in PR
> 85142, which isn't OK to check in.
> 
> There is a problem with atomics.  We only have builtins for the ones
> that can be implemented with a single instruction.  Adding -latomic
> unconditionally might fix it, but won't work for gcc builds and the gcc
> testsuite unless we also add paths pointing into the libatomic build
> dir.  I'm also concerned that this might cause build problems, if we end
> up trying to link with libatomic before we have built it.  The simplest
> solution might be to just add expanders for all of the missing atomics,
> even if they require multiple instructions, just like how all of the
> mainstream linux targets currently work.
> 
> There is a problem with the linker not searching the right set of dirs
> by default.  That is more a binutils problem than a gcc problem, but the
> linker might need some help from gcc to fix it, as the linker doesn't
> normally take -march and -mabi options.
> 
> There is a problem with libffi, which has RISC-V support upstream, but
> not in the FSF GCC copy.  This is needed for go language support.  There
> was also a dispute about go something naming, as to whether it should be
> riscv64 or riscv, with one person doing a port choosing the former and
> another person doing another port choosing the latter.
> 
> Those are all of the Linux specific ones I can remember at the moment. I
> might have missed some.
Are you guys using qemu user mode emulation for testing purposes?  When
I've set up a suitable riscv64 rootfs and try to do anything nontrivial
in it with qemu user mode emulation it immediately complains that my
kernel is too old -- which is quite odd as I've got a dozen or so of
these kinds of environments set up for testing which don't issue that
complaint.

I'd like to avoid full system emulation just from a cost standpoint..

Jeff

Re: GCC changes for Fedora + riscv64

[Stef O'Rear]

On Mon, Apr 9, 2018 at 6:37 PM, Jeff Law <law@redhat.com> wrote:
> On 04/09/2018 12:04 PM, Jim Wilson wrote:
>> On 04/08/2018 08:22 AM, Jeff Law wrote:
>>> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
>>>> I'd like to talk about what changes we (may) need to GCC in
>>>> Fedora to get it working on 64-bit RISC-V, and also (more
>>>> importantly) to ask your advice on things we don't fully
>>>> understand yet.  However, I don't know even what venue you'd
>>>> prefer to discuss this in.
>>
>> A discussion here is fine with me.  I know of a few issues.
>>
>> I have a work-in-progress --with-multilib-list patch in PR 84797 but it
>> isn't quite right yet, and needs to work more like the patch in PR
>> 85142, which isn't OK to check in.
>>
>> There is a problem with atomics.  We only have builtins for the ones
>> that can be implemented with a single instruction.  Adding -latomic
>> unconditionally might fix it, but won't work for gcc builds and the gcc
>> testsuite unless we also add paths pointing into the libatomic build
>> dir.  I'm also concerned that this might cause build problems, if we end
>> up trying to link with libatomic before we have built it.  The simplest
>> solution might be to just add expanders for all of the missing atomics,
>> even if they require multiple instructions, just like how all of the
>> mainstream linux targets currently work.
>>
>> There is a problem with the linker not searching the right set of dirs
>> by default.  That is more a binutils problem than a gcc problem, but the
>> linker might need some help from gcc to fix it, as the linker doesn't
>> normally take -march and -mabi options.
>>
>> There is a problem with libffi, which has RISC-V support upstream, but
>> not in the FSF GCC copy.  This is needed for go language support.  There
>> was also a dispute about go something naming, as to whether it should be
>> riscv64 or riscv, with one person doing a port choosing the former and
>> another person doing another port choosing the latter.
>>
>> Those are all of the Linux specific ones I can remember at the moment. I
>> might have missed some.
> Are you guys using qemu user mode emulation for testing purposes?  When
> I've set up a suitable riscv64 rootfs and try to do anything nontrivial
> in it with qemu user mode emulation it immediately complains that my
> kernel is too old -- which is quite odd as I've got a dozen or so of
> these kinds of environments set up for testing which don't issue that
> complaint.
>
> I'd like to avoid full system emulation just from a cost standpoint..

That error is produced by glibc if uname() returns a kernel version
older than the oldest with arch support, which for riscv is 4.15.0.
qemu-user does not fake uname(), so qemu-user for riscv will only work
if the host is running 4.15.0 or newer.

-s

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 04/09/2018 04:47 PM, Stef O'Rear wrote:
> On Mon, Apr 9, 2018 at 6:37 PM, Jeff Law <law@redhat.com> wrote:
>> On 04/09/2018 12:04 PM, Jim Wilson wrote:
>>> On 04/08/2018 08:22 AM, Jeff Law wrote:
>>>> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
>>>>> I'd like to talk about what changes we (may) need to GCC in
>>>>> Fedora to get it working on 64-bit RISC-V, and also (more
>>>>> importantly) to ask your advice on things we don't fully
>>>>> understand yet.  However, I don't know even what venue you'd
>>>>> prefer to discuss this in.
>>>
>>> A discussion here is fine with me.  I know of a few issues.
>>>
>>> I have a work-in-progress --with-multilib-list patch in PR 84797 but it
>>> isn't quite right yet, and needs to work more like the patch in PR
>>> 85142, which isn't OK to check in.
>>>
>>> There is a problem with atomics.  We only have builtins for the ones
>>> that can be implemented with a single instruction.  Adding -latomic
>>> unconditionally might fix it, but won't work for gcc builds and the gcc
>>> testsuite unless we also add paths pointing into the libatomic build
>>> dir.  I'm also concerned that this might cause build problems, if we end
>>> up trying to link with libatomic before we have built it.  The simplest
>>> solution might be to just add expanders for all of the missing atomics,
>>> even if they require multiple instructions, just like how all of the
>>> mainstream linux targets currently work.
>>>
>>> There is a problem with the linker not searching the right set of dirs
>>> by default.  That is more a binutils problem than a gcc problem, but the
>>> linker might need some help from gcc to fix it, as the linker doesn't
>>> normally take -march and -mabi options.
>>>
>>> There is a problem with libffi, which has RISC-V support upstream, but
>>> not in the FSF GCC copy.  This is needed for go language support.  There
>>> was also a dispute about go something naming, as to whether it should be
>>> riscv64 or riscv, with one person doing a port choosing the former and
>>> another person doing another port choosing the latter.
>>>
>>> Those are all of the Linux specific ones I can remember at the moment. I
>>> might have missed some.
>> Are you guys using qemu user mode emulation for testing purposes?  When
>> I've set up a suitable riscv64 rootfs and try to do anything nontrivial
>> in it with qemu user mode emulation it immediately complains that my
>> kernel is too old -- which is quite odd as I've got a dozen or so of
>> these kinds of environments set up for testing which don't issue that
>> complaint.
>>
>> I'd like to avoid full system emulation just from a cost standpoint..
> 
> That error is produced by glibc if uname() returns a kernel version
> older than the oldest with arch support, which for riscv is 4.15.0.
> qemu-user does not fake uname(), so qemu-user for riscv will only work
> if the host is running 4.15.0 or newer.
Hmm, makes sense since the hosts are running 4.10-ish

I'll hack around it somehow.  I'd really like to add riscv64 to the
tester and don't want to mess around with kernel updates.

Thanks,
jeff

Re: GCC changes for Fedora + riscv64

[Kito Cheng]

Hi Jeff:

You can use -r option (e.g. ./qemu-riscv64 -r 4.15) or set QEMU_UNAME
environment variable to change the uname for qemu.

On Tue, Apr 10, 2018 at 6:50 AM, Jeff Law <law@redhat.com> wrote:
> On 04/09/2018 04:47 PM, Stef O'Rear wrote:
>> On Mon, Apr 9, 2018 at 6:37 PM, Jeff Law <law@redhat.com> wrote:
>>> On 04/09/2018 12:04 PM, Jim Wilson wrote:
>>>> On 04/08/2018 08:22 AM, Jeff Law wrote:
>>>>> On 03/31/2018 12:27 PM, Richard W.M. Jones wrote:
>>>>>> I'd like to talk about what changes we (may) need to GCC in
>>>>>> Fedora to get it working on 64-bit RISC-V, and also (more
>>>>>> importantly) to ask your advice on things we don't fully
>>>>>> understand yet.  However, I don't know even what venue you'd
>>>>>> prefer to discuss this in.
>>>>
>>>> A discussion here is fine with me.  I know of a few issues.
>>>>
>>>> I have a work-in-progress --with-multilib-list patch in PR 84797 but it
>>>> isn't quite right yet, and needs to work more like the patch in PR
>>>> 85142, which isn't OK to check in.
>>>>
>>>> There is a problem with atomics.  We only have builtins for the ones
>>>> that can be implemented with a single instruction.  Adding -latomic
>>>> unconditionally might fix it, but won't work for gcc builds and the gcc
>>>> testsuite unless we also add paths pointing into the libatomic build
>>>> dir.  I'm also concerned that this might cause build problems, if we end
>>>> up trying to link with libatomic before we have built it.  The simplest
>>>> solution might be to just add expanders for all of the missing atomics,
>>>> even if they require multiple instructions, just like how all of the
>>>> mainstream linux targets currently work.
>>>>
>>>> There is a problem with the linker not searching the right set of dirs
>>>> by default.  That is more a binutils problem than a gcc problem, but the
>>>> linker might need some help from gcc to fix it, as the linker doesn't
>>>> normally take -march and -mabi options.
>>>>
>>>> There is a problem with libffi, which has RISC-V support upstream, but
>>>> not in the FSF GCC copy.  This is needed for go language support.  There
>>>> was also a dispute about go something naming, as to whether it should be
>>>> riscv64 or riscv, with one person doing a port choosing the former and
>>>> another person doing another port choosing the latter.
>>>>
>>>> Those are all of the Linux specific ones I can remember at the moment. I
>>>> might have missed some.
>>> Are you guys using qemu user mode emulation for testing purposes?  When
>>> I've set up a suitable riscv64 rootfs and try to do anything nontrivial
>>> in it with qemu user mode emulation it immediately complains that my
>>> kernel is too old -- which is quite odd as I've got a dozen or so of
>>> these kinds of environments set up for testing which don't issue that
>>> complaint.
>>>
>>> I'd like to avoid full system emulation just from a cost standpoint..
>>
>> That error is produced by glibc if uname() returns a kernel version
>> older than the oldest with arch support, which for riscv is 4.15.0.
>> qemu-user does not fake uname(), so qemu-user for riscv will only work
>> if the host is running 4.15.0 or newer.
> Hmm, makes sense since the hosts are running 4.10-ish
>
> I'll hack around it somehow.  I'd really like to add riscv64 to the
> tester and don't want to mess around with kernel updates.
>
> Thanks,
> jeff

Re: GCC changes for Fedora + riscv64

[Andreas Schwab]

On Apr 10 2018, Kito Cheng <kito.cheng@gmail.com> wrote:

> You can use -r option (e.g. ./qemu-riscv64 -r 4.15) or set QEMU_UNAME
> environment variable to change the uname for qemu.

But be aware that the emulated environment still assumes that all system
calls implemented by 4.15 are available, and qemu linux-user generally
just passes them through to the host kernel.  If the host kernel is too
old to provide a particular system call the emulation may malfunction.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: GCC changes for Fedora + riscv64

[Kito Cheng]

>> You can use -r option (e.g. ./qemu-riscv64 -r 4.15) or set QEMU_UNAME
>> environment variable to change the uname for qemu.
>
> But be aware that the emulated environment still assumes that all system
> calls implemented by 4.15 are available, and qemu linux-user generally
> just passes them through to the host kernel.  If the host kernel is too
> old to provide a particular system call the emulation may malfunction.

Yeah, but it's enough for running gcc testsuite.

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 04/10/2018 01:55 AM, Andreas Schwab wrote:
> On Apr 10 2018, Kito Cheng <kito.cheng@gmail.com> wrote:
> 
>> You can use -r option (e.g. ./qemu-riscv64 -r 4.15) or set QEMU_UNAME
>> environment variable to change the uname for qemu.
> 
> But be aware that the emulated environment still assumes that all system
> calls implemented by 4.15 are available, and qemu linux-user generally
> just passes them through to the host kernel.  If the host kernel is too
> old to provide a particular system call the emulation may malfunction.
Well understood.  For my needs it likely is a non-issue.

jeff

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 04/09/2018 11:01 PM, Kito Cheng wrote:
> Hi Jeff:
> 
> You can use -r option (e.g. ./qemu-riscv64 -r 4.15) or set QEMU_UNAME
> environment variable to change the uname for qemu.
That did the trick.  Exactly what I needed.

Thanks!

jeff

Re: GCC changes for Fedora + riscv64

[Richard W.M. Jones]

On Mon, Apr 09, 2018 at 04:37:30PM -0600, Jeff Law wrote:
> Are you guys using qemu user mode emulation for testing purposes?  When
> I've set up a suitable riscv64 rootfs and try to do anything nontrivial
> in it with qemu user mode emulation it immediately complains that my
> kernel is too old -- which is quite odd as I've got a dozen or so of
> these kinds of environments set up for testing which don't issue that
> complaint.
> 
> I'd like to avoid full system emulation just from a cost standpoint..

We're using full system emulation with upstream qemu.  It's
surprisingly simple to set up, it supports multiple virtual CPUs up to
‘-smp 8’, and provided you have a fast enough host hardware isn't too
bad for development.  Of course compiling GCC with bootstrapping is
still going to take a long time.  For instructions see the readme
here:

  https://fedorapeople.org/groups/risc-v/disk-images/

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: GCC changes for Fedora + riscv64

[Jeff Law]

On 04/10/2018 12:27 AM, Richard W.M. Jones wrote:
> On Mon, Apr 09, 2018 at 04:37:30PM -0600, Jeff Law wrote:
>> Are you guys using qemu user mode emulation for testing purposes?  When
>> I've set up a suitable riscv64 rootfs and try to do anything nontrivial
>> in it with qemu user mode emulation it immediately complains that my
>> kernel is too old -- which is quite odd as I've got a dozen or so of
>> these kinds of environments set up for testing which don't issue that
>> complaint.
>>
>> I'd like to avoid full system emulation just from a cost standpoint..
> 
> We're using full system emulation with upstream qemu.  It's
> surprisingly simple to set up, it supports multiple virtual CPUs up to
> ‘-smp 8’, and provided you have a fast enough host hardware isn't too
> bad for development.  Of course compiling GCC with bootstrapping is
> still going to take a long time.  For instructions see the readme
> here:
It's actually a continuous testing system rather than development :-)  A
typical target using user mode emulation takes 6-10 hours to to run
through its paces on one of our 56 processor build machines.  I'd hazard
a guess if we do full system emulation then we're likely looking at
doubling that or worse, which I'd like to avoid given the number of
targets I'm trying to get tested in a 24hr cycle.


> 
>   https://fedorapeople.org/groups/risc-v/disk-images/
Yea.  I actually looked at those briefly to confirm that kernel too old
issue triggered there too.  For my tester I actually use mini roots that
I cross build for the various targets.  The advantage is the contents of
the mini roots is consistent across the targets.


Thanks,

Jeff