Re: [PATCH 0/9] eBPF support for GNU binutils

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jiong Wang]

CCing BPF kernel community who is defining the ISA and various runtime stuff.

Also two inline comments below about the assembler

> On 20 May 2019, at 17:45, Jose E. Marchesi <jose.marchesi oracle ! com> wrote:
> 
> Hi people!
> 
> This patch series introduces support for eBPF, which is a virtual
> machine that resides in the Linux kernel.  Initially intended for
> user-level packet capture and filtering, eBPF is nowadays generalized
> to serve as a general-purpose infrastructure also for non-networking
> purposes.
> 
> The first patch is preparatory, and adds support to config.guess to
> recognize bpf-*-* triplets.  This will be submitted as a patch to the
> `config' project as soon as this series gets upstreamed.
> 
> The second and third patches add support for an ELF64 based eBPF
> target to BFD, in both little-endian and big-endian vectors.
> 
> The fourth patch adds a CGEN cpu description for eBPF, plus support
> code.  This description covers the full eBPF ISA.  Due to the 64-bit
> instruction fields used in some instructions, we needed to fix a
> bug/limitation in CGEN impacting 32-bit hosts.  The fix is in a patch
> submitted to CGEN last week, that is still waiting for review:
> http://www.sourceware.org/ml/cgen/2019-q2/msg00008.html None of the
> existing CGEN ports in binutils are impacted by that patch: the code
> generated for these remains exactly the same.
> 
> The fifth patch adds opcodes and disassembler support for eBPF, based
> on the CGEN description.
> 
> The sixth patch adds a GAS port, including a testsuite and manual
> updates.  By default the assembler generates objects using the same
> endianness than the host.  This can be overrided by the usual -EB and
> -EB command-line options.
> 
> Support for linking eBPF ELF files with ld/bfd is provided in the
> seventh patch.  A couple of simple tests are included.
> 
> The eighth patch adds support for eBPF to readelf, and makes a little
> adjustment in the `nm' testsuite to not fail in bpf-*-* targets.
> 
> Finally, the last patch adds myself as the maintainer of the BPF
> target.  We are committing to maintain this port.
> 
> Future work on the binutils port:
> * Support for semantic actions in bpf.cpu, and support code for a
>  simulator in sim/.
> * Support for ld.gold.
> 
> Next stop is GCC.  An eBPF backend is on the works.  We plan to
> upstream it before September.
> 
> Regressions tested in all targets.
> Regressions tested with --enable-targets=all
> Tested in 64-bit x86_64 host.
> Tested in 32-bit x86 host.
> 
> Oh, a little note regarding interoperability:
> 
> There is a clang/llvm based toolchain for eBPF.  However, at this
> moment compiled eBPF doesn't have established conventions.  The
> details on what is expected to be in an ELF file containing eBPF is
> determined, in practice, by what the llvm BPF backend supports and
> what the sample bpf_load.c in the Linux kernel source tree expects
> [1].
> 
> Despite using a different syntax for the assembler (the llvm assembler
> uses a C-ish expression-based syntax while the GNU assembler opts for
> a more classic assembly-language syntax) this implementation tries to
> provide inter-operability with clang/llvm generated objects.

I also noticed your implementation doesn’t seem to use the same sub-register
syntax as what LLVM assembler is doing.

  x register for 64-bit, and w register for 32-bit sub-register.

So:
  add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
  add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X

ASAICT, different register prefix for different register width is also adopted
by quite a few other GNU assembler targets like AArch64, X86_64.

> 
> In particular, the numbers of the relocations used for instruction
> fields are the same.  These are R_BPF_INSN_64 and R_BPF_INSN_DISP32.
> The later is resolved at load-time by bpf_load.c.

I think you missed the latest JMP32 instructions.

  https://github.com/torvalds/linux/blob/master/Documentation/networking/filter.txt#L870


> 
> [1] We expect/hope that the addition of eBPF support to the GNU
>    toolchain will help to mature the domain of compiled eBPF.  We
>    will certainly be working with the kernel people to that effect.
> 
> Salud!
> 
> Jose E. Marchesi (9):
>  config: recognize eBPF triplets
>  include: add elf/bpf.h
>  bfd: add support for eBPF
>  cpu: add eBPF cpu description
>  opcodes: add support for eBPF
>  gas: add support for eBPF
>  ld: add support for eBPF
>  binutils: add support for eBPF
>  binutils: add myself as the maintainer for BPF
> 
> ChangeLog                              |    4 +
> bfd/ChangeLog                          |   20 +
> bfd/Makefile.am                        |    4 +
> bfd/Makefile.in                        |    7 +
> bfd/archures.c                         |    4 +
> bfd/bfd-in2.h                          |    9 +
> bfd/config.bfd                         |    6 +
> bfd/configure                          |   30 +-
> bfd/configure.ac                       |    2 +
> bfd/cpu-bpf.c                          |   41 +
> bfd/elf64-bpf.c                        |  463 +++++++++
> bfd/libbfd.h                           |    5 +
> bfd/reloc.c                            |   13 +
> bfd/targets.c                          |    7 +
> binutils/ChangeLog                     |   13 +
> binutils/MAINTAINERS                   |    1 +
> binutils/readelf.c                     |    8 +
> binutils/testsuite/binutils-all/nm.exp |    3 +-
> config.sub                             |    4 +-
> cpu/ChangeLog                          |    5 +
> cpu/bpf.cpu                            |  647 +++++++++++++
> cpu/bpf.opc                            |  191 ++++
> gas/ChangeLog                          |   45 +
> gas/Makefile.am                        |    2 +
> gas/Makefile.in                        |    6 +
> gas/config/tc-bpf.c                    |  357 +++++++
> gas/config/tc-bpf.h                    |   51 +
> gas/configure                          |   38 +-
> gas/configure.ac                       |    6 +
> gas/configure.tgt                      |    1 +
> gas/doc/Makefile.am                    |    1 +
> gas/doc/Makefile.in                    |    6 +-
> gas/doc/all.texi                       |    1 +
> gas/doc/as.texi                        |   34 +
> gas/doc/c-bpf.texi                     |  364 +++++++
> gas/testsuite/gas/all/gas.exp          |    3 +
> gas/testsuite/gas/all/org-1.l          |    2 +-
> gas/testsuite/gas/all/org-1.s          |    2 +
> gas/testsuite/gas/bpf/alu-be.d         |   59 ++
> gas/testsuite/gas/bpf/alu.d            |   58 ++
> gas/testsuite/gas/bpf/alu.s            |   51 +
> gas/testsuite/gas/bpf/alu32-be.d       |   65 ++
> gas/testsuite/gas/bpf/alu32.d          |   64 ++
> gas/testsuite/gas/bpf/alu32.s          |   57 ++
> gas/testsuite/gas/bpf/atomic-be.d      |   12 +
> gas/testsuite/gas/bpf/atomic.d         |   11 +
> gas/testsuite/gas/bpf/atomic.s         |    5 +
> gas/testsuite/gas/bpf/bpf.exp          |   38 +
> gas/testsuite/gas/bpf/call-be.d        |   19 +
> gas/testsuite/gas/bpf/call.d           |   18 +
> gas/testsuite/gas/bpf/call.s           |   11 +
> gas/testsuite/gas/bpf/exit-be.d        |   11 +
> gas/testsuite/gas/bpf/exit.d           |   10 +
> gas/testsuite/gas/bpf/exit.s           |    2 +
> gas/testsuite/gas/bpf/jump-be.d        |   32 +
> gas/testsuite/gas/bpf/jump.d           |   31 +
> gas/testsuite/gas/bpf/jump.s           |   25 +
> gas/testsuite/gas/bpf/lddw-be.d        |   18 +
> gas/testsuite/gas/bpf/lddw.d           |   17 +
> gas/testsuite/gas/bpf/lddw.s           |    6 +
> gas/testsuite/gas/bpf/mem-be.d         |   30 +
> gas/testsuite/gas/bpf/mem.d            |   29 +
> gas/testsuite/gas/bpf/mem.s            |   24 +
> include/ChangeLog                      |    4 +
> include/elf/bpf.h                      |   45 +
> ld/ChangeLog                           |   15 +
> ld/Makefile.am                         |    2 +
> ld/Makefile.in                         |    4 +
> ld/configure                           |   28 +-
> ld/configure.tgt                       |    1 +
> ld/emulparams/elf64bpf.sh              |   10 +
> ld/testsuite/ld-bpf/bar.s              |    5 +
> ld/testsuite/ld-bpf/baz.s              |    5 +
> ld/testsuite/ld-bpf/bpf.exp            |   29 +
> ld/testsuite/ld-bpf/call-1.d           |   23 +
> ld/testsuite/ld-bpf/foo.s              |    5 +
> ld/testsuite/ld-bpf/jump-1.d           |   23 +
> ld/testsuite/lib/ld-lib.exp            |    1 +
> opcodes/ChangeLog                      |   24 +
> opcodes/Makefile.am                    |   17 +
> opcodes/Makefile.in                    |   23 +
> opcodes/bpf-asm.c                      |  590 ++++++++++++
> opcodes/bpf-desc.c                     | 1638 ++++++++++++++++++++++++++++++++
> opcodes/bpf-desc.h                     |  266 ++++++
> opcodes/bpf-dis.c                      |  624 ++++++++++++
> opcodes/bpf-ibld.c                     |  956 +++++++++++++++++++
> opcodes/bpf-opc.c                      | 1495 +++++++++++++++++++++++++++++
> opcodes/bpf-opc.h                      |  151 +++
> opcodes/configure                      |   19 +-
> opcodes/configure.ac                   |    1 +
> opcodes/disassemble.c                  |   35 +
> opcodes/disassemble.h                  |    1 +
> 92 files changed, 9116 insertions(+), 33 deletions(-)
> create mode 100644 bfd/cpu-bpf.c
> create mode 100644 bfd/elf64-bpf.c
> create mode 100644 cpu/bpf.cpu
> create mode 100644 cpu/bpf.opc
> create mode 100644 gas/config/tc-bpf.c
> create mode 100644 gas/config/tc-bpf.h
> create mode 100644 gas/doc/c-bpf.texi
> create mode 100644 gas/testsuite/gas/bpf/alu-be.d
> create mode 100644 gas/testsuite/gas/bpf/alu.d
> create mode 100644 gas/testsuite/gas/bpf/alu.s
> create mode 100644 gas/testsuite/gas/bpf/alu32-be.d
> create mode 100644 gas/testsuite/gas/bpf/alu32.d
> create mode 100644 gas/testsuite/gas/bpf/alu32.s
> create mode 100644 gas/testsuite/gas/bpf/atomic-be.d
> create mode 100644 gas/testsuite/gas/bpf/atomic.d
> create mode 100644 gas/testsuite/gas/bpf/atomic.s
> create mode 100644 gas/testsuite/gas/bpf/bpf.exp
> create mode 100644 gas/testsuite/gas/bpf/call-be.d
> create mode 100644 gas/testsuite/gas/bpf/call.d
> create mode 100644 gas/testsuite/gas/bpf/call.s
> create mode 100644 gas/testsuite/gas/bpf/exit-be.d
> create mode 100644 gas/testsuite/gas/bpf/exit.d
> create mode 100644 gas/testsuite/gas/bpf/exit.s
> create mode 100644 gas/testsuite/gas/bpf/jump-be.d
> create mode 100644 gas/testsuite/gas/bpf/jump.d
> create mode 100644 gas/testsuite/gas/bpf/jump.s
> create mode 100644 gas/testsuite/gas/bpf/lddw-be.d
> create mode 100644 gas/testsuite/gas/bpf/lddw.d
> create mode 100644 gas/testsuite/gas/bpf/lddw.s
> create mode 100644 gas/testsuite/gas/bpf/mem-be.d
> create mode 100644 gas/testsuite/gas/bpf/mem.d
> create mode 100644 gas/testsuite/gas/bpf/mem.s
> create mode 100644 include/elf/bpf.h
> create mode 100644 ld/emulparams/elf64bpf.sh
> create mode 100644 ld/testsuite/ld-bpf/bar.s
> create mode 100644 ld/testsuite/ld-bpf/baz.s
> create mode 100644 ld/testsuite/ld-bpf/bpf.exp
> create mode 100644 ld/testsuite/ld-bpf/call-1.d
> create mode 100644 ld/testsuite/ld-bpf/foo.s
> create mode 100644 ld/testsuite/ld-bpf/jump-1.d
> create mode 100644 opcodes/bpf-asm.c
> create mode 100644 opcodes/bpf-desc.c
> create mode 100644 opcodes/bpf-desc.h
> create mode 100644 opcodes/bpf-dis.c
> create mode 100644 opcodes/bpf-ibld.c
> create mode 100644 opcodes/bpf-opc.c
> create mode 100644 opcodes/bpf-opc.h
> 
> -- 
> 2.11.0

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

Hi Jiong.

    > Despite using a different syntax for the assembler (the llvm assembler
    > uses a C-ish expression-based syntax while the GNU assembler opts for
    > a more classic assembly-language syntax) this implementation tries to
    > provide inter-operability with clang/llvm generated objects.
    
    I also noticed your implementation doesn’t seem to use the same sub-register
    syntax as what LLVM assembler is doing.
    
      x register for 64-bit, and w register for 32-bit sub-register.
    
    So:
      add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
      add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X
    
    ASAICT, different register prefix for different register width is also adopted
    by quite a few other GNU assembler targets like AArch64, X86_64.

Right.  I opted for using different mnemonics for alu and alu64
instructions, as it seemed to be simpler.

What was your rationale for using sub-register notation?  Are you
planning to support instructions (or pseudo-instructions) mixing w and x
registers in the future?

    > In particular, the numbers of the relocations used for instruction
    > fields are the same.  These are R_BPF_INSN_64 and R_BPF_INSN_DISP32.
    > The later is resolved at load-time by bpf_load.c.
    
    I think you missed the latest JMP32 instructions.
    
      https://github.com/torvalds/linux/blob/master/Documentation/networking/filter.txt#L870

Oh thanks for spotting that.
Adding support for it :)

Re: [PATCH 0/9] eBPF support for GNU binutils

[David Miller]

From: Jiong Wang <jiong.wang@netronome.com>
Date: Tue, 21 May 2019 16:41:56 +0100

> CCing BPF kernel community who is defining the ISA and various runtime stuff.
> 
> Also two inline comments below about the assembler

Also, I already wrote such a binutils patch a year or two ago:

https://lwn.net/Articles/721473/

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jiong Wang]

> On 21 May 2019, at 18:06, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:
> 
> 
> Hi Jiong.
> 
>> Despite using a different syntax for the assembler (the llvm assembler
>> uses a C-ish expression-based syntax while the GNU assembler opts for
>> a more classic assembly-language syntax) this implementation tries to
>> provide inter-operability with clang/llvm generated objects.
> 
>    I also noticed your implementation doesn’t seem to use the same sub-register
>    syntax as what LLVM assembler is doing.
> 
>      x register for 64-bit, and w register for 32-bit sub-register.
> 
>    So:
>      add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
>      add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X
> 
>    ASAICT, different register prefix for different register width is also adopted
>    by quite a few other GNU assembler targets like AArch64, X86_64.
> 
> Right.  I opted for using different mnemonics for alu and alu64
> instructions, as it seemed to be simpler.
> 
> What was your rationale for using sub-register notation?  

It is the same instruction operating on different register classes, sub-register
is a new register class, so define separate notation for them. This also
simplifies compiler back-end when generating sub-register instructions, at
least for LLVM, and is likely for GCC as well. 

LLVM eBPF backend has full support for generating sub-register ISA,


> Are you
> planning to support instructions (or pseudo-instructions) mixing w and x
> registers in the future?
> 
>> In particular, the numbers of the relocations used for instruction
>> fields are the same.  These are R_BPF_INSN_64 and R_BPF_INSN_DISP32.
>> The later is resolved at load-time by bpf_load.c.
> 
>    I think you missed the latest JMP32 instructions.
> 
>      https://github.com/torvalds/linux/blob/master/Documentation/networking/filter.txt#L870
> 
> Oh thanks for spotting that.
> Adding support for it :)

Re: [PATCH 0/9] eBPF support for GNU binutils

[Alexei Starovoitov]

On Tue, May 21, 2019 at 8:41 AM Jiong Wang <jiong.wang@netronome.com> wrote:
> >
> > Despite using a different syntax for the assembler (the llvm assembler
> > uses a C-ish expression-based syntax while the GNU assembler opts for
> > a more classic assembly-language syntax) this implementation tries to
> > provide inter-operability with clang/llvm generated objects.
>
> I also noticed your implementation doesn’t seem to use the same sub-register
> syntax as what LLVM assembler is doing.
>
>   x register for 64-bit, and w register for 32-bit sub-register.
>
> So:
>   add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
>   add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X
>
> ASAICT, different register prefix for different register width is also adopted
> by quite a few other GNU assembler targets like AArch64, X86_64.

there is also Ed's assembler:
https://github.com/solarflarecom/ebpf_asm
It uses 2 ops style.
I think 3 ops style "add r0,r1,r2" is not a good fit for bpf isa.

I think we need to converge on one asm syntax for gas/bfd.
At this point we cannot change llvm's asm output,
so my preference would be to make gas accept it.
But I understand the implementation difficulties to fit it into bfd infra.
So I'm ok with more traditional asm the way Dave implemented it few
years back.
One asm syntax for gas and another asm syntax for clang is, imo, acceptable.

Jose, can you combine Dave's patches with yours?

I think Ed had an idea on how to specify BTF in asm syntax.
BTF has to be supported by the assembler as well
along with .btf.ext, lineinfo, etc
Currently llvm emits btf as '.byte 0x...', but that's far from ideal.

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

Hi Alexey.

    > > Despite using a different syntax for the assembler (the llvm assembler
    > > uses a C-ish expression-based syntax while the GNU assembler opts for
    > > a more classic assembly-language syntax) this implementation tries to
    > > provide inter-operability with clang/llvm generated objects.
    >
    > I also noticed your implementation doesn’t seem to use the same sub-register
    > syntax as what LLVM assembler is doing.
    >
    >   x register for 64-bit, and w register for 32-bit sub-register.
    >
    > So:
    >   add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
    >   add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X
    >
    > ASAICT, different register prefix for different register width is also adopted
    > by quite a few other GNU assembler targets like AArch64, X86_64.
    
    there is also Ed's assembler:
    https://github.com/solarflarecom/ebpf_asm

Oh interesting, didn't know about it.

    It uses 2 ops style.
    I think 3 ops style "add r0,r1,r2" is not a good fit for bpf isa.

I agree.  My port uses two-operand formats for most instructions,
including arithmetic instructions.  Examples:

sub %r2, 666
sub %r3, -666
sub %r4, 0x7eadbeef
sub %r5, %r6
    
    I think we need to converge on one asm syntax for gas/bfd.
    At this point we cannot change llvm's asm output,
    so my preference would be to make gas accept it.
    But I understand the implementation difficulties to fit it into bfd infra.
    So I'm ok with more traditional asm the way Dave implemented it few
    years back.
    One asm syntax for gas and another asm syntax for clang is, imo, acceptable.
    
    Jose, can you combine Dave's patches with yours?

As far as I can see my port uses basically the same syntax than Dave's.

The only significant difference is that mine uses a % prefix for
register names.  I prefer it this way in order to avoid collisions with
symbols.
    
    I think Ed had an idea on how to specify BTF in asm syntax.
    BTF has to be supported by the assembler as well
    along with .btf.ext, lineinfo, etc
    Currently llvm emits btf as '.byte 0x...', but that's far from ideal.

I haven't looked into BTF yet, but it would be certainly good, for
everyone to use the same directives to support it :)

Re: [PATCH 0/9] eBPF support for GNU binutils

[Edward Cree]

On 21/05/2019 19:18, Alexei Starovoitov wrote:
> I think Ed had an idea on how to specify BTF in asm syntax.
Specifically, see [1] for BTF as implemented in ebpf_asm, though note
 that it doesn't (yet) cover .btf.ext or lineinfo.

-Ed

[1]: https://github.com/solarflarecom/ebpf_asm/tree/btfdoc#type-definitions

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

    >> Despite using a different syntax for the assembler (the llvm assembler
    >> uses a C-ish expression-based syntax while the GNU assembler opts for
    >> a more classic assembly-language syntax) this implementation tries to
    >> provide inter-operability with clang/llvm generated objects.
    > 
    >    I also noticed your implementation doesn’t seem to use the same sub-register
    >    syntax as what LLVM assembler is doing.
    > 
    >      x register for 64-bit, and w register for 32-bit sub-register.
    > 
    >    So:
    >      add r0, r1, r2 means BPF_ALU64 | BPF_ADD | BFF_X
    >      add w0, w1, w1 means BPF_ALU | BPF_ADD | BPF_X
    > 
    >    ASAICT, different register prefix for different register width is also adopted
    >    by quite a few other GNU assembler targets like AArch64, X86_64.
    > 
    > Right.  I opted for using different mnemonics for alu and alu64
    > instructions, as it seemed to be simpler.
    > 
    > What was your rationale for using sub-register notation?  
    
    It is the same instruction operating on different register classes,
    sub-register is a new register class, so define separate notation
    for them. This also simplifies compiler back-end when generating
    sub-register instructions, at least for LLVM, and is likely for GCC
    as well.

    LLVM eBPF backend has full support for generating sub-register ISA,

Well, the way I read the spec, these look like different instructions
operating on the same registers, only with different semantics :)

But yeah, it is basically two different ways to look at the same thing,
at the ISA level.

Given that both llvm and ebpf_asm use some kind of sub-registers (using
different register names, or suffixes) I guess I could do the same... In
principle I don't have a strong preference, but I have to think about
it, and determine what would be the impact in my on-going GCC backend.

Thanks for the info.  Much appreciated.

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

    On 21/05/2019 19:18, Alexei Starovoitov wrote:
    > I think Ed had an idea on how to specify BTF in asm syntax.
    Specifically, see [1] for BTF as implemented in ebpf_asm, though note
    that it doesn't (yet) cover .btf.ext or lineinfo.
    
    [1]: https://github.com/solarflarecom/ebpf_asm/tree/btfdoc#type-definitions

Thanks for the reference.  I just checked out your `btfdoc' branch.  I
will take a look.

Where would you like to get feedback/suggestions/questions btw?
bpf@vger.kernel.org?

Re: [PATCH 0/9] eBPF support for GNU binutils

[Edward Cree]

On 21/05/2019 20:34, Jose E. Marchesi wrote:
>     On 21/05/2019 19:18, Alexei Starovoitov wrote:
>     > I think Ed had an idea on how to specify BTF in asm syntax.
>     Specifically, see [1] for BTF as implemented in ebpf_asm, though note
>     that it doesn't (yet) cover .btf.ext or lineinfo.
>     
>     [1]: https://github.com/solarflarecom/ebpf_asm/tree/btfdoc#type-definitions
>
> Thanks for the reference.  I just checked out your `btfdoc' branch.  I
> will take a look.
>
> Where would you like to get feedback/suggestions/questions btw?
For ebpf_asm?  Either through GitHub Issues, or this address (ecree@solarflare.com).

I'm not actually on bpf@vger (though maybe I ought to subscribe...)

BTW I'm really happy to see someone working on eBPF support in the GNU
 toolchain; being tied to an LLVM monoculture has been bad for eBPF imho.

-Ed

Re: [PATCH 0/9] eBPF support for GNU binutils

[Nick Clifton]

Hi Jose,

> The target is intended to be configured with target=bpf (or
> target=bpf-none-none) and in that case `is_elf_format' returns 0.
> 
> You must have configured with target=bpf-whatever-elf or
> bpf-whatever-linux or similar.  Hence all these tests get executed, and
> fail.

Right - that is exactly what happened.

> I could add checks to all the .exp files of the involved tests to skip
> if istarget bpf-*-*, if you think that is better.  But I think it is
> more reasonable to have is_elf_format return 0 for now, as other "funny"
> targets do.

I think either have is_elf_format return zero, or - better in my opinon -
just have config.bfd reject bpf-*-elf and bpf-*-linux-gnu with a "not yet
fully supported" message. 

Cheers
  Nick

Re: [PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

Hi Nick!
    
      Thank you very much for submitting such a nicely set up patch
      series.  I only have one problem with it: the new toolchain shows
      a lot of failures in the various testsuites:

I was gonna write "my testing results in no unexpected failures in any
of the testsuites" but then I realized what is going on.

The target is intended to be configured with target=bpf (or
target=bpf-none-none) and in that case `is_elf_format' returns 0.

You must have configured with target=bpf-whatever-elf or
bpf-whatever-linux or similar.  Hence all these tests get executed, and
fail.

That's the reason why I added bpf-*-* to the conditional in
testsuite/binutils-all/nm.exp, where you can find this (pre-existing)
comment:

# The target exceptions here are intended for targets that have ELF as
# an intermediate format or otherwise require the ELF-variant syntax
# for proper size annotation.  It would be nice if is_elf_format found
# all the ELF variants, but adding the patterns here to that proc then
# introduces a whole slew of new regressions in the GAS and LD testsuites.

It forgets to mention the BINUTILS testsuite :D

So yes, these tests were ought to be skipped, for the time being.

In time we will be promoting the port to a first-class ELF citizen, or
near enough: I definitely want DWARF support (we are working on a sim/
to be used with GDB) and other goodies.  Also, eBPF is evolving and the
kernel chaps are starting talking about dynamic linking and other crazy
ideas.  However many other ELF features will probably never make sense
for eBPF.

I could add checks to all the .exp files of the involved tests to skip
if istarget bpf-*-*, if you think that is better.  But I think it is
more reasonable to have is_elf_format return 0 for now, as other "funny"
targets do.

PS: in fact the same problem happens with other targets.  For example,
    if you configure with --target=mmix and with --target=mmix-none-elf,
    you will see that the second triggers unexpected failures in the
    testsuites.

Re: [PATCH 0/9] eBPF support for GNU binutils

[Nick Clifton]

Hi Jose,

  Thank you very much for submitting such a nicely set up patch
  series.  I only have one problem with it: the new toolchain shows
  a lot of failures in the various testsuites:

gas:
  FAIL: DWARF2 1
  FAIL: DWARF2 2
  FAIL: DWARF2 5
  FAIL: DWARF2 6
  FAIL: DWARF2 7
  FAIL: DWARF2 11
  FAIL: DWARF2 12
  FAIL: DWARF2 13
  FAIL: DWARF2 14
  FAIL: DWARF2 15
  FAIL: DWARF2 16
  FAIL: DWARF2 17
  FAIL: DWARF2 18
  FAIL: DWARF2 19
  FAIL: gas/elf/warn-2.s  (test for warnings, line )
  FAIL: gas/elf/warn-2.s (test for excess errors)
  FAIL: lns-duplicate
  FAIL: lns-common-1

binutils:
  FAIL: objdump compress debug sections
  FAIL: objdump compress debug sections 3
  FAIL: objcopy compress debug sections in archive with zlib-gabi
  FAIL: objdump compress debug sections 3 with zlib-gabi
  FAIL: v2 gnu build attribute notes (64-bit)
  FAIL: v3 gnu build attribute notes (64-bit)
  FAIL: assembler generated build notes
  FAIL: unordered .debug_info references to .debug_ranges
  FAIL: objdump decoded line
  FAIL: objdump -WL (reason: unexpected output)
  FAIL: objdump -WL
  FAIL: objdump -W for debug_ranges
  FAIL: readelf --debug-dump=loc pr18374
  FAIL: readelf --debug-dump=loc locview-1
  FAIL: readelf --debug-dump=loc locview-2
  FAIL: readelf -wiaoRlL dw5
  FAIL: readelf --debug-dump=links dwo

ld:
  FAIL: ld-discard/extern
  FAIL: ld-discard/start
  FAIL: ld-discard/static
  FAIL: PR ld/21703
  FAIL: PR ld/21703 -r
  FAIL: Symbol flags copy
  FAIL: ld-elf/group1
  FAIL: ld-elf/group10
  FAIL: ld-elf/group2
  FAIL: ld-elf/group3b
  FAIL: ld-elf/group4
  FAIL: ld-elf/group5
  FAIL: ld-elf/group6
  FAIL: ld-elf/group7
  FAIL: ld-elf/group8a
  FAIL: ld-elf/group8b
  FAIL: ld-elf/group9a
  FAIL: ld-elf/group9b
  FAIL: ld-elf/linkonce2
  FAIL: ld-elf/merge
  FAIL: ld-elf/merge2
  FAIL: ld-elf/merge3
  FAIL: ld-elf/pr12851
  FAIL: ld-elf/pr17550c
  FAIL: ld-elf/pr17550d
  FAIL: ld-elf/pr22677
  FAIL: ld-elf/pr22836-1a
  FAIL: ld-elf/pr22836-1b
  FAIL: ld-elf/pr22836-2
  FAIL: ld-elf/pr23658-1a
  FAIL: ld-elf/warn1
  FAIL: ld-elf/warn3
  FAIL: Generate empty import library
  FAIL: Generate import library
  FAIL: Linkonce sections with assembler generated notes
  FAIL: ld-elf/64ksec-r
  FAIL: ld-elf/64ksec
  FAIL: Linker setting GNU OSABI on STB_GNU_UNIQUE symbol (PR 10549)

Please could you take a look at these and determine if they represent
problems with the ebpf code, problems with the tests, or just something
that ought to be skipped.

Cheers
  Nick

[PATCH 0/9] eBPF support for GNU binutils

[Jose E. Marchesi]

Hi people!

This patch series introduces support for eBPF, which is a virtual
machine that resides in the Linux kernel.  Initially intended for
user-level packet capture and filtering, eBPF is nowadays generalized
to serve as a general-purpose infrastructure also for non-networking
purposes.

The first patch is preparatory, and adds support to config.guess to
recognize bpf-*-* triplets.  This will be submitted as a patch to the
`config' project as soon as this series gets upstreamed.
   
The second and third patches add support for an ELF64 based eBPF
target to BFD, in both little-endian and big-endian vectors.

The fourth patch adds a CGEN cpu description for eBPF, plus support
code.  This description covers the full eBPF ISA.  Due to the 64-bit
instruction fields used in some instructions, we needed to fix a
bug/limitation in CGEN impacting 32-bit hosts.  The fix is in a patch
submitted to CGEN last week, that is still waiting for review:
http://www.sourceware.org/ml/cgen/2019-q2/msg00008.html None of the
existing CGEN ports in binutils are impacted by that patch: the code
generated for these remains exactly the same.

The fifth patch adds opcodes and disassembler support for eBPF, based
on the CGEN description.

The sixth patch adds a GAS port, including a testsuite and manual
updates.  By default the assembler generates objects using the same
endianness than the host.  This can be overrided by the usual -EB and
-EB command-line options.

Support for linking eBPF ELF files with ld/bfd is provided in the
seventh patch.  A couple of simple tests are included.

The eighth patch adds support for eBPF to readelf, and makes a little
adjustment in the `nm' testsuite to not fail in bpf-*-* targets.

Finally, the last patch adds myself as the maintainer of the BPF
target.  We are committing to maintain this port.

Future work on the binutils port:
* Support for semantic actions in bpf.cpu, and support code for a
  simulator in sim/.
* Support for ld.gold.

Next stop is GCC.  An eBPF backend is on the works.  We plan to
upstream it before September.

Regressions tested in all targets.
Regressions tested with --enable-targets=all
Tested in 64-bit x86_64 host.
Tested in 32-bit x86 host.

Oh, a little note regarding interoperability:

There is a clang/llvm based toolchain for eBPF.  However, at this
moment compiled eBPF doesn't have established conventions.  The
details on what is expected to be in an ELF file containing eBPF is
determined, in practice, by what the llvm BPF backend supports and
what the sample bpf_load.c in the Linux kernel source tree expects
[1].

Despite using a different syntax for the assembler (the llvm assembler
uses a C-ish expression-based syntax while the GNU assembler opts for
a more classic assembly-language syntax) this implementation tries to
provide inter-operability with clang/llvm generated objects.

In particular, the numbers of the relocations used for instruction
fields are the same.  These are R_BPF_INSN_64 and R_BPF_INSN_DISP32.
The later is resolved at load-time by bpf_load.c.

[1] We expect/hope that the addition of eBPF support to the GNU
    toolchain will help to mature the domain of compiled eBPF.  We
    will certainly be working with the kernel people to that effect.

Salud!

Jose E. Marchesi (9):
  config: recognize eBPF triplets
  include: add elf/bpf.h
  bfd: add support for eBPF
  cpu: add eBPF cpu description
  opcodes: add support for eBPF
  gas: add support for eBPF
  ld: add support for eBPF
  binutils: add support for eBPF
  binutils: add myself as the maintainer for BPF

 ChangeLog                              |    4 +
 bfd/ChangeLog                          |   20 +
 bfd/Makefile.am                        |    4 +
 bfd/Makefile.in                        |    7 +
 bfd/archures.c                         |    4 +
 bfd/bfd-in2.h                          |    9 +
 bfd/config.bfd                         |    6 +
 bfd/configure                          |   30 +-
 bfd/configure.ac                       |    2 +
 bfd/cpu-bpf.c                          |   41 +
 bfd/elf64-bpf.c                        |  463 +++++++++
 bfd/libbfd.h                           |    5 +
 bfd/reloc.c                            |   13 +
 bfd/targets.c                          |    7 +
 binutils/ChangeLog                     |   13 +
 binutils/MAINTAINERS                   |    1 +
 binutils/readelf.c                     |    8 +
 binutils/testsuite/binutils-all/nm.exp |    3 +-
 config.sub                             |    4 +-
 cpu/ChangeLog                          |    5 +
 cpu/bpf.cpu                            |  647 +++++++++++++
 cpu/bpf.opc                            |  191 ++++
 gas/ChangeLog                          |   45 +
 gas/Makefile.am                        |    2 +
 gas/Makefile.in                        |    6 +
 gas/config/tc-bpf.c                    |  357 +++++++
 gas/config/tc-bpf.h                    |   51 +
 gas/configure                          |   38 +-
 gas/configure.ac                       |    6 +
 gas/configure.tgt                      |    1 +
 gas/doc/Makefile.am                    |    1 +
 gas/doc/Makefile.in                    |    6 +-
 gas/doc/all.texi                       |    1 +
 gas/doc/as.texi                        |   34 +
 gas/doc/c-bpf.texi                     |  364 +++++++
 gas/testsuite/gas/all/gas.exp          |    3 +
 gas/testsuite/gas/all/org-1.l          |    2 +-
 gas/testsuite/gas/all/org-1.s          |    2 +
 gas/testsuite/gas/bpf/alu-be.d         |   59 ++
 gas/testsuite/gas/bpf/alu.d            |   58 ++
 gas/testsuite/gas/bpf/alu.s            |   51 +
 gas/testsuite/gas/bpf/alu32-be.d       |   65 ++
 gas/testsuite/gas/bpf/alu32.d          |   64 ++
 gas/testsuite/gas/bpf/alu32.s          |   57 ++
 gas/testsuite/gas/bpf/atomic-be.d      |   12 +
 gas/testsuite/gas/bpf/atomic.d         |   11 +
 gas/testsuite/gas/bpf/atomic.s         |    5 +
 gas/testsuite/gas/bpf/bpf.exp          |   38 +
 gas/testsuite/gas/bpf/call-be.d        |   19 +
 gas/testsuite/gas/bpf/call.d           |   18 +
 gas/testsuite/gas/bpf/call.s           |   11 +
 gas/testsuite/gas/bpf/exit-be.d        |   11 +
 gas/testsuite/gas/bpf/exit.d           |   10 +
 gas/testsuite/gas/bpf/exit.s           |    2 +
 gas/testsuite/gas/bpf/jump-be.d        |   32 +
 gas/testsuite/gas/bpf/jump.d           |   31 +
 gas/testsuite/gas/bpf/jump.s           |   25 +
 gas/testsuite/gas/bpf/lddw-be.d        |   18 +
 gas/testsuite/gas/bpf/lddw.d           |   17 +
 gas/testsuite/gas/bpf/lddw.s           |    6 +
 gas/testsuite/gas/bpf/mem-be.d         |   30 +
 gas/testsuite/gas/bpf/mem.d            |   29 +
 gas/testsuite/gas/bpf/mem.s            |   24 +
 include/ChangeLog                      |    4 +
 include/elf/bpf.h                      |   45 +
 ld/ChangeLog                           |   15 +
 ld/Makefile.am                         |    2 +
 ld/Makefile.in                         |    4 +
 ld/configure                           |   28 +-
 ld/configure.tgt                       |    1 +
 ld/emulparams/elf64bpf.sh              |   10 +
 ld/testsuite/ld-bpf/bar.s              |    5 +
 ld/testsuite/ld-bpf/baz.s              |    5 +
 ld/testsuite/ld-bpf/bpf.exp            |   29 +
 ld/testsuite/ld-bpf/call-1.d           |   23 +
 ld/testsuite/ld-bpf/foo.s              |    5 +
 ld/testsuite/ld-bpf/jump-1.d           |   23 +
 ld/testsuite/lib/ld-lib.exp            |    1 +
 opcodes/ChangeLog                      |   24 +
 opcodes/Makefile.am                    |   17 +
 opcodes/Makefile.in                    |   23 +
 opcodes/bpf-asm.c                      |  590 ++++++++++++
 opcodes/bpf-desc.c                     | 1638 ++++++++++++++++++++++++++++++++
 opcodes/bpf-desc.h                     |  266 ++++++
 opcodes/bpf-dis.c                      |  624 ++++++++++++
 opcodes/bpf-ibld.c                     |  956 +++++++++++++++++++
 opcodes/bpf-opc.c                      | 1495 +++++++++++++++++++++++++++++
 opcodes/bpf-opc.h                      |  151 +++
 opcodes/configure                      |   19 +-
 opcodes/configure.ac                   |    1 +
 opcodes/disassemble.c                  |   35 +
 opcodes/disassemble.h                  |    1 +
 92 files changed, 9116 insertions(+), 33 deletions(-)
 create mode 100644 bfd/cpu-bpf.c
 create mode 100644 bfd/elf64-bpf.c
 create mode 100644 cpu/bpf.cpu
 create mode 100644 cpu/bpf.opc
 create mode 100644 gas/config/tc-bpf.c
 create mode 100644 gas/config/tc-bpf.h
 create mode 100644 gas/doc/c-bpf.texi
 create mode 100644 gas/testsuite/gas/bpf/alu-be.d
 create mode 100644 gas/testsuite/gas/bpf/alu.d
 create mode 100644 gas/testsuite/gas/bpf/alu.s
 create mode 100644 gas/testsuite/gas/bpf/alu32-be.d
 create mode 100644 gas/testsuite/gas/bpf/alu32.d
 create mode 100644 gas/testsuite/gas/bpf/alu32.s
 create mode 100644 gas/testsuite/gas/bpf/atomic-be.d
 create mode 100644 gas/testsuite/gas/bpf/atomic.d
 create mode 100644 gas/testsuite/gas/bpf/atomic.s
 create mode 100644 gas/testsuite/gas/bpf/bpf.exp
 create mode 100644 gas/testsuite/gas/bpf/call-be.d
 create mode 100644 gas/testsuite/gas/bpf/call.d
 create mode 100644 gas/testsuite/gas/bpf/call.s
 create mode 100644 gas/testsuite/gas/bpf/exit-be.d
 create mode 100644 gas/testsuite/gas/bpf/exit.d
 create mode 100644 gas/testsuite/gas/bpf/exit.s
 create mode 100644 gas/testsuite/gas/bpf/jump-be.d
 create mode 100644 gas/testsuite/gas/bpf/jump.d
 create mode 100644 gas/testsuite/gas/bpf/jump.s
 create mode 100644 gas/testsuite/gas/bpf/lddw-be.d
 create mode 100644 gas/testsuite/gas/bpf/lddw.d
 create mode 100644 gas/testsuite/gas/bpf/lddw.s
 create mode 100644 gas/testsuite/gas/bpf/mem-be.d
 create mode 100644 gas/testsuite/gas/bpf/mem.d
 create mode 100644 gas/testsuite/gas/bpf/mem.s
 create mode 100644 include/elf/bpf.h
 create mode 100644 ld/emulparams/elf64bpf.sh
 create mode 100644 ld/testsuite/ld-bpf/bar.s
 create mode 100644 ld/testsuite/ld-bpf/baz.s
 create mode 100644 ld/testsuite/ld-bpf/bpf.exp
 create mode 100644 ld/testsuite/ld-bpf/call-1.d
 create mode 100644 ld/testsuite/ld-bpf/foo.s
 create mode 100644 ld/testsuite/ld-bpf/jump-1.d
 create mode 100644 opcodes/bpf-asm.c
 create mode 100644 opcodes/bpf-desc.c
 create mode 100644 opcodes/bpf-desc.h
 create mode 100644 opcodes/bpf-dis.c
 create mode 100644 opcodes/bpf-ibld.c
 create mode 100644 opcodes/bpf-opc.c
 create mode 100644 opcodes/bpf-opc.h

-- 
2.11.0