[Bug default/22971] RFE: Support <xi:include> for managing abixml documents.

[Bug default/22971] RFE: Support <xi:include> for managing abixml documents.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

--- Comment #2 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to dodji from comment #1)
> "carlos at redhat dot com" <sourceware-bugzilla@sourceware.org> writes:
> 
> > Can we get this feature into libabigail?
> 
> Sure.  And very interesting feature.
> 
> Though, please keep in mind that initially, the abixml file was not
> meant to be edited (or consumed) by humans.  Its format is meant as a
> private implementation detail of Libabigail.  Things that interact with
> the abixml file are tools.
> 
> But then things change, don't they :-)
> 
> With that said, I would like to learn more about how you see the whole
> process, i.e, how you'd like to split the abixml file to begin with.
> 
> I think doing it by hand can be a nightmare as each abixml file has a
> number of invariants to comply with.
> 
> The way I see things is that we'd need to be able designate (in a high
> level-ish manner) which ABI artifact 'A' -- be it a type, function, or
> elf symbol -- should be put inside an ancillary abixml file 'AF'.  The
> tool would then make sure that the artifacts that 'A' depends on will
> also be moved to that 'AF' abixml file.  Things like that.

I disagree. You need not worry about any of this. You process <xi:include> at
the top-level and pretend to have a complete document that meets all of your
invariants.

> The main abixml file 'MF' would then include 'AF'.  We could even go
> further and saying that using xi:include for that or not is, again, an
> implementation detail, even if I think that xi:include seems to be the
> way to go :-)

It is up to you if you want to go this way, and I explain my use case to you
and you can design around this.

> > For example in the relationship between glibc and Fedora glibc, one being the
> > downstream of the other, in Fedora I would like move any ABI changes into a
> > distinct file and <xi:include> it into the baseline glibc abixml file.
> 
> I am not sure I understand the use case correctly.
> 
> I would thought that it's the Fedora ABI would have Fedora-specific ABI
> sub-parts.  And so the main Fedora ABI description file would include
> the "generic" upstream glibc common ABI and define its specific ABI
> sub-parts on top of that.
> 
> I guess I am understanding something backward.  What is it?

We are thinking of the same thing, but our process is slightly different.

Consider the following:

* Upstream master glibc has an abi/ directory, into which are SONAME named
files each being an abixml document that describes the full ABI of that SONAME.

* Downstream fedora is based on upstream glibc master, and would branch from
that and layer on top all of the fedora changes.

* During the rebase process which moves glibc master changes into the fedora
rawhide git branch (not dist-git) we have to reconcile the upstream abi/
changes with our downstream abi/ files. The reconciliation process is easier if
we have a way to split, on a file basis, the ABI differences.

Example:

Fedora Rawhide has 2 new GLIBC_PRIVATE symbols in libc.so.6 which are used by
ld.so, and 1 completely new data structure shared between them also
GLIBC_PRIVATE.

Merging from upstream glibc master to fedora rawhide would cause conflicts in
the libc.so.6 abixml file and ld.so abixml file.

It would be nice if we could <xi:include> a new set of functions and data
symbols and the definitions. Either by <xi:include>'ing some new ancilliary
format file, or by an <xi:include> of each sub section (would require you
support more than one of each element type e.g. <elf-function-symbols> because
the <xi:include>'d file must be valid XML with 1 root element).

Your suggestion appears to be better, in which you suggest:

* abi/ remains untouched.

* final-abi/ is created, and has libc.so.6 abixml which includes abi/libc.so.6
abixml baseline and modifies it.

What is hard here is that final-abi/ may have more than just additions, and may
have modifications to internal ABIs which we also want to track as invariants
(so we don't break libc.so.6 <-> ld.so private interfaces by accident).

This is the reason I was going to go with modifying abi/ in such a way as to
construct the final-abi, but doing so using <xi:include> to minimize merge
conflict errors.

Lastly, if a tool could produce a "abi xml delta" file which applies on top of
an existing baseline and modifies it accordingly (same named types override,
and new types are added, and removed types are marked removed [something not
specified in the xml currently]) then that would be what we could use too.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/22971] Support <xi:include> for managing abixml documents.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/22971] RFE: Support <xi:include> for managing abixml documents.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Support <xi:include> for    |RFE: Support <xi:include>
                   |managing abixml documents.  |for managing abixml
                   |                            |documents.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/22971] RFE: Support <xi:include> for managing abixml documents.

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

--- Comment #1 from dodji at redhat dot com ---
"carlos at redhat dot com" <sourceware-bugzilla@sourceware.org> writes:

> Can we get this feature into libabigail?

Sure.  And very interesting feature.


Though, please keep in mind that initially, the abixml file was not
meant to be edited (or consumed) by humans.  Its format is meant as a
private implementation detail of Libabigail.  Things that interact with
the abixml file are tools.

But then things change, don't they :-)

With that said, I would like to learn more about how you see the whole
process, i.e, how you'd like to split the abixml file to begin with.

I think doing it by hand can be a nightmare as each abixml file has a
number of invariants to comply with.

The way I see things is that we'd need to be able designate (in a high
level-ish manner) which ABI artifact 'A' -- be it a type, function, or
elf symbol -- should be put inside an ancillary abixml file 'AF'.  The
tool would then make sure that the artifacts that 'A' depends on will
also be moved to that 'AF' abixml file.  Things like that.

The main abixml file 'MF' would then include 'AF'.  We could even go
further and saying that using xi:include for that or not is, again, an
implementation detail, even if I think that xi:include seems to be the
way to go :-)


> For example in the relationship between glibc and Fedora glibc, one being the
> downstream of the other, in Fedora I would like move any ABI changes into a
> distinct file and <xi:include> it into the baseline glibc abixml file.

I am not sure I understand the use case correctly.

I would thought that it's the Fedora ABI would have Fedora-specific ABI
sub-parts.  And so the main Fedora ABI description file would include
the "generic" upstream glibc common ABI and define its specific ABI
sub-parts on top of that.

I guess I am understanding something backward.  What is it?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/22971] New: Support <xi:include> for managing abixml documents.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

            Bug ID: 22971
           Summary: Support <xi:include> for managing abixml documents.
           Product: libabigail
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: default
          Assignee: dodji at redhat dot com
          Reporter: carlos at redhat dot com
                CC: libabigail at sourceware dot org
  Target Milestone: ---

I would like to split up the abixml files into logical parts that can be
updated independently, but in order to do that I need support for traditional
XML include directives like <xi:include>.

Today libabigail uses libxml2, which supports <xi:include>, but I expect the
parser needs to be updated to trigger the inclusions instead of skipping over
nodes the parser doesn't understand.

I have verified that the parser skips <xi:include> and does not parse it.

Can we get this feature into libabigail?

For example in the relationship between glibc and Fedora glibc, one being the
downstream of the other, in Fedora I would like move any ABI changes into a
distinct file and <xi:include> it into the baseline glibc abixml file. This way
when you merge the ABI descriptions from glibc to Fedora you can do the merge a
little easier.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/22971] RFE: Support <xi:include> for managing abixml documents.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22971

--- Comment #3 from Carlos O'Donell <carlos at redhat dot com> ---
Example of what I'm thinking about:

abi/x86_64/libc.so.6.abi.xml

<abi-corpus path='./lib64/libc-2.27.9000.so' architecture='elf-amd-x86_64'
soname='libc.so.6'>
  <elf-needed>
    <dependency name='ld-linux-x86-64.so.2'/>
  </elf-needed>
  <elf-function-symbols>
    ...
  </elf-function-symbols>
  <!-- Include new elf function symbols in downstream. -->
  <xi:include href="../final-abi/x86_64/libc.so.6.efs.abi.xml"/>
  ...
</abi-corpus>

final-abi/x86_64/libc.so.6.efs.abi.xml

<elf-function-symbols>
  ... List of all new elf function symbols
</elf-functions-symbols>

Delta file suggestion:

abi/x86_64/libc.so.6.abi.xml

<abi-corpus path='./lib64/libc-2.27.9000.so' architecture='elf-amd-x86_64'
soname='libc.so.6'>
  <elf-needed>
    <dependency name='ld-linux-x86-64.so.2'/>
  </elf-needed>
  <elf-function-symbols>
    ...
  </elf-function-symbols>
  ...
</abi-corpus>

final-abi/x86_64/libc.so.6.abi.xml

<abi-corpus path='./lib64/libc-2.28.so' architecture='elf-amd-x86_64'
soname='libc.so.6'>
  <!-- Include baseline upstream corpus. -->
  <xi:include href="../abi/x86_64/libc.so.6.abi.xml"/>
  <elf-needed>
    <dependency name='ld-linux-x86-64.so.2'/>
  </elf-needed>
  <elf-function-symbols>
    ...
    <!-- The following symbols are removed from the baseline.  -->
    <elf-symbol-removed name='_IO_2_1_stderr_' size='224' version='GLIBC_2.2.5'
is-default-version='yes' type='object-type' binding='global-binding'
visibility='default-visibility' is-defined='yes'/>
  </elf-function-symbols>
  ...
</abi-corpus>

* Requires support for nested <abi-corpus>
* Requires support for override/deltion of ABI elements.

-- 
You are receiving this mail because:
You are on the CC list for the bug.