From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 06226385801B for ; Fri, 22 Apr 2022 16:17:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 06226385801B Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-280-f3HUH5U4MlGz3VlLUTdCLQ-1; Fri, 22 Apr 2022 12:17:36 -0400 X-MC-Unique: f3HUH5U4MlGz3VlLUTdCLQ-1 Received: by mail-pf1-f199.google.com with SMTP id i196-20020a6287cd000000b0050ab83e42c0so5685852pfe.7 for ; Fri, 22 Apr 2022 09:17:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=3/PuYsXWj2GMx4vthS2hJJuB4Y4DGomrMczHYV9mqCw=; b=EFI3UWJJBduXmhILgmuyETVUF5vS0wH6LpLovyhoAqHVBnVAMPV6iQFbW9jRLSX//o t5Plu0YEKV0rRYyDm6PHnLEat9gq6oPytX72+dRlbws9Ki0Tq4VfnZLVlhUkGWdND7p/ KKZCwpT9nziLmdYQVwd74Uq1YpFQOQtpsIpj5zM07dad1jjaKQjvZLAUG3y5v9rYcNGb q7O7+NGnOEENXAvHE2NXWAbo5qogmtVMlD6AMtj/7xBPwT8B+bNM73PYTAijKkDWoHTN 6TwAkuVAarWMkSuAbMomfmMnw5ZlNK/GBn+FxjElXciyNXcPb6SQq18f5hnd2JDpTtLQ H1vw== X-Gm-Message-State: AOAM530Ya4JoKAvY3IKZItHMRGy7bL9y0TCtI8i+bFULI4bBh1LA+MAL Jr/kbbvygphn2XqGxqFse0j1ao3SQjxcqFiO/7mvfOzLMZbubgg2ha446V2Kltp4MLIE/y+ZL8e GkOTmJjza7mgHCQtmqeK/crv5cfIOOnmcS3cPFLKGi+tUD4zPYamx0HpGM1Ep7lz8Xum6xU0= X-Received: by 2002:a17:90b:1e08:b0:1d2:8bb4:f565 with SMTP id pg8-20020a17090b1e0800b001d28bb4f565mr16933240pjb.4.1650644253834; Fri, 22 Apr 2022 09:17:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxCIwhdJZkN+vgiZc2Angsl8Kdc+Pl7PcAe8bceHTZW+U88u+2CsCdiwXPhzPTRqKuoNhYPLg== X-Received: by 2002:a17:90b:1e08:b0:1d2:8bb4:f565 with SMTP id pg8-20020a17090b1e0800b001d28bb4f565mr16933172pjb.4.1650644253058; Fri, 22 Apr 2022 09:17:33 -0700 (PDT) Received: from alien.localdomain ([47.208.199.57]) by smtp.gmail.com with ESMTPSA id q7-20020a17090a2e0700b001cda0b69a30sm6380696pjd.52.2022.04.22.09.17.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Apr 2022 09:17:32 -0700 (PDT) From: Ben Woodard To: libabigail@sourceware.org Cc: vsoch , Ben Woodard Subject: [PATCH] Add github actions to support workflows Date: Fri, 22 Apr 2022 09:17:16 -0700 Message-Id: <20220422161716.710496-1-woodard@redhat.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: application/octet-stream; x-default=true X-Spam-Status: No, score=-5.8 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Apr 2022 16:17:39 -0000 From: vsoch To facilitate collaboration with developers working on the BUILD-SI project who are using github, I have been maintaining a clone of the libabigail repo there. Having this repo also allows us to leverage some of the tooling that github provides, in particular the ability to have github actions test patches to verify that the builds succeed, and all the regression tests run successfully. This will allow it to better integrate with the normal agile workflow used by at least this community of developers. While containerizing the workflow we found that fedabipkgdiff had a subtle implicit dependency on system provided mimetypes. We found it expedient to add logic that eliminated that dependency. * add .github directory * add .github/workflows directory * add .github/workflows/build-container.yaml * add .github/workflows/libabigail.yaml * add .github/workflows/test.yaml * add docker directory * add docker/Dockerfile.fedora * add docker/Dockerfile.ubuntu * add docker/Dockerfile.fedora-base * add docker/Dockerfile.test * add README-DOCKER.md * tools/fedabipkgdiff Signed-off-by: vsoch Reviewed-by: Ben Woodard --- .github/workflows/build-container.yaml | 99 +++++++++++++++++++ .github/workflows/libabigail.yaml | 126 +++++++++++++++++++++++++ .github/workflows/test.yaml | 34 +++++++ README-DOCKER.md | 66 +++++++++++++ docker/Dockerfile.fedora | 38 ++++++++ docker/Dockerfile.fedora-base | 21 +++++ docker/Dockerfile.test | 28 ++++++ docker/Dockerfile.ubuntu | 30 ++++++ tools/fedabipkgdiff | 4 + 9 files changed, 446 insertions(+) create mode 100644 .github/workflows/build-container.yaml create mode 100644 .github/workflows/libabigail.yaml create mode 100644 .github/workflows/test.yaml create mode 100644 README-DOCKER.md create mode 100644 docker/Dockerfile.fedora create mode 100644 docker/Dockerfile.fedora-base create mode 100644 docker/Dockerfile.test create mode 100644 docker/Dockerfile.ubuntu diff --git a/.github/workflows/build-container.yaml b/.github/workflows/build-container.yaml new file mode 100644 index 00000000..b2b50e27 --- /dev/null +++ b/.github/workflows/build-container.yaml @@ -0,0 +1,99 @@ +name: Build Deploy Container + +on: + + # Always have a base image ready to go - this is a nightly build + schedule: + - cron: 0 3 * * * + + # Allow manual trigger of a build + workflow_dispatch: + inputs: + whatever: + description: 'This variable does not matter, its a GHA bug.' + + # On push to main we build and deploy images + push: + branches: + - master + + # Do build to test works on PR + pull_request: [] + + # Publish packages on release + release: + types: [published] + +jobs: + build: + permissions: + packages: write + strategy: + fail-fast: false + matrix: + + # Dockerfiles to build, container names to use, and to tag as libabigail:latest? + container: [["Dockerfile.fedora", "ghcr.io/woodard/libabigail-fedora", true], + ["Dockerfile.ubuntu", "ghcr.io/woodard/libabigail-ubuntu-22.04", false], + ["Dockerfile.fedora-base", "ghcr.io/woodard/libabigail-fedora-base", false]] + + runs-on: ubuntu-latest + name: Build + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Make Space For Build + run: | + sudo rm -rf /usr/share/dotnet + sudo rm -rf /opt/ghc + + # It's easier to reference named variables than indexes of the matrix + - name: Set Environment + env: + dockerfile: ${{ matrix.container[0] }} + uri: ${{ matrix.container[1] }} + isLatest: ${{ matrix.container[2] }} + run: | + echo "dockerfile=$dockerfile" >> $GITHUB_ENV + echo "uri=$uri" >> $GITHUB_ENV + echo "isLatest=$isLatest" >> $GITHUB_ENV + + - name: Pull previous layers for cache + run: docker pull ${uri}:latest || echo "No container to pull" + + - name: Build Container + run: | + container=$uri:latest + docker build -f docker/${dockerfile} -t ${container} . + if [[ "${isLatest}" == "true" ]]; then docker tag ${container} ghcr.io/woodard/libabigail:latest + fi + echo "container=$container" >> $GITHUB_ENV + + - name: GHCR Login + if: (github.event_name != 'pull_request') + uses: docker/login-action@v1 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Deploy + if: (github.event_name != 'pull_request') + run: | + docker push ${container} + if [[ "${isLatest}" == "true" ]]; then + docker push ghcr.io/woodard/libabigail:latest + fi + + - name: Tag and Push Release + if: (github.event_name == 'release') + run: | + tag=${GITHUB_REF#refs/tags/} + echo "Tagging and releasing ${uri}:${tag}" + docker tag ${uri}:latest ${uri}:${tag} + docker push ${uri}:${tag} + if [[ "${isLatest}" == "true" ]]; then + docker tag ${uri}:latest ghcr.io/woodard/libabigail:${tag} + docker push ghcr.io/woodard/libabigail:${tag} + fi diff --git a/.github/workflows/libabigail.yaml b/.github/workflows/libabigail.yaml new file mode 100644 index 00000000..a7c40d27 --- /dev/null +++ b/.github/workflows/libabigail.yaml @@ -0,0 +1,126 @@ +name: Libabigail ABI Checks +on: + pull_request: [] + +jobs: + +# get-release: +# container: ghcr.io/woodard/libabigail-ubuntu-22.04:latest +# runs-on: ubuntu-latest +# steps: +# - name: Organize Files +# run: | +# mkdir -p /abi +# cp /opt/abigail-env/.spack-env/view/lib/libabigail.so /abi/ +# cp /lib/x86_64-linux-gnu/libc.so.6 /abi/ +# cp /lib/x86_64-linux-gnu/libstdc++.so.6 /abi/ +# cp /lib/x86_64-linux-gnu/libsystemd.so.0 /abi/ + +# - name: Upload Libs +# uses: actions/upload-artifact@v3 +# with: +# name: release-libs +# path: | +# /abi/libabigail.so +# /abi/libc.so.6 +# /abi/libstdc++.so.6 +# /abi/libsystemd.so.0 + + + get-latest: + container: ghcr.io/woodard/libabigail-ubuntu-22.04:latest + runs-on: ubuntu-latest + steps: + - name: Organize Files + run: | + mkdir -p /abi + cp /opt/abigail-env/.spack-env/view/lib/libabigail.so /abi/ + cp /lib/x86_64-linux-gnu/libc.so.6 /abi/ + cp /lib/x86_64-linux-gnu/libstdc++.so.6 /abi/ + cp /lib/x86_64-linux-gnu/libsystemd.so.0 /abi/ + + - name: Upload Libs + uses: actions/upload-artifact@v3 + with: + name: latest-libs + path: | + /abi/libabigail.so + /abi/libc.so.6 + /abi/libstdc++.so.6 + /abi/libsystemd.so.0 + + get-pr: + container: ghcr.io/woodard/libabigail-ubuntu-22.04:latest + runs-on: ubuntu-latest + steps: + - name: Build Pull Request + uses: actions/checkout@v3 + - name: Build + run: | + rm -rf /src + cp -R $PWD /src + ls /src + . /opt/spack/share/spack/setup-env.sh + cd /opt/abigail-env + spack install + + - name: Organize Files + run: | + mkdir -p /abi + cp /opt/abigail-env/.spack-env/view/lib/libabigail.so /abi/ + cp /lib/x86_64-linux-gnu/libc.so.6 /abi/ + cp /lib/x86_64-linux-gnu/libstdc++.so.6 /abi/ + cp /lib/x86_64-linux-gnu/libsystemd.so.0 /abi/ + + - name: Upload Libs + uses: actions/upload-artifact@v3 + with: + name: pr-libs + path: | + /abi/libabigail.so + /abi/libc.so.6 + /abi/libstdc++.so.6 + /abi/libsystemd.so.0 + + abi: + runs-on: ubuntu-latest + needs: [get-latest, get-pr] # get-release + strategy: + fail-fast: false + matrix: + + # Testing every paired library for release vs pr and main vs. pr + libs: ["libabigail.so", + "libc.so.6", + "libstdc++.so.6", + "libsystemd.so.0"] + + # Artifact pairs (named) for comparison) + artifacts: [["pr-libs", "latest-libs"]] + #["pr-libs", "release-libs"]] + + steps: + - name: Download Previous Version + uses: actions/download-artifact@v2 + with: + name: ${{ matrix.artifacts[1] }} + path: previous/ + + - name: Download Pull Request Version + uses: actions/download-artifact@v2 + with: + name: ${{ matrix.artifacts[0] }} + path: current/ + + - name: Show Files + run: | + ls current/ + ls previous/ + + - name: Run Libabigail + uses: buildsi/libabigail-action@0.0.11 + env: + lib: ${{ matrix.libs }} + with: + abidiff: previous/${{ env.lib }} current/${{ env.lib }} + abidw: "--abidiff current/${{ env.lib }}" diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml new file mode 100644 index 00000000..f26f08cc --- /dev/null +++ b/.github/workflows/test.yaml @@ -0,0 +1,34 @@ +name: Test Libabigail + +on: + pull_request: [] + push: + branches: + - master + +jobs: + build: + permissions: + packages: read + strategy: + fail-fast: false + matrix: + + # Testing containers (build and run make check) + dockerfile: ["Dockerfile.test"] + + runs-on: ubuntu-latest + name: Build + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Make Space For Build + run: | + sudo rm -rf /usr/share/dotnet + sudo rm -rf /opt/ghc + + - name: Build Test Container + env: + dockerfile: ${{ matrix.dockerfile }} + run: docker build -f docker/${dockerfile} -t libabigail-test . diff --git a/README-DOCKER.md b/README-DOCKER.md new file mode 100644 index 00000000..ad2dd8e1 --- /dev/null +++ b/README-DOCKER.md @@ -0,0 +1,66 @@ +# Libabigail Docker + +Libabigail comes with two Dockerfile in [docker](docker) to build each of: + + - a Fedora base image (recommended) + - an Ubuntu base image. + +These containers are built and deployed on merges to the main branch and releases. + +### Usage + +Here is how to build the containers. Note that we build so it belongs to the same +namespace as the repository here. "ghcr.io" means "GitHub Container Registry" and +is the [GitHub packages](https://github.com/features/packages) registry that supports + Docker images and other OCI artifacts. + +```bash +$ docker build -f docker/Dockerfile.fedora -t ghcr.io/woodard/libabigail-fedora . +``` +```bash +$ docker build -f docker/Dockerfile.ubuntu -t ghcr.io/woodard/libabigail-ubuntu-22.04 . +``` + +Note that currently the fedora image is deployed to `ghcr.io/woodard/libabigail:latest`. + +### Shell + +To shell into a container (here is an example with ubuntu): + +```bash +$ docker run -it ghcr.io/woodard/libabigail-ubuntu-22.04 bash +``` + +Off the bat, you can find the abi executables: + +```bash +# which abidiff +/opt/abigail-env/.spack-env/view/bin/abidiff +``` + +Since the ubuntu base uses spack, you can interact with spack. +You can go to the environment in `/opt/abigail-env` and (given you +have the source code bound to `/src`) build and test again. + +```bash +$ spack install +``` + +And that's it! This workflow should make it easy to install development versions of libabigail with spack. +We will also use the "production" containers to grab libraries in: + +``` +$ ls /opt/abigail-env/.spack-env/view/ +bin include lib share +``` + +Note that the fedora container does not come with spack. + +### Testing + +We provide a testing container, which will use a fedora base and add new code to +compile, and then run `make check`. You can do this as follows on your local machine: + +```bash +$ docker build -f docker/Dockerfile.test -t test . +``` diff --git a/docker/Dockerfile.fedora b/docker/Dockerfile.fedora new file mode 100644 index 00000000..0257e99e --- /dev/null +++ b/docker/Dockerfile.fedora @@ -0,0 +1,38 @@ +FROM ghcr.io/woodard/libabigail-fedora-base as builder + +# docker build -f docker/Dockerfile.fedora -t ghcr.io/woodard/libabigail-fedora . + +WORKDIR /code +COPY . /code + +RUN mkdir -p build && \ + autoreconf -i && \ + cd build && \ + CXXFLAGS="-g3 -fvar-tracking-assignments \ + -gstatement-frontiers -gvariable-location-views -grecord-gcc-switches -pipe -Wall \ + -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -Wp,-D_GLIBCXX_ASSERTIONS \ + -fstack-protector-strong -fstack-clash-protection -fcf-protection \ + -fasynchronous-unwind-tables -O2" CFLAGS="-g3 -fvar-tracking-assignments \ + -gstatement-frontiers -gvariable-location-views -grecord-gcc-switches -pipe -Wall \ + -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -Wp,-D_GLIBCXX_ASSERTIONS \ + -fstack-protector-strong -fstack-clash-protection -fcf-protection \ + -fasynchronous-unwind-tables -O2" LDFLAGS="-Wl,--no-undefined" \ + ../configure --prefix=/opt/libabigail --enable-rpm=yes && \ + make -j4 && \ + make install + +FROM fedora:latest +COPY --from=builder /opt/libabigail /opt/libabigail +ENV PATH=/opt/libabigail/bin:$PATH + +# Runtime dependencies +RUN dnf install -y \ + python3-koji \ + python3-mock \ + python3-pyxdg \ + elfutils-libelf \ + elfutils-libs \ + libstdc++ \ + lbzip2 \ + shared-mime-info \ + six diff --git a/docker/Dockerfile.fedora-base b/docker/Dockerfile.fedora-base new file mode 100644 index 00000000..f803dd7f --- /dev/null +++ b/docker/Dockerfile.fedora-base @@ -0,0 +1,21 @@ +ARG fedora_version=latest +# ARG fedora_version=35 +FROM fedora:${fedora_version} as builder + +# docker build -f docker/Dockerfile.fedora-base -t ghcr.io/woodard/libabigail-fedora-base . + +RUN dnf install -y \ + autoconf \ + automake \ + cpio \ + elfutils-devel \ + gcc-c++ \ + lbzip2 \ + libtool \ + libxml2-devel \ + python3-koji \ + python3-mock \ + python3-pyxdg \ + shared-mime-info \ + six \ + wget diff --git a/docker/Dockerfile.test b/docker/Dockerfile.test new file mode 100644 index 00000000..52c6abd1 --- /dev/null +++ b/docker/Dockerfile.test @@ -0,0 +1,28 @@ +FROM ghcr.io/woodard/libabigail-fedora-base + +# docker build -f docker/Dockerfile.test -t test . + +RUN dnf install -y \ + file \ + diffutils \ + lbzip2 \ + elfutils-libs + +WORKDIR /code +COPY . /code + +RUN mkdir -p build && \ + autoreconf -i && \ + cd build && \ + CXXFLAGS="-g3 -fvar-tracking-assignments \ + -gstatement-frontiers -gvariable-location-views -grecord-gcc-switches -pipe -Wall \ + -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -Wp,-D_GLIBCXX_ASSERTIONS \ + -fstack-protector-strong -fstack-clash-protection -fcf-protection \ + -fasynchronous-unwind-tables -O2" CFLAGS="-g3 -fvar-tracking-assignments \ + -gstatement-frontiers -gvariable-location-views -grecord-gcc-switches -pipe -Wall \ + -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -Wp,-D_GLIBCXX_ASSERTIONS \ + -fstack-protector-strong -fstack-clash-protection -fcf-protection \ + -fasynchronous-unwind-tables -O2" LDFLAGS="-Wl,--no-undefined" \ + ../configure --prefix=/opt/libabigail --enable-rpm=yes && \ + make -j4 && \ + make check || (cat /tests/test-suite.log && exit 1) diff --git a/docker/Dockerfile.ubuntu b/docker/Dockerfile.ubuntu new file mode 100644 index 00000000..cb1dc166 --- /dev/null +++ b/docker/Dockerfile.ubuntu @@ -0,0 +1,30 @@ +ARG ubuntu_version=22.04 +ARG gcc_version=10.3.0 +FROM ghcr.io/rse-ops/gcc-ubuntu-${ubuntu_version}:gcc-${gcc_version} + +# docker build -t ghcr.io/woodard/libabigail-ubuntu-22.04 . + +# Install Libabigail to its own view +WORKDIR /opt/abigail-env +RUN . /opt/spack/share/spack/setup-env.sh && \ + spack env create -d . && \ + spack env activate . && \ + spack add libabigail@master && \ + spack --debug install + +# Prepare a source extraction of libabigail at /src (intended to be overwritten by user) +COPY . /src + +# Second run - should have deps cached +RUN . /opt/spack/share/spack/setup-env.sh && \ + + # This adds metadata for libabigail to spack.yaml + spack develop --path /src libabigail@master && \ + spack --debug install + # At this point you can spack install, and bind the code to /code to develop + +# ensure libabigail stuffs always on the path +RUN cd /opt/abigail-env && \ + spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh + +ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l", "-c"] diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff index dc80a6ef..c05bd8b1 100755 --- a/tools/fedabipkgdiff +++ b/tools/fedabipkgdiff @@ -213,6 +213,10 @@ def is_rpm_file(filename): mimetype = mimetypes.guess_type(filename)[0] if isfile else None isrpm = (mimetype == 'application/x-redhat-package-manager' or mimetype == 'application/x-rpm') + + # Most systems won't have rpm defined as a mimetype + if not mimetype and filename.endswith('.rpm'): + isrpm = True logger.debug('is_rpm_file(\'%s\'): isfile=%s, mimetype=\'%s\', isrpm=%s', filename, isfile, mimetype, isrpm) return isrpm -- 2.35.1