public inbox for gcc-cvs@sourceware.org help / color / mirror / Atom feed
From: Eugene Rozenfeld <erozen@gcc.gnu.org> To: gcc-cvs@gcc.gnu.org Subject: [gcc(refs/vendors/microsoft/heads/releases/gcc-11)] Add the contents of .github folder. Date: Fri, 29 Jul 2022 22:43:58 +0000 (GMT) [thread overview] Message-ID: <20220729224358.AD50438582A1@sourceware.org> (raw) https://gcc.gnu.org/g:ab70439c6f116d070992a3015ad590138e371a62 commit ab70439c6f116d070992a3015ad590138e371a62 Author: Eugene Rozenfeld <erozen@microsoft.com> Date: Fri Jul 29 15:42:26 2022 -0700 Add the contents of .github folder. Diff: --- .github/LICENSE.txt | 21 + .github/Mariner/specs/11.2.1/gcc.specs | 386 +++++++++++ .github/actions/build-composite/action.yaml | 42 ++ .github/actions/build-tarball/action.yaml | 89 +++ .github/actions/print-gcc-testset/action.yaml | 38 + .github/actions/pull-master-upstream/action.yaml | 50 ++ .github/actions/rebase-gcc-master/action.yaml | 50 ++ .github/actions/setup-vendor-branches/action.yaml | 52 ++ .github/actions/test-composite/action.yaml | 64 ++ .github/scripts/build-gcc.sh | 41 ++ .github/scripts/checkout-refs.sh | 42 ++ .github/scripts/common.py | 84 +++ .github/scripts/config.py | 116 ++++ .github/scripts/configure-gcc.sh | 41 ++ .github/scripts/delete-branches.sh | 25 + .github/scripts/downloadBuildArtifact.py | 222 ++++++ .github/scripts/gccWorkflow.py | 189 +++++ .github/scripts/gcc_release | 799 ++++++++++++++++++++++ .github/scripts/globals.py | 29 + .github/scripts/init.sh | 39 ++ .github/scripts/pushMergedVendorBranch.sh | 56 ++ .github/scripts/test-gcc.sh | 45 ++ .github/workflows/build.yaml | 88 +++ .github/workflows/delete-merge-branches.yaml | 52 ++ .github/workflows/package-rpm.yaml | 95 +++ .github/workflows/package-source-tarball.yaml | 72 ++ .github/workflows/test-gcc.yaml | 161 +++++ .github/workflows/update-main.yaml | 395 +++++++++++ .github/workflows/update-mirror-branches.yaml | 91 +++ 29 files changed, 3474 insertions(+) diff --git a/.github/LICENSE.txt b/.github/LICENSE.txt new file mode 100644 index 00000000000..63447fd8bbb --- /dev/null +++ b/.github/LICENSE.txt @@ -0,0 +1,21 @@ +Copyright (c) Microsoft Corporation. + +MIT License + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/.github/Mariner/specs/11.2.1/gcc.specs b/.github/Mariner/specs/11.2.1/gcc.specs new file mode 100644 index 00000000000..bd6e8a346bb --- /dev/null +++ b/.github/Mariner/specs/11.2.1/gcc.specs @@ -0,0 +1,386 @@ +%global security_hardening nofortify +%define _use_internal_dependency_generator 0 +Summary: Contains the GNU compiler collection +Name: gcc +Version: 11.2.1 +Release: 2%{?dist} +License: GPLv2+ +Vendor: Microsoft Corporation +Distribution: Mariner +Group: Development/Tools +URL: https://gcc.gnu.org/ +Source0: ~/output_dir/gcc-%{version}/gcc-%{version}.tar.xz +Requires: gcc-c++ = %{version}-%{release} +Requires: gmp +Requires: libgcc-atomic = %{version}-%{release} +Requires: libgcc-devel = %{version}-%{release} +Requires: libgomp-devel = %{version}-%{release} +Requires: libmpc +Requires: libstdc++-devel = %{version}-%{release} +Provides: cpp = %{version}-%{release} +Provides: gcc-plugin-devel = %{version}-%{release} +Provides: libasan = %{version}-%{release} +Provides: libasan%{?_isa} = %{version}-%{release} +Provides: libasan-static = %{version}-%{release} +Provides: libasan-static%{?_isa} = %{version}-%{release} +Provides: liblsan = %{version}-%{release} +Provides: liblsan%{?_isa} = %{version}-%{release} +Provides: liblsan-static = %{version}-%{release} +Provides: liblsan-static%{?_isa} = %{version}-%{release} +Provides: libtsan = %{version}-%{release} +Provides: libtsan%{?_isa} = %{version}-%{release} +Provides: libtsan-static = %{version}-%{release} +Provides: libtsan-static%{?_isa} = %{version}-%{release} +Provides: libubsan = %{version}-%{release} +Provides: libubsan%{?_isa} = %{version}-%{release} +Provides: libubsan-static = %{version}-%{release} +Provides: libubsan-static%{?_isa} = %{version}-%{release} +Provides: libquadmath = %{version}-%{release} +Provides: libquadmath-devel = %{version}-%{release} +Provides: libquadmath-devel%{?_isa} = %{version}-%{release} +#%if %{with_check} +#BuildRequires: autogen +#BuildRequires: dejagnu +#%endif + +%description +The GCC package contains the GNU compiler collection, +which includes the C and C++ compilers. + +%package -n gfortran +Summary: GNU Fortran compiler. +Group: Development/Tools +Requires: gcc = %{version}-%{release} +Provides: gcc-gfortran = %{version}-%{release} + +%description -n gfortran +The gfortran package contains GNU Fortran compiler. + +%package -n libgcc +Summary: GNU C Library +Group: System Environment/Libraries + +%description -n libgcc +The libgcc package contains GCC shared libraries for gcc. + +%package -n libgcc-atomic +Summary: GNU C Library for atomic counter updates +Group: System Environment/Libraries +Requires: libgcc = %{version}-%{release} +Provides: libatomic = %{version}-%{release} + +%description -n libgcc-atomic +The libgcc package contains GCC shared libraries for atomic counter updates. + +%package -n libgcc-devel +Summary: GNU C Library +Group: Development/Libraries +Requires: libgcc = %{version}-%{release} + +%description -n libgcc-devel +The libgcc package contains GCC shared libraries for gcc . +This package contains development headers and static library for libgcc. + +%package c++ +Summary: C++ support for GCC +Group: System Environment/Libraries +Requires: gcc = %{version}-%{release} +Requires: libstdc++-devel = %{version}-%{release} +Provides: gcc-g++ = %{version}-%{release} +Provides: g++ = %{version}-%{release} + +%description c++ +This package adds C++ support to the GNU Compiler Collection. +It includes support for most of the current C++ specification, +including templates and exception handling. + +%package -n libstdc++ +Summary: GNU C Library +Group: System Environment/Libraries +Requires: libgcc = %{version}-%{release} + +%description -n libstdc++ +This package contains the GCC Standard C++ Library v3, an ongoing project to implement the ISO/IEC 14882:1998 Standard C++ library. + +%package -n libstdc++-devel +Summary: GNU C Library +Group: Development/Libraries +Requires: libstdc++ = %{version}-%{release} +Provides: libstdc++-static = %{version}-%{release} + +%description -n libstdc++-devel +This is the GNU implementation of the standard C++ libraries. +This package includes the headers files and libraries needed for C++ development. + +%package -n libgomp +Summary: GNU C Library +Group: System Environment/Libraries + +%description -n libgomp +An implementation of OpenMP for the C, C++, and Fortran 95 compilers in the GNU Compiler Collection. + +%package -n libgomp-devel +Summary: Development headers and static library for libgomp +Group: Development/Libraries +Requires: libgomp = %{version}-%{release} + +%description -n libgomp-devel +An implementation of OpenMP for the C, C++, and Fortran 95 compilers in the GNU Compiler Collection. +This package contains development headers and static library for libgomp + +%prep +%setup -q +# disable no-pie for gcc binaries +sed -i '/^NO_PIE_CFLAGS = /s/@NO_PIE_CFLAGS@//' gcc/Makefile.in + +%build +CFLAGS="`echo " %{build_cflags} " | sed 's/-Werror=format-security/-Wno-error=format-security/'`" +CXXFLAGS="`echo " %{build_cxxflags} " | sed 's/-Werror=format-security/-Wno-error=format-security/'`" +FCFLAGS="`echo " %{build_fflags} " | sed 's/-Werror=format-security/-Wno-error=format-security/'`" +export CFLAGS +export CXXFLAGS +export FCFLAGS + +SED=sed \ +%configure \ + --enable-shared \ + --enable-threads=posix \ + --enable-__cxa_atexit \ + --enable-clocale=gnu \ + --enable-languages=c,c++,fortran \ + --disable-multilib \ + --disable-bootstrap \ + --enable-linker-build-id \ + --enable-plugin \ + --enable-default-pie \ + --with-system-zlib +make %{?_smp_mflags} + +%install +make %{?_smp_mflags} DESTDIR=%{buildroot} install +install -vdm 755 %{buildroot}/%{_libdir} +ln -sv %{_bindir}/cpp %{buildroot}/%{_libdir} +ln -sv gcc %{buildroot}%{_bindir}/cc +install -vdm 755 %{buildroot}%{_datarootdir}/gdb/auto-load%{_libdir} +mv -v %{buildroot}%{_lib64dir}/*gdb.py %{buildroot}%{_datarootdir}/gdb/auto-load%{_libdir} +chmod 755 %{buildroot}/%{_lib64dir}/libgcc_s.so.1 +rm -rf %{buildroot}%{_infodir} +#%find_lang %{name} --all-name + +%check +ulimit -s 32768 +# disable PCH tests is ASLR is on (due to bug in pch) +test `cat /proc/sys/kernel/randomize_va_space` -ne 0 && rm gcc/testsuite/gcc.dg/pch/pch.exp +# disable security hardening for tests +rm -f $(dirname $(gcc -print-libgcc-file-name))/../specs +# run only gcc tests +make %{?_smp_mflags} check-gcc +# Only 1 FAIL is OK +[ `grep ^FAIL testsuite/gcc/gcc.sum | wc -l` -ne 1 -o `grep ^XPASS testsuite/gcc/gcc.sum | wc -l` -ne 0 ] && exit 1 ||: +[ `grep "^FAIL: gcc.dg/cpp/trad/include.c (test for excess errors)" testsuite/gcc/gcc.sum | wc -l` -ne 1 ] && exit 1 ||: + +%post -p /sbin/ldconfig +%postun -p /sbin/ldconfig + +%files +%defattr(-,root,root) +%license COPYING +%{_libdir}/cpp +# Executables +%exclude %{_bindir}/*gfortran +%exclude %{_bindir}/*c++ +%exclude %{_bindir}/*g++ +%{_bindir}/* +# Libraries +%{_lib64dir}/* +%exclude %{_libexecdir}/gcc/%{_arch}-%{_host_vendor}-linux-gnu/%{version}/f951 +%exclude %{_libexecdir}/gcc/%{_arch}-%{_host_vendor}-linux-gnu/%{version}/cc1plus +%{_libdir}/gcc/* +# Library executables +%{_libexecdir}/gcc/* +# Man pages +%{_mandir}/man1/gcov.1.gz +%{_mandir}/man1/gcov-dump.1.gz +%{_mandir}/man1/gcov-tool.1.gz +%{_mandir}/man1/gcc.1.gz +%{_mandir}/man1/g++.1.gz +%{_mandir}/man1/cpp.1.gz +%{_mandir}/man1/lto-dump.1.gz +%{_mandir}/man7/*.gz +%{_datadir}/gdb/* + +%exclude %{_lib64dir}/libgcc* +%exclude %{_lib64dir}/libstdc++* +%exclude %{_lib64dir}/libsupc++* +%exclude %{_lib64dir}/libgomp* + +%files -n gfortran +%defattr(-,root,root) +%{_bindir}/*gfortran +%{_mandir}/man1/gfortran.1.gz +%{_libexecdir}/gcc/%{_arch}-%{_host_vendor}-linux-gnu/%{version}/f951 + +%files -n libgcc +%defattr(-,root,root) +%{_lib64dir}/libgcc_s.so.* + +%files -n libgcc-atomic +%defattr(-,root,root) +%{_lib64dir}/libatomic.so* + +%files -n libgcc-devel +%defattr(-,root,root) +%{_lib64dir}/libgcc_s.so +%{_lib64dir}/libcc1.* + +%files c++ +%defattr(-,root,root) +%{_bindir}/*c++ +%{_bindir}/*g++ +%{_libexecdir}/gcc/%{_arch}-%{_host_vendor}-linux-gnu/%{version}/cc1plus + +%files -n libstdc++ +%defattr(-,root,root) +%{_lib64dir}/libstdc++.so.* +%dir %{_datarootdir}/gcc-%{version}/python/libstdcxx +%{_datarootdir}/gcc-%{version}/python/libstdcxx/* + +%files -n libstdc++-devel +%defattr(-,root,root) +%{_lib64dir}/libstdc++.so +%{_lib64dir}/libstdc++.la +%{_lib64dir}/libstdc++.a +%{_lib64dir}/libstdc++fs.a +%{_lib64dir}/libsupc++.a +%{_lib64dir}/libsupc++.la + +%{_includedir}/c++/* + +%files -n libgomp +%defattr(-,root,root) +%{_lib64dir}/libgomp*.so.* + +%files -n libgomp-devel +%defattr(-,root,root) +%{_lib64dir}/libgomp.a +%{_lib64dir}/libgomp.la +%{_lib64dir}/libgomp.so +%{_lib64dir}/libgomp.spec + +%changelog +* Tue Jan 25 2022 Thomas Crain <thcrain@microsoft.com> - 11.2.0-2 +- Add provides for libasan, liblsan, libtsan, and libubsan (and their static counterparts) to the main package +- Remove CVE-2019-15847 nopatch file (not relevant to our version of GCC) + +* Mon Oct 18 2021 Andrew Phelps <anphel@microsoft.com> - 11.2.0-1 +- Update to version 11.2.0 + +* Fri Feb 05 2021 Joe Schmitt <joschmit@microsoft.com> - 9.1.0-11 +- Replace incorrect %%{_lib} usage with %%{_libdir} + +* Fri Jan 08 2021 Ruying Chen <v-ruyche@microsoft.com> - 9.1.0-10 +- Provide libquadmath and libquadmath-devel. + +* Tue Nov 03 2020 Joe Schmitt <joschmit@microsoft.com> - 9.1.0-9 +- Provide gcc-plugin-devel. + +* Mon Sep 28 2020 Ruying Chen <v-ruyche@microsoft.com> 9.1.0-8 +- Split gcc-c++ subpackage. +- Provide cpp, gcc-gfortran, libatomic, and listdc++-static. + +* Thu Sep 10 2020 Thomas Crain <thcrain@microsoft.com> - 9.1.0-7 +- Ignore CVE-2019-15847, as it applies to an unsupported ISA + +* Mon Jul 06 2020 Henry Beberman <henry.beberman@microsoft.com> - 9.1.0-6 +- Comment out with_check BuildRequires to break circular dependency in build graph. + +* Thu Jun 11 2020 Henry Beberman <henry.beberman@microsoft.com> - 9.1.0-5 +- Disable -Werror=format-security to build with hardened cflags + +* Sat May 09 2020 Nick Samson <nisamson@microsoft.com> - 9.1.0-4 +- Added %%license line automatically + +* Thu Apr 30 2020 Emre Girgin <mrgirgin@microsoft.com> - 9.1.0-3 +- Renaming mpc to libmpc + +* Thu Apr 09 2020 Emre Girgin <mrgirgin@microsoft.com> - 9.1.0-2 +- Add the "--enable-default-pie" flag in order to enforce ASLR-enabled binaries. + +* Tue Mar 17 2020 Andrew Phelps <anphel@microsoft.com> - 9.1.0-1 +- Update to version 9.1.0. License verified. Add libstdc++fs.a + +* Tue Jan 21 2020 Andrew Phelps <anphel@microsoft.com> - 7.3.0-6 +- Fixing build issues for multiple architectures + +* Tue Sep 03 2019 Mateusz Malisz <mamalisz@microsoft.com> - 7.3.0-5 +- Initial CBL-Mariner import from Photon (license: Apache2). + +* Fri Nov 02 2018 Alexey Makhalov <amakhalov@vmware.com> - 7.3.0-4 +- Use nofortify security_hardening instead of sed hacking +- Use %configure + +* Wed Sep 19 2018 Alexey Makhalov <amakhalov@vmware.com> - 7.3.0-3 +- Fix compilation issue for glibc-2.28 + +* Thu Aug 30 2018 Keerthana K <keerthanak@vmware.com> - 7.3.0-2 +- Packaging .a files (libstdc++-static files). + +* Wed Aug 01 2018 Srivatsa S. Bhat <srivatsa@csail.mit.edu> - 7.3.0-1 +- Update to version 7.3.0 to get retpoline support. + +* Tue Nov 14 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-7 +- Aarch64 support + +* Mon Oct 02 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-6 +- Added smp_mflags for parallel build + +* Mon Sep 25 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-5 +- Enable elfdeps for libgcc_s to generate libgcc_s.so.1(*)(64bit) provides + +* Mon Aug 28 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-4 +- Fix makecheck + +* Tue Aug 15 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-3 +- Fix compilation issue for glibc-2.26 + +* Tue Aug 15 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-2 +- Improve make check + +* Thu Mar 9 2017 Alexey Makhalov <amakhalov@vmware.com> - 6.3.0-1 +- Update version to 6.3 + +* Thu Mar 02 2017 Xiaolin Li <xiaolinl@vmware.com> - 5.3.0-6 +- Enabled fortran. + +* Wed Feb 22 2017 Alexey Makhalov <amakhalov@vmware.com> - 5.3.0-5 +- Added new plugin entry point: PLUGIN_TYPE_CAST (.patch) + +* Thu Sep 8 2016 Alexey Makhalov <amakhalov@vmware.com> - 5.3.0-4 +- Enable plugins and linker build id. + +* Tue May 24 2016 Priyesh Padmavilasom <ppadmavilasom@vmware.com> - 5.3.0-3 +- GA - Bump release of all rpms + +* Tue May 17 2016 Anish Swaminathan <anishs@vmware.com> - 5.3.0-2 +- Change package dependencies + +* Mon Mar 28 2016 Alexey Makhalov <amakhalov@vmware.com> - 5.3.0-1 +- Update version to 5.3 + +* Tue Nov 10 2015 Xiaolin Li <xiaolinl@vmware.com> - 4.8.2-6 +- Handled locale files with macro find_lang + +* Mon Nov 02 2015 Vinay Kulkarni <kulkarniv@vmware.com> - 4.8.2-5 +- Put libatomic.so into its own package. + +* Wed May 20 2015 Touseef Liaqat <tliaqat@vmware.com> - 4.8.2-4 +- Updated group. + +* Mon May 18 2015 Touseef Liaqat <tliaqat@vmware.com> - 4.8.2-3 +- Update according to UsrMove. + +* Fri May 15 2015 Divya Thaluru <dthaluru@vmware.com> - 4.8.2-2 +- Packaging .la files + +* Tue Apr 01 2014 baho-utot <baho-utot@columbus.rr.com> - 4.8.2-1 +- Initial build. First version diff --git a/.github/actions/build-composite/action.yaml b/.github/actions/build-composite/action.yaml new file mode 100644 index 00000000000..21d2e4b6f02 --- /dev/null +++ b/.github/actions/build-composite/action.yaml @@ -0,0 +1,42 @@ +# Composite yaml that's re-usable in multiple workflows to build gcc + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +inputs: + configjson: + description: 'Config object converted into json' + required: true + default: '' +runs: + using: "composite" + steps: + - name: Build gcc + run: | + chmod +x .github/scripts/gccWorkflow.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + python -c 'import sys; from gccWorkflow import *; GccWorkflow.Build(sys.argv[1])' "${CONFIG_JSON}" + shell: bash + env: + CONFIG_JSON: ${{ inputs.configJson }} \ No newline at end of file diff --git a/.github/actions/build-tarball/action.yaml b/.github/actions/build-tarball/action.yaml new file mode 100644 index 00000000000..3949c37b7f7 --- /dev/null +++ b/.github/actions/build-tarball/action.yaml @@ -0,0 +1,89 @@ +# Composite yaml to rebase to upstream gcc master + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +name: "build-tarball" +description: "Build tar.xz release of gcc" +runs: + using: "composite" + steps: + + # This step will set an environment variable for branch name. + # e.g. branch_name=releases/gcc-11 + - name: Set Environment Variable For Branch Name + shell: bash + run: echo "branch_name=${{ github.head_ref || github.ref_name }}" >> $GITHUB_ENV + + # This step will set an environment variable for release name. + # e.g. release_version=11.2.1 + - name: Set Environment Variable For Release Version + shell: bash + working-directory: ./gcc + run: echo "release_version=$(git show origin/${{ env.branch_name }}:gcc/BASE-VER)" >> $GITHUB_ENV + + # This is only to fix a bug in CBL-Mariner build procedure. + # Eventually we will be able to delete this step. + - name: Create Include Dir # to fix building GCC gfortran bug in Mariner + run: | + mkdir -p /usr/lib/gfortran/modules + shell: bash + + # Update the package manager. + - name: TDNF Update + run: | + unset HOME && tdnf update -y + shell: bash + + # Enable mariner-repos-extended repo. + # In newer versions of CBL-Mariner, you have to enable this repo to install libdwarf and a few other packages. + - name: TDNF Install mariner-repos-extended + run: | + unset HOME && tdnf install -y mariner-repos-extended + shell: bash + + # Install required packages. These packages are required to build gcc and create a RPM. + - name: Install Required Packages + run: | + unset HOME && tdnf install glibc-devel file dejagnu texinfo build-essential rpm-build git ca-certificates wget flex bison gdb gawk libgomp-devel gfortran libgomp lapack libdwarf make sudo autoconf automake libstdc++ libstdc++-devel sed gmp-devel mpfr-devel libmpc-devel lynx -y + shell: bash + + # In our fork of gcc_release script, gcc_release clones the repository locally. + # Because of that we need the release branch we are building to be available locally. + - name: Enable Release Branch Locally + working-directory: ./gcc + run: | + git fetch origin ${{ env.branch_name }}:${{ env.branch_name }} + shell: bash + + # This is just running gcc_relase script with bare minimum flags. + # The flags are : + # -f : build is final (don't append "snapshot" to the end of tar.xz files). + # -r : release version. + # -l : it runs locally (do not fork from a repository). + # -c : clone the target branch locally. + # -d : destination directory for gcc-$(release_version).tar.xz. + # for more information about the flags, take a look at the commit pushing gcc_release to .github/scripts/gcc_release + # WARNING: This is going to take a long long time. Go get a coffee, watch a tv show and/or sleep. + - name: Run gcc_release Script + working-directory: ./gcc + run: ./.github/scripts/gcc_release -f -r ${{ env.release_version }} -c -l -d ~/ sources tarfiles + shell: sh diff --git a/.github/actions/print-gcc-testset/action.yaml b/.github/actions/print-gcc-testset/action.yaml new file mode 100644 index 00000000000..8b4f18a540c --- /dev/null +++ b/.github/actions/print-gcc-testset/action.yaml @@ -0,0 +1,38 @@ +# Composite yaml that prints the GCC test set so it can be used as a YAML matrix variable for running tests + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +outputs: + testSet: + description: 'GCC test set in JSON representation' + value: ${{ steps.printGCCTestSet.outputs.testSet }} +runs: + using: "composite" + steps: + - name: Print GCC test set + id: printGCCTestSet + run: | + chmod +x .github/scripts/gccWorkflow.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'import sys; from gccWorkflow import *; GccWorkflow.PrintTestSet()' + shell: bash diff --git a/.github/actions/pull-master-upstream/action.yaml b/.github/actions/pull-master-upstream/action.yaml new file mode 100644 index 00000000000..0b589c87e7a --- /dev/null +++ b/.github/actions/pull-master-upstream/action.yaml @@ -0,0 +1,50 @@ +# Composite yaml to add the upstream gcc remote and checkout from their master branch + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +inputs: + masterRef: + description: 'Reference off of gcc/master to sync to' + required: false + default: 'gcc/master' + scriptsRef: + description: 'Commit reference to checkout .github scripts from' + required: true + default: '' + +runs: + using: "composite" + steps: + - name: Initialize environment and checkout from ${{ inputs.masterRef }} + run: | + python -m pip install requests + chmod +x .github/scripts/common.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'from common import *; RunScript(".github/scripts/init.sh", True)' + git fetch gcc master + python -c 'import sys; from common import *; RunScript(".github/scripts/checkout-refs.sh", True, sys.argv[1], sys.argv[2])' "${MASTER_REF}" "${SCRIPTS_REF}" + git log -1 + shell: bash + env: + SCRIPTS_REF: ${{ inputs.scriptsRef }} + MASTER_REF: ${{ inputs.masterRef }} \ No newline at end of file diff --git a/.github/actions/rebase-gcc-master/action.yaml b/.github/actions/rebase-gcc-master/action.yaml new file mode 100644 index 00000000000..941ae85a822 --- /dev/null +++ b/.github/actions/rebase-gcc-master/action.yaml @@ -0,0 +1,50 @@ +# Composite yaml to rebase to upstream gcc master + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +runs: + using: "composite" + steps: + - name: Initialize environment + run: | + python -m pip install requests + chmod +x .github/scripts/common.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'from common import *; RunScript(".github/scripts/init.sh", True)' + shell: bash + - name: Fetch + run: | + git fetch gcc master + shell: bash + - name: Rebase + run: | + git log gcc/master -1 + git config --global user.email "erozen@microsoft.com" + git config --global user.name "Eugene Rozenfeld" + git rebase gcc/master + shell: bash + - name: Check commits + run: | + git status + git log -15 + shell: bash diff --git a/.github/actions/setup-vendor-branches/action.yaml b/.github/actions/setup-vendor-branches/action.yaml new file mode 100644 index 00000000000..415165a1b78 --- /dev/null +++ b/.github/actions/setup-vendor-branches/action.yaml @@ -0,0 +1,52 @@ +# Composite yaml to add the upstream gcc remote, setup the vendor branches and checkout the specified vendor branch + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +inputs: + vendorRef: + description: 'Reference off of gcc/master to sync to' + required: true + default: '' + scriptsRef: + description: 'Commit reference to checkout .github scripts from' + required: true + default: '' + +runs: + using: "composite" + steps: + - name: Initialize environment and checkout from vendor branch + run: | + python -m pip install requests + chmod +x .github/scripts/common.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'from common import *; RunScript(".github/scripts/init.sh", True)' + echo -e "MS Automation\ngnutools@microsoft.com\ngcc\nMSAutomation\nme\nyes" > gitCustomization.txt + cat "gitCustomization.txt" | contrib/gcc-git-customization.sh + python -c 'from common import *; RunScript("contrib/git-fetch-vendor.sh", True, "microsoft")' + git branch -a + python -c 'import sys; from common import *; RunScript(".github/scripts/checkout-refs.sh", True, sys.argv[1], sys.argv[2])' "${VENDOR_REF}" "${SCRIPTS_REF}" + shell: bash + env: + SCRIPTS_REF: ${{ inputs.scriptsRef }} + VENDOR_REF: ${{ inputs.vendorRef }} diff --git a/.github/actions/test-composite/action.yaml b/.github/actions/test-composite/action.yaml new file mode 100644 index 00000000000..92aaaee4ecf --- /dev/null +++ b/.github/actions/test-composite/action.yaml @@ -0,0 +1,64 @@ +# Composite yaml that's re-usable in multiple workflows to run gcc tests + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +inputs: + configjson: + description: 'Config object converted into json' + required: true + default: '' + testSet: + description: 'gcc test set to run' + required: true + default: '' + githubtoken: + description: 'GitHub auth token' + required: true + default: '' + buildDownloaded: + description: 'Whether the build is already available on the machine this composite action is running on' + required: true + default: true +outputs: + configJson: + description: 'Config object that may have been updated' + value: ${{ steps.runTests.outputs.noSecretConfigJson }} +runs: + using: "composite" + steps: + - name: Download build and run tests + id: runTests + run: | + chmod +x .github/scripts/gccWorkflow.py + echo "$PYTHONPATH" + echo "$CONFIG_JSON" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "$BUILD_DOWNLOADED" + python -c 'import sys; from gccWorkflow import *; GccWorkflow.Test(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4])' "${CONFIG_JSON}" "${TEST_SET}" "${GITHUB_TOKEN}" "${BUILD_DOWNLOADED}" + shell: bash + env: + CONFIG_JSON: ${{ inputs.configJson }} + TEST_SET: ${{ inputs.testSet }} + GITHUB_TOKEN: ${{ inputs.githubtoken }} + BUILD_DOWNLOADED: ${{ inputs.buildDownloaded }} diff --git a/.github/scripts/build-gcc.sh b/.github/scripts/build-gcc.sh new file mode 100644 index 00000000000..d55cc37c26c --- /dev/null +++ b/.github/scripts/build-gcc.sh @@ -0,0 +1,41 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +cd ../objdir +$PWD/../gcc/configure --prefix=$HOME/GCC --enable-languages=c,c++,fortran + +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi + +make -j$(nproc) +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi + +sudo make install +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi \ No newline at end of file diff --git a/.github/scripts/checkout-refs.sh b/.github/scripts/checkout-refs.sh new file mode 100644 index 00000000000..bdbdf829e40 --- /dev/null +++ b/.github/scripts/checkout-refs.sh @@ -0,0 +1,42 @@ +# $1 is the reference to use for checking out GCC sources +# $2 is the reference to use for checking out the .github scripts + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +git checkout $1 -f +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi + +git fetch origin +refs/pull/*:refs/remotes/origin/refs/pull/* +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi + +git checkout $2 -- .github/ +exit_code=$? +if [ $exit_code != 0 ]; then + exit $exit_code +fi \ No newline at end of file diff --git a/.github/scripts/common.py b/.github/scripts/common.py new file mode 100644 index 00000000000..58de82132d9 --- /dev/null +++ b/.github/scripts/common.py @@ -0,0 +1,84 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +import globals +import logging +import subprocess +import requests +import time + +# Exception class to raise when a basic workflow error happens +class WorkflowError(Exception): + pass + +def GetLogger(): + if (globals.logger == None): + logging.basicConfig(level=logging.NOTSET) + logging.root.setLevel(logging.DEBUG) + globals.logger = logging.getLogger() + globals.logger.setLevel(logging.DEBUG) + return globals.logger + +def RaiseWorkflowError(error): + logging.shutdown() + raise WorkflowError(error) + +# This name needs to match what's in the build.yaml file +def GetGccBuildName(): + return 'build' + +def GetandPrintCurrentSHA(): + res = subprocess.run('git rev-parse HEAD', shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + + # Output for Github to pick up output for future jobs + print("::set-output name=currentSHA::" + str(res.stdout, 'utf-8')) + +def SendGetRestCmd(restCmd, restArgs, restHeader): + logger = GetLogger() + retryCount = 10 + while True: + res = requests.get(restCmd, params=restArgs, headers=restHeader) + if res.ok: + return res + + # Retry the command after a sleep + sleepTimeSeconds = 30 + logger.error("Sleeping for " + str(sleepTimeSeconds) + " seconds before retrying " + restCmd) + time.sleep(sleepTimeSeconds) + + retryCount = retryCount - 1 + if retryCount == 0: + RaiseWorkflowError("Max retry count hit with " + restCmd) + +# Run the provided script with the provided arguments +def RunScript(scriptPath, failOnNonZeroReturn, *args): + argsStr = "" + for a in args: + argsStr += " " + argsStr += a + + res = subprocess.run(''' + chmod +x ''' + scriptPath + ''' + ''' + scriptPath + argsStr, + shell=True, check=failOnNonZeroReturn, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + + return res \ No newline at end of file diff --git a/.github/scripts/config.py b/.github/scripts/config.py new file mode 100644 index 00000000000..709e340c4cc --- /dev/null +++ b/.github/scripts/config.py @@ -0,0 +1,116 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +import json +from json import JSONEncoder +import globals +from common import * +from downloadBuildArtifact import * +import sys + +# subclass JSONEncoder to convert the Config object into JSON +class ConfigEncoder(JSONEncoder): + def default(self, o): + return o.__dict__ + +class Config(object): + + def __init__(self, sha, token, buildArtifactID, workflowRunID, failedWorkflowErrorStr, failedWorkflowReturnCode): + self.commitSHA = sha # Commit SHA for the checkin + self.accessToken = token # Access token for REST APIs + self.gccBuildArtifactID = buildArtifactID # GCC Build artifact ID + self.runID = workflowRunID # Run ID of workflow (not job) that ran Setup() + self.failedWorkflowError = failedWorkflowErrorStr # Whether or not the workflow failed + self.failedReturnCode = failedWorkflowReturnCode # Failed workflow error code + logger = GetLogger() + logger.info('creating an instance of Config') + + @staticmethod + def Setup(githubObject, accessToken, isBuild): + logger = GetLogger() + + logger.debug("In Setup") + logger.debug("---") + logger.debug("Github object = " + githubObject) + githubJson = json.loads(githubObject) + logger.debug(githubJson["event"]) + commit = githubJson["sha"] + if ("pull_request" in githubJson["event"]): + commit = githubJson["event"]["pull_request"]["head"]["sha"] + + runID = githubJson["run_id"] + + logger.info("SHA = " + commit) + workflowName = githubJson["workflow"] + + # If this isn't a GCC build, wait for the GCC build to complete with the needed artifacts + if (not isBuild): + gccBuildArtifactID = WaitOnGccBuild(commit, accessToken) + else: + gccBuildArtifactID = 0 + + # Construct Config object + newConfig = Config(commit, accessToken, gccBuildArtifactID, runID, '', 0) + configJson = json.dumps(newConfig, cls=ConfigEncoder) + + # Output for Github to pick up output for future steps + print("::set-output name=configJson::" + configJson) + + # Set global config object + globals.configObj = newConfig + + @staticmethod + def PrintNoSecretConfigJson(): + # GitHub won't allow the printing of strings with secrets in them for future jobs + # so we need to clear out any secrets in our config object prior to printing it + + # Clear out the access token field + globals.configObj.accessToken = globals.configObj.accessToken.replace(globals.configObj.accessToken, '') + configJson = json.dumps(globals.configObj, cls=ConfigEncoder) + + # Output for Github to pick up output for future jobs + print("::set-output name=noSecretConfigJson::" + configJson) + + @staticmethod + def PrintNoSecretConfigJsonFromJson(configJson): + Config.Reload(configJson) + Config.PrintNoSecretConfigJson() + + @staticmethod + def Reload(configJson, accessToken=""): + loadedJson = json.loads(configJson) + + # If no access token is provided, use the one in the config already + # This is possible in cases where the config setup and reload functions are run on the same machine + if (accessToken==""): + accessToken=loadedJson["accessToken"] + + # Set global config object + globals.configObj = Config(loadedJson["commitSHA"], accessToken, loadedJson["gccBuildArtifactID"], loadedJson["runID"], loadedJson["failedWorkflowError"], loadedJson["failedReturnCode"]) + + @staticmethod + def RaiseErrorIfWorkflowFailed(): + if (globals.configObj.failedWorkflowError): + logger = GetLogger() + logger.error(globals.configObj.failedWorkflowError) + logging.shutdown() + sys.exit(globals.configObj.failedReturnCode) diff --git a/.github/scripts/configure-gcc.sh b/.github/scripts/configure-gcc.sh new file mode 100644 index 00000000000..69434f8a7bd --- /dev/null +++ b/.github/scripts/configure-gcc.sh @@ -0,0 +1,41 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +sudo apt update + +# Install gcc 7 and g++ 7 +sudo apt-get install gcc-7 g++-7 g++-7-multilib libstdc++-7-doc binutils-doc build-essential cpp-doc gcc-7-doc libstdc++6-7-dbg lib32stdc++6-7-dbg libx32stdc++6-7-dbg make autoconf automake libtool flex bison gdb gawk gcc-doc gfortran libgfortran3 libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan4-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg libquadmath0-dbg glibc-doc python lynx -y + +# Redirect gcc and g++ to the installed versions +sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 60 +sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 60 + +# Redirect cc and c++ to the installed gcc and g++ versions: +sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30 +sudo update-alternatives --set cc /usr/bin/gcc + +sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30 +sudo update-alternatives --set c++ /usr/bin/g++ + +sudo apt install -y texinfo +sudo apt-get install -y dejagnu +./contrib/download_prerequisites \ No newline at end of file diff --git a/.github/scripts/delete-branches.sh b/.github/scripts/delete-branches.sh new file mode 100644 index 00000000000..93582f1e4af --- /dev/null +++ b/.github/scripts/delete-branches.sh @@ -0,0 +1,25 @@ +# Deletes branches that match the regex and are older than the maximum acceptable branch age +# $1 is the regex to match the branch to +# $2 is the maximum acceptable branch age + +branchesStr=$(git for-each-ref --format='%(refname:short) %(committerdate:relative)' --sort=-committerdate refs/remotes/origin) +readarray -t branches <<<"$branchesStr" +maxAcceptableDateInSecs=$(date -d "$2 days ago" +%s) +regexMatch=$1 + +for (( i=0; i<${#branches[@]} ; i+=1 )) ; do + echo "${branches[i]}" + read -r branchName lastCommitRelative <<< "${branches[i]}" + # Remove commas in the date which can occur with older branches. e.g. "2 years, 2 months ago" + lastCommitRelative=$(echo "$lastCommitRelative" | sed 's;,;;' ) + lastCommitRelativeDate=$(date -d "$lastCommitRelative" +%s) + + echo "$lastCommitRelative" + echo "$lastCommitRelativeDate" + echo "$maxAcceptableDateInSecs" + if [ $lastCommitRelativeDate -lt $maxAcceptableDateInSecs ] + then + branchNameTrimmed=$( echo "$branchName" | sed 's;origin/;;' ) + [[ $branchNameTrimmed =~ $regexMatch ]] && echo "Deleting branch $branchNameTrimmed" && git push origin --delete $branchNameTrimmed + fi +done \ No newline at end of file diff --git a/.github/scripts/downloadBuildArtifact.py b/.github/scripts/downloadBuildArtifact.py new file mode 100644 index 00000000000..45336fb4317 --- /dev/null +++ b/.github/scripts/downloadBuildArtifact.py @@ -0,0 +1,222 @@ +# Requirements: +# sudo apt-get install python-pip +# sudo apt-get install python +# python -m pip install requests + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +import requests +import json +from datetime import datetime +import datetime as dt +import zipfile +import time +from common import * +import globals +import re +import tarfile +import os +import subprocess + +def WaitOnLatestWorkflow(branch, workflowName, accessToken, raiseErrorOnFailure): + logger = GetLogger() + logger.info("Looking for a " + workflowName + " build for branch: " + branch) + + latestWorkflowTime = dt.datetime(1970, 1, 1) + latestWorkflowId = 0 + + while latestWorkflowId == 0: + # Look at all workflow runs on the branch + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/runs?branch=" + branch, {}, {'Authorization': "token " + accessToken}) + runs = res.json()['workflow_runs'] + + # This could be optimized if we can guarantee that the results are sorted. It seems like the results are sorted from most recent to least recent, but there + # doesn't seem to be any documentation supporting this. + + for entry in runs: + if (entry['name'] == workflowName): + date = datetime.strptime(entry['created_at'], '%Y-%m-%dT%H:%M:%SZ') + if (latestWorkflowTime < date): + latestWorkflowTime = date + latestWorkflowId = entry['id'] + + + logger.info("Found workflow ID: " + str(latestWorkflowId)) + buildCompleted = False + conclusion = '' + # Wait until latestWorkflowId completes + while buildCompleted == False: + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/runs/" + str(latestWorkflowId), {}, {'Authorization': "token " + accessToken}) + if res.ok: + resJson = res.json() + status = resJson['status'] + if (status == 'completed'): + buildCompleted = True + conclusion = resJson['conclusion'] + + if not buildCompleted: + # Sleep for 30 seconds + sleepTimeSeconds = 30 + logger.error("Sleeping for " + str(sleepTimeSeconds) + " seconds while waiting on build number " + str(latestWorkflowId) + " to finish") + time.sleep(sleepTimeSeconds) + + logger.info("Run completed. Link available at:") + logger.info("https://github.com/microsoft/gcc/actions/runs/" + str(latestWorkflowId)) + + if raiseErrorOnFailure: + if conclusion != 'success': + RaiseWorkflowError("Workflow did not succeed. Conclusion was: " + conclusion) + +def FindGccBuildArtifact(workflowRunID, accessToken): + logger = GetLogger() + numArtifactRetries = 60 + while True: + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/runs/" + str(workflowRunID) + "/artifacts", {}, {'Authorization': "token " + accessToken}) + logger.debug("Artifact Json:" + json.dumps(res.json())) + + gccBuildArtifactID = 0 + for artifact in res.json()['artifacts']: + if (artifact['name'] == "gccBuild"): + gccBuildArtifactID = artifact['id'] + + if (gccBuildArtifactID != 0): + break + elif (gccBuildArtifactID == 0 and numArtifactRetries == 0): + RaiseWorkflowError("No gcc build artifact found") + + numArtifactRetries = numArtifactRetries-1 + + # We need this sleep here because even if the build job status is "completed", the gccBuild artifacts aren't + # immediately ready. If for some reason they aren't, sleep for a bit and retry. + + # Sleep for 30 seconds + sleepTimeSeconds = 30 + logger.info("Sleeping for " + str(sleepTimeSeconds) + " seconds while waiting on artifacts for workflow run ID " + str(workflowRunID)) + time.sleep(sleepTimeSeconds) + + logger.info("GCC Build artifact ID = " + str(gccBuildArtifactID)) + return gccBuildArtifactID + +def WaitOnGccBuild(commit, accessToken): + logger = GetLogger() + logger.info("Looking for a gcc build for " + commit) + + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/commits/" + commit + "/check-runs", {'check_name': GetGccBuildName()}, {'Authorization': "token " + accessToken}) + checkruns = res.json()['check_runs'] + latestBuildTime = dt.datetime(1970, 1, 1) + latestBuildId = 0 + + for entry in checkruns: + if (entry['name'] == GetGccBuildName()): + date = datetime.strptime(entry['started_at'], '%Y-%m-%dT%H:%M:%SZ') + + # take the most recent build run + if (latestBuildTime < date): + latestBuildTime = date + latestBuildId = entry['id'] + + if (latestBuildId == 0): + RaiseWorkflowError("No run found") + + logger.info("Found latest build workflow ID: " + str(latestBuildId)) + + while True: + # Get the run ID + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/jobs/" + str(latestBuildId), {}, {'Authorization': "token " + accessToken}) + jobJson = res.json() + workflowRunID = jobJson['run_id'] + + if jobJson['status'] == 'completed': + logger.debug(jobJson) + conclusion = jobJson['conclusion'] + if (conclusion == 'success'): + break + else: + logger.error("Job conclusion is: " + conclusion) + RaiseWorkflowError("Build run ID " + str(latestBuildId) + " failed") + else: + # Sleep for 30 seconds + sleepTimeSeconds = 30 + logger.error("Sleeping for " + str(sleepTimeSeconds) + " seconds while waiting on build number " + str(latestBuildId) + " to finish") + time.sleep(sleepTimeSeconds) + + logger.info("Workflow run ID = " + str(workflowRunID)) + + gccBuildArtifactID = FindGccBuildArtifact(workflowRunID, accessToken) + + return gccBuildArtifactID + +# Returns a list of artifact objects in the runId that match the regex +def GetArtifactObjsInRun(runId, regex): + r = re.compile(regex) + matchedList = [] + + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/runs/" + str(runId) + "/artifacts", {}, {'Authorization': "token " + globals.configObj.accessToken}) + artifactsList = res.json()['artifacts'] + for artifact in artifactsList: + name = artifact['name'] + if r.match(name): + matchedList.append(artifact) + + return matchedList + +def DownloadArtifact(artifactID): + logger = GetLogger() + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/artifacts/" + str(artifactID), {}, {'Authorization': "token " + globals.configObj.accessToken}) + name = res.json()['name'] + zipName = name + '.zip' + + res = SendGetRestCmd("https://api.github.com/repos/microsoft/gcc/actions/artifacts/" + str(artifactID) +"/zip", {}, {'Authorization': "token " + globals.configObj.accessToken}) + + logger.info("Downloading " + name + " zip from artifact ID " + str(artifactID)) + with open(zipName, 'wb') as f: + f.write(res.content) + + logger.info("Unzipping zip") + with zipfile.ZipFile(zipName, 'r') as zip_ref: + zip_ref.extractall(name) + print('currentCWD') + print(os.getcwd()) + logger.info("Done downloading") + + return name + +def DownloadBuildArtifact(): + logger = GetLogger() + + if (globals.configObj.gccBuildArtifactID): + gccBuildArtifactID = globals.configObj.gccBuildArtifactID + else: + gccBuildArtifactID = FindGccBuildArtifact(globals.configObj.runID, globals.configObj.accessToken) + + # TODO: Support downloading build artifact without a config object setup so the script can be used outside of workflows by developers + filename = DownloadArtifact(gccBuildArtifactID) + "/objdir.tar.gz" + res = subprocess.run('tar -xzf ' + filename + ' -m -C ..', + shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + + logger.info("tar cmd output = " + str(res.stdout, 'utf-8')) + + res = subprocess.run('rm ' + filename, + shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + + logger.info("rm cmd output = " + str(res.stdout, 'utf-8')) diff --git a/.github/scripts/gccWorkflow.py b/.github/scripts/gccWorkflow.py new file mode 100644 index 00000000000..411aac30434 --- /dev/null +++ b/.github/scripts/gccWorkflow.py @@ -0,0 +1,189 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from config import * +import subprocess +from downloadBuildArtifact import * +import sys +import globals +from common import * +import os +import json +import glob + +class GccWorkflow(object): + # Setup the config object and wait for any runs necessary to finish before proceeding onto the next job section + # to avoid exceeding the 6 hr limit on Github Actions that run on Github machines + @staticmethod + def Init(githubContext, accessToken, isInitForBuild=False): + Config.Setup(githubContext, accessToken, isInitForBuild) + + @staticmethod + def PrintTestSet(): + dictionary = { + "testSet" : ["check-target-libstdc++-v3", "check-gcc-c++", "check-gcc-c", "check-target-libgomp", "check-target-libitm", "check-target-libatomic"] + } + dictionaryJson = json.dumps(dictionary) + print("::set-output name=testSet::" + dictionaryJson) + + # Runs the configure script to set up gcc configuration environment prior to building and running tests + # Creates the objdir directory as part of this process + @staticmethod + def Configure(): + logger = GetLogger() + res = RunScript(".github/scripts/configure-gcc.sh", False) + logger.info("output = " + str(res.stdout, 'utf-8')) + + @staticmethod + def MakeObjDir(): + logger = GetLogger() + res = subprocess.run(''' + cd .. + mkdir objdir + ''', + shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + + logger.info("output = " + str(res.stdout, 'utf-8')) + + @staticmethod + def Build(configJson): + logger = GetLogger() + logger.info("Gcc Build start") + Config.Reload(configJson) + + GccWorkflow.Configure() + GccWorkflow.MakeObjDir() + + # Build + res = RunScript(".github/scripts/build-gcc.sh", False) + logger.info("build output = " + str(res.stdout, 'utf-8')) + + # TODO: Add better handling of errors to display + if (res.returncode != 0): + logger.error("GCC Build failed") + logging.shutdown() + sys.exit(res.returncode) + + @staticmethod + def Test(configJson, testSet, accessToken, buildDownloadedStr): + logger = GetLogger() + logger.info("Gcc Test start") + + # Convert the input string (because that's how it's passed via the composite template) into a boolean + buildDownloaded = buildDownloadedStr == 'True' or buildDownloadedStr == 'true' + Config.Reload(configJson, accessToken) + + GccWorkflow.Configure() + + if (not buildDownloaded): + logger.info("Downloading gccBuild artifact from 'build' workflow...") + GccWorkflow.MakeObjDir() + try: + DownloadBuildArtifact() + except: + logger.error("Could not download build artifact") + RaiseWorkflowError("Error downloading build artifact for GCC workflow") + + # The gcc build should be in the objdir folder one level above + currentDir = os.getcwd() + assert os.path.isdir(currentDir + "/../objdir") + + # Copy over downloaded build artifact into objdir directory that was created in the configure script + # For more details on the subprocess function and its parameters, + # see https://stackoverflow.com/questions/4256107/running-bash-commands-in-python + res = RunScript(".github/scripts/test-gcc.sh", False, testSet) + logger.info("test output = " + str(res.stdout, 'utf-8')) + + # TODO: Add better handling of errors to display + if (res.returncode != 0): + globals.configObj.failedWorkflowError = "GCC Test failed" + globals.configObj.failedReturnCode = res.returncode + + Config.PrintNoSecretConfigJson() + + @staticmethod + def ParseTestLogs(configJson, accessToken): + logger = GetLogger() + logger.info("GCC Parsing Test Logs") + + Config.Reload(configJson, accessToken) + + # Load in the tests known to fail in our vendor branch + with open('contrib/testsuite-management/x86_64-pc-linux-gnu.xfail') as xFailFile: + linesFailFile = {line.rstrip('\n') for line in xFailFile} + + newFailuresToAdd = [] + + # Look for logs in the artifacts directory that were downloaded in the yaml via download-artifact + logDirs = glob.glob('artifacts/master_*_logs') + + for logDir in logDirs: + logger.info ("Parsing log directory: " + logDir) + + # Each directory should have a failures.txt file + assert os.path.exists(logDir + '/failures.txt') + + # Create a rawFailures.txt file with just the unexpected failure lines which are between the + # "Unexpected results in this build" and "Expected results not present in this build" markers in the file + res = subprocess.run('cd ' + logDir + + ''' + sed '0,/^Unexpected results in this build/d' failures.txt > afterUnexpected.txt + sed '/^Expected results not present in this build/q' afterUnexpected.txt > rawFailures.txt + sed -r '/^\s*$/d' -i rawFailures.txt + ''', + shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + logger.info("sed output = " + str(res.stdout, 'utf-8')) + + with open(logDir + '/rawFailures.txt') as rawFailuresFile: + failsInThisDir = {line.rstrip('\n') for line in rawFailuresFile} + newFails = failsInThisDir - linesFailFile + + for newFail in newFails: + newFailuresToAdd.append(newFail) + + if len(newFailuresToAdd) == 0: + logger.info("No new failures detected") + return + + # Sort the list of failures so that they're in a predictable order + newFailuresToAdd.sort() + + with open('contrib/testsuite-management/x86_64-pc-linux-gnu.xfail', 'a') as xFailFile: + logger.info("New failures found:") + for newFail in newFailuresToAdd: + logger.info(newFail) + xFailFile.write("\n") + xFailFile.write(newFail) + + # Create a patch file + res = subprocess.run(''' + git diff contrib/testsuite-management/x86_64-pc-linux-gnu.xfail > newFailures.patch + ''', + shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + logger.info("git diff output = " + str(res.stdout, 'utf-8')) + + res = subprocess.run(''' + git add contrib/testsuite-management/x86_64-pc-linux-gnu.xfail + git commit -m "Update xfail with new failures" + ''', + shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + logger.info("git add and commit output = " + str(res.stdout, 'utf-8')) diff --git a/.github/scripts/gcc_release b/.github/scripts/gcc_release new file mode 100644 index 00000000000..428b439c8f9 --- /dev/null +++ b/.github/scripts/gcc_release @@ -0,0 +1,799 @@ +#! /bin/sh + +######################################################################## +# +# File: gcc_release +# Author: Jeffrey Law, Bernd Schmidt, Mark Mitchell +# Date: 2001-05-25 +# +# Contents: +# Script to create a GCC release. +# +# Copyright (c) 2001-2020 Free Software Foundation. +# +# This file is part of GCC. +# +# GCC is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GCC is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# <http://www.gnu.org/licenses/>. +# +######################################################################## + +######################################################################## +# Notes +######################################################################## + +# Here is an example usage of this script, to create a GCC 3.0.2 +# prerelease: +# +# gcc_release -r 3.0.2 +# +# This script will automatically use the head of the release branch +# to generate the release. + +######################################################################## +# Functions +######################################################################## + +# Issue the error message given by $@ and exit with a non-zero +# exit code. + +error() { + echo "gcc_release: error: $@" + exit 1 +} + +# Issue the informational message given by $@. + +inform() { + echo "gcc_release: $@" +} + +# Issue a usage message explaining how to use this script. + +usage() { +cat <<EOF +gcc_release -r release [-f] [further options] +gcc_release -s name:gitbranch [further options] + +Options: + + -r release Version of the form X.Y or X.Y.Z. + -s name:gitbranch Create a snapshot, not a real release. + + -d destination Local working directory where we will build the release + (default=${HOME}). + -f Create a final release (and update ChangeLogs,...). + -l Indicate that we are running on gcc.gnu.org. + -p previous-tarball Location of a previous tarball (to generate diff files). + -t tag Tag to mark the release in git. + -u username Username for upload operations. + -b local-git-repo Local git repository to speed up cloning. + -c Indicate that the cloning should be done from the repo the script lives in. +EOF + exit 1 +} + +# Change to the directory given by $1. + +changedir() { + cd $1 || \ + error "Could not change directory to $1" +} + +# Build the source tree that will be the basis for the release +# in ${WORKING_DIRECTORY}/gcc-${RELEASE}. + +build_sources() { + # If the WORKING_DIRECTORY already exists, do not risk destroying it. + if [ -r ${WORKING_DIRECTORY} ]; then + error "\`${WORKING_DIRECTORY}' already exists" + fi + # Create the WORKING_DIRECTORY. + mkdir "${WORKING_DIRECTORY}" \ + || error "Could not create \`${WORKING_DIRECTORY}'" + changedir "${WORKING_DIRECTORY}" + + # Check out the sources. + if [ $GIT_LOCALPATH -eq 1 ]; then + git -C ${GCCPATH} clone ${GCCPATH} ${SOURCE_DIRECTORY} --single-branch --branch "${GITBRANCH}" + # Since the repo is private, we don't use this: + # ${GIT} clone -q -b "${GITBRANCH}" "${GITROOT}" "`basename ${SOURCE_DIRECTORY}`" || \ + # error "Could not check out release sources" + elif [ -n "${GIT_REFERENCE}" ]; then + ${GIT} clone -q --dissociate --reference "${GIT_REFERENCE}" \ + -b "${GITBRANCH}" "${GITROOT}" "`basename ${SOURCE_DIRECTORY}`" || \ + error "Could not check out release sources" + else + ${GIT} clone -q -b "${GITBRANCH}" "${GITROOT}" "`basename ${SOURCE_DIRECTORY}`" || \ + error "Could not check out release sources" + fi + + # If this is a final release and it is not custom repository, make sure that the ChangeLogs + # and version strings are updated. + if [ ${FINAL} -ne 0 ] && [ $GIT_LOCALPATH -eq 0 ]; then + inform "Updating ChangeLogs and version files" + + grep -q "gcc-${RELEASE_MAJOR}/index.html gcc-${RELEASE_MAJOR}/changes.html" \ + ${SOURCE_DIRECTORY}/contrib/gennews ||\ + error "New release not listed in contrib/gennews" + + ${SOURCE_DIRECTORY}/contrib/gennews > NEWS ||\ + error "Could not regenerate NEWS files" + + grep -q "no releases of GCC ${RELEASE_MAJOR} have yet been made" NEWS &&\ + error "gcc-${RELEASE_MAJOR}/index.html has not been updated yet" + + grep -q "GCC ${RELEASE_MAJOR} has not been released yet" NEWS &&\ + error "gcc-${RELEASE_MAJOR}/changes.html has not been updated yet" + + thisindex="http:\/\/gcc.gnu.org\/gcc-${RELEASE_MAJOR}\/index.html" + thischanges="http:\/\/gcc.gnu.org\/gcc-${RELEASE_MAJOR}\/changes.html" + previndex="http:\/\/gcc.gnu.org\/gcc-`expr ${RELEASE_MAJOR} - 1`\/index.html" + sed -n -e "/^${thisindex}/,/^${thischanges}/p" NEWS |\ + sed -n -e "/Release History/,/References and Acknowledgments/p" |\ + grep -q "^[[:blank:]]*GCC ${RELEASE_MAJOR}.${RELEASE_MINOR}" ||\ + error "GCC ${RELEASE_MAJOR}.${RELEASE_MINOR} not mentioned "\ + "in gcc-${RELEASE_MAJOR}/index.html" + + sed -n -e "/^${thischanges}/,/^${previndex}/p" NEWS |\ + grep -q "^[[:blank:]]*GCC ${RELEASE_MAJOR}.${RELEASE_MINOR}" ||\ + error "GCC ${RELEASE_MAJOR}.${RELEASE_MINOR} not mentioned "\ + "in gcc-${RELEASE_MAJOR}/changes.html" + + rm -f NEWS + + commit_files="" + for x in `changedir ${SOURCE_DIRECTORY} && \ + find . -name ChangeLog`; do + # Update this ChangeLog file only if it does not yet contain the + # entry we are going to add. (This is a safety net for repeated + # runs of this script for the same release.) + if ! grep "GCC ${RELEASE} released." ${SOURCE_DIRECTORY}/${x} > /dev/null ; then + cat - ${SOURCE_DIRECTORY}/${x} > ${SOURCE_DIRECTORY}/${x}.new <<EOF +${LONG_DATE} Release Manager + + * GCC ${RELEASE} released. + +EOF + mv ${SOURCE_DIRECTORY}/${x}.new ${SOURCE_DIRECTORY}/${x} \ + || error "Could not update ${x}" + commit_files="${commit_files} ${x}" + fi + done + + # Update gcc/DEV-PHASE. + + if [ `cat ${SOURCE_DIRECTORY}/gcc/BASE-VER` != ${RELEASE} ]; then + [ ${RELEASE_MAJOR} -lt 5 ] && \ + error "Release number ${RELEASE} does not match BASE-VER" + if [ `cat ${SOURCE_DIRECTORY}/gcc/BASE-VER` \ + = ${RELEASE_MAJOR}.`expr ${RELEASE_MINOR} - 1`.1 \ + -a x${RELEASE_REVISION} = x0 ]; then + (changedir ${SOURCE_DIRECTORY}/gcc && \ + echo ${RELEASE} > BASE-VER) || \ + error "Could not update BASE-VER" + commit_files="${commit_files} gcc/BASE-VER" + else + error "Release number ${RELEASE} does not immediately follow BASE-VER" + fi + fi + (changedir ${SOURCE_DIRECTORY}/gcc && \ + : > DEV-PHASE) || \ + error "Could not update DEV-PHASE" + commit_files="${commit_files} gcc/DEV-PHASE" + + (changedir ${SOURCE_DIRECTORY} && \ + ${GIT} commit -q -m 'Update ChangeLog and version files for release' ${commit_files} && \ + ${GIT} push) || \ + error "Could not commit ChangeLog and version file updates" + + # Make sure we tag the sources for a final release. + TAG="releases/gcc-${RELEASE}" + fi + + # Tag the sources. + if [ -n "${TAG}" ]; then + inform "Tagging sources as ${TAG}" + # We don't want to overwrite an existing tag. So, if the tag + # already exists, issue an error message; the release manager can + # manually remove the tag if appropriate. + if (changedir ${SOURCE_DIRECTORY} && \ + ${GIT} rev-parse "refs/tags/${TAG}" > /dev/null 2>&1); then + error "Tag ${TAG} already exists" + fi + (changedir ${SOURCE_DIRECTORY} && \ + ${GIT} tag -s -m "GCC ${RELEASE} release" "${TAG}" && \ + ${GIT} push origin tag "${TAG}") || \ + error "Could not tag sources" + GITBRANCH=${TAG} + fi + + GITREV=`cd ${SOURCE_DIRECTORY} && ${GIT} rev-parse HEAD` + inform "Sources are commit ${GITREV}" + + # Make sure there are no uncommitted changes in the sources. + status=${WORKING_DIRECTORY}/gitstatus.$$ + (changedir ${SOURCE_DIRECTORY} && \ + ${GIT} status --porcelain --ignored > "$status") || \ + error "Could not get source directory status" + if [ -s "$status" ]; then + cat "$status" + error "Source directory has unexpected changes" + fi + rm "$status" + + # Remove .git from the sources. + rm -rf "${SOURCE_DIRECTORY}/.git" || \ + error "Could not remove .git from sources" + + # Run gcc_update on them to set up the timestamps nicely, and (re)write + # the LAST_UPDATED file containing the git tag/revision used. + changedir "gcc-${RELEASE}" + contrib/gcc_update --touch + echo "Obtained from git: ${GITBRANCH} revision ${GITREV}" > LAST_UPDATED + + # For a prerelease or real release, we need to generate additional + # files not present in git. + changedir "${SOURCE_DIRECTORY}" + if [ $SNAPSHOT -ne 1 ]; then + # Generate the documentation. + inform "Building install docs" + SOURCEDIR=${SOURCE_DIRECTORY}/gcc/doc + DESTDIR=${SOURCE_DIRECTORY}/INSTALL + export SOURCEDIR + export DESTDIR + ${SOURCE_DIRECTORY}/gcc/doc/install.texi2html + + # Regenerate the NEWS file. + contrib/gennews > NEWS || \ + error "Could not regenerate NEWS files" + + # Now, we must build the compiler in order to create any generated + # files that are supposed to go in the source directory. This is + # also a good sanity check to make sure that the release builds + # on at least one platform. + inform "Building compiler" + OBJECT_DIRECTORY=../objdir + num_cpus=1 + if type -p getconf 2>/dev/null; then + num_cpus=`getconf _NPROCESSORS_ONLN 2>/dev/null` + case "$num_cpus" in + '' | 0* | *[!0-9]*) num_cpus=1;; + esac + fi + contrib/gcc_build -d ${SOURCE_DIRECTORY} -o ${OBJECT_DIRECTORY} \ + -c "--enable-generated-files-in-srcdir --disable-multilib" \ + -m "-j$num_cpus" build || \ + error "Could not rebuild GCC" + fi + + # Move message catalogs to source directory. + mv ../objdir/gcc/po/*.gmo gcc/po/ + [ -f libcpp/po/cpplib.pot ] && mv ../objdir/libcpp/po/*.gmo libcpp/po/ + + # Create a "MD5SUMS" file to use for checking the validity of the release. + echo \ +"# This file contains the MD5 checksums of the files in the +# gcc-"${RELEASE}".tar.xz tarball. +# +# Besides verifying that all files in the tarball were correctly expanded, +# it also can be used to determine if any files have changed since the +# tarball was expanded or to verify that a patchfile was correctly applied. +# +# Suggested usage: +# md5sum -c MD5SUMS | grep -v \"OK$\" +#" > MD5SUMS + + find . -type f | + sed -e 's:^\./::' -e '/MD5SUMS/d' | + sort | + xargs md5sum >>MD5SUMS +} + +# Build a single tarfile. The first argument is the name of the tarfile +# to build, without any suffixes. They will be added automatically. The +# rest of the arguments are files or directories to include, and possibly +# other arguments to tar. + +build_tarfile() { + # Get the name of the destination tar file. + TARFILE="$1.tar.xz" + shift + + # Build the tar file itself. + (${TAR} cf - "$@" | ${XZ} > ${TARFILE}) || \ + error "Could not build tarfile" + FILE_LIST="${FILE_LIST} ${TARFILE}" +} + +# Build the various tar files for the release. + +build_tarfiles() { + inform "Building tarfiles" + + changedir "${WORKING_DIRECTORY}" + + # The GNU Coding Standards specify that all files should + # world readable. + chmod -R a+r ${SOURCE_DIRECTORY} + # And that all directories have mode 755. + find ${SOURCE_DIRECTORY} -type d -exec chmod 755 {} \; + + # Build one huge tarfile for the entire distribution. + build_tarfile gcc-${RELEASE} `basename ${SOURCE_DIRECTORY}` +} + +# Build .gz files. +build_gzip() { + for f in ${FILE_LIST}; do + target=${f%.xz}.gz + (${XZ} -d -c $f | ${GZIP} > ${target}) || error "Could not create ${target}" + done +} + +# Build diffs against an old release. +build_diffs() { + old_dir=${1%/*} + old_file=${1##*/} + case "$old_file" in + *.tar.xz) old_vers=${old_file%.tar.xz};; + *) old_vers=${old_file%.tar.bz2};; + esac + old_vers=${old_vers#gcc-} + inform "Building diffs against version $old_vers" + for f in gcc; do + if [ -e ${old_dir}/${f}-${old_vers}.tar.xz ]; then + old_tar=${old_dir}/${f}-${old_vers}.tar.xz + else + old_tar=${old_dir}/${f}-${old_vers}.tar.bz2 + fi + new_tar=${WORKING_DIRECTORY}/${f}-${RELEASE}.tar.xz + if [ ! -e $old_tar ]; then + inform "$old_tar not found; not generating diff file" + elif [ ! -e $new_tar ]; then + inform "$new_tar not found; not generating diff file" + else + build_diff $old_tar gcc-${old_vers} $new_tar gcc-${RELEASE} \ + ${f}-${old_vers}-${RELEASE}.diff.xz + fi + done +} + +# Build an individual diff. +build_diff() { + changedir "${WORKING_DIRECTORY}" + tmpdir=gccdiff.$$ + mkdir $tmpdir || error "Could not create directory $tmpdir" + changedir $tmpdir + case "$1" in + *.tar.bz2) + (${BZIP2} -d -c $1 | ${TAR} xf - ) || error "Could not unpack $1 for diffs" + ;; + *.tar.xz) + (${XZ} -d -c $1 | ${TAR} xf - ) || error "Could not unpack $1 for diffs" + ;; + esac + (${XZ} -d -c $3 | ${TAR} xf - ) || error "Could not unpack $3 for diffs" + ${DIFF} $2 $4 > ../${5%.xz} + if [ $? -eq 2 ]; then + error "Trouble making diffs from $1 to $3" + fi + ${XZ} ../${5%.xz} || error "Could not generate ../$5" + changedir .. + rm -rf $tmpdir + FILE_LIST="${FILE_LIST} $5" +} + +# Upload the files to the FTP server. +upload_files() { + inform "Uploading files" + + changedir "${WORKING_DIRECTORY}" + + # Make sure the directory exists on the server. + if [ $LOCAL -eq 0 ]; then + ${SSH} -l ${GCC_USERNAME} ${GCC_HOSTNAME} \ + mkdir -m 755 -p "${FTP_PATH}/diffs" + UPLOAD_PATH="${GCC_USERNAME}@${GCC_HOSTNAME}:${FTP_PATH}" + else + mkdir -p "${FTP_PATH}/diffs" \ + || error "Could not create \`${FTP_PATH}'" + UPLOAD_PATH=${FTP_PATH} + fi + + # Then copy files to their respective (sub)directories. + for x in gcc*.gz gcc*.xz; do + if [ -e ${x} ]; then + # Make sure the file will be readable on the server. + chmod a+r ${x} + # Copy it. + case ${x} in + *.diff.*) + SUBDIR="diffs/"; + ;; + *) + SUBDIR=""; + esac + ${SCP} ${x} ${UPLOAD_PATH}/${SUBDIR} \ + || error "Could not upload ${x}" + fi + done +} + +# Print description if snapshot exists. +snapshot_print() { + if [ -e ${RELEASE}/$1 ]; then + hash=`openssl sha256 ${RELEASE}/$1 | sed -e 's#(.*)##' -e 's# *= *#=#'` + hash2=`openssl sha1 ${RELEASE}/$1 | sed -e 's#(.*)##' -e 's# *= *#=#'` + + printf " %-37s%s\n\n %s\n %s\n\n" "$1" "$2" "$hash" "$hash2" \ + >> ${SNAPSHOT_README} + + echo " <tr><td><a href=\"$1\">$1</a></td>" >> ${SNAPSHOT_INDEX} + echo " <td>$2</td></tr>" >> ${SNAPSHOT_INDEX} + fi +} + +# Announce a snapshot, both on the web and via mail. +announce_snapshot() { + inform "Updating links and READMEs on the FTP server" + + TEXT_DATE=`date --date=$DATE +%B\ %d,\ %Y` + SNAPSHOT_README=${RELEASE}/README + SNAPSHOT_INDEX=${RELEASE}/index.html + + changedir "${SNAPSHOTS_DIR}" + echo \ +"Snapshot gcc-"${RELEASE}" is now available on + https://gcc.gnu.org/pub/gcc/snapshots/"${RELEASE}"/ +and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. + +This snapshot has been generated from the GCC "${BRANCH}" git branch +with the following options: "git://gcc.gnu.org/git/gcc.git branch ${GITBRANCH} revision ${GITREV}" + +You'll find: +" > ${SNAPSHOT_README} + + echo \ +"<html> + +<head> +<title>GCC "${RELEASE}" Snapshot</title> +</head> + +<body> +<h1>GCC "${RELEASE}" Snapshot</h1> + +<p>The <a href =\"http://gcc.gnu.org/\">GCC Project</a> makes +periodic snapshots of the GCC source tree available to the public +for testing purposes.</p> + +<p>If you are planning to download and use one of our snapshots, then +we highly recommend you join the GCC developers list. Details for +how to sign up can be found on the GCC project home page.</p> + +<p>This snapshot has been generated from the GCC "${BRANCH}" git branch +with the following options: <code>"git://gcc.gnu.org/git/gcc.git branch ${GITBRANCH} revision ${GITREV}"</code></p> + +<table>" > ${SNAPSHOT_INDEX} + + snapshot_print gcc-${RELEASE}.tar.xz "Complete GCC" + + echo \ +"Diffs from "${BRANCH}"-"${LAST_DATE}" are available in the diffs/ subdirectory. + +When a particular snapshot is ready for public consumption the LATEST-"${BRANCH}" +link is updated and a message is sent to the gcc list. Please do not use +a snapshot before it has been announced that way." >> ${SNAPSHOT_README} + + echo \ +"</table> +<p>Diffs from "${BRANCH}"-"${LAST_DATE}" are available in the +<a href=\"diffs/\">diffs/ subdirectory</a>.</p> + +<p>When a particular snapshot is ready for public consumption the LATEST-"${BRANCH}" +link is updated and a message is sent to the gcc list. Please do not use +a snapshot before it has been announced that way.</p> + +<hr /> + +<address> +<a href=\"mailto:gcc@gcc.gnu.org\">gcc@gcc.gnu.org</a> +<br /> +Last modified "${TEXT_DATE}" +</address> +</body> + +</html>" >> ${SNAPSHOT_INDEX} + + rm -f LATEST-${BRANCH} + ln -s ${RELEASE} LATEST-${BRANCH} + + inform "Sending mail" + + export QMAILHOST=gcc.gnu.org + mail -s "gcc-${RELEASE} is now available" gcc@gcc.gnu.org < ${SNAPSHOT_README} +} + +######################################################################## +# Initialization +######################################################################## + +LC_ALL=C +export LC_ALL + +# Today's date. +DATE=`date "+%Y%m%d"` +LONG_DATE=`date "+%Y-%m-%d"` + +GIT=${GIT:-git} +# The server containing the GCC repository. +GIT_SERVER="gcc.gnu.org" +# The path to the repository on that server. +GIT_REPOSITORY="/git/gcc.git" +# The username to use when connecting to the server. +GIT_USERNAME="${USER}" +# Flag to specify use of local git repository. +GIT_LOCALPATH=0 +# The machine to which files will be uploaded. +GCC_HOSTNAME="gcc.gnu.org" +# The name of the account on the machine to which files are uploaded. +GCC_USERNAME="gccadmin" +# The directory in which the files will be placed (do not use ~user syntax). +FTP_PATH=/var/ftp/pub/gcc +# The directory in which snapshots will be placed. +SNAPSHOTS_DIR=${FTP_PATH}/snapshots + +# The major number for the release. For release `3.0.2' this would be +# `3' +RELEASE_MAJOR="" +# The minor number for the release. For release `3.0.2' this would be +# `0'. +RELEASE_MINOR="" +# The revision number for the release. For release `3.0.2' this would +# be `2'. +RELEASE_REVISION="" +# The complete name of the release. +RELEASE="" + +# The name of the branch from which the release should be made, in a +# user-friendly form. +BRANCH="" + +# The name of the branch from which the release should be made, as used +# for our version control system. +GITBRANCH="" + +# The tag to apply to the sources used for the release. +TAG="" + +# The old tarballs from which to generate diffs. +OLD_TARS="" + +# Local gcc git checkout to speed up git cloning. +GIT_REFERENCE="" + +# The directory that will be used to construct the release. The +# release itself will be placed in a subdirectory of this directory. +DESTINATION=${HOME} +# The subdirectory. +WORKING_DIRECTORY="" +# The directory that will contain the GCC sources. +SOURCE_DIRECTORY="" +# PATH To GCC Repository containing this script while running +GCCPATH=$(cd "$(dirname "$0")" && pwd)/../../ + +# Non-zero if this is the final release, rather than a prerelease. +FINAL=0 + +# Non-zero if we are building a snapshot, and don't build gcc or +# include generated files. +SNAPSHOT=0 + +# Non-zero if we are running locally on gcc.gnu.org, and use local CVS +# and copy directly to the FTP directory. +LOCAL=0 + +# Major operation modes. +MODE_GZIP=0 +MODE_DIFFS=0 +MODE_SOURCES=0 +MODE_TARFILES=0 +MODE_UPLOAD=0 + +# List of archive files generated; used to create .gz files from .xz. +FILE_LIST="" + +# Programs we use. + +BZIP2="${BZIP2:-bzip2}" +XZ="${XZ:-xz --best}" +CVS="${CVS:-cvs -f -Q -z9}" +DIFF="${DIFF:-diff -Nrcpad}" +ENV="${ENV:-env}" +GZIP="${GZIP:-gzip --best}" +SCP="${SCP:-scp -p}" +SSH="${SSH:-ssh}" +TAR="${TAR:-tar}" + +######################################################################## +# Command Line Processing +######################################################################## + +# Parse the options. +while getopts "d:fr:u:t:p:s:lb:c" ARG; do + case $ARG in + d) DESTINATION="${OPTARG}";; + r) RELEASE="${OPTARG}";; + t) TAG="${OPTARG}";; + u) GIT_USERNAME="${OPTARG}";; + f) FINAL=1;; + s) SNAPSHOT=1 + BRANCH=${OPTARG%:*} + GITBRANCH=${OPTARG#*:} + ;; + l) LOCAL=1 + SCP=cp + PATH=~:/usr/local/bin:$PATH;; + p) OLD_TARS="${OLD_TARS} ${OPTARG}" + if [ ! -f ${OPTARG} ]; then + error "-p argument must name a tarball" + fi;; + b) GIT_REFERENCE="${OPTARG}";; + c) GIT_LOCALPATH=1;; + \?) usage;; + esac +done +shift `expr ${OPTIND} - 1` + +# Handle the major modes. +while [ $# -ne 0 ]; do + case $1 in + diffs) MODE_DIFFS=1;; + gzip) MODE_GZIP=1;; + sources) MODE_SOURCES=1;; + tarfiles) MODE_TARFILES=1;; + upload) MODE_UPLOAD=1;; + all) MODE_SOURCES=1; MODE_TARFILES=1; MODE_DIFFS=1; MODE_UPLOAD=1; + if [ $SNAPSHOT -ne 1 ]; then + # Only for releases and pre-releases. + MODE_GZIP=1; + fi + ;; + *) error "Unknown mode $1";; + esac + shift +done + +# Perform consistency checking. +if [ ${LOCAL} -eq 0 ] && [ -z ${GIT_USERNAME} ]; then + error "No username specified" +fi + +if [ ! -d ${DESTINATION} ]; then + error "\`${DESTINATION}' is not a directory" +fi + +if [ $SNAPSHOT -eq 0 ]; then + if [ -z ${RELEASE} ]; then + error "No release number specified" + fi + + # Compute the major and minor release numbers. + RELEASE_MAJOR=`echo $RELEASE | awk --assign FS=. '{ print $1; }'` + RELEASE_MINOR=`echo $RELEASE | awk --assign FS=. '{ print $2; }'` + RELEASE_REVISION=`echo $RELEASE | awk --assign FS=. '{ print $3; }'` + + if [ -z "${RELEASE_MAJOR}" ] || [ -z "${RELEASE_MINOR}" ]; then + error "Release number \`${RELEASE}' is invalid" + fi + + # Compute the full name of the release. + if [ -z "${RELEASE_REVISION}" ]; then + RELEASE="${RELEASE_MAJOR}.${RELEASE_MINOR}" + else + RELEASE="${RELEASE_MAJOR}.${RELEASE_MINOR}.${RELEASE_REVISION}" + fi + + # Compute the name of the branch, which is based solely on the major + # release number. + GITBRANCH="releases/gcc-${RELEASE_MAJOR}" + + # If this is not a final release, set various parameters accordingly. + if [ ${FINAL} -ne 1 ]; then + RELEASE="${RELEASE}-RC-${DATE}" + FTP_PATH="${SNAPSHOTS_DIR}/${RELEASE}" + else + FTP_PATH="${FTP_PATH}/releases/gcc-${RELEASE}/" + fi +else + RELEASE=${BRANCH}-${DATE} + FTP_PATH="${FTP_PATH}/snapshots/${RELEASE}" + + # If diffs are requested when building locally on gcc.gnu.org, we (usually) + # know what the last snapshot date was and take the corresponding tarballs, + # unless the user specified tarballs explicitly. + if [ $MODE_DIFFS -ne 0 ] && [ $LOCAL -ne 0 ] && [ -z "${OLD_TARS}" ]; then + LAST_DATE=`cat ~/.snapshot_date-${BRANCH}` + OLD_TARS=${SNAPSHOTS_DIR}/${BRANCH}-${LAST_DATE}/gcc-${BRANCH}-${LAST_DATE}.tar.bz2 + if [ ! -e $OLD_TARS ]; then + OLD_TARS=${SNAPSHOTS_DIR}/${BRANCH}-${LAST_DATE}/gcc-${BRANCH}-${LAST_DATE}.tar.xz + fi + fi +fi + +# Compute the name of the WORKING_DIRECTORY and the SOURCE_DIRECTORY. +WORKING_DIRECTORY="${DESTINATION}/gcc-${RELEASE}" +SOURCE_DIRECTORY="${WORKING_DIRECTORY}/gcc-${RELEASE}" + +# Set up GITROOT. +if [ $LOCAL -eq 0 ]; then + GITROOT="git+ssh://${GIT_USERNAME}@${GIT_SERVER}${GIT_REPOSITORY}" +else + GITROOT="/git/gcc.git" +fi +export GITROOT + +######################################################################## +# Main Program +######################################################################## + +# Set the timezone to UTC +TZ="UTC0" +export TZ + +# Build the source directory. + +if [ $MODE_SOURCES -ne 0 ]; then + build_sources +fi + +# Build the tar files. + +if [ $MODE_TARFILES -ne 0 ]; then + build_tarfiles +fi + +# Build diffs + +if [ $MODE_DIFFS -ne 0 ]; then + # Possibly build diffs. + if [ -n "$OLD_TARS" ]; then + for old_tar in $OLD_TARS; do + build_diffs $old_tar + done + fi +fi + +# Build gzip files +if [ $MODE_GZIP -ne 0 ]; then + build_gzip +fi + +# Upload them to the FTP server. +if [ $MODE_UPLOAD -ne 0 ]; then + upload_files + + # For snapshots, make some further updates. + if [ $SNAPSHOT -ne 0 ] && [ $LOCAL -ne 0 ]; then + announce_snapshot + + # Update snapshot date file. + changedir ~ + echo $DATE > .snapshot_date-${BRANCH} + + # Remove working directory + rm -rf ${WORKING_DIRECTORY} + fi +fi diff --git a/.github/scripts/globals.py b/.github/scripts/globals.py new file mode 100644 index 00000000000..a46aa997184 --- /dev/null +++ b/.github/scripts/globals.py @@ -0,0 +1,29 @@ + +# Declare config global variable +# https://docs.python.org/3/faq/programming.html#how-do-i-share-global-variables-across-modules + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +configObj = None + +logger = None \ No newline at end of file diff --git a/.github/scripts/init.sh b/.github/scripts/init.sh new file mode 100644 index 00000000000..4a218b23f91 --- /dev/null +++ b/.github/scripts/init.sh @@ -0,0 +1,39 @@ +# Script to run basic initialization for developers and GitHub workflows + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +# Setup upstream gcc remote +git ls-remote --exit-code --quiet gcc > /dev/null 2>&1 +if test $? != 0; then + echo "Adding gcc upstream remote" + git remote add gcc git://gcc.gnu.org/git/gcc.git +else + # Check if the URL ends with gcc.gnu.org. We don't check for a string explicitly + # since some devs may use ssh or other means to connect to the remote + gccUrl="$(git config remote.gcc.url)" + + if [[ "$gccUrl" != *"gcc.gnu.org/git/gcc.git" ]]; then + echo "ERROR: gcc upstream remote was found but it's not pointing to git://gcc.gnu.org/git/gcc.git. Please delete the local gcc upstream remote before running init again" + exit -1 + fi +fi \ No newline at end of file diff --git a/.github/scripts/pushMergedVendorBranch.sh b/.github/scripts/pushMergedVendorBranch.sh new file mode 100644 index 00000000000..252dc6f6963 --- /dev/null +++ b/.github/scripts/pushMergedVendorBranch.sh @@ -0,0 +1,56 @@ +# This script pushes the merged branch +# $1 is the temporary local branch to create +# $2 is the merge branch produced by the update-main.yaml workflow + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +BASEDIR=$(dirname $0) + +chmod +x ${BASEDIR}/init.sh +${BASEDIR}/init.sh +if test $? != 0; then + echo "ERROR: Initialization failed" + exit -1 +fi + +if [[ `git status --porcelain` ]]; then + echo "ERROR: You have local changes in your repository. Please stash or push them to a branch before running this script" + exit -1 +fi + +echo " +---- READ THIS ---- +When prompted for the 'local name for upstream repository', +enter 'gcc'. The other fields can be the default +---- END OF READ THIS ---- +" + +${BASEDIR}/../../contrib/gcc-git-customization.sh +${BASEDIR}/../../contrib/git-fetch-vendor.sh --enable-push microsoft +git fetch +git fetch gcc +git fetch origin +git checkout -B microsoft/main vendors/microsoft/main +git merge origin/$1 +git push vendors/microsoft microsoft/main + diff --git a/.github/scripts/test-gcc.sh b/.github/scripts/test-gcc.sh new file mode 100644 index 00000000000..7c43e6f18e3 --- /dev/null +++ b/.github/scripts/test-gcc.sh @@ -0,0 +1,45 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +cd ../objdir + +chmod +x $PWD -R +mkdir logs + +# Run tests +make -k $1 -j$(nproc) RUNTESTFLAGS="-v -v" 2>&1 >> logs/output.txt 2>&1 + +# Compare failures found to the failures that we're expecting +../gcc/contrib/testsuite-management/validate_failures.py >> logs/failures.txt 2>&1 +exit_code=$? + +# Print failures to console output so they can be viewed in the GitHub Action run +cat logs/failures.txt + +if [ $exit_code != 0 ]; then + exit $exit_code +fi + +# To run one test in particular, use +#make check-gcc RUNTESTFLAGS="-v -v dg.exp=c-c++-common/asan/asan-interface-1.c" + + diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml new file mode 100644 index 00000000000..d3e8b9cf980 --- /dev/null +++ b/.github/workflows/build.yaml @@ -0,0 +1,88 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +# This needs to match the string returned in GetGccBuildName() in common.py +name: build + +# Run this workflow on every new commit and pull request +on: + push: + branches: + - current + - 'releases/**' + - gcc-9.1.0 + pull_request: + branches: + - current + - 'releases/**' + - gcc-9.1.0 + workflow_dispatch: + +jobs: + build: + runs-on: ubuntu-18.04 + steps: + - name: checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: python -m pip install requests + shell: bash + + # Setup config + - name: Setup config + id: setupconfig + run: | + chmod +x .github/scripts/gccWorkflow.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "${GITHUB_CONTEXT}" + python -c 'import sys; from gccWorkflow import *; GccWorkflow.Init(sys.argv[1], sys.argv[2], True)' "${GITHUB_CONTEXT}" "${GITHUB_TOKEN}" + shell: bash + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Build GCC + uses: ./.github/actions/build-composite + with: + configjson: ${{ steps.setupconfig.outputs.configJson }} + + - name: Compress and archive objdir so that it can be uploaded + run: tar -czf objdir.tar.gz ../objdir + + - name: Upload build output + uses: actions/upload-artifact@v2 + with: + name: gccBuild + path: objdir.tar.gz diff --git a/.github/workflows/delete-merge-branches.yaml b/.github/workflows/delete-merge-branches.yaml new file mode 100644 index 00000000000..3e0930d62d6 --- /dev/null +++ b/.github/workflows/delete-merge-branches.yaml @@ -0,0 +1,52 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +# This workflow deletes the stale merge branches created by update-main.yaml +# Stale is defined by the branch being 14+ days old. Keeping the branch around for a little +# while can help us investigate failures. + +on: + # Run once a day at 5 am UTC + schedule: + - cron: "0 5 * * *" + +jobs: + delete: + runs-on: ubuntu-18.04 + outputs: + config: ${{ steps.printNoSecretJson.outputs.noSecretConfigJson }} + matrixTestSet: ${{ steps.printGCCTestSet.outputs.testSet }} + steps: + - name: checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Delete stale branches + run: | + chmod +x .github/scripts/delete-branches.sh + .github/scripts/delete-branches.sh "^merge-master-.*" 14 + shell: bash + + diff --git a/.github/workflows/package-rpm.yaml b/.github/workflows/package-rpm.yaml new file mode 100644 index 00000000000..1e667eb4b2c --- /dev/null +++ b/.github/workflows/package-rpm.yaml @@ -0,0 +1,95 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +name: package-rpm + +# Disable for now since packaging needs to run after the build. +# Run this workflow on every new commit +# For the time being, we are only running this workflow manually. +on: + # We use the push and pull_request events instead of the workflow_run event + # so that this workflow will show up during the PR + push: + branches: + - "releases/**" + pull_request: + branches: + - "releases/**" + workflow_dispatch: + +jobs: + package: + container: cblmariner2preview.azurecr.io/base/core:2.0 + runs-on: ubuntu-18.04 + steps: + + # We need to install a minimum number of packages to run the workflow: git and ca-certificates. + # We need to checkout the repository, and to checkout the repository we need git, since + # by default CBL-Mariner 2.0 image does not have git running. + - name: Install Minimal Required Packages + run: unset HOME && tdnf install git ca-certificates -y + shell: bash + + # Clone the GCC repository to path gcc. + - name: Checkout GCC Repository + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: false + path: gcc + + # Run composite job to generate gcc tar.xz files. + - name: Run Composite Job To Build Tarball + uses: ./gcc/.github/actions/build-tarball + + # Before running a complete build, check whether we have the .spec file to build for this version. + # If this fails, it means we don't have .spec file for ${{ env.release_version }} version. + # The file should be present in .github/Mariner/specs/${{ env.release_version }} directory inside gcc source tree. + - name: Check the SPEC File Does Exist in GCC Source Tree + working-directory: ./gcc + run: test -f .github/Mariner/specs/${{ env.release_version }}/gcc.specs + shell: bash + + # Copy gcc-${{ env.release_version }}.tar.xz to /usr/src/mariner/SOURCES/ + # /usr/src/mariner/SOURCES/ is a hardcoded path for rpmbuild in CBL-Mariner + # to look for tar.xz files when starts building packages. + - name: Copy GCC + run: mkdir -p /usr/src/mariner/SOURCES/ && cp ~/gcc-${{ env.release_version }}/gcc-${{ env.release_version }}.tar.xz /usr/src/mariner/SOURCES/ + shell: bash + + # Build RPM + # Simple rpmbuild command which will build GCC based on the spec file available in our repo. + # The flags are bare minimum, and we are forcing rpmbuild to build everything. + - name: Build RPM + working-directory: ./gcc + run: rpmbuild -bb --rebuild --clean --verbose --nocheck .github/Mariner/specs/11.2.1/gcc.specs + shell: bash + + # Upload gcc-${{ env.release_version }}-rpms folder to the Github Action artifact storage. + # The variable ${{ env.release_version }} will be set by ./gcc/.github/actions/build-tarball job. + - name: Upload GCC RPM + uses: actions/upload-artifact@v2 + with: + name: gcc-${{ env.release_version }}-rpms + path: /usr/src/mariner/RPMS/x86_64/ + if-no-files-found: error # 'warn' or 'ignore' are also available, defaults to `warn` diff --git a/.github/workflows/package-source-tarball.yaml b/.github/workflows/package-source-tarball.yaml new file mode 100644 index 00000000000..cad53faff3e --- /dev/null +++ b/.github/workflows/package-source-tarball.yaml @@ -0,0 +1,72 @@ +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +name: package-source-tarball + +# Disable for now since packaging needs to run after the build. +# Run this workflow on every new commit +# For the time being, we are only running this workflow manually. +on: + # We use the push and pull_request events instead of the workflow_run event + # so that this workflow will show up during the PR + push: + branches: + - "releases/**" + pull_request: + branches: + - "releases/**" + workflow_dispatch: + +jobs: + package: + container: cblmariner2preview.azurecr.io/base/core:2.0 + runs-on: ubuntu-18.04 + steps: + + # We need to install a minimum number of packages to run the workflow: git and ca-certificates. + # We need to checkout the repository, and to checkout the repository we need git, since + # by default CBL-Mariner 2.0 image does not have git running. + - name: Install Minimal Required Packages + run: unset HOME && tdnf install git ca-certificates -y + shell: bash + + # Clone the GCC repository to path gcc. + - name: Checkout GCC Repository + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: false + path: gcc + + # Run composite job to generate gcc tar.xz files. + - name: Run Composite Job To Build Tarball + uses: ./gcc/.github/actions/build-tarball + + # Upload gcc-${{ env.release_version }}.tar.xz file to the Github Action artifact storage. + # The variable ${{ env.release_version }} will be set by ./gcc/.github/actions/build-tarball job. + - name: Upload GCC tar.xz + uses: actions/upload-artifact@v2 + with: + name: gcc-${{ env.release_version }}.tar.xz + path: ~/gcc-${{ env.release_version }}/gcc-${{ env.release_version }}.tar.xz + if-no-files-found: error # 'warn' or 'ignore' are also available, defaults to `warn` diff --git a/.github/workflows/test-gcc.yaml b/.github/workflows/test-gcc.yaml new file mode 100644 index 00000000000..63b85b79636 --- /dev/null +++ b/.github/workflows/test-gcc.yaml @@ -0,0 +1,161 @@ +# This workflow runs the GCC tests in parallel + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +name: test-gcc + +# Run this workflow when a build run finishes on the branches listed +on: + # We use the push and pull_request events instead of the workflow_run event + # so that this workflow will show up during the PR + push: + branches: + - current + - 'releases/**' + - gcc-9.1.0 + pull_request: + branches: + - current + - 'releases/**' + - gcc-9.1.0 + # the problem with workflow_run is that the run isn't associated with a PR and won't show up in a PR + #workflow_run: + # workflows: ["build"] + # types: + # - completed + workflow_dispatch: + +jobs: + init: + runs-on: ubuntu-18.04 + outputs: + config: ${{ steps.printNoSecretJson.outputs.noSecretConfigJson }} + matrixTestSet: ${{ steps.printGCCTestSet.outputs.testSet }} + steps: + - name: checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: | + python -m pip install requests + shell: bash + + # Print the chunked set of GCC tests we'd like to run in a matrix later + - name: Print GCC test set + id: printGCCTestSet + uses: ./.github/actions/print-gcc-testset + + # Setup config + - name: Setup config + id: setupconfig + run: | + chmod +x .github/scripts/gccWorkflow.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'import sys; from gccWorkflow import *; GccWorkflow.Init(sys.argv[1], sys.argv[2], False)' "${GITHUB_CONTEXT}" "${GITHUB_TOKEN}" + shell: bash + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + # This should be the last step on this machine since it will clear out fields in the config json + - name: Print No Secret Json + id: printNoSecretJson + run: | + chmod +x .github/scripts/config.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'import sys; from config import *; Config.PrintNoSecretConfigJsonFromJson(sys.argv[1])' "${CONFIG_JSON}" + shell: bash + env: + CONFIG_JSON: ${{ steps.setupconfig.outputs.configJson }} + test: + needs: init + strategy: + matrix: ${{fromJSON(needs.init.outputs.matrixTestSet)}} + # Avoid cancelling other matrix chunks even if one fails + fail-fast: false + runs-on: ubuntu-18.04 + steps: + - run: echo ${{needs.init.outputs.output1}} ${{needs.init.outputs.output2}} + - run: echo $GITHUB_CONTEXT + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + - name: checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: python -m pip install requests + shell: bash + + # build.yaml creates the gccBuild artifact + - name: Run GCC tests + uses: ./.github/actions/test-composite + id: test-composite + with: + configjson: ${{needs.init.outputs.config}} + testSet: ${{ matrix.testSet }} + githubtoken: ${{ secrets.GITHUB_TOKEN }} + buildDownloaded: False + + - name: Move objdir to be in repo so logs inside can be uploaded + run: mv ../objdir objdir + + - name: Upload build output + uses: actions/upload-artifact@v2 + with: + name: ${{ matrix.testSet }}_logs + path: objdir/logs + + # We have to do this after because there's no REST API to upload build artifacts so we need to use the yaml one + - name: Fail workflow if there was a failure earlier + run: | + chmod +x .github/scripts/common.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "${GITHUB_CONTEXT}" + python -c 'import sys; from gccWorkflow import *; Config.Reload(sys.argv[1], sys.argv[2]); Config.RaiseErrorIfWorkflowFailed()' "${GITHUB_CONTEXT}" "${GITHUB_TOKEN}" + shell: bash + env: + GITHUB_CONTEXT: ${{steps.test-composite.outputs.configJson}} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + diff --git a/.github/workflows/update-main.yaml b/.github/workflows/update-main.yaml new file mode 100644 index 00000000000..8168f4302fd --- /dev/null +++ b/.github/workflows/update-main.yaml @@ -0,0 +1,395 @@ +# This workflow does the following things: +# 1. Takes the latest SHA from the upstream GCC master branch and starts a build and GCC test run +# 2. Any new failures encountered during the run and not in the .xfail file in vendors/microsoft/main will get added to a the .xfail file. +# 3. A new branch will be created in the GitHub repository with the gcc/master SHA we used earlier merged into vendors/microsoft/main and the newly added .xfail failures (if any) +# 4. A build and test-gcc workflow will then get kicked off against this new branch +# 5. This workflow will wait until the build and test-gcc workflows are done. +# 6. If the build and test-gcc steps are successful, it will print the set of Git commands for someone to run manually to actually push the changes to our vendor branch + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + + +name: update-main + +# Run this workflow every day at 3 am UTC +on: + schedule: + - cron: "0 3 * * *" + workflow_dispatch: + +jobs: + gcc-master-build: + name: Build the latest gcc/master toolchain + runs-on: ubuntu-18.04 + outputs: + config: ${{ steps.printNoSecretJson.outputs.noSecretConfigJson }} + masterSHA: ${{ steps.printCurrentSHA.outputs.currentSHA }} + matrixTestSet: ${{ steps.printGCCTestSet.outputs.testSet }} + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: python -m pip install requests + shell: bash + + # Add upstream gcc as remote and checkout from their master branch + - name: Add upstream remote and checkout upstream gcc master branch + uses: ./.github/actions/pull-master-upstream + with: + scriptsRef: ${{ github.sha }} + + # Print the current SHA so we know what master SHA we synced to and can sync to it later + - name: Print current SHA + id: printCurrentSHA + run: | + chmod +x .github/scripts/common.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'from common import *; GetandPrintCurrentSHA()' + shell: bash + + # Print the chunked set of GCC tests we'd like to run in a matrix later + - name: Print GCC test set + id: printGCCTestSet + uses: ./.github/actions/print-gcc-testset + + # Setup config + - name: Setup config + id: setupconfig + run: | + chmod +x .github/scripts/gccWorkflow.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "${GITHUB_CONTEXT}" + python -c 'import sys; from gccWorkflow import *; GccWorkflow.Init(sys.argv[1], sys.argv[2], True)' "${GITHUB_CONTEXT}" "${GITHUB_TOKEN}" + shell: bash + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Build GCC + uses: ./.github/actions/build-composite + with: + configjson: ${{ steps.setupconfig.outputs.configJson }} + + - name: Compress and archive objdir so that it can be uploaded + run: tar -czf objdir.tar.gz ../objdir + + - name: Upload build output + uses: actions/upload-artifact@v2 + with: + name: gccBuild + path: objdir.tar.gz + + # This should be the last step on this machine since it will clear out fields in the config json + - name: Print No Secret Json + id: printNoSecretJson + run: | + chmod +x .github/scripts/config.py + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + python -c 'import sys; from config import *; Config.PrintNoSecretConfigJsonFromJson(sys.argv[1])' "${CONFIG_JSON}" + shell: bash + env: + CONFIG_JSON: ${{ steps.setupconfig.outputs.configJson }} + + gcc-master-test: + name: Run tests against gcc/master bits built in the previous step + needs: gcc-master-build + strategy: + matrix: ${{fromJSON(needs.gcc-master-build.outputs.matrixTestSet)}} + # Avoid cancelling other matrix chunks even if one fails + fail-fast: false + + runs-on: ubuntu-18.04 + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: python -m pip install requests + shell: bash + + # Add upstream gcc as remote and checkout the SHA we used in the master build + - name: Add upstream remote and checkout SHA used in build step + uses: ./.github/actions/pull-master-upstream + with: + scriptsRef: ${{ github.sha }} + masterRef: ${{ needs.gcc-master-build.outputs.masterSHA }} + + # We need to download the build artifact here instead of in DownloadBuildArtifact() because artifacts are not available + # for download through the REST API until after the workflow has completed. We can only get the workflows from the + # previous steps through the download-artifact action. + - name: Download gccBuild artifact + uses: actions/download-artifact@v2 + with: + name: gccBuild + path: gccBuild + + - name: Unpack objdir outisde of the repository + run: | + tar -xzf gccBuild/objdir.tar.gz -m -C .. + rm -r gccBuild + shell: bash + + # build.yaml creates the gccBuild artifact + - name: Run GCC tests + uses: ./.github/actions/test-composite + with: + configjson: ${{needs.gcc-master-build.outputs.config}} + testSet: ${{ matrix.testSet }} + githubtoken: ${{ secrets.GITHUB_TOKEN }} + buildDownloaded: True + + - name: Move objdir to be in repo so logs inside can be uploaded + run: mv ../objdir objdir + + - name: Upload test logs + uses: actions/upload-artifact@v2 + with: + name: master_${{ matrix.testSet }}_logs + path: objdir/logs + + parse-failures: + name: Parse failures and start build and test runs with gcc/master merged into vendor/microsoft/main + needs: [gcc-master-test, gcc-master-build] + runs-on: ubuntu-18.04 + outputs: + newBranchName: ${{ steps.setBranchName.outputs.newBranchName }} + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + token: ${{ secrets.VICTORPAT }} # The basic ${{ github.token }} doesn't include "workflows" write permission access to modify workflows in the .github directory + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: | + python -m pip install requests + shell: bash + + # Add upstream gcc as remote, setup vendor branches and checkout vendors/microsoft/main + - name: Setup and checkout vendors/microsoft/main + uses: ./.github/actions/setup-vendor-branches + with: + scriptsRef: ${{ github.sha }} + vendorRef: vendors/microsoft/main + + # Download all artifacts from the previous jobs. This is not ideal but we have to use download-artifact + # since the artifacts aren't available to the REST API calls until after the workflow completes + # There's also no way to specify a regular expression for the artifact names we're interested in + # so we'll have to download the GCC build earlier, which isn't necessary for this step since we only + # care about the test logs. + # To make this faster, we can avoid downloading the GCC build in the future + - uses: actions/download-artifact@v2 + with: + path: artifacts + + - name: Parse failures + run: | + chmod +x .github/scripts/gccWorkflow.py + echo "$PYTHONPATH" + echo "$CONFIG_JSON" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "$BUILD_DOWNLOADED" + python -c 'import sys; from gccWorkflow import *; GccWorkflow.ParseTestLogs(sys.argv[1], sys.argv[2])' "${CONFIG_JSON}" "${GITHUB_TOKEN}" + shell: bash + env: + CONFIG_JSON: ${{needs.gcc-master-build.outputs.config}} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + # newFailures.patch is created in ParseTestLogs IFF there were failures in master run that aren't in the .xfail file + - name: Check if failures patch exists + id: check_failures_patch + uses: andstor/file-existence-action@v1 + with: + files: "newFailures.patch" + + - name: Upload failures patch + if: ${{ steps.check_failures_patch.outputs.files_exists }} == 'true' + uses: actions/upload-artifact@v2 + with: + name: newFailures.patch + path: newFailures.patch + + - name: Get current date + id: date + run: echo "::set-output name=date::$(date +'%Y-%m-%dT%H-%M-%S')" + + - name: Set new branch name + id: setBranchName + run: echo "::set-output name=newBranchName::${name}" + env: + name: merge-master-${{ needs.gcc-master-build.outputs.masterSHA }}-${{ steps.date.outputs.date }} + + # Use .github scripts in this branch in case the workflow is testing infrastructure changes + - name: Merge master SHA and push to our GitHub repo to start run + run: | + git checkout -b ${newBranchName} + git diff --exit-code --quiet .github/ > /dev/null 2>&1 + if test $? != 0; then + git add .github/* + git commit -m "Bring latest scripts from ${BRANCH_REF}" + fi + git fetch gcc master + git branch -a + git merge ${masterSHA} + git push origin HEAD -f + shell: bash + env: + newBranchName: ${{ steps.setBranchName.outputs.newBranchName }} + masterSHA: ${{ needs.gcc-master-build.outputs.masterSHA }} + BRANCH_REF: ${{ github.sha }} + + - name: Invoke build workflow + uses: benc-uk/workflow-dispatch@v1 + with: + workflow: build + token: ${{ secrets.VICTORPAT }} + ref: ${{ steps.setBranchName.outputs.newBranchName }} + + - name: Invoke test-gcc workflow + uses: benc-uk/workflow-dispatch@v1 + with: + workflow: test-gcc + token: ${{ secrets.VICTORPAT }} + ref: ${{ steps.setBranchName.outputs.newBranchName }} + + - name: Sleep before waiting on build and test runs + run: | + sleep 3m + shell: bash + + wait-build: + name: Waiting on build with gcc/master merged into vendor/microsoft/main + needs: [parse-failures] + runs-on: ubuntu-18.04 + outputs: + newFailuresPatchExists: ${{ steps.check_failures_patch.outputs.files_exists }} + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: | + python -m pip install requests + shell: bash + + - name: Wait for build + run: | + chmod +x .github/scripts/downloadBuildArtifact.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "${GITHUB_CONTEXT}" + python -c 'import sys; from downloadBuildArtifact import *; WaitOnLatestWorkflow(sys.argv[1], sys.argv[2], sys.argv[3], True)' "${mergeBranch}" "build" "${GITHUB_TOKEN}" + shell: bash + env: + mergeBranch: ${{ needs.parse-failures.outputs.newBranchName }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + wait-test: + name: Waiting on tests with gcc/master merged into vendor/microsoft/main + needs: [parse-failures, wait-build, gcc-master-build] + runs-on: ubuntu-18.04 + outputs: + newFailuresPatchExists: ${{ steps.check_failures_patch.outputs.files_exists }} + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Install requests package which is used in downloadBuildArtifact.py + - name: Pip Install Requests + run: | + python -m pip install requests + shell: bash + + - name: Wait for test-gcc + run: | + chmod +x .github/scripts/downloadBuildArtifact.py + echo "$PYTHONPATH" + export PYTHONPATH=${PYTHONPATH}:${PWD}/.github/scripts + echo "$PYTHONPATH" + echo "${GITHUB_CONTEXT}" + python -c 'import sys; from downloadBuildArtifact import *; WaitOnLatestWorkflow(sys.argv[1], sys.argv[2], sys.argv[3], True)' "${mergeBranch}" "test-gcc" "${GITHUB_TOKEN}" + shell: bash + env: + mergeBranch: ${{ needs.parse-failures.outputs.newBranchName }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + # The text below is marked with a warning header so that it's colored yellow in the logs and will stand out + # I couldn't find any documentation on setting text color in printf statements in a GitHub Yaml file so we're using this for now + - name: Run the following command to push the merge manually + run: | + printf "::warning::Run the following command to create temporary local git branch and push the changes to our vendor branch: \n" + printf ".github/scripts/pushMergedVendorBranch.sh ${mergeBranch}" + env: + mergeBranch: ${{ needs.parse-failures.outputs.newBranchName }} diff --git a/.github/workflows/update-mirror-branches.yaml b/.github/workflows/update-mirror-branches.yaml new file mode 100644 index 00000000000..7beb047cc8a --- /dev/null +++ b/.github/workflows/update-mirror-branches.yaml @@ -0,0 +1,91 @@ +# This workflow mirrors our vendor branches into our GitHub repository to trigger the CI workflow if any new changes are present. +# The workflow will automatically push these changes and will not require manual intervention + +# Copyright (c) Microsoft Corporation. + +# MIT License + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: + +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +name: update-mirror-branches + +# Run this workflow every 10 minutes +on: + schedule: + - cron: "0,10,20,30,40,50 * * * *" + workflow_dispatch: + +jobs: + sync: + runs-on: ubuntu-18.04 + strategy: + matrix: + upstreamBranch: [""] + githubBranch: [""] + + # upstream vendor/microsoft/main maps to current + # upstream vendor/microsoft/9.1.0 maps to gcc-9.1.0 + include: + - upstreamBranch: main + githubBranch: current + - upstreamBranch: '9.1.0' + githubBranch: gcc-9.1.0 + - upstreamBranch: 'releases/gcc-11' + githubBranch: 'releases/gcc-11' + + # Exclude the default case because we can't have empty matricies + exclude: + - upstreamBranch: "" + githubBranch: "" + + # Avoid cancelling other matrix chunks even if one fails + fail-fast: false + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + submodules: recursive + lfs: true + token: ${{ secrets.VICTORPAT }} # The basic ${{ github.token }} doesn't include "workflows" write permission access to modify workflows in the .github directory + + - name: Setup Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + + # Add upstream gcc as remote, setup vendor branches and checkout matrix vendor branch + - name: Setup and checkout vendors/microsoft/${{ matrix.upstreamBranch }} + uses: ./.github/actions/setup-vendor-branches + with: + scriptsRef: ${{ github.sha }} + vendorRef: vendors/microsoft/${{ matrix.upstreamBranch }} + + - name: Get current date + id: date + run: echo "::set-output name=date::$(date +'%Y-%m-%dT%H-%M-%S')" + + - name: Push to branch in GitHub repo + run: | + git checkout -b ${newBranchName} + git push origin -f ${newBranchName}:${githubBranch} + shell: bash + env: + newBranchName: update-mirror-${{ matrix.githubBranch }}-${{ steps.date.outputs.date }} + githubBranch: ${{ matrix.githubBranch }}
reply other threads:[~2022-07-29 22:43 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220729224358.AD50438582A1@sourceware.org \ --to=erozen@gcc.gnu.org \ --cc=gcc-cvs@gcc.gnu.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).