[gcc(refs/vendors/microsoft/heads/main)] This fixes two bugs in the update-main workflow:

[gcc(refs/vendors/microsoft/heads/main)] This fixes two bugs in the update-main workflow:

[Eugene Rozenfeld]

https://gcc.gnu.org/g:3bbfbe9828429a2dfb6efa07c09f60bff5b5c7a0

commit 3bbfbe9828429a2dfb6efa07c09f60bff5b5c7a0
Author: Victor Tong <vitong@microsoft.com>
Date:   Wed Jun 16 16:17:39 2021 -0700

    This fixes two bugs in the update-main workflow:
    
    Logs from the master test run weren't downloaded. I reached out to the GitHub support team and found out that the REST API won't return artifacts until after the workflow has completed. I've made a feature request to support this scenario. In the meantime, we have to use the download-artifact yaml action to download all artifacts (which takes longer than needed since we don't actually need the build bits when parsing failures), but it's the only clean way of implementing this for now since we can't have a yaml step in a loop.
    
    Fix the failures log parsing where it doesn't return any lines if there are unexpected failures but no expected failures. The reason is that the pattern matching looks for the "unexpected failures" and "expected results" strings and returns the line in between but if the latter is missing, it won't return anything. The fix is to first find all lines after "unexpected failures" and then look to remove all lines from "expected results" onwards, leaving behind all lines in between even if "expected results" wasn't found.
    
    A test run of the update-main workflow was done here: https://github.com/microsoft/gcc/actions/runs/947890871

Diff:
---
 .github/scripts/gccWorkflow.py     | 36 +++++++++++++++++++-----------------
 .github/workflows/update-main.yaml | 10 ++++++++++
 2 files changed, 29 insertions(+), 17 deletions(-)

diff --git a/.github/scripts/gccWorkflow.py b/.github/scripts/gccWorkflow.py
index 115f588fd7b..f63fd45b241 100644
--- a/.github/scripts/gccWorkflow.py
+++ b/.github/scripts/gccWorkflow.py
@@ -28,6 +28,7 @@ import globals
 from common import *
 import os
 import json
+import glob
 
 class GccWorkflow(object):
     # Setup the config object and wait for any runs necessary to finish before proceeding onto the next job section
@@ -137,29 +138,27 @@ class GccWorkflow(object):
 
         Config.Reload(configJson, accessToken)
 
-        # Download all logs
-        logDirs = set()
-        artifacts = GetArtifactObjsInRun(globals.configObj.runID, '.*_logs$')
-        for artifact in artifacts:
-            DownloadArtifact(artifact['id'])
-            logDirs.add(artifact['name'])
-
-            # Each directory should have a failures.txt file
-            assert os.path.exists(artifact['name'] + '/failures.txt')
-
         # Load in the tests known to fail in our vendor branch
         with open('contrib/testsuite-management/x86_64-pc-linux-gnu.xfail') as xFailFile:
             linesFailFile = {line.rstrip('\n') for line in xFailFile}
 
-        newFailuresToAdd = set()
+        newFailuresToAdd = []
+
+        # Look for logs in the artifacts directory that were downloaded in the yaml via download-artifact
+        logDirs = glob.glob('artifacts/master_*_logs')
 
         for logDir in logDirs:
             logger.info ("Parsing log directory: " + logDir)
-            # Create a rawFailures.txt file with just the unexpected failure lines
+
+            # Each directory should have a failures.txt file
+            assert os.path.exists(logDir + '/failures.txt')
+
+            # Create a rawFailures.txt file with just the unexpected failure lines which are between the
+            # "Unexpected results in this build" and "Expected results not present in this build" markers in the file
             res = subprocess.run('cd ' + logDir                
                 + '''
-                sed -n '/^Unexpected results in this build/,${p;/^Expected results not present in this build/q}' failures.txt > rawFailures.txt
-                sed -e '1d' -e '$d' -i rawFailures.txt 
+                sed '0,/^Unexpected results in this build/d' failures.txt > afterUnexpected.txt
+                sed '/^Expected results not present in this build/q' afterUnexpected.txt > rawFailures.txt
                 sed -r '/^\s*$/d' -i rawFailures.txt
                 ''',
                 shell=True, check=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
@@ -169,12 +168,16 @@ class GccWorkflow(object):
                 failsInThisDir = {line.rstrip('\n') for line in rawFailuresFile}
             newFails = failsInThisDir - linesFailFile
 
-            newFailuresToAdd = newFailuresToAdd | newFails
+            for newFail in newFails:
+                newFailuresToAdd.append(newFail)
 
         if len(newFailuresToAdd) == 0:
             logger.info("No new failures detected")
             return
 
+        # Sort the list of failures so that they're in a predictable order
+        newFailuresToAdd.sort()
+
         with open('contrib/testsuite-management/x86_64-pc-linux-gnu.xfail', 'a') as xFailFile:
             logger.info("New failures found:")
             for newFail in newFailuresToAdd:
@@ -182,8 +185,7 @@ class GccWorkflow(object):
                 xFailFile.write("\n")
                 xFailFile.write(newFail)
 
-
-        # Create a patch file 
+        # Create a patch file
         res = subprocess.run('''
                 git diff contrib/testsuite-management/x86_64-pc-linux-gnu.xfail > newFailures.patch
                 ''',
diff --git a/.github/workflows/update-main.yaml b/.github/workflows/update-main.yaml
index 9b77d0b0182..55c58e46bdd 100644
--- a/.github/workflows/update-main.yaml
+++ b/.github/workflows/update-main.yaml
@@ -220,6 +220,16 @@ jobs:
           scriptsRef: ${{ github.sha }}
           vendorRef: vendors/microsoft/main
 
+      # Download all artifacts from the previous jobs. This is not ideal but we have to use download-artifact
+      # since the artifacts aren't available to the REST API calls until after the workflow completes
+      # There's also no way to specify a regular expression for the artifact names we're interested in
+      # so we'll have to download the GCC build earlier, which isn't necessary for this step since we only
+      # care about the test logs.
+      # To make this faster, we can avoid downloading the GCC build in the future
+      - uses: actions/download-artifact@v2
+        with:
+          path: artifacts
+
       - name: Parse failures
         run: | 
           chmod +x .github/scripts/gccWorkflow.py