From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-obe.outbound.protection.outlook.com (unknown [52.101.56.26]) by sourceware.org (Postfix) with ESMTPS id 0FB163856947 for ; Fri, 5 Aug 2022 23:07:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0FB163856947 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jHK5b2sCBHc8lSkveHaX64RF7maIjIqyrf48YA7K2vVrHonSva6qu4SDN/e/jHX93Uu5ZOLrVzFQYcQTQZyRbITf7ZjIuy4xI0vALiDPkozLe5159uKkwh/crytLrRMRGqjZZHC7KaTPMhPbck5TOp62+rO1g2b43+eV7nH9h7jXV7m15N2X3gFkbjD3z7FacnZF31Ub3CPzEnD3Dm6ssmF4POit0+aJD3rN78PANpy45VGkgyRtf7HG0QdeqpvG+1WyocE2Y6E8C7oHWNLAGA0Lfyoj8qZ0VKMnL/T5Ujxm6/QPgEp8+9Hqc0KVlVv8niE9XXcrABl55hU+bJq+GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fUi5xirMDvDXzSR1skVEHYCCMA8A9/aafSYixsxrI78=; b=HHpy9/G5u/gDlYXd3MD2xXfHnvJFG20KVhz+Zkd+gDLy+2S/zEjR4IJonyE8GaBmcL9BEjn56M3HnfkfQ/JLjWUfzU2XAiwX8Qzv+bSU0rn/9kw2QAv7063G+DCMnkZ9o0XoxQtYCrD/z/DjWbOqhQWp1UaGw1VKoFy0fB9sP2lQgfv6n7Lf43k51TrQZibrvp3eKWTGcgQ8oywTYK1RU8qQi6s7QWriCLP5b5Hmxmjlb62ODhDKkRVoifXUDpaMOamyCteRfSGei5zSBK8gvdhnMHiv+TKM+7KgTCzDtP7DDUPWM9CbAv1J71SUruZyPXFdIch6sMBXaWMEx1BgjQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none Received: from MW2PR2101MB1770.namprd21.prod.outlook.com (2603:10b6:302:8::14) by SJ0PR21MB1920.namprd21.prod.outlook.com (2603:10b6:a03:296::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.7; Fri, 5 Aug 2022 23:07:54 +0000 Received: from MW2PR2101MB1770.namprd21.prod.outlook.com ([fe80::dd39:8367:809e:e909]) by MW2PR2101MB1770.namprd21.prod.outlook.com ([fe80::dd39:8367:809e:e909%6]) with mapi id 15.20.5525.005; Fri, 5 Aug 2022 23:07:54 +0000 From: Eugene Rozenfeld To: Xi Ruoyao , "gcc-patches@gcc.gnu.org" , Andi Kleen Subject: RE: [EXTERNAL] [PATCH] contrib: modernize gen_autofdo_event.py Thread-Topic: [EXTERNAL] [PATCH] contrib: modernize gen_autofdo_event.py Thread-Index: AQHYie2JEkBRmqc2dkmhN6vf71FiMq2hLDwg Date: Fri, 5 Aug 2022 23:07:54 +0000 Message-ID: References: <0c00f719092529e0a902fab5a65152f055836348.camel@xry111.site> In-Reply-To: <0c00f719092529e0a902fab5a65152f055836348.camel@xry111.site> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=239c8671-1f39-4188-9518-cd35845f0247; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2022-08-05T23:06:51Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f0bfbd6c-20d8-46a3-3c2d-08da77375436 x-ms-traffictypediagnostic: SJ0PR21MB1920:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TNxfz/HXPPKtFi760CmfrWAJaAMkC2Axu7VxvLGtZZziPczDuoZAsQCBy60Lpv+B43CdVVqzWiiAPMhqpAvh3aXvYwFrIRWz0LARIkMU9ivq/JvwjHupWE6cW2vPWj9IomQqxaGZq05DztNvZ0GR1k86ypvrVJ1xEhVeW94nGAhYPiSK/V3ZS6B57zvDXbl1G8ZGlyi1Tc1Epc9xGJHV0I3aOArlYib3iQAUHpI4uY+uAGFQ5ZLD3yH1r7TcMn8OAn+0jmIvx9TvSbCvFXV9RRA/a2hrSFXyFKPMitEW/TVMRLSV35+ZIw/3E5PN8fZExDrepb7wnxh9calso6C36F2v6Oiqw751sjBTafT+lzE07Ok2SLdiSVvFYVtIYu1RWJv7kSMsksJP1w8a3ESpfg3cEWPrLTFBYlSuhZRXqUfHwwdKOeg9KAJOhwqn3E7xl7T1aCqF/EB2KeLvy+H0u+uEtLxr6s7lt9NoMBrSU7Fj0jUiDcHw6CGNUzkHpa2+ag67+/F/s1OkU4xfQ1+a3/ZTJ6Zh//2zmb6LSaK6mhfyPr8As/Xxb7wIabc27Xc2gkX/5503gR+dKHxV4KNOGexO1oynYvP5UTOYB6oMkJGqbhu+aAah5fsrYKjO3zDoeW0CO9MF/yWJLoUZ0mmOUYIMxh4ZPFrL1hnv8+aK5MVwoksj47OORDs3B6jOIuqbtr7AkfYvp2ebAreb8Ew4mGeKqklgAD9KHa/JUynC4dx8042Y+QH5wMmHcOkFXjEKpnSY6RDGnJyN2jXe5iPdH0id7hurQ0obCWr0cOBXwb9sdYUdkP2a5JsHLVZ7lHlzUXNcc1L1ktGkbzad35LcV45NnB9Od9/4/LONPK33Ud5RSwetFZ4ziUbfaimK5VG7Izk8LeWBPEg8Iph+d4uYfg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR2101MB1770.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(39860400002)(376002)(346002)(366004)(136003)(396003)(451199009)(8676002)(66946007)(76116006)(83380400001)(38070700005)(66556008)(122000001)(66476007)(66446008)(64756008)(52536014)(8936002)(5660300002)(53546011)(186003)(66574015)(38100700002)(33656002)(71200400001)(7696005)(82950400001)(82960400001)(6506007)(316002)(2906002)(55016003)(110136005)(9686003)(8990500004)(10290500003)(41300700001)(86362001)(478600001)(2004002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?4+As5jCR2hW/WHiIqOJ7x+f0kua4u3wIv4AJTRC8XHowHQ20GZXj0zqQybr/?= =?us-ascii?Q?HmOIqKJcHlx1gMXFw5zHoonOKowKM0CcqHy92xp1nMH/52qYTmdX9wfDB2vR?= =?us-ascii?Q?t1ucawiIVvNkDbuXmXqT6jZUrzefXwozdG0fbxj+PUAyO3Du7/sRx2Qs+2eT?= =?us-ascii?Q?Bf2u7PDGXr+V2bGxVNBo3ZK753FeKkLhFFb2+hnktHtAhJhIElm36WMuTGSq?= =?us-ascii?Q?eptefTS6xLl/79Dwc3u8+4D1FMkS1ZhjFuFY5r0XCaXLCIcR1ukR9KPQtyDI?= =?us-ascii?Q?QZNVSQxY9FGt9F5QH4+56bMgZgoPFoqT+61B8IRKtBe4Y2tA2V/ybMjX8pDq?= =?us-ascii?Q?EWoz+54YWHh6bqycCWG4vCsX77+UQ3YGq0mWorc5Dy8S+dNh24lM00Mm2ZHR?= =?us-ascii?Q?V1EtD4BUJw3xJRQnFWAtMmX/zTy7Km2M8E4DyIHnvMfkxSXA/Mu+QAtqxyZI?= =?us-ascii?Q?zghOka8k1kozM1B2GEPn2m+2OsK5IVNGWcMueETYAGHc+UdFaeg/PQw1YD7j?= =?us-ascii?Q?VIPAJcWLHn8w2z846yrIvRNO9dgRLvLGQEkuKBy/P8tfIr+zECSfmqKrdngI?= =?us-ascii?Q?ObHPJw8RNRhVwgnzy+2ABWbu95DjK2BBwQE6a2ejPLFdNyWn947wlT+s+DWy?= =?us-ascii?Q?exSITTdKu9ZudAXMjX4CUlWgfFBOV37M601Tr/3daNKMK2/9tiI0Cc1Ql0GA?= =?us-ascii?Q?8MJBOsNk+4IuV+W/Qzpo3CIXeoaoKYvhSxIhJf4ELEsmdIxnBT4KfECHyvR5?= =?us-ascii?Q?RMaca5qi+S9Ca4lqV80KMN8Va+Zp+AJS0X+a4W2Rmih/0r5SA9bz5YLtSbVD?= =?us-ascii?Q?fYvirL5B+tjK23nFAlzhs9Av/UZB1DNqC3ZROuDMqbrZkyHRXA+Xr3ALxVvv?= =?us-ascii?Q?am77nrkOmM15nTf8/OhrfgjMH0KfPwxPp9xBJsnjHiZ4Xq70dmYaccXvlZV1?= =?us-ascii?Q?sz1qSaYzbROq6KX+TjjLgtPhviYVMGCVA06Qd3dwrk7jG50/VLEegz5r2bs5?= =?us-ascii?Q?W+RQyAaRlIbRODK+j5PBTesrgOIipyJ2ZjjdZvTVmjBI6hE5dmQkz8SebJ8/?= =?us-ascii?Q?02fBjs6Hv3PMsSEjmy3cDDNvfQJT3D/hCzimPJJuQVJY2VlSd9wRiouURDKl?= =?us-ascii?Q?YS6ud/yuVTOwzo4wY3cjrr3ZNJzz8OGkCo5dmso+L4WxsCNIPls3X3wAfCGs?= =?us-ascii?Q?2FSnc4BBtdf4/LyU9mJaXmhTKEHReJxNGH9FgSoDG2sl4/x9uzc3o3aru945?= =?us-ascii?Q?to1tAC/xolZPvKZkgYA/1lg+muDXMRPECQ9GJUb7/efNJt1cpc+y/fVrUfhS?= =?us-ascii?Q?Sfi71FfMIgi9Qi9cuP0zhmmsXs/9dHNVZjbxEfhg9F7pZ1jrKYqhz5Hdarib?= =?us-ascii?Q?J6ob9DmfV/p+INhF71wULd4Ysfm9qAMoWrSeyZ/85U12J6TGARk8hrvmt8rU?= =?us-ascii?Q?h4/du9whswMRC7CrOurq5Er5akn8luxPunP7ZKPn6/L/nu2Thkate1ep/C5S?= =?us-ascii?Q?2TCklfbgMIfBVYO5D/zCRDv30YLAXM33ItOXxhtkBYfQdxy722iiqFvjnmEj?= =?us-ascii?Q?lkPJB0ZKZMu57EVbJVGJPE8UV/xLivuDOQAJsPZAnDSAU5XaHqJRD/QCUyZJ?= =?us-ascii?Q?aTnUW8pyEV9TO7wer5J621rZSqvXlGpBPPaijv2iCa0W?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR21MB1920 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2022 23:08:01 -0000 The changes look good to me. Also adding Andi, the author of the script. Eugene -----Original Message----- From: Gcc-patches = On Behalf Of Xi Ruoyao via Gcc-patches Sent: Sunday, June 26, 2022 11:15 PM To: gcc-patches@gcc.gnu.org Subject: [EXTERNAL] [PATCH] contrib: modernize gen_autofdo_event.py Python 2 has been EOL'ed for two years. egrep has been deprecated for many= years and the next grep release will start to print warning if it is used. -E option may be unsupported by some non-POSIX grep implementations, but gc= c-auto-profile won't work on non-Linux systems anyway. contrib/ChangeLog: * gen_autofdo_event.py: Port to Python 3, and use grep -E instead of egrep. gcc/ChangeLog: * config/i386/gcc-auto-profile: Regenerate. --- contrib/gen_autofdo_event.py | 80 ++++++++++++++++---------------- gcc/config/i386/gcc-auto-profile | 31 +++++++------ 2 files changed, 57 insertions(+), 54 deletions(-) diff --git a/contrib/gen_autofdo_event.py b/contrib/gen_autofdo_event.py in= dex 1eb6f1d6d85..7da2876530d 100755 --- a/contrib/gen_autofdo_event.py +++ b/contrib/gen_autofdo_event.py @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/python3 # Generate Intel taken branches Linux perf event script for autofdo profil= ing. =20 # Copyright (C) 2016 Free Software Foundation, Inc. @@ -26,18 +26,19 @@ # Requires internet (https) access. This may require setting up a proxy #= with export https_proxy=3D... # -import urllib2 +import urllib.request import sys import json import argparse import collections +import os =20 baseurl =3D "https://download.01.org/perfmon" =20 -target_events =3D (u'BR_INST_RETIRED.NEAR_TAKEN', - u'BR_INST_EXEC.TAKEN', - u'BR_INST_RETIRED.TAKEN_JCC', - u'BR_INST_TYPE_RETIRED.COND_TAKEN') +target_events =3D ('BR_INST_RETIRED.NEAR_TAKEN', + 'BR_INST_EXEC.TAKEN', + 'BR_INST_RETIRED.TAKEN_JCC', + 'BR_INST_TYPE_RETIRED.COND_TAKEN') =20 ap =3D argparse.ArgumentParser() ap.add_argument('--all', '-a', help=3D'Print for all CPUs', action=3D'stor= e_true') @@ -71,47 +72,46 @@ def get_cpustr(): return "%s-%d-%X" % tuple(cpu)[:3] =20 def find_event(eventurl, model): - print >>sys.stderr, "Downloading", eventurl - u =3D urllib2.urlopen(eventurl) + print("Downloading", eventurl, file =3D sys.stderr) + u =3D urllib.request.urlopen(eventurl) events =3D json.loads(u.read()) u.close() =20 found =3D 0 for j in events: - if j[u'EventName'] in target_events: - event =3D "cpu/event=3D%s,umask=3D%s/" % (j[u'EventCode'], j[u= 'UMask']) - if u'PEBS' in j and j[u'PEBS'] > 0: + if j['EventName'] in target_events: + event =3D "cpu/event=3D%s,umask=3D%s/" % (j['EventCode'], j['U= Mask']) + if 'PEBS' in j and int(j['PEBS']) > 0: event +=3D "p" if args.script: eventmap[event].append(model) else: - print j[u'EventName'], "event for model", model, "is", eve= nt + print(j['EventName'], "event for model", model, "is",=20 + event) found +=3D 1 return found =20 if not args.all: - cpu =3D get_cpu_str() + cpu =3D get_cpustr() if not cpu: sys.exit("Unknown CPU type") =20 url =3D baseurl + "/mapfile.csv" -print >>sys.stderr, "Downloading", url -u =3D urllib2.urlopen(url) +print("Downloading", url, file =3D sys.stderr) u =3D=20 +urllib.request.urlopen(url) found =3D 0 cpufound =3D 0 for j in u: - n =3D j.rstrip().split(',') + n =3D j.rstrip().decode().split(',') if len(n) >=3D 4 and (args.all or n[0] =3D=3D cpu) and n[3] =3D=3D "co= re": - if args.all: - components =3D n[0].split("-") - model =3D components[2] - model =3D int(model, 16) + components =3D n[0].split("-") + model =3D components[2] + model =3D int(model, 16) cpufound +=3D 1 found +=3D find_event(baseurl + n[2], model) u.close() =20 if args.script: - print '''#!/bin/sh + print('''#!/bin/sh # Profile workload for gcc profile feedback (autofdo) using Linux perf. # Auto generated. To regenerate for new CPUs run # contrib/gen_autofdo_ev= ent.py --script --all in gcc source @@ -146,27 +146,27 @@ if grep -q hyperv= isor /proc/cpuinfo ; then echo >&2 "Warning: branch profiling may not be functional in VMs" fi =20 -case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo && - egrep "^model\s*:" /proc/cpuinfo | head -n1` in''' - for event, mod in eventmap.iteritems(): +case `grep -E -q "^cpu family\s*: 6" /proc/cpuinfo && + grep -E "^model\s*:" /proc/cpuinfo | head -n1` in''') + for event, mod in eventmap.items(): for m in mod[:-1]: - print "model*:\ %s|\\" % m - print 'model*:\ %s) E=3D"%s$FLAGS" ;;' % (mod[-1], event) - print '''*) + print("model*:\ %s|\\" % m) + print('model*:\ %s) E=3D"%s$FLAGS" ;;' % (mod[-1], event)) + print('''*) echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to = update script." - exit 1 ;;''' - print "esac" - print "set -x" - print 'if ! perf record -e $E -b "$@" ; then' - print ' # PEBS may not actually be working even if the processor supp= orts it' - print ' # (e.g., in a virtual machine). Trying to run without /p.' - print ' set +x' - print ' echo >&2 "Retrying without /p."' - print ' E=3D"$(echo "${E}" | sed -e \'s/\/p/\//\')"' - print ' set -x' - print ' exec perf record -e $E -b "$@"' - print ' set +x' - print 'fi' + exit 1 ;;''') + print("esac") + print("set -x") + print('if ! perf record -e $E -b "$@" ; then') + print(' # PEBS may not actually be working even if the processor supp= orts it') + print(' # (e.g., in a virtual machine). Trying to run without /p.') + print(' set +x') + print(' echo >&2 "Retrying without /p."') + print(' E=3D"$(echo "${E}" | sed -e \'s/\/p/\//\')"') + print(' set -x') + print(' exec perf record -e $E -b "$@"') + print(' set +x') + print('fi') =20 if cpufound =3D=3D 0 and not args.all: sys.exit('CPU %s not found' % cpu) diff --git a/gcc/config/i386/gcc-auto-profile b/gcc/config/i386/gcc-auto-pr= ofile index 56f64cbff1f..5ab224b041b 100755 --- a/gcc/config/i386/gcc-auto-profile +++ b/gcc/config/i386/gcc-auto-profile @@ -33,8 +33,15 @@ if grep -q hypervisor /proc/cpuinfo ; then echo >&2 "Warning: branch profiling may not be functional in VMs" fi =20 -case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo && - egrep "^model\s*:" /proc/cpuinfo | head -n1` in +case `grep -E -q "^cpu family\s*: 6" /proc/cpuinfo && + grep -E "^model\s*:" /proc/cpuinfo | head -n1` in model*:\ 46|\=20 +model*:\ 30|\ model*:\ 31|\ model*:\ 26|\ model*:\ 47|\ model*:\ 37|\=20 +model*:\ 44) E=3D"cpu/event=3D0x88,umask=3D0x40/$FLAGS" ;; model*:\ 55|\ model*:\ 77|\ model*:\ 76|\ @@ -43,6 +50,11 @@ model*:\ 95|\ model*:\ 87|\ model*:\ 133|\ model*:\ 122) E=3D"cpu/event=3D0xC4,umask=3D0xFE/p$FLAGS" ;; +model*:\ 28|\ +model*:\ 38|\ +model*:\ 39|\ +model*:\ 54|\ +model*:\ 53) E=3D"cpu/event=3D0x88,umask=3D0x41/$FLAGS" ;; model*:\ 42|\ model*:\ 45|\ model*:\ 58|\ @@ -63,23 +75,14 @@ model*:\ 165|\ model*:\ 166|\ model*:\ 85|\ model*:\ 85) E=3D"cpu/event=3D0xC4,umask=3D0x20/p$FLAGS" ;; -model*:\ 46|\= -model*:\ 30|\ -model*:\ 31|\ -model*:\ 26|\ -model*:\ 47|\ -model*:\ 37|\= -model*:\ 44) E=3D"cpu/event=3D0x88,umask=3D0x40/p$FLAGS" ;; -model*:\ 28|= \ -model*:\ 38|\ -model*:\ 39|\ -model*:\ 54|\ -model*:\ 53) E=3D"cpu/event= =3D0x88,umask=3D0x41/p$FLAGS" ;; model*:\ 126|\ model*:\ 140|\ model*:\ = 141|\ +model*:\ 143|\ model*:\ 106|\ model*:\ 108) E=3D"cpu/event=3D0xc4,umask=3D0x20/p$FLAGS" ;; +model*:\ 134|\ +model*:\ 150) E=3D"cpu/event=3D0xc4,umask=3D0xfe/p$FLAGS" ;; *) echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to = update script." exit 1 ;; -- 2.36.1