From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout-p-101.mailbox.org (mout-p-101.mailbox.org [IPv6:2001:67c:2050:0:465::101]) by sourceware.org (Postfix) with ESMTPS id C03A13858C5F for ; Thu, 5 Oct 2023 12:17:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C03A13858C5F Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=aarsen.me Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=aarsen.me Received: from smtp2.mailbox.org (smtp2.mailbox.org [10.196.197.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-101.mailbox.org (Postfix) with ESMTPS id 4S1Vv04LCcz9sjm; Thu, 5 Oct 2023 14:17:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aarsen.me; s=MBO0001; t=1696508268; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=uVCApJaPrME/XqHgYXGPq2Ns2DfHZUMfg+P5gQABByY=; b=ryTHzNkz/9bSyQjwl2ChmUW/LcFUWNTQlWk72ATSA5JjuQfuu/4vll9yGYl9tRUv8l760q nHICzVm4ByrvdipPsQlrfuzDR316qOk88BEMbZHdY1NKu/lJ/4C0DXOLRHU1aoigh3Lc/i Tuu8m1EC2e0QBcT0sialeFL5DPIzfX0IkhPWeJh7kEPYMnxfib8CBQIy7POZh2E7bUF81O I4qB7hfG5QbsXOdm7duVqH8W69O9KdwZDKDrEXnrlSDdJ/Ynsf9uNKyF4DDUUpt1I/83X2 O/qdeOYxibgDEq/UVQL06TvhTb8EvdVwMeHxSBgu4XEmYVoEKCSbEcYskeqFtA== References: <20231004221932.06980e3f@nz> From: Arsen =?utf-8?Q?Arsenovi=C4=87?= To: Sergei Trofimovich Cc: Richard Biener , gcc@gcc.gnu.org Subject: Re: Scaling -fmacro-prefix-map= to thousands entries Date: Thu, 05 Oct 2023 14:14:16 +0200 In-reply-to: Message-ID: <87zg0xs1p2.fsf@aarsen.me> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,KAM_INFOUSMEBIZ,KAM_SHORT,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Sergei Trofimovich via Gcc writes: > Sounds good. Do you have any preference over specific syntax? My > suggestions would be: > > 1. `-fmacro-prefix-map=3Dfile-name`: if `file-name` there is not in `key= =3Dval` > format then treat it as file > 2. `-fmacro-prefix-map=3D@file-name`: use @ as a signal to use file > 3. `fmacro-prefix-map-file=3Dfile-name`: use a new option > >> Btw, I thought we have response files to deal with command-line limits, >> why doesn't that work here? I see the driver expands response files >> but IIRC it also builds those when the command-line gets too large >> and uses it for the environment and the cc1 invocation? If it doesn't >> do the latter why not fix it that way? > > Yeah, in theory response files would extend the limit. In practice `gcc` > always extends response files internally into a single > `COLLECT_GCC_OPTIONS` option and hits the environment variable limit > very early: > > https://gcc.gnu.org/PR111527 Doesn't it make more sense to fix this? The issue is more general than just this option (even if manifesting like so today). Can the COLLECT_GCC_OPTIONS consumers deal with argfiles? > Example reproducer: > > $ for i in `seq 1 1000`; do printf -- "-fmacro-prefix-map=3D%0*d=3D%0= *d\n" 200 1 200 2; done > a.rsp > $ touch a.c; gcc @a.rsp -c a.c > gcc: fatal error: cannot execute 'cc1': execv: Argument list too long > compilation terminated. > > And if you want to look at the gory details: > > $ strace -f -etrace=3Dexecve -s 1000000 -v -v -v gcc @a.rsp -c a.c > ... > [pid 78] execve("cc1", ["cc1", "-quiet", "a.c", "-quiet", "-dumpba= se", "a.c", "-dumpbase-ext", ".c", "-mtune=3Dgeneric", "-march=3Dx86-64", > "-fmacro-prefix-map=3D...=3D...", > "-fmacro-prefix-map=3D...=3D...", > ...], > [..., > "COLLECT_GCC=3Dgcc", > "COLLECT_GCC_OPTIONS=3D'-fmacro-prefix-map=3D...=3D...' '-fmacro-pre= fix-map=3D...=3D...' ... '-c' '-mtune=3Dgeneric' '-march=3Dx86-64'"]) =3D -= 1 E2BIG (Argument list too long) > > Note how `gcc` not only expands response file into an argument list > (that is not too bad) but also duplicates the whole list as a single > `COLLECT_GCC_OPTIONS=3D...` environment variable with added quoting on > top. > > Would be nice if `gcc` just passed response files around as is :) =2D-=20 Arsen Arsenovi=C4=87 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIYEARYKAC4WIQT+4rPRE/wAoxYtYGFSwpQwHqLEkwUCZR6paRAcYXJzZW5AYWFy c2VuLm1lAAoJEFLClDAeosSTIncA/jmlwLkcAsJ+vBkAfR+nLi3KTZAOZsMZEjpc rqMVHQLCAQCrqWFd4qi5Gk9RuO0HWq0v9qxoCRhy5VNvmhqpGFEWAw== =FkmC -----END PGP SIGNATURE----- --=-=-=--