From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) by sourceware.org (Postfix) with ESMTPS id 0327E3994813 for ; Tue, 20 Jul 2021 17:00:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0327E3994813 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=SystematicSw.ab.ca Authentication-Results: sourceware.org; spf=none smtp.mailfrom=systematicsw.ab.ca Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTP id 5o0JmZL4X4bIn5t6omQtwZ; Tue, 20 Jul 2021 17:00:42 +0000 Received: from [192.168.1.104] ([68.147.0.90]) by cmsmtp with ESMTP id 5t6mmYZ3qB9dP5t6nmOTk4; Tue, 20 Jul 2021 17:00:42 +0000 X-Authority-Analysis: v=2.4 cv=Ac10o1bG c=1 sm=1 tr=0 ts=60f7013a a=T+ovY1NZ+FAi/xYICV7Bgg==:117 a=T+ovY1NZ+FAi/xYICV7Bgg==:17 a=IkcTkHD0fZMA:10 a=CCpqsmhAAAAA:8 a=e5mUnYsNAAAA:8 a=FhwSYsPFeF_qToIT26AA:9 a=QEXdDO2ut3YA:10 a=o1yDpi_DOQcA:10 a=ul9cdbp4aOFLsgKbc677:22 a=Vxmtnl_E_bksehYqCbjh:22 Reply-To: cygwin@cygwin.com To: cygwin@cygwin.com References: <782011494.20200910083521@yandex.ru> <5a2fdf46-93c8-048b-cadb-cb9d9212c716@SystematicSw.ab.ca> <20200910230426.5811f3e8@quarternote> <8f40571c-1a37-8e4b-f1bd-ecf40175d0d7@gmail.com> <179bbaf0-02b4-1c63-0083-5fa2a8833ea9@cornell.edu> <06e9cf44-8cc8-267e-12b3-e8a866a01c80@gmail.com> <87h7s45a4c.fsf@Rainer.invalid> <3305b90c-41f2-7377-092d-0f151a83da1c@cornell.edu> <9c515b79-42e2-5a9b-1996-9281f3c2b0e8@cornell.edu> From: Brian Inglis Organization: Systematic Software Subject: Re: postinstall: fontconfig abnormal exit Message-ID: Date: Tue, 20 Jul 2021 11:00:39 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <9c515b79-42e2-5a9b-1996-9281f3c2b0e8@cornell.edu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-CA Content-Transfer-Encoding: 8bit X-CMAE-Envelope: MS4xfC0jtycNcRdN6UZBsrl1nSFyP/m/GFRuGEqUG2iPCIAnmBaTZeyKgZoXELQlWZZlbWxztDlJPmlAY9QYKLTpHdRMtWJYLXnMqlSdgGCz/G3sjaAY9793 2eu3N/rcVQQeVXlJKsYjgztALzotKYo414S68pC53csAkKKiclJCW8C0DBRc5FT+4X/8wwdYDeyrS/YNpWQpxCQ6AkTP6oM2WC0= X-Spam-Status: No, score=-1161.1 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, NICE_REPLY_A, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2021 17:00:47 -0000 On 2020-09-12 06:56, Ken Brown via Cygwin wrote: > On 9/12/2020 1:18 AM, Brian Inglis wrote: >> On 2020-09-11 15:13, Ken Brown via Cygwin wrote: >>> On 9/11/2020 4:30 PM, Achim Gratz wrote: >>>> Ken Brown via Cygwin writes: >>>>> Unfortunately, this doesn't yet fix the problem with >>>>> fontconfig_dtd.sh.  The latter will now succeed if it is run after >>>>> libxml2.sh, but not if it is run first.  I'm not aware of any way to >>>>> force setup to run one postinstall script before another. >>>> >>>> Multiple ways: >>>> >>>> 1. Make the libxml postinstall script sort lexically before any >>>> others that depend on it. Obviously this is brittle, but it >>>> might work in this particular instance (autorebase does this). >>>> >>>> 2. Make the libxml catalog creation a perpetual postinstall >>>> script with prefix "0p_". That only works if it doesn't depend >>>> on other postinstall scripts having their work completed. >>>> >>>> 3. Implement and use the stratified postinstall concept originally >>>> outlined at: >>>> https://sourceware.org/legacy-ml/cygwin-apps/2014-12/msg00148.html >>>> >>>> 4. Use the package dependency order to order the postinstall script >>>> activation.  With libzypp we should have the correct information, we'd >>>> just need to somehow make the packagemeta iterator use it. That still >>>> won't work if we have dependency loops. >>>> >>>> 5. (Try to) Run any failed postinstalls again in setup and bail only if >>>> the number of fails does not decrease from the last iteration. >>>> >>>> At the time it was deemed too complicated and so we only use >>>> the "end" strata for the perpetual postinstall scripts. As >>>> said then, there would need to be some serious discussion on >>>> how to coordinate the strata assignments. >>> There's no dependency relation between libxml2 and >>> libfontconfig-common, so #4 wouldn't fix the problem. And some of >>> the other suggestions would require work on setup.exe that >>> someone would have to do. The current problem is simple to fix >>> and shouldn't have to wait for that. >>> >>> I like your idea of using perpetual postinstall scripts. I think >>> the way to do it is probably to make fontconfig_dtd.sh perpetual >>> with prefix "zp_". That way if libfontconfig-common is installed >>> without libxml2 but then libxml2 is installed later, the >>> xmlcatalog command gets run. We would have to check that no harm >>> is done if that xmlcatalog command gets run more than once. >> As libxml2 supplies /usr/bin/xmlcatalog, that postinstall script >> should both conditionally create the catalog and only if just >> created, also conditionally add fonts.dtd, > No, the libxml2 postinstall script shouldn't add fonts.dtd. That > should be added only when/if fontconfig is installed. So it's an > appropriate task for a fontconfig postinstall script. >> [TL;DR: We do not want to add more permanent postinstall scripts >> unless essential! > Making fontconfig_dtd.sh perpetual may or may not be essential, but it's > the best way I can think of to solve the problem of this bug report. > Can you propose a better way? >> Permanent postinstall scripts can greatly extend the run time of >> the Cygwin Setup program, when fontconfig, man-db, or tex just seem >> to decide sometimes for some reason that another whole new world of >> fonts, man pages, or whatever has appeared that they have to >> reprocess, > If you see a perpetual postinstall script doing something time-consuming > that you think is unnecessary, please make a bug report. Unfortunately it's often too late to get any information once you notice setup is still running in one script. >> pushing the Setup run time and service (cron) downtime to hours > I've never seen this.  Please give details. I have never been able to figure out what texlive postinstall command keeps going. I have multiple distros man pages available under Cygwin for convenience, so I manually run man-db postinstall after updates (nohup ... &), to try to avoid long setup postinstalls, but sometimes setup man-db postinstall does lengthy updates anyway. For fontconfig fc-cache-1 appears to have been creating thousands (on Cygwin 64 millions) of small <1KB /var/cache/fontconfig/%8x-%4x-%4x-%4x-%12x-le{64,32d8}.cache-7 files. The problems could have originally been caused by an old bug: https://gitlab.freedesktop.org/fontconfig/fontconfig/-/issues/107 combined with many font additions around that time, mainly working on Cygwin 64, where I use X, and manually run fontconfig postinstall script, to try to avoid long setup postinstalls, whereas on Cygwin 32 I don't use X or manually run postinstall scripts, just get run after setup. I have about 200 Windows MS font files, 1000 non-MS font files, and about 800 Cygwin font files, from multiple distros and elsewhere, including some with full BMP coverage, some with SMP coverage, some for fallback code points, others with group ranges. I rm'ed -rf /var/cache/fontconfig/ with a few thousand files on Cygwin 32 and rebuilt it okay with only 65 cache files. I tried rm -rf /var/cache/fontconfig/ on Cygwin 64 but got many permission errors and killed it. I gave up waiting for ls -U to show any results on Cygwin 64 or Explorer on that dir to show any file details, but cmd /c dir | less displays the base info for hundreds of thousands of files, and wc reports millions. I am still waiting for an elevated cmd to rmdir /s /q fontconfig there! Do you know why fc-cache-1 is run rather than fc-cache and what the difference is? What would give you useful information once I have the fontconfig cache cleared? Might running FC_DEBUG=65535 fc-cache-1 -fsv provide useful info? Or do so with strace? Would running file on the font files give enough info about properties to be of any help? What is the best approach to get the minimal cache files recreated? What is the best approach to avoid thousand of cache files in future? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]