Re: postinstall: fontconfig abnormal exit

[Brian Inglis <Brian.Inglis@SystematicSw.ab.ca>]

On 2020-09-12 06:56, Ken Brown via Cygwin wrote:
> On 9/12/2020 1:18 AM, Brian Inglis wrote:
>> On 2020-09-11 15:13, Ken Brown via Cygwin wrote:
>>> On 9/11/2020 4:30 PM, Achim Gratz wrote:
>>>> Ken Brown via Cygwin writes:
>>>>> Unfortunately, this doesn't yet fix the problem with
>>>>> fontconfig_dtd.sh.  The latter will now succeed if it is run after
>>>>> libxml2.sh, but not if it is run first.  I'm not aware of any way to
>>>>> force setup to run one postinstall script before another.
>>>>
>>>> Multiple ways:
>>>>
>>>> 1. Make the libxml postinstall script sort lexically before any
>>>> others that depend on it. Obviously this is brittle, but it
>>>> might work in this particular instance (autorebase does this).
>>>>
>>>> 2. Make the libxml catalog creation a perpetual postinstall
>>>> script with prefix "0p_". That only works if it doesn't depend
>>>> on other postinstall scripts having their work completed.
>>>>
>>>> 3. Implement and use the stratified postinstall concept originally
>>>> outlined at:
>>>> https://sourceware.org/legacy-ml/cygwin-apps/2014-12/msg00148.html
>>>>
>>>> 4. Use the package dependency order to order the postinstall script
>>>> activation.  With libzypp we should have the correct information, we'd
>>>> just need to somehow make the packagemeta iterator use it. That still
>>>> won't work if we have dependency loops.
>>>>
>>>> 5. (Try to) Run any failed postinstalls again in setup and bail only if
>>>> the number of fails does not decrease from the last iteration.
>>>>
>>>> At the time it was deemed too complicated and so we only use
>>>> the "end" strata for the perpetual postinstall scripts.  As
>>>> said then, there would need to be some serious discussion on
>>>> how to coordinate the strata assignments.

>>> There's no dependency relation between libxml2 and 
>>> libfontconfig-common, so #4 wouldn't fix the problem. And some of
>>> the other suggestions would require work on setup.exe that
>>> someone would have to do. The current problem is simple to fix
>>> and shouldn't have to wait for that.
>>>
>>> I like your idea of using perpetual postinstall scripts.  I think
>>> the way to do it is probably to make fontconfig_dtd.sh perpetual
>>> with prefix "zp_". That way if libfontconfig-common is installed
>>> without libxml2 but then libxml2 is installed later, the
>>> xmlcatalog command gets run.  We would have to check that no harm
>>> is done if that xmlcatalog command gets run more than once.

>> As libxml2 supplies /usr/bin/xmlcatalog, that postinstall script 
>> should both conditionally create the catalog and only if just
>> created, also conditionally add fonts.dtd,

> No, the libxml2 postinstall script shouldn't add fonts.dtd. That
> should be added only when/if fontconfig is installed. So it's an
> appropriate task for a fontconfig postinstall script.

>> [TL;DR: We do not want to add more permanent postinstall scripts 
>> unless essential!

> Making fontconfig_dtd.sh perpetual may or may not be essential, but it's 
> the best way I can think of to solve the problem of this bug report.  
> Can you propose a better way?

>> Permanent postinstall scripts can greatly extend the run time of
>> the Cygwin Setup program, when fontconfig, man-db, or tex just seem
>> to decide sometimes for some reason that another whole new world of
>> fonts, man pages, or whatever has appeared that they have to
>> reprocess,

> If you see a perpetual postinstall script doing something time-consuming 
> that you think is unnecessary, please make a bug report.

Unfortunately it's often too late to get any information once you notice 
setup is still running in one script.

>> pushing the Setup run time and service (cron) downtime to hours

> I've never seen this.  Please give details.

I have never been able to figure out what texlive postinstall command 
keeps going.

I have multiple distros man pages available under Cygwin for 
convenience, so I manually run man-db postinstall after updates (nohup 
... &), to try to avoid long setup postinstalls, but sometimes setup 
man-db postinstall does lengthy updates anyway.

For fontconfig fc-cache-1 appears to have been creating thousands (on 
Cygwin 64 millions) of small <1KB 
/var/cache/fontconfig/%8x-%4x-%4x-%4x-%12x-le{64,32d8}.cache-7 files.

The problems could have originally been caused by an old bug:

https://gitlab.freedesktop.org/fontconfig/fontconfig/-/issues/107

combined with many font additions around that time, mainly working on 
Cygwin 64, where I use X, and manually run fontconfig postinstall 
script, to try to avoid long setup postinstalls, whereas on Cygwin 32 I 
don't use X or manually run postinstall scripts, just get run after setup.

I have about 200 Windows MS font files, 1000 non-MS font files, and 
about 800 Cygwin font files, from multiple distros and elsewhere, 
including some with full BMP coverage, some with SMP coverage, some for 
fallback code points, others with group ranges.

I rm'ed -rf /var/cache/fontconfig/ with a few thousand files on Cygwin 
32 and rebuilt it okay with only 65 cache files.

I tried rm -rf /var/cache/fontconfig/ on Cygwin 64 but got many 
permission errors and killed it.

I gave up waiting for ls -U to show any results on Cygwin 64 or Explorer 
on that dir to show any file details, but cmd /c dir | less displays the 
base info for hundreds of thousands of files, and wc reports millions.

I am still waiting for an elevated cmd to rmdir /s /q fontconfig there!

Do you know why fc-cache-1 is run rather than fc-cache and what the 
difference is?

What would give you useful information once I have the fontconfig cache 
cleared?
Might running FC_DEBUG=65535 fc-cache-1 -fsv provide useful info?
Or do so with strace?
Would running file on the font files give enough info about properties 
to be of any help?

What is the best approach to get the minimal cache files recreated?

What is the best approach to avoid thousand of cache files in future?

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]