Re: compiling coreutils with cygport

[Eric Blake <eblake@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4439 bytes --]

On 07/13/2012 03:25 PM, Jeff Janes wrote:
> If I use setup.exe to download the src for coreutils (just by checking
> the "Src?" checkbox in the gui) , and then use cygport to build it, it
> fails.  The reason it fails  seems to be that ginstall.exe does not
> have the proper manifest file so windows refuses to execute it.  This
> is the same error you get if you try to build coreutils downloaded
> from upstream gnu sources.

>   GEN    install.1
> help2man: can't get `--help' info from install.td/install

Wow.  I'm glad I build the coreutils package on my Windows XP box, where
I'm not impacted by newer Window's stupidity.

> 
> If I delete the two patch files which setup.exe automatically
> downloads into /usr/src (coreutils-8.15-1.src.patch and
> coreutils-8.15-1.cygwin.patch), and then run cygport as before, it
> compiles fine.

But then it doesn't have any cygwin-specific patches, so you aren't
getting the full coreutils experience that I have intended to package.
(Hmm, that reminds me, I'm due for a coreutils upgrade for cygwin soon)

>  Also, it works if I delete all the files which
> setup.exe downloads to /usr/src except for the
> "coreutils-8.15-1.cygport" one, and then run:
> 
> cygport coreutils-8.15-1.cygport download prep compile
> 
> So it looks like the setup.exe is giving me a coreutils tar ball which
> has already been patched to work with cygwin, but is also giving me
> some patch files which undo that and cause it to be broken again.  I
> can work around that by deleting the patch files, but it doesn't seem
> like that should be necessary.  Is the problem in cygport, or
> setup.exe, or in my understanding of how these things are supposed to
> work together?

setup.exe gives you the working tarball (which I compiled on a windows
box that doesn't need a manifest file to be built), as well as the
patches that are included in that tarball.  If you don't apply the
patches, then you aren't building the cygwin version of coreutils, at
which point, if things work, great, but it's not quite relevant here.
So the real trick is to figure out how to make the upstream make process
work with the need for manifest files during the build (rather than my
current solution of only creating manifest files after the build), and
unfortunately, since I don't have access to anything newer than XP, I
can't really test that.  I'll be relying on you (or anyone else really
bothered by this) to provide patches.

> As an aside, the reason I wanted to build coreutils from source is
> that I was trying to figure out why cut.exe is so slow, even under
> LANG=C.  It is slow both compared to a functionally equivalent
> invocation of cygwin's awk.exe, and compared to running cut on a linux
> VM.   What I found was that cut.exe is slow because the "putchar"
> function is slow under cygwin.   cut.exe uses "putchar" to generate
> its output, while awk.exe does not. 

Actually, coreutils tries to use all the *_unlocked variants of stdio,
since those have better speed in single-threaded programs (all but sort
are single-threaded, and sort skips stdio).  I do know that cygwin has
putchar_unlocked, but does not have quite as many *_unlocked variants as
Linux, so it may be a case where the configure logic to detect _unlocked
functions went wrong and cygwin ends up using putchar instead of
putchar_unlocked as a result.  I'll have to look into it more.

It may also be an efficiency thing - calling one putchar per byte is
slower than calling operations to put a line at a time, just due to the
sheer overhead of a function call.  But looking at cut.c, it really is
calling fwrite() for the bulk of a line, and saving putchar() for just
the line terminator, which means the bulk of the processing isn't
suffering from the putchar() slowdowns.

>  I'm kind of at a dead-end there,
> as I can't figure out where putchar itself is implemented.

In newlib (cygwin's libc).  http://sourceware.org/newlib/

http://sourceware.org/cgi-bin/cvsweb.cgi/src/newlib/libc/stdio/putchar_u.c?annotate=1.3&cvsroot=src

>  So I've
> just learned to use awk.exe rather than cut.exe when I care about
> performance.

Interesting observation, and may be one worth making upstream that cut.c
isn't being as efficient as possible.

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]