grep-3.0-2 issues within Makefile

grep-3.0-2 issues within Makefile

[Daniel Fort]

grep-3.0-2 binary will not function as expected when the -v option is
used in a Makefile.

Resolution - downgrade to grep-3.0-1.

When using Cygwin to build Magic Lantern users stated reporting a
build errors on new Cygwin installs around November 2016. The
resolution was to downgrade grep to the previous version.

The discussion and instructions on how to prepare Cygwin to compile
Magic Lantern are in this forum topic:

http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435

Using a Cygwin install that includes the default grep-3.0-1 will
result in errors when running the follow Makefile code:

localsyms: libtcctmp.o
    @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
        | grep -v "^tcc_new$$" \
        | grep -v "^tcc_delete$$" \
        | grep -v "^tcc_add_file$$" \
        | grep -v "^tcc_relocate$$" \
        | grep -v "^tcc_get_symbol$$" \
        | grep -v "^tcc_add_symbol$$" \
        | grep -v "^tcc_set_options$$" \
        | grep -v "^tcc_load_offline_section$$" \
        > $@

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

grep-3.0-2 issues within Makefile

[Daniel Fort]

grep-3.0-2 binary will not function as expected when the -v option is
used in a Makefile.

Resolution - downgrade to grep-3.0-1.

When using Cygwin to build Magic Lantern users stated reporting a
build errors on new Cygwin installs around November 2016. The
resolution was to downgrade grep to the previous version.

The discussion and instructions on how to prepare Cygwin to compile
Magic Lantern are in this forum topic:

http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435

Using a Cygwin install that includes the default grep-3.0-1 will
result in errors when running the follow Makefile code:

localsyms: libtcctmp.o
    @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
        | grep -v "^tcc_new$$" \
        | grep -v "^tcc_delete$$" \
        | grep -v "^tcc_add_file$$" \
        | grep -v "^tcc_relocate$$" \
        | grep -v "^tcc_get_symbol$$" \
        | grep -v "^tcc_add_symbol$$" \
        | grep -v "^tcc_set_options$$" \
        | grep -v "^tcc_load_offline_section$$" \
        > $@

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: grep-3.0-2 issues within Makefile

[Marco Atzeri]

On 29/05/2017 08:34, Daniel Fort wrote:
> grep-3.0-2 binary will not function as expected when the -v option is
> used in a Makefile.

Please note the last grep announcement
https://sourceware.org/ml/cygwin-announce/2017-02/msg00035.html

and the changes between text and binary mounts.

> Resolution - downgrade to grep-3.0-1.
>
> When using Cygwin to build Magic Lantern users stated reporting a
> build errors on new Cygwin installs around November 2016. The
> resolution was to downgrade grep to the previous version.
>
> The discussion and instructions on how to prepare Cygwin to compile
> Magic Lantern are in this forum topic:
>
> http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435
>
> Using a Cygwin install that includes the default grep-3.0-1 will
> result in errors when running the follow Makefile code:
>
> localsyms: libtcctmp.o
>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
>         | grep -v "^tcc_new$$" \
>         | grep -v "^tcc_delete$$" \
>         | grep -v "^tcc_add_file$$" \
>         | grep -v "^tcc_relocate$$" \
>         | grep -v "^tcc_get_symbol$$" \
>         | grep -v "^tcc_add_symbol$$" \
>         | grep -v "^tcc_set_options$$" \
>         | grep -v "^tcc_load_offline_section$$" \
>         > $@
>
> --

and what is the error ?

Regards
Marco

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: grep-3.0-2 issues within Makefile

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1094 bytes --]

On 05/29/2017 03:52 AM, Marco Atzeri wrote:
> On 29/05/2017 08:34, Daniel Fort wrote:
>> grep-3.0-2 binary will not function as expected when the -v option is
>> used in a Makefile.
> 
> Please note the last grep announcement
> https://sourceware.org/ml/cygwin-announce/2017-02/msg00035.html
> 
> and the changes between text and binary mounts.
> 

>> Using a Cygwin install that includes the default grep-3.0-1 will
>> result in errors when running the follow Makefile code:
>>
>> localsyms: libtcctmp.o
>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \

Most likely, $(READELF) is producing \r\n-terminated output. The
solution, then, is to rewrite the line to:

$(READELF) $< -Ws | tr -d '\r' | $(AWK) ...

> 
> and what is the error ?

Most likely, grep is not filtering as expected, because now that it is
treating your data as binary rather than text, your explicit $ anchor is
only matching \n instead of \r\n.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: grep-3.0-2 issues within Makefile

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 589 bytes --]

On 05/29/2017 06:39 AM, Eric Blake wrote:

>>> localsyms: libtcctmp.o
>>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
> 
> Most likely, $(READELF) is producing \r\n-terminated output.

That said, WHAT is $(READELF) actually expanding to? If it is the cygwin
binutils version, it should NOT be outputting \r\n in the first place.
Generally, you don't get \r\n output unless you are mixing non-cygwin
programs into the pipeline.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: grep-3.0-2 issues within Makefile

[Brian Inglis]

On 2017-05-29 05:44, Eric Blake wrote:
> On 05/29/2017 06:39 AM, Eric Blake wrote:
>>>> localsyms: libtcctmp.o
>>>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
>> Most likely, $(READELF) is producing \r\n-terminated output.
> That said, WHAT is $(READELF) actually expanding to? If it is the cygwin
> binutils version, it should NOT be outputting \r\n in the first place.
> Generally, you don't get \r\n output unless you are mixing non-cygwin
> programs into the pipeline.

Cygwin .o files are not ELF format and not recognized as such by readelf

$ readelf src/astro/sofa/20160503_a/c/build/zr.o -Ws
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start

or file

$ file src/astro/sofa/20160503_a/c/build/zr.o
src/astro/sofa/20160503_a/c/build/zr.o: data

running Cygwin readelf on cross builds which produce ELF .o don't
generate "\r":

$ file util/*.o
util/ntp-keygen.o:      ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/ntp-keygen-opts.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/ntptime.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/tickadj.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/version.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info

$ readelf util/*.o -Ws | wc -lwcL
    460    3479   26678      84
$ readelf util/*.o -Ws | grep $'\r' | wc -lwcL
      0       0       0       0

and it's not the Cygwin mingw binutils

$ /usr/bin/x86_64-w64-mingw32-readelf util/*.o -Ws | grep $'\r' | wc -lwcL
      0       0       0       0

and just to prove this detects "\r"

$ echo $'\r' | grep $'\r' | wc -lwcL
      1       0       2       0

so culprit must be native Mingw binutils readelf.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: grep-3.0-2 issues within Makefile

[Daniel Fort]

Wow, fantastic help on this list.

> Most likely, $(READELF) is producing \r\n-terminated output. The
> solution, then, is to rewrite the line to:
>
> $(READELF) $< -Ws | tr -d '\r' | $(AWK) ...
>
>>
>> and what is the error ?
>
> Most likely, grep is not filtering as expected, because now that it is
> treating your data as binary rather than text, your explicit $ anchor is
> only matching \n instead of \r\n.

Bingo. That fix worked. Testing it now to make sure it works on other
the other platforms supported by Magic Lantern.

> Eric Blake, Principal Software Engineer

Thank you Eric!

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple