R does not handle package non-ASCII DESCRIPTION files properly

R does not handle package non-ASCII DESCRIPTION files properly

[Markus Hoenicka]

Hi,

I've contacted the processx package maintainer on a problem reported 
here previously (see 
https://cygwin.com/pipermail/cygwin/2020-April/244667.html). He 
suggested to try the github version but that triggered a different type 
of error which does not seem to be package-specific.

1st try: build from a local copy of the github package sources with 
default locale settings on a German Windows 10 system


> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source")
Installing package into ‘/usr/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘processx’ ...
** using staged installation
Error : Invalid DESCRIPTION file

Authors@R field gives no person with maintainer role, valid email
address and non-empty name.

See section 'The DESCRIPTION file' in the 'Writing R Extensions'
manual.

ERROR: installing package DESCRIPTION failed for package ‘processx’
* removing ‘/usr/lib/R/site-library/processx’
Warning message:
In install.packages("/cygdrive/c/Temp/processx-master", repos = NULL,  :
   installation of package ‘/cygdrive/c/Temp/processx-master’ had 
non-zero exit status


2nd try: same as above, but with LC_ALL=en_US.UTF-8
same result as above

3rd try: edit DESCRIPTION file and remove accented characters in author 
name
(change "Gábor", "Csárdi" to "Gabor", "Csardi")

> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source")
Installing package into ‘/usr/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘processx’ ...
** using staged installation
** libs
gcc -ggdb -O2 -pipe -Wall -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong 
--param=ssp-buffer-size=4 -std=gnu99 
-fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/R/R-3.6.3-1.x86_64/build=/usr/src/debug/R-3.6.3-1 
-fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/R/R-3.6.3-1.x86_64/src/R-3.6.3=/usr/src/debug/R-3.6.3-1 
  -Wall tools/px.c -o tools/px
[...]

So this circumvents the problems in processing the non-ASCII description 
file (which is properly tagged as UTF-8 BTW), but it results in the same 
compilation failure as reported in the message above.

As mentioned in the other report, Cygwin is a fresh installation as of 
April 28, R package version is R 3.6.3.1.

regards,
Markus


-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Marco Atzeri]

Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
> Hi,
> 
> I've contacted the processx package maintainer on a problem reported 
> here previously (see 
> https://cygwin.com/pipermail/cygwin/2020-April/244667.html). He 
> suggested to try the github version but that triggered a different type 
> of error which does not seem to be package-specific.
> 
> 1st try: build from a local copy of the github package sources with 
> default locale settings on a German Windows 10 system
> 
> 
>> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source") 
>>
> Installing package into ‘/usr/lib/R/site-library’
> (as ‘lib’ is unspecified)
> * installing *source* package ‘processx’ ...
> ** using staged installation
> Error : Invalid DESCRIPTION file
> 
> Authors@R field gives no person with maintainer role, valid email
> address and non-empty name.
> 
> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
> manual.
> 
> ERROR: installing package DESCRIPTION failed for package ‘processx’
> * removing ‘/usr/lib/R/site-library/processx’
> Warning message:
> In install.packages("/cygdrive/c/Temp/processx-master", repos = NULL,  :
>    installation of package ‘/cygdrive/c/Temp/processx-master’ had 
> non-zero exit status
> 
> 
> 2nd try: same as above, but with LC_ALL=en_US.UTF-8
> same result as above
> 
> 3rd try: edit DESCRIPTION file and remove accented characters in author 
> name
> (change "Gábor", "Csárdi" to "Gabor", "Csardi")
> 
>> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source") 
>>
> Installing package into ‘/usr/lib/R/site-library’
> (as ‘lib’ is unspecified)
> * installing *source* package ‘processx’ ...
> ** using staged installation
> ** libs
> gcc -ggdb -O2 -pipe -Wall -Werror=format-security 
> -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong 
> --param=ssp-buffer-size=4 -std=gnu99 
> -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/R/R-3.6.3-1.x86_64/build=/usr/src/debug/R-3.6.3-1 
> -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/R/R-3.6.3-1.x86_64/src/R-3.6.3=/usr/src/debug/R-3.6.3-1 
>   -Wall tools/px.c -o tools/px
> [...]
> 
> So this circumvents the problems in processing the non-ASCII description 
> file (which is properly tagged as UTF-8 BTW), but it results in the same 
> compilation failure as reported in the message above.
> 
> As mentioned in the other report, Cygwin is a fresh installation as of 
> April 28, R package version is R 3.6.3.1.
> 
> regards,
> Markus
> 

Noted. I will look on both the two issues but it could take a bit.
I never looked at the build process of R packages in such details

Marco
R cygwin package maintainer

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Marco Atzeri]

Am 30.04.2020 um 22:17 schrieb Marco Atzeri:
> Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
>> Hi,
>>
>> I've contacted the processx package maintainer on a problem reported 
>> here previously (see 
>> https://cygwin.com/pipermail/cygwin/2020-April/244667.html). He 
>> suggested to try the github version but that triggered a different 
>> type of error which does not seem to be package-specific.

the original problem is caused by the lack of $(LIBR)
after $(CLIENT_OBJECTS) in src/Makevars

Cygwin as Windows need the link library after the objects.

$ grep SHLIB_LINK Makevars*
Makevars:       $(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS) 
$(PKG_LIBS)
Makevars.win:   $(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS) 
$(LIBR) $(SHLIB_LIBADD) $(PKG_LIBS)


the documentations

https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars

suggest the form

$(SHLIB): $(OBJECTS)
         $(SHLIB_LINK) -o $ $(OBJECTS) $(ALL_LIBS)

and this should work as
  /usr/lib/R/etc/Makeconf
defines

ALL_LIBS = $(PKG_LIBS) $(SHLIB_LIBADD) $(LIBR) $(LIBINTL) $(LIBS)


so try with
$(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS) $(ALL_LIBS)

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Markus Hoenicka]

Am 2020-05-01 12:44, schrieb Marco Atzeri via Cygwin:
> Am 30.04.2020 um 22:17 schrieb Marco Atzeri:
>> Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
>>> Hi,
>>> 
>>> I've contacted the processx package maintainer on a problem reported 
>>> here previously (see 
>>> https://cygwin.com/pipermail/cygwin/2020-April/244667.html). He 
>>> suggested to try the github version but that triggered a different 
>>> type of error which does not seem to be package-specific.
> 
> the original problem is caused by the lack of $(LIBR)
> after $(CLIENT_OBJECTS) in src/Makevars
> 
> Cygwin as Windows need the link library after the objects.
> 
> $ grep SHLIB_LINK Makevars*
> Makevars:       $(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS)
> $(PKG_LIBS)
> Makevars.win:   $(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS)
> $(LIBR) $(SHLIB_LIBADD) $(PKG_LIBS)
> 
> 
> the documentations
> 
> https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars
> 
> suggest the form
> 
> $(SHLIB): $(OBJECTS)
>         $(SHLIB_LINK) -o $ $(OBJECTS) $(ALL_LIBS)
> 
> and this should work as
>  /usr/lib/R/etc/Makeconf
> defines
> 
> ALL_LIBS = $(PKG_LIBS) $(SHLIB_LIBADD) $(LIBR) $(LIBINTL) $(LIBS)
> 
> 
> so try with
> $(SHLIB_LINK) -o client$(SHLIB_EXT) $(CLIENT_OBJECTS) $(ALL_LIBS)

Hi Marco,

thanks for looking into this. I can confirm that the processx package 
compiles just fine with the modification of Makevars that you suggested. 
I'll report this to the processx maintainer and ask him to fix this.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Markus Hoenicka]

Am 2020-04-30 22:17, schrieb Marco Atzeri via Cygwin:
> Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
>> Hi,
>> 
>> I've contacted the processx package maintainer on a problem reported 
>> here previously (see 
>> https://cygwin.com/pipermail/cygwin/2020-April/244667.html). He 
>> suggested to try the github version but that triggered a different 
>> type of error which does not seem to be package-specific.
>> 
>> 1st try: build from a local copy of the github package sources with 
>> default locale settings on a German Windows 10 system
>> 
>> 
>>> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source")
>> Installing package into ‘/usr/lib/R/site-library’
>> (as ‘lib’ is unspecified)
>> * installing *source* package ‘processx’ ...
>> ** using staged installation
>> Error : Invalid DESCRIPTION file
>> 
>> Authors@R field gives no person with maintainer role, valid email
>> address and non-empty name.
>> 
>> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
>> manual.
>> 
>> ERROR: installing package DESCRIPTION failed for package ‘processx’
>> * removing ‘/usr/lib/R/site-library/processx’
>> Warning message:
>> In install.packages("/cygdrive/c/Temp/processx-master", repos = NULL,  
>> :
>>    installation of package ‘/cygdrive/c/Temp/processx-master’ had 
>> non-zero exit status
>> 
>> 

[...]

>> 
> 
> Noted. I will look on both the two issues but it could take a bit.
> I never looked at the build process of R packages in such details
> 

Hi,

kind of "while you're at it": I've noticed one additional problem with 
non-ASCII characters. This may or may not be related to the problem 
reported above. Several packages throw an error during installation, in 
my case questionr, readr, pls, and vegan. The output of the latter is 
shown here:

installing to /usr/lib/R/site-library/00LOCK-vegan/00new/vegan/libs
** R
** data
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
Error in Rd_info(db[[i]]) :
   missing/empty \title field in 
'/tmp/RtmprL23H1/R.INSTALLffe51949f2e2/vegan/man/sipoo.Rd'
Rd files must have a non-empty \title.
See chapter 'Writing R documentation' in manual 'Writing R Extensions'.
* removing ‘/usr/lib/R/site-library/vegan’


The file in question (man/sipoo.Rd) starts like this:

\encoding{UTF-8}
\name{sipoo}
\alias{sipoo}
\alias{sipoo.map}
\docType{data}
\title{ Birds in the Archipelago of Sipoo (Sibbo and Borgå)}

Please note that the encoding is set to UTF-8 explicitely. Nonetheless, 
the a with overring in "Borgå" seems to mess up the title, which makes R 
think the \title field is missing. Changing the å to a circumvents this 
problem. I've also noticed that non-ASCII characters outside the title 
field do not hurt at all.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Brian Inglis]

On 2020-05-05 02:47, Markus Hoenicka wrote:
> Am 2020-04-30 22:17, schrieb Marco Atzeri via Cygwin:
>> Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
>>> I've contacted the processx package maintainer on a problem reported here
>>> previously (see https://cygwin.com/pipermail/cygwin/2020-April/244667.html).
>>> He suggested to try the github version but that triggered a different type of
>>> error which does not seem to be package-specific.
>>>
>>> 1st try: build from a local copy of the github package sources with default
>>> locale settings on a German Windows 10 system
>>>
>>>> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source")

>>> Installing package into ‘/usr/lib/R/site-library’
>>> (as ‘lib’ is unspecified)
>>> * installing *source* package ‘processx’ ...
>>> ** using staged installation
>>> Error : Invalid DESCRIPTION file
>>>
>>> Authors@R field gives no person with maintainer role, valid email
>>> address and non-empty name.
>>>
>>> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
>>> manual.
>>>
>>> ERROR: installing package DESCRIPTION failed for package ‘processx’
>>> * removing ‘/usr/lib/R/site-library/processx’
>>> Warning message:
>>> In install.packages("/cygdrive/c/Temp/processx-master", repos = NULL,  :
>>>    installation of package ‘/cygdrive/c/Temp/processx-master’ had non-zero
>>> exit status

>> Noted. I will look on both the two issues but it could take a bit.
>> I never looked at the build process of R packages in such details

> kind of "while you're at it": I've noticed one additional problem with non-ASCII
> characters. This may or may not be related to the problem reported above.
> Several packages throw an error during installation, in my case questionr,
> readr, pls, and vegan. The output of the latter is shown here:
> 
> installing to /usr/lib/R/site-library/00LOCK-vegan/00new/vegan/libs
> ** R
> ** data
> ** inst
> ** byte-compile and prepare package for lazy loading
> ** help
> *** installing help indices
> Error in Rd_info(db[[i]]) :
>   missing/empty \title field in
> '/tmp/RtmprL23H1/R.INSTALLffe51949f2e2/vegan/man/sipoo.Rd'
> Rd files must have a non-empty \title.
> See chapter 'Writing R documentation' in manual 'Writing R Extensions'.
> * removing ‘/usr/lib/R/site-library/vegan’
> 
> The file in question (man/sipoo.Rd) starts like this:
> 
> \encoding{UTF-8}
> \name{sipoo}
> \alias{sipoo}
> \alias{sipoo.map}
> \docType{data}
> \title{ Birds in the Archipelago of Sipoo (Sibbo and Borgå)}
> 
> Please note that the encoding is set to UTF-8 explicitely. Nonetheless, the a
> with overring in "Borgå" seems to mess up the title, which makes R think the
> \title field is missing. Changing the å to a circumvents this problem. I've also
> noticed that non-ASCII characters outside the title field do not hurt at all.

Add "Encoding: UTF-8" to the package DESCRIPTION file and remove from .Rd files,
as documented near the bottom of the section:

https://cran.r-project.org/doc/manuals/R-exts.html#The-DESCRIPTION-file

referenced by:

https://stackoverflow.com/questions/18080303/rd-files-with-different-encoding

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

Re: R does not handle package non-ASCII DESCRIPTION files properly

[Marco Atzeri]

Am 05.05.2020 um 16:55 schrieb Brian Inglis:
> On 2020-05-05 02:47, Markus Hoenicka wrote:
>> Am 2020-04-30 22:17, schrieb Marco Atzeri via Cygwin:
>>> Am 30.04.2020 um 17:28 schrieb Markus Hoenicka:
>>>> I've contacted the processx package maintainer on a problem reported here
>>>> previously (see https://cygwin.com/pipermail/cygwin/2020-April/244667.html).
>>>> He suggested to try the github version but that triggered a different type of
>>>> error which does not seem to be package-specific.
>>>>
>>>> 1st try: build from a local copy of the github package sources with default
>>>> locale settings on a German Windows 10 system
>>>>
>>>>> install.packages("/cygdrive/c/Temp/processx-master",repos=NULL,type="source")
> 
>>>> Installing package into ‘/usr/lib/R/site-library’
>>>> (as ‘lib’ is unspecified)
>>>> * installing *source* package ‘processx’ ...
>>>> ** using staged installation
>>>> Error : Invalid DESCRIPTION file
>>>>
>>>> Authors@R field gives no person with maintainer role, valid email
>>>> address and non-empty name.
>>>>
>>>> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
>>>> manual.
>>>>
>>>> ERROR: installing package DESCRIPTION failed for package ‘processx’
>>>> * removing ‘/usr/lib/R/site-library/processx’
>>>> Warning message:
>>>> In install.packages("/cygdrive/c/Temp/processx-master", repos = NULL,  :
>>>>     installation of package ‘/cygdrive/c/Temp/processx-master’ had non-zero
>>>> exit status
> 
>>> Noted. I will look on both the two issues but it could take a bit.
>>> I never looked at the build process of R packages in such details
> 
>> kind of "while you're at it": I've noticed one additional problem with non-ASCII
>> characters. This may or may not be related to the problem reported above.
>> Several packages throw an error during installation, in my case questionr,
>> readr, pls, and vegan. The output of the latter is shown here:
>>
>> installing to /usr/lib/R/site-library/00LOCK-vegan/00new/vegan/libs
>> ** R
>> ** data
>> ** inst
>> ** byte-compile and prepare package for lazy loading
>> ** help
>> *** installing help indices
>> Error in Rd_info(db[[i]]) :
>>    missing/empty \title field in
>> '/tmp/RtmprL23H1/R.INSTALLffe51949f2e2/vegan/man/sipoo.Rd'
>> Rd files must have a non-empty \title.
>> See chapter 'Writing R documentation' in manual 'Writing R Extensions'.
>> * removing ‘/usr/lib/R/site-library/vegan’
>>
>> The file in question (man/sipoo.Rd) starts like this:
>>
>> \encoding{UTF-8}
>> \name{sipoo}
>> \alias{sipoo}
>> \alias{sipoo.map}
>> \docType{data}
>> \title{ Birds in the Archipelago of Sipoo (Sibbo and Borgå)}
>>
>> Please note that the encoding is set to UTF-8 explicitely. Nonetheless, the a
>> with overring in "Borgå" seems to mess up the title, which makes R think the
>> \title field is missing. Changing the å to a circumvents this problem. I've also
>> noticed that non-ASCII characters outside the title field do not hurt at all.
> 
> Add "Encoding: UTF-8" to the package DESCRIPTION file and remove from .Rd files,
> as documented near the bottom of the section:
> 
> https://cran.r-project.org/doc/manuals/R-exts.html#The-DESCRIPTION-file
> 
> referenced by:
> 
> https://stackoverflow.com/questions/18080303/rd-files-with-different-encoding
> 

the original issue was in a package that was following the rule, so 
there is something at fault in the Cygwin R package reading UTF-8 files.

May be is the same root cause that is causing a single
fault in reading/writing during build test.

I am looking on the 4.0.0 before releasing but I am not a R user
so it takes time.

Regards
Marco