From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 56398 invoked by alias); 24 May 2017 10:14:38 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 56364 invoked by uid 89); 24 May 2017 10:14:37 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=1.9 required=5.0 tests=AWL,BAYES_50,FREEMAIL_FROM,LIKELY_SPAM_SUBJECT,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=no version=3.3.2 spammy=ronald, Ronald, UD:C.UTF-8, CUTF8 X-HELO: out1-smtp.messagingengine.com Received: from out1-smtp.messagingengine.com (HELO out1-smtp.messagingengine.com) (66.111.4.25) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 24 May 2017 10:14:36 +0000 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 6195620969 for ; Wed, 24 May 2017 06:14:38 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute6.internal (MEProxy); Wed, 24 May 2017 06:14:38 -0400 X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 42B07626D8; Wed, 24 May 2017 06:14:38 -0400 (EDT) Message-Id: <1495620878.1850033.986938960.3CBADAA7@webmail.messagingengine.com> From: Ronald Fischer To: cygwin@cygwin.com MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" In-Reply-To: <1221683706.20170524130359@yandex.ru> References: <1495612367.2760331.986814392.79C77EB2@webmail.messagingengine.com> <1221683706.20170524130359@yandex.ru> Subject: Re: Bug: grep behaves incorrectly under the locale C.UTF-8, if a file contains Umlaut characters Date: Wed, 24 May 2017 10:20:00 -0000 X-IsSubscribed: yes X-SW-Source: 2017-05/txt/msg00382.txt.bz2 > > If I grep the file using, say, > > > $ grep . X >Y > > > (i.e. select every non-empty line and write the result to Y), this works > > fine, if LANG is set to one of: UTF-8, C, C.de_DE, C.en_EN, en_EN, > > de_DE. > > > However, if LANG is set to C.UTF-8, two things happen: > > > - grep classifies the file as binary file and produces the error message > > "Binary file X matches" > > This is an intended behavior, upstream decision since mid-2015, I recall. Might be, but this still does not explain the issues 1., 2. and 3., which I layed out in detail below. Note that never said that the fact, that grep classifies certain characters as binary, would by itself a bug. Or is the intended behaviour, that with C.UTF-8 (and *only* with this setting), the resulting standard output of grep is interspersed with "Binary file matches" lines? If this is the case, I really would like to se a justification for this decision. > > > - Both the grepped lines (i.e. in our example the non-empty lines) AND > > the error message end up in the standard output (i.e. in file Y). > > > IMO, there are several problems with this: > > > 1. It's hard to see, why an umlaut character makes the file X binary > > under encoding C.UTF-8, but not under encoding UTF-8 or C.en_EN > > > 2. If grep classifies a file as binary, I think the desired behaviour > > would be to NOT produce any output, unless the -a flag has been > > supplied. > > > 3. If grep writes a message "Binary file ... matches", this message > > should go to stderr, not stdout. The stdout is supposed to contain only > > a subset of the input lines. > > Ronald -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple