Re: Pipes truncating data in cygwin from main and cygwin-3_4-branch

[Takashi Yano <takashi.yano@nifty.ne.jp>]

On Mon, 14 Aug 2023 20:51:39 +0900
キャロウ　マーク wrote:
> I have a problem that looks like pipes are truncating data when I cat a file to my program’s stdin. A simple `cat foo | cat > bar` works fine. bar ends up identical to foo. It is more complicated than that. My application is doing this
> std::stringstream buffer;
> buffer << std::cin.rdbuf();
> std::istream* isp = &buffer;
> Initial reads after this work fine. Once the app has read everything up to the payload data in the file, it does
> 
> off_t dataStart = (off_t)(isp->tellg());
> isp->seekg(0, ios_base::end);
> off_t dataEnd = (off_t)(isp->tellg());
> dataSizeInFile = dataEnd - dataStart;
> The tellg result shows the size is significantly less than the actual file data. 43k less in a 170k file. It is seemingly being truncated somewhere.
> 
> Later the app does
> 
>    isp->seekg(0);
>    std::streambuf* _streambuf = (isp->rdbuf());
> and starts reading from _streambuf. All data read from _streambuf is gibberish.
> 
> The application code makes no distinction between a pipe and stdin redirection from a file. It just uses std::cin. stdin redirection still works.
> 
> I created a minimal reproducer. More on that in a moment.
> 
> I first encountered this in Git for Windows 2.41.0. I had no problem in previous versions. I reported this to the Git for Windows project. See https://github.com/git-for-windows/git/issues/4464. You can find the minimal reproducer over there. It consists of 2 parts, a script and a small c++ program. The script finds the size of the target file then cats it to the test program passing the file size as a command line option. The test program does what I have described above and compares the file size determined from the seek to the end with the provided size.
> 
> A G4W project member reports that the problem reproduces on vanilla Cygwin in the branches mentioned in the subject and that G4W and MSYS2 are on the cygwin-3_4-branch release train. He recommends reporting the bug to you, You can find his(?) full comment here <https://github.com/git-for-windows/git/issues/4464#issuecomment-1671137446>.

Your test case does not work in command prompt as well.

Try
type testfile | test-pipe sizeoftestfile
in command prompt. It will fail.

New pipe implementation since cygwin 3.4.x provides the pipes
more similar to pipe in command prompt for non-cygwin apps.

Since your test case is compiled with cl.exe, it is non-cygwin
apps.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>