dd -- bs>=2G, error writing to /dev/null - possible bug? (win7; coreutils 8.26)

dd -- bs>=2G, error writing to /dev/null - possible bug? (win7; coreutils 8.26)

[Josef Frank]

Dear all,


dd utility has problems to write to /dev/null with bs>=2^31,
while bs<2^31 is ok.

Increasing bs further to 2^33 leads to extended error message

For description see below


jf




Steps to reproduce:


$ dd if=/dev/zero of=/dev/null bs=2147483647 count=1
1+0 records in
1+0 records out
2147483647 bytes (2.1 GB, 2.0 GiB) copied, 0.522774 s, 4.1 GB/s


$ dd if=/dev/zero of=/dev/null bs=2147483648 count=1
dd: error writing '/dev/null'
1+0 records in
0+0 records out
0 bytes copied, 0.517421 s, 0.0 kB/s


$ dd if=/dev/zero of=/dev/null bs=8589934591 count=1
dd: error writing '/dev/null'
1+0 records in
0+0 records out
0 bytes copied, 2.13474 s, 0.0 kB/s


$ dd if=/dev/zero of=/dev/null bs=8589934592 count=1
dd: error writing '/dev/null': No space left on device
1+0 records in
0+0 records out
0 bytes copied, 2.08004 s, 0.0 kB/s



In contrast the following lines (also coreutils 8.26) run fine (without
reporting any error):

head -c 2147483648 /dev/zero > /dev/null

head -c 8589934592 /dev/zero > /dev/null




Configuration:

Applies to several different machines with different amounts of main 
memory (up to 32GB, >24GB available), and sufficient free space on 
/var/tmp (100-180GB), all using Windows 7


$ uname -a
CYGWIN_NT-6.1 rab 2.7.0(0.306/5/3) 2017-02-12 13:18 x86_64 Cygwin


$ dd --version
dd (coreutils) 8.26
Packaged by Cygwin (8.26-2)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Paul Rubin, David MacKenzie, and Stuart Kemp.


$ cygcheck dd
Found: C:\cygwin64\bin\dd.exe
Found: C:\cygwin64\bin\dd.exe
C:\cygwin64\bin\dd.exe
   C:\cygwin64\bin\cygwin1.dll
     C:\windows\system32\KERNEL32.dll
       C:\windows\system32\API-MS-Win-Core-RtlSupport-L1-1-0.dll
       C:\windows\system32\ntdll.dll
       C:\windows\system32\KERNELBASE.dll
       C:\windows\system32\API-MS-Win-Core-ProcessThreads-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Heap-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Memory-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Handle-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Synch-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-File-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-IO-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-ThreadPool-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-LibraryLoader-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-NamedPipe-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Misc-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-SysInfo-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Localization-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-ProcessEnvironment-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-String-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Debug-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-ErrorHandling-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Fibers-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Util-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Core-Profile-L1-1-0.dll
       C:\windows\system32\API-MS-Win-Security-Base-L1-1-0.dll
   C:\cygwin64\bin\cygintl-8.dll
     C:\cygwin64\bin\cygiconv-2.dll

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: dd -- bs>=2G, error writing to /dev/null - possible bug? (win7; coreutils 8.26)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1011 bytes --]

On Mar 12 00:33, Josef Frank wrote:
> 
> Dear all,
> 
> 
> dd utility has problems to write to /dev/null with bs>=2^31,
> while bs<2^31 is ok.
> 
> Increasing bs further to 2^33 leads to extended error message
> 
> For description see below

Thanks for the testcase.  It's a combination of two bugs:

- At a crucial point in write(2), Cygwin used an int variable where
  it should have used a ssize_t variable.  This fouled up the return
  value for lengths >= 2^31.

- When writing to the Windows NUL device, NtWriteFile only returns the
  lower 32 bits od the number of bytes written.  This fouls up the
  return value of write(2) for lengths >= 2^32.

I fixed the first problem and created a workaround for the second.

I uploaded new developer snapshots to https://cygwin.de/snapshots/
Please give 'em a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: dd -- bs>=2G, error writing to /dev/null - possible bug? (win7; coreutils 8.26)

[Marco Atzeri]

On 12/03/2017 12:39, Corinna Vinschen wrote:
> On Mar 12 00:33, Josef Frank wrote:
>>
>> Dear all,
>>
>>
>> dd utility has problems to write to /dev/null with bs>=2^31,
>> while bs<2^31 is ok.
>>
>> Increasing bs further to 2^33 leads to extended error message
>>
>> For description see below
>
> Thanks for the testcase.  It's a combination of two bugs:
>
> - At a crucial point in write(2), Cygwin used an int variable where
>   it should have used a ssize_t variable.  This fouled up the return
>   value for lengths >= 2^31.
>
> - When writing to the Windows NUL device, NtWriteFile only returns the
>   lower 32 bits od the number of bytes written.  This fouls up the
>   return value of write(2) for lengths >= 2^32.
>
> I fixed the first problem and created a workaround for the second.
>
> I uploaded new developer snapshots to https://cygwin.de/snapshots/
> Please give 'em a try.

May be
https://cygwin.com/snapshots/

>
>
> Thanks,
> Corinna

Cheers
Marco


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: dd -- bs>=2G, error writing to /dev/null - possible bug? (win7; coreutils 8.26)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

On Mar 12 15:49, Marco Atzeri wrote:
> On 12/03/2017 12:39, Corinna Vinschen wrote:
> > On Mar 12 00:33, Josef Frank wrote:
> > > 
> > > Dear all,
> > > 
> > > 
> > > dd utility has problems to write to /dev/null with bs>=2^31,
> > > while bs<2^31 is ok.
> > > 
> > > Increasing bs further to 2^33 leads to extended error message
> > > 
> > > For description see below
> > 
> > Thanks for the testcase.  It's a combination of two bugs:
> > 
> > - At a crucial point in write(2), Cygwin used an int variable where
> >   it should have used a ssize_t variable.  This fouled up the return
> >   value for lengths >= 2^31.
> > 
> > - When writing to the Windows NUL device, NtWriteFile only returns the
> >   lower 32 bits od the number of bytes written.  This fouls up the
> >   return value of write(2) for lengths >= 2^32.
> > 
> > I fixed the first problem and created a workaround for the second.
> > 
> > I uploaded new developer snapshots to XXXXXXXXXXXXXXXXXXXXXXXXXXXX
> > Please give 'em a try.
> 
> May be
> https://cygwin.com/snapshots/

Oops, right :)


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]