[ANNOUNCEMENT] Updated: tar-1.33-3

[ANNOUNCEMENT] Updated: tar-1.33-3

[Achim Gratz]

GNU tar has been updated to the latest upstream release 1.33.

This is a new upstream release.  Upstream release notes can be found
alongside other documentation in /usr/share/doc/tar/.

ZStandard compression support is enabled in this release.


Homepage:
https://www.gnu.org/software/tar

GNU Tar provides the ability to create tar archives, as well as various
other kinds of manipulation. For example, you can use Tar on previously
created archives to extract files, to store additional files, or to
update or list files which were already stored.

Initially, tar archives were used to store files conveniently on
magnetic tape. The name "Tar" comes from this use; it stands for tape
archiver. Despite the utility's name, Tar can direct its output to
available devices, files, or other programs (using pipes), it can even
access remote devices or files (as archives).


-- 
              *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

If you want to unsubscribe from the cygwin-announce mailing list, look
at the "List-Unsubscribe: " tag in the email header of this message.
Send email to the address specified there. It will be in the format:

cygwin-announce-unsubscribe-you=yourdomain.com@cygwin.com

If you need more information on unsubscribing, start reading here:

http://sourceware.org/lists.html#unsubscribe-simple

Please read *all* of the information on unsubscribing that is available
starting at this URL.

tar 1.33 symlinks : Cannot change mode to...

[Vincent Rivière]

On 10/01/2021 à 14:50, Achim Gratz wrote:
> GNU tar has been updated to the latest upstream release 1.33.

Unfortunately it has reintroduced this old bug:
https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html

This happens when symlinks are located before the target in the source 
archive. The example given in the above link produces this output:

$ tar -xf directory.tar
tar: directory/1: Cannot change mode to rwxr-xr-x: No such file or directory
tar: directory/2: Cannot change mode to rwxr-xr-x: No such file or directory
tar: directory/3: Cannot change mode to rwxr-xr-x: No such file or directory
tar: Exiting with failure status due to previous errors

Note that files and symlinks are properly extracted, but $? is set to 2 so 
scripts can fail.

tar 1.32-2 : ok
tar 1.33-1 : bug

This needs to be fixed.

Regards,

-- 
Vincent Rivière

Re: tar 1.33 symlinks : Cannot change mode to...

[Achim Gratz]

Vincent Rivière writes:
> On 10/01/2021 à 14:50, Achim Gratz wrote:
>> GNU tar has been updated to the latest upstream release 1.33.
>
> Unfortunately it has reintroduced this old bug:
> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html

It's quite unlikely the same bug since there is actually a test for that.

> This happens when symlinks are located before the target in the source
> archive. The example given in the above link produces this output:
>
> $ tar -xf directory.tar
> tar: directory/1: Cannot change mode to rwxr-xr-x: No such file or directory
> tar: directory/2: Cannot change mode to rwxr-xr-x: No such file or directory
> tar: directory/3: Cannot change mode to rwxr-xr-x: No such file or directory
> tar: Exiting with failure status due to previous errors

Cannot reproduce.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: tar 1.33 symlinks : Cannot change mode to...

[Vincent Rivière]

On 13/01/2021 at 17:52, Ken Brown via Cygwin wrote:
> This turned out to be a Cygwin bug, which has now been fixed.

I upgraded again tar to version 1.34-1: fail.
Then I upgraded to cygwin 3.20.0-1 using official setup-x86.exe: pass!

Many thanks for this fix. Highly appreciated.

Kind regards,

-- 
Vincent Rivière

Re: tar 1.33 symlinks : Cannot change mode to...

[Michael Soegtrop]

> have you tried the Cygwin snapshot as suggested in:
> 
> https://sourceware.org/pipermail/cygwin/2021-January/247418.html

The snapshot is hard to use in headless CI systems because the provided 
archive contains a cygwin dll, so it cannot be patched from within 
cygwin and my experience with doing such things in PowerShell is limited.

Instead I am now restoring the old version using:

wget 
http://mirrors.kernel.org/sourceware/cygwin/x86/release/tar/tar-1.32-2.tar.xz 
-O /tmp/tar-1.32-2.tar.xz
tar xvf /tmp/tar-1.32-2.tar.xz -C /

which can be run from the cygwin console to restore the previous tar 
after cygwin setup.

I will try the snapshot later locally.

Best regards,

Michael

Re: tar 1.33 symlinks : Cannot change mode to...

[Michael Soegtrop]

Hi Marco,

> have you tried the Cygwin snapshot as suggested in:
> 
> https://sourceware.org/pipermail/cygwin/2021-January/247418.html

sorry, I somehow missed this message. I will try this and report if it 
works.

Thanks!

Best regards,

Michael

Re: tar 1.33 symlinks : Cannot change mode to...

[Marco Atzeri]

On 30.01.2021 15:58, Michael Soegtrop via Cygwin wrote:
> Dear Cygwin Team,
> 
> is there an update on this? Apparently tar is broken on Cygwin 32 since 
> the latest update (it works fine on 64 bit Cygwin) and afaik there is no 
> way to install specific package versions from the command line as would 
> be possible via the Setup UI, so I can't easily roll back to the 
> previous version of tar. IMHO this is a very severe issue - tar is 
> ubiquitous.
> 
> Best regards,
> 
> Michael

have you tried the Cygwin snapshot as suggested in:

https://sourceware.org/pipermail/cygwin/2021-January/247418.html

Regards
Marco

Re: tar 1.33 symlinks : Cannot change mode to...

[Michael Soegtrop]

Dear Cygwin Team,

is there an update on this? Apparently tar is broken on Cygwin 32 since 
the latest update (it works fine on 64 bit Cygwin) and afaik there is no 
way to install specific package versions from the command line as would 
be possible via the Setup UI, so I can't easily roll back to the 
previous version of tar. IMHO this is a very severe issue - tar is 
ubiquitous.

Best regards,

Michael

Re: tar 1.33 symlinks : Cannot change mode to...

[Ken Brown]

On 1/13/2021 4:07 PM, Morgan King via Cygwin wrote:
> On 13 January 2021 19:02, Corinna Vinschen via Cygwin wrote:
>> On Jan 13 11:52, Ken Brown via Cygwin wrote:
>>> On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
>>>> Hello,
>>>>
>>>> I am experiencing the same issue with tar and I am able to reproduce it
>> following the steps at:
>>>> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
>>>>
>>>> This issue only appears to occur when using 32-bit Cygwin, I am unable to
>>>> reproduce it using 64-bit Cygwin.
>>>
>>> This turned out to be a Cygwin bug, which has now been fixed.  As soon
>>> as Corinna has a chance to upload new snapshots to
>>> https://cygwin.com/snapshots/, you'll be able to test it.
>>
>> Snapshot is up!
>>
>>
>> Corinna
> 
> Hello,
> 
> I have downloaded the new snapshot and this fixes the issue.
> 
> Thank you very much for this.

Thanks for testing.

Ken

RE: tar 1.33 symlinks : Cannot change mode to...

[Morgan King]

On 13 January 2021 19:02, Corinna Vinschen via Cygwin wrote:
> On Jan 13 11:52, Ken Brown via Cygwin wrote:
> > On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
> > > Hello,
> > >
> > > I am experiencing the same issue with tar and I am able to reproduce it
> following the steps at:
> > > https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
> > >
> > > This issue only appears to occur when using 32-bit Cygwin, I am unable to
> > > reproduce it using 64-bit Cygwin.
> >
> > This turned out to be a Cygwin bug, which has now been fixed.  As soon
> > as Corinna has a chance to upload new snapshots to
> > https://cygwin.com/snapshots/, you'll be able to test it.
> 
> Snapshot is up!
> 
> 
> Corinna

Hello,

I have downloaded the new snapshot and this fixes the issue.

Thank you very much for this.

Regards,
Morgan

Re: tar 1.33 symlinks : Cannot change mode to...

[Corinna Vinschen]

On Jan 13 11:52, Ken Brown via Cygwin wrote:
> On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
> > Hello,
> > 
> > I am experiencing the same issue with tar and I am able to reproduce it following the steps at:
> > https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
> > 
> > This issue only appears to occur when using 32-bit Cygwin, I am unable to reproduce it using 64-bit Cygwin. I need to use 32-bit Cygwin since I use a 32-bit application which depends on Cygwin.
> 
> This turned out to be a Cygwin bug, which has now been fixed.  As soon as
> Corinna has a chance to upload new snapshots to
> https://cygwin.com/snapshots/, you'll be able to test it.

Snapshot is up!


Corinna

Re: tar 1.33 symlinks : Cannot change mode to...

[Ken Brown]

On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
> Hello,
> 
> I am experiencing the same issue with tar and I am able to reproduce it following the steps at:
> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
> 
> This issue only appears to occur when using 32-bit Cygwin, I am unable to reproduce it using 64-bit Cygwin. I need to use 32-bit Cygwin since I use a 32-bit application which depends on Cygwin.

This turned out to be a Cygwin bug, which has now been fixed.  As soon as 
Corinna has a chance to upload new snapshots to https://cygwin.com/snapshots/, 
you'll be able to test it.

Thanks for the report, and thanks especially for pointing out that the problem 
occurred only on 32-bit Cygwin.

Ken

Re: tar 1.33 symlinks : Cannot change mode to...

[Corinna Vinschen]

On Jan 12 12:49, Ken Brown via Cygwin wrote:
> On 1/12/2021 12:15 PM, Corinna Vinschen wrote:
> > On Jan 12 11:55, Ken Brown via Cygwin wrote:
> > > [...]
> > > This appears to be a bug in fstat in 32-bit Cygwin.  Here's what I'm seeing
> > > in gdb, using an unoptimized build of cygwin1.dll.  There is a call to
> > > fstatat in the tar source file gnu/fchmodat.c:87.  This ultimately leads to
> > > a call to fstat, whose definition in syscalls.cc is the following in the
> > > 32-bit case:
> > 
> > No, wait.  struct stat == struct _stat64 since Cygwin 1.5.0!
> > 
> > With Cygwin 1.5.0, the old entry point fstat was only retained to serve
> > old 32 bit applications buil.t under Cygwin 1.3.x or earlier.
> > 
> > Consequentially, newer 32 bit apps, built with Cygwin 1.5.0 or later,
> > are redirected to fstat64, see NEW_FUNCTIONS in Makefile.in.
> > 
> > If tar is actually calling fstat, something fishy is going on.
> 
> It's not tar that's calling fstat, it's Cygwin's fstatat at
> syscalls.cc:4855. This appears to be a bug that I introduced in commit
> 84252946.  I'll fix it.

Oh, right, this code should call fstat64, or resolve to an fhandler
and call its fstat method.

> Thanks for setting me straight.

Never mind, shit happens.


Corinna

Re: tar 1.33 symlinks : Cannot change mode to...

[Achim Gratz]

Ken Brown via Cygwin writes:
[…]
> I don't see how that could be expected to work, since several members
> of struct __stat32 have different sizes than the corresponding members
> of struct stat.  I wonder if it worked by accident in the past, but
> the problem is just showing up with newer gcc because of changes in
> how it pads the two structs?

I've just rebuilt tar-1.32 with gcc10 and see this error there too now.
I guess your finding saves me the bother of rolling back binutils and
gcc…

> Corinna, can you shed some light on this?

That'd be highly appreciated.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: tar 1.33 symlinks : Cannot change mode to...

[Ken Brown]

On 1/12/2021 12:15 PM, Corinna Vinschen wrote:
> On Jan 12 11:55, Ken Brown via Cygwin wrote:
>> On 1/11/2021 6:50 PM, Ken Brown via Cygwin wrote:
>>> On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
>>>> Hello,
>>>>
>>>> I am experiencing the same issue with tar and I am able to reproduce
>>>> it following the steps at:
>>>> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
>>>>
>>>> This issue only appears to occur when using 32-bit Cygwin, I am
>>>> unable to reproduce it using 64-bit Cygwin.
>>>
>>> I can confirm this.  Running the tar command under gdb shows a problem
>>> at gnu/fchmodat.c:94:
>>>
>>> 94            if (S_ISLNK (st.st_mode))
>>>
>>> Here st is a stat structure for directory/1, so S_ISLNK() should be
>>> true.  It is indeed true on 64-bit Cygwin but not on 32-bit Cygwin.
>>> Someone needs to look more closely and find out why this happened.  I'll
>>> try to do it tomorrow if no one beats me to it.
>>
>> This appears to be a bug in fstat in 32-bit Cygwin.  Here's what I'm seeing
>> in gdb, using an unoptimized build of cygwin1.dll.  There is a call to
>> fstatat in the tar source file gnu/fchmodat.c:87.  This ultimately leads to
>> a call to fstat, whose definition in syscalls.cc is the following in the
>> 32-bit case:
> 
> No, wait.  struct stat == struct _stat64 since Cygwin 1.5.0!
> 
> With Cygwin 1.5.0, the old entry point fstat was only retained to serve
> old 32 bit applications buil.t under Cygwin 1.3.x or earlier.
> 
> Consequentially, newer 32 bit apps, built with Cygwin 1.5.0 or later,
> are redirected to fstat64, see NEW_FUNCTIONS in Makefile.in.
> 
> If tar is actually calling fstat, something fishy is going on.

It's not tar that's calling fstat, it's Cygwin's fstatat at syscalls.cc:4855. 
This appears to be a bug that I introduced in commit 84252946.  I'll fix it. 
Thanks for setting me straight.

Ken

Re: tar 1.33 symlinks : Cannot change mode to...

[Corinna Vinschen]

On Jan 12 11:55, Ken Brown via Cygwin wrote:
> On 1/11/2021 6:50 PM, Ken Brown via Cygwin wrote:
> > On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
> > > Hello,
> > > 
> > > I am experiencing the same issue with tar and I am able to reproduce
> > > it following the steps at:
> > > https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
> > > 
> > > This issue only appears to occur when using 32-bit Cygwin, I am
> > > unable to reproduce it using 64-bit Cygwin.
> > 
> > I can confirm this.  Running the tar command under gdb shows a problem
> > at gnu/fchmodat.c:94:
> > 
> > 94            if (S_ISLNK (st.st_mode))
> > 
> > Here st is a stat structure for directory/1, so S_ISLNK() should be
> > true.  It is indeed true on 64-bit Cygwin but not on 32-bit Cygwin. 
> > Someone needs to look more closely and find out why this happened.  I'll
> > try to do it tomorrow if no one beats me to it.
> 
> This appears to be a bug in fstat in 32-bit Cygwin.  Here's what I'm seeing
> in gdb, using an unoptimized build of cygwin1.dll.  There is a call to
> fstatat in the tar source file gnu/fchmodat.c:87.  This ultimately leads to
> a call to fstat, whose definition in syscalls.cc is the following in the
> 32-bit case:

No, wait.  struct stat == struct _stat64 since Cygwin 1.5.0!

With Cygwin 1.5.0, the old entry point fstat was only retained to serve
old 32 bit applications buil.t under Cygwin 1.3.x or earlier.

Consequentially, newer 32 bit apps, built with Cygwin 1.5.0 or later,
are redirected to fstat64, see NEW_FUNCTIONS in Makefile.in.

If tar is actually calling fstat, something fishy is going on.

> After the call to stat64_to_stat32, buf looks like this:
> 
> (gdb) p/o *buf
> $22 = {st_dev = 026106753173, st_ino = 017510000040000120777, st_mode = 01001,
>   st_nlink = 01, st_uid = 013777346014, st_gid = 03415154434,
>   st_rdev = 013777334323, st_size = 0161040234413777346014, st_atim = {
>     tv_sec = 0200000, tv_nsec = 0}, st_mtim = {tv_sec = 01, tv_nsec = 0},
>   st_ctim = {tv_sec = 06533716, tv_nsec = 025000000}, st_blksize = 0,
>   st_blocks = 0, st_birthtim = {tv_sec = 0, tv_nsec = 03}}
> 
> Note that many of the values have been corrupted or shifted.

NHo, they are not.  The problme is jsut that buf is defined as
`struct stat' for API compatibility, but older (pre Cygwin-1.5.0)
32 bit apps expect the old 32 bit struct __stat32.  Therefore,
to evaluate the content of buf correctly, you have to cast it to
struct __stat32 * in GDB.  But... see above.


Corinna

Re: tar 1.33 symlinks : Cannot change mode to...

[Ken Brown]

On 1/11/2021 6:50 PM, Ken Brown via Cygwin wrote:
> On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
>> Hello,
>>
>> I am experiencing the same issue with tar and I am able to reproduce it 
>> following the steps at:
>> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
>>
>> This issue only appears to occur when using 32-bit Cygwin, I am unable to 
>> reproduce it using 64-bit Cygwin.
> 
> I can confirm this.  Running the tar command under gdb shows a problem at 
> gnu/fchmodat.c:94:
> 
> 94            if (S_ISLNK (st.st_mode))
> 
> Here st is a stat structure for directory/1, so S_ISLNK() should be true.  It is 
> indeed true on 64-bit Cygwin but not on 32-bit Cygwin.  Someone needs to look 
> more closely and find out why this happened.  I'll try to do it tomorrow if no 
> one beats me to it.

This appears to be a bug in fstat in 32-bit Cygwin.  Here's what I'm seeing in 
gdb, using an unoptimized build of cygwin1.dll.  There is a call to fstatat in 
the tar source file gnu/fchmodat.c:87.  This ultimately leads to a call to 
fstat, whose definition in syscalls.cc is the following in the 32-bit case:

extern "C" int
fstat (int fd, struct stat *buf)
{
   struct stat buf64;
   int ret = fstat64 (fd, &buf64);
   if (!ret)
     stat64_to_stat32 (&buf64, (struct __stat32 *) buf);
   return ret;
}

After the call to fstat64, buf64 looks like this:

(gdb) p/o buf64
$20 = {st_dev = 016465473173, st_ino = 01240000000006533716,
   st_mode = 0120777, st_nlink = 01, st_uid = 0601751, st_gid = 0601001,
   st_rdev = 0, st_size = 01, st_atim = {tv_sec = 013777346014,
     tv_nsec = 03415154434}, st_mtim = {tv_sec = 013777334323, tv_nsec = 0},
   st_ctim = {tv_sec = 013777346014, tv_nsec = 03421004710},
   st_blksize = 0200000, st_blocks = 0, st_birthtim = {tv_sec = 013777346014,
     tv_nsec = 03415154434}}

After the call to stat64_to_stat32, buf looks like this:

(gdb) p/o *buf
$22 = {st_dev = 026106753173, st_ino = 017510000040000120777, st_mode = 01001,
   st_nlink = 01, st_uid = 013777346014, st_gid = 03415154434,
   st_rdev = 013777334323, st_size = 0161040234413777346014, st_atim = {
     tv_sec = 0200000, tv_nsec = 0}, st_mtim = {tv_sec = 01, tv_nsec = 0},
   st_ctim = {tv_sec = 06533716, tv_nsec = 025000000}, st_blksize = 0,
   st_blocks = 0, st_birthtim = {tv_sec = 0, tv_nsec = 03}}

Note that many of the values have been corrupted or shifted.  (A few are 
expected to change.)  I think the problem is the cast in the call

   stat64_to_stat32 (&buf64, (struct __stat32 *) buf);

I don't see how that could be expected to work, since several members of struct 
__stat32 have different sizes than the corresponding members of struct stat.  I 
wonder if it worked by accident in the past, but the problem is just showing up 
with newer gcc because of changes in how it pads the two structs?

Corinna, can you shed some light on this?

Ken

Re: tar 1.33 symlinks : Cannot change mode to...

[Marco Atzeri]

On 12.01.2021 09:18, ASSI wrote:
> Ken Brown via Cygwin writes:
>> I can confirm this.  Running the tar command under gdb shows a problem
>> at gnu/fchmodat.c:94:
>>
>> 94            if (S_ISLNK (st.st_mode))
> 
> That should be foolproof, but apparently isn't.

> 
> Unfortunately the handful of commits that might change the behaviour for
> this situation elsewhere are knotty, so bisecting will not really
> help much I'm afraid.
> 
> 
> 
> Regards,
> Achim.
> 

rebuild 1.32 to verify that is not a change in Cygwin definitions,
Python rebuild was it by that.

Re: tar 1.33 symlinks : Cannot change mode to...

[ASSI]

Ken Brown via Cygwin writes:
> I can confirm this.  Running the tar command under gdb shows a problem
> at gnu/fchmodat.c:94:
>
> 94            if (S_ISLNK (st.st_mode))

That should be foolproof, but apparently isn't.

> Here st is a stat structure for directory/1, so S_ISLNK() should be
> true.  It is indeed true on 64-bit Cygwin but not on 32-bit Cygwin.
> Someone needs to look more closely and find out why this happened.
> I'll try to do it tomorrow if no one beats me to it.

I might remember wrongly, but I don't think that code was changed
between the two versions.  What's the call stack at that point?

Unfortunately the handful of commits that might change the behaviour for
this situation elsewhere are knotty, so bisecting will not really
help much I'm afraid.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: tar 1.33 symlinks : Cannot change mode to...

[Ken Brown]

On 1/11/2021 2:57 PM, Morgan King via Cygwin wrote:
> Hello,
> 
> I am experiencing the same issue with tar and I am able to reproduce it following the steps at:
> https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html
> 
> This issue only appears to occur when using 32-bit Cygwin, I am unable to reproduce it using 64-bit Cygwin.

I can confirm this.  Running the tar command under gdb shows a problem at 
gnu/fchmodat.c:94:

94            if (S_ISLNK (st.st_mode))

Here st is a stat structure for directory/1, so S_ISLNK() should be true.  It is 
indeed true on 64-bit Cygwin but not on 32-bit Cygwin.  Someone needs to look 
more closely and find out why this happened.  I'll try to do it tomorrow if no 
one beats me to it.

Ken

Re: tar 1.33 symlinks : Cannot change mode to...

[Morgan King]

Hello,

I am experiencing the same issue with tar and I am able to reproduce it following the steps at:
https://lists.gnu.org/archive/html/bug-tar/2011-08/msg00006.html

This issue only appears to occur when using 32-bit Cygwin, I am unable to reproduce it using 64-bit Cygwin. I need to use 32-bit Cygwin since I use a 32-bit application which depends on Cygwin.

This should be fixed.

Regards,

Morgan