/usr/bin/fallocate -v -d -l 2 myfile takes forever

/usr/bin/fallocate -v -d -l 2 myfile takes forever

[Cedric Blancher]

Good morning!

rm -f myfile
touch myfile
/usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Mark Geisert]

On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
> Good morning!
> 
> rm -f myfile
> touch myfile
> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
                              ^ length goes here, before filename

I believe you need to
     chattr +S myfile
after the 'touch' to make the file sparsifiable.

Also, I'm unsure if '-d' makes sense when initially allocating a sparse 
file.  If I leave it off, fallocate does the right thing.

I'm separately investigating the 'takes forever' report.

..mark

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Brian Inglis]

On 2024-01-28 23:44, Cedric Blancher via Cygwin wrote:
> rm -f myfile
> touch myfile
> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5

It takes much longer without Cygwin 3.5!

Your fallocate(1) option -l has no length numeric argument and no -o offset 
numeric argument to specify the start of the range.

Takes no time on my system when used properly:

$ uname -srvmo
CYGWIN_NT-10.0-19045 3.5.0-0.613.g2c5433e5da82.x86_64 2024-01-24 15:14 UTC 
x86_64 Cygwin
$ smartctl -i /dev/sda
smartctl 7.4 2023-08-01 r5530 [x86_64-pc-cygwin-w10-22H2] (cygwin-7.4-1)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

Warning: Limited functionality due to missing admin rights
=== START OF INFORMATION SECTION ===
Model Family:     Seagate BarraCuda 3.5 (CMR)
Device Model:     ST1000DM010-2EP102
Serial Number:    W9AAWPXN
Firmware Version: CC43
Device is:        In smartctl database 7.3/5528
ATA Version is:   [No Information Found]
Local Time is:    Mon Jan 29 01:11:59 2024 MST
SMART support is: Available - device has SMART capability.
                   Enabled status cached by OS, trying SMART RETURN STATUS cmd.
SMART support is: Enabled

$ rm -f myfile; touch myfile; llgo myfile
removed 'myfile'
-rw-r--r-- 1 0 Jan 29 00:50 myfile
$ time fallocate -v -o 0 -l 64MiB myfile; llgo myfile

real    0m0.049s
user    0m0.000s
sys     0m0.015s
-rw-r--r-- 1 64M Jan 29 00:50 myfile
$ time fallocate -v -o 0 -l 64MiB -d myfile; llgo myfile
myfile: 64 MiB (67108864 bytes) converted to sparse holes.

real    0m0.146s
user    0m0.015s
sys     0m0.078s
-rw-r--r-- 1 64M Jan 29 00:50 myfile

Now perhaps the fallocate(1) design, option checking, diagnostics, defaults, and 
output, could use some work: using SI multipliers for data sizes (see ls --si!)

Perhaps show some useful commands and output that demonstrate your claim, 
including the actual test release used from `uname -srvmo` and provide some 
information about your drive (type, speed, size) and system, like output from 
`cygcheck -hrsv` attached as a text file?

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Mark Geisert]

On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote:
> On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
>> Good morning!
>>
>> rm -f myfile
>> touch myfile
>> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
>                               ^ length goes here, before filename
> 
> I believe you need to
>      chattr +S myfile
> after the 'touch' to make the file sparsifiable.
> 
> Also, I'm unsure if '-d' makes sense when initially allocating a sparse 
> file.  If I leave it off, fallocate does the right thing.
> 
> I'm separately investigating the 'takes forever' report.

It's taking forever because it's in an infinite loop looking for data 
followed by a hole, which is not present in a new file being created. 
This is due to the '-d' option being specified.  fallocate ought to 
diagnose this condition rather than looping forever, but here we are.

It's possible we have an old fallocate from an old util-linux package. 
I'll try to investigate this further when I have time.

..mark

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Cedric Blancher]

On Mon, 29 Jan 2024 at 09:33, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
>
> On 2024-01-28 23:44, Cedric Blancher via Cygwin wrote:
> > rm -f myfile
> > touch myfile
> > /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
>
> It takes much longer without Cygwin 3.5!
>
> Your fallocate(1) option -l has no length numeric argument and no -o offset
> numeric argument to specify the start of the range.

Apologies, I typed this from a screenshot, and made a mistake.
The line should be:
/usr/bin/fallocate -v -d -l 2 myfile
like in the Subject

Offset is 0 i this case, because no -o was given

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Cedric Blancher]

On Mon, 29 Jan 2024 at 10:46, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
>
> On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote:
> > On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
> >> Good morning!
> >>
> >> rm -f myfile
> >> touch myfile
> >> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
> >                               ^ length goes here, before filename
> >
> > I believe you need to
> >      chattr +S myfile
> > after the 'touch' to make the file sparsifiable.
> >
> > Also, I'm unsure if '-d' makes sense when initially allocating a sparse
> > file.  If I leave it off, fallocate does the right thing.
> >
> > I'm separately investigating the 'takes forever' report.
>
> It's taking forever because it's in an infinite loop looking for data
> followed by a hole, which is not present in a new file being created.
> This is due to the '-d' option being specified.  fallocate ought to
> diagnose this condition rather than looping forever, but here we are.
>
> It's possible we have an old fallocate from an old util-linux package.
> I'll try to investigate this further when I have time.
>

Any update on this issue?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Mark Geisert]

On 3/5/2024 12:46 AM, Cedric Blancher via Cygwin wrote:
> On Mon, 29 Jan 2024 at 10:46, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
>>
>> On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote:
>>> On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
>>>> Good morning!
>>>>
>>>> rm -f myfile
>>>> touch myfile
>>>> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
>>>                                ^ length goes here, before filename
>>>
>>> I believe you need to
>>>       chattr +S myfile
>>> after the 'touch' to make the file sparsifiable.
>>>
>>> Also, I'm unsure if '-d' makes sense when initially allocating a sparse
>>> file.  If I leave it off, fallocate does the right thing.
>>>
>>> I'm separately investigating the 'takes forever' report.
>>
>> It's taking forever because it's in an infinite loop looking for data
>> followed by a hole, which is not present in a new file being created.
>> This is due to the '-d' option being specified.  fallocate ought to
>> diagnose this condition rather than looping forever, but here we are.
>>
>> It's possible we have an old fallocate from an old util-linux package.
>> I'll try to investigate this further when I have time.
>>
> 
> Any update on this issue?

Thanks for the reminder. I'll check repology to see if we can maybe 
support a newer util-linux. No guarantees its fallocate is any better, 
but lets first see if it builds at all. Give me a couple days, please.

..mark

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Mark Geisert]

On 3/5/2024 12:46 AM, Cedric Blancher via Cygwin wrote:
> On Mon, 29 Jan 2024 at 10:46, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
>>
>> On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote:
>>> On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
>>>> Good morning!
>>>>
>>>> rm -f myfile
>>>> touch myfile
>>>> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5
>>>                                ^ length goes here, before filename
>>>
>>> I believe you need to
>>>       chattr +S myfile
>>> after the 'touch' to make the file sparsifiable.
>>>
>>> Also, I'm unsure if '-d' makes sense when initially allocating a sparse
>>> file.  If I leave it off, fallocate does the right thing.
>>>
>>> I'm separately investigating the 'takes forever' report.
>>
>> It's taking forever because it's in an infinite loop looking for data
>> followed by a hole, which is not present in a new file being created.
>> This is due to the '-d' option being specified.  fallocate ought to
>> diagnose this condition rather than looping forever, but here we are.
>>
>> It's possible we have an old fallocate from an old util-linux package.
>> I'll try to investigate this further when I have time.
>>
> 
> Any update on this issue?

Now that I've reviewed the whole email thread, I don't believe we ever 
received feedback from you on whether leaving off the '-d' option and 
doing the 'chattr +S' made any difference.

..mark

Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever

[Mark Geisert]

On 3/5/2024 2:43 AM, Mark Geisert via Cygwin wrote:
> On 3/5/2024 12:46 AM, Cedric Blancher via Cygwin wrote:
>> On Mon, 29 Jan 2024 at 10:46, Mark Geisert via Cygwin 
>> <cygwin@cygwin.com> wrote:
>>>
>>> On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote:
>>>> On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote:
>>>>> Good morning!
>>>>>
>>>>> rm -f myfile
>>>>> touch myfile
>>>>> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 
>>>>> 3.5
>>>>                                ^ length goes here, before filename
>>>>
>>>> I believe you need to
>>>>       chattr +S myfile
>>>> after the 'touch' to make the file sparsifiable.
>>>>
>>>> Also, I'm unsure if '-d' makes sense when initially allocating a sparse
>>>> file.  If I leave it off, fallocate does the right thing.
>>>>
>>>> I'm separately investigating the 'takes forever' report.
>>>
>>> It's taking forever because it's in an infinite loop looking for data
>>> followed by a hole, which is not present in a new file being created.
>>> This is due to the '-d' option being specified.  fallocate ought to
>>> diagnose this condition rather than looping forever, but here we are.
>>>
>>> It's possible we have an old fallocate from an old util-linux package.
>>> I'll try to investigate this further when I have time.
>>>
>>
>> Any update on this issue?
> 
> Thanks for the reminder. I'll check repology to see if we can maybe 
> support a newer util-linux. No guarantees its fallocate is any better, 
> but lets first see if it builds at all. Give me a couple days, please.

A newer util-linux package has been uploaded. I find that its fallocate 
operates identically to the earlier one for your example.  So, to 
summarize, when allocating a brand new file:
   1) don't specify '-d' option,
   2) after creating file with 'touch', do 'chattr +S' on it,
   3) using fallocate on the file without step 2 won't sparsify it.

It's easier to see fallocate is working properly by giving it a file 
size large enough to make a difference by sparsifying. My testing shows 
that specifying '-l 2m' to create a 2MB file makes the file occupy 64 
blocks if it's sparsified, 2048 blocks if it isn't. Use 'stat <file>' to 
get the block size of the file. 'ls -l' will show 2MB either way.
HTH,

..mark