[RFC] DT_WHT

[RFC] DT_WHT

[Rical Jasan]

There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
is defined in dirent/dirent.h and appears to be used once in io/fts.c:

#if defined FTS_WHITEOUT && 0
                if (dp->d_type == DT_WHT)
                        p->fts_flags |= FTS_ISW;
#endif

For an entry in the manual, I was thinking something along the lines of:

"This macro is obsolete and has no effect.  It was used to..."

but I'm not sure what to say about what it was used for.

Alternatively, could/should it be removed?  It looks like that might
extend beyond just DT_WHT, as FTS_WHITEOUT also appears to be a NOOP:

$ grep -rFn FTS_WHITEOUT .
./io/fts.h:75:#define FTS_WHITEOUT	0x0080		/* return whiteout information */
./io/fts.c:628:#if defined FTS_WHITEOUT && 0
./io/fts.c:629:	if (ISSET(FTS_WHITEOUT))
./io/fts.c:772:#if defined FTS_WHITEOUT && 0
./io/fts.c:881:#if defined FTS_WHITEOUT && 0

Lastly, I can't find a reference to it in any of the standards I've
managed to obtain so far (not to say that's exhaustive at all).  Is this
also BSD?

Thank you,
Rical

Re: [RFC] DT_WHT

[Carlos O'Donell]

On 02/21/2018 02:16 AM, Rical Jasan wrote:
> There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
> is defined in dirent/dirent.h and appears to be used once in io/fts.c:
> 
> #if defined FTS_WHITEOUT && 0
>                 if (dp->d_type == DT_WHT)
>                         p->fts_flags |= FTS_ISW;
> #endif
> 
> For an entry in the manual, I was thinking something along the lines of:
> 
> "This macro is obsolete and has no effect.  It was used to..."
> 
> but I'm not sure what to say about what it was used for.
> 
> Alternatively, could/should it be removed?  It looks like that might
> extend beyond just DT_WHT, as FTS_WHITEOUT also appears to be a NOOP:
> 
> $ grep -rFn FTS_WHITEOUT .
> ./io/fts.h:75:#define FTS_WHITEOUT	0x0080		/* return whiteout information */
> ./io/fts.c:628:#if defined FTS_WHITEOUT && 0
> ./io/fts.c:629:	if (ISSET(FTS_WHITEOUT))
> ./io/fts.c:772:#if defined FTS_WHITEOUT && 0
> ./io/fts.c:881:#if defined FTS_WHITEOUT && 0
> 
> Lastly, I can't find a reference to it in any of the standards I've
> managed to obtain so far (not to say that's exhaustive at all).  Is this
> also BSD?

Does this help?

https://www.sourceware.org/ml/libc-alpha/2012-04/msg00298.html

I think it should be entirely removed, along with the removal
of FTS_WHITEOUT.

-- 
Cheers,
Carlos.

Re: [RFC] DT_WHT

[Ben Hutchings]

On Wed, 2018-02-21 at 02:16 -0800, Rical Jasan wrote:
> There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
> is defined in dirent/dirent.h and appears to be used once in io/fts.c:
> 
> #if defined FTS_WHITEOUT && 0
>                 if (dp->d_type == DT_WHT)
>                         p->fts_flags |= FTS_ISW;
> #endif
> 
> For an entry in the manual, I was thinking something along the lines of:
> 
> "This macro is obsolete and has no effect.  It was used to..."
> 
> but I'm not sure what to say about what it was used for.
[...]
> Lastly, I can't find a reference to it in any of the standards I've
> managed to obtain so far (not to say that's exhaustive at all).  Is this
> also BSD?

Some Linux filesystems support white-out directory entries for use with
union filesystems.  At least the coda and xfs implementations of
getdents() appear to be capable of setting d_type to DT_WHT.

Ben.

-- 
Ben Hutchings
Software Developer, Codethink Ltd.

Re: [RFC] DT_WHT

[Rical Jasan]

On 03/01/2018 08:24 AM, Ben Hutchings wrote:
> On Wed, 2018-02-21 at 02:16 -0800, Rical Jasan wrote:
>> There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
>> is defined in dirent/dirent.h and appears to be used once in io/fts.c:
>>
>> #if defined FTS_WHITEOUT && 0
>>                 if (dp->d_type == DT_WHT)
>>                         p->fts_flags |= FTS_ISW;
>> #endif
>>
>> For an entry in the manual, I was thinking something along the lines of:
>>
>> "This macro is obsolete and has no effect.  It was used to..."
>>
>> but I'm not sure what to say about what it was used for.
> [...]
>> Lastly, I can't find a reference to it in any of the standards I've
>> managed to obtain so far (not to say that's exhaustive at all).  Is this
>> also BSD?
> 
> Some Linux filesystems support white-out directory entries for use with
> union filesystems.  At least the coda and xfs implementations of
> getdents() appear to be capable of setting d_type to DT_WHT.

The code as-is isn't used, though.  Other "#if ... 0" blocks also use
some apparently whiteout-related macros that aren't defined anywhere.

Do you think instead of removing the unused code it should be changed to
actually do something?

Thanks,
Rical

Re: [RFC] DT_WHT

[Ben Hutchings]

On Thu, 2018-03-01 at 15:02 -0800, Rical Jasan wrote:
> On 03/01/2018 08:24 AM, Ben Hutchings wrote:
> > On Wed, 2018-02-21 at 02:16 -0800, Rical Jasan wrote:
> > > There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
> > > is defined in dirent/dirent.h and appears to be used once in io/fts.c:
> > > 
> > > #if defined FTS_WHITEOUT && 0
> > >                 if (dp->d_type == DT_WHT)
> > >                         p->fts_flags |= FTS_ISW;
> > > #endif
> > > 
> > > For an entry in the manual, I was thinking something along the lines of:
> > > 
> > > "This macro is obsolete and has no effect.  It was used to..."
> > > 
> > > but I'm not sure what to say about what it was used for.
> > 
> > [...]
> > > Lastly, I can't find a reference to it in any of the standards I've
> > > managed to obtain so far (not to say that's exhaustive at all).  Is this
> > > also BSD?
> > 
> > Some Linux filesystems support white-out directory entries for use with
> > union filesystems.  At least the coda and xfs implementations of
> > getdents() appear to be capable of setting d_type to DT_WHT.
> 
> The code as-is isn't used, though.  Other "#if ... 0" blocks also use
> some apparently whiteout-related macros that aren't defined anywhere.
> 
> Do you think instead of removing the unused code it should be changed to
> actually do something?

I have no opinion on that code, but it sounded like you proposed to
remove the macro definition too.  I wanted to point out that the macro
is meaningful and may be useful to applications.

Ben.

-- 
Ben Hutchings
Software Developer, Codethink Ltd.

Re: [RFC] DT_WHT

[Florian Weimer]

On 03/05/2018 02:50 AM, Ben Hutchings wrote:
> I have no opinion on that code, but it sounded like you proposed to
> remove the macro definition too.  I wanted to point out that the macro
> is meaningful and may be useful to applications.

Useful in what sense?  What are its semantics?

Thanks,
Florian

Re: [RFC] DT_WHT

[Rical Jasan]

On 03/04/2018 05:50 PM, Ben Hutchings wrote:
> On Thu, 2018-03-01 at 15:02 -0800, Rical Jasan wrote:
>> On 03/01/2018 08:24 AM, Ben Hutchings wrote:
>>> On Wed, 2018-02-21 at 02:16 -0800, Rical Jasan wrote:
>>>> There is one undocumented DT_* macro in manual/filesys.texi: DT_WHT.  It
>>>> is defined in dirent/dirent.h and appears to be used once in io/fts.c:
>>>>
>>>> #if defined FTS_WHITEOUT && 0
>>>>                 if (dp->d_type == DT_WHT)
>>>>                         p->fts_flags |= FTS_ISW;
>>>> #endif
>>>>
>>>> For an entry in the manual, I was thinking something along the lines of:
>>>>
>>>> "This macro is obsolete and has no effect.  It was used to..."
>>>>
>>>> but I'm not sure what to say about what it was used for.
>>>
>>> [...]
>>>> Lastly, I can't find a reference to it in any of the standards I've
>>>> managed to obtain so far (not to say that's exhaustive at all).  Is this
>>>> also BSD?
>>>
>>> Some Linux filesystems support white-out directory entries for use with
>>> union filesystems.  At least the coda and xfs implementations of
>>> getdents() appear to be capable of setting d_type to DT_WHT.
>>
>> The code as-is isn't used, though.  Other "#if ... 0" blocks also use
>> some apparently whiteout-related macros that aren't defined anywhere.
>>
>> Do you think instead of removing the unused code it should be changed to
>> actually do something?
> 
> I have no opinion on that code, but it sounded like you proposed to
> remove the macro definition too.  I wanted to point out that the macro
> is meaningful and may be useful to applications.

It's a complete NOOP, though, so why should we provide it?

Rical

Re: [RFC] DT_WHT

[Ben Hutchings]

On Mon, 2018-03-05 at 10:47 +0100, Florian Weimer wrote:
> On 03/05/2018 02:50 AM, Ben Hutchings wrote:
> > I have no opinion on that code, but it sounded like you proposed to
> > remove the macro definition too.  I wanted to point out that the macro
> > is meaningful and may be useful to applications.
> 
> Useful in what sense?  What are its semantics?

A white-out entry is a placeholder written to the upper layer of a
union filesystem that hides an entry in the lower layer.

You'll never see these directory entries when looking at the union
filesystem, but can do if you look at the upper layer directly.

Ben.

-- 
Ben Hutchings
Software Developer, Codethink Ltd.

Re: [RFC] DT_WHT

[Ben Hutchings]

On Mon, 2018-03-05 at 02:49 -0800, Rical Jasan wrote:
> On 03/04/2018 05:50 PM, Ben Hutchings wrote:
> > On Thu, 2018-03-01 at 15:02 -0800, Rical Jasan wrote:
> > > On 03/01/2018 08:24 AM, Ben Hutchings wrote:
> > > > On Wed, 2018-02-21 at 02:16 -0800, Rical Jasan wrote:
> > > > > There is one undocumented DT_* macro in manual/filesys.texi:
> > > > > DT_WHT.  It
> > > > > is defined in dirent/dirent.h and appears to be used once in io/fts.c:
> > > > > 
> > > > > #if defined FTS_WHITEOUT && 0
> > > > >                 if (dp->d_type == DT_WHT)
> > > > >                         p->fts_flags |= FTS_ISW;
> > > > > #endif
> > > > > 
> > > > > For an entry in the manual, I was thinking something along the lines of:
> > > > > 
> > > > > "This macro is obsolete and has no effect.  It was used to..."
> > > > > 
> > > > > but I'm not sure what to say about what it was used for.
> > > > 
> > > > [...]
> > > > > Lastly, I can't find a reference to it in any of the standards I've
> > > > > managed to obtain so far (not to say that's exhaustive at all).  Is this
> > > > > also BSD?
> > > > 
> > > > Some Linux filesystems support white-out directory entries for use with
> > > > union filesystems.  At least the coda and xfs implementations of
> > > > getdents() appear to be capable of setting d_type to DT_WHT.
> > > 
> > > The code as-is isn't used, though.  Other "#if ... 0" blocks also use
> > > some apparently whiteout-related macros that aren't defined anywhere.
> > > 
> > > Do you think instead of removing the unused code it should be changed to
> > > actually do something?
> > 
> > I have no opinion on that code, but it sounded like you proposed to
> > remove the macro definition too.  I wanted to point out that the macro
> > is meaningful and may be useful to applications.
> 
> It's a complete NOOP, though, so why should we provide it?

For the same reason glibc defines the other DT_* macros.

Ben.

-- 
Ben Hutchings
Software Developer, Codethink Ltd.

Re: [RFC] DT_WHT

[Florian Weimer]

On 03/05/2018 05:20 PM, Ben Hutchings wrote:
> On Mon, 2018-03-05 at 10:47 +0100, Florian Weimer wrote:
>> On 03/05/2018 02:50 AM, Ben Hutchings wrote:
>>> I have no opinion on that code, but it sounded like you proposed to
>>> remove the macro definition too.  I wanted to point out that the macro
>>> is meaningful and may be useful to applications.
>>
>> Useful in what sense?  What are its semantics?
> 
> A white-out entry is a placeholder written to the upper layer of a
> union filesystem that hides an entry in the lower layer.
> 
> You'll never see these directory entries when looking at the union
> filesystem, but can do if you look at the upper layer directly.

That's not documented anywhere.  The kernel never sets this flag.  The 
Coda file system simply copies it from the network, and I cannot find 
the current server code anywhere.

For all we know, DT_WHT might be visible on the union file system, and 
applications have to skip those entries manually if they want to avoid 
“file not found” errors.  People assumed that inode 0 had this property 
(but this wasn't really true).

Furthermore, for true whiteout entry support, I'd expect an S_IFWHT mode 
flag, but that's just not there in the upstream kernel.

Thanks,
Florian