Incorrect Python errors when using os.remove to delete a directory

Incorrect Python errors when using os.remove to delete a directory

[Adam Dinwoodie]

If I use os.remove in Python to remove a directory, I expect it to fail
with an OSError on Python2 or a IsADirectoryError on Python3.  On
Python2, I get OSError, but with the wrong error code, whereas on
Python3 I get completely the wrong exception.

Simple testcases:

    $ rm -rf testdir && mkdir testdir && python -c 'import os; os.remove("testdir")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    OSError: [Errno 1] Operation not permitted: 'testdir'

    $ rm -rf testdir && mkdir testdir && python3 -c 'import os; os.remove("testdir")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    PermissionError: [Errno 1] Operation not permitted: 'testdir'

On my handy CentOS box, I see the following instead:

    $ rm -rf testdir && mkdir testdir && python -c 'import os; os.remove("testdir")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    OSError: [Errno 21] Is a directory: 'testdir'

    $ rm -rf testdir && mkdir testdir && python3 -c 'import os; os.remove("testdir")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    IsADirectoryError: [Errno 21] Is a directory: 'testdir'

Now I've realised what's going wrong I can work around it, but it'd be
nice if the correct error were raised in the first place.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Incorrect Python errors when using os.remove to delete a directory

[Michael Wild]

On Fri, Nov 27, 2015 at 3:51 PM, Adam Dinwoodie <adam@dinwoodie.org> wrote:
> If I use os.remove in Python to remove a directory, I expect it to fail
> with an OSError on Python2 or a IsADirectoryError on Python3.  On
> Python2, I get OSError, but with the wrong error code, whereas on
> Python3 I get completely the wrong exception.
>
> Simple testcases:
>
>     $ rm -rf testdir && mkdir testdir && python -c 'import os; os.remove("testdir")'
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     OSError: [Errno 1] Operation not permitted: 'testdir'
>
>     $ rm -rf testdir && mkdir testdir && python3 -c 'import os; os.remove("testdir")'
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     PermissionError: [Errno 1] Operation not permitted: 'testdir'
>
> On my handy CentOS box, I see the following instead:
>
>     $ rm -rf testdir && mkdir testdir && python -c 'import os; os.remove("testdir")'
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     OSError: [Errno 21] Is a directory: 'testdir'
>
>     $ rm -rf testdir && mkdir testdir && python3 -c 'import os; os.remove("testdir")'
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     IsADirectoryError: [Errno 21] Is a directory: 'testdir'
>
> Now I've realised what's going wrong I can work around it, but it'd be
> nice if the correct error were raised in the first place.

This is a bug in Python itself, and not Cygwin specific. At least I
get the same behavior with my Anaconda installation.

Not really a fix for the bug itself, but usually I consider it good
practice to first call os.isdir().

Cheers

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Incorrect Python errors when using os.remove to delete a directory

[Adam Dinwoodie]

On Fri, Nov 27, 2015 at 04:09:52PM +0100, Michael Wild wrote:
> On Fri, Nov 27, 2015 at 3:51 PM, Adam Dinwoodie wrote:
> > If I use os.remove in Python to remove a directory, I expect it to fail
> > with an OSError on Python2 or a IsADirectoryError on Python3.  On
> > Python2, I get OSError, but with the wrong error code, whereas on
> > Python3 I get completely the wrong exception.
> 
> This is a bug in Python itself, and not Cygwin specific. At least I
> get the same behavior with my Anaconda installation.

So, having dug a bit more, the problem here is that Python is calling unlink,
which is setting errno to EPERM.  On Linux systems, it's set to EISDIR instead,
but Cygwin's unlink conforms to POSIX[0], and the POSIX standard specifies the
errno in this case should be EPERM.[1]

So, depending on your perspective, the bug is either (a) that Cygwin conforms
to POSIX and not some other standard, (b) that Python3 doesn't correctly
distinguish between the different reasons a POSIX-compliant "unlink" might
return EPERM, or (c) there is no bug, and per [1] the calling code should
handle both exceptions ("Applications written for portability to both
POSIX.1-2008 and the LSB should be prepared to handle either error code.").

I think claiming (a) is not going to achieve much, and distinguishing between
(b) and (c) is something to take upstream to the Python mailing lists or
similar.

[0]: https://cygwin.com/cygwin-api/compatibility.html#std-susv4
[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/unlink.html

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple