os.mkdir(path) returns OSError when directory does not exist

Question

I am calling os.mkdir to create a folder with a certain set of generated data. However, even though the path I specified has not been created, the os.mkdir(path) raises an OSError that the path already exists.

For example, I call:

os.mkdir(test)

This call results in OSError: [Errno 17] File exists: 'test' even though I don't have a test directory or a file named test anywhere.

NOTE: the actual path name I use is not "test" but something more obscure that I'm sure is not named anywhere.

Help, please?

score 60 · Accepted Answer · edited May 04 '18 at 17:09

60

Greg's answer is correct but doesn't go far enough. OSError has sub-error conditions, and you don't want to suppress them all every time. It's prudent to trap just expected OS errors.

Do additional checking before you decide to suppress the exception, like this:

import errno
import os

try:
    os.mkdir(dirname)
except OSError as exc:
    if exc.errno != errno.EEXIST:
        raise
    pass

You probably don't want to suppress errno.EACCES (Permission denied), errno.ENOSPC (No space left on device), errno.EROFS (Read-only file system) etc. Or maybe you do want to -- but that needs to be a conscious decision based on the specific logic of what you're building.

Greg's code suppresses all OS errors; that's unsafe just like except Exception is unsafe.

edited May 04 '18 at 17:09

jtpereyda

5,465
8
46
67

answered Mar 28 '14 at 16:30

Chris Johnson

17,500
5
69
74

3

This is the only correct answer. The "look before you leap" approach using os.path.exists that others have suggested creates a race condition: it's entirely possible for the file/dir to be created between the exists and mkdir/makedirs calls, which would generate an unhandled exception. – krait Apr 21 '14 at 21:35
1

This is **not** the "only correct answer" since the race condition is not a relevant problem in *all* (perhaps even most) use cases. – oseiskar Jun 28 '16 at 12:26
2

@oseiskar, when the right way is as easy or easier than doing it the wrong way, to not do it the right way every time is irresponsible. If you do it the right way every time, you don't have to waste time & energy thinking about it. Good developers focus on developing sound habits. – Chris Johnson Jun 28 '16 at 14:01
1

@ChrisJohnson true, but in this case the "as easy" is questionable in terms of readability. The correct balance between these different aspects depends on your use case. Sacrificing readability for resistance to race conditions that never happen in your environment is premature optimization. – oseiskar Jun 29 '16 at 08:43
2

@oseiskar, what I showed is the normal approach for a Python developer. If it's not readable, then whoever's reading it doesn't understand Python very well. Unless you're running on a single-threaded OS, the potential for this particular race condition always exists. – Chris Johnson Jun 29 '16 at 08:53
Two criticisms: 1) the `pass` is superfluous and 2) I'd rather the `raise` be bare (preserving the traceback - maybe unimportant here, but I've had problems with losing tracebacks due to well-meaning coders raising a new error, losing the traceback, and thus my clue for what to fix, before.) – Aaron Hall Dec 15 '16 at 22:25
Aaron, I agree. The `pass` is just a harmless habit, but the `raise` has a real effect. I'll revise that. – Chris Johnson Apr 20 '17 at 01:07
Seems it is a kind of “Ask forgiveness not permission” https://stackoverflow.com/questions/12265451/ask-forgiveness-not-permission-explain – Akif Oct 05 '18 at 05:09
@akif yes exactly. – Chris Johnson Oct 05 '18 at 10:50

1'' · Answer 2 · 2017-10-27T12:48:23.963

55

In Python 3.2 and above, you can use:

os.makedirs(path, exist_ok=True)

to avoid getting an exception if the directory already exists. This will still raise an exception if path exists and is not a directory.

edited Oct 27 '17 at 12:48

answered Oct 08 '17 at 07:09

1''

23,546
28
128
192

1

Using `exist_ok=True` in `makedirs` would still raise `FileExistsError` if target path exists and it is not a directory (file, block device, ...) :( – nadrimajstor Oct 27 '17 at 11:04
7

@nadrimajstor That's a good point, this isn't a drop-in replacement for catching `FileExistsError`, although often you do want an exception when you're trying to create a directory on top of a file. – 1'' Oct 27 '17 at 12:49

score 34 · Answer 3 · edited Jun 17 '20 at 09:47

34

Just check if the path exist. if not create it

import os    
if not os.path.exists(test):
    os.makedirs(test)

edited Jun 17 '20 at 09:47

Josh Correia

2,133
1
17
29

answered Sep 24 '13 at 05:19

lordkain

2,863
1
11
17

8

This answer is incorrect and dangerous. It creates a race condition, since the directory could be created by another process after the `exists` call and before the `makedirs` call. See my answer for a proper solution. – Chris Johnson Jan 05 '15 at 17:43
1

hehe how many miliseconds do you think there will be between the two statements :) but yes you are correct :) – lordkain Jan 05 '15 at 19:59
1

It will happen just often enough to drive you crazy trying to debug it :) – Chris Johnson Jan 06 '15 at 07:03
2

in was kind of environment are you working? and how many processes are there? – lordkain Jan 06 '15 at 07:15
6

Someone famous once said: A one in a million chance is next Tuesday. – Be Kind To New Users Jan 24 '17 at 05:05
Seems like the scheduler likes to slice between OS calls because this particular case is more likely than you'd think. I just ran into it today with two threads calling into the same library that has basically this code in a @property & with a hardcoded directory name. – Sumudu Fernando Sep 26 '17 at 00:29
Just to play devils advocate - if your code is the only thing on the system that would or could create that folder then your race condition will never occur. Still, I'd prefer errno.EEXIST as then you don't even have to ask that question. – Roger Heathcote Jan 14 '20 at 23:06
Race conditions are good to eliminate, but this thread seems overly focused on the race condition in the example code, when a zillion other more common issues can arise and there is no exception handling here at all. Permission errors? Folder in which you are creating the new folder doesn't exist, or worse, is actually a file and not a folder? Out of drive space? Folder name contains invalid characters or is too long? Networked drive is not accessible? Disk is corrupt? Out of memory? Every one of these and many more are not handled. – Michael Krebs May 14 '20 at 13:51

score 3 · Answer 4 · edited Jun 12 '19 at 17:40

3

I also faced the same problem, specially, when the string 'test' contains the multiple directory name. So when 'test' contains the single directory -

if not os.path.exists(test):
    try:
        os.makedir(test)
    except:
        raise OSError("Can't create destination directory (%s)!" % (test))

If the 'test' contains multiple directory like '\dir1\dir2' then -

if not os.path.exists(test):
    try:
        os.makedirs(test)
    except:
        raise OSError("Can't create destination directory (%s)!" % (test))

edited Jun 12 '19 at 17:40

Josh Correia

2,133
1
17
29

answered Jul 02 '15 at 15:25

Anupam Bera

469
3
9

The `if` serves no purpose. – Chris Johnson Jul 14 '16 at 08:55
This `if` statement protects from overwriting the existing directory with the same name. – Anupam Bera Aug 26 '16 at 12:49
3

No it doesn't. The operating system will prevent the overwriting whether you check before or not. Unless you are running on a single-threaded operating system, another process can create the directory after the time you check the directory doesn't exist and the time you try to create it. The whole point of catching the exception is to deal with what happens if the directory already exists, whether it came into existence last week or 1 microsecond ago. The `if` serves no purpose. The `try/except` is the only reliable approach. – Chris Johnson Aug 27 '16 at 21:36

score 1 · Answer 5 · answered Sep 24 '13 at 05:18

1

You have a file there with the name test. You can't make a directory with that exact same name.

answered Sep 24 '13 at 05:18

CT Zhu

44,804
14
99
113

score 1 · Answer 6 · answered Nov 20 '13 at 13:34

Happened to me on Windows, maybe this is the case:

Like you I was trying to :

os.mkdir(dirname)

and got OSError: [Errno 17] File exists: '<dirname>'. When I ran:

os.path.exists(dirname)

I got false, and it drove me mad for a while :)

The problem was: In a certain window I was at the specific directory. Even though it did not exists at that time (I removed it from linux). The solution was to close that window \ navigate to somewhere else. Shameful, I know ...

score 0 · Answer 7 · answered Sep 24 '13 at 05:16

0

I don't know the specifics of your file system. But if you really want to get around this maybe use a try/except clause?

try:
    os.mkdir(test)
except OSError:
    print "test already exists"

You can always do some kind of debugging in the meanwhile.

answered Sep 24 '13 at 05:16

Greg

4,911
1
21
31

1

The error message is incorrect. The `OSError` exception can happen for many reasons, e.g. insufficient permissions or a read-only file system. You can't conclude that the directory already exists. You need to check the value of `exc.errno` to determine the cause of the exception. – Chris Johnson Apr 20 '17 at 01:13

score 0 · Answer 8 · edited Jun 12 '19 at 17:39

0

Maybe there's a hidden folder named test in that directory. Manually check if it exists.

ls -a

Create the file only if it doesn't exist.

if not os.path.exists(test):
    os.makedirs(test)

edited Jun 12 '19 at 17:39

Josh Correia

2,133
1
17
29

answered Sep 24 '13 at 05:21

Tabraiz Ali

667
9
34

This answer is incorrect and dangerous. It creates a race condition, since the directory could be created by another process after the `exists` call and before the `makedirs` call. See my answer for a proper solution. – Chris Johnson Apr 20 '17 at 01:15
If it's hidden, it can't possibly be called `test`, right? It must be `.test` which is a different string – sox with Monica Jan 31 '19 at 13:59

score 0 · Answer 9 · answered Sep 06 '19 at 12:02

0

For python 2.7

from distutils.dir_util import mkpath
mkpath(path)

answered Sep 06 '19 at 12:02

Ahsan aslam

989
2
15
30

score 0 · Answer 10 · answered May 13 '20 at 21:58

Simple answer that does not require any additional import, does not suppress errors such as "permission denied", "no space left on device" etc., yet accepts that the directory may already exist:

import os

try:
    os.mkdir(dirname)
except FileExistsError :
    pass
except :
    raise

os.mkdir(path) returns OSError when directory does not exist

10 Answers10

Linked