0

I am running a parallelized(using multiprocessing module) python script during data processing and am doing many operations on pandas dataframe. However, during the execution when I used strace on one of the process instance, I observe below: 

stat("/home/anaconda3/lib/python3.5/site-packages/pandas/operator",0x7ffcef5329b0) = -1 ENOENT (No such file or directory)
stat("/home/anaconda3/lib/python3.5/site-packages/pandas/operator.pye",0x7ffcef5329b0) = -1 ENOENT (No such file or directory)  
stat("/home/anaconda3/lib/python3.5/site-packages/pandas",{st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0

I think the file operator.pye must belong to python pyconcrete module, but since I did not used pyconcerete to encrypt my python script, so my running script should not be looking for them.

So my question is:
Are the missing directory(s)/file(s) a pandas bug, or someone else have seen something like this before, and can suggest what could be the cause of it.

OR I can ignore these messages.

Really appreciate some comments!

gmatharu
  • 63
  • 8
  • 1
    My guess: Inside the pandas source there are several `import operator` (operator being a stdlib module) statements. When the interpreter executes an import statement, I believe it checks all directories in the module loading path and loads the first matching file. So the trace is probably from python's import handling. If I am correct, you should probably see many similar traces with other file/directory names. – codeape Sep 14 '17 at 11:40
  • @[codepae](https://stackoverflow.com/users/3571/codeape) I see this message repeatedely for these 2 files : **operator** and **pandas** only that no such file exist, but all other are fine, for example: `clock_gettime(CLOCK_REALTIME, {1505388024, 898907379}) = 0 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=118, ...}) = 0` but I was wondering weather it is an issue or I can ignore it. – gmatharu Sep 14 '17 at 11:43

1 Answers1

0

You should ignore this. It is not indicative of a problem unless it is happening thousands of times per process.

More on where Python searches for imports: How does python find a module file if the import statement only contains the filename?

John Zwinck
  • 207,363
  • 31
  • 261
  • 371