2

I am trying to execute this command using Python:

findSyntax = "find . -maxdepth 2 -name '.config' | cpio -updm ../test1/"
subprocess.Popen(findSyntax.split(' '))

But this command just would not work. When I execute this command, it will start listing all the files (not just .config) under the . directory beyond the maxdepth 2... which is a long list.

What am I missing here! Can someone point it out? Thanks.

NOTE: I've tried running subProcess.run as well with same results. I was able to get just the find part working using os.system() command.

EDIT: I just wanted to clarify that this command will copy the files found with the exact directory structure intact to the new location (creating subdirectories if necessary). I've tried this command on bash terminal, and it works fine. But I couldn't get it to work with Python.

EDIT2: So, the whole command works with os.system(), but I couldn't figure out how to make it work with subprocess. os.system() is supposed to be deprecated, so I would be very interested in figuring out the solution using subprocess instead.

rrlamichhane
  • 630
  • 1
  • 8
  • 22
  • You should say what you are expecting to happen instead. – Daniel Roseman Jan 10 '17 at 08:33
  • You should dissect things. First make sure that your find "finds" what you expect it to find - when you run it manually from the command line! Then: you might want to use the shlex module instead of just splitting your command. – GhostCat Jan 10 '17 at 08:41
  • @GhostCat, I've Edited the question with some details. The command works fine on shell. I don't know about his shlex module, but I'll google it. – rrlamichhane Jan 10 '17 at 09:58
  • Check your paths. If you can run the command via bash it should work exactly the same via `os.system()` – Roman Jan 10 '17 at 10:35

1 Answers1

3

Please look at this good answer and this also helps

But in essence, you can't use your above subprocess command with a pipe.

Lets run through the simple example of getting all py files in the current directory: (ls | grep py)
This is broken:

import subprocess
subprocess.call(['ls', '|', 'grep', 'py'])

Because subprocess does only one process at a time, and by piping you are really creating 2 processes.

The simple but limited (to platform) way is to use os.system

import os
os.system('ls | grep py')

This literally just passes a shell command to the system to execute.

However, you should do it with subprocess by defining your pipes:

# Get all files and pass the stdout to a pipe
p1 = subprocess.Popen(['ls'], stdout=subprocess.PIPE)
# then pass that pipe to another process as stdin and do part 2
output = subprocess.check_output(['grep', 'py'], stdin=p1.stdout)
print(output)

So, a copy paste for your example:

import subprocess
p1 = subprocess.Popen("find . -maxdepth 2 -name '.config'".split(), stdout=subprocess.PIPE)
output = subprocess.check_output("cpio -updm ../test1/".split(), stdin=p1.stdout)

Or with os:

os.system("find . -maxdepth 2 -name '.config' | cpio -updm ../test1/")
Community
  • 1
  • 1
Roman
  • 6,398
  • 6
  • 50
  • 87
  • thanks for the response. I tried your example but it didn't work either. I tried to debug by looking into stdout data as explained [here](http://stackoverflow.com/questions/17411966/printing-stdout-in-realtime-from-a-subprocess-that-requires-stdin), and it only printed `b''`. I'm not sure what's going on here. – rrlamichhane Jan 10 '17 at 10:07
  • 1
    have you tried running your command with `os.system("find . -maxdepth 2 -name '.config' | cpio -updm ../test1/")`? – Roman Jan 10 '17 at 10:27
  • @rrlamichhane Oh, I suspect that perhaps your paths are not right. Instead of looking in your current (python script) path`.`, enter the absolute path. – Roman Jan 10 '17 at 10:31
  • You can always start a bash command and give the "piped" part to that as a whole! – GhostCat Jan 10 '17 at 10:41
  • @Roman, not sure why it never crossed my mind to try the whole command with `os.system()`, but I just tried it and it does work. I know that Python wants to deprecate `os.system()`, but shouldn't that mean the new replacements like `subprocess` should be able to run commands that `os.system()` can? – rrlamichhane Jan 10 '17 at 17:17
  • @rrlamichhane really? I've found no mention about it. Check out the official docs: https://docs.python.org/2/library/os.html#os.system. And yes, subprocess can do everything system can, but it does it differently, so you can't just enter the string with pipes – Roman Jan 10 '17 at 18:37
  • ` # Get all files and pass the stdout to a pipe p1 = subprocess.Popen(['systeminfo'], stdout=subprocess.PIPE) # then pass that pipe to another process as stdin and do part 2 os = subprocess.check_output('findstr /B /C:"OS Name" /C:"OS Version"'.split(' '), stdin=p1.stdout)` doesnt work – Bluscream Jul 02 '17 at 07:32