3

i have a script that first reads in a table data with pandas, then does some computations:

import numpy as np
import sys
import pandas as pd
originaldata = pd.read_csv('../file.txt', sep='\t', skiprows = 31)
originaldata  = originaldata.fillna(0)
.... (rest of the code)

I would like to pass the file name in command line rather than typing it in the code each time there is a different file. is my code right? 

import numpy as np
import sys
import pandas as pd
filename = sys.argv[-1]
originaldata = pd.read_csv('filename', sep='\t', skiprows = 31)
originaldata  = originaldata.fillna(0)
.... (rest of the code)

so on the command line i would type:

$python  program1.py  file.txt

update:

so from the comments i learned that one mistake is that the 

originaldata = pd.read_csv('filename', sep='\t', skiprows = 31)

should be

originaldata = pd.read_csv(filename, sep='\t', skiprows = 31)

and instead of using sys.argv[-1] , argparse is more useful. so here i changed my code to:

import argparse
parser = argparse.ArgumentParser(description='program1 ')
parser.add_argument('-i','--input', help='Input file name',required=True)
originaldata = pd.read_csv(args.input , sep='\t', skiprows = 31)

note i'm using Anaconda 's Spyder IDLE which contains the pandas module, for my program, it runs well within the IDLE but is having problem with command line. Turns out Anaconda uses pythonwin , so after setting path to pythonwin, on my command line i typed : 

$pythonwin program1.py file.txt

then a 'python for Win32' popped up and says 'still can't get my hands on win32ui' . i'm so confused now. does it have anything to do with me using a 64 bit computer?

Jessica
  • 2,004
  • 4
  • 19
  • 40
  • 1
    take out the quotes around `filename` so that you use the variable reference, not a string literal `'filename'` – R Nar Nov 17 '15 at 20:09
  • @R Nar thanks, is everything else correct? – Jessica Nov 17 '15 at 20:12
  • 1
    Why not just run it and see? – yurib Nov 17 '15 at 20:12
  • @yurib i tried using windows command prompt , setting path to python.exe and it still doesn't work, i am not sure why. – Jessica Nov 17 '15 at 20:14
  • 1
    @Jessica "doesn't work" ? do you get an error message? if so, you should add it to the problem description, also the exact command you're executing. – yurib Nov 17 '15 at 20:14
  • @yurib i'm using anaconda 's Spyder IDLE so it has the pandas modules and my script runs, but when i tried the command line: '$python program1.py file.txt ' it gives an error saying no module named 'pandas' . – Jessica Nov 17 '15 at 20:17
  • 1
    @Jessica it's an important piece of information, please edit the question and mention the error. you're most likely just missing an import statement, try adding `import pandas as pd` to the imports – yurib Nov 17 '15 at 20:20
  • looks like you still have to install the `pandas` module from online (its not a standard module) – R Nar Nov 17 '15 at 20:30
  • but the pandas module is included in Anaconda and i forgot to include here that i imported the pandas module. – Jessica Nov 17 '15 at 20:30
  • add the following to your script. **for i in sys.path: print(i)**. then check out what it says for both command line and IDLE. you may also be using a different Python exe in both cases, I would suggest using **which python** if you were on Linux/OSX, but you can check out the PATH manually. You can also start python on the command line wo specifiying a script and check out sys.path. – JL Peyret Nov 17 '15 at 20:36
  • @JL Peyret turns out Anaconda uses pythonwin , so on my command line i typed : pythonwin program1.py file.txt then a 'python for Win32' popped up and says 'still can't get my hands on win32ui' . i'm so confused now. does it have anything to do with me using a 64 bit computer? – Jessica Nov 17 '15 at 20:45
  • check this out for a which equivalent on Windows http://stackoverflow.com/questions/304319/is-there-an-equivalent-of-which-on-the-windows-command-line. now, sorry for the bad news, but I don't think there is a shortcut to sorting out your sys.path in both cases (IDLE which works and command line which doesn't). One approach is to figure out where pythonwin is coming from and checking if there isn't a lil ol python.exe in that neighborhood that you can use by explicitly providing its whole ...\python.exe path on command line. and, no, 64 bit computer has little to do with it. – JL Peyret Nov 17 '15 at 21:31
  • I would also try to settle on 1 Python, either Anaconda's or your base system's (wherever you got that from) and adjust your environment to always use it. On MacOS I have left the system Python alone, but explicitly modified my user environment to always call my dev Python. Pain in the neck, but that's the way it is. – JL Peyret Nov 17 '15 at 21:38

1 Answers1

4

This line has a problem:

originaldata = pd.read_csv('filename', sep='\t', skiprows = 31)

It should be:

originaldata = pd.read_csv(filename, sep='\t', skiprows = 31)

This line will generally work, but....

filename = sys.argv[-1]

I wouldn't do it this way. You're just getting the final argument. Why not be more precise? I'd either use argparse, which may be overkill in this case, but I'd probably just use sys.argv[1] and possibly warn the user if they use an incorrect number of parameters by testing the length of sys.argv

To solve the error, try adding:

import pandas as pd

You may need to pip install pandas using the version of python you are running the script with.

jgritty
  • 10,684
  • 3
  • 33
  • 58