0

I'll try to explain this, hopefully it makes sense.

I had installed a virtual env a while ago in the directory: desktop/project_website/test_project/env to do some websites.

I also installed scrapy on the desktop/project_website/test_project/env/Scripts dir.

The problem is that whenever I use the command prompt to run scrapy I need to be in this specific directory, so all the files that scrapy creates are made in here, not in the directory where my project is (that is desktop/project_website/Scrapy)

So how would I would I be able to run scrapy in the dir that I want? Thanks.

tadm123
  • 6,523
  • 4
  • 20
  • 35

1 Answers1

4

You can create virtualenv in any directory: 

virtualenv some/directory
# or windows
virtualenv some\directory

Then you can activate that virtualenv which will change your $PATH environment variables to read python related things from virtual environments directory instead of your system's:

source some/directory/bin/activate
# or for windows
some\directory\Scripts\activate

and to deactivate type: deactivate

See more at official documentation of virtualenv

Once you have virtualenv activated when you call scrapy the scrapy of virtual environment will be called instead of system one and any packages you install via pip(if the python version in your virtualenv has it) will be installed to the virtual environment.
You can always test what will run by using:

$ which scrapy
some/directory/bin/scrapy
# or for windows
$ where scrapy
some\directory\bin\scrapy
Granitosaurus
  • 17,068
  • 2
  • 45
  • 66
  • I probably should had include this. I already activated the virtual env, and went to the `Project_website/Scrapy` directory, tried running and is not finding it – tadm123 Mar 01 '17 at 08:24
  • `(Test_Project) C:\Users\A\Desktop\Project_Website\Scrapy>scrapy 'scrapy' is not recognized as an internal or external command, operable program or batch file.` – tadm123 Mar 01 '17 at 08:24
  • Are you sure scrapy is installed in the virtual environment? check if `virtualenv\bin\scrapy` is there. – Granitosaurus Mar 01 '17 at 08:30
  • I don't have a `bin` directory but the `scrapy.exe` file is in `C:\Users\A\Desktop\Project_Website\Test_Project\env\Scripts` – tadm123 Mar 01 '17 at 08:33
  • What does `where python` says? and `echo %path`? – Granitosaurus Mar 01 '17 at 08:37
  • `C:\Users\A\Desktop\Project_Website\Test_Project\env\Scripts\python.exe` `C:\Python\Python35\python.exe` – tadm123 Mar 01 '17 at 08:39
  • What about copying pasting the `scrapy.exe` file into the directory that I want. Would that work? Edit: Unfortunately, this doesn't work. – tadm123 Mar 01 '17 at 08:44
  • Seems like the `virtualenv\Scripts` directory is not being added to your path for some reason. You can try recreating the virtual environment, maybe it's broken. You can also add virtual environment to PATH manually: `set PATH=%PATH%;C:\virtualenvironment\Scripts` see http://stackoverflow.com/questions/9546324/adding-directory-to-path-environment-variable-in-windows – Granitosaurus Mar 01 '17 at 08:53
  • Thanks, I'll try to go over it. – tadm123 Mar 01 '17 at 08:54
  • oh you can also probably just run scrapy by calling absolute path: `C:\virtualenv\Scripts\scrapy.exe crawl myspider` – Granitosaurus Mar 01 '17 at 08:54
  • seems like that isn't working either: `ImportError: No module named 'quotes.settings'; 'quotes' is not a package` – tadm123 Mar 01 '17 at 09:00
  • running `scrapy.exe` alone from other directory without all the other files that came when you install it is gonna give those errors as expected. oh well, I'll try to read the link that you posted. no worries – tadm123 Mar 01 '17 at 09:03