27

Question is straightfoward, but some context may help.

I'm trying to deploy scrapy while using selenium and phantomjs as downloader. But the problem is that it keeps on saying permission denied when trying to deploy. So I want to change the path of ghostdriver.log or just disable it. Looking at phantomjs -h and ghostdriver github page I couldn't find the answer, my friend google let me down also.

$ scrapy deploy
Building egg of crawler-1370960743
'build/scripts-2.7' does not exist -- can't clean it
zip_safe flag not set; analyzing archive contents...
tests.fake_responses.__init__: module references __file__
Deploying crawler-1370960743 to http://localhost:6800/addversion.json
Server response (200):
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/scrapyd/webservice.py", line 18, in render
    return JsonResource.render(self, txrequest)
  File "/usr/lib/pymodules/python2.7/scrapy/utils/txweb.py", line 10, in render
    r = resource.Resource.render(self, txrequest)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 216, in render
    return m(request)
  File "/usr/lib/pymodules/python2.7/scrapyd/webservice.py", line 66, in render_POST
    spiders = get_spider_list(project)
  File "/usr/lib/pymodules/python2.7/scrapyd/utils.py", line 65, in get_spider_list
    raise RuntimeError(msg.splitlines()[-1])
RuntimeError: IOError: [Errno 13] Permission denied: 'ghostdriver.log
Sam Stoelinga
  • 4,264
  • 7
  • 36
  • 52

2 Answers2

33

When using the PhantomJS driver add the following parameter:

driver = webdriver.PhantomJS(service_log_path='/var/log/phantomjs/ghostdriver.log')

Related code, would be nice to have an option to turn off logging though, seems thats not supported:

selenium/webdriver/phantomjs/service.py

class Service(object):
    """
    Object that manages the starting and stopping of PhantomJS / Ghostdriver
    """

    def __init__(self, executable_path, port=0, service_args=None, log_path=None):
        """
        Creates a new instance of the Service

        :Args:
         - executable_path : Path to PhantomJS binary
         - port : Port the service is running on
         - service_args : A List of other command line options to pass to PhantomJS
         - log_path: Path for PhantomJS service to log to
        """

        self.port = port
        self.path = executable_path
        self.service_args= service_args
        if self.port == 0:
            self.port = utils.free_port()
        if self.service_args is None:
            self.service_args = []
        self.service_args.insert(0, self.path)
        self.service_args.append("--webdriver=%d" % self.port)
        if not log_path:
            log_path = "ghostdriver.log"
        self._log = open(log_path, 'w')
Sam Stoelinga
  • 4,264
  • 7
  • 36
  • 52
  • On windows, can I just make my log file anywhere? – User Jan 11 '14 at 21:39
  • @macdonjo I think if you supply a "Windows friendly" path you can put it anywhere you want as long as the proccess has permission to access files in the path you supplied. I'm on Linux though – Sam Stoelinga Jan 12 '14 at 09:31
  • Has anyone tried using '/dev/null' as log path to disable logging? –  Jun 13 '14 at 13:03
  • 13
    Ok, figured it out. To disable logging, use `os.path.devnull` as path. Should work on any system with null device. –  Jun 13 '14 at 17:21
  • I have a related question: `How can we found these selenium phantomjs usages?` I think it's very hard for me to find these usages in http://www.seleniumhq.org/docs/ – Simin Jie May 13 '17 at 03:02
  • @SiminJie Welcome to StackOverflow! If you have a separate but related question, just ask a new question and feel free to link to the other question. Thanks! – jtpereyda Dec 12 '17 at 00:46
16
    #Reduce logging level
    driver = webdriver.PhantomJS(service_args=["--webdriver-loglevel=SEVERE"])

    #Remove logging
    import os
    driver = webdriver.PhantomJS(service_log_path=os.path.devnull)
harishvc
  • 443
  • 8
  • 19