I tried to run scrapy spiders in a flask app, and there is a API in github Arachne.But there is something wrong when I run the demo project in my mac.The app.py looks like this:
app = Arachne(__name__)
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource,
logFormatter=http.combinedLogFormatter,
logPath="logs/"+datetime.now().strftime("%Y-%m-%d.web.log"))
reactor.listenTCP(8080, site)
if __name__ == '__main__':
reactor.run()
But when I run this file,there is an error:
twisted.internet.error.CannotListenError: Couldn't listen on any:8080: [Errno 48] Address already in use.
I tried to solve this by the method which is mentioned in this question:twisted python server port already in use.And the result after I use the command to kill the process in my terminal like this:
sunzhendeMacBook-Pro:~ Rabbit$ lsof -n -iTCP:8080 -sTCP:LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
Python 5151 Rabbit 3u IPv4 0x738fcf82e37369b9 0t0 TCP *:http-alt (LISTEN)
However,when I run the demo project the Address already in use problem still there.I've also tried to change another port number, but the program has no result if I change the port.
So I wonder who has ever used Arachne API has encountered this problem and can give me a solution.Or does anybody has other good way to run multi spiders in flask app?
