I'm trying to create a program that uses a scrapy
sipder multiple times for different uses. My structure is like the following:
def init_crawl(OUTPUT_FILE, ticker):
OUTPUT_FILE = "file:///" + OUTPUT_FILE
process = CrawlerProcess({
'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
'FEED_URI': OUTPUT_FILE,
'LOG_ENABLED': False,
})
process.crawl(SeccrawlerSpider, ticker=ticker)
process.start()
class GUIapp():
def __init__(self, master):
# stuff
def execute(self, ticker):
# tiker is user input
init_crawl(file_loc, ticker)
def main():
master = tk.Tk()
app = GUIapp(master)
master.mainloop()
if __name__ == "__main__":
main()
This works fine the first time, but I always get a twisted.internet.error.ReactorNotRestartable
when I input a second ticker after the getting the results for the first ticker. I've tried several possible solutions on SO relating to this issue but none of them seem to be directly applicable to this. What am I missing?