0

I have a Scrapy script that parses from a french site. I use the command scrapy runspider test.py -o test2.csv to get output. The output is not displaying non-english letters such as "Premium Sécurité de tuyau." How can I modify settings or item dictionary values encoding to display the name as shown. Visit amazon french for an example of French characters that I must parse.

If I use:

response.xpath('//*[@id="productTitle"]/text()').extract_first().strip().encode("utf-8").decode("ascii","ignore")

I get "Gardena 8169-30 Premium Sécurité de tuyau d'arrosage 19 mm (1,9 cm (0,75"))"

I checked this scrapy encoding question but I think it is for python2 (I use python3). Basically, what I am looking for is getting the name as it displayed on the site.

Craig
  • 55
  • 7

1 Answers1

1

You need to import from CSV manually in Excel and select UTF-8 encoding.

Is it possible to force Excel recognize UTF-8 CSV files automatically?

gangabass
  • 9,476
  • 2
  • 21
  • 33
  • `response.xpath('//*[@id="productTitle"]/text()').extract_first().strip().encode("utf-8")` like this, then I import manually ? –  May 12 '19 at 06:30
  • You don't need to use `.encode('utf-8")`. Scrapy save output as UTF-8 by default. It's Excel issue. – gangabass May 12 '19 at 07:08