13

I would like to parse an HTML document using lxml. I am using python 3.2.3 and lxml 2.3.4 ( http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml )

I am using the etree.iterparse to parse the document, but it returns the following run-time error:

Traceback (most recent call last):
  File "D:\Eclipse Projects\Python workspace\Crawler\crawler.py", line 12, in <module>
    for event, elements in etree.iterparse(some_file_like):
  File "iterparse.pxi", line 491, in lxml.etree.iterparse.__next__ (src/lxml\lxml.etree.c:98565)
  File "iterparse.pxi", line 512, in lxml.etree.iterparse._read_more_events (src/lxml\lxml.etree.c:98768)
TypeError: reading file objects must return plain strings

The question is: How to solve this run-time error?

Thank you very much.

Here is the code:

from io import StringIO
from lxml import etree

some_file_like = StringIO("<root><a>data</a></root>")

for event, elements in etree.iterparse(some_file_like): #<-- Run-time error happens here
    print("%s, %4s, %s" % (event, elements.tag, elements.text))
Avaris
  • 32,127
  • 6
  • 70
  • 67
Ababneh A
  • 936
  • 4
  • 14
  • 29

1 Answers1

23

Your StringIO buffer has unicode string. iterparse works with file like objects that return bytes. The following buffer should work with iterparse:

from io import BytesIO
some_file_like = BytesIO("<root><a>data</a></root>".encode('utf-8'))
Imran
  • 76,055
  • 23
  • 93
  • 124
  • Thank you for the feedback. I tried your suggestion but it gave the following run-time error: TypeError: initial_value must be str or None, not bytes – Ababneh A Apr 20 '12 at 08:12
  • Apparently you need to use `BytesIO` for bytes and `StringIO` for strings (unlike old `StringIO` which could be used for both). Fixed my answer. – Imran Apr 20 '12 at 08:18
  • I've an unicode file which needs to be processed as bytes. How can I do that? – user Jul 08 '13 at 07:09
  • 2
    @user I think you should open the file in binary mode. – Imran Jul 08 '13 at 10:32