1

I am trying to read data from ORACLE using PANDAS:

pd.read_sql(sql, myORACLEinfo)

where sql is a select statement

I don't own the ORACLE table, nor I can make change to the table date.

The pd.read_sql is returning this error:

 refresh 'charmap' codec can't decode byte 0x81 in position 1101:
 character maps to <undefined>

is there a way to escape/ignore this error.
I tried adding: encoding='iso-8859-1' to the alchemy create_engine settings,

assuming ORACLE's default encoding is iso-8859-1, but this did not help

Barbaros Özhan
  • 39,060
  • 9
  • 21
  • 45
jscriptor
  • 443
  • 6
  • 17
  • try this pd.read_sql_query(sql,myORACLEinfo) – tawab_shakeel Jul 13 '19 at 13:48
  • you can have a look at [this link](https://stackoverflow.com/questions/9233027/unicodedecodeerror-charmap-codec-cant-decode-byte-x-in-position-y-character) – Barbaros Özhan Jul 13 '19 at 14:18
  • Which interface are you using to connect? I'd prefer `cx_Oracle.connect` and it works fine for me using the default encoding with Python 3.x. – Kaushik Nayak Jul 13 '19 at 16:10
  • what character set is used in your Oracle DB: `select * from nls_database_parameters where parameter='NLS_CHARACTERSET'` ? – MaxU Jul 13 '19 at 16:35
  • Hi MaxU. Thanks for the QRY. I ran it and got 'AL32UTF8'. I am not the DBA and have no control over the setting. – jscriptor Jul 13 '19 at 19:38
  • Hi Kaushik Nayak. I am using sqlalchemy.create_engine which under the hood id using cx_Oracle. I am usinf Python 3.7 – jscriptor Jul 13 '19 at 19:41
  • After so many failed attempts and so many goole searches, I landed here: https://stackoverflow.com/questions/45279863/how-to-use-charset-and-encoding-in-create-engine-of-sqlalchemy-to-create . using charset=utf8mb4 eliminated the error. I have no idea why. – jscriptor Jul 13 '19 at 19:44
  • charset=al32utf8 works as well. – jscriptor Jul 13 '19 at 20:01

0 Answers0