json module bug in Python 3.4.1?

Question

I'm trying to use SharePoint 2010's REST API, which was going swimmingly until i ran into this:

Traceback (most recent call last):
  File "TestJSON.py", line 21, in <module>
    json.loads(s)
  File "c:\Python33\lib\json\__init__.py", line 316, in loads
    return _default_decoder.decode(s)
  File "c:\Python33\lib\json\decoder.py", line 351, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\Python33\lib\json\decoder.py", line 367, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting ',' delimiter: line 1 column 14 (char 13)

Test case:

import json
s='''{"etag": "W/\"1\""}'''
json.loads(s)

Python 3.3.5 gives the same error. Did i find a bug in the JSON library?

Update:

The actual error i'm getting (preceded by affected part) is this:

>>>>>>Err:tration?$filter=Modified%20gt%20datetime\'2014-04-30T00:00:00.000Z\'&$orderby=Mo<<<<<<<
Traceback (most recent call last):
  File "TestURL.py", line 41, in <module>
    j = json.loads(body)
  File "c:\Python33\lib\json\__init__.py", line 316, in loads
    return _default_decoder.decode(s)
  File "c:\Python33\lib\json\decoder.py", line 351, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\Python33\lib\json\decoder.py", line 367, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Invalid \escape: line 4005 column 114 (char 314020)

from

body = response.read().decode("utf-8")
print(">>>>>>Err:{}<<<<<<<".format(body[314020-40:314020+40]))

No, you specified invalid JSON. `{"etag": "W/"1""}` is **not** properly formatted JSON. — Martijn Pieters, Jun 24 '14 at 15:12
I suspect your diagnosis of the SharePoint REST JSON output is incorrect. What is the actual value *from SharePoint* that trips up `json.loads()`? — Martijn Pieters, Jun 24 '14 at 15:14
Use `{!r}` to show us the contents of the string in a reproducable format. — Martijn Pieters, Jun 24 '14 at 15:23
`"tration?$filter=Modified%20gt%20datetime\\'2014-04-30T00:00:00.000Z\\'&$orderby=Mo"` — Cees Timmerman, Jun 24 '14 at 15:26
Note the ``\`` there; in JSON that needs to be doubled. In a `repr()` that'd be ``\\\\``. — Martijn Pieters, Jun 24 '14 at 15:30
No, that's the escaped output as it should be in source code. The literal characters are in the question. — Cees Timmerman, Jun 24 '14 at 15:44

score 3 · Answer 1 · answered Jun 24 '14 at 15:12

3

The string literal is not escaped correctly. Make sure the string really represent JSON.

>>> s = r'''{"etag": "W/\"1\""}'''  # NOTE: raw string literal
>>> json.loads(s)
{'etag': 'W/"1"'}

answered Jun 24 '14 at 15:12

falsetru

314,667
49
610
551

1

According to [this answer](http://stackoverflow.com/a/3020108/819417), MicrosoftSharePointTeamServices 14.0.0.6029's JSON output is incorrect. – Cees Timmerman Jun 24 '14 at 15:22

score 1 · Accepted Answer · answered Jun 24 '14 at 15:45

The \' sequence is invalid JSON. Single quotes do not need to be escaped, making this an invalid string escape.

You could try to repair it after the fact:

import re

data = re.sub(r"(?<!\\)\\'", "'", data)

before loading it with JSON. This replaces \' with plain ', provided the backslash wasn't already escaped by a preceding \.

Since single quotes can only appear in string values, this should be safe.

json module bug in Python 3.4.1?

2 Answers2