313

I'm using the python requests module to send a RESTful GET to a server, for which I get a response in JSON. The JSON response is basically just a list of lists.

What's the best way to coerce the response to a native Python object so I can either iterate or print it out using pprint?

Martijn Pieters
  • 889,049
  • 245
  • 3,507
  • 2,997
felix001
  • 12,330
  • 27
  • 79
  • 103

2 Answers2

555

Since you're using requests, you should use the response's json method.

import requests

response = requests.get(...)
data = response.json()

It autodetects which decoder to use.

Martijn Pieters
  • 889,049
  • 245
  • 3,507
  • 2,997
pswaminathan
  • 7,116
  • 1
  • 20
  • 26
  • 2
    Just keep in mind that it have appeared somewhere in between v0.12 and v1.0 so that for example Ubuntu 12.04 deb-package for python-requests does not have this function yet (it is v0.8). You can `pip install requests` though instead of using deb package. – timurb Oct 22 '13 at 16:11
  • 6
    I am a little curious what is now `data` is it a list of list or a dictionary. – Krishna Oza May 30 '15 at 18:05
  • 8
    @Krishna_Oza `data` mirrors the structure of the JSON it's reading. For example, if the response is: `[{"a": {...}}]`, `data` is a list, with `list[0] == {'a': {...}}`. If the response is `{"a": "b", "c": "d"}`, `data` is a dict. Does that answer your question? – pswaminathan May 30 '15 at 21:06
  • this approach is essentially the same as the json.loads(response) approach in other answers, with the exception that the latter may be easier to tab through if you're using a Jupyter environment – beep_check Aug 30 '18 at 03:26
  • shouldn't it be `data = response.json`? Otherwise, I get `TypeError: 'dict' object is not callable` – CGFoX Oct 28 '18 at 19:20
  • 1
    @CGFoX what version are you running? I'm still seeing the API work the same way on the latest version: ```>>> import requests >>> r = requests.get('http://httpbin.org/get') >>> r.json > >>> r.json() {'args': {}, ...}``` – pswaminathan Oct 29 '18 at 14:47
  • I'm using Python 3.7, but I don't think that's the issue. See [this thread with the same problem](https://stackoverflow.com/a/31481487/2745116) – CGFoX Oct 30 '18 at 10:48
  • @CGFoX that is the _request_ object, not the _response_. The response JSON needs to be parsed, which is why it is a method. – pswaminathan Nov 01 '18 at 15:54
  • 5
    Upvote for this because I was usually using the json.loads(response.text) method until on some large jsons I found that using respons.json() was much faster than the other way. – Paul R. Jan 28 '19 at 14:01
  • 1
    @pswaminathan what a fantastic answer! This saved me a lot of headache. Was using the response.text which was messing this up. Thanks again :) – Helen Neely Oct 12 '19 at 20:07
  • Even with this method, I get an 'invalid control character' JSONDecodeError. Any insight into this, @Martijn Pieters? – sFishman May 18 '20 at 08:20
  • @sFishman: this method nor any other can't protect you from invalid JSON data. I can't tell you how to fix your problem because there could be any number of reasons you are fetching data that can't be decoded as JSON. – Martijn Pieters May 18 '20 at 19:16
378

You can use json.loads:

import json
import requests

response = requests.get(...)
json_data = json.loads(response.text)

This converts a given string into a dictionary which allows you to access your JSON data easily within your code.

Or you can use @Martijn's helpful suggestion, and the higher voted answer, response.json().

Scott Skiles
  • 2,933
  • 2
  • 26
  • 50
Simeon Visser
  • 106,727
  • 18
  • 159
  • 164
  • 3
    ok great, however each of the elements would still be unicode. – felix001 Jun 01 '13 at 21:43
  • @felix001: yes, although you can convert any data using `str()`. On the other hand unicode data isn't bad to have around (in preparation for the future). – Simeon Visser Jun 01 '13 at 21:45
  • 186
    Much, much better to use `response.json()`, as it'll do a *better job* of figuring out the encoding used. (Disclaimer, I wrote some of that code). – Martijn Pieters May 19 '16 at 20:24
  • @MartijnPieters: then how can I use requests json parser later on a memcached text of the response? i.e. having the output of `response.text()`? – neurino Mar 28 '19 at 14:40
  • @MartijnPieters, found: `requests.compat.json.loads(resp_text)` – neurino Mar 28 '19 at 14:59
  • 1
    @neurino you want the standard library [`json` module](https://docs.python.org/3/library/json.html). The `requests.compat` module is there to bridge different Python versions and `requests.compat.json` is the same thing as `json` on practically every system you’d care about. – Martijn Pieters Mar 28 '19 at 15:13