-2

I have a list of dictionaries which comes as a unicode and the dictionary comes out as a string. I want to remove the unicode bit from the list and convert string dictionaries to normal dicts, to make it a complete list of dicts.

This is how it looks as of now :

u'["{"status": "fulfillable", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "fulfillable", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'41_test\'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}, {"status": "fulfillable", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'5003\'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": null, "complete_time": "2018-01-15T07:16:21.996532+00:00", "priority": 1, "pps_id": null, "pps_seat_name": null, "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": null}, "type": "PICK"}", "{"status": "complete", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:58.395", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "complete", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:58.443", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'41_test\'"]}}]}]}, "actuals": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"pdfa_values": {"product_sku": "41_test"}}}], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}, {"status": "complete", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:58.542", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'5003\'"]}}]}]}, "actuals": {"containers": [{"products": [], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": "7", "complete_time": "2018-01-15T07:16:57.003651+00:00", "priority": 1, "pps_id": "1", "pps_seat_name": "front_1", "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": "admin"}, "type": "PICK"}"]'

I tried to use encode() method which removes the u bit from the list but still leaves it as a string. Also the dicts inside the list are string, which needs to be as dicts only.

If I use ast.literal_eval(), it gives an error - seems it's not able to evaluate.

In [60]: new = ast.literal_eval(data) File "", line 1 ["{"status": "fulfillable", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "fulfillable", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = '41_test'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}, {"status": "fulfillable", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = '5003'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": null, "complete_time": "2018-01-15T07:16:21.996532+00:00", "priority": 1, "pps_id": null, "pps_seat_name": null, "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": null}, "type": "PICK"}", "{"status": "complete", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:58.395", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "complete", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:58.443", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = '41_test'"]}}]}]}, "actuals": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"pdfa_values": {"product_sku": "41_test"}}}], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}, {"status": "complete", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:58.542", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = '5003'"]}}]}]}, "actuals": {"containers": [{"products": [], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": "7", "complete_time": "2018-01-15T07:16:57.003651+00:00", "priority": 1, "pps_id": "1", "pps_seat_name": "front_1", "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": "admin"}, "type": "PICK"}"] ^ SyntaxError: invalid syntax

Ayush
  • 769
  • 1
  • 9
  • 27
  • 3
    That's a JSON string. Just use a JSON parser? – cs95 Jan 19 '18 at 06:08
  • 2
    Possible duplicate of [Convert string representation of list to list in Python](https://stackoverflow.com/questions/1894269/convert-string-representation-of-list-to-list-in-python) – Advay Umare Jan 19 '18 at 06:09
  • @advay.umare ast.literal_eval() is unable to evaluate this, even after it's decoded. – Ayush Jan 19 '18 at 06:11
  • `my_Dict=json.loads(my_string_text)` – Joran Beasley Jan 19 '18 at 06:14
  • @Ayush then you might use json. As shown in another answer there by Ryan. – Advay Umare Jan 19 '18 at 06:14
  • Yes, that's what I thought to do but it did not help. 384 raise ValueError("No JSON object could be decoded") ValueError: Expecting ',' delimiter: line 1 column 5 (char 4) – Ayush Jan 19 '18 at 06:15
  • You have two problems; you need to unwrap the Python `repr()` around the entire string, and then decode the embedded JSON. Neither of these is an uncommon or hard problem, though a single duplicate won't solve all of it. Still, voting to close. – tripleee Jan 19 '18 at 06:17
  • @Ayush I think the encodings not right. How is this string getting generated? – Advay Umare Jan 19 '18 at 06:36
  • It's response from django view HttpResponse, I'm using request.get to make the call. – Ayush Jan 19 '18 at 06:41
  • then add headers in request for json response.. – Advay Umare Jan 19 '18 at 06:57

1 Answers1

0

dump your unicode list of dicts and then load using json.

x = u'["{"status": "fulfillable", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "fulfillable", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'41_test\'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}, {"status": "fulfillable", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:23.412", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'5003\'"]}}]}]}, "actuals": {"containers": []}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": null, "complete_time": "2018-01-15T07:16:21.996532+00:00", "priority": 1, "pps_id": null, "pps_seat_name": null, "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": null}, "type": "PICK"}", "{"status": "complete", "externalServiceRequestId": "15_jan_003", "updatedOn": "2018-01-15T07:12:58.395", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": []}, "serviceRequests": [{"status": "complete", "externalServiceRequestId": "15_jan_00_01", "updatedOn": "2018-01-15T07:12:58.443", "createdOn": "2018-01-15T07:12:22.826", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'41_test\'"]}}]}]}, "actuals": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"pdfa_values": {"product_sku": "41_test"}}}], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}, {"status": "complete", "externalServiceRequestId": "11_jan_025", "updatedOn": "2018-01-15T07:12:58.542", "createdOn": "2018-01-15T07:12:22.827", "expectations": {"containers": [{"products": [{"productQuantity": 20, "productAttributes": {"filter_parameters": ["product_sku = \'5003\'"]}}]}]}, "actuals": {"containers": [{"products": [], "state": "complete", "barcode": null, "type": "VIRTUAL"}]}, "type": "PICK_LINE"}], "actuals": {"containers": []}, "attributes": {"pps_bin_id": "7", "complete_time": "2018-01-15T07:16:57.003651+00:00", "priority": 1, "pps_id": "1", "pps_seat_name": "front_1", "order_options": {"pick_after_time": "2017-10-01 09:15:00", "pick_before_time": "2018-11-04 09:20:00"}, "user_name": "admin"}, "type": "PICK"}"]'

y = json.dumps(x)
z = json.loads(y) ## will give normal string notation of you thing

now you can convert your strings dicts to normal dicts using this. 

op = [ast.literal_eval(a) for a in z]
Prakash Palnati
  • 2,416
  • 16
  • 32