HTTP Request for CSV file yields a string of characters - how do I decode?

Question

I have an API I am accessing using the python requests package. When the data set requested exceeds an internal size limit, the standard endpoint will return a failure and the response code indicates to use their "export" endpoint which serves a CSV instead.

However, the returned CSV data is, what appears to be to me, an encoded string.

Here is an example of the output (trimmed to show start and end)

{"Data":"VGVybWluYWwgTnVtsQ2FsZW5kYXIgVGltZSxUcmFuc2FjdGlvbiBUeXBlLE...GhvbGRlciB0cmFuc2FjdGlvbiIsIkRlbmllZCI="}

What I've tried so far is extracting the string, and then tried various encode/decode combinations, but nothing is yielding a human readable CSV.

The response headers are:

{'Content-Length': '556895', 
 'Content-Type': 'application/json; charset=utf-8', 
 'Server': 'Microsoft-HTTPAPI/2.0', 
 'Date': 'Fri, 24 Apr 2020 17:13:31 GMT'}

Does anyone have experience with this? I expected a byte-string or a byte array, but this is just a utf-8 string.

I apologize if this is something simple I am overlooking, or if I have left out crucial information.

EDIT: My request looks like this:

    api = myAPIObject()
    api.authenticate(username, passwd)
    CR_ID = 5341345
    res = requests.post(api.CR_export_url, 
                        headers=api.auth_header, 
                        data={'ID': CR_ID,
                              'ExportFormat': 'CSV', 
                              'ShouldIncludeHeaders':True})

This is per the documentation of the api.

score 0 · Answer 1 · answered Apr 25 '20 at 16:24

0

It looks like it is not decoding the file properly.

What decoding method are you using? Are you able to provide some of the code that you are using to make the request?

answered Apr 25 '20 at 16:24

dilloncwheeler

41
5

Thanks for the suggestion, I've updated my comment and added the request code - it's very simple, and per the specifications of the API Documentation – Djones4822 Apr 25 '20 at 17:44
I would doublecheck the documentation. You are using a POST, which generally only used to update or insert data to a website. Typically to get the data, you would be using a GET. The data you are getting back is most likely a "receipt" from your post. Hopefully, this helps. https://stackoverflow.com/questions/3477333/what-is-the-difference-between-post-and-get – dilloncwheeler Apr 25 '20 at 20:19
I completely agree, however the documentation is clear and simple. It states a POST with those parameters... I mostly am wondering if this is a style of data transmission I am not familiar with. I've also considered the possibility that something is wrong............ – Djones4822 Apr 25 '20 at 22:59

score 0 · Accepted Answer · answered Apr 26 '20 at 18:20

The data is Base64 encoded, the giveaway is that at the end of the data string there is the signature = sign

simply doing this yielded the expected results

import base64

data = resp.json()['Data']
decoded_data = base64.b64decode(data)
print(decoded_data)

HTTP Request for CSV file yields a string of characters - how do I decode?

2 Answers2