How to encode Unicode objects in Python?

Question

I've been trying to find a solution here for my problem for a long time but nothing seems to work. I am trying to authenticate with the Coinbase Pro API in Python but I am getting

"Unicode-Objects must be encoded before hashing" error.

Please see the code below for your reference.

import json, hmac, hashlib, time, requests, base64
from requests.auth import AuthBase

# Create custom authentication for Exchange
class CoinbaseExchangeAuth(AuthBase):
    def __init__(self, api_key, secret_key, passphrase):
        self.api_key = api_key
        self.secret_key = secret_key
        self.passphrase = passphrase

    def __call__(self, request):
        timestamp = str(time.time())
        message = timestamp + request.method + request.path_url + (request.body or '')
        hmac_key = base64.b64decode(self.secret_key)
        signature = hmac.new(hmac_key, message, hashlib.sha256)
        signature_b64 = signature.digest().encode('base64').rstrip('\n')

        request.headers.update({
            'CB-ACCESS-SIGN': signature_b64,
            'CB-ACCESS-TIMESTAMP': timestamp,
            'CB-ACCESS-KEY': self.api_key,
            'CB-ACCESS-PASSPHRASE': self.passphrase,
            'Content-Type': 'application/json'
        })
        return request

api_url = 'https://api.pro.coinbase.com/'
auth = CoinbaseExchangeAuth(api_key='XXXX',
                            secret_key='XXXX',
                            passphrase='XXXX')

# Get accounts
r = requests.get(api_url + 'accounts', auth=auth)

It would be great if someone can point me to the right direction. I couldn't find the correct solution after searching.

@t.stv I am actually getting this error now: File "/Users/asanga/Documents/python_projects/coinbasepro/main.py", line 22, in __call__ hmac_key = base64.b64decode(self.secret_key) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/base64.py", line 87, in b64decode return binascii.a2b_base64(s) binascii.Error: Invalid base64-encoded string: number of data characters (85) cannot be 1 more than a multiple of 4 — Nant, Feb 10 '20 at 21:15
so your secret_key contains some strange stuff. Have you tried to call encode on the secret_key? — tstoev, Feb 10 '20 at 21:18
@t.stv I've created a new pair of keys and tried to run the code again but now I am getting the "Unicode-objects must be encoded before hashing" error again — Nant, Feb 10 '20 at 21:24
@t.stv stacetrack below: File "/Users/asanga/Documents/python_projects/coinbasepro/main.py", line 23, in __call__ signature = hmac.new(hmac_key, message, hashlib.sha256) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/hmac.py", line 153, in new return HMAC(key, msg, digestmod) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/hmac.py", line 93, in __init__ self.update(msg) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/hmac.py", line 102, in self.inner.update(msg) — Nant, Feb 10 '20 at 21:25
@t.stv still didn't work, I am getting the same error message. — Nant, Feb 10 '20 at 21:36
check this thread https://stackoverflow.com/questions/7585307/how-to-correct-typeerror-unicode-objects-must-be-encoded-before-hashing — tstoev, Feb 10 '20 at 21:39
After some further debugging I found that the main problem lies in this line: signature = hmac.new(hmac_key, message, hashlib.sha256) — Nant, Feb 10 '20 at 21:54

Raihan Kabir · Answer 1 · 2020-02-10T21:50:52.773

1

hashlib takes bytes or bytearray. Try this -

...
message = message.encode('UTF-8')
hmac_key = self.secret_key.encode('UTF-8')
signature = hmac.new(hmac_key, message, hashlib.sha256)

edited Feb 10 '20 at 21:50

answered Feb 10 '20 at 21:24

Raihan Kabir

372
1
9

@Nant, did you follow this one? https://developers.coinbase.com/docs/wallet/api-key-authentication – Raihan Kabir Feb 10 '20 at 21:35
That's for authenticating to the coinbase wallet not the coinbase pro exchange. I've seen that article but unfortunately that's not quite what I want – Nant Feb 10 '20 at 21:36
@Nant, updated the solution. Try now :-) hope it helps! – Raihan Kabir Feb 10 '20 at 21:54
I am getting this error now: File "/Users/asanga/Documents/python_projects/coinbasepro/main.py", line 31, in __call__ signature_b64 = signature.digest().encode('base64').rstrip('\n') AttributeError: 'bytes' object has no attribute 'encode' – Nant Feb 10 '20 at 21:59
Well, `hashlib` takes bytes right? and you can achieve it with UTF-8 encoding. Is it necessary to encode in base64??? – Raihan Kabir Feb 10 '20 at 22:02
I had to change a few things but generally, this worked. Thanks for your help. See my answer for the solution abover. – Nant Feb 10 '20 at 22:33
1

Yes, it should work fine as well. Because both converts to bytes, `base64` and `UTF-8` and the `hmac` takes bytes like objects. – Raihan Kabir Feb 11 '20 at 04:15

score 1 · Accepted Answer · answered Feb 10 '20 at 22:33

After some debugging I found that the following solution worked for me:

def __call__(self, request):
    print(request.body)
    timestamp = str(time.time())
    message = (timestamp + (request.method).upper() + request.path_url + (request.body or ''))
    message = message.encode('UTF-8')
    hmac_key = base64.b64decode(self.secret_key)
    signature = hmac.new(hmac_key, message, hashlib.sha256).digest()
    signature_b64 = base64.b64encode(signature)

score 1 · Answer 3 · answered Mar 08 '20 at 21:22

Here is how I solved it, message.encode('utf-8')

   def __call__(self, request):
    timestamp = str(time.time())
    message = timestamp + request.method + request.path_url +     (request.body or '')
    hmac_key = base64.b64decode(self.secret_key)
    signature = hmac.new(hmac_key, message.encode('utf-8'), hashlib.sha256)
    signature_b64 = base64.b64encode(signature.digest())

How to encode Unicode objects in Python?

3 Answers3