I am running my Python script in Windows PowerShell, and the script should run another program using Popen, then pipe the output of that program (Mercurial, actually) for use in my script. I am getting an encoding error when I try to execute my script in PowerShell.
I am quite sure it is happening because Python is not using the correct encoding that PowerShell is using, when getting the output of the Popen call. The problem is that I don't know how to tell Python to use the correct encoding.
My script looks like
# -*- coding: utf-8 -*-
#... some imports
proc = Popen(["hg", "--cwd", self.path, "--encoding", "UTF-8"] + list(args), stdout=PIPE, stderr=PIPE)
#... other code
When I run this script on Linux, I have no problems whatsoever. I can also run the script in Windows 7 Home Premium 64-bit using PowerShell with no problems. The PowerShell in this Windows 7 is using the code page 850, that is, the output of chcp
is 850
("ibm850").
However, when I run the script in a Windows 7 Starter 32-bits using a PowerShell that has by default the encoding cp437 (chcp
= 437
), I get the following error from Python (version 2.7.2):
File "D:\Path\to\myscript.py", line 55, in hg_command
proc = Popen(["hg", "--cwd", self.path, "--encoding", "UTF-8"] + list(args), stdout=PIPE, stderr=PIPE)
File "C:\Program files\Python27\lib\subprocess.py", line 679, in __init__
errread, errwrite)
File "C:\Program files\Python27\lib\subprocess.py", line 852, in _execute_child
args = list2cmdline(args)
File "C:\Program files\Python27\lib\subprocess.py", line 615, in list2cmdline
return ''.join(result)
UnicodeDecodeError: 'utf8' codec cant decode byte 0xe3 in position 0: unexpected end of data
I have tried the following, with no success (i.e., the above error report stays the same):
- Remove the line
# -*- coding: utf-8 -*-
from my script. - Remove the
-- encoding UTF-8
option for running Mercurial through Popen in my script. - Change the encoding to
chcp 850
in PowerShell before executing my script. - Many other miscellaneous Python hacks I've found in other Stack Overflow answers.
For my specific details, my whole source code is available here in BitBucket. hgapi.py
is the script that gives the error.
UPDATE: The script is being called by this other script, which is setting the encoding like this
sys.setdefaultencoding("utf-8")
This line looks important, because if I comment it out, I get a different error:
UnicodeDecoreError: 'ascii' codec cant decode byte 0xe3 in position 0: ordinal not in range(128)