4

I am trying to make an environment which allows printing of unicode characters both in Python 2.7 and 3.4, under Windows.

By using Cmder / ConEmu as a terminal, UTF-8 is fully supported out of the box.

However, Python support isn't so simple.

Python 3.3+

If I set chcp 65001, Python 3.3+ detects it and runs fine.

Python 2.7

If I set chcp 65001 it simply doesn't work at all. Python produces an error line while IPython doesn't work at all.

Apparently, by specifying set PYTHONIOENCODING=utf-8 both Python and IPython work and allows printing of some unicode characters.

For example I can do:

print u'\xc1'

but I cannot do 

print u'\xc1\xc1'

which produces: IOError: [Errno 0] Error

Still, it's better than nothing, at least most Python 2.7 programs would run unmodified.

My question is that if I add 

@chcp 65001 > nul
@set PYTHONIOENCODING=utf-8

to the end of cmder/vendor/init.bat, would that have any side-effects on Python 3.3+?

Also, is there any way to let unmodified Python scripts print u'\xc1\xc1' under Windows?

hyperknot
  • 12,019
  • 22
  • 87
  • 143
  • `b'\xc1'.decode('utf-8')` raise `UnicodeDecodeError`. What result do you expect? 1. print Unicode instead of bytestrings 2. don't change chcp; it just complicates things 3. [Use `win-unicode-console` on Python 3](http://stackoverflow.com/a/30551552/4279) to print Unicode to Windows console. Test the [`development` branch of `win-unicode-console`](https://github.com/Drekin/win-unicode-console/tree/development) on Python 2. – jfs Aug 07 '15 at 19:24
  • @J.F.Sebastian sorry, that was obviously my typo. I've fixed it as well as added the error message to `u'\xc1\xc1'`. I am looking for a way to let unmodified Python scripts work on Windows console, not for including additional modules. – hyperknot Aug 07 '15 at 21:15
  • @zsero: click the link in my previous comment. It provides solutions for **unmodified** scripts. – jfs Aug 07 '15 at 21:28
  • @J.F.Sebastian ok, using the development branch of win-unicode-console *and* putting it in `sitecustomize.py` enables seamless unicode printing in Python 2 under windows. It needs to be in sitecustomize otherwise it breaks ipython. – hyperknot Aug 07 '15 at 23:00
  • if it works for you then you could [post your own answer](http://stackoverflow.com/help/self-answer) – jfs Aug 07 '15 at 23:05
  • I'll do it then. It'll take some time to write it all up. – hyperknot Aug 07 '15 at 23:11
  • 1
    Note that ConEmu solves the remaining problem with fonts. The console window managed by conhost.exe limits the choice of fonts based on the system locale and doesn't support fonts that mix half-width and full-width glyphs. This rigid design makes life difficult for people who need a mix of East-Asian and Western languages in a single window. So we need to recommend win-unicode-console + pyreadline + ConEmu to get something close to the default terminal environment in a modern Linux system. – Eryk Sun Aug 07 '15 at 23:20
  • 1
    win-unicode-console now supports also Python 2. – user87690 Aug 08 '15 at 11:46
  • where are you setting \@chcp 65001 > nul \@set PYTHONIOENCODING=utf-8 etc...? environment? Want to try to solve. I have utf8 prb with neovim myself. Thanks, – user6273920 Nov 14 '17 at 15:43

0 Answers0