Unicode in PowerShell with Python? Alternative shells in Windows?

Question

I want a shell that supports Unicode on Windows. PowerShell as it ships doesn't seem to.

PowerShell V2 (Windows 7 x64):

PS C:\> powershell
Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.

PS C:\> python
Python 2.6.2 (r262:71605, Apr 14 2009, 22:46:50) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> unicode_char=unichr(0xf12)
>>> unicode_char
u'\u0f12'
>>> print unicode_char
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\python26\lib\encodings\cp437.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0f12' in position 0: character maps to <undefined>
>>>

I get similar results with PowerShell ISE, even though some places around the web claim it to be Unicode supporting or whatever...

The Python Integrated Development Environment (IDLE) 2.6.2 seems to work fine:

>>> unicode_char=unichr(0xf12)
>>> print unicode_char
༒
>>>

IDLE is very slow, and I would prefer another shell, any ideas? Can I make this work in PowerShell?

related: [Python, Unicode, and the Windows console](http://stackoverflow.com/q/5419/4279) — jfs, Sep 23 '15 at 19:18

score 6 · Accepted Answer · edited Jan 04 '17 at 18:44

6

The Windows console subsystem is not Unicode, but code page based. You can play around with setting the code page:

PS> chcp 65001
PS> ipy64.exe
>>> print unichr(0x3a9)
Ω

I couldn't get (0xF12) to give the right character with that codepage. Perhaps it is available on another code page.

ISE can display Unicode and accept Unicode input, for example,

PS> [char]0xf12
༒
PS> [char]0xe4
ä
PS> [char]0x3a9
Ω

However, ISE doesn't seem to play well with the IronPython interpreter.

Further to the point that ISE seems to handle Unicode coming from a native application via stdout:

$src = @'
namespace Foo {
    public class Bar
    {
        public static void Baz()
        {
            System.Console.Out.WriteLine("\u0f12");
            System.Console.Out.WriteLine("\u00e4");
            System.Console.Out.WriteLine("\u03a9");
        }
    }
}
'@

Add-Type -TypeDefinition $src

[Foo.Bar]::Baz()
༒
ä
Ω

edited Jan 04 '17 at 18:44

Peter Mortensen

28,342
21
95
123

answered Jan 20 '10 at 22:51

Keith Hill

173,872
36
316
347

Yeah normal python 2.6.2 cannot print these characters in ISE, I get the same error I do with regular powershell. – 8steve8 Jan 21 '10 at 00:44
Unfortunately you're running WNU Windows: WNU's Not Unix – Peter Seale Jan 21 '10 at 03:17
Windows consoles are not code page based, they are Unicode-based just like all components of Windows. Code pages are only for legacy applications. – Philipp Jun 11 '11 at 20:41

score 1 · Answer 2 · edited Jan 04 '17 at 18:47

1

The PowerShell itself is Unicode, but the classic console has problems with Unicode. But PowerShell ISE is definitely Unicode.

Try this:

PS C:\> $a = [char]0xf12
PS C:\> echo $a

This is Python not playing nice.

You can try chcp 65001 before starting Python (to set the code page to UTF-8). No promise though (I did not try to see if it works, no Python installed on this machine).

edited Jan 04 '17 at 18:47

Peter Mortensen

28,342
21
95
123

answered Jan 21 '10 at 09:39

Mihai Nita

5,148
23
23

score 0 · Answer 3 · edited Jan 04 '17 at 18:45

0

Maybe you should try IPython.

edited Jan 04 '17 at 18:45

Peter Mortensen

28,342
21
95
123

answered Jan 20 '10 at 21:36

Enchantner

1,424
3
19
37

score 0 · Answer 4 · answered Jan 21 '10 at 14:15

0

Note that, when issuing chcp 65001 in a Windows console the font must be Lucida Console and not any of the bitmap fonts. Theoretically, other monospaced ttf fonts should work, but they don't (practically). It's an issue of attributes that MS checks in monospaced fonts intended for the console, and Lucida Console includes them.

There is at least one issue open for the Python Windows console unicode problems here.

answered Jan 21 '10 at 14:15

tzot

81,264
25
129
197

1

I believe Consolas should work also (its TrueType). That's the font I use in my console windows. – Keith Hill Jan 21 '10 at 17:57
Oh, yes, you're probably right, but since my MS Windows experience never reached Vista or 7, I wouldn't know. – tzot Jan 22 '10 at 21:49

score 0 · Answer 5 · edited Oct 08 '14 at 17:07

0

Both the normal Windows console and PowerShell ISE support Unicode. See the blog posts by Michael Kaplan for details:

edited Oct 08 '14 at 17:07

Community

1
1

answered Jun 11 '11 at 20:49

Philipp

43,805
12
78
104

Good find. Thanks for sharing! – Keith Hill Jun 14 '11 at 17:29

Unicode in PowerShell with Python? Alternative shells in Windows?

5 Answers5

Linked