Powershell: Setting Encoding for Get-Content Pipeline

Question

I have a file saved as UCS-2 Little Endian I want to change the encoding so I ran the following code:

cat tmp.log -encoding UTF8 > new.log

The resulting file is still in UCS-2 Little Endian. Is this because the pipeline is always in that format? Is there an easy way to pipe this to a new file as UTF8?

So UCS-2 Little Endian is utf16-le. – js2010 Mar 11 '21 at 22:44 — js2010, Mar 11 '21 at 22:44

score 56 · Accepted Answer · edited May 23 '17 at 11:55

56

As suggested here:

Get-Content tmp.log | Out-File -Encoding UTF8 new.log

edited May 23 '17 at 11:55

Community

1
1

answered Sep 16 '08 at 20:41

Lars Truijens

40,852
6
117
137

Thanks for answering this... It helped me figure out a problem reading an xml file encoded in UTF-8 – Peter Walke Nov 17 '10 at 23:49
2

See http://stackoverflow.com/questions/5596982/using-powershell-to-write-a-file-in-utf-8-without-the-bom – Lars Truijens Apr 14 '11 at 22:15
Get-Content hgrc.tmp | Out-File -Encoding ASCII hgrc saved my bitbucket script. Thanks. – sonjz Dec 17 '13 at 21:20

driis · Answer 2 · 2011-09-03T14:14:47.127

31

I would do it like this:

get-content tmp.log -encoding Unicode | set-content new.log -encoding UTF8

My understanding is that the -encoding option selects the encdoing that the file should be read or written in.

edited Sep 03 '11 at 14:14

answered Sep 16 '08 at 20:36

driis

151,614
43
262
332

10

"Get-Help Set-Content" does not identify -encoding as an option, but it does in fact work. – jedatu Sep 17 '08 at 04:05
8

@jedatu: That's because this parameter comes from the FileSystem provider. – Joey Jan 10 '11 at 10:53

score 8 · Answer 3 · answered Jun 05 '14 at 10:33

8

load content from xml file with encoding.

(Get-Content -Encoding UTF8 $fileName)

answered Jun 05 '14 at 10:33

user572730

108
1
2

Do not use `Get-Content` to read XML files, because this requires prior knowledge of the file encoding. It's much better to let the XML parser read the file on it's own, as this correctly deals with any encoding the file might have. `$xml = (New-Object System.Xml.XmlDocument).Load($fileName)` – Tomalak Sep 25 '17 at 09:45

score 1 · Answer 4 · answered Jul 20 '15 at 16:45

1

If you are reading an XML file, here's an even better way that adapts to the encoding of your XML file:

$xml = New-Object -Typename XML
$xml.load('foo.xml')

answered Jul 20 '15 at 16:45

sba923

459
5
8

score 0 · Answer 5 · answered May 24 '21 at 20:20

PowerShell's get-content/set-content encoding flag doesn't handle all encoding types. You may need to use IO.File, for example to load a file using Windows-1252:

$myString = [IO.File]::ReadAllText($filePath, [Text.Encoding]::GetEncoding(1252))

Text.Encoding::GetEncoding Text.Encoding::GetEncodings

Powershell: Setting Encoding for Get-Content Pipeline

5 Answers5

Linked