3

On powershell core (7.1.2) or 7.2, I don't have the accented french characters "é" or "è" or "à", I have this:

Liste des mises à jours déjà installées dans la dernière mise à jour

It works on powershell windows 5.1. the text is like this:

Liste des mises à jours déjà installées dans la dernière mise à jour

I use the IDE visual studio code in the 2 cases.

Whats is the solution please? I have already tried to change the encoding : utf8 or utf 16 for example with the french language pack

I tried this code for example, but it doesn't work

#affiche la tentative de mise à jour"
function affichetentative {
 
    #nom du serveur
    $b=HOSTNAME.EXE
    #chemin  où  est lecalisé le fichier de log du serveur
    $path="C:\LOGS\log_$b.txt"
    #affichage du texte ci-dessous dans un fichier de log
    $c=Get-Date
    $a="Liste des mises à jours déjà installées dans la dernière mise à jour le {0} " -f  $c
    ADD-content -path $path -Encoding utf8BOM -value  $a  
        
    }
 affichetentative

I have this output in my file:

Liste des mises à jours déjà installées dans la dernière mise à jour le 16/02/2021 22:13:41

even if i configure utf8BOM or UTF-16LE on visual studio code, it doesn't work

geekdu972
  • 127
  • 8
  • I'm ok on Windows 10 with that. Maybe this is your terminal\console\font problems? Try printing file with this text to console without powershell (using `type` or `cat`) – filimonic Feb 16 '21 at 17:56

1 Answers1

3

The implication is that your file is UTF8-encoded, but without a BOM.

  • Caveat: If you use Add-Content to append to an existing file (that isn't empty), PowerShell matches the (possibly inferred) existing encoding and ignores an -Encoding argument.

While PowerShell (Core) 7+ reads such files correctly, Windows PowerShell does not, because it assumes ANSI encoding in the absence of a BOM; this applies both to files read explicitly with Get-Content and implicitly read source-code files.

Any file you want both PowerShell editions to interpret correctly by default should be UTF8-encoded with a BOM (or UTF16-LE-encoded, which PowerShell calls Unicode, which always has a BOM).

The pitfall is that modern editors such as Visual Studio Code create BOM-less files by default, because UTF8 is assumed to be the default encoding nowadays, and because some utilities, notably those with a Unix heritage, do not expect a BOM and may even misinterpret as data.

  • However, as filimonic points out, you can configure Visual Studio Code to create UTF8 files with BOM by default (optionally just for PowerShell source-code files) - see the docs.

From Windows PowerShell, the problem can be demonstrated as follows:

# Use a .NET API directly to create a test file.
# .NET APIs create BOM-*less* UTF-8 files by default.
[IO.File]::WriteAllText(
  "$PWD/test.txt",
  'Liste des mises à jours déjà installées dans la dernière mise à jour'
)

# Now read it with Get-Content on Windows PowerShell, which
# results in MISINTERPRETATION.
# Note: In PowerShell (Core) 7+, this works correctly.
Get-Content test.txt

You'll get:

Liste des mises à jours déjà installées dans la dernière mise à jour

because the UTF8 encoding was misinterpreted as the active ANSI code page's encoding.

By passing -Encoding utf8 to Get-Content, you could avoid the problem.

mklement0
  • 245,023
  • 45
  • 419
  • 492
  • It works for me with the encoding unicode or utf32. Thanks! https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/add-content?view=powershell-7.1 – geekdu972 Feb 16 '21 at 18:31
  • Glad to hear it, @geekdu972. `Unicode` is PowerShell-speak for UTF16-LE, which definitely works (as does UTF32), but note that you'll get much larger files than with UTF8 (with BOM), if the file is primarily composed of ASCII-range characters. – mklement0 Feb 16 '21 at 18:44
  • @mklement0 When working with files you can use encoding `$utf8bom = [System.Text.UTF8Encoding]::new($true)` which keeps BOM record `[System.IO.File]::WriteAllLines($path, $lines, $utf8bom)`, `[System.IO.File]::WriteAllText($path, $text, $utf8bom)` – filimonic Feb 16 '21 at 18:50
  • @filimonic, yes, but if you _do_ want a BOM in PowerShell code, just use `-Encoding utf8` in Windows PowerShell, and `-Encoding utf8bom` in PowerShell (Core) 7+; if you _don't_ want a BOM in Windows PowerShell, you indeed have to use .NET APIs - see [this answer](https://stackoverflow.com/a/51847431/45375) and [this answer](https://stackoverflow.com/a/34969243/45375) for a cmdlet-like wrapper function. (In PowerShell (Core) 7+ you'll get BOM-less UTF8 by default, consistently across all cmdlets). – mklement0 Feb 16 '21 at 18:56