How to use here-string in PowerShell to write pip.ini and .condarc file properly?

Question

Disclaimer: I am getting more and more accustomed with PowerShell, but I am rather inexperienced with PowerShell.

I would like to use PowerShell together with here-string syntax to write the pip.ini and .condarc configuration files to use the Python package managers pip, conda, respectively. With the .condarc, there is no error, but I think I had to rewrite it again in Notepad++ to really make it work - I think that it is a file encoding issue:

mkdir 'C:\ProgramData\conda\.condarc'
echo @"
show_channel_urls: true
allow_other_channels: false
report_errors: false
remote_read_timeout_secs: 120
"@ > C:\ProgramData\conda\.condarc

And the following gives an error for pip.ini because of [global]:

mkdir 'C:\ProgramData\pip\pip.ini'
echo @"
[global]
index = https://xxxx/nexus/repository/xxxx/pypi
index-url = https://xxxx:8443/nexus/repository/xxxx/simple
trusted-host = xxxx:8443
"@ > C:\ProgramData\pip\pip.ini

Get-Content pip.ini works well, but pip config list -v returns:

PS C:\Program Files> pip config list -v
Configuration file could not be loaded.
File contains no section headers.
file: 'C:\\ProgramData\\pip\\pip.ini', line: 1
'ÿþ[\x00g\x00l\x00o\x00b\x00a\x00l\x00]\x00\n'

Remark: xxxx represents sensitive company information, therefore replaces real text.

I also tried to escape the square brackets with `, but without success.

Is there a way to specify some file encoding like UTF-8 above or can the problem be somehow solved in another automatized way?

Answer 1

In Windows Powershell, the redirection operators use Unicode encoding (in other words, UTF-16 with the little-endian byte order.) That's why you see such weird file content. Either

• run your code from PowerShell Core (pwsh.exe), or
• use the Out-File cmdlet with its Encoding parameter instead of > redirector.

However, note that Windows PowerShell tends to add a UTF-8 byte order mark using the following code snippet:

$MyRawString = @"
[global]
index = https://xxxx/nexus/repository/xxxx/pypi
index-url = https://xxxx:8443/nexus/repository/xxxx/simple
trusted-host = xxxx:8443
"@
$MyPath = "C:\ProgramData\pip\pip.ini"
$MyRawString | Out-file -FilePath $MyPath -Encoding utf8

Solution for Windows PowerShell. Using .NET's UTF8Encoding class and passing $False to the constructor seems to work (stolen from this M. Dudley's answer):

$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($MyPath, $MyRawString, $Utf8NoBomEncoding)

Explanation Get-Help About_Redirection

• Windows PowerShell (powershell.exe)

When you are writing to files, the redirection operators use Unicode encoding. If the file has a different encoding, the output might not be formatted correctly. To redirect content to non-Unicode files, use the Out-File cmdlet with its Encoding parameter.

• PowerShell Core (pwsh.exe, version 6+)

When you are writing to files, the redirection operators use UTF8NoBOM encoding. If the file has a different encoding, the output might not be formatted correctly. To write to files with a different encoding, use the Out-File cmdlet with its Encoding parameter.

Note that there is an error in the online version of the About_Redirection Help topic for PowersShell 5.1…

Answer 2

Out-File Default in Windows PowerShell 5.1 is Unicode in PowerShell Core 7+ it's UTF8.

You're using mkdir to create a file which doesn't make sense to me.

There's no need to pre-create the file anyhow. So, combining the above maybe something like:

@"
show_channel_urls: true
allow_other_channels: false
report_errors: false
remote_read_timeout_secs: 120
"@ | Out-File  C:\ProgramData\conda\.condarc

@"
[global]
index = https://xxxx/nexus/repository/xxxx/pypi
index-url = https://xxxx:8443/nexus/repository/xxxx/simple
trusted-host = xxxx:8443
"@ | Out-File C:\ProgramData\pip\pip.ini