CODEWITHSUNDEEP
powershellencoding
Maxime Rossini
Maxime Rossini

Reputation: 3879

Handle ASCII output when executing external exe within powershell

When invoking AWS CLI executable from within powershell (v5.1), special characters are not handled properly by powershell. Here is an example:

# This works: powershell is forwarding the output directly to stdout without touching it
> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vérification est {####}."
        [...]
    }
}

# This doesn't work: powershell handles the command's output encoding improperly
> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId | Out-Host
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vÚrification est {####}."
        [...]
    }
}

The problem is exactly the same when you want to save the output to a variable ($foo = aws ...) or redirect it to a file (| Out-File "foo.json" or > "foo.json"). Changing the output encoding (| Out-File "foo.json" -Encoding xxx) does not solve the issue, because the parsing error is made when powershell decodes the exe output, not when it outputs it to the file. The issue is not present when doing the same thing from CMD.

We could switch to the AWS tools for powershell (which should handle encoding properly) instead of using the standard AWS CLI, but I was wondering how you would solve this issue when there is no powershell alternative.

Upvotes: 1

Views: 241

Answers (1)

Maxime Rossini
Maxime Rossini

Reputation: 3879

As mentioned by @AdminOfThings in this comment, the encoding used by powershell to decode the output from an exe is managed by [Console]::OutputEncoding (whereas $OutputEncoding manages the the encoding used to send data to an exe).

The value of this variable in my case was ibm850 :

> [Console]::OutputEncoding

IsSingleByte      : True
BodyName          : ibm850
EncodingName      : Europe de l'Ouest (DOS)
HeaderName        : ibm850
WebName           : ibm850
WindowsCodePage   : 1252
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : False
IsMailNewsSave    : False
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : False
CodePage          : 850

Switching to iso-8859-1 solved my issue:

> [Console]::OutputEncoding = [System.Text.Encoding]::Default
> [Console]::OutputEncoding

IsSingleByte      : True
BodyName          : iso-8859-1
EncodingName      : Europe de l'Ouest (Windows)
HeaderName        : Windows-1252
WebName           : Windows-1252
WindowsCodePage   : 1252
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 1252

Then:

> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId | Out-Host
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vérification est {####}."
        [...]
    }
}

Upvotes: 1

Related Questions