PowerShell function that converts the raw body of a web-request response to a string based on the given character encoding.

ConvertTo-BodyWithEncoding.ps1

<#

  Prerequisites: Window PowerShell v5.1 and PowerShell (Core), on all supported platforms. (May work in earlier versions.)
  License: MIT
  Author:  Michael Klement <[email protected]>

  DOWNLOAD and INSTANT DEFINITION OF THE FUNCTION:

    irm https://gist.github.com/mklement0/209a9506b8ba32246f95d1cc238d564d/raw/ConvertTo-BodyWithEncoding.ps1 | iex

  The above directly defines the function below in your session and offers guidance for making it available in future
  sessions too. To silence the guidance information, append 4>$null
  CAVEAT: If you run this command *from a script*, you'll get a spurious warning about dot-sourcing, which you can ignore
  and suppress by appending 3>$null. However, it's best to avoid calling this command from scripts, because later versions
  of this Gist aren't guaranteed to be backward-compatible; howevever, you can modify the command to lock in a
  *specific revision* of this Gist, which is guaranteed not to change: see the instructions at
  https://gist.github.com/mklement0/880624fd665073bb439dfff5d71da886?permalink_comment_id=4296669#gistcomment-4296669

  DOWNLOAD ONLY:

    irm https://gist.github.com/mklement0/209a9506b8ba32246f95d1cc238d564d/raw | Set-Content -Encoding utf8 ./ConvertTo-BodyWithEncoding.ps1

  The above downloads to the specified file, which you then need to dot-source to make the function available
  in the current session (again, use 4>$null to silence the guidance information):

     . ./ConvertTo-BodyWithEncoding.ps1
   
  To learn what the function does:
    * see the next comment block
    * or, once downloaded and defined, invoke the function with -? or pass its name to Get-Help.

  To define an ALIAS for the function, (also) add something like the following to your $PROFILE:
  
    Set-Alias ctb ConvertTo-BodyWithEncoding

 #>


function ConvertTo-BodyWithEncoding {

<#
.SYNOPSIS
Converts the raw body of a web-request response to a string based on the given character encoding.

.DESCRIPTION
Use this command if the .Content property of a web-response object returned by Invoke-WebRequest
misinterprets the response due to either a missing or an invalid "charset" attribute in the
"Content-Type" header field of the response.

The raw bytes are decoded based on the specified character encoding, which defaults to UTF-8.

Note that the default encoding assumed by Invoke-WebRequest / Invoke-RestMethod is
ISO-8859-1 (loosely speaking, Windows-1252) in PowerShell versions up to v7.2.x,
and UTF-8 since v7.3.0

.PARAMETER InputObject
A web-request response as output by Invoke-WebRequest.
Best to provide it via the pipeline.

.PARAMETER Encoding
The character encoding to use to decode the response body's raw bytes into a string.
Defaults to UTF-8.

Pass a [Text.Encoding] instance, a code-page number (e.g. 1251), or a name (e.g. "utf-16le")
In the latter two cases, the value is passed to [Text.Encoding]::GetEncoding().

.EXAMPLE
Invoke-WebRequest https://www.ukrlib.com.ua/ | ConvertTo-BodyWithEncoding -Encoding 1251

.NOTES
This command's code is also used in the form of a function at https://stackoverflow.com/a/47961370/45375
#>

[CmdletBinding(PositionalBinding=$false)]
param(
  [Parameter(Mandatory, ValueFromPipeline)]
  [Microsoft.PowerShell.Commands.WebResponseObject] $InputObject,
  # The encoding to use; defaults to UTF-8
  [Parameter(Position=0)]
  $Encoding = [System.Text.Encoding]::Utf8
)

begin {
  if ($Encoding -isnot [System.Text.Encoding]) {
    try {
      $Encoding = [System.Text.Encoding]::GetEncoding($Encoding)
    }
    catch { 
      throw
    }
  }
}

process {
  $Encoding.GetString(
     $InputObject.RawContentStream.ToArray()
  )
}

}


# --------------------------------
# GENERIC INSTALLATION HELPER CODE
# --------------------------------
#    Provides guidance for making the function persistently available when
#    this script is either directly invoked from the originating Gist or
#    dot-sourced after download.
#    IMPORTANT: 
#       * DO NOT USE `exit` in the code below, because it would exit
#         the calling shell when Invoke-Expression is used to directly
#         execute this script's content from GitHub.
#       * Because the typical invocation is DOT-SOURCED (via Invoke-Expression), 
#         do not define variables or alter the session state via Set-StrictMode, ...
#         *except in child scopes*, via & { ... }
if ($MyInvocation.Line -eq '') {
  # Most likely, this code is being executed via Invoke-Expression directly 
  # from gist.github.com
  # To simulate for testing with a local script, use the following:
  # Note: Be sure to use a path and to use "/" as the separator.
  #  iex (Get-Content -Raw ./script.ps1)
  # Derive the function name from the invocation command, via the enclosing
  # script name presumed to be contained in the URL.
  # NOTE: Unfortunately, when invoked via Invoke-Expression, $MyInvocation.MyCommand.ScriptBlock
  #       with the actual script content is NOT available, so we cannot extract
  #       the function name this way.
  & {
    
    param($invocationCmdLine)
    
    # Try to extract the function name from the URL.
    $funcName = $invocationCmdLine -replace '^.+/(.+?)(?:\.ps1).*$', '$1'
    if ($funcName -eq $invocationCmdLine) {
      # Function name could not be extracted, just provide a generic message.
      # Note: Hypothetically, we could try to extract the Gist ID from the URL
      #       and use the REST API to determine the first filename.
      Write-Verbose -Verbose "Function is now defined in this session."
    } 
    else {
      # Indicate that the function is now defined and also show how to
      # add it to the $PROFILE or convert it to a script file.
      Write-Verbose -Verbose @"
Function `"$funcName`" is now defined in this session.

* If you want to add this function to your `$PROFILE, run the following:

   "``nfunction $funcName {``n`${function:$funcName}``n}" | Add-Content `$PROFILE

* If you want to convert this function to a script file that you can invoke
  directly, run:
  
  "`${function:$funcName}" | Set-Content ./$funcName.ps1 -Encoding $('utf8' + ('', 'bom')[[bool] (Get-Variable -ErrorAction Ignore IsCoreCLR -ValueOnly)])
  
"@
    }
  } $MyInvocation.MyCommand.Definition # Pass the original invocation command line to the script block.
}
else {
  # Invocation presumably as a local file after manual download, 
  # either dot-sourced (as it should be) or mistakenly directly.  
  & {
    param($originalInvocation)
    # Parse this file to reliably extract the name of the embedded function, 
    # irrespective of the name of the script file.
    $ast = $originalInvocation.MyCommand.ScriptBlock.Ast
    $funcName = $ast.Find( { $args[0] -is [System.Management.Automation.Language.FunctionDefinitionAst] }, $false).Name
    if ($originalInvocation.InvocationName -eq '.') {
      # Being dot-sourced as a file.
      
      # Provide a hint that the function is now loaded and provide
      # guidance for how to add it to the $PROFILE.
      Write-Verbose -Verbose @"
Function `"$funcName`" is now defined in this session.

If you want to add this function to your `$PROFILE, run the following:

    "``nfunction $funcName {``n`${function:$funcName}``n}" | Add-Content `$PROFILE

"@
    }
    else {
      # Mistakenly directly invoked.
      # Issue a warning that the function definition didn't take effect and
      # provide guidance for reinvocation and adding to the $PROFILE.
      Write-Warning @"
This script contains a definition for function "$funcName", but this definition
only takes effect if you dot-source this script.

To define this function for the current session, run:
  
  . "$($originalInvocation.MyCommand.Path)"
  
"@
    } 
  }  $MyInvocation # Pass the original invocation info to the helper script block.
}