CODEWITHSUNDEEP
powershellweb-scrapingcookiessession-cookies
Emil Schlitzohr
Emil Schlitzohr

Reputation: 55

Using Chrome Cookies for Invoke-WebRequest in Powershell

Is there a way to use already obtained cookies from an existing Chrome web session in Powershell for the SessionVariable?

I know the fields for SessionVariable are:

Headers               : {}
Cookies               : System.Net.CookieContainer
UseDefaultCredentials : False
Credentials           : 
Certificates          : 
UserAgent             : Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) 
                        WindowsPowerShell/5.1.17134.407
Proxy                 : 
MaximumRedirection    : -1

Am I able to do something to set these values initially for a webrequest, such as: ? Would I need specific headers for a session when it could be multiple/different types of requests?

$ua = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
    $Params = @{
        Headers = $headers 
        Cookies = $cookies 
        UseDefaultCredentials = "False" 
        Credentials = $creds
        UserAgent = $ua
        Proxy = ""
        MaximumRedirection = "-1"
    }

If something like that can be done, I am also a little confused how I would input the cookies. I found this from Getting Cookies using PowerShell :

$webrequest = Invoke-WebRequest -Uri $url -SessionVariable websession 
$cookies = $websession.Cookies.GetCookies($url) 
 
# Here, you can output all of $cookies, or you can go through them one by one. 
 
foreach ($cookie in $cookies) { 
     # You can get cookie specifics, or just use $cookie 
     # This gets each cookie's name and value 
     Write-Host "$($cookie.name) = $($cookie.value)" 
}

I have a JSON format export of cookies from any given site in this format, but I can also cut that down to just "name" = "value" if that is what is needed.

[
  {
    "domain": "",
    "expirationDate": "",
    "hostOnly": "",
    "httpOnly": "",
    "name": "",
    "path": "",
    "sameSite": "",
    "secure": "",
    "session": "",
    "storeId": "",
    "value": "",
    "id": ""
  },
  {
    "domain": "",
    etc...
  }
]

I didn't know how to designate each section as a $cookie in $cookies, nor how to add them in a different manner since it will not be from a URL/WebRequest but instead a JSON.

Thanks for any help!

Upvotes: 4

Views: 7421

Answers (1)

Mavaddat Javid
Mavaddat Javid

Reputation: 686

Here's the solution as PowerShell script using mySQLite and Bouncy Castle (GitHub Logo Gist):

# One time setup
if (-not (Get-Package 'Portable.BouncyCastle' -ErrorAction Ignore)) {
    if (-not (Get-PackageSource -Name NuGet -ErrorAction Ignore)) {
        Register-PackageSource -Name NuGet -Location https://api.nuget.org/v3/index.json -ProviderName NuGet | Set-PackageSource -Trusted
    }
    Install-Package -Name 'Portable.BouncyCastle' -Source NuGet -Scope CurrentUser -SkipDependencies
}
# Download the MySQLite repository (for PowerShell <5 without PowerShellGet)
if (-not(Get-Module -Name MySQLite -ErrorAction Ignore) -and ($PSVersionTable.PSVersion.Major -lt 5)) {
    $RepositoryZipUrl = 'https://api.github.com/repos/jdhitsolutions/MySQLite/zipball/master'
    Invoke-RestMethod -Uri $RepositoryZipUrl -OutFile 'MySQLite.zip'
    # Unblock the zip
    Unblock-File 'MySQLite.zip'
    # Extract the MySQLite folder to a module path (e.g. $env:USERPROFILE\Documents\WindowsPowerShell\Modules\)
    Expand-Archive -Path 'MySQLite.zip' -DestinationPath $($env:PSModulePath -split [System.IO.Path]::PathSeparator | Where-Object { (Test-Path -Path $_ -PathType Container -ErrorAction SilentlyContinue) -and $(try { $tmp = New-Item -Path $_ -Name ([System.IO.Path]::GetRandomFileName()) -ItemType File -Value (Get-Random) -ErrorAction SilentlyContinue; Remove-Item -Path $tmp; $true | Write-Output } catch { $false | Write-Output } ) } | Select-Object -First 1) -Force -Confirm
}
elseif (-not(Get-Module -Name MySQLite -ErrorAction Ignore) -and ($PSVersionTable.PSVersion.Major -ge 5)) {
    #Simple alternative, if you have PowerShell ≥5, or the PowerShellGet module:
    Install-Module MySQLite -Repository PSGallery -Scope CurrentUser
}

# Import the MySQLite module
Import-Module MySQLite    #Alternatively, Import-Module \\Path\To\MySQLite
# Import Bouncy Castle Classes
Get-Package 'Portable.BouncyCastle' | ForEach-Object { Add-Type -LiteralPath ($_.Source | Split-Path | Get-ChildItem -Filter 'netstandard*' -Recurse -Directory | Get-ChildItem -Filter *.dll -Recurse -File ).FullName }
# Specify for which domain you want to retrieve cookies
$domain = 'mavaddat.ca'
$cookiesPath = "$env:LOCALAPPDATA\Google\Chrome Beta\User Data\Default\Network\Cookies" # "$env:LOCALAPPDATA\Microsoft\Edge\User Data\Default\Network\Cookies" # "$env:APPDATA\Opera Software\Opera Stable\Cookies"

# Investigate the db structure
Get-MySQLiteTable -Path $cookiesPath -Detail

# Based on the schema of table `cookies`, form the query
$query = "SELECT name,encrypted_value,path,host_key FROM `"main`".`"cookies`" WHERE `"host_key`" LIKE '%$domain%' ESCAPE '\' LIMIT 0, 49999;"

# Or, get all cookies for all domains
$query = "SELECT name,encrypted_value,path,host_key FROM `"main`".`"cookies`" LIMIT 0, 49999;"

# Read the cookies from the SQLite
$cookies = Invoke-MySQLiteQuery -Path $cookiesPath -Query $query

# Get Chromium cookie master key
$localStatePath = "$env:LOCALAPPDATA\Google\Chrome Beta\User Data\Local State" # "$env:LOCALAPPDATA\Microsoft\Edge\User Data\Local State" # "$env:APPDATA\Opera Software\Opera Stable\Local State"
$cookiesKeyEncBaseSixtyFour = (Get-Content -Path $localStatePath | ConvertFrom-Json).'os_crypt'.'encrypted_key'
$cookiesKeyEnc = [System.Convert]::FromBase64String($cookiesKeyEncBaseSixtyFour) | Select-Object -Skip ([System.Text.Encoding]::UTF8.GetBytes('DPAPI').Count)
$cookiesKey = [System.Security.Cryptography.ProtectedData]::Unprotect($cookiesKeyEnc, $null, [System.Security.Cryptography.DataProtectionScope]::LocalMachine)

# Create a web session object for the IWR work
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Prep the cipher elements
$cipher = [Org.BouncyCastle.Crypto.Modes.GcmBlockCipher]::new([Org.BouncyCastle.Crypto.Engines.AesEngine]::new())

# Stuff the cookies into the session
foreach ($cookie in $cookies) {
    $path = [string]::IsNullOrEmpty($cookie.path) ? '/' : $cookie.path
    try {
        $cipherStream = [System.IO.MemoryStream]::new($cookie.encrypted_value)
        $cipherReader = [System.IO.BinaryReader]::new($cipherStream)
    
    
        <# 
        # We don't need to keep the non-secret payload; however, this is how to retrieve it (spoiler alert, it's 'v10')
        $nonSecretPayload = $cipherReader.ReadBytes(([System.Text.Encoding]::ASCII.GetBytes('v10').Count)) # 
        
        # if you want to read this, just use 
        [System.Text.Encoding]::Default.GetString($nonSecretPayload) | Out-Host
        #>

        # Alternatively, if you don't care about 'v10', move the stream pointer past it
        $cipherReader.BaseStream.Position = [System.Text.Encoding]::ASCII.GetBytes('v10').Count

        $nonce = $cipherReader.ReadBytes([System.Security.Cryptography.AesGcm]::NonceByteSizes.MinSize)

        $parameters = [Org.BouncyCastle.Crypto.Parameters.AeadParameters]::new( ([Org.BouncyCastle.Crypto.Parameters.KeyParameter]::new($cookiesKey)), ([System.Security.Cryptography.AesGcm]::TagByteSizes.MaxSize * [byte]::MaxValue.GetShortestBitLength()), $nonce)
        $cipher.Init($false, $parameters)
        $cipherText = $cipherReader.ReadBytes($cookie.encrypted_value.Length)
        $plainText = [byte[]]::new($cipher.GetOutputSize($cipherText.Length))
        if (-not [string]::IsNullOrEmpty($plainText)) {
            try {
                $len = $cipher.ProcessBytes($cipherText, 0, $cipherText.Length, $plainText, 0)
                $bytesDeciphered = $cipher.DoFinal($plainText, $len)
                Write-Verbose "Deciphered $bytesDeciphered bytes"
            }
            catch [System.Management.Automation.MethodInvocationException] {
                # if inner exception [Org.BouncyCastle.Crypto.InvalidCipherTextException]
                if ($_.Exception.InnerException -is [Org.BouncyCastle.Crypto.InvalidCipherTextException]) {
                    Write-Error 'Invalid Cipher Text'
                }
                else {
                    Write-Error $_ # Echo the error unless you have a better way to handle
                }
                continue
            }
            finally {
                $cipher.Reset()
            }
            try {
                $session.Cookies.Add([System.Net.Cookie]::new(($cookie.name), [System.Text.Encoding]::Default.GetString($plainText), $path, ($cookie.host_key -replace '^\.')))
            }
            catch [System.Management.Automation.MethodInvocationException] {
                if ($_.Exception.InnerException -is [System.Net.CookieException]) {
                    $session.Cookies.Add([System.Net.Cookie]::new(($cookie.name), [System.Web.HttpUtility]::UrlEncode([System.Text.Encoding]::Default.GetString($plainText)), $path, ($cookie.host_key -replace '^\.')))
                }
                else {
                    Write-Error $_ # Echo the error unless you have a better way to handle
                }
            }
        }
    }
    finally {
        $cipherStream.Dispose()
        $cipherReader.Dispose()
    }
}

# Remove sensitive objects
$cipherReader = $null
$cipherStream = $null
$cookiesKey = $null
$cookiesKeyEnc = $null
$cookiesKeyEncBaseSixtyFour = $null
$nonce = $null
$cipher = $null
$cipherText = $null
$plainText = $null

Remove-Variable cipher, cipherReader, cipherStream, cookiesKey, cookiesKeyEnc, cookiesKeyEncBaseSixtyFour, nonce, cipherText, plainText

# Do IWR Work
Invoke-WebRequest -Uri $domain -WebSession $session

How I realized the above

There are two Chromium files that need to be queried:

  1. User Data local state
  2. Cookies in SQLite

The User Data is a JSON formatted file that keeps the encryption key for the cookies. For me (using Chrome Beta 107.0.5304.18 on Windows 11), this is at %LOCALAPPDATA%\Google\Chrome Beta\User Data\Local State.

The Chromium-based browser uses cookies stored in a binary SQLite database file. See Superuser question on how to find this database for Chrome, "Is there a way to watch cookies and their values live?".

For me, this file is named Cookies and it lives at %LOCALAPPDATA%\Google\Chrome Beta\User Data\Default\Network\Cookies. Using MySQLite, I can see that the database has two tables with the following schemas:

Cookies table

Name Type Schema
creation_utc INTEGER "creation_utc" INTEGER NOT NULL
host_key TEXT "host_key" TEXT NOT NULL
name TEXT "name" TEXT NOT NULL
value TEXT "value" TEXT NOT NULL
path TEXT "path" TEXT NOT NULL
expires_utc INTEGER "expires_utc" INTEGER NOT NULL
is_secure INTEGER "is_secure" INTEGER NOT NULL
is_httponly INTEGER "is_httponly" INTEGER NOT NULL
last_access_utc INTEGER "last_access_utc" INTEGER NOT NULL
has_expires INTEGER "has_expires" INTEGER NOT NULL DEFAULT 1
is_persistent INTEGER "is_persistent" INTEGER NOT NULL DEFAULT 1
priority INTEGER "priority" INTEGER NOT NULL DEFAULT 1
encrypted_value BLOB "encrypted_value" BLOB DEFAULT ''
samesite INTEGER "samesite" INTEGER NOT NULL DEFAULT -1
source_scheme INTEGER "source_scheme" INTEGER NOT NULL DEFAULT 0
source_port INTEGER "source_port" INTEGER NOT NULL DEFAULT -1
is_same_party INTEGER "is_same_party" INTEGER NOT NULL DEFAULT 0

Meta table

Name Type Schema
key LONGVARCHAR "key" LONGVARCHAR NOT NULL UNIQUE
value LONGVARCHAR "value" LONGVARCHAR

See stackoverflow question "How to read Brave Browser cookie database encrypted values in C# (.NET Core)?" Additionally, user @michael-fromberger details the Chromium-based cookie structure here: Google Chrome Encrypted Cookies — this details where many seemingly adventitious strings (DPAPI, v10) in the above code originate. I also followed the PowerShell cookie method of this gist by @lawrencegripper as a template.

Upvotes: 2

Related Questions