CODEWITHSUNDEEP
regexpowershell
EGAD
EGAD

Reputation: 51

Extract data from .log file with Regex

I'm trying to extract data using Regex positive lookbehind. I have created a .ps1 file with the following content:

$input_path = ‘input.log’

$output_file = ‘Output.txt’

$regex = ‘(?<=    "name": ")(.*)(?=",)|(?<=    "fullname": ")(.*)(?=",)|(?<=Start identity token validation\r\n)(.*)(?=ids: Token validation success)|(?<=  "ClientName": ")(.*)(?=",\r\n  "ValidateLifetime": false,)’

select-string -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } >$output_file

The input file looks like this: 

08:15.27.47-922: T= 11 ids: Start end session request
08:15.27.47-922: T= 11 ids: Start end session request validation
08:15.27.47-922: T= 11 ids: Start identity token validation
08:15.27.47-922: T= 11 ids: Token validation success
{
  "ClientId": "te_triouser",
  "ClientName": "TE Trio User",
  "ValidateLifetime": false,
  "Claims": {
    "iss": "http://sv-trio17.adm.linkoping.se:34000/core/",
    "aud": "te_triouser",
    "exp": "1552054900",
    "nbf": "1552054600",
    "nonce": "f1ae9044-25f9-4e7f-b39f-bd7bdcb9dc8d",
    "iat": "1552054600",
    "at_hash": "Wv_7nNe42gUP945FO4p0Wg",
    "sid": "9870230d92cb741a8674313dd11ae325",
    "sub": "23223",
    "auth_time": "1551960154",
    "idp": "tecs",
    "name": "tele2",
    "canLaunchAdmin": "1",
    "isLockedToCustomerGroup": "0",
    "customerGroupId": "1",
    "fullname": "Tele2 Servicekonto Test",
    "tokenIdentifier": "2Ljta5ZEovccNlab9QXb8MPXOqaBfR6eyKst/Dc4bF4=",
    "tokenSequence": "bMKEXP9urPigRDUguJjvug==",
    "tokenChecksum": "NINN0DDZpx7zTlxHqCb/8fLTrsyB131mWoA+7IFjGhAV303///kKRGQDuAE6irEYiCCesje2a4z47qvhEX22og==",
    "idpsrv_lang": "sv-SE",
    "CD_UserInfo": "23223 U2 C1",
    "amr": "optional"
  }
}

If i run the regex through http://regexstorm.net/tester i get the right matches. But when i run my script with powershell on my computer I dont get the matches where I have \r\n in the regex question. I only get the matches from the first two regex questions.

Upvotes: 5

Views: 341

Answers (1)

user6811411
user6811411

Reputation:

  • I agree with @AdminOfThings to use Get-Content with the -raw parameter.
  • also don't use typographic quotes in scripts.
  • If the number of leading spaces aren't really fixed replace with one space and + or * quantifier.
  • make the \r optional => \r?.

A minimal complete verifiable example should also include your expected output.

EDIT changed Regex to be better readable

The following script 

## Q:\Test\2019\03\22\SO_55298614.ps1

$input_path = 'input.log'
$output_file = 'Output.txt'

$regexes = ('(?<= *"(full)?name": ")(.*)(?=",)',
            '(?<=Start identity token validation\r?\n)(.*)(?=ids: Token validation success)',
            '(?<= *"ClientName": ")(.*)(?=",\r?\n *"ValidateLifetime": false,)')

$regex = [RegEx]($regexes -join'|')


Get-Content $input_path -Raw | Select-String -pattern $regex -AllMatches | 
   ForEach-Object { $_.Matches.Value }

yields this sample output:

> Q:\Test\2019\03\22\SO_55298614.ps1
08:15.27.47-922: T= 11
TE Trio User
tele2
Tele2 Servicekonto Test

Upvotes: 2

Related Questions