CODEWITHSUNDEEP
powershellscripting
SADDAS16
SADDAS16

Reputation: 19

How to parse logs and mask specific characters using Powershell

I have a problem that I really hope to get some help with. It's rather complex but I will try and keep my explanation as simple and objective as possible. In a nutshell, I have log files that contain thousands of lines. Each line consists of information like date/time, source, type and message. In this case the message contains a variable size ...999 password that I need to mask. Basically the message looks something like this (its an ISO message):

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

For each line I need to zero in on password length identifier (008) do a count on it and then proceed to mask the number of following characters, which would be PASSWORD in this case. I would change it to something like XXXXXXXX instead so once done the line would look like this:

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008XXXXXXXX6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

I honestly have no idea how to start doing this with PowerShell. I need to loop though each line in the log file, and identify the number of characters to mask.

I've kept this high level as a starting point, there are some other complexities that I hope to figure out at a later time, like the fact that there are different types of messages and depending on the type the password length starts at another character position. I might be able to build on my aforementioned question first but if anyone understands what I mean then I would appreciate some help or tips about that too.

Any help is appreciated.

Thanks!

Additional information to original post:

Firstly, thank you to everyone for your answers thus far, its been greatly appreciated. Now that I have a baseline for how your answers are being formulated based on my information I feel I need to provide some more details.

1) There was a question about whether or not the password starting position is fixed and the logic behind it.

2) This is what I know and these are the steps I hope to accomplish with the script.

What I know: - There are 3 different msg types that contain passwords. I've figured out where the starting position of the password is for each msg type based on the bitmap and the data elements present. For example 0210 contains one in this case:

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

What I need to do:

  1. Pass the log file to the script
  2. For each line in the log identify if the line has a msg type that contains a password
  3. If the message type contains a password then determine length of password by reading the preceding 3 digits to the password ("ans ...999" which means alphanumeric - special with length max of 999 and 3 digit length info). Lets say the character position of the password would be 107 in this case for arguments sake, so we know to read the 3 numbers before it.
  4. Starting at the character position of the password, mask the number of characters required with XXX. Loop through log until complete.

Upvotes: 1

Views: 1221

Answers (2)

Mike Frank
Mike Frank

Reputation: 389

It does seem as though you're indicating the position of the password and the length of the password will vary. As long as you have the '008' and something like '666' to indicate a starting and stopping point something like this should work. 

$filePath = '.\YourFile.log'

(Get-Content $filePath) | ForEach-Object {
    $startIndex = $_.IndexOf('008') + 3
    $endIndex = $_.IndexOf('666', $startIndex)
    $passwordLength = $endIndex - $startIndex
    $passwordToReplace = $_.Substring($startIndex,$passwordLength)
    $obfuscation = New-Object 'string' -ArgumentList 'X', $passwordLength
    $_.Replace($passwordToReplace, $obfuscation)
} | Set-Content $filePath

If the file is too large to load into memory then you will have to StreamReader and StreamWriter to write the content to a new file and delete the old.

Upvotes: 1

mklement0
mklement0

Reputation: 439767

Assuming a fixed position where the password-length field starts, based on your sample line (if that position is variable, as you've hinted at, you need to tell us more): 

$line = '22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff'

$posStart = 62 # fixed 0-based pos. where length-of-password field stats
$pwLenFieldLen = 3  # length of length-of-password field
$pwLen = [int] $line.SubString($posStart, $pwLenFieldLen) # extract password length
$pwSubstitute = 'X' * $pwLen # determine the password replacement string

# replace the password with all Xs
$line -replace "(?<=^.{$($posStart + $pwLenFieldLen)}).{$pwLen}(?=.*)", $pwSubstitute

Note: This is not the most efficient way to do it, but it is concise.

Upvotes: 0

Related Questions