CODEWITHSUNDEEP
pythonsqlregexsedbase64
hankivstmb
hankivstmb

Reputation: 171

Find, decode and replace all base64 values in text file

I have a SQL dump file that contains text with html links like:

<a href="http://blahblah.org/kb/getattachment.php?data=NHxUb3Bjb25fZGF0YS1kb3dubG9hZF9ob3d0by5wZGY=">attached file</a>

I'd like to find, decode and replace the base64 part of the text in each of these links.

I've been trying to use Python w/ regular expressions and base64 to do the job. However, my regex skills are not up to the task.

I need to select any string that starts with 

'getattachement.php?data='

and ends with 

'"'

I then need to decode the part between 'data=' and '&quot' using base64.b64decode()

results should look something like:

<a href="http://blahblah.org/kb/4/Topcon_data-download_howto.pdf">attached file</a>

I think the solution will look something like:

import re
import base64
with open('phpkb_articles.sql') as f:
    for line in f:
        re.sub(some_regex_expression_here, some_function_here_to_decode_base64)

Any ideas?

EDIT: Answer for anyone who's interested.

import re
import base64
import sys


def decode_base64(s):
    """
    Method to decode base64 into ascii
    """
    # fix escaped equal signs in some base64 strings
    base64_string = re.sub('%3D', '=', s.group(1))
    decodedString = base64.b64decode(base64_string)

    # substitute '|' for '/'
    decodedString = re.sub('\|', '/', decodedString)

    # escape the spaces in file names
    decodedString = re.sub(' ', '%20', decodedString)

    # print 'assets/' + decodedString + '&quot'  # Print for debug
    return 'assets/' + decodedString + '&quot'


count = 0

pattern = r'getattachment.php\?data=([^&]+?)&quot'

# Open the file and read line by line
with open('phpkb_articles.sql') as f:
    for line in f:
        try:
            # globally substitute in new file path
            edited_line = re.sub(pattern, decode_base64, line)
            # output the edited line to standard out
            sys.stdout.write(edited_line)
        except TypeError:
            # output unedited line if decoding fails to prevent corruption
            sys.stdout.write(line)
            # print line
            count += 1

Upvotes: 2

Views: 3580

Answers (1)

R Nar
R Nar

Reputation: 5515

you already have it, you just need the small pieces:

pattern: r'data=([^&]+?)&quot' will match anything after data= and before &quot

>>> pat = r'data=([^&]+?)&quot'
>>> line = '<a href="http://blahblah.org/kb/getattachment.php?data=NHxUb3Bjb25fZGF0YS1kb3dubG9hZF9ob3d0by5wZGY=">attached file</a>'
>>> decodeString = re.search(pat,line).group(1) #because the b64 string is capture by grouping, we only want group(1)
>>> decodeString
'NHxUb3Bjb25fZGF0YS1kb3dubG9hZF9ob3d0by5wZGY='

you can then use str.replace() method as well as base64.b64decode() method to finish the rest. I dont want to just write your code for you but this should give you a good idea of where to go.

Upvotes: 2

Related Questions