CODEWITHSUNDEEP
regexperl
m0skit0
m0skit0

Reputation: 25874

Regex not matching data and dates

I have an SQL Select dump with many lines each looks like this:

07/11/2011 16:48:08,07/11/2011 16:48:08,'YD','MANUAL',0,1,'text','text','text','text',,,,'text',,,0,0,

I want to do 2 things to each line:

  1. Replace all dates with Oracle's sysdate function. Dates can also come without hour (like 07/11/2011).
  2. Replace all null values with null string

Here's my attempt:

$_ =~ s/,(,|\n)/,null$1/g;                  # Replace no data by "null"
$_ =~ s/\d{2}\/\d{2}\/d{4}.*?,/sysdate,/g;  # Replace dates by "sysdate"

But this would transform the string to:

07/11/2011 16:48:08,07/11/2011 16:48:08,'YD','MANUAL',0,1,'text','text','text','text',null,,null,'text',null,,0,0,null

while I expect it to be

sysdate,sysdate,'YD','MANUAL',0,1,'text','text','text','text',null,null,null,'text',null,null,0,0,null

I don't understand why dates do not match and why some ,, are not replaced by null.

Any insights welcome, thanks in advance.

Upvotes: 1

Views: 243

Answers (4)

user557597
user557597

Reputation:

\d{2}\/\d{2}\/d{4}.*?, didn't work because the last d wasn't escaped.
If a , can be on either side, or begin/end of string, you could do it in 2 steps:

step 1
s/(?:^|(?<=,))(?=,|\n)/null/g
expanded:

/
  (?:  ^           # Begining of line, ie: nothing behind us
     | (?<=,)      # Or, a comma behind us
  )
     # we are HERE!, this is the place between characters
  (?=  ,           # A comma in front of us
     | \n          # Or, a newline in front of us
  )
/null/g
# The above regex does not consume, it just inserts 'null', leaving the
# same search position (after the insertion, but before the comma).

# If you want to consume a comma, it would be done this way:
s/(?:^|(?<=,))(,|\n)/null$1/xg
# Now the search position is after the 'null,'

step 2
s/(?:^|(?<=,))\d{2}\/\d{2}\/\d{4}.*?(?=,|\n)/sysdate/g

Or, you could combine them into a single regex, using the eval modifier:
$row =~ s/(?:^|(?<=,))(\d{2}\/\d{2}\/\d{4}.*?|)(?=,|\n)/ length $1 ? 'sysdate' : 'null'/eg;

Broken down it looks like this 

s{
   (?: ^ | (?<=,) )  # begin of line or comma behind us
   (                 # Capt group $1
       \d{2}/\d{2}/\d{4}.*?     # date format and optional non-newline chars
     |                          # Or, nothing at all
   )                 # End Capt group 1
  (?= , | \n )       # comma or newline in front of us
}{
   length $1 ? 'sysdate' : 'null'
}eg

If there is a chance of non-newline whitespace padding, it could be written as:

$row =~ s/(?:^|(?<=,))(?:([^\S\n]*\d{2}\/\d{2}\/\d{4}.*?)|[^\S\n]*)(?=,|\n)/ defined $1 ? 'sysdate' : 'null'/eg;

Upvotes: 1

FailedDev
FailedDev

Reputation: 26940

You want to replace something. Usually lookaheads are a better option for this : 

$subject =~ s/(?<=,)(?=,|$)/null/g;

Explanation : 

"
(?<=       # Assert that the regex below can be matched, with the match ending at this position (positive lookbehind)
   ,          # Match the character “,” literally
)
(?=        # Assert that the regex below can be matched, starting at this position (positive lookahead)
              # Match either the regular expression below (attempting the next alternative only if this one fails)
      ,          # Match the character “,” literally
   |          # Or match regular expression number 2 below (the entire group fails if this one fails to match)
      \$          # Assert position at the end of the string (or before the line break at the end of the string, if any)
)
"

Secodnly you wish to replace the dates : 

$subject =~ s!\d{2}/\d{2}/\d{4}.*?(?=,)!sysdate!g;

That's almost the same with your original regex. Just replace the last , with lookahead. (If you don't want to replace it , don't match it.)

# \d{2}/\d{2}/\d{4}.*?(?=,)
# 
# Match a single digit 0..9 «\d{2}»
#    Exactly 2 times «{2}»
# Match the character “/” literally «/»
# Match a single digit 0..9 «\d{2}»
#    Exactly 2 times «{2}»
# Match the character “/” literally «/»
# Match a single digit 0..9 «\d{4}»
#    Exactly 4 times «{4}»
# Match any single character that is not a line break character «.*?»
#    Between zero and unlimited times, as few times as possible, expanding as needed (lazy) «*?»
# Assert that the regex below can be matched, starting at this position (positive lookahead) «(?=,)»
#    Match the character “,” literally «,»

Upvotes: 1

Lucas
Lucas

Reputation: 14979

You could do this:

$ cat perlregex.pl
use warnings;
use strict;

my $row = "07/11/2011 16:48:08,07/11/2011 16:48:08,'YD','MANUAL',0,1,'text','text','text','text',,,,'text',,,0,0,\n";

print( "$row\n" );
while ( $row =~ /,([,\n])/ ) { $row =~ s/,([,\n])/,null$1/; }
print( "$row\n" );
$row =~ s/\d{2}\/\d{2}\/\d{4}.*?,/sysdate,/g;
print( "$row\n" );

Which results in this:

$ ./perlregex.pl
07/11/2011 16:48:08,07/11/2011 16:48:08,'YD','MANUAL',0,1,'text','text','text','text',,,,'text',,,0,0,

07/11/2011 16:48:08,07/11/2011 16:48:08,'YD','MANUAL',0,1,'text','text','text','text',null,null,null,'text',null,null,0,0,null

sysdate,sysdate,'YD','MANUAL',0,1,'text','text','text','text',null,null,null,'text',null,null,0,0,null

This could certainly be optimized, but it gets the point across.

Upvotes: 1

Gilles Qu&#233;not
Gilles Qu&#233;not

Reputation: 185871

Maybe .*? is too greedy, try:

$_ =~ s/\d{2}\/\d{2}\/d{4}[^,]+,/sysdate,/g;

Upvotes: 0

Related Questions