How do I match over multiple lines in perl

Question

Lets say I have a netlist file formatted like so for each module:

module module_name1(in1, in2,
    in3, in4, in5,
    out1, out2, out3
    out4, out5);

There are many of these throughout the netlist. I want to be able to grab the module name and the list of ports. Here is what I have so far:

use strict;
use warnings;

my $input_file = $ARGV[0];
open (my $INFILE, $input_file) or die "$input_file cannot be opened.\n";

my $outfile = "verilog.port.txt";
open (my $OUTFILE, '>', $outfile) or die "\nUnable to create $outfile\n";

my ($module_name,$port_list);

while (<>) {
  if ($_ =~ /module (\w+)\((.+)\)/m) {
    $module_name = $1;
    $port_list = $2;
    print $OUTFILE "Module Name: $module_name Port list: $port_list\n"
  }
}
close $INFILE;

close $OUTFILE;

This will only work if the module is instantiated in only 1 line. For example if :

module module_name2(in1, in2, out1, out2);

and I will get something like:

Module Name: module_name2 Port list: in1, in2, out1, out2

However if the module is created over multiple lines like in my first example, my reg expression cannot pick it up. So I was wondering if there is a way to match through multiple lines using perl.

Answer 1

I have to disagree that reading line-by-line is 'inappropriate' when perl has the .. range operator.

Take the OP code and modify as such:

while (<>) {
    if (/module/ .. /\)/) {
        $module_name = $1 if /module\s+(\w+)/;
        my $done=/\)/;
        s/.*\(//; s/\).*//;s/,\s+/, /g;
        chomp;
        $port_list .= $_;
        print $OUTFILE "Module Name: $module_name Port list $port_list\n" if $done;
    }
}

In other words, from lines matching /module/ to lines matching /)/, accumulate the port list.

Answer 2

You are reading files line by line, you need to read them either by paragraph (chunks separated by a empty line) or the entire file if if there is no such separation; otherwise $_ contains only one line and will not match.

Also, the /m flag is not what you are looking for (/m makes ^/$ match beginning/end of lines), you need /s which makes . include newlines (see: the perlreref documentation page, the perlop page is a bit confusing)

By paragraph, this one liner should do the trick:

$ perl -l -00 -ne 'if ( /module (\w+)\((.+)\)/s) { @ports = split(/\s*,\s*/,$2); print "Module name: $1 Ports: " . join(", ", @ports)}' <<'EOF'
> module module_name1(in1, in2,
>     in3, in4, in5,
>     out1, out2, out3,
>     out4, out5);
>
>
> module module_name2(in21, in22,
>     in23, in24, in25,
>     out21, out22, out23,
>     out24, out25);
> EOF
Module name: module_name1 Ports: in1, in2, in3, in4, in5, out1, out2, out3, out4, out5
Module name: module_name2 Ports: in21, in22, in23, in24, in25, out21, out22, out23, out24, out25

You can use -MO=Deparse to see the entire code:

perl -MO=Deparse -l -00 -ne 'if ( /module (\w+)\((.+)\)/s) { @ports = split(/\s*,\s*/,$2); print "Module name: $1 Ports: " . join(", ", @ports)}'
BEGIN { $/ = ""; $\ = "\n"; }
LINE: while (defined($_ = <ARGV>)) {
    chomp $_;
    if (/module (\w+)\((.+)\)/s) {
        @ports = split(/\s*,\s*/, $2, 0);
        print "Module name: $1 Ports: " . join(', ', @ports);
    }
}

If you don't have empty lines separating the modules, you will need to get the entire file at once (slurp)

perl -l -0777 -ne 'while (/module (\w+)\((.+?)\);/sg) { @ports = split(/\s*,\s*/,$2); print "Module name: $1 Ports: " . join(", ", @ports)}' <<'EOF'
> module module_name1(in1, in2,
>     in3, in4, in5,
>     out1, out2, out3,
>     out4, out5);
> module module_name2(in21, in22,
>     in23, in24, in25,
>     out21, out22, out23,
>     out24, out25);
> EOF
Module name: module_name1 Ports: in1, in2, in3, in4, in5, out1, out2, out3, out4, out5
Module name: module_name2 Ports: in21, in22, in23, in24, in25, out21, out22, out23, out24, out25

Again, you can use -MO=Deparse to see what is happening:

perl -MO=Deparse -l -0777 -ne 'while (/module (\w+)\((.+?)\);/sg) { @ports = split(/\s*,\s*/,$2); print "Module name: $1 Ports: " . join(", ", @ports)}'
BEGIN { $/ = undef; $\ = "\n"; }
LINE: while (defined($_ = <ARGV>)) {
    chomp $_;
    while (/module (\w+)\((.+?)\);/gs) {
        @ports = split(/\s*,\s*/, $2, 0);
        print "Module name: $1 Ports: " . join(', ', @ports);
    }
}

The key element in these approaches is the -0 flag which in the -00 form sets $/ to the empty string enabling paragraph mode, and in -0777 form sets $/ to undef enabling slurp mode (reading the entire file) (see also: $RS in the perlvar manual.)

A important caveat: -l sets the $\ variable to $/ (which by default is "\n"), and in this case it has to used before -0 in the command line if you want the output to be separated by new lines.

For a more elegant approach, you can use the following script:

#!/bin/perl

use warnings;
use strict;

use File::Slurp;
use Data::Dumper;

my $data = read_file($ARGV[0]);

my %modules = $data =~ /module (\w+)\((.+?)\);/sg;

$modules{$_} = [split(/\s*,\s*/, $modules{$_})] for keys(%modules);

print Dumper(\%modules);

This would give you a data structure containing all the information needed - see https://ideone.com/BuuR8I for a live demo

Answer 3

See following code snippet for one of many possible solutions

NOTE: OP is missing in posted data block , after out3

#!/usr/bin/perl 
#
# vim: ai:ts=4:sw=4
#

use strict;
use warnings;
use feature 'say';

use Data::Dumper;

my $debug = 0;          # debug flag

my $data = do { local $/; <DATA> };

$data =~ s/[ \n]+/ /g;

my @lines = split ';', $data;

say Dumper(\@lines) if $debug;

for (@lines) {
    next unless /module\s+(.*)?\((.*)\)/;
    say "Module: $1 -- Ports: $2";
}


__DATA__
module module_name1(in1, in2,
    in3, in4, in5,
    out1, out2, out3,
    out4, out5);


module module_name2(in21, in22,
    in23, in24, in25,
    out21, out22, out23,
    out24, out25);

Output

Module: module_name1 -- Ports: in1, in2, in3, in4, in5, out1, out2, out3, out4, out5
Module: module_name2 -- Ports: in21, in22, in23, in24, in25, out21, out22, out23, out24, out25