Matching and Extracting data using regular expression

Question

Problem: To find a matching string and to extract data from the matched string. There are a number of command strings which has keywords and data.

Command Examples:

1. Ask name to call me
2. Notify name that do this action
3. Message name that request

Keywords: Ask, Notify, Message, to, that. Data:

Input strings:

1. Ask peter to call me
2. Notify Jenna that I am going to be away
3. Message home that I am running late

My problem consists of two problems 1) Find matching command 2) Extract data

Here is what I am doing: I create multiple regular expressions: "Ask[[:s:]][[:w:]]+[[:s:]]to[[:s:]][[:w:]]+" or "Ask([^\t\n]+?)to([^\t\n]+?)" "Notify[[:s:]][[:w:]]+[[:s:]]that[[:s:]][[:w:]]+" or "Notify([^\t\n]+?)that([^\t\n]+?)"

void searchExpression(const char *regString)
{
    std::string str;
    boost::regex callRegEx(regString, boost::regex_constants::icase);
    boost::cmatch im;

    while(true) {
       std::cout << "Enter String: ";
       getline(std::cin, str);
       fprintf(stderr, "str %s regstring %s\n", str.c_str(), regString);

       if(boost::regex_search(str.c_str(), im, callRegEx)) {
             int num_var = im.size() + 1;
             fprintf(stderr, "Matched num_var %d\n", num_var);
             for(int j = 0; j <= num_var; j++) {
                    fprintf(stderr, "%d) Found %s\n",j, std::string(im[j]).c_str());
             }
      }
      else {
          fprintf(stderr, "Not Matched\n");
      }
   }
}

I am able to Find a matching string, I am not able to extract the data. Here is the output:

input_string: Ask peter to call Regex Ask[[:s:]][[:w:]]+[[:s:]]to[[:s:]][[:w:]]+
Matched num_var 2
0) Found Ask peter to call
1) Found
2) Found

I would like to extract peter and call from Ask Peter to call.

Answer 1

This code reads the command strings from the file "commands.txt", searches for the regular expressions and prints the parts whenever there is a match.

#include <iostream>
#include <fstream> 
#include <string>
#include <boost/regex.hpp>

const int NumCmdParts = 4;
std::string CommandPartIds[] = {"Verb", "Name", "Preposition", "Content"};

int main(int argc, char *argv[])
{

    std::ifstream ifs;
    ifs.open ("commands.txt", std::ifstream::in);
    if (!ifs.is_open()) {
      std::cout << "Error opening file commands.txt" << std::endl;
      exit(1);
    }

    std::string cmdStr;

    // Pieces of regular expression pattern
    // '(?<Verb>' : This is to name the capture group as 'Verb'
    std::string VerbPat = "(?<Verb>(Ask)|(Notify|Message))";
    std::string SeparatorPat = "\\s*";  
    std::string NamePat = "(?<Name>\\w+)";

    // Conditional expression. if (Ask) (to) else (that)
    std::string PrepositionPat = "(?<Preposition>(?(2)(to)|(that)))";
    std::string ContentPat = "(?<Content>.*)";

    // Put the pieces together to compose pattern
    std::string TotalPat = VerbPat + SeparatorPat + NamePat + SeparatorPat
                            + PrepositionPat + SeparatorPat + ContentPat;

    boost::regex actions_re(TotalPat);
    boost::smatch action_match;

    while (getline(ifs, cmdStr)) {
        bool IsMatch = boost::regex_search(cmdStr, action_match, actions_re);
        if (IsMatch) {          
          for (int i=1; i <= NumCmdParts; i++) {     
            std::cout << CommandPartIds[i-1] << ": " << action_match[CommandPartIds[i-1]] << "\n";
          }
        }
    }   

    ifs.close();
}

Answer 2

Since you're really wanting to parse a grammar, you should consider Boost's parser generator.

You'd simply write the whole thing top-down:

auto sentence  = [](auto&& v, auto&& p) { 
    auto verb     = lexeme [ no_case [  as_parser(v) ] ];
    auto name     = lexeme [ +graph ];
    auto particle = lexeme [ no_case [  as_parser(p) ] ];
    return confix(verb, particle) [ name ]; 
};

auto ask     = sentence("ask",     "to")   >> lexeme[+char_];
auto notify  = sentence("notify",  "that") >> lexeme[+char_];
auto message = sentence("message", "that") >> lexeme[+char_];

auto command = ask | notify | message;

This is a Spirit X3 grammar for it. Read lexeme as "keep whole word" (don't ignore spaces).

Here, "name" is taken to be anything up to the expected particle¹

If you just want to return the raw string matched, this is enough:

Live On Coliru

#include <iostream>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/directive/confix.hpp>

namespace x3 = boost::spirit::x3;

namespace commands {
    namespace grammar {
        using namespace x3;

        auto sentence  = [](auto&& v, auto&& p) { 
            auto verb     = lexeme [ no_case [  as_parser(v) ] ];
            auto name     = lexeme [ +graph ];
            auto particle = lexeme [ no_case [  as_parser(p) ] ];
            return confix(verb, particle) [ name ]; 
        };

        auto ask     = sentence("ask",     "to")   >> lexeme[+char_];
        auto notify  = sentence("notify",  "that") >> lexeme[+char_];
        auto message = sentence("message", "that") >> lexeme[+char_];

        auto command = ask | notify | message;

        auto parser  = raw [ skip(space) [ command ] ];
    }
}

int main() {
    for (std::string const input : {
            "Ask peter to call me",
            "Notify Jenna that I am going to be away",
            "Message home that I am running late",
            })
    {
        std::string matched;

        if (parse(input.begin(), input.end(), commands::grammar::parser, matched))
            std::cout << "Matched: '" << matched << "'\n";
        else
            std::cout << "No match in '" << input << "'\n";
    }

}

Prints:

Matched: 'Ask peter to call me'
Matched: 'Notify Jenna that I am going to be away'
Matched: 'Message home that I am running late'

BONUS

Of course, you'd actually want to extract the relevant bits of information.

Here's how I'd do that. Let's parse into a struct:

struct Command {
    enum class Type { ask, message, notify } type;
    std::string name;
    std::string message;
};

And let's write our main() as:

commands::Command cmd;

if (parse(input.begin(), input.end(), commands::grammar::parser, cmd))
    std::cout << "Matched: " << cmd.type << "|" << cmd.name << "|" << cmd.message << "\n";
else
    std::cout << "No match in '" << input << "'\n";

Live On Coliru

#include <iostream>
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/directive/confix.hpp>

namespace x3 = boost::spirit::x3;

namespace commands {

    struct Command {
        enum class Type { ask, message, notify } type;
        std::string name;
        std::string message;

        friend std::ostream& operator<<(std::ostream& os, Type t) { return os << static_cast<int>(t); } // TODO
    };

}

BOOST_FUSION_ADAPT_STRUCT(commands::Command, type, name, message)

namespace commands {

    namespace grammar {
        using namespace x3;

        auto sentence  = [](auto type, auto&& v, auto&& p) { 
            auto verb     = lexeme [ no_case [  as_parser(v) ] ];
            auto name     = lexeme [ +graph ];
            auto particle = lexeme [ no_case [  as_parser(p) ] ];
            return attr(type) >> confix(verb, particle) [ name ]; 
        };

        using Type = Command::Type;
        auto ask     = sentence(Type::ask,     "ask",     "to")   >> lexeme[+char_];
        auto notify  = sentence(Type::notify,  "notify",  "that") >> lexeme[+char_];
        auto message = sentence(Type::message, "message", "that") >> lexeme[+char_];

        auto command // = rule<struct command, Command> { }
                     = ask | notify | message;

        auto parser  = skip(space) [ command ];
    }
}

int main() {
    for (std::string const input : {
            "Ask peter to call me",
            "Notify Jenna that I am going to be away",
            "Message home that I am running late",
            })
    {
        commands::Command cmd;

        if (parse(input.begin(), input.end(), commands::grammar::parser, cmd))
            std::cout << "Matched: " << cmd.type << "|" << cmd.name << "|" << cmd.message << "\n";
        else
            std::cout << "No match in '" << input << "'\n";
    }

}

Prints

Matched: 0|peter|call me
Matched: 2|Jenna|I am going to be away
Matched: 1|home|I am running late

---

¹ I'm no English linguist so I don't know whether that is the correct grammatical term :)