CODEWITHSUNDEEP
nlpstanford-nlp
Richard Bender
Richard Bender

Reputation: 1134

Extracting relationship from NER parse

I'm working on a problem that at the very least seems to require named entity recognition, but I'm not sure how to go farther than the NER parse. What I'm trying to do is parse information (likely from tweets) regarding scheduling of events. So, for example, I'd like to be able to automatically resolve the yes/no answer to the question of "Are The Beatles playing tomorrow?" from short messages like:

"The Beatles cancelled their show tomorrow" or "The Beatles' show is still on tomorrow"

I know NER will get me close as it will identify the band of interest and the time (if it's indicated), but there are many ways to express the concepts I'm interested in, for example:

"The Beatles are on for tomorrow" or "The Beatles won't be playing tomorrow."

How can I go from an NER parsed representation to extracting the information of interest? Any suggestions would be much appreciated.

Upvotes: 0

Views: 79

Answers (1)

Nikita Astrakhantsev
Nikita Astrakhantsev

Reputation: 4749

I guess you should search by event detection (optionally - in Twitter); maybe, also by question answering systems, if your example with yes/no questions wasn't just an illustration: if you know user needs in advance, this information may increase the quality of the system.

For start, there are some papers about event detection in Twitter: here and here.

As a baseline, you can create a list with positive verbs for your domain (to be, to schedule) and negative verbs (to cancel, to delay) - just start from manual list and expand it by synonyms from some dictionary, e.g. WordNet. Also check for negations - again, by presence of pre-specified words ('not' in different forms) in a tweet. Then, if there is a negation, you just invert the meaning.

Since you work with Twitter and most likely there would be just one event mentioned in a tweet, it can work pretty well.

Upvotes: 1

Related Questions