CODEWITHSUNDEEP
pythonnlpnltkdependency-parsing
James Ko
James Ko

Reputation: 34489

How to detect statements made about entities using NLP

I want to research using NLP to detect negative/non-constructive comments, i.e. those that frequently arise when discussing politics online. I am curious to know that if given a sentence like this:

You're a liberal dweeb. Clinton is ruining the US with her inappropriate behavior as president.

Whether it's possible to not only deduce the entities (you, Clinton) using NER but also get a tree of the statements made about each entity:

+-----------------+                             +------------------------+
|                 |                             |                        |
|                 |                             |                        |
|       you       |                             |          Clinton       |
|                 +------+                      |                        +------+
|                 |      |                      |                        |      |
+--+--------------+      |                      |                        |      |
   |                     |                      +--+---------------------+      |
   |                     |                         |                            |
   |                     |                         |                            |
 +-+-------+        +----+-----+                   |                  +---------+----------+
 |         |        |          |              +----+---------+        |                    |
 |         |        |   dweeb  |              |              |        |                    |
 |  liberal|        |          |              |  ruining US  |        | has inappropriate  |
 |         |        +----------+              |              |        | behavior as pres.  |
 +---------+                                  |              |        |                    |
                                              +--------------+        +--------------------+

Is something like this possible with NLP?

Upvotes: 0

Views: 394

Answers (2)

Adnan S
Adnan S

Reputation: 1882

Yes, what you are looking for is certainly possible with NLP. There are two approaches you should research further.

1) The faster approach in terms of coding but requiring investment in labelling and training annotated data is to use the "relation extractor" feature of an NLP framework like Stanford NLP, Spacy, etc. However, you will have to do some customisation and training of the default models. Here's a link to a sample blog article about doing this with NLTK but you should look for a more recent article if you pursue this route.

2) The slower approach in terms of coding but not requiring data labelling and annotation is mentioned by @Gabriel in running a dependency parser and entity NER pipeline on your sentences and then using a set of manual rules in your code for extraction of the relations.

Upvotes: 0

Gabriel
Gabriel

Reputation: 587

A constituency parser or dependency parser, possibly plus some kind of semantic analysis to give you more info about named and non-named entities, may be what you're looking for. Try pasting some example sentences into http://corenlp.run/ or into http://demo.ark.cs.cmu.edu/parse, which applies a dependency parse and a semantic parse, to see if it's the type of thing you're looking for.

Upvotes: 1

Related Questions