NLTK -> Using Stanford Dependency Parser ->

Question

There seems to be a lot of contradicting documentation for NLTK (where is the definitive source for NLTK/StanfordNLP documentation?).

My question: what is the preferred method to call the StanfordParser from nltk? This is my code, but something is incorrect in the java call.

from nltk.parse.stanford import StanfordDependencyParser
import os

parser_home = '/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/'

# os.environ['CLASSPATH'] = parser_home

parser = StanfordDependencyParser(
    model_path = parser_home + 'stanford-parser.jar',
    path_to_models_jar = parser_home +  'stanford-parser-3.9.1-models.jar',
    verbose = True
)

result = parser.raw_parse('Here is an example sentence.')

Here's my error. Any help appreciated. I haven't found an exact match to mine. I'm setting the classpath, but I'm not sure that's required.

[Found stanford-parser\.jar: /Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser.jar]
[Found stanford-parser-(\d+)(\.(\d+))+-models\.jar: /Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser-3.9.1-models.jar]
/Users/myname/anaconda3/envs/nlp/lib/python3.6/site-packages/ipykernel_launcher.py:12: DeprecationWarning: The StanfordDependencyParser will be deprecated
Please use nltk.parse.corenlp.StanforCoreNLPDependencyParser instead.
  if sys.path[0] == '':

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.RuntimeException: /Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser.jar: expecting BEGIN block; got PK��aL    META-INF/��PKPK��aLMETA-INF/MANIFEST.MFE��
    at edu.stanford.nlp.parser.lexparser.LexicalizedParser.confirmBeginBlock(LexicalizedParser.java:536)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParser.getParserFromTextFile(LexicalizedParser.java:546)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParser.getParserFromFile(LexicalizedParser.java:406)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParser.loadModel(LexicalizedParser.java:186)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParser.main(LexicalizedParser.java:1400)

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-18-052e46a6f6aa> in <module>()
----> 1 result = parser.raw_parse('Here is an example sentence.')

~/anaconda3/envs/nlp/lib/python3.6/site-packages/nltk/parse/stanford.py in raw_parse(self, sentence, verbose)
    132         :rtype: iter(Tree)
    133         """
--> 134         return next(self.raw_parse_sents([sentence], verbose))
    135 
    136     def raw_parse_sents(self, sentences, verbose=False):

~/anaconda3/envs/nlp/lib/python3.6/site-packages/nltk/parse/stanford.py in raw_parse_sents(self, sentences, verbose)
    150             '-outputFormat', self._OUTPUT_FORMAT,
    151         ]
--> 152         return self._parse_trees_output(self._execute(cmd, '\n'.join(sentences), verbose))
    153 
    154     def tagged_parse(self, sentence, verbose=False):

~/anaconda3/envs/nlp/lib/python3.6/site-packages/nltk/parse/stanford.py in _execute(self, cmd, input_, verbose)
    216                 cmd.append(input_file.name)
    217                 stdout, stderr = java(cmd, classpath=self._classpath,
--> 218                                       stdout=PIPE, stderr=PIPE)
    219 
    220             stdout = stdout.replace(b'\xc2\xa0', b' ')

~/anaconda3/envs/nlp/lib/python3.6/site-packages/nltk/__init__.py in java(cmd, classpath, stdin, stdout, stderr, blocking)
    134     if p.returncode != 0:
    135         print(_decode_stdoutdata(stderr))
--> 136         raise OSError('Java command failed : ' + str(cmd))
    137 
    138     return (stdout, stderr)

OSError: Java command failed : ['/usr/bin/java', '-mx1000m', '-cp', '/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser-3.9.1-models.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser-3.9.1-javadoc.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/ejml-0.23.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser-3.9.1-sources.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/slf4j-api.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser-3.9.1-models.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser.jar:/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/slf4j-api-1.7.12-sources.jar', 'edu.stanford.nlp.parser.lexparser.LexicalizedParser', '-model', '/Users/myname/Documents/nlp/stanford-parser-full-2018-02-27/stanford-parser.jar', '-sentences', 'newline', '-outputFormat', 'conll2007', '-encoding', 'utf8', '/var/folders/kg/y1g8nszj77z0pm6mzplqv7580000gp/T/tmp93uyyya_']

Answer 1

After digging around, it seems that the StanfordDependencyParser class has been deprecated in NLTK:

• Discussion on GitHub of people having similar issues
• Proposal of more elegant interface

The new, improved way:

First, download the full CoreNLP files from here, then start a CoreNLP server (I chose port 9010) in the downloaded folder by running the below command. The folder looks like the stanford-parser-full-2018-02-27 directory, for you:

$ java -mx1g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9010 -timeout 15000

Then, run this code:

from nltk.parse.corenlp import CoreNLPParser
parser = CoreNLPParser(url='http://localhost:{somePort}'
next(
     parser.raw_parse('The quick brown fox sucks at jumping.')
     ).pretty_print()
                ROOT
                 |
                 S
       __________|__________________________
      |                     VP              |
      |                 ____|___            |
      |                |        PP          |
      |                |     ___|_____      |
      |                |    |         S     |
      |                |    |         |     |
      NP               |    |         VP    |
  ____|__________      |    |         |     |
 DT   JJ    JJ   NN   VBZ   IN       VBG    .
 |    |     |    |     |    |         |     |
The quick brown fox  sucks  at     jumping  .

Also, fun fact, once the server is running, you can navigate to http://localhost:9010 (or whatever port you've chosen) and view a nice little interface to tinker around with.