Python subprocess call precompiled java

Question

This works from the windows command line:

c:\mallet\bin\mallet run

I've tried

subprocess.call(['c:\mallet\bin\mallet', 'run'])

and get an error

WindowsError: [Error 2] The system cannot find the file specified

and I've tried

subprocess.call(['c:/mallet/bin/mallet', 'run'])

and get the error

WindowsError: [Error 193] %1 is not a valid Win32 application

What do I have to pass to subprocess.call()?

For completeness sake the complete command I would like to pass is:

bin\mallet run cc.mallet.topics.tui.DMRLoader texts.txt features.txt instance.mallet

My vague idea is that this is a precompiled java class that I'm calling somehow, but I don't really understand what I'm doing here.

Here are the two mallet-files in the folder bin:

mallet.bat

@echo off

rem This batch file serves as a wrapper for several
rem  MALLET command line tools.

if not "%MALLET_HOME%" == "" goto gotMalletHome

echo MALLET requires an environment variable MALLET_HOME.
goto :eof

:gotMalletHome

set MALLET_CLASSPATH=%MALLET_HOME%\class;%MALLET_HOME%\lib\mallet-deps.jar
set MALLET_MEMORY=1G
set MALLET_ENCODING=UTF-8

set CMD=%1
shift

set CLASS=
if "%CMD%"=="import-dir" set CLASS=cc.mallet.classify.tui.Text2Vectors
if "%CMD%"=="import-file" set CLASS=cc.mallet.classify.tui.Csv2Vectors
if "%CMD%"=="import-smvlight" set CLASS=cc.mallet.classify.tui.SvmLight2Vectors
if "%CMD%"=="train-classifier" set CLASS=cc.mallet.classify.tui.Vectors2Classify
if "%CMD%"=="train-topics" set CLASS=cc.mallet.topics.tui.Vectors2Topics
if "%CMD%"=="infer-topics" set CLASS=cc.mallet.topics.tui.InferTopics
if "%CMD%"=="estimate-topics" set CLASS=cc.mallet.topics.tui.EstimateTopics
if "%CMD%"=="hlda" set CLASS=cc.mallet.topics.tui.HierarchicalLDATUI
if "%CMD%"=="prune" set CLASS=cc.mallet.classify.tui.Vectors2Vectors
if "%CMD%"=="split" set CLASS=cc.mallet.classify.tui.Vectors2Vectors
if "%CMD%"=="bulk-load" set CLASS=cc.mallet.util.BulkLoader
if "%CMD%"=="run" set CLASS=%1 & shift

if not "%CLASS%" == "" goto gotClass

echo Mallet 2.0 commands: 
echo   import-dir        load the contents of a directory into mallet instances (one per file)
echo   import-file       load a single file into mallet instances (one per line)
echo   import-svmlight   load a single SVMLight format data file into mallet instances (one per line)
echo   train-classifier  train a classifier from Mallet data files
echo   train-topics      train a topic model from Mallet data files
echo   infer-topics      use a trained topic model to infer topics for new documents
echo   estimate-topics   estimate the probability of new documents given a trained model
echo   hlda              train a topic model using Hierarchical LDA
echo   prune             remove features based on frequency or information gain
echo   split             divide data into testing, training, and validation portions
echo Include --help with any option for more information


goto :eof

:gotClass

set MALLET_ARGS=

:getArg

if "%1"=="" goto run
set MALLET_ARGS=%MALLET_ARGS% %1
shift
goto getArg

:run

java -Xmx%MALLET_MEMORY% -ea -Dfile.encoding=%MALLET_ENCODING% -classpath %MALLET_CLASSPATH% %CLASS% %MALLET_ARGS%

:eof

and mallet

#!/bin/bash


malletdir=`dirname $0`
malletdir=`dirname $malletdir`

cp=$malletdir/class:$malletdir/lib/mallet-deps.jar:$CLASSPATH
#echo $cp

MEMORY=1g

JAVA_COMMAND="java -Xmx$MEMORY -ea -Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -classpath $cp"

CMD=$1
shift

help()
{
cat <<EOF
Mallet 2.0 commands: 

  import-dir         load the contents of a directory into mallet instances (one per file)
  import-file        load a single file into mallet instances (one per line)
  import-svmlight    load SVMLight format data files into Mallet instances
  train-classifier   train a classifier from Mallet data files
  classify-dir       classify data from a single file with a saved classifier
  classify-file      classify the contents of a directory with a saved classifier
  classify-svmlight  classify data from a single file in SVMLight format
  train-topics       train a topic model from Mallet data files
  infer-topics       use a trained topic model to infer topics for new documents
  evaluate-topics    estimate the probability of new documents under a trained model
  hlda               train a topic model using Hierarchical LDA
  prune              remove features based on frequency or information gain
  split              divide data into testing, training, and validation portions

Include --help with any option for more information
EOF
}

CLASS=

case $CMD in
    import-dir) CLASS=cc.mallet.classify.tui.Text2Vectors;;
    import-file) CLASS=cc.mallet.classify.tui.Csv2Vectors;;
        import-svmlight) CLASS=cc.mallet.classify.tui.SvmLight2Vectors;;
    train-classifier) CLASS=cc.mallet.classify.tui.Vectors2Classify;;
        classify-dir) CLASS=cc.mallet.classify.tui.Text2Classify;;
        classify-file) CLASS=cc.mallet.classify.tui.Csv2Classify;;
        classify-svmlight) CLASS=cc.mallet.classify.tui.SvmLight2Classify;;
    train-topics) CLASS=cc.mallet.topics.tui.Vectors2Topics;;
    infer-topics) CLASS=cc.mallet.topics.tui.InferTopics;;
    evaluate-topics) CLASS=cc.mallet.topics.tui.EvaluateTopics;;
    hlda) CLASS=cc.mallet.topics.tui.HierarchicalLDATUI;;
    prune) CLASS=cc.mallet.classify.tui.Vectors2Vectors;;
    split) CLASS=cc.mallet.classify.tui.Vectors2Vectors;;
    bulk-load) CLASS=cc.mallet.util.BulkLoader;;
    run) CLASS=$1; shift;;
    *) echo "Unrecognized command: $CMD"; help; exit 1;;
esac

$JAVA_COMMAND $CLASS $*

Answer 1

When you call a program without extension, the Windows shell will try several standard extensions (.BAT, .EXE, ...) in order to guess the file you are trying to call.

If you want to execute your program without a shell to perform that look-up phase, you need to pass the full name of the batch you are trying to execute -- incl. the .BAT extension:

subprocess.call(['c:/mallet/bin/mallet.bat', 'run'])

Answer 2

Make sure that you pass the shell = True argument to subprocess.call(). However, it poses security issues, so make sure you look at the documentation and understand how it works.

subprocess.call(['c:/mallet/bin/mallet', 'run'], shell = True)

Also, when using strings to identify paths that contain backslashes, make it a raw string (r"This is a raw string!"), so that it doesn't implement anything else (such as newline tokens).

If my above suggestions don't work, there are only two things that I can thing of:

1. The file you are trying to execute may not be an application file (.exe file). I don't work with Windows so I'm not sure about this one, but it could likely be a possibility.
2. Something in the file(s) is broken or something like that.

subprocess.call() docs

subprocess.call() Security Issues

Answer 3

Well, maybe the problem is with the backslashes.

From their docs:

The backslash (\) character is used to escape characters that otherwise have a special meaning, such as newline, backslash itself, or the quote character.

So you probably should do:

subprocess.call(['c:\\mallet\\bin\\mallet', 'run'])