CODEWITHSUNDEEP
pythonjsonsubprocess
Michael
Michael

Reputation: 353

Running a terminal command for every line in a JSON file

I have a json file with tweets mined from Tweepy. To get a sentiment analysis for every tweet, I am attempting to use the API from www.text-processing.com. You can view my present (flawed) code below.

fname = 'python.json'
with open(fname, 'r') as f:
    for line in f:
        tweet = json.loads(line)
        # Draw out the text from each tweet
        tweet_words = tweet['text']
        bash_com = 'curl -d "text={}".format(tweet_words) http://text-processing.com/api/sentiment/'
        subprocess.Popen(bash_com)
        output = subprocess.check_output(['bash','-c', bash_com])
        with open('sentiment.json', 'w') as s:
            s.write(output)

It returns the following error:

CalledProcessError: Command '['bash', '-c', 'curl -d "text={}".format(tweet_words) http://text-processing.com/api/sentiment/']' returned non-zero exit status 1

I surmise this is mainly because I am using the format() function for a command that is suppose to go through the terminal (via the subprocess module). The terminal command I want to run for every tweet in my json file is:

curl -d "text=word" http://text-processing.com/api/sentiment/

Does anyone know of a good solution? Thanks!

Upvotes: 0

Views: 699

Answers (1)

Francisco
Francisco

Reputation: 11496

bash_com is equal to 'curl -d "text={}".format(tweet_words) http://text-processing.com/api/sentiment/'

You're not calling format because it's inside of the string.

You shouldn't do that anyway, since it's insecure, you can use subprocess.check_output directly:

output = subprocess.check_output(['curl', '-d', "text=" + tweet_words, 'http://text-processing.com/api/sentiment/'])

But you should be using an HTTP library such as requests instead:

requests.post('http://text-processing.com/api/sentiment/', {'text': tweet_words})

Upvotes: 1

Related Questions