CODEWITHSUNDEEP
pythoncsvanalysis
Chris Chalmers
Chris Chalmers

Reputation: 87

TweetSentiments.com API Python script

I don’t have much experience with python but I’d like a way to store a response from TweetSentiments.com API.

For more info visit - http://intridea.com/blog/2010/11/29/sentiment-analysis-using-tweetsentimentscom-api

I have a CSV file full of different tweets and would like to be able to read from the file to the API (csvreader.fieldnames??)with a query like ‘http://data.tweetsentiments.com:8080/api/analyze.json?q=’.

It seems the only response possible is JSON but I’d prefer to return the results to another CSV file with fields like ‘tweet’ and ‘rating’.

So I am interested in creating a python script that –

Reads CSV tweet file > construct query > Interrogates API > format JSON response > writes to CSV file.

I just need a foundation as I’m struggling to find some example code.

Tried using the cURL command from the command line ‘curl ”http://data.tweetsentiments.com:8080/api/analyze.json?q=”’

But got a ‘400 – Bad Request’ back.

I figured if I got a response back I could use this as a starting point.

Extra info – being implemented on WinXp.

Thanks in advance for the advice!

<---Update!--->

import csv
import urllib
import simplejson as json

Tweets=[] ## Creates empty list to store tweets.

TweetWriter = csv.writer(open('test.csv', 'w'), dialect='excel', delimiter=' ',quotechar='|')
TweetReader = csv.reader(open("C:\StoredTweets.csv", "r"))

for row in TweetReader:

    #TweetList.append(rows)
    Tweets.append({ 'tweet': row[0], 'date': row[1] }) ## Stores from CSV in list.

for rows in Tweets:

    #print TweetList
    data = urllib.urlencode({'Tweet': row[0], 'Date': row[1]}) ##Takes Tweet and date to construct query.
    #print data
    API_request = urllib.urlopen("http://data.tweetsentiments.com:8080/api/analyze.json?q=", data) ## Adds query to end of URL and queries API
    print API_request
    result = json.load(urllib.request({'API_request'}))
    print result
    TweetWriter.write(result) ## Writes API Response to CSV file.

It’s the json.load part I can’t seem to get the hang of. I’ve looked at loads of different examples and can’t seem to understand it. The API_request should hold my json structure I think? Should the json reponse be stored in another list? I keep getting told “AttributeError: ‘module’ object had no attribute ‘request’. Thanks for taken the time to go over this with me! 

C:\>python TweetSentiment.py
<!DOCTYPE html>
<html>
<head>
  <style type="text/css">
  body { text-align:center;font-family:helvetica,arial;font-size:22px;
    color:#888;margin:20px}
  #c {margin:0 auto;width:500px;text-align:left}
  </style>
</head>
<body>
  <h2>Sinatra doesn't know this ditty.</h2>
  <img src='/__sinatra__/404.png'>
  <div id="c">
    Try this:
    <pre>post '/api/analyze' do
  "Hello World"
end</pre>
  </div>
</body>
</html>

Upvotes: 1

Views: 389

Answers (1)

Katriel
Katriel

Reputation: 123762

This is the sort of thing Python is great for -- good choice! There are a few different things you need to do, in order.

  • Read the CSV file.

    You will want to use the csv module; specifically, csv.Reader or csv.DictReader (depending on whether you want tuples or dictionaries of each row.

  • Construct the query.

    Have a look at urllib.urlencode (in Python2 -- the name has changed slightly in Python 3, to urllib.parse.quote).

  • Interrogate API.

    urllib.request. Plenty of documentation for this! You may need to do some HTTPS stuff, include an API key, that sort of thing.

  • Format JSON response.

    json.load will return you a dictionary from the urllib.Request object.

  • Write CSV.

    csv.writer.

Is there any particular step in here you'd like help with?

Here's a very quick idea of what the code might end up looking like.

with open(...) as inputs, open(..., "w") as outputs:
    inputs = csv.reader(inputs)
    outputs = csv.writer(outputs)

    for line in inputs:
        query = urllib.urlencode(...)
        result = json.load(urllib.request(...))
        outputs.write(result["spam"], result["ham"])

Upvotes: 3

Related Questions