CODEWITHSUNDEEP
pythondjangoamazon-s3boto
markwalker_
markwalker_

Reputation: 12859

How can I store URLs from Amazon S3 bucket in Django sqlite db to display & comment on?

I've been using Django for a couple of days & setup a basic blog from a tutorial with django comments.

I've got a totally separate python script that generates screenshots and uploads them to Amazon S3, now I'd like my django app to display all the images in the bucket and use a comment system on the images. Preferably I'd do this by just storing the URLs in my sqlite db, which I've got hard-coded currently to display all images in the db and has comments enabled on these.

My model:

(Does this need a foreign key to the django comments or is that just part of the Django Magic?!)

class Image(models.Model):
imgUrl=models.CharField(max_length=200)
meta=models.CharField(max_length=300)
def __unicode__(self):
    return self.imgUrl

My bucket structure: https://s3-eu-west-1.amazonaws.com/bucket/revision/process/images.png

Almost all the tutorials and packages I'm finding are based on upload/download rather than a simple for keys in bucket type approach that I want.

One of my problems is understanding how I can integrate my Boto functions with Django if I'm using Base.html. In an earlier tutorial I had an index page which had a view and could call functions from there. But base doesn't need that so I'm starting to get a little lost.

Upvotes: 1

Views: 2162

Answers (2)

Haimei
Haimei

Reputation: 13005

This is my code to store result from s3 to mysql by boto, django.

from demo.models import Movies
import boto
from boto.s3.key import Key
import string
from django.db import connection, transaction

def movietitle(b):
    key = b.get_key('netflix/movie_titles.txt')
    content = key.get_contents_as_string()
    line = content.split('\n')
    args = []
    for imovie in line:
        if len(imovie) > 0:
            imovie = imovie.split(',')
            movieid = imovie[0]
            year = imovie[1]
            title = imovie[2]
            iargs = [string.atoi(movieid),title,year]
            args.append(iargs)
    cursor = connection.cursor()
    sql = "insert into demo_movies(MovieID,MovieName,ReleaseYear) values(%s,%s,%s)"
    cursor.executemany(sql,args)
    transaction.commit_unless_managed()
    cursor.close()

Upvotes: 0

snies
snies

Reputation: 3521

haven't looked up if boto api changed, but this is how it worked last time i looked

from boto.s3.connection import S3Connection
from boto.s3.key import Key
import s3config

conn    = S3Connection(s3config.passwd, s3config.secret)
bucket  = conn.get_bucket(s3config.bucket)
s3_path = '/some/path/in/your/bucket'
keys    = bucket.list(s3_path)
# or if you want all keys:
# keys   = bucket.get_all_keys()

for key in keys:
  print key
  # here you can download or do other stuff
  # with the keys like get some metadata
  print key.name
  print key.etag
  print key.size
  print key.last_modified

#s3config.py
passwd = 'BLABALBALABALA'
secret = 'xvdwv3efefefefefef'
bucket = 'name-of-your-bucket'

Update:

Amazon s3 is a key value store, where key is a string. So nothing prevents you from putting in keys like:

/this/string/key/looks/like/a/unix/path
/folder/images/fileA.jpg
/folder/images/fileB.jpg
/folder/images/folderX/fileX1.jpg

now bucket.list(prefix="/folder/images/") would yield the latter three. Look here for further details:

Upvotes: 2

Related Questions