GAE renders 'none' but i can see the db entities in datastore viewer

Question

I'm having problem with accessing datastore entities. I'm new to GAE and Python, so be kind. I have been sourcing code from online tutorials and others example code and that's been going OK, until now.

Ultimately i want to be able to strip an email attachment, do some simple changes to the file(s) and merge with other .kml file(s) (or even better .kmz, but one thing at a time :) .

step1. was to send emails to the app's appspotmail address = done :)

step2. ...which writes to a db.model = i can see in dashboard/datastore viewer that the entities are there, so no problem = done :)

step3. - render a template to show the most recent,say 20 emails received. this list would dynamically update..... :( Hmmmmm

What happens instead is that i see none rendered for all variables in index.html. Can you perhaps point me in the right direction? thanks!

Dav-o

here is yaml.app

application: racetracer
version: 1
runtime: python27
api_version: 1
threadsafe: false

libraries:                                                                      
- name: jinja2                                                                  
  version: latest   

handlers:
- url: /_ah/mail/.+ 
  script: handle_incoming_email.py
  login: admin

- url: /favicon.ico
  static_files: images/favicon.ico
  upload: images/favicon.ico

- url: /static/css
  static_dir: static/css

- url: /static/html
  static_dir: static/index.html

- url: .*
  script: main.app

inbound_services:
- mail

here is handle_incoming_email.py

import logging, email
import cgi
import datetime
import os.path
import os

from os import path
from google.appengine.ext import webapp 
from google.appengine.ext import db
from google.appengine.ext.webapp.mail_handlers import InboundMailHandler 
from google.appengine.ext.webapp.util import run_wsgi_app 
from google.appengine.ext.webapp.template import render
import jinja2
import wsgiref.handlers

from main import *
from baseController import *
from authController import *

class Vault(db.Model):
    #fromAddress= db.UserProperty()
    fromAddress= db.StringProperty()
    subject = db.StringProperty(multiline=True)
    #date = db.DateTimeProperty(auto_now_add=True)
    date = db.StringProperty(multiline=True)
    name = db.StringProperty(multiline=True)

class ProcessEmail(InboundMailHandler): 
    def receive(self, mailMessage): 
        logging.info("Email from: " + mailMessage.sender + " received ok")
        logging.info("Email subject: " + mailMessage.subject )
        logging.info("Email date: " + mailMessage.date )

        fromAddress = mailMessage.sender
        subject = mailMessage.subject
        date = mailMessage.date

        if not hasattr(mailMessage, 'attachments'):
            logging.info("Oi, the email has no attachments")
            # raise ProcessingFailedError('Email had no attached documents')
        else:
            logging.info("Email has %i attachment(s) " % len (mailMessage.attachments))

            for attach in mailMessage.attachments:
                name = attach[0] #this is the file name
                contents = str(attach[1]) #and this is the attached files contents
                logging.info("Attachment name: " + name )
                logging.info("Attachment contents: " + contents)
                vault = Vault()
                vault.fromAddress = fromAddress
                vault.subject = subject
                vault.date = date
                vault.name = name
                vault.contents = contents #.decode() EncodedPayload.decode()
                    vault.put()      

                #f = open("aardvark.txt", "r")
                #blah = f.read(30);
                #logging.info("the examined string is : " + blah)
                #f.close() # Close opened file

app = webapp.WSGIApplication([
  ProcessEmail.mapping()
], debug=True)

def main():
    run_wsgi_app(app)

if __name__ == "__main__":
    main()

and here is basecontroller.py

from authController import *
from handle_incoming_email import *
import logging
import os
import webapp2
import jinja2
import wsgiref.handlers

from google.appengine.ext.webapp import template 
from google.appengine.ext import db
from google.appengine.api import users 

TEMPLATE_DIR = os.path.join(os.path.dirname(__file__))
jinja_environment = \
    jinja2.Environment(loader=jinja2.FileSystemLoader(TEMPLATE_DIR))

class Vault(db.Model):
    #fromAddress= db.UserProperty()
    fromAddress= db.StringProperty()
    subject = db.StringProperty(multiline=True)
    #date = db.DateTimeProperty(auto_now_add=True)
    date = db.StringProperty(multiline=True)
    name = db.StringProperty(multiline=True)

class MainPage(webapp2.RequestHandler):    
    def get(self):
        logging.info("this is MainPage in baseController")
        list = db.GqlQuery("SELECT * FROM Vault ORDER BY date DESC").fetch(10)
        logging.info("this is in list: " + str(list))

        vault = Vault()

        fromAddress = vault.fromAddress
        subject = vault.subject
        date = vault.date
        name = vault.name

        values = {
                'fromAddress': fromAddress,
            'subject': subject,
            'date': date,
            'name': name,
            }
        templ = jinja_environment.get_template('index.html')
        self.response.out.write(templ.render(values))                        

def main():
    wsgiref.handlers.CGIHandler().run(app)

if __name__ == "__main__":
  main()

i read in a tutorial that rendering templates is a superior to just using the html in the .py, but maybe i've broken it down in too many files? Their logic was that different people do the front end and backend, so get used to it now. Anyway the above intends to render to index.html which is:

index.html

{% extends "base.html" %}
{% block title %} racetracer {% endblock %}
{% block content %}
<h1>Racetracer </h1>
<h3>Files recently received :</h3>
<br>
        <table>
        <tr>
            <th>Email address:                  </th>
            <th>-----------                     </th>
            <th>Subject:                        </th>
            <th>Date Received:                  </th>
            <th>Attachment:                     </th>
        </tr>           
        <tr>                
            <td> {{ fromAddress }}              </td>
            <td> ---------------                </td>   
            <td> {{ subject }}                  </td>
            <td> {{ date }}                     </td>
            <td> {{ name }}                     </td>
        <blockquote>{{ content|escape }}</blockquote>                     
        </p>
        </tr>                               
    </tr>    
    </table>      
<br>

<hr>
<form action="/sign" method="post">
    <textarea name="content" rows="1" cols="20"></textarea>
    <input type="submit" value="Submit (in index:)">
</form>
{% endblock %}

and base.html....

<html><head>
<link type="text/css" rel="stylesheet" href="/static/css/main.css" /></head>

<body> From the file base out in the racetracer folder<br>
right now you are logged in as : 
    {% if user %}
        {{ user.nickname }}!
        [<a href="{{ logout }}"><b>sign out</b></a>]
    {% else %}
        anonymous, who are you?
        [<a href="{{ login }}"><b>sign in</b></a>]
    {% endif %}
    {% block content %}
    {% endblock %}
</body></html>

and here is main.py

import os
import webapp2
from handle_incoming_email import *
from baseController import *
from authController import *
from google.appengine.ext.webapp import template 

app = webapp2.WSGIApplication([
    ('/', MainPage),
    #('/ProcessEmail', ProcessEmail),
], debug=True)

def main():
    wsgiref.handlers.CGIHandler().run(app)

if __name__ == "__main__":
  main()

Thanks for your help! it will be MUCH appreciated.

Answer 1

You're getting None because it IS None. You're declaring the Vault object but not assigning any values to it. The values would be in the list entity as a result of the GQL query.

vault = Vault() # This creates an empty object (assuming you aren't doing any dynamic init

fromAddress = vault.fromAddress  # Still empty....
subject = vault.subject
date = vault.date
name = vault.name

Also, you shouldn't assign list as a var since that's reserved for Py.

What you want to do is set something like:

my_list = YOUR GQL QUERY.fetch()

# Create a formatted list
values = []
for ml in my_list:
    value = {
            'fromAddress': ml.fromAddress,
        'subject': ml.subject,
        'date': ml.date,
        'name': ml.name,
        }

    values.append(value) # You could just append the dictionary directly here, of course

Then use the values list in you template params

** Update **

Your GQL query looks good as compared to your datastore Vault model.

First, make sure you've changed the 'list' var to 'my_list'. Next, if you want to see the contents of the retrieved object printed as a readable string, add this to your model:

    def __unicode__(self):
       return ('%s %s %s' % (self.key,self.fromAddress,self.subject)) # etc... for the vars you want printed with you print the object

Check your logging.info(my_list) and see if it prints anything more readable.

If not, run a for loop on the list and log the key and/or fromAddress, so:

for vault in my_list:
   logging.info('the key = %s'%vault.key)

If that's not returning anything, go directly to the interactive console within your development environment and run that query:

from google.appengine.ext import db
from *vault_file* import Vault

my_list = db.GqlQuery("SELECT * FROM Vault ORDER BY date DESC").fetch(10) # run query

for vault in my_list:
   print vault.key
   print vault.fromAddress

Let me know where these tests leave you and I will continue to update.

* Update #2 *

So now that you can confirm your getting datastore values
you want to set a template page parameter equal to the vault list, so:

params = dict(values=my_list)
self.response.out.write(templ.render(params)) 

And now on your page, you need to loop through your list to print each dictionary item:

{% for vault in values %}
    <tr>
       <td>{{vault.fromAddress}}}</td>
       etc...
    </tr>

{% endfor %}

That work?