Google App Engine to Rescue: 2009

Sunday, November 8, 2009

Enriching our RunAroundUser model with utility methods

Very much like PHP runaround application I have added some static and some instance methods which provider useful information such as:

Get the logged in user (either native user or the one which has used Facebook)
Get the name and email of user (from facebook or the one stored in DataStore)
Register the Facebook user in our datastore
Register the user to Facebook (Still in construction ......)
Methods to login and logout the user

Reading and Writing Cookies

Since runaround application uses Cookies to store the username, in this post we will see how to read and write using webapp framework.

Reading cookies is quite simple:

if request.cookies.has_key(USER_COOKIE_NAME):
            username = request.cookies[USER_COOKIE_NAME]

Writing cookies:

I have created a small function that should help:

def writeCookie(response,username):
    cookie = Cookie.SimpleCookie()
    cookie[USER_COOKIE_NAME] = username
    cookie[USER_COOKIE_NAME]["Path"] = "/"
    
    h = re.compile('^Set-Cookie:').sub('',cookie.output(),count=1)
         
    response.headers.add_header('Set-Cookie',str(h))

Facebook APIs and Using PyFacebook

Facebook platform expose access to its data using RESTful interfaces. Your application can request profile, friend, Page, group, photo, and event data using these interfaces.

Here you would find all the Facebook APIs and their documentation along with some examples.

To get data from Facebook platform you are essentially making HTTP requests (GET or POST) with XML or JSON formatted data in requests and response. The APIs in the above mentioned link give enough information about the format of these messages.

PyFacebook is a pythonic library that takes care of making these HTTP requests and expose a nice object oriented interfaces for your application to use. One downside of PyFacebook is lack of documentation and even when you will look at the source code you may not get a good idea of how to use it (at least it was the case for me). The reason behind it is how this library has been written and demonstrates the power of dynamic languages and meta programming. It is really wonderful.

Here is an example of how from your application you would like to get the name of the user for whom you have his facebook uid.

pyfacebookObj.users.getInfo([fb_uid], ['name'])[0]

Here pyfacebookObj is an object of Facebook class exposed by PyFacebook. If you look inside the PyFacebook code you would not find a field "users" so from where this field is coming and as the above code snippet shows this field is essentially an object of class which has "getInfo" method.

So what PyFacebook is doing is that it is creating a class for each of the Facebook exposed data service with corresponding methods at runtime. These are called Proxy classes which are generating by an API specification. The interesting thing is that PyFacebook did not need to write lot of classes (and methods).

Magical.

Rendering Cross Domain Receiver page

class XdReceiverPage(webapp.RequestHandler):
    def get(self):
        template_values = {}
        path = os.path.join(os.path.dirname(__file__), 'templates/xd_receiver.html')
        self.response.out.write(template.render(path, template_values))

In the base.html template now we need to tell Facebook JavaScript where to find our cross domain receiver file.

This shows that you can place xd_receiver.html anywhere as long as you can tell Facebook JavaScript where to find it.

and here is how the xd_receiver.html file looks like:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<body>
<script src="http://static.ak.connect.facebook.com/js/api_lib/v0.4/XdCommReceiver.js" type="text/javascript"></script>
</body>
</html>

Facebook documentation also describes details of xd_receiver here

Integrating Facebook Connect

After looking at the user and authentication management in the PHP code I figured that both of them so intertwined that it would be difficult to develop them separately so I am going to start integrating Facebook Connect for user authentication.

Facebook wiki also gives the description of a login session here

Facebook Connect is a technology used to delegate the authentication of users to Facebook platform. In a way it competes with technologies like openID, SAML and CardSpace.

What I really liked in Facebook Connect is their use of cross-domain communication channel which enable it to ask the user for the credentials at the service provider (runaround in our case) website instead of redirecting to Facebook. I would also like to mention that the protocols that Facebook Connect use is proprietary unlike SAML and OpenID which are open protocols.

The library that I will using will be pyfacebook to communicate with Facebook platform.

First you need to sign up with Facebook developer which is described in great detail here.

It would be a good idea to also read Anatomy of a Facebook Connect Site but anyway in later posts I am going to explain the steps need to be taken in our Google App Engine webapp framework based runaround application.

Sunday, November 1, 2009

Authentication & Session Management in Runaround application

As per the design of Runaround application :

User can login using his/her facebook account or native account at runaround
For Facebook Connect authentication the session is managemed by Facebook and runaround make use of Facebook client APIs to get a handle on session
For native account authentication (in the sample app) there is no server session, only a cookie based session is used.

We all know that Cookie based session is not secure and the demo app duly mentions it here

Since we are in learning mode and trying to port the PHP application to Google App Engine it may be simple to have similar functionality (even though it is not secure). But later we must do the right thing to have server based sessions.

Session management in Google App Engine

It would have been really nice if session management was part of Google SDK. Here are some open source alternatives available:

Saturday, October 31, 2009

Uploaded the application to App Engine

The application is available at http://ks-exp.appspot.com/runaround/

Populated the code in Github

The GitHub repository is:

http://github.com/ksachdeva/Google-App-Engine-experiments

Handler for the Home Page

class MainPage(webapp.RequestHandler):
    def get(self):
        template_values = {}
        path = os.path.join(os.path.dirname(__file__), 'templates/index_loggedout.html')
        self.response.out.write(template.render(path, template_values))

and updating _URLs as:

_URLS = [
    ('/runaround/', MainPage),
    ('/runaround/register/',RegisterPage),
]

As evident from these code snippets our home page is going to only render the page index_loggedout.html irrespective of use authentication status.

We need to now work on creating the login handler and have some sort of session management.

Templates for the Main Page

index_loggedout.html : Template to display when user is not logged in.

{% extends "base.html" %}

{% block content %}

  <img src="/ra/static/images/runaround_image.jpg" class="welcome_img" />

  <div class="welcome_dialog">
   <h3>Welcome to the Run Around</h3>

   <p>This is a simple site where you can log your runs and chart progress on your workout routine.</p>

  <div class="clearfix">
   <form action="/runaround/login/" method="post">
      <div class="login_sector">
       <div class="login_prompt">
        <b>Login</b>:
       </div>
       <div class="clearfix">
        <label>Username:</label>
        <input name="username" class="inputtext" type="text" size="20" value="" />
       </div>
         <div class="clearfix">
          <label>Password:</label>
          <input name="password" class="inputtext" type="password" size="20" value=""/>
         </div>
         <input id="submit" class="inputsubmit" value="Login" name="submit" type="submit" />
      </div>
      
      
      <div class="login_sector_fb">
       <div class="login_prompt">Or <b>login</b> with Facebook:</div>
        {% fbconnect_button "medium" %}
      </div>
      
    </form>
    
   </div>
   
    <div class="signup_container">
     Don't have an account? <a href="/runaround/register">Register Now!</a>
    </div>
    
  </div>
 
{% endblock content %}

index.html: Template to display when user is logged in

{% extends "base.html" %}

{% block content %}

 <div class="clearfix">
  <div class="bluebox">
  
   <h3>Where did you run recently?</h3>
   <form action="index.php" method="post">

    <table class="add_run_table">
       
       <tr>
        <td class="editor_key">
          <label id="label_route" for="route">Where did you go?</label>
         </td>
         <td class="editor_value">
           <input id="route" class="inputtext" type="text" size="20" value="" name="route"/>
         </td>
        </tr>
        
        <tr>
        <td class="editor_key">
          <label id="label_miles" for="miles">Number of miles</label>
         </td>
         <td class="editor_value">
           <input id="miles" class="inputtext" type="text" size="20" value="" name="miles"/>
         </td>
        </tr>
        
       <tr>
        <td class="editor_key"><label>Date (MM/DD/YYYY)</label></td>
         <td class="editor_value">
          <input id="date_month" class="inputtext datefield" name="date_month" type="text" size="2" maxlength="2" />
          /<input id="date_day" class="inputtext datefield" name="date_day" type="text" size="2" maxlength="2" />
          /<input id="date_year" class="inputtext datefield" name="date_year" type="text" size="4" maxlength="4" />
         </td>
         </tr>
      </table>
  
    <p id="publish_fb_checkbox" style="'.$style.'" >
        <img src="http://static.ak.fbcdn.net/images/icons/favicon.gif" /> 
        <input type="checkbox" name="publish_to_facebook" checked /> 
        Publish this run to Facebook
         </p>
    <input class="inputsubmit" type="submit" name="submit" value="Add Run"/>
    
   </form>
   
  </div>
 </div>
  
{% endblock content %}

The main page of runaround application

Depending on your logged in status the home page of runaround application (http://www.somethingtoputhere.com/therunaround/index.php) displays different pages.

When user is not logged in:

When user is logged in:

[ Note that our registration page is now capable of creating an account for user but still does not connect to Facebook. In the PHP based application when you use registration page it creates an account with Facebook also. We do not have this functionality yet. ]

Making the password field in Registration form

As shown in the register.html template file the "form" is sent by the handler to the template engine and we have not written the corresponding html. This html is generated based on the registration model we created.

In the registration model we have db.StringProperty as the type of the password field so the template engine renders a simple textbox for it and we would like to have a password field so user (or rather people looking at his screen) can not see the password being typed in this input field.

In django forms it is quite simple, you just have to say that the widget for this field should be PasswordInput as described here. Even though it is called djangoforms the functionalities are not quite similar at App Engine.

Here is how I solved it (with the help of information here) :

Extend db.StringProperty:

class PasswordProperty(db.StringProperty):
    def __init__(self, size=None, maxlength=None, password=False,
                 cssClass=None, **kwargs):
        self._size = size
        self._maxlength = maxlength
        self._password = password
        self._cssClass = cssClass
        super(PasswordProperty, self).__init__(**kwargs)

    def get_form_field(self, **kwargs):
        defaults = {}
        attrs={}
        if self._size:
            attrs['size']=self._size
        if self._maxlength:
            attrs['maxlength']=self._maxlength
        if self._cssClass:
            attrs['class']=self._cssClass
        if self._password:
            defaults['widget']=PasswordInput(attrs)
        else:
            defaults['widget']=TextInput(attrs)
        defaults.update(kwargs)
        return super(PasswordProperty, self).get_form_field(**defaults)

and now our model would look like this:

class RunAroundUser(db.Model):
    username = db.StringProperty(required=True)
    password = PasswordProperty(password=True,required=True)
    name     = db.StringProperty(required=True)
    email    = db.EmailProperty(required=True)
    fb_uid   = db.StringProperty()

Note that the password field is now of type PasswordProperty. I still need to refine the original idea that was here and then the PasswordProperty would be more refined and would not need to have "password=True" attribute.

Registering templates in App Engine

Although in App Engine we are using django templates language the loading of custom template tags is bit different.

In django you can use {% load my_custom_tags %} in your templates to load the custom template tags. This does not worked for me in the App Engine.

Here is how I made it work. In you handler file (which is runaround/runaround.py for us in this example) I have a code snippet

from google.appengine.ext.webapp import template

template.register_template_library('runaround.templatetags.runaround_tags')

runaround/templatetags/runaround_tags.py is the name of file/module which contains the template tags. Its contents were described in this post.

Handler for Registration Page

In my runaround application I have runaround.py, and you must have seen in the app.yaml mentioned in previous post that request to all the urls which have the pattern "/runaround.*: are sent to this python script i.e. runaround/runaround.py

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.webapp import template

import os
import cgi

from runaround import models

template.register_template_library('runaround.templatetags.runaround_tags')

from runaround import rforms

class RegisterPage(webapp.RequestHandler):
    def get(self):
        template_values = {
            'form' : rforms.RegisterForm() 
        }
        path = os.path.join(os.path.dirname(__file__), 'templates/register.html')
        self.response.out.write(template.render(path, template_values))
        
    def post(self):
        data = rforms.RegisterForm(data=self.request.POST)
        if data.is_valid():
            # save the data and redirect to the run page
            entity = data.save(commit=False)
            entity.put()
            self.redirect('/runaround/')
        else:
            # re-print the form
            template_values = {
                'form' : data
            }
            path = os.path.join(os.path.dirname(__file__), 'templates/register.html')
            self.response.out.write(template.render(path, template_values))

_URLS = [
    ('/runaround/register/',RegisterPage),
]

application = webapp.WSGIApplication(_URLS, debug=True)

def main():
    run_wsgi_app(application)

if __name__ == "__main__":
    main()

Registration Form

I have created a file rforms.py in my runaround application and for now the contents look like this:

from google.appengine.ext.webapp import template

from google.appengine.ext.db import djangoforms

from django.newforms.widgets import PasswordInput

import models

class RegisterForm(djangoforms.ModelForm):
    class Meta:
        model = models.RunAroundUser
        exclude = ['fb_uid']

So as the code shows we are inheriting from djangoforms.ModelForm which helps in maintaining the needed information at one place only. Since we would not ask user to enter "fb_uid" which is some internal information we can exclude it.

Looks pretty neat.

app.yaml and static directory structure

application: "PUT_NAME_OF_APPENGINE_ID_HERE"

version: 1

runtime: python

api_version: 1

handlers:

- url: /ra

static_dir: static/runaround

- url: /runaround.*

script: runaround/runaround.py

and our static directory looks something like this:

All the css, images and javascript files shown in this image are retrieved from the PHP runaround application.

Master and Registration Page templates

base.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 

<html xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml"> 
  <head> 
   <title>The Run Around</title> 
    <link type="text/css" rel="stylesheet" href="/ra/static/css/style.css" /> 
    <script type="text/javascript" src="/ra/static/js/base.js"></script> 
  
  </head> 
  
  <body>
  
   <div id="header">
    <div class="header_content">
     <a href="/runaround" class="title">
      <img src="/ra/static/images/runaround_logo.gif" />
     </a>
     
      <div class="account">Hello Guest | 
     <a href="/runaround/register">Register for an account</a>
    </div>
   
    </div>
   </div>
   
   <div class="body_content">
   
    {% block content %} {% endblock %}
    
   </div>
   
   <div class="footer_stuff">This is an awesome running app.</div>

 <script src="http://static.ak.connect.facebook.com/js/api_lib/v0.4/FeatureLoader.js.php" type="text/javascript"></script>
     <script type="text/javascript">
       FB.init("edbc02640bc38965940b56cece534ad5", "/runaround/xd_receiver");
     </script>
 <script type="text/javascript" src="/ra/static/js/fbconnect.js"></script> 
  
  </body>
  
</html>

Nothing fancy here, a simple and typical django base template page which other templates/pages would inherit from.

register.html

{% extends "base.html" %}

{% block content %}

<div class="register">
 <div class="login_sector">
  <h2>Do you use Facebook?</h2>
    Use Facebook to register for The Run Around:
   <br/><br />
   
   {% fbconnect_button "medium" %}
   
    </div>

 <div class="login_sector_fb">
  <h2>Register</h2>
  <form action="/runaround/register/" method="post">
   
   <table>
   
     {{ form }}
      
   </table>
   
   <input type="hidden" name="save" value="1">
   <input type="submit" class="inputsubmit" value="Register" style="margin-left: 80px">

  </form>

 </div>
</div>

{% endblock content %}

The "form" variable shown in the above template would be passed by the handler to the template engine, which we will discuss in next few posts.

Tuesday, October 27, 2009

Templatetag for displaying Facebook Connect button

You normally create a templatetags directory in your application which should contain __int__.py so as to make it a package. In App Engine however there is no need. Here is my python module runaround_tags.py which contain this code:

from google.appengine.ext.webapp import template

register = template.create_template_register()

def fbconnect_button(button_size="medium"):
    return """
        
    """
    
register.simple_tag(fbconnect_button)

Model for Registration information

Here is my model which would contain the registration information

class RunAroundUser(db.Model):
    username = db.StringProperty(required=True)
    password = db.StringProperty(required=True)
    name     = db.StringProperty(required=True)
    email    = db.EmailProperty(required=True)
    fb_uid   = db.StringProperty()

This model is similar to the user class in PHP application but does not yet contain the utilities methods. We will talk about fb_uid field later when we would be integrating Facebook Connect.

Registration Page

Here is how the registration page of runaround application looks like :

pretty nice page. Good thing is that I do not need to worry about look and feed or layout (read 'css and images') of the application.

In the above image you can see that we need to build

A form which will register the user
Display Facebook connect button

Since all the pages in runaround application has same design i.e. header and footer with main area where the real action is it may be a good idea to use django templates and django templatetags.

Monday, October 26, 2009

To use django or webapp framework

I have some experience developing web applications using django but there are some problems using it in app engine. The app engine models are different from that of django and session middleware along with many other nice features of django would not work on app engine.

App Engine does have django as one of its built in libraries but I think I would give webapp framework a go. It is simple, elegant and seems good enough for my simple application.

May be I will port my application to django once I have learned about webapp and we can then compare them also at that time.

If you want to use django on app engine you may want to check out following projects which make it possible:

Configuring Pydev for App Engine application development

No need for me to describe it. Here is the link:

http://code.google.com/appengine/articles/eclipse.html

SDK, Libraries and IDE

So lets setup the development environment and here is mine (assuming Python 2.5 is installed) :

Google App Engine Python SDK
PyFaceBook : A thin wrapper for accessing Facebook's RESTful API through Python.
Aptana Pydev

I am sure I will need to use some more libraries as I made progress.

The runaround Facebook application

In my search for an ideal not-so-simple-but-not-so-complex project I stumbled upon the runaround application. This is decent web application which demonstrates following functionalities :

Registration of user
Authenticate user using username-password or Facebook Connect
A simple form for user to enter his runarounds and a way to display them
Post the runarounds on the Facebook wall
Pull the runarounds of your Facebook friends and display them

If I can implement these functionalities as an app engine web application I think I should be able to get a good idea on how it works and how to use its framework(s). I would also get to learn how to integrate Facebook connect and potentially other Facebook APIs. This is what I call "killing two birds with one stone"

The runaround application has been written in PHP and its design is described in detail here with links to its sources in subversion repository.

App Engine to rescue

The idea of some one else worrying about the scalability of your application is very appealing and that brings me to Google App Engine. I know that even though App Engine is going to do heavy lifting in terms of running and maintaining the servers & databases (read 'BigTable'), a developer needs to make right and informed decisions regarding creating data model, using appropriate caching mechanism and following many other software design principles.

This is why I am starting this blog where I would be writing down my experiences with App Engine as I learn and play with it.

I have always believed that the best way to learn a language and/or software technology is create a not-so-simple-but-no-so-complex project and that is what I intend to do here.

Sunday, November 8, 2009

Sunday, November 1, 2009

Saturday, October 31, 2009

Tuesday, October 27, 2009

Monday, October 26, 2009

Blog Archive