Let’s Make a CSSI Project

The final project is your chance to bring together all you’ve learned about the command line, HTML, CSS, JavaScript, Git, GitHub, Python, AppEngine, Jinja, the Users API, the Maps API, third-party APIs, and Google Cloud Datastore. Here’s a walkthrough of building a trivial webapp using these technologies. Your own project will be more intense than this one, of course, but will move through many of the same steps outlined here.

Define the Project

At the end of Week 2 of CSSI, you completed project ideation and wrote your design document.

Let’s pretend we have a design document for a simple application with these highlights:

Audience and Purpose
The app allows a community of people to post small text notes to a single, global channel. (Notes can be thoughts, advice, fortunes, aphorisms, whatever; they’re all just text.) Anyone can read the notes but one must login with a Google Account to create notes.
Features
If a note starts with /loc then the content will be replaced with a Google Map centered on the location appearing after loc. If the note starts with /gif then the post will be replaced with a G-rated image from Giphy (using the text after gif as a search string).
Handlers and Pages
HandlerDescriptionPage
GET /Show the “home page” with the most recent 20 notes and a link to add a note.main.html
GET /notesShow a page with a submit-a-note form.note_entry.html
POST /notesAccepts the submitted note, writes it to the data store, and redirects to the home page where the notes are displayed.—
Future work
Author profiles with bios and avatars; comments on notes; multiple channels; tags and categories on notes (similar to Twitter hashtags); ratings on notes; deleting notes.

Create and Clone the GitHub Repo

Before you do any programming, set up your project at GitHub, and get your whole team on board.

My project here is called rtoal/notes. Your project name will of course be different, so please read this writeup with that in mind, and make the necessary adjustments on your own.
  1. While logged in at github.com, select New Repository.
  2. Enter the repo name, a description, keep it public, initialize it with a README (please check that box!), select Python for the .gitignore and MIT License for the license. Create the repository.
  3. You should now looking at your project page on GitHub showing three files (.gitignore, LICENSE, and README.md). You should also see that you have one commit.

    cssi-initial-repo.png

  4. Go to Settings then Collaborators and add the people you will be working with. (They will have to accept your invitation.)
  5. Clone the repo, and check that you’re ready to go: 
    $ git clone git@github.com:rtoal/notes.git
$ cd notes
$ ls -aF
./   ../   .git/   .gitignore   LICENSE   README.md

Make a Minimal App

We’ve already gone through all the details for setting up AppEngine projects already, so we’ll just jump right in. We will create folders and files with this structure: 

  .
  ├── app.yaml
  ├── main.py
  ├── scripts
  │   └── notes.js
  ├── styles
  │   └── notes.css
  └── templates
      ├── main.html
      └── note_entry.html

Let’s go.

  1. We know we want to use Jinja and some local stylesheets and scripts, so let’s start with our complete app.yaml:

    app.yaml
    runtime: python27
api_version: 1
threadsafe: yes

handlers:
- url: "/styles"
  static_dir: styles
- url: "/scripts"
  static_dir: scripts
- url: "/images"
  static_dir: images
- url: ".*"
  script: main.app

libraries:
- name: jinja2
  version: "latest"

  2. And our initial main.py:

    main.py
    import webapp2
import jinja2

jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader('templates'),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)

class MainHandler(webapp2.RequestHandler):
    def get(self):
        template = jinja_env.get_template('main.html')
        self.response.write(template.render())

class NoteHandler(webapp2.RequestHandler):
    def get(self):
        template = jinja_env.get_template('note_entry.html')
        self.response.write(template.render())
    def post(self):
        note = self.request.get('note')
        print('Received: {}\nWill write to datastore later'.format(note))
        self.redirect('/')

app = webapp2.WSGIApplication([
    ('/', MainHandler),
    ('/notes', NoteHandler),
], debug=True)

  3. And our placeholder templates:

    main.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Notes</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Notes</h1>
    <p>Notes will go here</p>
    <p><a href="/notes">Add a new note</a></p>
    <script src="/scripts/notes.js"></script>
  </body>
</html>
    note_entry.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Add a Note</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Add a Note</h1>
    <form method="post" action="/notes">
      <p><textarea name="note" rows="5" cols="50"></textarea></p>
      <p><input type="submit" value="Add"></p>
    </form>
  </body>
</html>

  4. We don’t care yet about scripts and stylesheets, but it is nice to make sure they are wired in. Here’s all we need for now:

    notes.css
    body {
  margin: 0;
  padding: 20px;
  height: 100vh;
  background-image: radial-gradient(gold, skyblue)
}
    notes.js
    console.log('Real scripts coming soon')
  5. Make sure it works! Run a development server for testing: 
    $ dev_appserver.py app.yaml
  6. Open your browser at localhost:8080 and make sure you see the home page. Try the links and forms to make sure your app pages are wired together properly. Please keep in mind we are not saving any notes yet.

    cssi-project-initial.png

Commit and Deploy the Minimal App

We have the basic app in place, so let’s commit.

  1. See how things look with git status: 
    $ git status
  2. You should notice your new files are untracked, and that main.pyc is not listed. Yay! That’s because we asked GitHub to make us a nice .gitignore file, which knows how to ignore .pyc files. Now let’s add and commit and push: 
    $ git add app.yaml index.yaml
$ git commit -m 'Add initial AppEngine files'
$ git push
  3. Do a git status to make sure everything is clean
  4. Take a look at your project on github.com to see that you now have two commits and all the files made it up.
  5. Might as well do an initial deploy now! Create your project on the Google Cloud Console, and do a gcloud app deploy as we learned how to do earlier. Test that the app works on appspot.com.

Add Authentication

We want anyone to be able to read notes (on the home page) but only allow logged in users to post notes. This makes sense, since we want to track the author of each note, and what better way to track the author of the note than to require authors to login?

  1. Update app.yaml so that the /notes handler requires login. (Note this app does not need a login page! Instead, whenever an authenticated handler is hit, Google App Engine will put up the login prompt automatically.

    app.yaml
    runtime: python27
api_version: 1
threadsafe: yes

handlers:
- url: "/styles"
  static_dir: styles
- url: "/scripts"
  static_dir: scripts
- url: "/images"
  static_dir: images
- url: "/notes"
  script: main.app
  login: required
- url: ".*"
  script: main.app

libraries:
- name: jinja2
  version: "latest"

  2. Update main.py so that if a user is logged in, we will send their nickname and a logout url to the template. Upon logout, we will direct the user to the home page. Good thing that the Google Users API has that create_logout_url function.

    main.py
    import webapp2
import jinja2
from google.appengine.api import users

jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader('templates'),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)

class MainHandler(webapp2.RequestHandler):
    def get(self):
        notes = ['Fake note 1', 'Fake note 2']
        user = users.get_current_user()
        logout_url = users.create_logout_url('/') if user else None
        template = jinja_env.get_template('main.html')
        self.response.write(template.render({
            'notes': notes,
            'nickname' : user.nickname() if user else None,
            'logout_url': logout_url }))

class NoteHandler(webapp2.RequestHandler):
    def get(self):
        user = users.get_current_user()
        template = jinja_env.get_template('note_entry.html')
        self.response.write(template.render({
            'nickname' : user.nickname(),
            'logout_url': users.create_logout_url('/')}))
    def post(self):
        note = self.request.get('note')
        print('Received: {}\nWill write to datastore later'.format(note))
        self.redirect('/')

app = webapp2.WSGIApplication([
    ('/', MainHandler),
    ('/notes', NoteHandler),
], debug=True)

  3. Update the templates, so that if a user is logged in, we render their nickname and a logout button. (Again, this app is not using a login button, just a logout button. Login is triggered by trying to post a note, rather than with an explicit login link.)

    4. main.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Notes</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Notes</h1>
    {% if nickname %}
      <p>Welcome, {{ nickname }} | <a href="{{ logout_url }}">Logout</a></p>
    {% endif %}
    <section>{{ notes }}</section>
    <p><a href="/notes">Add a new note</a></p>
    <script src="/scripts/notes.js"></script>
  </body>
</html>
    note_entry.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Add a Note</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Add a Note</h1>
    {% if nickname %}
      <p>Welcome, {{ nickname }} | <a href="{{ logout_url }}">Logout</a></p>
    {% endif %}
    <form method="post" action="/notes">
      <p><textarea name="note" rows="5" cols="50"></textarea></p>
      <p><input type="submit" value="Add"></p>
    </form>
  </body>
</html>
  4. Test by clicking around. When running with dev_appserver.py, you get a simulated login widget that comes up when needed. When running on GAE on the cloud, you get a real Google authentication page.

    cssi-project-2.png

  5. Make sure you practice with logging out and back in a few times.
  6. Git add, commit, and push.
  7. At this point, you can gcloud app deploy if you wish. It’s more important to commit frequently than it is to deploy frequently.

Datastore Time

When working on large projects with large teams, connecting to a datastore comes late in the process, and we tend to develop applications with fake data. Sometimes even the choice of a datastore is postponed until very late in the game, too. Since CSSI project week is very short and projects are done in groups of only three, and we know we will be using Google Cloud Datastore, it’s fine to build our datastore models right away.

  1. Create the file models.py in the project folder, with one model, for our notes. Our note authors will simply be logged-in users that have Google Accounts; the Users API exposes a unique id which we will use. The note content will be a string, and we will keep track of the timestamp at which the note was created. Look at how DataStore can do this for us automatically:

    models.py
    from google.appengine.ext import ndb

class Note(ndb.Model):
    user_id = ndb.StringProperty()
    content = ndb.StringProperty()
    timestamp = ndb.DateTimeProperty(auto_now_add=True)

  2. Update main.py to use the model. We need to both fetch models on the main page, and put a new object into the datastore in the post handler.

    3. main.py
    import webapp2
import jinja2
from google.appengine.api import users
from models import Note

jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader('templates'),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)

class MainHandler(webapp2.RequestHandler):
    def get(self):
        notes = Note.query().order(-Note.timestamp).fetch(limit=20)
        user = users.get_current_user()
        logout_url = users.create_logout_url('/') if user else None
        template = jinja_env.get_template('main.html')
        self.response.write(template.render({
            'notes': notes,
            'nickname' : user.nickname() if user else None,
            'logout_url': logout_url }))

class NoteHandler(webapp2.RequestHandler):
    def get(self):
        user = users.get_current_user()
        template = jinja_env.get_template('note_entry.html')
        self.response.write(template.render({
            'nickname' : user.nickname(),
            'logout_url': users.create_logout_url('/')}))
    def post(self):
        note = self.request.get('note')
        user = users.get_current_user()
        Note(user_id=user.user_id(), content=note).put()
        self.redirect('/')

app = webapp2.WSGIApplication([
    ('/', MainHandler),
    ('/notes', NoteHandler),
], debug=True)

  3. Update the template pages so the notes are shown. And while we’re growing our app, let’s touch up the styles as well.

    main.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Notes</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Notes</h1>
    {% if nickname %}
      <div>Welcome, {{ nickname }} | <a href="{{ logout_url }}">Logout</a></div>
    {% endif %}
    <section>
    {% for note in notes %}
      <article>
        {{ note.timestamp }} ({{note.user_id}}) {{ note.content }}
      </article>
    {% endfor %}
    </section>
    <p id="addbutton"><a href="/notes">Add a new note</a></p>
    <script src="/scripts/notes.js"></script>
  </body>
</html>
    note_entry.html
    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Add a Note</title>
    <link rel="stylesheet" href="/styles/notes.css">
  </head>
  <body>
    <h1>Add a Note</h1>
    {% if nickname %}
      <div>Welcome, {{ nickname }} | <a href="{{ logout_url }}">Logout</a></div>
    {% endif %}
    <form method="post" action="/notes">
      <p><textarea name="note" rows="5" cols="50"></textarea></p>
      <p><input type="submit" value="Add"></p>
    </form>
  </body>
</html>
    notes.css
    body {
  margin: 0;
  padding: 0;
  height: 100vh;
  background-image: radial-gradient(gold, skyblue);
  font-family: Arial;
}

h1 {
  background-color: purple;
  color: white;
  font-size: 32px;
  margin: 0;
  padding: 20px 0;
  text-align: center;
}

div {
  background-color: #faf;
  padding: 10px;
}

article {
  font-size: 18px;
  background-color: lightsteelblue;
  border: 1px solid red;
  border-radius: 5px;
  margin: 20px;
  padding: 8px;
}

p#addbutton {
  margin-left: 20px;
}

p#addbutton a {
  font-size: 16px;
  color: white;
  background-color: blue;
  padding: 8px;
  text-decoration: none;
}

textarea, input {
  margin-left: 20px;
}
  4. Re-run the app locally to make sure you clear the datastore: 
    $ dev_appserver.py app.yaml --clear_datastore=yes
  5. Go to localhost:8080 and refresh if necessary. Add a note, something like “First Note!!!”. This should take you to a simulated login page. (If you were running on the cloud, you would see a real Google Auth page).
  6. Add the note. But wait! WHERE IS THE NOTE? WHY DID YOU NOT SEE IT? Hmmmm. Refresh the page. Now it is there!

    cssi-project-3.png

  7. Okay so the reason you did not see the post right away is that it takes some time for a put to the datastore to actually land there. When we say .put(), AppEngine does not wait for the put to complete. Instead .put() only initiates the process of putting while the code keeps running. In our app, we started the put process but we redirected to the home page and read the most recent notes, and this happened before the .put() finished. When we (as a slow human) waited a couple seconds and refreshed the page, it was finally there. This can be fixed by adding a new handler that returns the timestamp of the most recent note. A bit of JavaScript on the notes page calls this handler 1.5 seconds after the page is loaded. If the timestamp from the database does not match the timestamp of the first note on the page, the page gets reloaded. By that time, the new note will have made it to the database. And if not, well, the user can manually reload.

    main.py
    import webapp2
import jinja2
from google.appengine.api import users
from models import Note

jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader('templates'),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)

class MainHandler(webapp2.RequestHandler):
    def get(self):
        notes = Note.query().order(-Note.timestamp).fetch(limit=20)
        user = users.get_current_user()
        logout_url = users.create_logout_url('/') if user else None
        template = jinja_env.get_template('main.html')
        self.response.write(template.render({
            'notes': notes,
            'nickname' : user.nickname() if user else None,
            'logout_url': logout_url }))

class NoteHandler(webapp2.RequestHandler):
    def get(self):
        user = users.get_current_user()
        template = jinja_env.get_template('note_entry.html')
        self.response.write(template.render({
            'nickname' : user.nickname(),
            'logout_url': users.create_logout_url('/')}))
    def post(self):
        note = self.request.get('note')
        user = users.get_current_user()
        Note(user_id=user.user_id(), content=note).put()
        self.redirect('/')

class MostRecentNoteHandler(webapp2.RequestHandler):
    def get(self):
        note = Note.query().order(-Note.timestamp).get()
        self.response.headers['Content-Type'] = 'text-plain'
        self.response.write(str(note.timestamp if note else ''))

app = webapp2.WSGIApplication([
    ('/', MainHandler),
    ('/notes', NoteHandler),
    ('/most-recent-note', MostRecentNoteHandler),
], debug=True)
    notes.js
    window.addEventListener('load', event => {
  setTimeout(() => {
    fetch('/most-recent-note').then(r => r.text()).then(most_recent => {
      let displayed = document.querySelector('article');
      displayed = displayed ? displayed.textContent.trim() : '';
      if (!displayed.startsWith(most_recent)) {
          window.location.reload();
      }
    });
  }, 1500);
});
  8. Time to save our work! Add! Commit! Push!

Using Giphy

  1. We want to replace all notes that begin with /gif with an actual gif from Giphy. We just need to add one statement (it’s a long one) to the end of our script:

    notes.js
    window.addEventListener('load', event => {
  setTimeout(() => {
    fetch('/most-recent-note').then(r => r.text()).then(most_recent => {
      let displayed = document.querySelector('article');
      displayed = displayed ? displayed.textContent.trim() : '';
      if (!displayed.startsWith(most_recent)) {
          window.location.reload();
      }
    });
  }, 1500);
});

document.querySelectorAll('article div').forEach(div => {
  if (div.textContent && div.textContent.trim().startsWith('/gif ')) {
    const q = encodeURIComponent(div.textContent.slice(5));
    const baseUrl = 'https://api.giphy.com/v1/gifs/search';
    const url = `${baseUrl}?api_key=dc6zaTOxFJmzC&limit=1&rating=g&q=${q}`;
    fetch(url).then(r => r.json()).then(result => {
      const imageUrl = result.data[0].images.original.url;
      div.innerHTML = `<img src=${imageUrl}>`;
    });
  }
});
  2. Test.
  3. Commit.

Clean up!

Before deploying, we should clean the code. Things to consider:

I extended the app a bit. You can see the code on GitHub.