Google Cloud Datastore

What We Will Be Doing

We’ve already made sites with templates.

We will make an app that stores (and retrieves!) data on the cloud. In AppEngine land, one of the services that stores and retrives data is called Datastore.

The Plan

We’re going to learn by doing! Doing the Hogwarts Lab in the curriculum, that is. So clone the CSSI Labs Repository if you have not yet done so. Open up your editor and navigate to appEngine-DataStore/labs/hogwarts-query.

Also, open up the curriculum to the DataStore Intro Page.

What’s in This Lab?

The datastore will be holding:

Kinds

The Kinds are Student, Wand, House, Course, Enrollment, Teacher

Entities

Each individual student, wand, house, etc. is an Entity.

Properties

Entities have properties. For example every student has a student id, first name, and last name. Houses have a name, mascot, and list of students.

Set Up

Once the repo is cloned and you have navigated to the directory, get oriented with the project files:

    .
    ├── README.md
    ├── app.yaml
    ├── hogwarts_models.py
    ├── main.py
    ├── seed_hogwarts_db.py
    └── templates
        └── houselist.html

First, let’s all peek at app.yaml and main.py. Ask if there are any structural things in those files you haven’t seen before.

Next, browse the file hogwarts_models.py.

Next, go back to main.py and look more carefully at the LoadDataHandler. It calls a function. Where does the function that it calls come from? Find that file and browse it too.

Concepts

In the file seed_hogwarts_db.py, what do you think that:

• .put() does?
• so many variables end in _key for?

Code Along

STEP 1 Go to proper folder, and:

$ dev_appserver.py app.yaml --clear_datastore=yes --enable_console=True

Note that command was more complex than before, right?

STEP 2 In your browser, go to: http://localhost:8080/seed-data.

STEP 3 Now go to your interactive console, http://localhost:8000/. Practice listing entities. Cross reference with the code in the seed data file.

STEP 4 Delete any code in the interactive console, and enter and run:

from google.appengine.ext import ndb
from hogwarts_models import Student, House, Teacher, Enrollment, Wand, Course
print(Teacher.query().fetch())

Note something interesting: .query() makes a query, but .fetch() gets all the results from the query

STEP 5 Let’s write some queries and learn as we go.

As we encounter these queries, write down some key words that you encounter (e.g., query, fetch, get, count, filter, order, ...)

We will so some together and you can work on some on your own or with a partner. Don’t forget to help each other. Don’t forget the PEP as well.

For CSSI, we use the NDB library, not the Firestore! Documentation is here.

• What are the names of all the houses?

  houses = House.query().fetch()
  for house in houses:
      print(house.name)
  

• What are the names of all the students, in alphabetical order (by first name)?

  students = Student.query().order(Student.first_name).fetch()
  for student in students:
      print(student.first_name + ' ' + student.last_name)
  

• Who is the mascot of Ravenclaw?

  ravenclaw = House.query().filter(House.name == "Ravenclaw").get()
  print(ravenclaw.mascot)
  

• What is Weasley's Student Id?

  print(Student.query().filter(Student.last_name == "Weasley").get().student_id)
  

• How many courses are there in the datastore?

  print(Course.query().count())
  

• How many students are there in the datastore?

  print(Student.query().count())
  

• How many students are there in Slytherin?

  ravenclaw_students = House.query().filter(House.name == "Slytherin").get().students
  print(len(ravenclaw_students))
  

  Should there be a better way?

There is a limit of 1000 results from fetch().

In general, you should pass a smaller value to fetch and paginate. But, wait, what does this limit mean for “max” queries?

Connecting DataStore to the App

The first connection is done for you! Look in main.py to see the route, and the actual database call!

Find out the app url before revealing 🙂

More Queries Please

• How many classes did Severus Snape teach?

  snape = Teacher.query().filter(Teacher.name == "Severus Snape").get()
  print(len(snape.classes_taught))
  

• Who is the teacher with the most teaching experience?

  teachers = Teacher.query().fetch()
  teacher = max(teachers, key=lambda teacher: teacher.years_experience)
  print(teacher.name + ' has been teaching ' + str(teacher.years_experience) + ' years')
  

• Who is the student enrolled in the most classes?
• What is the mascot of Gryffindor?
• Which house has the Badger mascot?

  print(House.query().filter(House.mascot == "Badger").get())
  

• Which house is Hermione in?

  hermione_key = Student.query().filter(Student.first_name == 'Hermione').get().key
  print(House.query().filter(House.students == hermione_key).get())
  

• Who teaches Defense Against the Dark Arts?
• What is the core of Harry's wand?
• Who has the wand with the Unicorn core?
• Where is Potions taught?
• How many classes is Draco enrolled in?
• What are the names of the classes that Draco is enrolled in?
• Who has the shortest wand?

Now You Can Do Anything

The lab is infinitely expandable.

Oh One More Thing

That lab we did...was run locally at localhost:8080 with the development app server. When you run locally, the datastore is actually saved on your machine. In order to really store your data on the cloud, you just have to deploy your app.

Summary