Overview

In this assignment we will augment our frontend knowledge of creating beutiful websites by learning how to design backends, the designs that provide your sites with data and handle complex logic. You will build a replica of the API for the quiz website Kahoot. The assignment will help you practice more complex database and API design techniques and help you learn how to test a backend without using a frontend.

Setup

Database (Mongo)

First, you need to install mongodb, the database client for this api.

On OSX something like this should work:

brew uninstall --force mongodb
brew tap mongodb/brew
brew install mongodb-community

Pull Starter

Now, go ahead and pull the backend starterpack that contains the pacakges and boilerplate needed for a express and mongo backend.

We're going to be building an API, where users can make requests to create or respond to quizzes. We will be using a backend starterpack to start, which sets us up with an express node server with a tiny bit of boilerplate as well as linting and babel. You could easily recreate this, but for now we'll save you some time by providing it for you.

🚀 This is similar to cloning a repo with git clone, but with a key difference. Instead of cloning from one remote location, we will add a new "remote" (basically a reference to a repo hosted on Github) so that we can pull code from there but also retain the reference to our repo for this project. That way, we don't modify the starterpack when we want to push our changes to Github.

# make sure you are in your project directory
git remote add starter STARTER_REPO_URL
git pull starter main

Then run these following commands in your project directory to start our new node+express app in dev reloading mode.

npm install
npm start

Intro Express

Express is a server side web framework for Node.js. What it does for us is provide a way to listen for and respond to incoming web requests. Today, we will be creating API endpoints to respond to certain CRUD-style requests.

Recall that the src/server.js file is the entry point for our API. Note how we are setting the route:

// default index route
app.get('/', (req, res) => {
  res.send('hi');
});

The 2nd parameter to .get() is a function that takes 2 arguments: request and response.

request is an express object that contains, among other things, any data that was part of the request. For instance, the JSON parameters we would POST or PUT in our asynchronous axios calls would be available as req.body.parameterName.

response is another special express object that contains, among other things, a method named send that allows us a send back a response to the client. When your API call gets back JSON data this is how it is returned. Consider res.send() the equivalent of a network based return statement. This is important. You can only have 1 res.send(). Note that it is good practice to return res.send() unless you intend to run code after sending the response.

We'll add more routing in shortly, but first let's set up our database!

Mongo Database Server

We will need a database to store the information aboout the kahoot quizzes and the responses from players. For this version of the assignment, we will be using the non-relational MongoDB as our database. We've already installed mongodb using Homebrew.

In assignment NNN, you will replicate the API with PostgreSQL, a relational database.

🚀 You may need to run the brew services start mongodb-community process, which your node app will connect to. This is a background server process.

Recall you can interface with your database from the commandline using the mongo shell with the command mongo. You can also play around with a more graphical client mongodb compass community (just make sure to download the community version).

Mongoose

To interface with mongo for our API, we will use a module called mongoose. Mongoose is a an object model for mongo. This allows us to treat data that we are storing in mongo as objects that have a nice API for querying, saving, validating, etc. rather than writing Mongo queries directly. Mongo is in general considered a schema-less store. We store JSON documents in a large object tree similarly to firebase. However, with Mongoose we are able to specify a schema for our objects. This is purely in code and allows use to validate and assert our data before inserting it into the database to protect us from pesky bugs.

🚀 Install mongoose: npm install mongoose

🚀 Add this snippet to get mongoose initialized with our database at the bottom of server.js. We will wrap this in an anonymous async function so that we can await our call to connect to the database:

import mongoose from 'mongoose';

// DB Setup
(async () => {
  // connect mongo
  const mongoURI = process.env.MONGODB_URI || 'mongodb://localhost/kahootAPI';
  await mongoose.connect(mongoURI);

  // start listening
  app.listen(port, () => {
    console.log(`Listening on port ${port}`);
  });
})();

Models

We're going to create a data model to work with. A data model in mongoose is initialized from a schema, which is a description of the structure of the object. This is much more like what you might be familiar with statically typed classes in Java.

🚀 Create a directory src/models and a file inside this directory named room_model.js. This is very similar to the express-mongo assignment.

import mongoose, { Schema } from 'mongoose';

// schema
const RoomSchema = new Schema({
  creator: String,
  questions: [String],
  answers: [String],
  submissions: [],
}, {
  toObject: { virtuals: true },
  toJSON: { virtuals: true },
  timestamps: true,
});

// class
const RoomModel = mongoose.model('Room', RoomSchema);

export default RoomModel;

Take a look at the submissions field of the Room model above. We know that the questions and answers fields should both be the same length, since each answer corresponds to one question. We could validate that these two fields have the same number of elements when a user tries to create a game, but why not combine these two fields into an array of nested documents. Our model will look a lot cleaner, and we will get the validation for free!

We also know that the submissions to a room will be an array, in this case an array of responses from various different players. We should also check that the submissions follow a consistent format. Otherwise, players could submit more answers than the number of questions or submit the wrong type of data in response to the questions. This might cause problems when we later try to evaluate the submissions.

To solve this issue, also create a file submission_model.js that contains a SubmissionModel. This represents a submission (a series of answers to the room's questions) by one player. The submission will indicate the player who submitted the response and the responses themselves. The user field is a username string (attached to that room) that we let them define. Right now, there could be duplicates in a given room.

EC: What type of validation might we want on usernames? What would we have to do if we wanted usernames to persist across rooms?

While including the the SubmissionSchema directly in the RoomSchema would work just fine, it would also mean that all the submissions would be stored inside with the rooms that they are for. For the purpose of this assignment, we are going to take a different approach, using mongoose's populate() function to retrieve the submission information only when we need it and storing the submissions in a separate collection. This will become more clear once we write the controllers for rooms and submissions. Write your SubmissionSchema to reflect this change:

const SubmissionSchema = new Schema({
  user: {
    type: String,
    required: true,
  },
  responses: [String],
}, {
  toObject: { virtuals: true },
  toJSON: { virtuals: true },
  timestamps: true,
});

We hope you do not leave this assignment thinking that there are not alternative approaches. We could have the submissions stored inside of the room document or even have the rooms not store any information about which submissions they own and instead have the submissions know the rooms to which they belong. All of these are valid, but here we chose to take the approach of storing references to the submissions inside the room document, but keeping the bulk of the information in a separate collection.

Now, let's update the RoomSchema to include our new combined question/answer format and submission schema.

import mongoose, { Schema } from 'mongoose';

// schema
const RoomSchema = new Schema({
  creator: String,
  questions: [{ prompt: String, answer: String }],
  submissions: [{
    type: Schema.Types.ObjectId,
    ref: 'Submission',
  }],
}, {
  toObject: { virtuals: true },
  toJSON: { virtuals: true },
  timestamps: true,
});

// class
const RoomModel = mongoose.model('Room', RoomSchema);

export default RoomModel;

Finally, let's add a virtual to our room schema to compute the "scoreboard" to compare how players are performing in a given game. Add the following code to the bottom of room_model.js:

function generateScoreboard() {
  const scoreboard = this.submissions.map((submission) => {
    let numCorrect = 0;
    submission.responses.forEach((response, index) => {
      if (response === this.questions[index].answer) {
        numCorrect += 1;
      }
    });
    return { Player: `${submission.player}`, 'Correct Answers': `${numCorrect}` };
  });

  // sort the scoreboard by decreasing number of correct answers
  scoreboard.sort().reverse();
  return scoreboard;
}

RoomSchema.virtual('scoreboard').get(generateScoreboard);

Controllers

🚀 Create a directory src/controllers and a file inside this named room_controller.js. This controller will be responsible for handling all the requests related to rooms (aka games) for our API:

export const createRoom = (roomInitInfo) => {
    
};

export const deleteRoom = (roomID) => {
    
};

// for room creators checking the status of the game or room properties
export const getAllRoomInfo = (roomID) => {

};

// for players checking the status of the game
export const getLimitedRoomInfo = () => {

};

export const getRoomInfo = (roomID) => {

};

export const getScoreboard = async (roomID) => {
  
};

Also add a file submission_controller.js that will handle submissions-related requests.

export const submit = async (roomID, user, responses) => {
    
};

These are empty right now, but we will come back to them soon.

Routing

Before we finish the controllers, let's define our routes. Each route defines a url that a client of the api can hit to either retrieve data or send new data to be stored. Our api will listen to requests at these routes and respond accordingly, usually by calling one of the controller methods we defined above. In the first assignment, we put our routes in src/server.js, now we'll move them out to a new file, src/routes.js:

// default index route
router.route('/rooms')
  .get(async (req, res) => {
    try {
      const rooms = await Rooms.getAllRooms();
      return res.json(rooms);
    } catch (err) {
      return res.status(500).json(`${err}`);
    }
  })
  .post(async (req, res) => {
    const roomInitInfo = req.body;

    try {
      const result = await Rooms.createRoom(roomInitInfo);
      return res.json(`Created room with id: ${result.id}`);
    } catch (err) {
      return res.status(500).json({ err });
    }
  });

router.route('/rooms/:id')
  .get(async (req, res) => {
    const roomID = req.params.id;
    try {
      const roomInfo = await Rooms.getRoomInfo(roomID);
      return res.json(roomInfo);
    } catch (err) {
      return res.status(404).send(`ERROR: ${err}`);
    }
  })
  .delete(async (req, res) => {
    const roomID = req.params.id;
    try {
      await Rooms.deleteRoom(roomID);
      return res.json({ message: `Room ${roomID} deleted successfully` });
    } catch (err) {
      return res.status(500).json({ err });
    }
  });

router.get('/rooms/:id/scoreboard', async (req, res) => {
  const roomID = req.params.id;
  try {
    const scoreboard = await Rooms.getScoreboard(roomID);
    return res.json(scoreboard);
  } catch {
    return res.status(404).json({ message: 'Room not found' });
  }
});

router.post('/rooms/:id/submission', async (req, res) => {
  const roomID = req.params.id;
  const { player, responses } = req.body;

  try {
    await Rooms.submit(roomID, player, responses);
    return res.json({ message: 'Submitted successfully' });
  } catch (err) {
    return res.status(500).json(`${err}`);
  }
});

export default router;

Recall that, when we reference req.params.id, the :id defined in the route. When the user makes a GET request to the route rooms/33, for example, req.params.id is 33.

🚀 Running into trouble? Don't forget to import your Room and Submission controller functions:

import * as Rooms from './controllers/room_controller';
import * as Submissions from './controllers/submission_controller';

Back to Controllers

Now we'll finish the controllers for both the submissions and rooms.

createRoom

We should first implement the createRoom endpoint. Without any rooms, we can't play!

🚀 We use the new keyword to initialize a new room, then give it the appropriate properties and save it using the .save() method:

export const createRoom = (roomInitInfo) => {
  const newRoom = new Room();
  newRoom.creator = roomInitInfo.creator;
  newRoom.questions = roomInitInfo.questions;
  newRoom.answers = roomInitInfo.answers;
  newRoom.submissions = [];

  return newRoom.save();
};

The .save() method returns a promise, so we can await this method when we call it.

🚀 Let's see if this works! For this assignment, we do not have a frontend, so we will instead use Postman to test our API. Postman is a tool that lets us make requests to an api. You can download Postman as a desktop app or just use the web interface by creating an account and navigating to web.postman.co.

Start up your api and, using Postman, test that creating a room via this endpoint returns a success message.

deleteRoom

🚀The complement to creating a room is deleting it. Write the deleteRoom method to handle this.

export const deleteRoom = (roomID) => {
  return Room.findByIdAndDelete(roomID);
};

getRoomInfo

We need an endpoint for two groups to be able to access the information about a room: the room creator/administrator and the players participating in that room. The issue is that these groups have different privileges in terms of what they should be allowed to know about a room.

The creator should obviously be able to see all of the associated data to make sure they have set up the room correctly or to see what participants have submitted. The players, however, should only be able to see the questions (so they can submit answers to them!) and the scoreboard (to see how they are doing). You can imagine the game would be pretty boring if players got the answers alongside the questions.

Therefore, we need some way to return different information to these two different parties. We could have two different endpoints, but there is a risk that the route for the "creator" endpoint is accidentally exposed, which would lead to cheating. Instead, we can have an optional query parameter that is a "password" to access the extra, protected information about the room. The creator will specify the parameter when creating the room and include it in subsequent requests to this endpoint. Depending if a request to GET rooms/:id has a password (and it is correct), we can pass the request to two different controller methods.

To support this behavior, we will need to modify our rooms/:id route:

const roomID = req.params.id;
const { roomKey } = req.query;
if (roomKey) {
  try {
    const room = await Rooms.getAllRoomInfo(roomID, roomKey);
    return res.json(room);
  } catch (err) {
    return res.status(500).json({ err });
  }
} else {
  try {
    const roomInfo = await Rooms.getLimitedRoomInfo(roomID);
    return res.json(roomInfo);
  } catch (err) {
    return res.status(500).json({ err });
  }
}

We will obviously need to split getRoomInfo into getAllRoomInfo and getLimitedRoomInfo as well. Finally, we will need to modify our model and

Deploy to Render?

MongoDB Atlas

Wait, but we don't have a database on our remote server! The problem is that Heroku does not support easy storage, there is no "hard drive" to save a database file on for instance. Every Heroku process (what runs your code every time you push), is called a Dyno - and Dynos don't get their own filesystems. They get what Heroku calls an ephemeral filesystem, more of a temporary scratchpad.

To run a mongoDB process with remote access there are several options, but we'll choose the cloud mongo option offered by mongodb.com.

1. Create an account at cloud.mongodb.co
2. Select the free Shared Clusters.
3. Pick most the defaults, in particular under Cluster Tier Select the M0 Sandbox(which is free). Don't turn on backups as that will add cost.
4. This will create a "Project 0" with "Cluster 0". You are limited to 1 free cluster per project, so later on you may want to create more. For now it is probably fine to use the same database for all your projects.
5. Create an access username and password. Save them.
   
   {: .medium .fancy}
   
   {: .medium .fancy}
6. In Network Access, select Allow Access From Anywhere
   
   {: .medium .fancy}
7. Click Clusters -> Connect -> Connect Your Application
8. Copy the connection string into a safe place and replace <password> with the password you saved earlier.

Connect Heroku to Mongo

1. Now you need to connect to a mongo database. Go to dashboard.heroku.com.
2. Go to Settings -> Reveal Config Vars
   This is where you can add environment variables — a great place for things like api keys and connection strings.
3. Add a key MONGODB_URI and paste the connection string you saved above into it. Remember to replace <password> with the actual password.

Testing!

We will test your api using Curl, a CLI for making web requests.