# Banter - Project Proposal


Banter is a social network for students. It is a fullstack application that can be retro-fitted into any school's IT infrastructure to enhance the student experience.

## Contents

[TOC]

## Overview

Banter is a Social Network and Marketplace for students. Here is the complete overview of the application. The app's main features are as follows -

- Social network for students to exchange info and connect with others on campus.
- Provides a news feed for students with the latest information in and around their campus.
- Students will be able to see and create events on their campus.
- Students will be able to buy and sell anything. (Marketplace for students)
- Students will be able to create and join clubs.
- Students will be able to create content in the clubs.

## Components

We will be using a wide range of open source frameworks and tools to decompose the application into microservices. These are some of the components that we expect to use.

### Tools and Libraries

| Name | Version | Purpose
| -------- | ------- | --------
| [VS Code](https://code.visualstudio.com/) | *1.72* | Code editor.
| [Git](https://git-scm.com/doc) | *2.38* | Version Control system.
| [Github](https://github.com/) | *N/A* | Git projects hosting service.
| [Python](https://docs.python.org/3/) | *3.10.7* | Primary language for our backend.
| [HTML](https://developer.mozilla.org/en-US/docs/Web/HTML) | *5* | Front end markup language.
| [CSS](https://developer.mozilla.org/en-US/docs/Web/CSS) | *3* | Front end styling markup language.
| [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript) | *ES6* | Primary frontend language.
| [JQuery](https://api.jquery.com/) | *3.6.1* | Front end JS library.
| [React](https://reactjs.org/docs/getting-started.html) | *18.2* | Primary frontend UI framework.
| [SQL MariaDB](https://mariadb.org/documentation/) | *10.11* | Primary Database for all microservices. Flavor of MySQL.
| [Bootstrap](https://getbootstrap.com/docs/5.2) | *5.2* | Front end library for UI styling and components.
| [RabbitMQ](https://www.rabbitmq.com/documentation.html) | *3.11* | Message queue for the microservices event bus.
| [Kafka](https://kafka.apache.org/documentation/) | *3.3* | Message queue for the microservices event bus.
| [Docker](https://docs.docker.com/) | *20.10.14* | Container runtime for our services.
| [Kubernetes](https://kubernetes.io/docs/home/) | *1.23* | Container orchertrator for our microservices.

### Cloud Providers

We will use either of the below options as our primary cloud provider.

- [GCP](https://cloud.google.com/docs)
- [Linode](https://www.linode.com/docs/)

> *Linode* is a relatively new **`IaaS`** platform with very cheap prices compared to GCP. [color=#2E8D5D]

### Kubernetes Specs

Provider | CPU | RAM | Storage | Price/Month
---------| -------- | -------- | -------- | -------
Linode | 1 | 2GB | 50GB | $10

As you can see the prices are very nominal for a *Linode Kubernetes Cluster* ([`LKE`](https://www.linode.com/products/kubernetes/)) and we will go ahead with this provider for our cloud computational needs. Mentioned specs are more than sufficent to build and test out our prototype. We would ideally like to use [Terraform](https://www.terraform.io/docs) for easy provisioning/tracking of the services.
> **`Terraform`** has a provider for `Linode`.
> The Kubernetes container runtime will be `Docker`. [color=#3069DE]

### Event Bus

- [Apache Kafka](https://kafka.apache.org/documentation/)
- [RabbitMQ](https://www.rabbitmq.com/documentation.html)

As we are not yet sure about the pros and cons of `RabbitMQ` and `Kafka` we have included both in this proposal. As we explore more and do a deep dive, we will zero in on one of them or use both of them for different events.

### CDN

- [Cloudinary](https://cloudinary.com/documentation)

We will be using `Cloudinary` as our CDN service. This is a free CDN service that provides a REST API for our static assets like *Images* and *Videos* and abstracts the storage and management of assets.

## Architecture

The primary actors are -

- **User**: There are 4 types of users. We will enable a RBAC with the following roles/user archetypes.
- ***`Student`***: The default primary role with following functions.
- `Add`/`edit`/`remove` **posts** on news feed.
- `Add`/`edit`/`remove` **clubs** in thier school.
- `Add`/`remove` self as **club** members
- `Add`/`remove` **events** on campus
- `Add`/`remove` **products** for sale
- `Buy` **products**. `Cancel` **orders**.
- ***`Business Owner`***: A minimal role for business owners on campuses to start advertisement campaigns in their school.
- ***`School Admin`***: All privileges of student with some elevated privileges for school management.
- ***`Super Admin`***: All privileges of School Admin with site wide access and dashboards with granular control of site actions and management.
- **School**: All the content that is generated on the app will be invariably tied to one school. Hence, this entity is a primary actor.
- **Store**: The `Store` service will house the `Order` service also and will will be interact heavily with the Product services to provide endpoints for
- Order Creation
- Order Management
- Creating a Storefront

:::info
:memo: There are other actors in the application but their features/behaviors are self explanatory and will become more apparent in the upcoming sections of this document
:::

### Entity Relationships

The following is our ERD outlining the major actors in our application.


### Database Schema


The above diagram shows the database schema for **Banter**. It features extensive relationships centered around all the entities. However, the most important actors/entities are the following.

:::info
All the content generated on the app are centered around :busts_in_silhouette: **User** and :school: **School** entities. Hence they are of great importance.
:::

<!-- ### Simplified DB Schema

 -->

### Current Architecture


The current architecture is a `Flask` application with various modules as can be seen above. While the app functions perfectly fine in the current state. It is highly coupled and difficult to scale. Decomposing to microservices will help maintain seperation of concerns and make the individual services independent, scalable and reduce any SPOFs.

### Proposed Architecture

The proposed architecture is a fully decoupled and highly cohesive microservice architecture with every service having a single responsibility taking care of one business domain requirement.

#### App Architecture


<br>

#### Marketplace Architecture *(Ext of previous)*


As you can see above, the proposed architecture is a decomposition of the existing monolith architecture into 10 different microservices. Each microservice will represent one business logic and is decomposed as such. The overall goal is to convert the applicaion into several services that each have a single responsibility. We aim to do this by adopting an event driven architecture.

The services are -

- **Web App Service**: This will be the front end React application that the users interact with. Will use *Bootstrap* components for UI elements.
- **User Service**: This service will be responsible for registering and signing in users and will provide an interface for other services within the application to fetch user data. This service will feature functions such as
- Register `User`
- Signin user
- Fetch user details
- **Billboard/Newsfeed Service**: This service will be responsible for the news feed in the application and will feature functions such as -
- Fetch posts
- For `All`, `User`, `School`
- Create/Delete post
- **Events Service**: This service will provide endpoints for creating and managing Events on a campus. This will have functions such as -
- Fetch Events
- For `All`, `User`, `School`
- Insert/Create Events
- **Clubs Service**: The clubs service will be responsible for facilitating users to create/join clubs and interact with other users. This service will feature endpoints for -
- Create/delete `Club`
- Join/Leave club memberships for `User`
- Fetch clubs
- For `All`, `School`
- Fetch club details/Info
- For `Club`
- Fetch club members
- For `Club`
- **CDN Service**: This is a service that we will build for other services to work with and handle static data like `Images` & `Video` content generated by the users. This service will use the [Cloudinary](https://) API internally and provide a layer of abstraction for our other internal services. These are the functions we will support -
- Upload Image/Video
- *Input*: `Object` (Image/Video)
- *Response*: Static url for hosted object
- Fetch Resource Object (Image/Video)

:::danger
:wastebasket: &nbsp; We will not support resource deletion of uploaded assets on our CDN provider.
:::
- **Ads Service**: This service will aggregate the ads created by business owners. The ads can then be requested by any other service to display along with its data. It will provide an API for other services to -
- Fetch Ads and dynamically inject into other content like `Newsfeed` and `Store`.
- For `School`
- Create/Delete ads.
> ***Example***: The `Newsfeed` service can request `Ads` and then inject it into the Newsfeed posts for the users to see. [color=#FD7E14]
- **Product Service**: This service will provide an API and all necessary endpoints to operate the `Storefront` on the app. Every school will have 1 storefront. This service will also internally use the `CDN` service to store and retrieve product images. Its functions are -
- Fetch products
- For `All`, `School`, `User`
- Create/Delete Products
- **Orders Service**: This service will be one of the primary services in the application which facilitates students to buy and sell products with other students. This service will be loosely coupled with the `Product` service to create an e-commerce ecosystem. This service will have functions such as -
- Create Order
- `Order` created with `Product` & `User`.
- Fetch Order
- For `All`, `User`
- Delete Order
- **Message Queue**: This service will form the backbone of the entire application architecture. We will use the `Publish`/`Subscribe` pattern to store and queue messages for our overarching event driven architecture. We propose to use `Apache Kafka` or `RabbitMQ` as the message broker. All other services will be subscribed to specified topics and also laterally publish certain messages for others to consume.
> More info on ***Message Queue*** and the ***Messages*** pattern can be seen below in the **Design** section. [color=#FD7E14]

:::info
Every service and/or sub-services listed here will emit **message**(s). This message will propogate through the mesaage queue and will be broadcast to other services that are **listening** or **subscribed** to this message topic. :incoming_envelope:
:::

## Design

The different backend services will be written in **`Python`**.
The front end application will be built with **`React`** using **`JavaScript`**.
**`Maria DB`** will be used as the datastore for all the services.
The message queue/ event bus will be built using **`RabbitMQ`** and/or **`Apache Kafka`**.
**`Docker`** will be used as the container runtime and **`Kubernetes`** for the container orchestration.

Now lets take a deeper look into some aspects of the design model and architecture.

### Microservice

Essentially, every microservice will have the following design. We will pair up every service with its own highly available database. Every service will store its data only in its coupled DB. Every service will typically be 2-3 containers structured as follows -

- **1** Service
- **1** Database (`Maria DB` or `Mongo DB`)
- **1** Database Manager (`Adminer` or `Mongo Express`)

> *Database Manger* is optional and we would typically attach other containers as well for monitoring and logging on a need basis. (**`Sidecar Pattern`**) [color=#FD7E14]

:::warning
:pushpin: &nbsp; **Services will fetch/receive data from other services through network requests only or by subscribing to messages from the message queue.**
:::

:::danger
:warning: &nbsp; **A service will never read/query from another service's database directly.**
:::


### Kubernetes Pod

Every microservice will be packaged as a Pod which will look like the image shown below. Services and their datastores will be in seperate containers. Both the containers will form a `Pod`.


### Message Queue / Event Bus

The most important part of this entire architecture will be the message queue.
This will be built using `RabbitMQ` and/or `Apache Kafka`. It will form the backbone of the entire app and will facilitate the event driven architecture. Every service in the application will emit events onto the message queue when different events are triggered. Services will listen to all the events and consume events relavant to them.


> ***Example*** - When an *order* is created in the `Order` service, it will emit a message onto the queue. This message will have type/topic of `order_created`. Another service such as the `Product` service will be subscribed to this topic and will consume the `order_created` message and subsequently get the details from this message and update its relevant internal schema accordingy. [color=#FD7E14]

### Message Format

The message format will play a crucial role in our application. A message in our system is fundamentally a `JSON` object with the following properties -
- **Message type/topic**: This will contain an identifier for the type/topic of event the message is corresponds to.
- **Payload**: The date contained within a message
- User details if message is of type `user_created`.
- Order details if message is of type `order_created`, etc.

Similarly, different messages will be defined for different events that can can be emitted by the services.

#### Sample `Message` emitted by the `Users` Service

{
    topic: 'user_login'
    payload: {
        user_id: 'user_XXXX',
        email: '[e-mail]',
        school: {
            school_id: 'XXXXXXX',
            school_name: 'ABC UNI'
        }
    }
}

Any services subscribed to the `user_login` topic will receive and process this message and update/fetch data accordingly.

#### Sample `Message` emitted by the `Orders` Service

{
    topic: 'order_created',
    payload: {
        order_id: 'XXXXXXXX',
        created_at: '12-17-2022',
        order_total: 150,
        order_curency: '$',
        buyer_id: 'user_ABCD',
        products: [
            { 
                product_id: 'XYXYXYX',
                sold_by: 'user_XXXX'
            },
            { 
                product_id: 'YYYYXXXX',
                sold_by: 'user_YYYY'
            },
        ]
    }
}

Any services listening for the `order_created` topic can process the above message.

> ***Example*** - The `Product` service can subscribe to `order_created` topic and process these type of messages and update the product inventory and mark the appropriate product as **`SOLD`**. [color=#FD7E14]

:::warning
:pushpin: &nbsp; The application is going to be comprised of numerous unique message types. At this point in time it is too premature to list out all possible message types. We will create message types as and when we define the scope of each service.
:::

## Roadmap

This section outlines our *implementation plan* for this project.

### Project Management

We plan to use [**`Agile Kanban`**](https://www.atlassian.com/agile/kanban) to manage the project.
We will set up a task board as follows -

Backlog | TODO | In Progress | Completed | Blocked | Bugs | Ready
-|-|-|-|-|-|-|

A requirement/feature will start out in the backlog and we will then progress it through the different stages as shown above. The final stage is `Ready` wherethe task is fully tested and ready to be deployed and used.

Tasks will be tagged with priority **`1-5`**. **`1`** being highest and **`5`** being lowest. Tasks will also be tagged with the developer's name.

We will have an internal priority list for the services and implement them in the descending order of priority. The overall tasks can be loosely described as follows in the backlog.

We will also sync up twice a week to check up on project progress. We would also would be using `Git` to track all branches and all the Team's commits.

### Project Backlog

The following table is a crude list of our backlog. More items would be added and/or deleted as we finetune the scope during development.

# | Task
-----|----- |
1| Requirements Elicitation
2| Design & Mockups
4| User Service
5| Billboard Service
6| Event Service
7| Clubs Service
8| Products Service
9| Orders Service
10| Ads Services
11 | Message Queue Service
12| E2E Testing
13 | Deployment

### Future Scope

- Better user onboarding with App intro and walkthroughs.
- Integrate Stripe/Venmo payments for easy checkout.
- Write automated tests for regression.
- Make the entire system mobile-first. Build a mobile app.
- More robust Admin features with more granular control and an exhaustive dashboard for school data management.
- SSO with School Credentials and Auth Providers.
- Event RSVPs with add to calendar and Ticket purchasing system.
- Integrate the Official Campus store with Banter for a one-stop-shop to pick up school merchandise.
- Improve scalability and performance.
- Derive insights from user data from Phase 1 to improve UX in phase 2.
- Put in a group chat/personal chat feature.
- Put in AdSense for advertisement & monetization.

:::info
:computer: &nbsp; These are our just the stretch goals. If possible,this is what we would try to incorporate once we develop all the primary goals outlined in the backlog.
:::

## Test Plan

This section outlines all the details of our testing strategies along with other details.

### Challenges

- Changes to one service might break API compatability.
- Extra checks to make sure services work together.
- Bigger testable surface since every service exposes multiple endpoints based on its function.
- Polyglot system since services will be in different languages generally, but we are trying to keep it contained to `Python` only.

### Strategy

We will employ 3 phases for the testing of all services. They are outlined below -

#### Unit Testing

- Extensive tests will be run on each and every container to check the functionality of the code.
- We will use `PyTest` to run the predefined test cases per functionality and put in place a simple `CI piplene` to run tests on code push.

#### Integration Testing

- When services are combined, things can break due to API compatibility or crash of services. In that case Integration tests are important.
- We will test message emmission onto the `Message Queue` and proper consumption by interested Services.
- We will design and execute lot of test cases to cover all the cases including the edge cases.

#### E2E Testing

Test complete user flow in the folowing cases -

- Login `Student` -> Select `Product` -> Buy `Product` -> Check `Orders`
- Login `School Admin` -> Create `Ad` -> Login different `Student` -> See `Ad`
- Login `Student` -> Create `Club` -> Login different `Student` -> Join `Club`

> More test cases might be added at a later point. [color=#FD7E14]

### Test Requirements

- As `Docker` containers are used to develop the code, testing will be done using the same containers in the same environment in the interest of time.
- Testing will be done using the `PyTest` framework so code can be thoroughly tested automatically.

### Scope

- Tests related to creating and joining `Clubs`, creating `Posts` on `Newsfeed`.
- Testing of School `Admin features`, tests related to creating, and deleting ads, as well as the functionality of the basic user.
- Testing the components of `Super Admin` dashboard which will be populated with real time data & statistics.
- Testing of `Order` creation process, starting from adding to `Cart` to the order placement.
- Testing of some/all the `Messeges` in the `Message Queue`.
- Testing of `Product` management.

### Risks/Assumptions

We are making some key assumptions and outlining some risks in this section. They are -

#### Risks

- Team not fully experienced with developing microservices. Lot of learning involved.
- Scope variation. Our scope is currently elastic. We will only decrease the scope as needed but never increase it.
- Team communication and conflicting schedules.
- Technology issues. We might face issues when adopting new technologies that we haven't used before. Will incur some overhead.

#### Assumptions

- Team is sufficiently equipped to take on and handle a project of this scale.
- We will deliver everything outlined in this document.
- Project scope will remain same or decrease.
- Testing covereage will be atleast `75%`.
- For the project to go without a problem, the system & all its dependencies must be interoperable, reliable, and stable.

## Conclusion

The primary goal of this project is to first and foremost familiarize ourselves with the Microservices architecture and to decompose a monolith application into several microservices. Secondly this app fills a need in the school eccommerce space by providing a safe and fun way for students to interact and barter their school products. Thirdly it incorporates the complete Tech Stack from the Front End UI design to the back end framework and services design. This project will cement and augment our current experience in building applications with the latest frameworks and tech.

## Author

Aneesh Melkot |
:-----:|