# Banter - Midpoint Evaluation


This document highlights our project in its current state. We have given a brief of all the work completed so far and mentioned the upcoming action items. We recommend you read this doc [here](https://hackmd.io/@aneeshmelkot/HJn4cNDNi) and not as a PDF.

## Contents

[TOC]

## Context

**Banter** is a social network for students. It is a fullstack application that can be retro-fitted into any school's IT infrastructure to enhance the student experience.

Our application is a fully decompoed, loosly coupled and highly cohesive microservice architecture where every service is independent, isolated and individually upgradable, maintainable and deployable.

### Architecture


### Project Structure

Here is the project structure of our application. All services are build using Python Flask and follow a structure as seen for the `Billboard` directory below. Every service also has a `tests` directory that houses some unit tests for that service. All other services would more or less have the same structure. The `Client` is built using React and its directory is as such.
The infra directory contains the k8s deployments and services defined for all the microservices in our application. Currently you can see only **3** depl files in this directory.

.
└── banter_micro/
    ├── advertisement/
    ├── billboard/
    │   ├── static/
    │   ├── tests/
    │   ├── .dockerignore
    │   ├── Dockerfile
    │   ├── app.py
    │   └── models.py
    ├── client/
    │   ├── node_modules/
    │   ├── public/
    │   ├── src/
    │   ├── .gitignore
    │   ├── package.json
    │   └── README.md
    ├── clubs/
    ├── event_bus/
    ├── events/
    ├── infra/
    │   └── k8s/
    │       ├── billboard.yaml
    │       ├── user.yaml
    │       └── events.yaml
    ├── shop/
    └── user/

### Services

As mentioned in our initial proposal, our services have the following design. We have paired up every service with its own highly available database. Every service is storing its data only in its coupled DB. Every service is typically be 2-3 containers structured as follows -

- **1** Service
- **1** Database (`Maria DB` or `Mongo DB`)
- **1** Database Manager (`Adminer` or `Mongo Express`)

> *Database Manager* is optional and we have attached other containers as well for monitoring and logging on a need basis. (**`Sidecar Pattern`**) [color=#FD7E14]

:::warning
:pushpin:   **Services will fetch/receive data from other services through network requests only or by subscribing to messages from the message queue.**
:::

:::danger
:warning:   **A service will never read/query from another service's database directly.**
:::


### Docker

This is the `Dockerfile` for the service container creation. All services would have a slightly modified version of this Dockerfile.

`Dockerfile`

FROM python:3.10.7-slim-buster

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt

COPY . .

CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]

For local multi container testing, we have put in this `docker-compose.yaml` for all the services with respective service names -

`docker-compose.yaml`

name: banter-billboard
services:
  web:
    build: .
    ports:
      - 5050:5000
  db:
    image: mariadb:10.9.2
    restart: always
    ports:
      - 3306:3306
    environment:
      - MARIADB_ROOT_PASSWORD=${MARIADB_ROOT_PASSWORD}
      - MARIADB_DATABASE=${MARIADB_DATABASE}
    volumes:
      - banter-db-data:/var/lib/mysql
  adminer:
    image: adminer
    restart: always
    ports:
      - 8085:8080
volumes:
  banter-db-data:
    driver: local

The above compose fill will containerize the Service along with the DB and a service called Adminer for DB management. Similar compose file in place for other completed services.

### Kubernetes

We are using the following k8s types.

- Deployment
- Service
- PersistentVolume
- Secret

> The following YAMLs that you see are for **1 service** in our application.

This is the `yaml` for `Deployments` and `Services` of one of our Microservices. All other services' yamls look similar but with respective service names instead. It is all composed in 1 yaml file `<service_name.yaml>` with `---` as the delimeter.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  labels:
    app: banter-billboard
spec:
  replicas: 3
  selector:
    matchLabels:
      app: banter-billboard
  template:
    metadata:
      labels:
        app: banter-billboard
    spec:
      containers:
        - name: banter-billboard
          image: docker.io/library/banter-billboard
          imagePullPolicy: Never
          ports:
            - containerPort: 5000
          env:
            - name: DATABASE_PATH
              value: "mysql+pymysql://root:admin@banter-billboard-db-svc/banter"
            - name: SECRET_KEY
              value: "secret"
---
apiVersion: v1
kind: Service
metadata:
  name: banter-billboard-svc
spec:
  ports:
  - port: 5000
    protocol: TCP
    targetPort: 5000
  selector:
    app: banter
  type: LoadBalancer

The following is the yaml file for the service's DB deployment. All services have a similar deployment file for the DB as every service has a DB attached.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: banter-billboard-db-deployment
  labels:
    app: banter-billboard-db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: banter-billboard-db
  template:
    metadata:
      labels:
        app: banter-billboard-db
    spec:
      containers:
        - name: maria-db
          image: mariadb
          imagePullPolicy: Always
          env:
            - name: MARIADB_ROOT_PASSWORD
              valueFrom:
                 secretKeyRef:
                   name: mariadb-credentials
                   key: MARIADB_ROOT_PASSWORD
            - name: MARIADB_DATABASE
              value: banter
          ports:
            - containerPort: 3306
              name: db-container
          volumeMounts:
              - name: maria-persistent-storage
                mountPath: /var/lib/mysql
        - name: adminer
          image: adminer
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
      volumes:
        - name: maria-persistent-storage
          persistentVolumeClaim:
            claimName: maria-pv-claim
---
apiVersion: v1
kind: Service
metadata:
  name: banter-billboard-db-svc
  labels:
    app: banter-billboard-db
spec:
  ports:
  - port: 3306
    protocol: TCP
    name: maria-db
  selector:
    app: maria-db
  type: NodePort

All deployments are ephemeral. Hence, all Databases will need a `PersistentVolume` to retain the date. Here we have defined a persistent volume storage in K8s

apiVersion: v1
kind: PersistentVolume
metadata:
  name: maria-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: maria-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Lastly all secrets in k8s are stored in a `Secret` as follows. Here the value itself is Base64 encoded. This is a Secret file for one of the service's DB Root Password.

apiVersion: v1
kind: Secret
metadata:
  name: mariadb-credentials
type: Opaque
data:
  MARIADB_ROOT_PASSWORD: YWRtaW4=

## Completed Services

Here, we have outlined all the work that we have completed so far.

### Client Service/ UI

Built using React.

#### Login/Register



#### Home Page


<br>

#### Billboard/ Newsfeed Page


#### Events Page


### User Service

This service was implemented to handle all the user's data.
The service has the following schema and attributes -


#### USER

Column | Type | Comment
:---|:---:|:---:
**id** | varchar(20) | PK (User's ID)
name |varchar(30) | Full Name
password| varchar(255) | Hashed and stored
bio |varchar(1000) | Small self description
dob | date | Date of birth
major | varchar(255)| Field of study
phone | varchar(15) | Phone Number
dp | varchar(255)| URL for profile picture to be stored using CDN
school_id | int(11) | FK - School ID

#### SCHOOL
Column | Type | Comment
:---|:---:|:---:
**id** | int | PK (School ID) Auto Increment
name | varchar(255) | School Name
desc | varchar(1000) | Short description
state | varchar (255) | State in which school is situated
city | varchar(255) | City in which school is situated
country | varchar (255) | Country in which school is situated

#### ROLE
Column | Type | Comment
:---|:---:|:---:
id | int | PK role id
name | varchar(255) | Role name
desc | varchar(255) | Role short desc

#### USER_ROLES
Column | Type | Comment
:---|:---:|:---:
id | int | PK Auto Increment
user_id | varchar(20) | FK User ID
role_id | int | FK Role ID

Furthermore, the User service has endpoints for -
- Creating/Fetching `User`
- Creating/Fetching `School`
- Assigning/Fetching `User` <-> `User Role`

#### Message Formats / API Responses

The following are the topics of messages emitted by the **User** service. These are subject to change. The API response of this service also would look similar when other services query it.

##### User Created

{
    "topic": "user_created",
    "payload": {
        "user_id": "XXXX",
        "name": "Aneesh",
        "school_id": 1235,
        "phone": "12334566",
        "major": "CS",
        "dp" : "http://dp.uri.com/XXXX",
    }
}

##### User Logged In

{
    "topic": "user_logged_in",
    "payload": {
        "user_id": "XXXX",
        "name": "Aneesh",
        "school_id": 1235,
        "phone": "12334566",
        "major": "CS",
        "dp" : "http://dp.uri.com/XXXX",
    }
}

##### School Created

{
    "topic": "school_created",
    "payload": {
        "school_id": "XXXX",
        "name": "UTA",
        "desc": "Awesome",
        "state": "TX",
        "city": "Arlington",
        "country" : "USA",
    }
}

### Billboard Service

The Billboard service handles the newsfeed and posts.


#### Billboard
Column | Type | Comment
:---|:---:|:---:
id | int(11) | PK Auto Increment
title | varchar(255) | Post title
content | varchar(5000) | Long post content
school_id | int(11) | School ID
img_url | varchar(255) | Post image (Optional)
student_id | varchar(20) | Post author
cat_id | int(11) | FK Post Category ID
created_at | datetime NULL | Post timestamp

#### Category
Column | Type | Comment
:---|:---:|:---:
id | int(11) | PK Auto Increment
name | varchar(255) | Category Name
desc | varchar(255) | Short description

#### Message Formats / API Responses

The following are the topics of messages emitted by the **Billboard** service. These are subject to change. The API response of this service also would look similar when other services query it.

##### Post Created

{
    "topic": "billboard_post_created",
    "payload" : {
        "id": 123,
        "title": "Cool Post",
        "content": "This is a cool post.",
        "school_id": "12345",
        "url": "http://url.post.image/123",
        "student_id": "XXXXX",
        "cat_id": 2,
        "created_at": 2022-10-15 14:35:50 
    }
}

##### Category Created

{
    "topic": "billboard_category_created",
    "payload": {
        "id": 5,
        "name": "Sports",
        "desc": "This is the sports category."
    }
}

### Events Service

The Events service handles the events in a school.


Column | Type | Comment
:---|:---:|:---:
id | int(11) | PK Auto Increment |
name | varchar(50) | Event Name
desc | varchar(255) | Short Event description
date_time | datetime | Event date and time
location | varchar(50) | Event location
school_id | int(11) | School ID
student_id | varchar(255) | Event creator

#### Message Formats / API Responses

The following are the topics of messages emitted by the **Events** service. These are subject to change. The API response of this service also would look similar when other services query it.

{
    "topic": "event_created",
    "payload": {
        "id": 1234,
        "name": "Haloween Party",
        "desc": "Come to the haloween party. It's gonna be dope",
        "date_time": 2022-10-30 19:00:00,
        "location": "Brazzos Park",
        "school_id": 12345,
        "student_id": "XXXXX"
    }
}

## Testing

### Unit Testing

- Tests have been executed on each and every container to check the functionality of the code.
- We have used `PyTest` to run the predefined test cases per functionality and put in place a simple `CI piplene` to run tests on code push.

At its current state, the services are running on our machines in a local kubernetes cluster using Minikube. The completed services are working in isolation and some rudimentary unit tests have been written to ensure quality.

:::info
Integration and E2E tests will be put in place once the Event Bus is complete.
:::

## Action Items

- **Event Bus**
This service will be our next priority as it will form the backbone of the application.
- Clubs Service
- Products Service
- Order Service
- CDN Service

## Conclusion
We are on track to complete the remaining services. We **might** scrap the ecommerce aspect of the project due to complexity or as an alternative, deliver the storefront as a **Monolith**.

## Author

Aneesh Melkot|
:-----: |