# HACK 75

## Info
- Brach: https://netskope.atlassian.net/browse/HACK-75
- Folder: https://github.com/ns-andylee/ministack-ui

#### Ministack IP
- Add to `/etc/host`: `10.136.215.193 andylee.docker.local`
- Supervisor config: `/etc/supervisor/conf.d/`

## Wish List
* Logs viewer (show containers and processes, `tail -f`)
* Be able to grep logs across multiple containers and processes
* [TODO] Convert log timestamp to browser local time when displaying
* Ministack dashboard
* container status
* redis status
* [TODO] Parse Netskope-Qs-Url header and automatically find the corresponding log file
* [TODO] Watch certain msg patterns in logs and apply certain actions automatically (i.e. automatically restarting apicuisvc when seeing DANGER] async queue is full !!! in UI log

---

## Features
- Dashboard
- container status
- redis status
- Logs viewer

## Dependencies / Stacks
* React & React Bootstrap
* Nodejs server
* https://www.npmjs.com/package/node-ssh
* Use websocket
* Package: https://www.npmjs.com/package/react-use-websocket
* Sample codes: https://github.com/tigoe/websocket-examples
* Have one npm project and install everything there

## Architecture
- React frontend
- Node.js
- Express

## Interface

### Logs Viewer

#### Client (web socket)

- init

{
  action: "init",
  data: {
    host: "<host>"
  }
}

- watch log

{
  action: "watch_log",
  data: {
      container: "<docker container name>",
      [x] svconfig: "<sv config>",
      process: "<process_name>"
  }
}

- stop watch log

{
  action: "stop_watch_log",
  data: {
      container: "<docker container name>",
      [x] svconfig: "<sv config>",
      process: "<process_name>"
  }
}

#### Server (web socket)

> URL: GET localhost:3000/ws/

- send logs

{
  action: "log",
  data: {
    container: "container",
    svconfig: "svconfig",
    process: "process"
    lines: ["", ""]
  }
}

#### REST

##### Logs Viewer

- GET `/ministacks/<host>/logs/containers`
- return `{ data: ["container1", "container2", ...] }`
- GET `/ministacks/<host>/logs/containers/<container>/svconfigs`
- return `{ data: ["config1", "config2", ...] }`

##### Dashboard

- GET `/ministacks/<host>/dashboard/containers_status`
- return `{ data: [{ container: "", status: "" }, ...]}`
- status: `"ok"` | `"error"`
- GET `/ministacks/<host>/dashboard/redis_status`
- return `{ data: [{ container: "redis5", status: "", memory_usage: "80" }, ...]}`
- status: `"ok"` | `"error"`

##### Process viewer

- GET `/ministacks/<host>/processes/<container>`
- return `{data: [{process: '', status: '', uptime: ''}, ...]`
- [
{
process: 'apiconnector:ui00',
status: 'RUNNING',
uptime: '2:08:30'
},
{
process: 'apiprofiles:apiprofiles00',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
},
{
process: 'apiservice:apiservice00',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
},
{
process: 'cfgagentv2',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
},
{
process: 'iaas-statussvc:iaas-statussvc00',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
},
{
process: 'incident-wkr',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
},
{
process: 'mongos',
status: 'RUNNING',
uptime: '6 days, 20:31:04'
}
]

- POST `/ministacks/<host>/processes/<container>/restart/<process_name>`
- request body: none
- return `{data: {process: '', status: '', uptime: ''}`

##### Container page

- GET `/ministacks/<host>/containers`
- GET `/ministacks/<host>/containers/<container>/processes/`
- POST `/ministacks/<host>/containers/<container>/processes/<process>/restart`

---

## Implementation Note

- `tail -f simulation`
- `setInterval`
* wc -l filename, compare with line number of last time
* sed -n -e "320,330 p" -e "330 q" ui-*.log
* https://stackoverflow.com/questions/3137094/how-to-count-lines-in-a-document

---

## TODO
- Feat: Logs: support multiple tabs
- Feat: Logs: better log viewer UI
- [sharon] Bug: Logs context and send multiple times problem
- [shirley] Feat: navigate by URL, Bug: cannot watch log (watch_log action is not sent)

New features
- Feat: Full log viewer (including old rotation files)

- Monitor Docker containers
- Supervisord process
-
- redis?

- View .ns package version
- process status / restart button
- log UI: display process name after clicking config name

---

## Demo

> - Requirement: upload demo video
> - Hackathon theme: Improve Quality and Fix Things
> - https://sites.google.com/netskope.com/hackathon2021/how-it-works

### Outline

- Motivation
- What we do: a trouble shooting tool for ministack
- How is it related to the theme?
- save time on touble shooting -> have more time to invest in quality?
- Introduction to the tool's feature and demo
- Conclusion & Future work
- Customize Dashboard
- Figure out how to do interactive commands properly (tail -f)
-
- Better error handling, UI loading status indicator

- Known issue
- ?

- Future TODO
- Log: convert timestamp to local browser time
- Support monitoring redis status and management (i.e. flush redis memory when mem usage is too high)
- Support processes not managed by supervisord, such as php in webui, mysql, mongodb
- Architecture: move this web server into Ministack