---
# System prepended metadata

title: HACK 75

---

# HACK 75

## Info
- Brach: https://netskope.atlassian.net/browse/HACK-75
- Folder: https://github.com/ns-andylee/ministack-ui

#### Ministack IP
- Add to `/etc/host`: `10.136.215.193 andylee.docker.local`
- Supervisor config: `/etc/supervisor/conf.d/`

## Wish List
* Logs viewer (show containers and processes, `tail -f`)
    * Be able to grep logs across multiple containers and processes
    * [TODO] Convert log timestamp to browser local time when displaying
* Ministack dashboard
    * container status
    * redis status
* [TODO] Parse Netskope-Qs-Url header and automatically find the corresponding log file
* [TODO] Watch certain msg patterns in logs and apply certain actions automatically (i.e. automatically restarting apicuisvc when seeing DANGER] async queue is full !!! in UI log

---

## Features
- Dashboard
    - container status
    - redis status
- Logs viewer

## Dependencies / Stacks
* React & React Bootstrap
* Nodejs server
* https://www.npmjs.com/package/node-ssh 
* Use websocket 
    * Package: https://www.npmjs.com/package/react-use-websocket
    * Sample codes: https://github.com/tigoe/websocket-examples
* Have one npm project and install everything there

## Architecture
- React frontend
- Node.js
    - Express

## Interface

### Logs Viewer

#### Client (web socket)

- init
```js
{
  action: "init",
  data: {
    host: "<host>"
  }
}
```


- watch log
```js
{
  action: "watch_log",
  data: {
      container: "<docker container name>",
      [x] svconfig: "<sv config>",
      process: "<process_name>"
  }
}
```
- stop watch log
```js
{
  action: "stop_watch_log",
  data: {
      container: "<docker container name>",
      [x] svconfig: "<sv config>",
      process: "<process_name>"
  }
}
```

#### Server (web socket)

> URL: GET localhost:3000/ws/

- send logs
```js
{
  action: "log",
  data: {
    container: "container",
    svconfig: "svconfig",
    process: "process"
    lines: ["", ""]
  }
}
```

#### REST

##### Logs Viewer

- GET `/ministacks/<host>/logs/containers`
    - return `{ data: ["container1", "container2", ...] }`
- GET `/ministacks/<host>/logs/containers/<container>/svconfigs`
    - return `{ data: ["config1", "config2", ...] }`

##### Dashboard

- GET `/ministacks/<host>/dashboard/containers_status`
    - return `{ data: [{ container: "", status: "" }, ...]}`
        - status: `"ok"` | `"error"`
- GET `/ministacks/<host>/dashboard/redis_status`
    - return `{ data: [{ container: "redis5", status: "", memory_usage: "80" }, ...]}`
        - status: `"ok"` | `"error"`


##### Process viewer

- GET `/ministacks/<host>/processes/<container>`
    - return `{data: [{process: '', status: '', uptime: ''}, ...]`
    - [
  {
    process: 'apiconnector:ui00',
    status: 'RUNNING',
    uptime: '2:08:30'
  },
  {
    process: 'apiprofiles:apiprofiles00',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  },
  {
    process: 'apiservice:apiservice00',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  },
  {
    process: 'cfgagentv2',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  },
  {
    process: 'iaas-statussvc:iaas-statussvc00',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  },
  {
    process: 'incident-wkr',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  },
  {
    process: 'mongos',
    status: 'RUNNING',
    uptime: '6 days, 20:31:04'
  }
]

- POST `/ministacks/<host>/processes/<container>/restart/<process_name>`
    - request body: none
    - return `{data: {process: '', status: '', uptime: ''}`




##### Container page

- GET `/ministacks/<host>/containers`
- GET `/ministacks/<host>/containers/<container>/processes/`
- POST `/ministacks/<host>/containers/<container>/processes/<process>/restart`





---

## Implementation Note

- `tail -f simulation`
- `setInterval`
    * wc -l filename, compare with line number of last time
    * sed -n -e "320,330 p" -e "330 q" ui-*.log
    * https://stackoverflow.com/questions/3137094/how-to-count-lines-in-a-document 

---

## TODO
- Feat: Logs: support multiple tabs 
- Feat: Logs: better log viewer UI
- [sharon] Bug: Logs context and send multiple times problem
- [shirley] Feat: navigate by URL, Bug: cannot watch log (watch_log action is not sent)


New features
- Feat: Full log viewer (including old rotation files)

- Monitor Docker containers
    - Supervisord process
    - 
    - redis?


- View .ns package version
- process status / restart button
- log UI: display process name after clicking config name

---

## Demo

> - Requirement: upload demo video
> - Hackathon theme: Improve Quality and Fix Things
> - https://sites.google.com/netskope.com/hackathon2021/how-it-works

### Outline

- Motivation
- What we do: a trouble shooting tool for ministack
- How is it related to the theme?
    - save time on touble shooting -> have more time to invest in quality?
- Introduction to the tool's feature and demo
- Conclusion & Future work
    - Customize Dashboard
    - Figure out how to do interactive commands properly (tail -f)
    - 
    - Better error handling, UI loading status indicator

- Known issue
    - ?


- Future TODO
    - Log: convert timestamp to local browser time
    - Support monitoring redis status and management (i.e. flush redis memory when mem usage is too high)
    - Support processes not managed by supervisord, such as php in webui, mysql, mongodb
    - Architecture: move this web server into Ministack


