---
# System prepended metadata

title: Serverless Analytic Report Script
tags: [' feature', metrics]

---

---
tags: metrics, feature
author: Oliver.W & Meng Zong  
---

# Serverless Analytic Report Script

[![hackmd-github-sync-badge](https://hackmd.io/OFRKg3OVQhag-Go6BfSVOQ/badge)](https://hackmd.io/OFRKg3OVQhag-Go6BfSVOQ)


> Task: [🧩 Event based analytics report](https://app.asana.com/0/1200011502754281/1200025989019536)  
> Site Visit data: [Tencent MTA Board](https://mta.qq.com/mta/overview/ctr_all_app_new)  
> Report Sheet: [Weekly Analytics Report](https://docs.google.com/spreadsheets/d/1QtExHdlvIenoVDxyFLWWH_5pMiuWIK0piAhiy75QXko/edit?usp=sharing)  
> Project repo: [Analytics report](https://github.com/serverlessinc/Analytics-report)

## Goal

As a Product Manager,  
I want to see useful metrics data and analytics reports based on weekly and monthly,  
So that I can verify if a product direction is correct and how users respond to the latest change.

## Design

This is the design of metrics that need to implemented for serverless analytics.

---
Analytics on user info.
- [x] Total Users：Total users that use serverless
- [x] Active Users: The users that do deploy durint the period
- [x] New Users：New users during the period
- [ ] New Active Users：New users that did deploy durint the period

---
Analytics on API Calls.
- [ ] API-Init times: Time of call init API.
    - [ ] API-Init times - Tencent: Time of call init API from tencent.
    - [ ] API-Init times - CLI: Time of call init API from CLI.
> Init info not exist on platform engine
- [ ] API-Dev times: Time of call dev API.
> Dev info not exist on platform engine
- [x] API-Deploy times: Time of call deploy API.
    - [ ] API-Deploy times - Tencent: Time of call deploy API from tencent.
    - [ ] API-Deploy times - CLI: Time of call deploy API from CLI.
- [x] API-Remove times: Time of call remove api.
    - [ ] API-Remove times - Tencent: Time of call remvoe API form tencent.
    - [ ] API-Deploy times - CLI: Time of call remove API from CLI
> Need a way to tell request from Tencent or Users' CLI, Tencent also use CLI behind the scene.  
> Consider use the IP to tell if is from Tencent.

---
Analytics on serverless components usage.
- [x] Total Components: Total deployed components.
    - [x] Total Components - Active: Total active compnents. 
    - [x] Total Components - InActive: Total inactive components.
    - [x] Total Components - Error: Total Error components.
- [x] New Components: New deployed components during the period.
- [x] Active Components: Deployed components during the period.
- [ ] Active Components 2+: Deployed 2+ times component during the period.
- [ ] Active Components 8+: Deployed 8+ times component during the period.
> Can't tell how may time deployed in a period from engine, the only info have is `lastDeployedAt`
---
Analytics on serverless function usage.
- [x] Total SLS Runtimes:  Deployed serverless functoin services in rumtiles.
- [x] Active SLS Runtimes:  Deployed serverless functoin services in rumtiles during the period.
---
Analytics on site visitors.
- [x] CNsite Visitor: Visitor of the serverless CN site.
- [x] Console Visitor: Visitor of the Tencent Serverless Console.
---
Funnel report
- [ ] New Users: Weekly new download users not deployed or fist deploy in this week.
- [ ] Init users: Weekly new download users execute init commands.
- [ ] Deploy Users: Weekly new download users execute deploy commands.
- [ ] Active User: Weekly new download users complete a success deployment. 
- [ ] 3+ deploy users: Weekly new download users complete 3+ deployment.
- [ ] Deploy after a week. Weekly new download users complete deploy in the next week

## Data Source

| Metrics               | Source            | Note                                                                        |
|:--------------------- |:----------------- |:--------------------------------------------------------------------------- |
| Total Users           | Serverless Engine | Count total users in Engine                                                 |
| Active Users          | Serverless Engine | Count unique users that deploye a componet in the period                    |
| New Users             | Cacultated        | Based on previous weekly total User.                                        |
| Registered Components | Serverless Engine | Count total amount of components(not instances).                            |
| Registered Templates  | Serverless Engine | Count total amount of templates(not instances).                             |
| Components-active     | Serverless Engine | Count total amount of component instances that in Active status.            |
| Components-inActive   | Serverless Engine | Count total amount of component instances that in InActive status.          |
| Components-error      | Serverless Engine | Count total amount of component instances that in error status.             |
| New Components        | Cacultated        | Based on previous weekly total components                                   |
| Active Components     | Serverless Engine | Components(not instance) that deployed in the period. Based on `deployedAt` |
| API-Removes           | Serverless Engine | Count total calls of deploy API                                             |
| API-Deploy            | Serverless Engine | Count total calls of deploy API.                                            |
| Total Components      | Cacultated        | Count Sum of Total-{name}                                                   |
| Active Components     | Cacultated        | Count Sum of Active-{name}                                                  |
| Total-{name}          | Serverless Engine | Total instances (active status only) in Engine in the filter list           |
| Active-{name}         | Serverless Engine | Active instances that deployed in the period in the filter list             |
| Total SLS             | Cacultated        | Count Sum of Total-SLS-{runtime}                                            |
| Active SLS            | Cacultated        | Count Sum of Active-SLS-{runtime}                                           |
| Total-SLS-{runtime}   | Serverless Engine | Total Runtimes (active status only) in Engine                               |
| Active-SLS-{runtime}  | Serverless Engine | Active Runtimes that deployed in the period                                 |
| Funnel-New            | Serverless Engine | New users(first do deploy) include anonymous users in the period                             |
| Funnel-Init           | Serverless Engine | Users do init from New users in the period                                  |
| Funnel-Deploy         | Serverless Engine | Users do deploy from init in the period                                     |
| Funnel-Active         | Serverless Engine | Users complete deploy from deploy in the period                             |
| Funnel-3+Active       | Serverless Engine | Users complete 3+ deploy from deploy in the period                          |
| Funnel-Retain         | Serverless Engine | Users complete a deploy in the next period.                                 |
| Init Counts           | Serverless Engine | Total init commands counts in the period                                    |
| Deploy Counts         | Serverless Engine | Total deploy commands counts in the period                                  |
| CNsite Visitor        | Tencent MTA       | Sum daily UV as Period UV                                                   |
| Console Visitor       | Tencent MTA       | Sum daily UV as Period UV                                                   |

### Component list (Only caculate these components in `Total-{name}` and `Active-{name}`)

> Need to update the list when there have a new componets to analytics.

There are some components registered in prod that for test

* website: react-start will be count as website.
* express
* scf
* multi-scf
* http
* nextjs
* egg
* koa
* nestjs
* nuxtjs
* postgresql
* springboot
* laravel
* flask
* django
* wordpress
* discuz-q
* thinkphp
* websocket
* mongodb
* cynosdb

### Runtime List

* Nodejs10.15
* Nodejs12.16
* Python2.7
* Python3.6
* PHP5
* PHP7
* Java8
* Go1
* CustomRuntime

## Dev

* Will use Serverless to generate CSV file with the raw data above. 
* The script will execute auto on every Monday go generate report on last week.
* The script can be called manually and generate date on a specific data range.
* Oliver Will inport the raw data to Google sheet to generate the report.

## Functionality
### SCF
1. SCF Instance-Dev: [cn-analytics-report](https://console.cloud.tencent.com/scf/list-detail?rid=4&ns=default&id=cn-analytics-report&tab=codeTab)
2. SCF Instance-Prod: [cn-analytics-report](https://console.cloud.tencent.com/scf/list-detail?rid=1&ns=default&id=cn-analytics-report&tab=codeTab)

### Run

#### Cron Job
1. The scf CRON task will run automatically at each Monday 4am

#### Manual Execute
2. You can also run it manually, you need to pass `begin time` and `end time`, just like: ![](https://i.imgur.com/vasjwZ8.png)
3. You can add a field `extra` in the payload to generate a app-info doc. @ole3021 
```
{
  "begin": "2021-03-15 00:00:00",
  "end": "2021-03-21 23:59:59",
  "extra": true
}
```

#### create user profiles for `mixpanel`
- ticket: https://app.asana.com/0/1200011502754281/1200832744926110/f
- You can run it manually to create user profile for mixpanel project

```json=
{
    "mixpanelUserProfile": true
}
```

#### Clean up old data for analytics
- ticket: https://app.asana.com/0/1200011502754281/1200835434749891/f
- Add the param when run script manually
```json=
{
  clean: true
}
```


## Usage

- Download `dev result csv` from [COS](https://console.cloud.tencent.com/cos5/bucket/setting?type=filelist&bucketName=analytics-report-1300963013&projectId=0&path=%252F&region=ap-shanghai)
    > Dev Data:  report-dev.csv   
    > Prod Data:  report-prod.csv 

## Notes

* China Serverless Platform not use envent service to collect the behaive info. (Only used for sync the debug info from engine.)