---
tags: metrics, feature
author: Oliver.W & Meng Zong
---

# Serverless Analytic Report Script

[](https://hackmd.io/OFRKg3OVQhag-Go6BfSVOQ)

> Task: [🧩 Event based analytics report](https://app.asana.com/0/1200011502754281/1200025989019536)
> Site Visit data: [Tencent MTA Board](https://mta.qq.com/mta/overview/ctr_all_app_new)
> Report Sheet: [Weekly Analytics Report](https://docs.google.com/spreadsheets/d/1QtExHdlvIenoVDxyFLWWH_5pMiuWIK0piAhiy75QXko/edit?usp=sharing)
> Project repo: [Analytics report](https://github.com/serverlessinc/Analytics-report)

## Goal

As a Product Manager,
I want to see useful metrics data and analytics reports based on weekly and monthly,
So that I can verify if a product direction is correct and how users respond to the latest change.

## Design

This is the design of metrics that need to implemented for serverless analytics.

---
Analytics on user info.
- [x] Total Users：Total users that use serverless
- [x] Active Users: The users that do deploy durint the period
- [x] New Users：New users during the period
- [ ] New Active Users：New users that did deploy durint the period

---
Analytics on API Calls.
- [ ] API-Init times: Time of call init API.
- [ ] API-Init times - Tencent: Time of call init API from tencent.
- [ ] API-Init times - CLI: Time of call init API from CLI.
> Init info not exist on platform engine
- [ ] API-Dev times: Time of call dev API.
> Dev info not exist on platform engine
- [x] API-Deploy times: Time of call deploy API.
- [ ] API-Deploy times - Tencent: Time of call deploy API from tencent.
- [ ] API-Deploy times - CLI: Time of call deploy API from CLI.
- [x] API-Remove times: Time of call remove api.
- [ ] API-Remove times - Tencent: Time of call remvoe API form tencent.
- [ ] API-Deploy times - CLI: Time of call remove API from CLI
> Need a way to tell request from Tencent or Users' CLI, Tencent also use CLI behind the scene.
> Consider use the IP to tell if is from Tencent.

---
Analytics on serverless components usage.
- [x] Total Components: Total deployed components.
- [x] Total Components - Active: Total active compnents.
- [x] Total Components - InActive: Total inactive components.
- [x] Total Components - Error: Total Error components.
- [x] New Components: New deployed components during the period.
- [x] Active Components: Deployed components during the period.
- [ ] Active Components 2+: Deployed 2+ times component during the period.
- [ ] Active Components 8+: Deployed 8+ times component during the period.
> Can't tell how may time deployed in a period from engine, the only info have is `lastDeployedAt`
---
Analytics on serverless function usage.
- [x] Total SLS Runtimes: Deployed serverless functoin services in rumtiles.
- [x] Active SLS Runtimes: Deployed serverless functoin services in rumtiles during the period.
---
Analytics on site visitors.
- [x] CNsite Visitor: Visitor of the serverless CN site.
- [x] Console Visitor: Visitor of the Tencent Serverless Console.
---
Funnel report
- [ ] New Users: Weekly new download users not deployed or fist deploy in this week.
- [ ] Init users: Weekly new download users execute init commands.
- [ ] Deploy Users: Weekly new download users execute deploy commands.
- [ ] Active User: Weekly new download users complete a success deployment.
- [ ] 3+ deploy users: Weekly new download users complete 3+ deployment.
- [ ] Deploy after a week. Weekly new download users complete deploy in the next week

## Data Source

| Metrics | Source | Note |
|:--------------------- |:----------------- |:--------------------------------------------------------------------------- |
| Total Users | Serverless Engine | Count total users in Engine |
| Active Users | Serverless Engine | Count unique users that deploye a componet in the period |
| New Users | Cacultated | Based on previous weekly total User. |
| Registered Components | Serverless Engine | Count total amount of components(not instances). |
| Registered Templates | Serverless Engine | Count total amount of templates(not instances). |
| Components-active | Serverless Engine | Count total amount of component instances that in Active status. |
| Components-inActive | Serverless Engine | Count total amount of component instances that in InActive status. |
| Components-error | Serverless Engine | Count total amount of component instances that in error status. |
| New Components | Cacultated | Based on previous weekly total components |
| Active Components | Serverless Engine | Components(not instance) that deployed in the period. Based on `deployedAt` |
| API-Removes | Serverless Engine | Count total calls of deploy API |
| API-Deploy | Serverless Engine | Count total calls of deploy API. |
| Total Components | Cacultated | Count Sum of Total-{name} |
| Active Components | Cacultated | Count Sum of Active-{name} |
| Total-{name} | Serverless Engine | Total instances (active status only) in Engine in the filter list |
| Active-{name} | Serverless Engine | Active instances that deployed in the period in the filter list |
| Total SLS | Cacultated | Count Sum of Total-SLS-{runtime} |
| Active SLS | Cacultated | Count Sum of Active-SLS-{runtime} |
| Total-SLS-{runtime} | Serverless Engine | Total Runtimes (active status only) in Engine |
| Active-SLS-{runtime} | Serverless Engine | Active Runtimes that deployed in the period |
| Funnel-New | Serverless Engine | New users(first do deploy) include anonymous users in the period |
| Funnel-Init | Serverless Engine | Users do init from New users in the period |
| Funnel-Deploy | Serverless Engine | Users do deploy from init in the period |
| Funnel-Active | Serverless Engine | Users complete deploy from deploy in the period |
| Funnel-3+Active | Serverless Engine | Users complete 3+ deploy from deploy in the period |
| Funnel-Retain | Serverless Engine | Users complete a deploy in the next period. |
| Init Counts | Serverless Engine | Total init commands counts in the period |
| Deploy Counts | Serverless Engine | Total deploy commands counts in the period |
| CNsite Visitor | Tencent MTA | Sum daily UV as Period UV |
| Console Visitor | Tencent MTA | Sum daily UV as Period UV |

### Component list (Only caculate these components in `Total-{name}` and `Active-{name}`)

> Need to update the list when there have a new componets to analytics.

There are some components registered in prod that for test

* website: react-start will be count as website.
* express
* scf
* multi-scf
* http
* nextjs
* egg
* koa
* nestjs
* nuxtjs
* postgresql
* springboot
* laravel
* flask
* django
* wordpress
* discuz-q
* thinkphp
* websocket
* mongodb
* cynosdb

### Runtime List

* Nodejs10.15
* Nodejs12.16
* Python2.7
* Python3.6
* PHP5
* PHP7
* Java8
* Go1
* CustomRuntime

## Dev

* Will use Serverless to generate CSV file with the raw data above.
* The script will execute auto on every Monday go generate report on last week.
* The script can be called manually and generate date on a specific data range.
* Oliver Will inport the raw data to Google sheet to generate the report.

## Functionality
### SCF
1. SCF Instance-Dev: [cn-analytics-report](https://console.cloud.tencent.com/scf/list-detail?rid=4&ns=default&id=cn-analytics-report&tab=codeTab)
2. SCF Instance-Prod: [cn-analytics-report](https://console.cloud.tencent.com/scf/list-detail?rid=1&ns=default&id=cn-analytics-report&tab=codeTab)

### Run

#### Cron Job
1. The scf CRON task will run automatically at each Monday 4am

#### Manual Execute
2. You can also run it manually, you need to pass `begin time` and `end time`, just like: 
3. You can add a field `extra` in the payload to generate a app-info doc. @ole3021

{
  "begin": "2021-03-15 00:00:00",
  "end": "2021-03-21 23:59:59",
  "extra": true
}

#### create user profiles for `mixpanel`
- ticket: https://app.asana.com/0/1200011502754281/1200832744926110/f
- You can run it manually to create user profile for mixpanel project

{
    "mixpanelUserProfile": true
}

#### Clean up old data for analytics
- ticket: https://app.asana.com/0/1200011502754281/1200835434749891/f
- Add the param when run script manually

{
  clean: true
}

## Usage

- Download `dev result csv` from [COS](https://console.cloud.tencent.com/cos5/bucket/setting?type=filelist&bucketName=analytics-report-1300963013&projectId=0&path=%252F&region=ap-shanghai)
> Dev Data: report-dev.csv
> Prod Data: report-prod.csv

## Notes

* China Serverless Platform not use envent service to collect the behaive info. (Only used for sync the debug info from engine.)