---
# System prepended metadata

title: Collecting Statistics

---

# Collecting Statistics

[Wiki page](https://github.com/iiasa/ene/wiki/KPI)

## Stats were provided like this so far:
Statistics were downloaded from awstats
    - e.g. http://awstats.jump5.iiasa.ac.at/awstats/awstats.pl?urlfilter=%2Fengage-api%2F&urlfilterex=%2Fengage-api%2Fmetrics&output=urldetail&config=db1.ene.iiasa.ac.at&year=2019&month=all&framename=mainright
The downloaded files included various individual pages for specific URLs/instances (filtering by servlet path), downloads, ...

Then, analyzing the downloaded pages happened via bash/python scripts by Peter. The scripts will be provided by Peter in a repository - undocumented, but for future reference on what and how things were collected. E.g. To get number of unique visitors, the numberf from the backend (db1) and frontend (data.ene) were combined (added).


## What was collected from scenario databases

Information provided by [awstats](http://awstats.foix2.iiasa.ac.at/awstats/awstats.pl?config=data.ene.iiasa.ac.at):

- visits
- unique visitors
- pages
- page hits
- bandwidth
- downloads (by file, total size)


## What was collected from Scenario Explorer/IXMP backends

- registered user logins
- anonymous logins
- new user accounts
- total downloads and per file

## TODOs
- omit metrics from access logs
    - currently this has to be done manually (filtering URLs); metrics shouldn't show up in logs at all though, see the "metrics" requirements

## Central place for cleaned+combined statistics
We could automate the scripts that analyze the awstats information, and collect this information (and graphs) on a central place.
- unique visitors etc. combined for db1 and data.ene
- which (timeseries) DB to use to store statistics?
- how to transfer statistics from log files to timeseries DB?
- can we make print-ready charts from Grafana dashboard?