---
tags: ECK, Kibana, Usage tracking, Monitoring, Elastic
---

# Alkemio reporting setup

## Terms
- Integration - streamlined way to connect your data to the Elastic Stack. They can include Data ingestion, storage, and transformation rules, Configuration options, Pre-built custom dashboards and visualizations, Documentation.

The APM integration receives performance data from your APM agents, validates and processes it, and then transforms the data into Elasticsearch documents. Removing this logic from APM agents help keeps them light, prevents certain security risks, and improves compatibility across the Elastic Stack.
**The Elastic integration runs on Elastic Agent.**
- Elastic Agent - **Elastic integrations are powered by Elastic Agent.** Single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. A single agent makes it easier and faster to deploy monitoring across your infrastructure. Each agent has a single policy you can update to add integrations for new data sources, security protections, and more.
- Elastic Agent policy - collection of inputs and settings that defines the data to be collected by an Elastic Agent. Each Elastic Agent can only be enrolled in a single policy.
- Kibana Fleet UI & Fleet server - enables you to manage Elastic Agent installations.
Fleet Server is a component that connects Elastic Agents to Fleet. It supports many Elastic Agent connections and serves as a control plane for updating agent policies, collecting status information, and coordinating actions across Elastic Agents. It also provides a scalable architecture. As the size of your agent deployment grows, you can deploy additional Fleet Servers to manage the increased workload.
Fleet uses Elastic Agent policies in two ways:
- Policies are stored in a plain-text YAML file and sent to each Elastic Agent to configure its inputs.
- Policies provide a visual representation of an Elastic Agents configuration in the Fleet UI.
- APM agent (real time user monitoring, nodejs agent, python agent, etc.) - Agents of this kind run either in a pod (backend server) or on user's machine (front-end). They can ping the OS, track user action in the browser, track performance.

## Alkemio infrastructure

1. RUM agent is installed on the client machines, via the alkemio client-web application. It is actively sending event data to a configured endpoint behind an Ingress route. This request is load balanced to one of multiple Elastic Agents running under an Elastic Agent policy, in which agent we have the APM integration installed.
2. The Elastic Agent communicates with the Fleet server, that is running under an Elastic Agent in the following way:

1. When a new agent policy is created, the Fleet UI saves the policy to a Fleet index in Elasticsearch.
2. To enroll in the policy, Elastic Agents send a request to Fleet Server, using the enrollment key generated for authentication. 
3. Fleet Server monitors Fleet indices, picks up the new agent policy from Elasticsearch, then ships the policy to all Elastic Agents enrolled in that policy. Fleet Server may also write updated policies to the Fleet index to manage coordination between agents.
4. Elastic Agent uses configuration information in the policy to collect and send data to Elasticsearch.
5. Elastic Agent checks in with Fleet Server for updates, maintaining an open connection.
6. When a policy is updated, Fleet Server retrieves the updated policy from Elasticsearch and sends it to the connected Elastic Agents.
7. To communicate with Fleet about the status of Elastic Agents and the policy rollout, Fleet Server writes updates to Fleet indices.

3. Polcies

## Data gathered
### 1. User data
- Authenticated users (indexed)
- user id
- Not authenticated users (indexed)
- unique id set into a cookie, with long persistance
- Custom context, applied for every user (not indexed)
- authenticated (boolean)
- ip
- location (geo location object with `lat` and `lon` fields)
- domain (the domain of the email)

### 2. RUM agent data
- Load time of Static Assets (JS, CSS, images, fonts, etc.)
- API requests (XMLHttpRequest and Fetch)
- Single page application navigations
- User interactions (click events that trigger network activity)
- User-centric metrics (Long tasks, FCP, LCP, FID, etc.)
- Page information (URLs visited and referrer)
- Network connection information
- JavaScript errors
- [Distributed tracing](https://www.elastic.co/guide/en/apm/agent/rum-js/current/distributed-tracing-guide.html#enable-cors)
- [Breakdown Metrics](https://www.elastic.co/guide/en/apm/agent/rum-js/current/breakdown-metrics-docs.html)

### 3. Runtime fields - field that is evaluated at query time. Runtime fields enable you to:
- Add fields to existing documents without reindexing your data
- Start working with your data without understanding how it’s structured
- Override the value returned from an indexed field at query time
- Define fields for a specific use without modifying the underlying schema

The following code, for the runtime field `user.alkemio (boolean)`, computes if a transaction is performed by an user who's email domain is `alkem.io`.

if (doc.containsKey('transaction.custom.domain')) {
    def domain = doc['transaction.custom.domain'];
    emit(domain == 'alkem.io');
    return;
}

Other examples include `transaction.custom.authenticated`, `hub.nameId`, `challenge.nameId`

### 4. [Elastic index](https://www.elastic.co/blog/what-is-an-elasticsearch-index) vs [Elastic Data stream](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html) vs [Kibana Data view](https://www.elastic.co/guide/en/kibana/master/data-views.html)
**An index** is like a ‘database' in a relational database. It has a mapping which defines multiple types.
When we are talking about indices Elastic is using **inverted index**.
Building an inverted index by itself is a complex process with a a long list of steps. On the image below you can see the simplest scenario where you have a set of document (sentences for simplicity) which need to be indexed.

1.The sentences are parsed, analyzed and tokenized into tokens (terms).
2. A list is appended with new terms or existing terms are correlated with a document where this terms can be found.
3. When a query is performed in the simplest scenario where just key words are entered, the returns multiple arrays with matching documents
4. The resulting arrays are intersected so all the returned document are matching both the terms

---

It is also a logical namespace which maps to one or more primary shards and can have zero or more replica shards to distribute data around the cluster.

---

**A data stream** lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data.

---

**Kibana** requires a **data view** to access the Elasticsearch data that you want to explore. A data view selects the data to use and allows you to define properties of the fields.
It can point to one or more **indices**, **data streams**, or **index aliases**. For example, a data view can point to your log data from yesterday, or all indices that contain your data.

## APM Dashboard
- Unique authenticated users (alkemio team is excluded)
- transaction.custom.authenticated: true
- user.alkemio: false
- Horizontal: timestamp
- Vertical: unique_count(user.id)
- Unique NOT authenticated users
- transaction.custom.authenticated: false
- Horizontal: timestamp
- Vertical: unique_count(user.id)
- Top 10 Hubs with most user activity (alkemio team is excluded)
- Map of user location
- Control to filter the data by environment

## Top 10 Hubs with most user activity
1. Data view name
2. Filter applied to the data in KIbana Query Language (KQL) - `hub.nameID field NOT empty` AND `user.alkemio = false` (user activity not related to the Alkemio team - based on the user's email domain)
3. Time range filter applied to the data
4. Filtered available fields (from 3 and 4)
5. Search over the filtered available fields
6. If working with timeseries data use the @timestamp field
7. Vertical axis setup

<img src="https://i.imgur.com/bIIzctk.png" style="float: left; width: 250px; padding-right: 30px" />

- Our vertical axis are all documents in this Data view (filtered from 3.) that have `hub.nameID NOT empty` and are NOT associated with the Alkemio team (filtered from 2.)
- Zero values are excluded and
- We have named the Horizontal axis.
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>

8. Break down setup

<img src="https://i.imgur.com/s36kBsI.png" style="float: left; width: 250px; padding-right: 30px" />

- Break down the vertical axis by top 10 values of `hub.nameID` and rank them by occurance.
- Don't display the rest values (grouped by "Other")
- We have named the Break down