# Logging

Logging is an important feature to identify and diagnose problems that are happening in our application. Having a logger means we can pinpoint the areas of our code that are causing bugs, analyze the most common errors and debug them.

## Logging diagram

<iframe style="border:none" width="800" height="450" src="https://whimsical.com/embed/KnLdAnY5YYJpL5Lvy7nPdF@2Ux7TurymNBL1SKckztS"></iframe>

## How to log data on Shopping Bag xAPI

First, you need to have a `LogSource`. You should create or reuse a LogSource inside `src/lib/logging`.
A LogSource have four properties: `formatters`, `context`, `level` and `log`.
You can use the snippets below as a starting point:

export type LogSource = {
    formatters?: { 
        error?: (err: unknown) => LogEvent;  // These formatters should be used
        info?: (data: unknown) => LogEvent;  // to format any data if necessary.
        debug?: (data: unknown) => LogEvent; // The prop key should 
                                             // be a LOG_LEVELS value.
    };
    context: string; // This prop is responsible for separating the contexts.
                     // Every LogSource should have its own context, and should
                     // never be a duplicate.
    level: LOG_LEVELS; // Indicate the allowed log levels. error, info...
    log: CreateLogEventFunction; // This should be a function that creates 
                                 // a LogEvent object and send to 
                                 // RequestContextStore.
};

You can pass custom formatters and log function if necessary, but the default ones should be OK in most of the cases.

Here's an example of a `LogSource` implementation (yes, you can - and should - Copy+Paste this one).

const BAG_EXAMPLE_CONTEXT = "GetBagExampleService";
export const BagExampleLogSource: LogSource = {
  formatters: {},
  context: BAG_EXAMPLE_CONTEXT,
  level: getLoggingLevel(BAG_EXAMPLE_CONTEXT),
  log(data, level, context = BAG_EXAMPLE_CONTEXT): void {
    defaultLog(this.level, this.context, this.formatters)(data, level, context);
  },
};

After you create a `LogSource`, to log something you just invoke the `log` method as seen below:

import { BagExampleLogSource } from "src/lib/logging/logSources";
import { LOG_LEVELS } from "src/lib/logging/logging.types";

BagExampleLogSource.log("Handling GET /get-bag-example", LOG_LEVELS.info);
BagExampleLogSource.log("Example log message"); // defaults to "info"

The log method has three parameters:

export type CreateLogEventFunction = (
  data: string | Partial<LogEvent>, // if string, it will handle as a 'message'.
                                    // if object, it must be a partial LogEvent
  level?: LOG_LEVELS, // to inform the level of the log that you are creating
  context?: string // in case you need to change the context of it. not advisable
) => void;

This should create a LogEvent and send it to the Request Context Store.

The `log` function can be customized for each `LogSource`.
In case you don't need customization, you can just use `defaultLog` (see example above).

## Request Context Store

The Request Context Store is a wrapper to interact with the [AsyncLocalStorage](https://nodejs.org/api/async_context.html#class-asynclocalstorage).
It is a store and exists only during the lifetime of a Request.

When the request comes in, the store is created with a `LogEntry`:

export type LogEntry = {
  shopperId: string;
  requestId: string;
  requestIdSource: "client" | "xapi";
  hasError: boolean;
  request: XAPIRequestLog;
  response: XAPIResponseLog;
  events: LogEvent[];
}

An important property of `LogEntry` is `events`:

- An array of `LogEvents` that are logged through the lifetime of the request.
- This array holds all the information that are logged using a LogSource.log() method:

export type LogEvent = {
  context: string;
  level: LOG_LEVELS;
  timestamp: number;
  request?: ExternalRequest;
  response?: ExternalResponse;
  data?: EventData;
  stackTrace?: EventErrorStackTrace;
  errorDetails?: string;
  message?: string;
  name?: string;
  additionalInfo?: Record<string, unknown>;
}

After the request is complete, the LogEntry is sent to Pino.

## Pino

[Pino](https://github.com/pinojs/pino) is a Node.js logger. It's similar to `Winston`, which we currently use in the xAPI, but has the advantage of having less overhead, thus being usually faster than the alternatives.

You can set the level baseline (hierarchy based in severity defined by RFC5424) that Pino will use.
Pino will only log an event if the event level is higher than the baseline level.

As an example, if Pino has `level = error`, it will only log errors, ignoring events that have the level of `debug` and `info`.

The current levels that Pino support are these:

| Level | trace | debug | info | warn | error | fatal |
|:-----:|:-----:|:-----:|:----:|:----:|:-----:|:-----:|
| Value | 10 | 20 | 30 | 40 | 50 | 60 |

Pino will stream the information logged to the **transports**. Common transports that are being used by xAPI are the Console Terminal, that will send all the information to the standard output, and New Relic, which will receive all the errors that are being logged.

## NewRelic

NewRelic is a monitoring platform which provides tools to observe and analyze data from our application. This data is sent through a Pino's transport. It also collects data automatically from requests and responses.

Within NewRelic, dashboards can be created and also alerts to monitor the error rates on a specific feature or application.
Currently, these alerts are integrated with PagerDuty.

To set up NewRelic in our application you will need to configure your `.env.dev` with these properties: `NEW_RELIC_APP_NAME`, `NEW_RELIC_LICENSE_KEY` and `NEW_RELIC_ENABLED`. You can check the `.env.dev.example` that exists in our repository.

You can get the dev License Key by accessing Cloud Foundry and getting the environment variables on `shopping-bag-stage`.

NewRelic uses a SQL-like syntax called NRQL to query the information. Here is an example to retrieve errors that were sent to NewRelic in our application:

SELECT * FROM TransactionError WHERE appName = 'shopping-bag-stage-azeus'

Remember to substitute `shopping-bag-stage-azeus` with the actual environment variable.

[Example Query](https://onenr.io/0EPwJ4MnDj7)

[Example Dashboard](https://onenr.io/0gbRKBVklwE)

## Splunk

Splunk is also an observability platform. The difference is that it reads from the standard output and log all the events that appears in the console terminal.