# Implementing APM and Logging with Serilog and ELK Stack in .NET Core Minimal API

### 1. Introduction:
> This guide outlines the integration of Application Performance Monitoring (APM) and logging into a .NET Core Minimal API application using ELK Stack (Elasticsearch, Logstash, Kibana) and Serilog for structured logging.
>
### 2. Prerequisites:

A running ELK Stack instance.
Basic understanding of .NET Core and C#.

version: '3.9'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.1
    environment:
      - discovery.type=single-node
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
    ports:
      - 9200:9200
    volumes:
      - esdata:/usr/share/elasticsearch/data

  kibana:
    image: docker.elastic.co/kibana/kibana:7.17.1
    depends_on:
      - elasticsearch
    ports:
      - 5601:5601
    environment:
      - ELASTICSEARCH_URL=http://elasticsearch:9200

  apm-server:
    image: docker.elastic.co/apm/apm-server:7.17.1
    cap_add:
      - CHOWN
      - DAC_OVERRIDE
      - SETGID
      - SETUID
    cap_drop:
      - ALL
    depends_on:
      - elasticsearch
    ports:
      - 8200:8200
    environment:
      - setup.kibana.host=kibana:5601
      - setup.template.settings.index.number_of_replicas=0
      - output.elasticsearch.hosts=["elasticsearch:9200"]
      - apm-server.rum.enabled=true  # Optional: Enable RUM (Real User Monitoring)

volumes:
  esdata: {}  # Persistent storage for Elasticsearch data

### 3. Project Setup:

#### 1. Create a new .NET Core 7 Minimal API project:

dotnet new webapi -o ElkIntegrationApi

Install required NuGet packages:

cd ElkIntegrationApi
dotnet add package Serilog
dotnet add package Serilog.Sinks.File
dotnet add package Serilog.Sinks.Elasticsearch
dotnet add package Elastic.Apm.AspNetCore

#### 2. Configuration with appsettings.json:

Create an appsettings.json file with the following structure, replacing placeholders with your actual values:

{
  "Logging": {
    "MinimumLevel": "Information", // Optional: Set the minimum logging level
    "Serilog": {
      "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "logs/elk_logs.txt",
            "rollingInterval": "Day" // Optional: Set rolling interval for file logs
          }
        },
        {
          "Name": "Elasticsearch",
          "Args": {
            "url": "http://localhost:9200", // Replace with your Elasticsearch URL
            "autoTemplate": "Default", // Optional: Set auto-template name
            "buffer": {
              "bufferSize": 100,
              "flushPeriod": "00:00:05" // Optional: Set flush period in format "hh:mm:ss"
            },
            "indexFormat": "elk-logs-{0:yyyyMMdd}" // Optional: Set dynamic index format
          }
        }
      ]
    }
  },
  "ElasticApm": {
    "ServiceName": "MyAwesomeService", // Replace with your service name
    "ServerUrls": "http://localhost:8200", // Replace with your APM server URL(s)
    "Environment": "Development" // Optional: Set environment
    # Additional APM settings (optional)
  }
}

#### 3. Program.cs with Combined Configuration:

Modify your Program.cs file to read the configuration from appsettings.json and utilize APM settings:

using System.Text.Json;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Serilog;
using Elastic.Apm.AspNetCore;

public class Program
{
    public static void Main(string[] args)
    {
        var configuration = new ConfigurationBuilder()
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
            .Build();

        // Read Serilog and APM configuration from appsettings.json
        Log.Logger = new LoggerConfiguration()
            .ReadFrom.Configuration(configuration)
            .CreateLogger();

        var app = WebApplication.Create(args);

        app.UseAllElasticApm(); // Enables auto-instrumentation (optional)

        app.MapGet("/", () =>
        {
            using (var transaction = Apm.Current?.StartTransaction("GET /")) // Start APM transaction
            {
                Log.Information("Hello from ElkIntegrationApi!");
                transaction?.Annotate("http.method", "GET"); // Add APM annotations (optional)
                transaction?.Annotate("http.url", "/"); // Add APM annotations (optional)
                return Results.Content("Hello from ElkIntegrationApi!", "text/plain");
            }
        });

        app.Run();
    }
}

### 4. Explanation:

* The **appsettings.json** file stores both Serilog and Elastic APM configuration options.
* Serilog settings define sinks and options for log writing to file and Elasticsearch.
* Elastic APM settings provide the service name, server URLs, and an optional environment.
* The Program.cs code reads the configuration and creates the Serilog logger based on Serilog settings.
* The **app.UseAllElasticApm()** call enables auto-instrumentation for common scenarios using APM settings.

* **Key Auto-instrumented Areas:**
* **HTTP Requests and Responses:**
Tracks the duration, status codes, request methods, and URLs of HTTP requests.
Captures details like request headers, body size, and response headers (configurable).
* **ASP.NET Core MVC/Razor Pages Actions:**
Measures the execution time of controller actions and Razor Pages handlers.
Provides context information like action name, route values, and view rendering time (if applicable).
* **Database Calls (ADO.NET, Entity Framework):**
Monitors the duration and type of database operations (e.g., SELECT, UPDATE, DELETE).
Captures command text and parameters (when available).
* **Exceptions:**
Records exception type, message, stack trace, and associated HTTP request information (if any).
* **ASP.NET Core Middleware:**
Tracks the execution time of custom middleware components in your application pipeline.
* **Customization and Configuration:**

While app.UseAllElasticApm() provides a convenient starting point, you can tailor auto-instrumentation behavior using various configuration options:

* **Disabling Auto-instrumentation:**
If you need granular control, call app.UseElasticApm(configuration => { configuration.AutoInstrument = false; }); to disable auto-instrumentation completely.
* **Enabling/Disabling Specific Areas:**
Use options like configuration.CaptureRequestData = false; or configuration.CaptureMvc = false; to control which areas are automatically captured.
* **Custom Instrumentation:**
For scenarios beyond pre-defined areas, employ the ITracer interface to manually instrument specific code sections using methods like StartTransaction and EndTransaction.

* Within the endpoint handler, we create an APM transaction to track the specific request and add annotations for additional context.