# Complete Grafana k6 Load Testing Tutorial

> **Project Repository**: [https://github.com/Yang92047111/Grafana_k6_Load_Testing_Quick_Start](https://github.com/Yang92047111/Grafana_k6_Load_Testing_Quick_Start)

This tutorial provides a comprehensive guide to setting up and using Grafana, Prometheus, and k6 for monitoring and load testing cloud-native applications.

## Table of Contents
1. [Introduction to k6](#Introduction-to-k6)
2. [How k6 Works](#How-k6-Works)
3. [Project Structure](#Project-Structure)
4. [Quick Start Guide](#Quick-Start-Guide)
5. [Setting Up the Environment](#Setting-Up-the-Environment)
6. [Creating a Go Backend Service](#Creating-a-Go-Backend-Service)
7. [Dockerizing the Application](#Dockerizing-the-Application)
8. [Writing k6 Load Tests](#Writing-k6-Load-Tests)
9. [Running k6 Tests](#Running-k6-Tests)
10. [Monitoring and Analysis](#Monitoring-and-Analysis)
11. [CI/CD Pipeline](#CI/CD-Pipeline)
12. [Troubleshooting](#Troubleshooting)

## Introduction to k6

k6 is a developer-centric, free and open-source load testing tool built for making performance testing a productive and enjoyable experience. It's designed to test the reliability and performance of systems under load.

### Key Features:
- **JavaScript-based**: Write tests in modern JavaScript (ES6+)
- **Developer-friendly**: CLI tool with great developer experience
- **Cloud and on-premise**: Run locally or in the cloud
- **Rich metrics**: Built-in metrics and custom metrics support
- **Integrations**: Works with CI/CD pipelines and monitoring systems

## How k6 Works

k6 follows a straightforward architecture for load testing:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   k6 CLI Tool   │    │  Test Scripts    │    │   Target API    │
│                 │    │  (JavaScript)    │    │   (Go Service)  │
│ ┌─────────────┐ │    │                  │    │                 │
│ │Load Testing │ │───▶│ ┌──────────────┐ │───▶│ ┌─────────────┐ │
│ │   Engine    │ │    │ │HTTP Requests │ │    │ │Endpoints    │ │
│ └─────────────┘ │    │ │Checks & Thresholds│ │ │Handlers     │ │
│                 │    │ │Virtual Users │ │    │ │             │ │
│ ┌─────────────┐ │    │ └──────────────┘ │    │ └─────────────┘ │
│ │  Metrics    │ │    │                  │    │                 │
│ │ Collection  │ │    └──────────────────┘    └─────────────────┘
│ └─────────────┘ │              │                       │
└─────────────────┘              │                       │
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│    Results      │    │   Monitoring     │    │    Logs &       │
│   & Reports     │    │   (Grafana)      │    │   Responses     │
└─────────────────┘    └──────────────────┘    └─────────────────┘

### k6 Test Lifecycle:
1. **Init**: Code runs once per VU (Virtual User) to set up test data
2. **Setup**: Runs once before the test starts
3. **VU Code**: Main test function that runs for each iteration
4. **Teardown**: Runs once after the test completes

## Project Structure

This tutorial implements a complete cloud-native load testing solution with the following structure:

grafana-k6-quick-start/
├── .dockerignore              # Docker build optimization
├── .github/workflows/         # CI/CD pipeline (GitHub Actions)
├── backend/                   # Go REST API service
│   ├── handlers/             # HTTP request handlers + tests
│   ├── middleware/           # HTTP middleware + tests
│   ├── models/              # Data models and structures
│   ├── main.go              # Application entry point
│   ├── Dockerfile           # Container configuration
│   └── go.mod/go.sum        # Go module dependencies
├── monitoring/                # Observability stack
│   ├── grafana/             # Dashboards and provisioning
│   └── prometheus/          # Prometheus configuration
├── tests/                     # k6 load testing scripts
│   ├── smoke-test.js        # Basic functionality validation
│   ├── load-test.js         # Standard load testing
│   ├── stress-test.js       # High load stress testing
│   ├── spike-test.js        # Sudden load spike testing
│   └── comprehensive-test.js # Full CRUD testing
├── docker-compose.yml         # Local development stack
├── Makefile                   # Build and deployment commands
└── README.md                 # Project documentation

## Quick Start Guide

### Option A: Local Development (API only)

# Clone the repository (if needed)
git clone <repository-url>
cd grafana-k6-quick-start

# Quick setup - downloads dependencies
make dev-setup

# Start the API server
make run

# In another terminal, test the API
curl http://localhost:8080/health

# Run a quick smoke test
make k6-smoke

### Option B: Full Setup with Monitoring

# Complete development environment with Docker
make dev-setup-full

# Or step by step:
make deps tidy          # Setup Go dependencies
make docker-build       # Build Docker image
make compose-up         # Start monitoring stack

# Access services:
# API: http://localhost:8080
# Grafana: http://localhost:3000 (admin/admin)
# Prometheus: http://localhost:9090

### Available Make Commands

make help              # Show all available commands
make test              # Run unit tests
make build             # Build Go application
make docker-build      # Build Docker image
make k6-smoke          # Run smoke test
make k6-load           # Run load test
make monitoring-up     # Start monitoring stack
make dev-teardown      # Clean up environment

## Setting Up the Environment

### Prerequisites:
- Docker and Docker Compose
- Go 1.21+
- k6

### Installation Commands:

# Install k6 (macOS)
brew install k6

# Install k6 (Linux)
sudo gpg -k
sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6

# Verify installation
k6 version

## Creating a Go Backend Service

Let's create a simple REST API service in Go that we'll use as our testing target.

### Project Structure:

backend/
├── main.go
├── handlers/
│   └── handlers.go
├── models/
│   └── models.go
├── go.mod
└── Dockerfile

### main.go

package main

import (
    "log"
    "net/http"
    "os"
    "time"

    "github.com/gorilla/mux"
    "your-project/handlers"
)

func main() {
    r := mux.NewRouter()
    
    // Routes
    r.HandleFunc("/health", handlers.HealthCheck).Methods("GET")
    r.HandleFunc("/api/users", handlers.GetUsers).Methods("GET")
    r.HandleFunc("/api/users", handlers.CreateUser).Methods("POST")
    r.HandleFunc("/api/users/{id}", handlers.GetUser).Methods("GET")
    r.HandleFunc("/api/users/{id}", handlers.UpdateUser).Methods("PUT")
    r.HandleFunc("/api/users/{id}", handlers.DeleteUser).Methods("DELETE")
    
    // Middleware
    r.Use(handlers.LoggingMiddleware)
    r.Use(handlers.CORSMiddleware)
    
    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    
    srv := &http.Server{
        Handler:      r,
        Addr:         ":" + port,
        WriteTimeout: 15 * time.Second,
        ReadTimeout:  15 * time.Second,
    }
    
    log.Printf("Server starting on port %s", port)
    log.Fatal(srv.ListenAndServe())
}

### handlers/handlers.go

package handlers

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "strconv"
    "time"
    
    "github.com/gorilla/mux"
    "your-project/models"
)

var users = make(map[int]models.User)
var userID = 1

func HealthCheck(w http.ResponseWriter, r *http.Request) {
    response := map[string]interface{}{
        "status":    "healthy",
        "timestamp": time.Now().Unix(),
        "service":   "user-api",
    }
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(response)
}

func GetUsers(w http.ResponseWriter, r *http.Request) {
    // Simulate some processing time
    time.Sleep(50 * time.Millisecond)
    
    userList := make([]models.User, 0, len(users))
    for _, user := range users {
        userList = append(userList, user)
    }
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(userList)
}

func CreateUser(w http.ResponseWriter, r *http.Request) {
    var user models.User
    if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
        http.Error(w, "Invalid JSON", http.StatusBadRequest)
        return
    }
    
    // Simulate processing time
    time.Sleep(100 * time.Millisecond)
    
    user.ID = userID
    user.CreatedAt = time.Now()
    users[userID] = user
    userID++
    
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(user)
}

func GetUser(w http.ResponseWriter, r *http.Request) {
    vars := mux.Vars(r)
    id, err := strconv.Atoi(vars["id"])
    if err != nil {
        http.Error(w, "Invalid user ID", http.StatusBadRequest)
        return
    }
    
    user, exists := users[id]
    if !exists {
        http.Error(w, "User not found", http.StatusNotFound)
        return
    }
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(user)
}

func UpdateUser(w http.ResponseWriter, r *http.Request) {
    vars := mux.Vars(r)
    id, err := strconv.Atoi(vars["id"])
    if err != nil {
        http.Error(w, "Invalid user ID", http.StatusBadRequest)
        return
    }
    
    existingUser, exists := users[id]
    if !exists {
        http.Error(w, "User not found", http.StatusNotFound)
        return
    }
    
    var updatedUser models.User
    if err := json.NewDecoder(r.Body).Decode(&updatedUser); err != nil {
        http.Error(w, "Invalid JSON", http.StatusBadRequest)
        return
    }
    
    // Simulate processing time
    time.Sleep(75 * time.Millisecond)
    
    updatedUser.ID = id
    updatedUser.CreatedAt = existingUser.CreatedAt
    updatedUser.UpdatedAt = time.Now()
    users[id] = updatedUser
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(updatedUser)
}

func DeleteUser(w http.ResponseWriter, r *http.Request) {
    vars := mux.Vars(r)
    id, err := strconv.Atoi(vars["id"])
    if err != nil {
        http.Error(w, "Invalid user ID", http.StatusBadRequest)
        return
    }
    
    if _, exists := users[id]; !exists {
        http.Error(w, "User not found", http.StatusNotFound)
        return
    }
    
    delete(users, id)
    w.WriteHeader(http.StatusNoContent)
}

// Middleware
func LoggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        next.ServeHTTP(w, r)
        log.Printf("%s %s %v", r.Method, r.URL.Path, time.Since(start))
    })
}

func CORSMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Access-Control-Allow-Origin", "*")
        w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
        w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
        
        if r.Method == "OPTIONS" {
            w.WriteHeader(http.StatusOK)
            return
        }
        
        next.ServeHTTP(w, r)
    })
}

### models/models.go

package models

import "time"

type User struct {
    ID        int       `json:"id"`
    Name      string    `json:"name"`
    Email     string    `json:"email"`
    Age       int       `json:"age"`
    CreatedAt time.Time `json:"created_at"`
    UpdatedAt time.Time `json:"updated_at,omitempty"`
}

### go.mod

module your-project

go 1.21

require github.com/gorilla/mux v1.8.0

## Dockerizing the Application

### Dockerfile

# Build stage
FROM golang:1.21-alpine AS builder

WORKDIR /app

# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download

# Copy source code
COPY . .

# Build the application
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .

# Final stage
FROM alpine:latest

RUN apk --no-cache add ca-certificates
WORKDIR /root/

# Copy the binary from builder stage
COPY --from=builder /app/main .

# Expose port
EXPOSE 8080

# Command to run
CMD ["./main"]

### docker-compose.yml

version: '3.8'

services:
  user-api:
    build: ./backend
    ports:
      - "8080:8080"
    environment:
      - PORT=8080
    healthcheck:
      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

## Writing k6 Load Tests

### Basic Load Test (load-test.js)

import http from 'k6/http';
import { check, sleep } from 'k6';
import { Counter, Rate, Trend } from 'k6/metrics';

// Custom metrics
const errorRate = new Rate('error_rate');
const customTrend = new Trend('custom_duration');
const httpReqFailed = new Counter('http_req_failed');

// Test configuration
export const options = {
  stages: [
    { duration: '30s', target: 10 },   // Ramp up to 10 users
    { duration: '1m', target: 10 },    // Stay at 10 users
    { duration: '30s', target: 50 },   // Ramp up to 50 users
    { duration: '2m', target: 50 },    // Stay at 50 users
    { duration: '30s', target: 0 },    // Ramp down to 0 users
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'], // 95% of requests must be below 500ms
    http_req_failed: ['rate<0.05'],   // Error rate must be below 5%
    error_rate: ['rate<0.1'],         // Custom error rate
  },
};

// Base URL - adjust based on your setup
const BASE_URL = 'http://localhost:8080';
// const BASE_URL = 'http://user-api.local'; // For ingress

// Test data
const users = [
  { name: 'John Doe', email: '[e-mail]', age: 30 },
  { name: 'Jane Smith', email: '[e-mail]', age: 25 },
  { name: 'Bob Johnson', email: '[e-mail]', age: 35 },
];

export function setup() {
  // This runs once before all VUs start
  console.log('Starting load test setup...');
  
  // Health check
  const healthResponse = http.get(`${BASE_URL}/health`);
  if (healthResponse.status !== 200) {
    throw new Error('Health check failed');
  }
  
  return { message: 'Setup complete' };
}

export default function(data) {
  // Main test function - runs for each VU iteration
  
  // Health check
  let response = http.get(`${BASE_URL}/health`);
  const healthCheckPassed = check(response, {
    'health check status is 200': (r) => r.status === 200,
    'health check has status field': (r) => JSON.parse(r.body).status === 'healthy',
  });
  
  if (!healthCheckPassed) {
    errorRate.add(1);
    httpReqFailed.add(1);
  }
  
  // Get all users
  response = http.get(`${BASE_URL}/api/users`);
  check(response, {
    'get users status is 200': (r) => r.status === 200,
    'get users response time < 500ms': (r) => r.timings.duration < 500,
  });
  
  // Create a new user
  const userData = users[Math.floor(Math.random() * users.length)];
  userData.email = `${userData.name.replace(' ', '').toLowerCase()}-${Date.now()}@example.com`;
  
  const createUserParams = {
    headers: {
      'Content-Type': 'application/json',
    },
  };
  
  response = http.post(`${BASE_URL}/api/users`, JSON.stringify(userData), createUserParams);
  const createUserPassed = check(response, {
    'create user status is 201': (r) => r.status === 201,
    'create user has id': (r) => JSON.parse(r.body).id > 0,
    'create user response time < 1000ms': (r) => r.timings.duration < 1000,
  });
  
  if (createUserPassed && response.status === 201) {
    const createdUser = JSON.parse(response.body);
    
    // Get the created user
    response = http.get(`${BASE_URL}/api/users/${createdUser.id}`);
    check(response, {
      'get user by id status is 200': (r) => r.status === 200,
      'get user by id returns correct user': (r) => JSON.parse(r.body).id === createdUser.id,
    });
    
    // Update the user
    const updatedUserData = {
      ...userData,
      age: userData.age + 1,
    };
    
    response = http.put(`${BASE_URL}/api/users/${createdUser.id}`, JSON.stringify(updatedUserData), createUserParams);
    check(response, {
      'update user status is 200': (r) => r.status === 200,
      'update user age changed': (r) => JSON.parse(r.body).age === updatedUserData.age,
    });
    
    // Delete the user
    response = http.del(`${BASE_URL}/api/users/${createdUser.id}`);
    check(response, {
      'delete user status is 204': (r) => r.status === 204,
    });
  } else {
    errorRate.add(1);
  }
  
  // Record custom metrics
  customTrend.add(response.timings.duration);
  
  // Sleep between 1-3 seconds
  sleep(Math.random() * 2 + 1);
}

export function teardown(data) {
  // This runs once after all VUs have finished
  console.log('Load test teardown complete');
}

### Spike Test (spike-test.js)

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '10s', target: 10 },   // Below normal load
    { duration: '1m', target: 10 },    // Normal load
    { duration: '10s', target: 100 },  // Around the breaking point
    { duration: '3m', target: 100 },   // Beyond the breaking point
    { duration: '10s', target: 10 },   // Scale down. Recovery stage.
    { duration: '3m', target: 10 },    // Recovery stage.
    { duration: '10s', target: 0 },    // Scale down to 0 users.
  ],
  thresholds: {
    http_req_duration: ['p(99)<1500'], // 99% of requests must be below 1.5s
    http_req_failed: ['rate<0.1'],     // Error rate must be below 10%
  },
};

const BASE_URL = 'http://localhost:8080';

export default function() {
  const response = http.get(`${BASE_URL}/health`);
  check(response, {
    'spike test - status is 200': (r) => r.status === 200,
    'spike test - response time OK': (r) => r.timings.duration < 2000,
  });
  sleep(1);
}

### Stress Test (stress-test.js)

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '2m', target: 100 },  // Ramp up to 100 users
    { duration: '5m', target: 100 },  // Stay at 100 users
    { duration: '2m', target: 200 },  // Ramp up to 200 users
    { duration: '5m', target: 200 },  // Stay at 200 users
    { duration: '2m', target: 300 },  // Ramp up to 300 users
    { duration: '5m', target: 300 },  // Stay at 300 users
    { duration: '2m', target: 400 },  // Ramp up to 400 users
    { duration: '5m', target: 400 },  // Stay at 400 users
    { duration: '10m', target: 0 },   // Ramp down to 0 users
  ],
  thresholds: {
    http_req_duration: ['p(99)<2000'], // 99% of requests must be below 2s
    http_req_failed: ['rate<0.1'],     // Error rate must be below 10%
  },
};

const BASE_URL = 'http://localhost:8080';

export default function() {
  const responses = http.batch([
    ['GET', `${BASE_URL}/health`],
    ['GET', `${BASE_URL}/api/users`],
  ]);
  
  check(responses[0], {
    'stress test - health check status is 200': (r) => r.status === 200,
  });
  
  check(responses[1], {
    'stress test - users endpoint status is 200': (r) => r.status === 200,
  });
  
  sleep(1);
}

### Smoke Test (smoke-test.js)

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 1,        // 1 user looping for 1 minute
  duration: '1m',
  thresholds: {
    http_req_duration: ['p(99)<1500'], // 99% of requests must be below 1.5s
    http_req_failed: ['rate<0.01'],    // Error rate must be below 1%
  },
};

const BASE_URL = 'http://localhost:8080';

export default function() {
  const response = http.get(`${BASE_URL}/health`);
  check(response, {
    'smoke test - status is 200': (r) => r.status === 200,
    'smoke test - service is healthy': (r) => JSON.parse(r.body).status === 'healthy',
  });
  sleep(1);
}

## Running k6 Tests

### Test Execution Commands:

# Run smoke test (quick validation)
k6 run smoke-test.js

# Run basic load test
k6 run load-test.js

# Run stress test
k6 run stress-test.js

# Run spike test
k6 run spike-test.js

# Run test with custom VUs and duration
k6 run --vus 50 --duration 30s load-test.js

# Run test and output results to file
k6 run --out json=results.json load-test.js

# Run test with custom thresholds
k6 run --threshold http_req_duration=p(95)<200 load-test.js

### k6 in Docker:

# Run k6 tests in Docker
docker run --rm -i grafana/k6:latest run - <load-test.js

# Run with volume mount for test files
docker run --rm -v "$PWD:/tests" grafana/k6:latest run /tests/load-test.js

# Run with network access to local services
docker run --rm --network=host -v "$PWD:/tests" grafana/k6:latest run /tests/load-test.js

### k6 in Kubernetes:

#### k6-job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: k6-load-test
spec:
  template:
    spec:
      containers:
      - name: k6
        image: grafana/k6:latest
        command: ["k6", "run", "--vus", "50", "--duration", "5m", "/scripts/load-test.js"]
        volumeMounts:
        - name: k6-scripts
          mountPath: /scripts
        env:
        - name: K6_OUT
          value: "json=/tmp/results.json"
      volumes:
      - name: k6-scripts
        configMap:
          name: k6-scripts
      restartPolicy: Never
  backoffLimit: 1

#### Create ConfigMap for test scripts:

kubectl create configmap k6-scripts --from-file=load-test.js

kubectl apply -f k6-job.yaml

# Check job status
kubectl get jobs
kubectl logs job/k6-load-test

## Monitoring and Analysis

### Setting up Grafana and Prometheus with Docker Compose:

The monitoring stack is already configured in the `docker-compose.yml` file with Grafana and Prometheus services.

### Start the monitoring stack:

# Start all services including monitoring
docker compose up -d

# Access the services:
# Grafana: http://localhost:3000 (admin/admin)  
# Prometheus: http://localhost:9090

### Run k6 with Prometheus output:

# Run k6 with Prometheus remote write output (when using docker-compose)
k6 run --out experimental-prometheus-rw=http://localhost:9090/api/v1/write load-test.js

# Alternative: Run k6 in Docker with Prometheus output
docker compose exec k6 k6 run --out experimental-prometheus-rw=http://prometheus:9090/api/v1/write /tests/load-test.js

### k6 Grafana Dashboard Configuration:

1. **Access Grafana**: http://localhost:3000 (admin/admin)

2. **Add Prometheus Data Source**:
- URL: http://localhost:9090
- Access: Server (default)
- HTTP Method: GET

3. **Import k6 Dashboard**:
- Go to + → Import
- Use dashboard ID: 19665 (k6 Prometheus dashboard)
- Or create custom dashboard with these queries:

#### Key Grafana Queries (PromQL):

# HTTP Request Duration (95th percentile)
histogram_quantile(0.95, rate(k6_http_req_duration_bucket[5m]))

# HTTP Request Rate
rate(k6_http_reqs_total[5m])

# Error Rate (percentage)
rate(k6_http_req_failed_total[5m]) / rate(k6_http_reqs_total[5m]) * 100

# Virtual Users
k6_vus

# Data Received Rate
rate(k6_data_received_total[5m])

# Data Sent Rate  
rate(k6_data_sent_total[5m])

# HTTP Request Duration Average
rate(k6_http_req_duration_sum[5m]) / rate(k6_http_req_duration_count[5m])

# Checks Pass Rate
rate(k6_checks_passed_total[5m]) / (rate(k6_checks_passed_total[5m]) + rate(k6_checks_failed_total[5m])) * 100

## Advanced k6 Testing Patterns

### Data-Driven Testing:

#### users-data.json

[
  {"name": "Alice Johnson", "email": "[e-mail]", "age": 28},
  {"name": "Bob Smith", "email": "[e-mail]", "age": 34},
  {"name": "Charlie Brown", "email": "[e-mail]", "age": 22},
  {"name": "Diana Prince", "email": "[e-mail]", "age": 31},
  {"name": "Eve Davis", "email": "[e-mail]", "age": 26}
]

#### data-driven-test.js

import http from 'k6/http';
import { check } from 'k6';

const userData = JSON.parse(open('./users-data.json'));

export const options = {
  vus: 10,
  duration: '2m',
};

const BASE_URL = 'http://localhost:8080';

export default function() {
  // Pick a random user from the dataset
  const user = userData[Math.floor(Math.random() * userData.length)];
  
  // Modify email to make it unique
  user.email = `${user.name.replace(' ', '').toLowerCase()}-${__ITER}@example.com`;
  
  const response = http.post(`${BASE_URL}/api/users`, JSON.stringify(user), {
    headers: { 'Content-Type': 'application/json' },
  });
  
  check(response, {
    'user created successfully': (r) => r.status === 201,
    'user has correct name': (r) => JSON.parse(r.body).name === user.name,
  });
}

### Session-Based Testing:

#### session-test.js

import http from 'k6/http';
import { check, group, sleep } from 'k6';

export const options = {
  vus: 20,
  duration: '5m',
};

const BASE_URL = 'http://localhost:8080';

export default function() {
  // Simulate user session
  group('User Registration Flow', () => {
    const userData = {
      name: `User ${__VU}-${__ITER}`,
      email: `user-${__VU}-${__ITER}@example.com`,
      age: Math.floor(Math.random() * 50) + 18,
    };
    
    // Create user
    let response = http.post(`${BASE_URL}/api/users`, JSON.stringify(userData), {
      headers: { 'Content-Type': 'application/json' },
    });
    
    const userId = check(response, {
      'user created': (r) => r.status === 201,
    }) ? JSON.parse(response.body).id : null;
    
    if (userId) {
      sleep(1);
      
      // Get user profile
      group('Profile Operations', () => {
        response = http.get(`${BASE_URL}/api/users/${userId}`);
        check(response, {
          'profile retrieved': (r) => r.status === 200,
        });
        
        sleep(2);
        
        // Update user profile
        userData.age += 1;
        response = http.put(`${BASE_URL}/api/users/${userId}`, JSON.stringify(userData), {
          headers: { 'Content-Type': 'application/json' },
        });
        
        check(response, {
          'profile updated': (r) => r.status === 200,
        });
      });
      
      sleep(1);
      
      // Clean up - delete user
      response = http.del(`${BASE_URL}/api/users/${userId}`);
      check(response, {
        'user deleted': (r) => r.status === 204,
      });
    }
  });
  
  sleep(Math.random() * 5 + 1);
}

### Performance Testing Best Practices:

#### comprehensive-test.js

import http from 'k6/http';
import { check, group, sleep, fail } from 'k6';
import { Rate, Trend, Counter } from 'k6/metrics';

// Custom metrics
const errorRate = new Rate('errors');
const responseTimeTrend = new Trend('response_time');
const customCounter = new Counter('custom_counter');

export const options = {
  stages: [
    { duration: '1m', target: 20 },
    { duration: '3m', target: 20 },
    { duration: '1m', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.05'],
    errors: ['rate<0.1'],
    response_time: ['p(95)<600'],
  },
  ext: {
    loadimpact: {
      name: 'Comprehensive API Test',
      projectID: 123456,
    },
  },
};

const BASE_URL = __ENV.BASE_URL || 'http://localhost:8080';

export function setup() {
  // Setup phase - run once
  console.log(`Testing API at: ${BASE_URL}`);
  
  const response = http.get(`${BASE_URL}/health`);
  if (response.status !== 200) {
    fail('API is not healthy');
  }
  
  return {
    baseUrl: BASE_URL,
    testData: generateTestData(100),
  };
}

export default function(data) {
  const testUser = data.testData[Math.floor(Math.random() * data.testData.length)];
  
  group('API Health Check', () => {
    const response = http.get(`${data.baseUrl}/health`);
    const success = check(response, {
      'health check passes': (r) => r.status === 200,
      'has healthy status': (r) => JSON.parse(r.body).status === 'healthy',
    });
    
    if (!success) {
      errorRate.add(1);
    }
    
    responseTimeTrend.add(response.timings.duration);
  });
  
  group('User CRUD Operations', () => {
    // Create
    let response = http.post(`${data.baseUrl}/api/users`, JSON.stringify(testUser), {
      headers: { 'Content-Type': 'application/json' },
    });
    
    const createSuccess = check(response, {
      'user created successfully': (r) => r.status === 201,
      'response has user id': (r) => JSON.parse(r.body).id > 0,
    });
    
    if (!createSuccess) {
      errorRate.add(1);
      return; // Skip remaining operations if creation fails
    }
    
    const userId = JSON.parse(response.body).id;
    customCounter.add(1);
    
    sleep(0.5);
    
    // Read
    response = http.get(`${data.baseUrl}/api/users/${userId}`);
    check(response, {
      'user retrieved successfully': (r) => r.status === 200,
      'correct user returned': (r) => JSON.parse(r.body).id === userId,
    }) || errorRate.add(1);
    
    sleep(0.5);
    
    // Update
    testUser.age += 1;
    response = http.put(`${data.baseUrl}/api/users/${userId}`, JSON.stringify(testUser), {
      headers: { 'Content-Type': 'application/json' },
    });
    
    check(response, {
      'user updated successfully': (r) => r.status === 200,
      'age was updated': (r) => JSON.parse(r.body).age === testUser.age,
    }) || errorRate.add(1);
    
    sleep(0.5);
    
    // Delete
    response = http.del(`${data.baseUrl}/api/users/${userId}`);
    check(response, {
      'user deleted successfully': (r) => r.status === 204,
    }) || errorRate.add(1);
  });
  
  sleep(Math.random() * 2 + 1);
}

export function teardown(data) {
  console.log('Test completed');
  console.log(`Base URL used: ${data.baseUrl}`);
}

function generateTestData(count) {
  const names = ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve', 'Frank', 'Grace', 'Henry'];
  const domains = ['example.com', 'test.com', 'demo.com'];
  
  const users = [];
  for (let i = 0; i < count; i++) {
    users.push({
      name: `${names[i % names.length]} ${i}`,
      email: `user${i}@${domains[i % domains.length]}`,
      age: Math.floor(Math.random() * 50) + 18,
    });
  }
  
  return users;
}

## Continuous Integration Integration

### GitHub Actions Workflow:

#### .github/workflows/load-test.yml

name: Load Testing with k6

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * *'  # Run daily at 2 AM

jobs:
  load-test:
    runs-on: ubuntu-latest
    
    services:
      api:
        image: user-api:latest
        ports:
          - 8080:8080
        env:
          PORT: 8080
        options: >-
          --health-cmd "wget --quiet --tries=1 --spider http://localhost:8080/health"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Setup Go
      uses: actions/setup-go@v4
      with:
        go-version: '1.21'
    
    - name: Build Docker image
      run: |
        cd backend
        docker build -t user-api:latest .
    
    - name: Install k6
      run: |
        sudo gpg -k
        sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
        echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
        sudo apt-get update
        sudo apt-get install k6
    
    - name: Wait for API to be ready
      run: |
        timeout 300 bash -c 'until curl -f http://localhost:8080/health; do sleep 5; done'
    
    - name: Run smoke test
      run: k6 run tests/smoke-test.js
    
    - name: Run load test
      run: k6 run tests/load-test.js --out json=results.json
    
    - name: Upload test results
      uses: actions/upload-artifact@v3
      if: always()
      with:
        name: k6-test-results
        path: results.json
    
    - name: Comment PR with results
      if: github.event_name == 'pull_request'
      uses: actions/github-script@v6
      with:
        script: |
          const fs = require('fs');
          try {
            const results = fs.readFileSync('results.json', 'utf8');
            const metrics = JSON.parse(results);
            // Process and comment results...
          } catch (error) {
            console.log('No results file found');
          }

## CI/CD Pipeline

The project includes a comprehensive GitHub Actions pipeline that automates testing, building, and deployment. Here's what the pipeline includes:

### Pipeline Overview

# .github/workflows/ci-cd.yml
name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:           # Unit tests with coverage
  build:          # Docker image build and push
  load-test:      # Automated k6 testing
  helm-test:      # Helm chart validation
  security-scan:  # Security vulnerability scanning
  deploy-staging: # Deploy to staging environment
  deploy-production: # Deploy to production
  post-deploy-tests: # Production smoke tests

### Pipeline Features
- **Unit Testing**: Go test suite with coverage reporting
- **Security Scanning**: Gosec (static analysis) + Trivy (image scanning)
- **Load Testing**: Automated k6 tests against built containers
- **Helm Validation**: Chart linting and templating verification
- **Multi-stage Deployment**: Staging → Production with approvals
- **Post-deployment Testing**: Smoke tests in production

### Using the Pipeline
The pipeline automatically triggers on:
- **Pull Requests**: Runs tests and validation only
- **Develop Branch**: Deploys to staging environment
- **Main Branch**: Deploys to production environment

## Troubleshooting

### Common Issues and Solutions

#### Docker Build Problems

# Test Docker connectivity
make docker-test

# Clean Docker cache
make docker-clean

# Test network connectivity
make network-test

# If Docker issues persist, use local development
make dev-setup
make run

#### API Not Starting

# Check if port is in use
lsof -i :8080

# Check API logs
make compose-logs

# Test API directly
go run backend/main.go

#### k6 Tests Failing

# Verify API is healthy
curl http://localhost:8080/health

# Test with verbose output
k6 run --http-debug tests/smoke-test.js

# Check if base URL is correct
export BASE_URL=http://localhost:8080
k6 run tests/smoke-test.js

#### Kubernetes Deployment Issues

# Check pod status
kubectl get pods
kubectl describe pod <pod-name>

# Check service connectivity
kubectl get svc
kubectl port-forward service/user-api 8080:80

# Check Helm deployment
helm status user-api
helm list

#### Monitoring Stack Issues

# Check Prometheus connection
kubectl port-forward -n monitoring service/prometheus 9090:9090
curl http://localhost:9090/-/healthy

# Verify Grafana datasource
# Login to Grafana → Configuration → Data Sources

# Check k6 output format
k6 run --out experimental-prometheus-rw=http://localhost:9090/api/v1/write tests/smoke-test.js

### Make Commands for Troubleshooting

make help              # Show all available commands
make docker-test       # Test Docker connectivity
make network-test      # Test network connectivity
make test-coverage     # Run tests with coverage report
make k8s-status        # Show Kubernetes resource status
make monitoring-up     # Restart monitoring stack

## Conclusion and Next Steps

This comprehensive tutorial has covered:

1. **k6 Fundamentals**: Understanding how k6 works and its architecture
2. **Environment Setup**: Local development with Docker
3. **Backend Service**: Go-based REST API with proper endpoints
4. **Container Orchestration**: Docker containerization
5. **Load Testing Scenarios**: Various test types (smoke, load, stress, spike)
6. **Monitoring Integration**: Grafana and Prometheus for metrics visualization
7. **Advanced Patterns**: Data-driven and session-based testing
8. **CI/CD Integration**: Automated testing in GitHub Actions

### Next Steps:

1. **Scale Testing**: Experiment with distributed k6 testing across multiple nodes
2. **Advanced Monitoring**: Integrate with Prometheus, Jaeger for distributed tracing
3. **Security Testing**: Add authentication and authorization testing scenarios
4. **Database Integration**: Include database load testing scenarios
5. **Cloud Deployment**: Deploy to cloud providers (AWS EKS, GCP GKE, Azure AKS)
6. **Performance Tuning**: Use k6 results to optimize application performance
7. **Custom Extensions**: Develop k6 extensions for specific testing needs

### Key Takeaways:

- **Start Small**: Begin with smoke tests before running intensive load tests
- **Monitor Everything**: Always collect metrics and monitor system behavior
- **Test Early**: Integrate performance testing into your development workflow
- **Realistic Scenarios**: Design tests that mirror real user behavior
- **Infrastructure Matters**: Ensure your test environment resembles production
- **Continuous Improvement**: Use test results to drive performance optimizations

This setup provides a solid foundation for comprehensive load testing with k6, enabling you to ensure your applications can handle production traffic effectively.