---
# System prepended metadata

title: "\U0001F4DA Kubernetes Ingress vs Gateway API: Complete Tutorial"
tags: ['#CloudNative']

---

# 📚 Kubernetes Ingress vs Gateway API: Complete Tutorial

**Repository:** https://github.com/Yang92047111/Kubernetes-Ingress-Gateway-API

## Table of Contents
1. [Introduction](#Introduction)
2. [Kubernetes Ingress](#Kubernetes-Ingress)
3. [Gateway API](#Gateway-API)
4. [Key Differences](#Key-Differences)
5. [Architecture Diagrams](#Architecture-Diagrams)
6. [Practical Examples](#Practical-Examples)
7. [Migration Path](#Migration-Path)
8. [Best Practices](#Best-Practices)
9. [Conclusion](#Conclusion)

---

## Introduction

Kubernetes provides two main approaches for exposing services to external traffic: **Ingress** and the newer **Gateway API**. This tutorial explores both approaches, their differences, and when to use each one.

### What Problem Do They Solve?

Both Ingress and Gateway API solve the challenge of:
- Exposing internal Kubernetes services to external clients
- Managing HTTP/HTTPS routing
- Handling SSL termination
- Load balancing traffic
- Managing traffic policies

---

## Kubernetes Ingress

### Overview
Kubernetes Ingress is the traditional way to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. It was introduced in Kubernetes 1.1 and has been the standard for years.

### Core Components

#### 1. Ingress Resource
Defines the routing rules and configuration:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-app-ingress
  namespace: routing-experiment
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: localhost
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: go-app
            port:
              number: 80
```

#### 2. Ingress Controller
The actual implementation that processes Ingress resources:
- **NGINX Ingress Controller** (most popular)
- **Traefik**
- **HAProxy**
- **Istio Gateway**
- **AWS Load Balancer Controller**

#### 3. IngressClass
Specifies which controller should handle the Ingress:

```yaml
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
```

### How Ingress Works

1. **Define Routes**: Create an Ingress resource with routing rules
2. **Controller Processing**: Ingress controller watches for Ingress resources
3. **Configuration Update**: Controller updates its configuration (e.g., NGINX config)
4. **Traffic Routing**: External traffic flows through the controller to services

### Ingress Capabilities

✅ **Strengths:**
- Simple and well-established
- Wide controller ecosystem
- Good for basic HTTP/HTTPS routing
- Mature tooling and documentation

❌ **Limitations:**
- Limited to HTTP/HTTPS only
- Controller-specific annotations for advanced features
- No native support for other protocols (TCP, UDP, gRPC)
- Configuration tied to implementation details
- Limited traffic splitting capabilities

---

## Gateway API

### Overview
Gateway API is the next-generation standard for service networking in Kubernetes. It's designed to be more expressive, extensible, and role-oriented than Ingress.

### Core Components

#### 1. GatewayClass
Defines the type of gateway and its controller:

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy-gateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
```

#### 2. Gateway
Represents the actual gateway instance:

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: go-app-gateway
  namespace: routing-experiment
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 8080
    allowedRoutes:
      namespaces:
        from: Same
```

#### 3. Route Resources
Define how requests are routed to services:

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: go-app-route
  namespace: routing-experiment
spec:
  parentRefs:
  - name: go-app-gateway
  hostnames:
  - "localhost"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: go-app
      port: 80
```

### Route Types
- **HTTPRoute**: HTTP/HTTPS traffic
- **GRPCRoute**: gRPC traffic
- **TCPRoute**: TCP traffic
- **UDPRoute**: UDP traffic
- **TLSRoute**: TLS-encrypted traffic

### How Gateway API Works

1. **Install GatewayClass**: Define the gateway implementation
2. **Create Gateway**: Instantiate a gateway with listeners
3. **Configure Routes**: Create route resources that reference the gateway
4. **Traffic Flow**: External traffic flows through gateway to services

### Gateway API Capabilities

✅ **Strengths:**
- Protocol-agnostic (HTTP, gRPC, TCP, UDP, TLS)
- Role-oriented design (infrastructure vs application teams)
- Extensible without vendor-specific annotations
- Built-in support for advanced features (traffic splitting, header manipulation)
- Clear separation of concerns
- Standardized across implementations

❌ **Limitations:**
- Newer standard (less mature ecosystem)
- More complex for simple use cases
- Limited controller implementations (growing)

---

## Key Differences

| Aspect | Ingress | Gateway API |
|--------|---------|-------------|
| **Protocols** | HTTP/HTTPS only | HTTP, HTTPS, gRPC, TCP, UDP, TLS |
| **Extensibility** | Annotations (vendor-specific) | Native API fields (standardized) |
| **Role Separation** | Single resource | Multiple resources for different roles |
| **Traffic Management** | Basic routing | Advanced (splitting, mirroring, etc.) |
| **Maturity** | Stable, widely adopted | Beta/Stable, growing adoption |
| **Complexity** | Simple for basic use cases | More powerful but complex |
| **Multi-tenancy** | Limited | Built-in support |
| **Configuration** | Single Ingress resource | GatewayClass + Gateway + Routes |

---

## Architecture Diagrams

### Ingress Architecture

```
┌─────────────────┐    ┌───────────────────┐    ┌─────────────────┐
│   External      │    │   Ingress         │    │   Kubernetes    │
│   Client        │    │   Controller      │    │   Services      │
│                 │    │   (e.g., NGINX)   │    │                 │
├─────────────────┤    ├───────────────────┤    ├─────────────────┤
│                 │    │                   │    │                 │
│  HTTP Request   │───▶│  ┌─────────────┐  │───▶│  go-app:80      │
│  Host: localhost│    │  │   Ingress   │  │    │                 │
│  Path: /healthz │    │  │   Rules     │  │    │  ┌───────────┐  │
│                 │    │  │             │  │    │  │    Pod    │  │
│                 │    │  │ localhost/  │  │    │  │    Pod    │  │
│                 │    │  │ → go-app:80 │  │    │  │    Pod    │  │
│                 │    │  └─────────────┘  │    │  └───────────┘  │
└─────────────────┘    └───────────────────┘    └─────────────────┘

Flow:
1. Client sends HTTP request
2. Ingress Controller receives traffic
3. Controller applies Ingress rules
4. Traffic routed to appropriate service
5. Service forwards to pods
```

### Gateway API Architecture

```
┌─────────────────┐    ┌───────────────────┐    ┌─────────────────┐
│   External      │    │   Gateway         │    │   Kubernetes    │
│   Client        │    │   Implementation  │    │   Services      │
│                 │    │   (e.g., Envoy)   │    │                 │
├─────────────────┤    ├───────────────────┤    ├─────────────────┤
│                 │    │                   │    │                 │
│  HTTP Request   │───▶│  ┌─────────────┐  │───▶│  go-app:80      │
│  Host: localhost│    │  │   Gateway   │  │    │                 │
│  Path: /healthz │    │  │   :8080     │  │    │  ┌───────────┐  │
│                 │    │  │             │  │    │  │    Pod    │  │
│                 │    │  │ HTTPRoute   │  │    │  │    Pod    │  │
│                 │    │  │ Rules       │  │    │  │    Pod    │  │
│                 │    │  └─────────────┘  │    │  └───────────┘  │
└─────────────────┘    └───────────────────┘    └─────────────────┘

Components Relationship:
┌─────────────────┐
│  GatewayClass   │ ──┐
│  (Controller)   │   │
└─────────────────┘   │
                      ▼
┌─────────────────┐   ┌─────────────────┐
│    Gateway      │◄──│   HTTPRoute     │
│  (Listeners)    │   │   (Rules)       │
└─────────────────┘   └─────────────────┘

Flow:
1. GatewayClass defines the controller
2. Gateway creates listeners on specific ports
3. HTTPRoute defines routing rules
4. Client traffic flows through gateway to services
```

### Resource Relationships

#### Ingress Model
```
┌─────────────────┐
│  IngressClass   │
│                 │
│  controller:    │
│  k8s.io/        │
│  ingress-nginx  │
└─────────────────┘
         │
         ▼
┌─────────────────┐
│     Ingress     │
│                 │
│  ingressClass:  │
│  nginx          │
│                 │
│  rules:         │
│  - host: app.com│
│    paths: [/]   │
└─────────────────┘
```

#### Gateway API Model
```
┌─────────────────┐
│  GatewayClass   │
│                 │
│  controller:    │
│  envoy-gateway  │
└─────────────────┘
         │
         ▼
┌─────────────────┐    ┌─────────────────┐
│     Gateway     │◄───│   HTTPRoute     │
│                 │    │                 │
│  class: envoy   │    │  parentRefs:    │
│  listeners:     │    │  - gateway      │
│  - port: 8080   │    │                 │
│    protocol:HTTP│    │  rules:         │
└─────────────────┘    │  - matches: [/] │
                       └─────────────────┘
```

---

## Practical Examples

Let's examine the actual configurations from this project:

### Example 1: Basic HTTP Routing

#### Ingress Approach
```yaml
# Single resource handles everything
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-app-ingress
  namespace: routing-experiment
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  ingressClassName: nginx
  rules:
  - host: localhost
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: go-app
            port:
              number: 80
      - path: /healthz
        pathType: Exact
        backend:
          service:
            name: go-app
            port:
              number: 80
```

#### Gateway API Approach
```yaml
# 1. Gateway (Infrastructure team responsibility)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: go-app-gateway
  namespace: routing-experiment
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 8080

---
# 2. HTTPRoute (Application team responsibility)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: go-app-route
  namespace: routing-experiment
spec:
  parentRefs:
  - name: go-app-gateway
  hostnames:
  - "localhost"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: go-app
      port: 80
  - matches:
    - path:
        type: Exact
        value: /healthz
    backendRefs:
    - name: go-app
      port: 80
```

### Example 2: Advanced Traffic Management

#### Traffic Splitting (Gateway API)
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-route
spec:
  parentRefs:
  - name: go-app-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api
    backendRefs:
    - name: go-app-v1
      port: 80
      weight: 90
    - name: go-app-v2
      port: 80
      weight: 10
```

#### Header-based Routing (Gateway API)
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: header-route
spec:
  parentRefs:
  - name: go-app-gateway
  rules:
  - matches:
    - headers:
      - name: x-version
        value: v2
    backendRefs:
    - name: go-app-v2
      port: 80
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: go-app-v1
      port: 80
```

### Example 3: Multi-Protocol Support

#### Gateway API with Multiple Protocols
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: multi-protocol-gateway
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      certificateRefs:
      - name: tls-cert
  - name: grpc
    protocol: HTTP
    port: 9090

---
# HTTP routes
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: web-route
spec:
  parentRefs:
  - name: multi-protocol-gateway
    sectionName: http
  rules:
  - backendRefs:
    - name: web-service
      port: 80

---
# gRPC routes
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GRPCRoute
metadata:
  name: grpc-route
spec:
  parentRefs:
  - name: multi-protocol-gateway
    sectionName: grpc
  rules:
  - backendRefs:
    - name: grpc-service
      port: 9090
```

---

## Migration Path

### From Ingress to Gateway API

#### Step 1: Assess Current Setup
```bash
# List current ingresses
kubectl get ingress -A

# Check ingress controllers
kubectl get ingressclass
```

#### Step 2: Install Gateway API CRDs
```bash
# Install standard CRDs
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

# Install experimental CRDs (for advanced features)
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/experimental-install.yaml
```

#### Step 3: Install Gateway Controller
```bash
# Example: Envoy Gateway
kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/v1.0.0/install.yaml

# Example: Istio Gateway
istioctl install --set values.pilot.env.EXTERNAL_ISTIOD=false
```

#### Step 4: Create Equivalent Gateway Resources
```bash
# Convert existing Ingress to Gateway API
# This varies by use case and requirements
```

#### Step 5: Test and Validate
```bash
# Test routing functionality
curl -H "Host: localhost" http://gateway-ip:8080/healthz

# Compare behavior with original Ingress
```

#### Step 6: Gradual Migration
- Run both systems in parallel
- Gradually move traffic to Gateway API
- Deprecate old Ingress resources

---

## Best Practices

### Ingress Best Practices

1. **Use IngressClass**
   ```yaml
   spec:
     ingressClassName: nginx  # Always specify
   ```

2. **Avoid Controller-Specific Annotations When Possible**
   ```yaml
   annotations:
     # Prefer standard annotations
     kubernetes.io/ingress.class: nginx
   ```

3. **Implement Health Checks**
   ```yaml
   annotations:
     nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
     nginx.ingress.kubernetes.io/healthcheck-path: "/healthz"
   ```

### Gateway API Best Practices

1. **Role-Based Resource Management**
   ```yaml
   # Infrastructure team manages
   - GatewayClass
   - Gateway
   
   # Application team manages
   - HTTPRoute
   - Service
   ```

2. **Use Appropriate Route Types**
   ```yaml
   # For HTTP services
   kind: HTTPRoute
   
   # For gRPC services
   kind: GRPCRoute
   
   # For TCP services
   kind: TCPRoute
   ```

3. **Implement Cross-Namespace References Carefully**
   ```yaml
   spec:
     parentRefs:
     - name: shared-gateway
       namespace: infrastructure
   ```

4. **Use ReferenceGrant for Cross-Namespace Access**
   ```yaml
   apiVersion: gateway.networking.k8s.io/v1beta1
   kind: ReferenceGrant
   metadata:
     name: allow-routes
     namespace: infrastructure
   spec:
     from:
     - group: gateway.networking.k8s.io
       kind: HTTPRoute
       namespace: applications
     to:
     - group: ""
       kind: Service
   ```

### Security Best Practices

1. **TLS Configuration**
   ```yaml
   # Gateway API
   listeners:
   - name: https
     protocol: HTTPS
     port: 443
     tls:
       mode: Terminate
       certificateRefs:
       - name: tls-secret
   ```

2. **Network Policies**
   ```yaml
   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: gateway-access
   spec:
     podSelector:
       matchLabels:
         app: gateway
     ingress:
     - from:
       - namespaceSelector:
           matchLabels:
             name: gateway-system
   ```

### Monitoring and Observability

1. **Metrics Collection**
   ```yaml
   # Prometheus annotations
   annotations:
     prometheus.io/scrape: "true"
     prometheus.io/port: "9090"
     prometheus.io/path: "/metrics"
   ```

2. **Logging Configuration**
   ```yaml
   # Structured logging
   annotations:
     nginx.ingress.kubernetes.io/enable-access-log: "true"
   ```

---

## Testing Your Setup

### Unit Testing (Go Application)
```go
func TestHealthHandler(t *testing.T) {
    req, err := http.NewRequest("GET", "/healthz", nil)
    if err != nil {
        t.Fatal(err)
    }

    rr := httptest.NewRecorder()
    handler := http.HandlerFunc(healthHandler)
    handler.ServeHTTP(rr, req)

    if status := rr.Code; status != http.StatusOK {
        t.Errorf("wrong status code: got %v want %v", status, http.StatusOK)
    }
}
```

### Integration Testing
```bash
# Test Ingress
curl -H "Host: localhost" http://localhost/healthz

# Test Gateway API
curl -H "Host: localhost" http://localhost:8080/healthz

# Load testing
hey -n 1000 -c 10 http://localhost/healthz
```

### Validation Scripts
```bash
#!/bin/bash
# test-routing.sh

echo "Testing Ingress routing..."
INGRESS_RESPONSE=$(curl -s -H "Host: localhost" http://localhost/healthz)
echo "Ingress response: $INGRESS_RESPONSE"

echo "Testing Gateway API routing..."
GATEWAY_RESPONSE=$(curl -s -H "Host: localhost" http://localhost:8080/healthz)
echo "Gateway response: $GATEWAY_RESPONSE"

if [ "$INGRESS_RESPONSE" = "$GATEWAY_RESPONSE" ]; then
    echo "✅ Both approaches return the same response"
else
    echo "❌ Responses differ"
fi
```

---

## Performance Considerations

### Ingress Performance
- **Pros**: Mature implementations, optimized for HTTP
- **Cons**: Limited protocol support, controller-specific optimizations

### Gateway API Performance
- **Pros**: Protocol-native optimizations, better resource separation
- **Cons**: Additional resource overhead, newer implementations

### Benchmarking
```bash
# Ingress performance test
ab -n 10000 -c 100 http://localhost/

# Gateway API performance test
ab -n 10000 -c 100 http://localhost:8080/

# Compare latency and throughput
```

---

## Troubleshooting

### Common Ingress Issues

1. **Controller Not Ready**
   ```bash
   kubectl get pods -n ingress-nginx
   kubectl logs -n ingress-nginx deployment/ingress-nginx-controller
   ```

2. **Wrong IngressClass**
   ```bash
   kubectl get ingressclass
   kubectl describe ingress <name>
   ```

3. **DNS/Host Issues**
   ```bash
   # Add to /etc/hosts
   echo "127.0.0.1 localhost" >> /etc/hosts
   ```

### Common Gateway API Issues

1. **CRDs Not Installed**
   ```bash
   kubectl get crd | grep gateway
   ```

2. **Gateway Status**
   ```bash
   kubectl describe gateway <name>
   kubectl get gateway <name> -o yaml
   ```

3. **Route Configuration**
   ```bash
   kubectl describe httproute <name>
   ```

### Debug Commands
```bash
# Check gateway status
kubectl get gateways -A

# Check route status
kubectl get httproutes -A

# Check gateway class
kubectl get gatewayclasses

# Detailed route inspection
kubectl describe httproute <route-name>

# Controller logs
kubectl logs -n gateway-system deployment/gateway-controller
```

---

## Conclusion

### When to Use Ingress
- ✅ Simple HTTP/HTTPS routing needs
- ✅ Mature, well-understood requirements
- ✅ Existing infrastructure based on Ingress
- ✅ Small teams with limited complexity

### When to Use Gateway API
- ✅ Multi-protocol support needed (gRPC, TCP, UDP)
- ✅ Advanced traffic management requirements
- ✅ Large organizations with role separation
- ✅ Future-proof infrastructure investments
- ✅ Complex routing and traffic policies

### Migration Strategy
1. **Start Small**: Begin with new applications
2. **Learn Gradually**: Train teams on Gateway API concepts
3. **Parallel Implementation**: Run both systems during transition
4. **Measure Impact**: Compare performance and operational overhead
5. **Full Migration**: Move existing applications when ready

### Future Direction
The Kubernetes community is moving toward Gateway API as the standard for service networking. While Ingress remains stable and supported, Gateway API represents the future of Kubernetes networking with its:
- Enhanced expressiveness
- Protocol extensibility
- Role-oriented design
- Vendor-neutral approach

Both approaches solve real problems, and the choice depends on your specific requirements, team structure, and long-term strategy.

---

## Resources and Further Reading

### Official Documentation
- [Kubernetes Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/)
- [Gateway API](https://gateway-api.sigs.k8s.io/)
- [Gateway API Concepts](https://gateway-api.sigs.k8s.io/concepts/api-overview/)

### Implementation Guides
- [NGINX Ingress Controller](https://kubernetes.github.io/ingress-nginx/)
- [Envoy Gateway](https://gateway.envoyproxy.io/)
- [Istio Gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/)

### Tools and Utilities
- [kubectl](https://kubernetes.io/docs/reference/kubectl/)
- [kind](https://kind.sigs.k8s.io/) (Kubernetes in Docker)
- [hey](https://github.com/rakyll/hey) (HTTP load testing)
- [curl](https://curl.se/) (HTTP client)

---

## Project Repository

The complete source code and examples for this tutorial can be found in the [Kubernetes-Ingress-Gateway-API GitHub Repository](https://github.com/Yang92047111/Kubernetes-Ingress-Gateway-API).
