owned this note
owned this note
Published
Linked with GitHub
---
title: 'Resolved - Auth via Keycloak & traefik forwar-auth proxy eng'
disqus: hackmd
---
Auth via Keycloak & traefik forwar-auth proxy
===
---
[TOC]
## Resolved
resolved using the configuration from https://github.com/sleighzy/k3s-traefik-forward-auth-openid-connect
## Overview
If an application isn't able to authenticate users itself it should be done via OIDC applications.
## The proposed scheme
---
```mermaid
sequenceDiagram
participant client
participant proxy
participant kc
participant app
client ->> app:
app ->> proxy: redirect via middleware if not auth-ed
proxy ->> kc: ask login/pw
kc ->> proxy: check if user login succesfully
proxy ->> client:
client ->> app:
note over proxy: traefik forward-auth proxy
note over kc: keycloak
```
## Resources
| component | url | details |
|:---------------------------:| ------------------------------------- | ------------------------------------------ |
| traefik forward-auth proxy | https://auth.team.stage.company.com/ | - |
| kc | keycloak.team.stage.company.com | realm - `internal-users`; client - `proxy` |
| app | https://myapp.team.stage.company.com/ | - |
### proxy manifests
<details>
```yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: traefik-forward-auth
namespace: stage
labels:
app: traefik-forward-auth
spec:
selector:
matchLabels:
app: traefik-forward-auth
template:
metadata:
labels:
app: traefik-forward-auth
spec:
terminationGracePeriodSeconds: 60
containers:
- image: mesosphere/traefik-forward-auth:3.0.3
name: traefik-forward-auth
ports:
- containerPort: 4181
protocol: TCP
env:
- name: PROVIDER_URI
value: https://keycloak.team.stage.company.com/auth/realms/internal-users
- name: CLIENT_ID
value: proxy
- name: CLIENT_SECRET
valueFrom:
secretKeyRef:
name: traefik-forward-auth
key: CLIENT_SECRET
- name: SECRET
valueFrom:
secretKeyRef:
name: traefik-forward-auth
key: SECRET
# experimental
- name: AUTH_HOST
value: "auth.team.stage.company.com"
- name: COOKIE_DOMAIN
value: "team.stage.company.com"
- name: LOG_LEVEL
value: trace
# resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: traefik-forward-auth
namespace: stage
labels:
app: traefik-forward-auth
spec:
type: ClusterIP
selector:
app: traefik-forward-auth
ports:
- name: auth-http
port: 4181
targetPort: 4181
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: traefik-forward-auth
namespace: stage
spec:
forwardAuth:
address: http://traefik-forward-auth.stage:4181
trustForwardHeader: true
authResponseHeaders:
- X-Forwarded-User
---
apiVersion: v1
kind: Service
metadata:
name: traefik-forward-auth-ui-external
namespace: stage
annotations:
external-dns.alpha.kubernetes.io/hostname: auth.team.stage.company.team
spec:
type: ExternalName
externalName: lb-traefik.team.stage.company.team
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-forward-auth-http
namespace: stage
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`auth.team.stage.company.com`)
middlewares:
- name: redirect-https
namespace: traefik
services:
- kind: Service
name: traefik-forward-auth
port: 4181
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-forward-auth-https
namespace: stage
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`auth.team.stage.company.com`)
services:
- kind: Service
name: traefik-forward-auth
port: 4181
tls:
secretName: wildcard-stage-team-company-com
```
</details>
### app manifests
<details>
```yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: stage
spec:
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: stage
spec:
selector:
app: myapp
ports:
- port: 80
targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: myapp-ui-external
namespace: stage
annotations:
external-dns.alpha.kubernetes.io/hostname: myapp.team.stage.company.com
spec:
type: ExternalName
externalName: lb-traefik.team.stage.company.com
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myapp-http
namespace: stage
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`myapp.team.stage.company.com`)
middlewares:
- name: redirect-https
namespace: traefik
services:
- kind: Service
name: myapp
port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myapp-https
namespace: stage
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`myapp.team.stage.company.com`)
middlewares:
- name: traefik-forward-auth
namespace: stage
services:
- kind: Service
name: myapp
port: 80
tls:
secretName: wildcard-stage-team-company-com
```
</details>
### kc
|k |v |
|:-------------------:|--------------------------------------------|
| realm | internal-users |
| client | proxy |
| secret | MyClientAuthSecret |
| Valid Redirect URIs | https://auth.team.stage.company.com/_oauth |
## Desired State
1. Client sends a request to `app`'s url.
2. Middleware `proxy` is attached to `app`'s `IngressRoute`. It tweaks the client request, before it gets to `app`. The client is going to be redirected to `proxy`'s `Service`.
3. If client is not authenticated in OIDC (in our case it is Keycloak) `proxy` redirects the request to `kc` **realm** endpoint. If client can be identified via login / pass, and they have permissions to access `app` (keycloak's client scope), `kc` extends the request payload with some user info and cookies (csrf token).
4. After it -> `kc` will redirect the request back to `proxy`'s special path -> `/_oauth`.
5. `proxy` will see cookie in the request payload and will finally redirect the client to the `app`.
## Current State
I have an issue at the step **5** - **proxy** instead of redirection client to `app` **origin**, it redirects the request back to `kc`, and it gets to infinite loop till the request falls with `ERR_TO_MANY_REDIRECTS` error.
## Logs
### kc
No useful logs in `kc` with severity `INFO`
I have session in `kc`:
[Session](https://ah-public-pictures.hb.bizmrg.com/nx/kc-client-proxy.png)
### proxy
```shell
time="2022-03-03T15:29:41Z" level=debug msg="Authenticate request" headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-us] Cookie:[_forward_auth_csrf=MySecretToken] User-Agent:[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15] X-Forwarded-For:[94.X.X.X] X-Forwarded-Host:[auth.team.stage.company.com] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[traefik-7b758f795-j2x9w] X-Real-Ip:[94.X.X.X]]" rule=default source_ip=94.X.X.X
time="2022-03-03T15:29:41Z" level=debug msg="sending CSRF cookie and a redirect to OIDC login" source_ip=94.X.X.X
```
## browserdata
### kc -> proxy json
<details>
```json
{
"_initiator": {
"type": "other"
},
"_priority": "VeryHigh",
"_resourceType": "document",
"cache": {},
"connection": "926909",
"request": {
"method": "POST",
"url": "https://keycloak.team.stage.company.com/auth/realms/internal-users/login-actions/authenticate?session_code=<SESSION_CODE>&execution=<EXECUTION>&client_id=proxy&tab_id=<TAB_ID>",
"httpVersion": "HTTP/1.1",
"headers": [
{
"name": "Host",
"value": "keycloak.team.stage.company.com"
},
{
"name": "Connection",
"value": "keep-alive"
},
{
"name": "Content-Length",
"value": "68"
},
{
"name": "Cache-Control",
"value": "max-age=0"
},
{
"name": "sec-ch-ua",
"value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google Chrome\";v=\"98\""
},
{
"name": "sec-ch-ua-mobile",
"value": "?0"
},
{
"name": "sec-ch-ua-team",
"value": "\"macOS\""
},
{
"name": "Upgrade-Insecure-Requests",
"value": "1"
},
{
"name": "Origin",
"value": "null"
},
{
"name": "Content-Type",
"value": "application/x-www-form-urlencoded"
},
{
"name": "User-Agent",
"value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.XX.XX.XX Safari/537.36"
},
{
"name": "Accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
},
{
"name": "Sec-Fetch-Site",
"value": "same-origin"
},
{
"name": "Sec-Fetch-Mode",
"value": "navigate"
},
{
"name": "Sec-Fetch-User",
"value": "?1"
},
{
"name": "Sec-Fetch-Dest",
"value": "document"
},
{
"name": "Accept-Encoding",
"value": "gzip, deflate, br"
},
{
"name": "Accept-Language",
"value": "en-US,en;q=0.9"
},
{
"name": "Cookie",
"value": "AUTH_SESSION_ID=<AUTH_SESSION_ID>.keycloak-0; AUTH_SESSION_ID_LEGACY=<AUTH_SESSION_ID>.keycloak-0; KC_RESTART=<KC_RESTART>; traefik-sticky=http://10.50.37.3:8080"
}
],
"queryString": [
{
"name": "session_code",
"value": "<SESSION_CODE>"
},
{
"name": "execution",
"value": "<EXECUTION>"
},
{
"name": "client_id",
"value": "proxy"
},
{
"name": "tab_id",
"value": "<TAB_ID>"
}
],
"cookies": [
{
"name": "AUTH_SESSION_ID",
"value": "<AUTH_SESSION_ID>.keycloak-0",
"path": "/auth/realms/internal-users/",
"domain": "keycloak.team.stage.company.com",
"expires": "1969-12-31T23:59:59.000Z",
"httpOnly": true,
"secure": true,
"sameSite": "None"
},
{
"name": "AUTH_SESSION_ID_LEGACY",
"value": "<AUTH_SESSION_ID>.keycloak-0",
"path": "/auth/realms/internal-users/",
"domain": "keycloak.team.stage.company.com",
"expires": "1969-12-31T23:59:59.000Z",
"httpOnly": true,
"secure": true
},
{
"name": "KC_RESTART",
"value": "<KC_RESTART>",
"path": "/auth/realms/internal-users/",
"domain": "keycloak.team.stage.company.com",
"expires": "1969-12-31T23:59:59.000Z",
"httpOnly": true,
"secure": true
},
{
"name": "traefik-sticky",
"value": "http://10.50.37.3:8080",
"path": "/",
"domain": "keycloak.team.stage.company.com",
"expires": "1969-12-31T23:59:59.000Z",
"httpOnly": false,
"secure": true
}
],
"headersSize": 1892,
"bodySize": 68,
"postData": {
"mimeType": "application/x-www-form-urlencoded",
"text": "username=a.horbach%40company.com&password=PASSWORD&credentialId=",
"params": [
{
"name": "username",
"value": "a.horbach%40company.com"
},
{
"name": "password",
"value": "PASSWORD"
},
{
"name": "credentialId",
"value": ""
}
]
}
},
"response": {
"status": 302,
"statusText": "Found",
"httpVersion": "HTTP/1.1",
"headers": [
{
"name": "Cache-Control",
"value": "no-store, must-revalidate, max-age=0"
},
{
"name": "Content-Length",
"value": "0"
},
{
"name": "Content-Security-Policy",
"value": "frame-src 'self'; frame-ancestors 'self'; object-src 'none';"
},
{
"name": "Date",
"value": "Mon, 07 Mar 2022 16:08:07 GMT"
},
{
"name": "Location",
"value": "https://auth.team.stage.company.com/_oauth?state=<STATE>%3Ahttps%3A%2F%2Fmyapp.team.stage.company.com%2F&session_state=<AUTH_SESSION_ID>&code=<CODE>.<AUTH_SESSION_ID>.<SOME_SECRET>"
},
{
"name": "P3p",
"value": "CP=\"This is not a P3P policy!\""
},
{
"name": "Referrer-Policy",
"value": "no-referrer"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_LOCALE=; Version=1; Comment=Expiring cookie; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/internal-users/; Secure; HttpOnly"
},
{
"name": "Set-Cookie",
"value": "KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/internal-users/; Secure; HttpOnly"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_IDENTITY=<KEYCLOAK_IDENTITY>; Version=1; Path=/auth/realms/internal-users/; SameSite=None; Secure; HttpOnly"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_IDENTITY_LEGACY=<KEYCLOAK_IDENTITY>; Version=1; Path=/auth/realms/internal-users/; Secure; HttpOnly"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_SESSION=internal-users/<KEYCLOAK_SESSION>/<AUTH_SESSION_ID>; Version=1; Expires=Thu, 10-Mar-2022 16:08:07 GMT; Max-Age=259200; Path=/auth/realms/internal-users/; SameSite=None; Secure"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_SESSION_LEGACY=internal-users/<KEYCLOAK_SESSION>/<AUTH_SESSION_ID>; Version=1; Expires=Thu, 10-Mar-2022 16:08:07 GMT; Max-Age=259200; Path=/auth/realms/internal-users/; Secure"
},
{
"name": "Set-Cookie",
"value": "KEYCLOAK_REMEMBER_ME=; Version=1; Comment=Expiring cookie; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/internal-users/; Secure; HttpOnly"
},
{
"name": "Strict-Transport-Security",
"value": "max-age=31536000; includeSubDomains"
},
{
"name": "X-Content-Type-Options",
"value": "nosniff"
},
{
"name": "X-Frame-Options",
"value": "SAMEORIGIN"
},
{
"name": "X-Robots-Tag",
"value": "none"
},
{
"name": "X-Xss-Protection",
"value": "1; mode=block"
}
],
"cookies": [
{
"name": "KEYCLOAK_LOCALE",
"value": "",
"path": "/auth/realms/internal-users/",
"domain": "keycloak.team.stage.company.com",
"expires": "2022-03-07T16:08:07.690Z",
"httpOnly": true,
"secure": true,
"sameSite": "None"
}
],
"content": {
"size": 0,
"mimeType": "x-unknown",
"compression": 0
},
"redirectURL": "https://auth.team.stage.company.com/_oauth?state=<STATE>%3Ahttps%3A%2F%2Fmyapp.team.stage.company.com%2F&session_state=<AUTH_SESSION_ID>&code=<CODE>.<AUTH_SESSION_ID>.<SOME_SECRET>",
"headersSize": 3284,
"bodySize": 0,
"_transferSize": 3284,
"_error": null
},
"serverIPAddress": "34.251.109.144",
"startedDateTime": "2022-03-07T16:08:07.683Z",
"time": 253.26799997128546,
"timings": {
"blocked": 7.564999960236252,
"dns": -1,
"ssl": -1,
"connect": -1,
"send": 0.12899999999999995,
"wait": 243.43899999404326,
"receive": 2.13500001700595,
"_blocked_queueing": 7.103999960236251
}
}
```
</details>
### proxy -> kc json
<details>
```json
{
"_initiator": {
"type": "other"
},
"_priority": "VeryHigh",
"_resourceType": "document",
"cache": {},
"connection": "927096",
"request": {
"method": "GET",
"url": "https://auth.team.stage.company.com/_oauth?state=<STATE>%3Ahttps%3A%2F%2Fmyapp.team.stage.company.com%2F&session_state=<SESSION_STATE>&code=<CODE>.<SESSION_STATE>.<SOME_SECRET>",
"httpVersion": "HTTP/1.1",
"headers": [
{
"name": "Host",
"value": "auth.team.stage.company.com"
},
{
"name": "Connection",
"value": "keep-alive"
},
{
"name": "Cache-Control",
"value": "max-age=0"
},
{
"name": "Upgrade-Insecure-Requests",
"value": "1"
},
{
"name": "User-Agent",
"value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.XX.XX.XX Safari/537.36"
},
{
"name": "Accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
},
{
"name": "Sec-Fetch-Site",
"value": "same-site"
},
{
"name": "Sec-Fetch-Mode",
"value": "navigate"
},
{
"name": "Sec-Fetch-User",
"value": "?1"
},
{
"name": "Sec-Fetch-Dest",
"value": "document"
},
{
"name": "sec-ch-ua",
"value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google Chrome\";v=\"98\""
},
{
"name": "sec-ch-ua-mobile",
"value": "?0"
},
{
"name": "sec-ch-ua-team",
"value": "\"macOS\""
},
{
"name": "Accept-Encoding",
"value": "gzip, deflate, br"
},
{
"name": "Accept-Language",
"value": "en-US,en;q=0.9"
},
{
"name": "Cookie",
"value": "_forward_auth_csrf=<STATE>"
}
],
"queryString": [
{
"name": "state",
"value": "<STATE>%3Ahttps%3A%2F%2Fmyapp.team.stage.company.com%2F"
},
{
"name": "session_state",
"value": "<SESSION_STATE>"
},
{
"name": "code",
"value": "<CODE>.<SESSION_STATE>.<SOME_SECRET>"
}
],
"cookies": [
{
"name": "_forward_auth_csrf",
"value": "<STATE>",
"path": "/",
"domain": ".team.stage.company.com",
"expires": "2022-03-08T04:07:41.865Z",
"httpOnly": true,
"secure": true
}
],
"headersSize": 1044,
"bodySize": 0
},
"response": {
"status": 302,
"statusText": "Found",
"httpVersion": "HTTP/1.1",
"headers": [
{
"name": "Content-Length",
"value": "0"
},
{
"name": "Date",
"value": "Mon, 07 Mar 2022 16:08:08 GMT"
},
{
"name": "Location",
"value": "https://keycloak.team.stage.company.com/auth/realms/internal-users/protocol/openid-connect/auth?client_id=proxy&redirect_uri=https%3A%2F%2Fauth.team.stage.company.com%2F_oauth&response_type=code&scope=openid+profile+email+groups&state=<FORWARD_AUTH_CSRF>%3Ahttps%3A%2F%2Fauth.team.stage.company.com%2F"
},
{
"name": "Set-Cookie",
"value": "_forward_auth_csrf=<FORWARD_AUTH_CSRF>; Path=/; Domain=team.stage.company.com; Expires=Tue, 08 Mar 2022 04:08:08 GMT; HttpOnly; Secure"
}
],
"cookies": [
{
"name": "_forward_auth_csrf",
"value": "<FORWARD_AUTH_CSRF>",
"path": "/",
"domain": "team.stage.company.com",
"expires": "2022-03-08T04:08:08.000Z",
"httpOnly": true,
"secure": true
}
],
"content": {
"size": 0,
"mimeType": "x-unknown",
"compression": 0
},
"redirectURL": "https://keycloak.team.stage.company.com/auth/realms/internal-users/protocol/openid-connect/auth?client_id=proxy&redirect_uri=https%3A%2F%2Fauth.team.stage.company.com%2F_oauth&response_type=code&scope=openid+profile+email+groups&state=<FORWARD_AUTH_CSRF>%3Ahttps%3A%2F%2Fauth.team.stage.company.com%2F",
"headersSize": 585,
"bodySize": 0,
"_transferSize": 585,
"_error": null
},
"serverIPAddress": "IP",
"startedDateTime": "2022-03-07T16:08:07.939Z",
"time": 364.34300001596284,
"timings": {
"blocked": 3.593000016462058,
"dns": 54.203,
"ssl": 95.025,
"connect": 223.143,
"send": 0.1769999999999925,
"wait": 82.00700002304464,
"receive": 1.2199999764561653,
"_blocked_queueing": 3.251000016462058
}
}
```
</details>
## Links
- https://doc.traefik.io/traefik/middlewares/http/forwardauth/
- https://brianturchyn.net/traefik-forwardauth-support-with-keycloak/
- https://geek-cookbook.funkypenguin.co.nz/ha-docker-swarm/traefik-forward-auth/
- https://itnext.io/how-to-implement-a-sso-middleware-for-traefik-v2-on-kubernetes-dcd9d45cc875
- https://github.com/thomseddon/traefik-forward-auth