MUNGE - HackMD

MUNGE === ###### tags: `K8s / app / Slurm Operator` ###### tags: `Kubernetes`, `k8s`, `app`, `Slurm`, `MUNGE` [TOC] ## Guide ### 文件源頭 - [Slurm Workload Manager - Documentation](https://slurm.schedmd.com/documentation.html) - [Installation Guide](https://slurm.schedmd.com/quickstart_admin.html) - [Super Quick Start](https://slurm.schedmd.com/quickstart_admin.html#quick_start) Install [MUNGE](https://dun.github.io/munge/) for authentication. Make sure that all nodes in your cluster have the same munge.key. Make sure the MUNGE daemon, munged, is started before you start the Slurm daemons. ### [MUNGE](https://dun.github.io/munge/) > MUNGE (MUNGE Uid 'N' Gid Emporium) is an authentication service for creating and validating credentials. It is designed to be highly scalable for use in an HPC cluster environment. It allows a process to authenticate the UID and GID of another local or remote process within a group of hosts having common users and groups. These hosts form a security realm that is defined by a shared cryptographic key. Clients within this security realm can create and validate credentials without the use of root privileges, reserved ports, or platform-specific methods. > MUNGE（MUNGE Uid 'N' Gid Emporium）是一種驗證服務，用於建立和驗證憑證。它專為高效能運算（HPC）叢集環境中的高度擴充性而設計。它允許一個程序驗證另一個本地或遠端程序的使用者識別碼（UID）與群組識別碼（GID），前提是這些程序所在的主機屬於擁有共用使用者與群組的群組主機。這些主機構成一個「安全領域」，此領域是透過共用的加密金鑰所定義。在這個安全領域中的用戶端，無需使用 root 權限、保留連接埠或平台特定的方法，就能建立並驗證憑證。 ## 指令用法 ### munged ``` # munged -h Usage: munged [OPTIONS] -h, --help Display this help -L, --license Display license information -V, --version Display version information -f, --force Force daemon to run if possible -F, --foreground Run daemon in the foreground (do not fork) -M, --mlockall Lock all pages in memory -s, --stop Stop daemon bound to socket -S, --socket=PATH Specify local socket [/run/munge/munge.socket.2] -v, --verbose Be verbose --benchmark Disable timers to reduce noise while benchmarking --group-check-mtime=BOOL Specify whether to check "/etc/group" mtime [1] --group-update-time=INT Specify seconds between group info updates [3600] --key-file=PATH Specify key file [/etc/munge/munge.key] --log-file=PATH Specify log file [/var/log/munge/munged.log] --max-ttl=INT Specify maximum time-to-live (in seconds) [3600] --num-threads=INT Specify number of threads to spawn [2] --origin=ADDRESS Specify origin address via hostname/IPaddr/interface --pid-file=PATH Specify PID file [/run/munge/munged.pid] --seed-file=PATH Specify PRNG seed file [/var/lib/munge/munged.seed] --syslog Redirect log messages to syslog --trusted-group=GROUP Specify trusted group/GID for directory checks ``` ### munge ``` $ munge -h Usage: munge [OPTIONS] -h, --help Display this help -L, --license Display license information -V, --version Display version information -n, --no-input Discard all input for payload -s, --string=STRING Input payload from STRING -i, --input=FILE Input payload from FILE -o, --output=FILE Output credential to FILE -c, --cipher=STRING Specify cipher type -C, --list-ciphers Display a list of supported ciphers -m, --mac=STRING Specify MAC type -M, --list-macs Display a list of supported MACs -z, --zip=STRING Specify compression type -Z, --list-zips Display a list of supported compressions -u, --restrict-uid=UID Restrict credential decoding by UID -U, --uid=UID Specify credential UID (requires root/CAP_SETUID) -g, --restrict-gid=GID Restrict credential decoding by GID -G, --gid=GID Specify credential GID (requires root/CAP_SETGID) -t, --ttl=INTEGER Specify time-to-live (in seconds; 0=dfl -1=max) -S, --socket=STRING Specify local domain socket for munged By default, payload read from stdin, credential written to stdout. ``` ### unmunge ``` # unmunge -h Usage: unmunge [OPTIONS] -h, --help Display this help -L, --license Display license information -V, --version Display version information -i, --input=FILE Input credential from FILE -n, --no-output Discard all output -m, --metadata=FILE Output metadata to FILE -o, --output=FILE Output payload to FILE -k, --keys=STRING Specify subset of metadata keys to output -K, --list-keys Display list of metadata keys -N, --numeric Display metadata values numerically -S, --socket=STRING Specify local domain socket for munged By default, credential read from stdin, metadata & payload written to stdout. ``` ## 參考資料 - [munge认证服务配置-集群搭建步骤4](https://blog.csdn.net/xhk12345678/article/details/124710893) > munge是一項認證服務，用於產生和驗證憑證。應用於大規模的HPC叢集中，slurm官方推薦使用 munge 作為 slurm 作業排程系統的認證服務，用於主控節點和運算節點之間的認證通訊。設定時分為 server 端和 client 端，其中 server 端應作為 slurm 調度服務的主控節點，用於管理認證服務。 --- --- # soperator/munge > https://github.com/nebius/soperator/pkgs/container/soperator%2Fmunge ## For docker ### pull image ```bash $ docker pull ghcr.io/nebius/soperator/munge:1.19.0-jammy-slurm24.05.5 ``` ### run image ``` $ docker run --rm -it --entrypoint="" ghcr.io/nebius/soperator/munge:1.19.0-jammy-slurm24.05.5 bash ``` - 要加 `--entrypoint` 參數來跳過原本的 init 腳本 - 沒有覆寫 `--entrypoint`，會遇到： ``` Link users from jail Bind-mount munge key from K8S secret mount: /etc/munge/munge.key: mount point is not a directory. ``` - `I have no name!@stage-kube01:/#` -> 如何改成 root name? ```bash # 建立最簡單版本的 /etc/passwd 和 /etc/group echo 'root:x:0:0:root:/root:/bin/bash' > /etc/passwd echo 'root:x:0:' > /etc/group ``` - 查看 `/etc/munge/munge.key` ``` $ ll /etc/munge total 12 drwx------ 2 root root 4096 Feb 26 09:07 ./ drwxr-xr-x 1 root root 4096 Jun 3 06:13 ../ -rwx------ 1 root root 128 Feb 26 09:07 munge.key* ``` - 查看 munge & unmunge ``` $ whereis munge munge: /usr/bin/munge /etc/munge $ whereis unmunge unmunge: /usr/bin/unmunge ``` ### start munged ```bash # 建立 socket 目錄 $ mkdir -p /run/munge # 確保 /etc/group 存在 $ touch /etc/group # 若 minimal 系統沒有此檔，可臨時建空檔案 # 前景模式方便看 log $ munged --foreground --log-file /dev/stdout ``` - logs (執行 unmunge ，驗證失敗的 log) ``` munged: Notice: Starting munge-0.5.14 daemon (pid 18) munged: Info: Set origin address to 172.17.0.2 (eth0) munged: Info: Seeded PRNG with 128 bytes from getrandom() munged: Info: Seeded PRNG with 1024 bytes from "/var/lib/munge/munged.seed" munged: Info: Updating supplementary group mapping every 3600 seconds munged: Info: Enabled supplementary group mtime check of "/etc/group" munged: Info: Created socket "/run/munge/munge.socket.2" munged: Info: Found 0 users with supplementary groups in 0.000 seconds munged: Info: Created 2 work threads munged: Info: Failed to match armor prefix munged: Info: Failed to base64-decode credential munged: Info: Invalid credential version 70 munged: Info: Invalid credential version 246 munged: Info: Invalid credential version 7 munged: Info: Replayed credential from 172.17.0.2 munged: Info: Failed to base64-decode credential munged: Info: Invalid credential munged: Info: Invalid credential munged: Info: Failed to base64-decode credential munged: Info: Replayed credential from 172.17.0.2 munged: Info: Expired credential from 172.17.0.2 munged: Info: Expired credential from 172.17.0.2 ``` ### test munge & unmunge ```bash $ munge -n MUNGE:AwQFAAA2v0O7sBhYQpdrFynuAqg6FEuYQ+jUQWYfhNZt1fDyfSNl+SDGfiJockY9++FMOzk6qFsbHv8uUTMI4ZncHV3z0cutrwPoFfF4UeGxeT2zq2IZcKAp3n8zv+LwCteHOgI=: $ munge -n | unmunge STATUS: Success (0) ENCODE_HOST: 0540fa5f5042 (172.17.0.2) ENCODE_TIME: 2025-06-03 07:07:45 +0000 (1748934465) DECODE_TIME: 2025-06-03 07:07:45 +0000 (1748934465) TTL: 300 CIPHER: aes128 (4) MAC: sha256 (5) ZIP: none (0) UID: ??? (0) GID: ??? (0) LENGTH: 0 # unmunge 錯誤訊息 unmunge: Error: Invalid credential ``` - **經過 MUNGE 加密封裝之後的結果，它包含了：** - 加密後的 payload - 憑證的 metadata（如 encode timestamp、TTL、UID、GID、MAC、壓縮等） - 一組 base64 編碼後的封裝格式通常儲存成檔案或透過管道傳輸，用於身份驗證流程 - **MUNGE 產生的憑證是「有壽命限制的」。每個憑證都內含：** - 加密資料（payload） - 發行時間（encode timestamp） - 有效期限（TTL，Time-To-Live） - **如何避免憑證過期？** - 用 `--ttl` 參數指定存活時間（單位：秒）： ``` echo "Hello, Soperator!" | munge --ttl=600 > test.cred ``` 這樣憑證就會在 10 分鐘內有效（預設時間通常是幾分鐘）。 - **名詞解釋** | 名稱 | 意思 | |-----|-----| | payload | 原始要加密的資料（e.g., "Hello, world!"） | | credential | MUNGE 加密後的字串（包含 metadata） | | token（非官方）| 有時人們口語上也會說成 token，但不準確 | ## ctr command ``` $ sudo ctr -n k8s.io run --rm -t ghcr.io/nebius/soperator/munge:1.19.0-jammy-slurm24.05.5 my-munge bash ``` --- --- # 自行準備 Dockerfile (Installing from git) ```dockerfile= # Use Ubuntu as the base image FROM ubuntu:22.04 # Prevent interactive prompts during package installation ENV DEBIAN_FRONTEND=noninteractive # Install build tools and dependencies including sudo RUN apt update && apt install -y \ sudo \ git \ build-essential \ autoconf \ automake \ libtool \ pkg-config \ libssl-dev \ python3-docutils \ ca-certificates \ && rm -rf /var/lib/apt/lists/* # Create system group and user for MUNGE RUN groupadd --system munge && \ useradd --system --gid munge --create-home --home-dir /var/lib/munge --shell /usr/sbin/nologin munge && \ echo "munge ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/munge && \ chmod 0440 /etc/sudoers.d/munge # Clone and build MUNGE from source # Set working directory WORKDIR /opt # Clone and build MUNGE from source RUN git clone https://github.com/dun/munge.git && \ cd munge && \ ./bootstrap && \ ./configure \ --prefix=/usr \ --sysconfdir=/etc \ --localstatedir=/var \ --runstatedir=/run && \ make && \ make check && \ make install # Create necessary directories and set ownership/permissions RUN mkdir -p /etc/munge /var/log /var/lib/munge /run/munge && \ chown -R munge:munge /etc/munge /var/log /var/lib/munge /var/lib/munge /run/munge && \ chmod 0700 /etc/munge /var/lib/munge && chmod 0711 /run/munge # Create the MUNGE key using the official tool and correct user RUN sudo -u munge /usr/sbin/mungekey --verbose && \ chmod 0600 /etc/munge/munge.key # Switch to the munge user USER munge WORKDIR /home/munge # Expose volume for socket if used externally VOLUME ["/run/munge"] # Run munged in foreground for containerized use CMD ["sudo", "-u", "munge", "/usr/sbin/munged", "--foreground", "--log-file", "/var/log/munged.log"] ``` --- --- # README.md # MUNGE Docker Image This repository provides a Docker image for [MUNGE (MUNGE Uid 'N' Gid Emporium)](https://github.com/dun/munge), built from source using the official installation guide. It is useful for authentication within HPC, Slurm, or other distributed systems. ## ✅ Features - Built from latest MUNGE source on GitHub - Follows [official installation instructions](https://github.com/dun/munge/wiki/Installation-Guide) - Includes: - `munge` system user - `sudo` access for `munge` - Custom-generated `/etc/munge/munge.key` - Foreground execution of `munged` for container use --- ## 🧱 Build Instructions Clone this repository and build the image: ``` docker build -t munge:0.5.15 . ``` --- ## 🚀 Usage Run the container: ``` docker run --rm -it munge:0.5.15 ``` Or run in detached mode: ``` docker run -d --name munge-test munge:0.5.15 ``` --- ## 🧪 Test MUNGE Once inside the container (or using `docker exec`): ``` munge -n | unmunge ``` You should see output similar to: ``` STATUS: Success (0) ENCODE_HOST: 7fb7cb0a64f6 (172.17.0.4) ENCODE_TIME: 2025-05-28 10:09:55 +0000 (1748426995) DECODE_TIME: 2025-05-28 10:09:55 +0000 (1748426995) TTL: 300 CIPHER: aes128 (4) MAC: sha256 (5) ZIP: none (0) UID: munge (999) GID: munge (999) LENGTH: 0 ``` To test across terminals: ``` munge -n > token unmunge < token ``` --- ## 🛠️ Container Details - Base image: `ubuntu:22.04` - Default user: `munge` (with `sudo` access) - Log output: `/var/log/munged.log` - MUNGE socket directory: `/run/munge` (exposed as volume) --- ## 📦 Notes - `munged` runs in foreground (`--foreground`) for compatibility with Docker. - The `/etc/munge/munge.key` is auto-generated at build time and owned by `munge:munge`. --- ## 🔒 Security Notice If you plan to use this in production or across multiple nodes: - Share the same `munge.key` file across nodes. - Keep `/etc/munge/munge.key` secret and with strict `0400` permissions. --- ## 🔗 Related - [MUNGE GitHub](https://github.com/dun/munge) - [Official Installation Guide](https://github.com/dun/munge/wiki/Installation-Guide) --- ## 📜 License This Dockerfile and related files are provided under MIT license. MUNGE itself is licensed under GPLv3. {%hackmd vaaMgNRPS4KGJDSFG0ZE0w %}