---
# System prepended metadata

title: 2021-01-06 Linux 5.11 SRv6 End.DT4

---

---
lang: ja
breaks: false
---

<style>
.ui-infobar, #doc.markdown-body { max-width: 1100px; }
</style>

# 2021-01-06 Linux 5.11 SRv6 End.DT4


- 参加者: hayakawa, kawakami, saito, slankdev
- kernel: 5.11
- tinet: https://github.com/tinynetwork/tinet/tree/master/examples/basic_srv6/linux/vpn_v4_per_vrf
- 最新のiproute2が必要だった。slankdev/frrには入ってないので注意

- ポイント
    - net.vrf.strict_mode=1
        - vrfごとに, table番号を重複できないようにする?
        - (1)重複するケース
    - vrftable, tableの違い
        - vrftable: vrfと紐づいたtableでしかやる
        - table: vrfに関わらず routing table
        - End.DT4はvrftableのみ
        - End.DT6はvrftable,tableどちらもつかえるよ
    - (少し逸れるが) vrfでtcpdumpできるようになった

(1)重複するケース
```
ip link add vrf1 table 100
ip link add vrf2 table 100

net.vrf.strict_mode=1にすると
ip link add vrf1 table 100
ip link add vrf2 table 100 <--これが設定失敗する?
```

- 検証項目
    - (1) VPNv4 per VRF
    - (2) End.DT4 を設定したのち vrf deleteすると?
        - きっと, tableは残るから, end.dt4はあるまんまかな..

## tinet 環境
![](https://i.imgur.com/av29WyU.png)

```
nodes:
  - name: R1
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R2#net0 }
      - { name: net1, type: direct, args: C1#net0 }
      - { name: net2, type: direct, args: C10#net0 }
  - name: R2
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R1#net0 }
      - { name: net1, type: direct, args: C2#net0 }
      - { name: net2, type: direct, args: C20#net0 }
  - name: C1
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R1#net1 }
  - name: C2
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R2#net1 }
  - name: C10
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R1#net2 }
  - name: C20
    image: slankdev/frr
    interfaces:
      - { name: net0, type: direct, args: R2#net2 }

node_configs:

  - name: R1
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.forwarding=1'
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.all.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.default.forwarding=1'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.lo.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.net0.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.net1.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv4.conf.all.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.default.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.lo.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.net0.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.net1.rp_filter=0'
      - cmd: sysctl -w 'net.vrf.strict_mode=1'
      - cmd: ip -6 addr add fc00:1::1/64 dev lo
      - cmd: ip -6 addr add 2001:12::1/64 dev net0
      - cmd: ip link add vrf101 type vrf table 101
      - cmd: ip link add vrf110 type vrf table 110
      - cmd: ip link set vrf101 up
      - cmd: ip link set vrf110 up
      - cmd: ip link set dev net1 master vrf101
      - cmd: ip link set dev net2 master vrf110
      - cmd: ip -4 addr add 10.0.1.1/24 dev net1
      - cmd: ip -4 addr add 10.0.1.1/24 dev net2
      - cmd: ip sr tunsrc set fc00:1::1
      - cmd: ip route add fc00:2::/64 via 2001:12::2
      - cmd: ip -4 route add 10.0.2.0/24 encap seg6 mode encap segs fc00:2::101 dev net0 table 101
      - cmd: ip -4 route add 10.0.2.0/24 encap seg6 mode encap segs fc00:2::110 dev net0 table 110
      - cmd: ip -6 route add fc00:1::101/128 encap seg6local action End.DT4 vrftable 101 dev net0
      - cmd: ip -6 route add fc00:1::110/128 encap seg6local action End.DT4 vrftable 110 dev net0

  - name: R2
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.forwarding=1'
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.all.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.default.forwarding=1'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.lo.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.net0.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv6.conf.net1.seg6_enabled=1'
      - cmd: sysctl -w 'net.ipv4.conf.all.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.default.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.lo.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.net0.rp_filter=0'
      - cmd: sysctl -w 'net.ipv4.conf.net1.rp_filter=0'
      - cmd: sysctl -w 'net.vrf.strict_mode=1'
      - cmd: ip -6 addr add fc00:2::1/64 dev lo
      - cmd: ip -6 addr add 2001:12::2/64 dev net0
      - cmd: ip link add vrf101 type vrf table 101
      - cmd: ip link add vrf110 type vrf table 110
      - cmd: ip link set vrf101 up
      - cmd: ip link set vrf110 up
      - cmd: ip link set dev net1 master vrf101
      - cmd: ip link set dev net2 master vrf110
      - cmd: ip -4 addr add 10.0.2.1/24 dev net1
      - cmd: ip -4 addr add 10.0.2.1/24 dev net2
      - cmd: ip sr tunsrc set fc00:2::1
      - cmd: ip route add fc00:1::/64 via 2001:12::1
      - cmd: ip -4 route add 10.0.1.0/24 encap seg6 mode encap segs fc00:1::101 dev net0 table 101
      - cmd: ip -4 route add 10.0.1.0/24 encap seg6 mode encap segs fc00:1::110 dev net0 table 110
      - cmd: ip -6 route add fc00:2::101/128 encap seg6local action End.DT4 vrftable 101 dev net0
      - cmd: ip -6 route add fc00:2::110/128 encap seg6local action End.DT4 vrftable 110 dev net0

  - name: C1
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: ip addr add 10.0.1.2/24 dev net0
      - cmd: ip route replace default via 10.0.1.1
  - name: C10
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: ip addr add 10.0.1.2/24 dev net0
      - cmd: ip route replace default via 10.0.1.1
  - name: C2
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: ip addr add 10.0.2.2/24 dev net0
      - cmd: ip route replace default via 10.0.2.1

  - name: C20
    cmds:
      - cmd: sysctl -w 'net.ipv6.conf.all.disable_ipv6=0'
      - cmd: sysctl -w 'net.ipv6.conf.default.disable_ipv6=0'
      - cmd: ip addr add 10.0.2.2/24 dev net0
      - cmd: ip route replace default via 10.0.2.1

```


## メモ

Conntrack使えなかった

```
root@R1:/# iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 45 packets, 2848 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 17 packets, 976 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy DROP 17 packets, 976 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      net1    0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED

root@C1:/# echo "hogehoge" | nc 10.0.2.2 9999

[root@localhost vpn_v4_per_vrf]# conntrack -S
cpu=0   	found=0 invalid=84 insert=0 insert_failed=0 drop=0 early_drop=0 error=0 search_restart=0
```

Conntrackをするには往路と復路両方でIPv4パケットが見える必要がある

LWTベースのSRv6encap/decapだと往路と復路でIPv4, IPv6両方が見える

よってだめ

## End.DT4を使った場合のパケットフロー

```
Outgoing
tap -> ip_rcv -> PREROUTING (ipv4 & in-device = tap) -> vrf_ip_rcv -> PREROUTING (ipv4 & in-device = vrf) -> routing decision (ipv4) -> seg6_input (encap) -> routing decision (ipv6) -> ip6_forward -> FORWARD (ipv6 & in-device = XXX) -> ip6_output -> POSTROUTING (ipv6 & in-device = XXX & out-device = phy) -> phy

Incoming
phy -> ipv6_rcv -> PREROUTING (ipv6 & in-device = phy) -> routing decision (ipv6) -> seg6_local_input -> input_action_end_dt4 (decap) -> vrf_ip_rcv -> PREROUTING (ipv4 & in-device = vrf) -> routing decision (ipv4) -> ip_forward -> FORWARD (ipv4 & in-device = XXX) -> ip_output -> POSTROUTING (ipv4 & in-device = XXX & & out-device = tap) -> tap
```

## 疑問