SD-WAN FMG API w/ Postman - Base Config === ###### tags: `SD-WAN`, `API`, `FortiOS 6.2.3`, `FMG`, `API`, `ADVPN`, `BGP`, # Verify Environment 1. Connect to both Branch FGTs (Fortigates) and verify that the managed switch is 'Connected' by navigating to **WiFi & Switch Controller > Managed FortiSwitch**: ![](https://i.imgur.com/SCMXkTz.png) 2. Enable RPC in FMG (FortiManager) by navigating to **System Setting > Admin > Administrators** and click on our 'admin' user and make sure that 'JSON API Access' is set to 'Read-Write'. ![](https://i.imgur.com/yU9Ykxr.png) ## Software Version ``` datacenter_fgt # get system status Version: FortiGate-VM64-KVM v6.2.3,build1066,191219 (GA) Virus-DB: 76.00774(2020-04-17 12:20) Extended DB: 1.00000(2018-04-09 18:07) Extreme DB: 1.00000(2018-04-09 18:07) IPS-DB: 15.00821(2020-04-17 00:59) IPS-ETDB: 0.00000(2001-01-01 00:00) APP-DB: 15.00819(2020-04-16 01:32) INDUSTRIAL-DB: 15.00821(2020-04-17 00:59) Serial-Number: FGVM01TM19004119 IPS Malicious URL Database: 2.00615(2020-04-16 04:55) Botnet DB: 4.00637(2020-03-31 23:36) License Status: Valid License Expiration Date: 2020-08-17 VM Resources: 1 CPU/1 allowed, 2012 MB RAM Log hard disk: Not available Hostname: datacenter_fgt Operation Mode: NAT Current virtual domain: root Max number of virtual domains: 10 Virtual domains status: 1 in NAT mode, 0 in TP mode Virtual domain configuration: disable FIPS-CC mode: disable Current HA mode: standalone Branch point: 1066 Release Version Information: GA FortiOS x86-64: Yes System time: Fri Apr 17 23:45:18 2020 ``` ``` fmg # get system status Platform Type : FMG-VM64-KVM Platform Full Name : FortiManager-VM64-KVM Version : v6.2.2-build1183 191008 (GA) Serial Number : FMG-VMTM19007414 BIOS version : 04000002 Hostname : fmg Max Number of Admin Domains : 10000 Max Number of Device Groups : 10000 Admin Domain Configuration : Enabled HA Mode : Stand Alone Branch Point : 1183 Release Version Information : GA Current Time : Fri Apr 17 23:48:21 CEST 2020 Daylight Time Saving : Yes Time Zone : (GMT+1:00) Brussels, Copenhagen, Madrid, Paris. x86-64 Applications : Yes Disk Usage : Free 76.22GB, Total 79.79GB File System : Ext4 License Status : Valid ``` # Setup Postman Client 1. Go to **Postman > Preferences** and in the **Settings** window disable (turn OFF) **SSL certificate verification**. ![](https://i.imgur.com/WUZt8E8.png) 2. From the [github](https://github.com/fortinet-solutions-cse/postman_collections/tree/6.2.3) repository Click on the ++"SD-WAN lab - Base Configuration.postman_collection.json"++ file. The click on **Raw**. ![](https://i.imgur.com/2xDTYiv.png) 3. Copy the contents of the JSON file and in Postman click on **Import** and in the window select **Past Raw Text** and paste the contents there. ![](https://i.imgur.com/A5xFW2c.png) ![](https://i.imgur.com/ubGa7B8.png) ![](https://i.imgur.com/vRnQspl.png) 4. Click on the gearwheel (upper right corner on macOS) and click **Add** environment using a name of your choice and including these variables then click **Add** Then in the top right ensure your Environment is selected: - **ip**: Use the IP and port (ip:port) given by your FortiPoC - **username**: admin - **password**: fortinet - **adom**: DEMO ![](https://i.imgur.com/lX6wC8A.png) 5. Check everything is working in Postman by clicking on the **Login'** method and then **Send**. You should receive an answer with 'Status: 200 OK', and in the body reply you should see a JSON containing 'code: 0' and 'message: OK: ![](https://i.imgur.com/iAXkBtL.png) # Register FGTs in FMG ## Discover Devices 1. In Postman collection in the **Setup** folder we will use the following methods: - **1 - Discover / Probe Devices FGT 1** - **2 - Discover / Probe Devices FGT 2** - **3 - Discover / Probe Devices FGT DC** In the request body the method is asking FMG to access the FGT specified in the body data and fetch all data from it. ![](https://i.imgur.com/RvmFHIt.png) We will use two of the returned attributes, namely: `sn` (serial number) and `name` (hostname). If you are curious to see just how we use them in Postman and they populate variables in your environment, look at the **Test** tab. ![](https://i.imgur.com/YVQHAcz.png) The nice thing about JSON is lines up with dictionary and list objects in Python. - line 1: First the JSON response object (`{...}`) is placed into the `jsonData` var. - line 2: To get the `sn` value we must go to the first item (`[0]`) in the `results` list (the only item!) which is the dict `data`. Then there is another nested `device` dict under that, and finally the `sn` key provides us the value. - line 3: The same is done to get the `name` value to set the `hostname` var. - line 5: Then environment variables such as `sn_fgt_dc` is set to the `sn` value. - line 6: Also the `name_fgt_dc` environment variable is set to the value of `hostname`. ```javascript= var jsonData = pm.response.json(); sn = jsonData.result[0].data.device.sn; hostname = jsonData.result[0].data.device.name; pm.environment.set("sn_fgt_dc", sn); pm.environment.set("name_fgt_dc", hostname); ``` ```json { [...] "result": [ { "data": { "device": { [...] "name": "branch1_fgt", [...] "sn": "FGVM01TM19004830", [...] } }, [...] }, [...] } ] } ``` ![](https://i.imgur.com/SgFoZ9D.png) ## Add Discovered Devices 2. The next request will be to add devices by running: - **4 - Add Devices** Note how we specify the serial number ( sn attribute): we use the environment variable that we set during device discovery. ![](https://i.imgur.com/i2awdM7.png) You can see that the reply contains `taskid` and a number. This means that this task is asynchronous and FortiManager is still performing the task in the background. If you want to check the status of the task in FortiManager, you can do it with Postman. We have another request whose name is **Get Tasks**, under **Monitor** folder. When issuing **Get Tasks** request, please note that you can filter by task ID. Just append the task ID obtained in the previous request to the url inside the ‘Get Tasks’ body: ![](https://i.imgur.com/oYwntIb.png) ![](https://i.imgur.com/airsdzD.png) When checking the response, you will see that there are several attributes returned inside data, namely `adom`, `end_tm`, `flags`, `history`, `id`, `line`, etc. For our purposes pay attention to `line` attribute, as it contains the `percent` of the task completion: ![](https://i.imgur.com/bOUkoCa.png) 3. In FMG navigate to **Device Manager > Device & Groups** and you should see the FGT devices: ![](https://i.imgur.com/PFJbd3W.png) ## Configure Logging We are using the built-in FAZ (FortiAnalyzer) functionality of FMG. The following configuration must be added to each FGT for it to be able to send logs to FMG: ``` config log fortianalyzer setting set status enable set server 192.68.0.15 set upload-option realtime set serial <fmg_serial_number> set certificate-verification disable set reliable enable end ``` 1. Use the following two Postman requests to add a CLI script to FMG for this: - **4.1 - Get FMG Serial Number** - **4.2 - Add Script - FAZ Configuration** The first method will populate the `sn_fmg` Postman environment variable which is used in the second method request body. ![](https://i.imgur.com/DgnkC0p.png) 2. You will now find the CLI script called “FAZ Configuration” in the FMG. Navigate to **Device Manager > Scripts**: ![](https://i.imgur.com/AV6FlcX.png) ![](https://i.imgur.com/YhtWeLD.png) 3. Execute this script on the FMG database for all the three FGTs, using the following Postman request: - **4.3 - Exec Script - FAZ Configuration** The script is executed against FMG database, the configuration is NOT applied to the actual devices yet. In order to apply it, we must install the configuration. Let’s postpone it for now - we will do it in the next chapter, after adding few more things. 4. Later, when you install the configuration, you will see the logs arriving at FAZ. But then, if you login to the FGT WebUI, you would notice an error message of ++"FortiGate not authorized to send any log types. Please grant permission on logging device."++ under the **Security Fabric > Settings** page. To ensure that is not a problem, let's fix it at this stage by logging into the FMG CLI and run the following command: ``` exec log device permissions ALL all enable ``` ![](https://i.imgur.com/dkaGeeF.png) # Overlay Configuration ## Map Underlay Interfaces to Dynamic Interfaces 1. Create dynamic interfaces to map port1, port4, port5, Wireless VLAN ( `vl_fap_mgmt` ) and Wired VLAN (`vl_lan`) of the three FGTs with the Postman request: - **5 - Add Dynamic Interface** 2. In FMG navigate to **Policy & Objects > Object Configurations** to verify the dynamic interface configuration: ![](https://i.imgur.com/ehp8KFu.png) ## VPN Manager 1. Now we need to create two overlay networks (Dial-up Topology) in the VPN Manager, one over MPLS and one over INET. For this we will use a couple of requests from our Postman collection: - **6 - Add VPN Manager VPN Table** - **7 - Add VPN Manager Node** The former will create the two VPN communities (or tables as they are called by the API) and the latter will add the nodes to them. 2. After running each navigate to **VPN Manager > IPsec VPN** in FMG: ![](https://i.imgur.com/e6WYuTv.png) ![](https://i.imgur.com/9WIFHz1.png) ![](https://i.imgur.com/KXJjmy3.png) 3. FMG after running the second Postman request: ![](https://i.imgur.com/aKzlmon.png) ![](https://i.imgur.com/3ieu7mb.png) ![](https://i.imgur.com/qaGdlH3.png) The configuration should be created with the following settings on the Spoke: - Disable IP Assignment - Disable `add-route` (because we will use dynamic routing) - Disable `net-device` and set `tunnel-search next-hop` - Set `auto-discovery-receiver enable` (for ADVPN) On the Hub: - Accept any peer ID - Disable DHCP - Disable `add-route` (because we will use dynamic routing) - Disable `net-device` and set `tunnel-search next-hop` - Set `auto-discovery-sender enable` (for ADVPN) The actual configuration that will be pushed to the FGTs will be the following: ``` branch1_fgt# show vpn ipsec phase1-interface config vpn ipsec phase1-interface edit "OL_MPLS_0" set interface "port4" set ike-version 2 set keylife 28800 set peertype any set net-device disable set proposal aes128-sha256 aes256-sha256 set comments "[created by FMG VPN Manager]" set remote-gw 172.16.2.5 set psksecret ENC 3980epjHj55ukRxJefwl6J4ivuTNyAvjCaUQ55Ot4QPnkZENtN/ITPZNRW/GGr1YTJ/iSXTAgII+AuSF5Ww/BpYyNh9pTxufDNiPrXKOOSxpxB9kai58M1NeybrlLx2Vxp4T3Ebp9RzxYTydAhcr4b2TQvIi9YD09GG4dEtWmQ9aAptdzUrmFiu0GcBgol3WchKo2w== next edit "OL_INET_0" set interface "port1" set ike-version 2 set keylife 28800 set peertype any set net-device disable set proposal aes128-sha256 aes256-sha256 set comments "[created by FMG VPN Manager]" set remote-gw 100.64.1.5 set psksecret ENC bk1N4Ym/ve9oowrXSNOnfg8DB7QKrGN9XJhrAHjWN6j6B8OeeKvhGyjiniVnTzu8Z98fNBZk3KarM3xaDnqGlsrK/sX2Bog4dyOBrg/Z4ebzKatmU9g1HJLe9q4GnZptCMaTxhTpce9vQaOCP3v353o5KcGx0EJNq7U00Q6gUWXJAWUrIPra2bo1bakR+po4DvMMwQ== next end ``` ``` branch1_fgt # show vpn ipsec phase2-interface config vpn ipsec phase2-interface edit "OL_INET_0_0" set phase1name "OL_INET_0" set proposal aes128-sha256 aes256-sha256 set keepalive enable set comments "[created by FMG VPN Manager]" set keylifeseconds 1800 next edit "OL_MPLS_0_0" set phase1name "OL_MPLS_0" set proposal aes128-sha256 aes256-sha256 set keepalive enable set comments "[created by FMG VPN Manager]" set keylifeseconds 1800 next end ``` ``` datacenter_fgt # show vpn ipsec phase1-interface config vpn ipsec phase1-interface edit "OL_MPLS_0" set type dynamic set interface "port4" set ike-version 2 set keylife 28800 set peertype any set net-device disable set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256 set add-route disable set dpd on-idle set comments "[created by FMG VPN Manager]" set tunnel-search nexthop set psksecret ENC 5XBml+fkNHIuCaYmyxqWUHu08N80/BZIu0oVb67MPoGofeNHSnGu/yhr/qQ54tBqPhB1AcvBYbuHuVo5uZl9/GyS6FJBeSqjnVWkgyvRPT5+QWK2XqeAP3BjlFEY9fF6S1nSQB/ROTJ0xfOLN QRX5r8R2P47jgaIfrv2fb2zSCwUAdz+hLjwQOiDrdDkP0mUS7Nghw== set dpd-retryinterval 60 next edit "OL_INET_0" set type dynamic set interface "port1" set ike-version 2 set keylife 28800 set peertype any set net-device disable set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256 set add-route disable set dpd on-idle set comments "[created by FMG VPN Manager]" set tunnel-search nexthop set psksecret ENC 6gE5VSRQizP47pWipunqYPfwFK+pp/DPqnE9mVv+o4sOOhfuAJhTF0Bm2EhhI4KJLE2TgY9XujpXwEtg5+GB7454od56+Zjx0oMoFzfMZQbUikLfAzr3QHUZHOLyqmtrlaOXczYt2WcLwXvCu 5GY4/dBeHEMnvUJA020/Bq3Z7wYUZYxGArzVHt7pEzkQkMWfnQi4g== set dpd-retryinterval 60 next end ``` ``` datacenter_fgt # show vpn ipsec phase2-interface config vpn ipsec phase2-interface edit "OL_INET_0_0" set phase1name "OL_INET_0" set proposal aes128-sha256 aes256-sha256 set keepalive enable set comments "[created by FMG VPN Manager]" set keylifeseconds 1800 next edit "OL_MPLS_0_0" set phase1name "OL_MPLS_0" set proposal aes128-sha256 aes256-sha256 set keepalive enable set comments "[created by FMG VPN Manager]" set keylifeseconds 1800 next end ``` > **NOTE**: After the configuration and policies are installed on the FortiGates, you will probably notice the following issues in regards to the `config vpn ipsec phase1-interface` hierarchy: > - On the spokes `set tunnel-search next-hop` is not set to search for a tunnel using the next-hop (vs `selectors`) which enables use of a dynamic routing protocol. > - On the spokes `auto-discovery-receiver enable` is not set to enable ADVPN features. > - On the hub `set auto-discovery-sender enable` is not set to enable ADVPN features. > **NOTE**: ADVPN (Auto Discovery VPN) allows spokes to dynamically establish tunnels directly between each other (called shortcuts) in what is a traditionally a hub and spoke architecture. Fortinet ADVPN was introduced in FortiOS 5.4. This is a FortiManager misbehavior. We are going to fix it by creating and executing a couple of scripts (it can be done via GUI if desired). However, before doing that, let’s complete the following part first... ## Create Policy Package 1. We are now going to create the firewall policies that we want to push to the FGTs. Create two policy packages: one for the Branches and one for the Data Center. For this purpose we have to use the Postman requests below: - **8 - Add Policy Package** - **9 - Add Firewall Policy - Branches PP - No Interfaces** - **10 - Add Firewall Policy - DataCenter PP - No Interfaces** 2. In FMG navigate to **Policy & Objects > Policy Objects**: The following is seen in FMG after running the **8 - Add Policy Package** request: ![](https://i.imgur.com/Fl86YbJ.png) ![](https://i.imgur.com/vp4vDK3.png) The following is seen in FMG after running the **9 - Add Firewall Policy - Branches PP - No Interfaces** request: ![](https://i.imgur.com/AOxcyKp.png) The following is seen in FMG after running **10 - Add Firewall Policy - DataCenter PP - No Interfaces** request: ![](https://i.imgur.com/vbS4oYn.png) > **NOTE**: Pay attention to the policies and note the following important nuances: > 1. There is a policy on the Hub (`Overlay to INET`) that implies that sometimes Branch traffic towards the Internet will flow via the Hub. That will be part of our SD-WAN configuration. > 2. The virtual sd-wan interface is not used in the policies. Why is that remarkable? Consider again Branch traffic towards the Internet. Think what policy on Branch will be matched when the traffic goes to the Internet directly and what policies on Branch/Hub will be matched when it goes via the Hub. Now see on which of these policies NAT is enabled. Now you probably understand why having a single policy with the virtual sd-wan interface would not be acceptable. At this point the Tunnel interfaces are not available, therefore we are using `any` instead in the source and destination interfaces as you can check in the body of the request. We’ll modify the policies later. 3. Use the Postman requests under Monitor folder to check out the policies: - **Get Policy Package** - **Get Firewall Policy -- Branches PP** - **Get Firewall Policy -- DataCenter PP** ![](https://i.imgur.com/y1PM8y9.png) ## Install Configuration & Policies 1. It's time to install the configuration and the policies on the three FortiGates. We will use the following Postman requests: - **11 - Install policy - Hub** - **12 - Install policy - Branches** You will see that these requests take longer to execute but FortiManager returns the answer immediately. Again, we have an asynchronous request, whose status can be check by issuing a **Get Tasks** Postman request. Remember to check the `taskid` returned in the response. ![](https://i.imgur.com/StsxDso.png) ![](https://i.imgur.com/MMK9Ca8.png) 2. In FMG check the status by navigating to **Device Manager > Device & Groups**: ![](https://i.imgur.com/NPjf3UQ.png) ![](https://i.imgur.com/p17H4xJ.png) ## Tunnel Interfaces 1. Configure the tunnel interfaces using the following Postman requests: - **13 - Configure FGT-1 Interfaces** - **14 - Configure FGT-2 Interfaces** - **15 - Configure FGT-DC Interfaces** These requests will act on the device model of the FortiManager to modify the interfaces of each FortiGate. Remember currently the tunnel interfaces were created when we created the phase1 and phase2 ipsec configurations but the access and ip addressing has not been set: ``` branch1_fgt # show system interface OL_INET_0 config system interface edit "OL_INET_0" set vdom "root" set type tunnel set snmp-index 124 set interface "port1" next end ``` ``` branch1_fgt # show system interface OL_MPLS_0 config system interface edit "OL_MPLS_0" set vdom "root" set type tunnel set snmp-index 123 set interface "port4" next end ``` 2. In FMG navigate to **Device Manager > Device & Groups**. Select one of the **Managed Devices** and then click on **System: Dashboard** and change it to **System: Interface** and you will see that IP addressing and access has been set for the tunnel interfaces. ![](https://i.imgur.com/C2RazdJ.png) ![](https://i.imgur.com/NeaRQJu.png) The actual configuration that will be pushed to the FGTs will be the following: ``` branch1_fgt # show system interface OL_MPLS_0 config system interface edit "OL_MPLS_0" set vdom "root" set ip 10.0.10.1 255.255.255.255 set allowaccess ping set type tunnel set remote-ip 10.0.10.5 255.255.255.0 set snmp-index 123 set interface "port4" next end ``` ``` branch1_fgt # show system interface OL_INET_0 config system interface edit "OL_INET_0" set vdom "root" set ip 10.0.11.1 255.255.255.255 set allowaccess ping set type tunnel set remote-ip 10.0.11.5 255.255.255.0 set snmp-index 124 set interface "port1" next end ``` ``` datacenter_fgt # show system interface OL_MPLS_0 config system interface edit "OL_MPLS_0" set vdom "root" set ip 10.0.10.5 255.255.255.255 set allowaccess ping set type tunnel set remote-ip 10.0.10.254 255.255.255.0 set alias "OL_MPLS" set snmp-index 113 set interface "port4" next end ``` ``` datacenter_fgt # show system interface OL_INET_0 config system interface edit "OL_INET_0" set vdom "root" set ip 10.0.11.5 255.255.255.255 set allowaccess ping set type tunnel set remote-ip 10.0.11.254 255.255.255.0 set alias "OL_INET" set snmp-index 114 set interface "port1" next end ``` 3. Next, create a couple of scripts in FMG to modify the settings of the FGT that went wrong in the previous steps (`set tunnel search next-hop` and auto discovery settings for ADVPN). Execute the following Postman request: - **16 - Add Script - Fix VPN in Branches** - **17 - Add Script - Fix VPN in Hub** 4. In FMG navigate to **Device Manager > Scripts** to verify the scripts were added: ![](https://i.imgur.com/TisLXzt.png) ![](https://i.imgur.com/GwW9HYJ.png) 4. The above step will create a couple of scripts in FMG. However they are not executed yet. To achieve this goal, please open and execute these requests: - **18 - Exec Script - Fix VPN in Branches** - **19 - Exec Script - Fix VPN in Hub** These requests are asynchronous, so please check they are finished before going on. Use **Get Tasks** Postman requests or check FGT status in FMG’s Device Manager. The script is executed against FMG database, the configuration is NOT applied to the actual devices yet. In order to apply it, we must install the configuration. Let’s postpone it for now. You can see this is setup in the FMG database by navigating to **Device Manager > Device & Groups** then select one of the **Managed Devices** and click on **CLI Configurations**. Then select **vpn > ipsec > phase1-interface**. Then for the branch device verify that`set tunnel-search next-hop` is set (vs `selectors`) and that `set auto-discovery-receiver enable` is configured. ![](https://i.imgur.com/xtUX57y.png) ![](https://i.imgur.com/MyXBKjJ.png) 5. Next, create dynamic interfaces for the Overlay Tunnel interfaces. Use this request: - **20 - Add Dynamic Interface - Overlays** 6. In FMG navigate to **Policy & Objects > Object Configurations** and under **Interface** the `OL_INET` and `OL_MPLS` dyanmic interfaces have been added: ![](https://i.imgur.com/3Ueo13t.png) ![](https://i.imgur.com/KbFuDY5.png) 7. Now that we have two dynamic interfaces for the tunnels, we can adjust the firewall policy to use the corresponding interfaces. We can use the following Postman requests: - **21 - Add Firewall Policy - Branches PP** - **22 - Add Firewall Policy - DataCenter PP** 8. These requests are exactly the same as 9 and 10 but they include now the proper names in the interfaces. After executing them, in FMG navigate to **Policy & Objects > Policy Packages** and select one of the policy packages and you should see the proper interfaces in the From/To columns: ![](https://i.imgur.com/r9v7bzw.png) ![](https://i.imgur.com/m7RT9GO.png) 9. You can now install the Policy Package on the three FGTs. Do it either manually or using the Postman requests from before: - **23 - Install policy - Hub** - **24 - Install policy - Branches** 11. In FMG navigate to **Device Manager > Device & Groups** and ensure that the device Config Status is Synchronized and the Policy Package Status good: ![](https://i.imgur.com/mShsIPM.png) 10. After installation is completed, make sure the VPN tunnels are up by navigating to **VPN Manager > Monitor**. The dynamic interfaces will show a status of down - this is normal: ![](https://i.imgur.com/2D8nS0v.png) 11. Log out of the API session using the following Postman request: - **Logout** ![](https://i.imgur.com/AaNb6S4.png) 12. In FMG if you navigate to **System Settings > Dashboard** thenon the **System Information** widget you should only have the 1 admin session: ![](https://i.imgur.com/O1kO3dw.png) ![](https://i.imgur.com/Q58j5yP.png) # BGP FortiOS 6.2.2 introduced a feature that allows withdrawing BGP routes based on the configured performance SLA for SD-WAN. ## BGP Communities The Branches will set the following communities on the prefixes that they advertise: - `SLA_OK_MPLS` = `65000:200`: prefixes advertised over MPLS overlay, when it meets the SLA - `SLA_OK_INET` = `65000:201`: prefixes advertised over INET overlay, when it meets the SLA - `SLA_KO_MPLS` = `65000:100`: prefixes advertised over MPLS overlay, when it doesn’t meet the SLA (or when there is no SLA configured) - `SLA_KO_INET` = `65000:101`: prefixes advertised over INET overlay, when it doesn’t meet the SLA (or when there is no SLA configured) ## Hub Configuration ### `router community-list` Confiugre the BGP communities: ``` config router community-list edit "SLA_KO_INET" config rule edit 1 set action permit set match "65000:201" next end next edit "SLA_KO_MPLS" config rule edit 1 set action permit set match "65000:200" next end next edit "SLA_OK_INET" config rule edit 1 set action permit set match "65000:101" next end next edit "SLA_OK_MPLS" config rule edit 1 set action permit set match "65000:100" next end next end ``` ### `router route-map` Next, we configure the route-map, which will serve two purposes: 1. We want to be able to add/remove routes on the Hub, depending on the SLA status measured on the Branches. This way the Hub itself does not need to measure the SLA. To achieve this goal, we are going to set higher weight on the prefixes advertised with `SLA_OK_*` communities and use tags. Weight is a value that is assigned to our prefixes as a locally significant value. Weight is a simple number in the range of 0 through 65535, and the higher the weight value, the higher the preference for that path. We are also going to mark the `SLA_OK_*` communities with route-tag 1 and the `SLA_KO_*` communities - with route-tag 2. This marking will help us later on. We are going to apply these manipulations, using route-map-in . Below is the relevant part of the route-map: ``` config router route-map edit "OL_INET_IN" config rule edit 1 set match-community "SLA_KO_INET" set set-weight 100 set set-route-tag 2 next edit 100 set set-weight 200 set set-route-tag 1 next end next edit "OL_MPLS_IN" config rule edit 3 set match-community "SLA_KO_MPLS" set set-weight 100 set set-route-tag 2 next edit 100 set set-weight 200 set set-route-tag 1 next end next end ``` 2. The Hub is going to receive from the Branches the same routes on the two overlay links: MPLS and INET. For example, in our lab, the Hub is going to receive 10.0.1.0/24 from Branch-1 on both MPLS and INET. Since we configure iBGP multipath, the Hub will advertise both received routes to Branch-2 over both MPLS and INET. This means that the Branch-2 is going to receive four routes total to 10.0.1.0/24 instead of two. We are going to avoid this, using `route-map` out on the Hub: prefixes will be advertised only on the overlay on which they have been received. Below is the relevant part of the route- map: The `route-map` will be configured to not advertise the MPLS routes over the INET overlay and vice versa: ``` config router route-map edit "OL_INET_OUT" config rule edit 1 set action deny set match-community "SLA_OK_MPLS" next edit 2 set action deny set match-community "SLA_KO_MPLS" next edit 100 next end next edit "OL_MPLS_OUT" config rule edit 1 set action deny set match-community "SLA_OK_INET" next edit 2 set action deny set match-community "SLA_KO_INET" next edit 100 next end next end ``` ### `router bgp` BGP configuration with the above route-map: ``` config router bgp set as 65000 set keepalive-timer 5 set holdtime-timer 15 set ibgp-multipath enable set network-import-check disable set additional-path enable set scan-time 20 config neighbor-group edit "OL_INET" set advertisement-interval 1 set soft-reconfiguration enable set remote-as 65000 set route-map-in "OL_INET_IN" set route-map-out "OL_INET_OUT" set additional-path both set route-reflector-client enable next edit "OL_MPLS" set advertisement-interval 1 set soft-reconfiguration enable set remote-as 65000 set route-map-in "OL_MPLS_IN" set route-map-out "OL_MPLS_OUT" set additional-path both set route-reflector-client enable next end config neighbor-range edit 1 set prefix 10.0.10.0 255.255.255.0 set neighbor-group "OL_MPLS" next edit 2 set prefix 10.0.11.0 255.255.255.0 set neighbor-group "OL_INET" next end config network edit 1 set prefix 172.16.0.0 255.255.255.0 next end end ``` ### `router policy` Before ADVPN shortcut between Branches is established, the traffic starts flowing via the Hub. The Hub receives traffic from one Branch and forwards it to the other. We do not want the Hub to ECMP the traffic over the two overlays. Rather, we want it to forward the traffic only over the same overlay on which it has been received. So policy routing will set traffic that comes in over the MPLS overlay interface to be sent out the MPLS overlay interface only and vice versa. ``` config router policy edit 1 set input-device "OL_MPLS_0" set output-device "OL_MPLS_0" next edit 2 set input-device "OL_INET_0" set output-device "OL_INET_0" next end ``` ## Branch Configuration ### `router route-map` This shows the associated branch BGP communities that will be configured: ``` config router route-map edit "SLA_KO_INET" config rule edit 1 set set-community "65000:201" next end next edit "SLA_KO_MPLS" config rule edit 1 set set-community "65000:200" next end next edit "SLA_OK_INET" config rule edit 1 set set-community "65000:101" next end next edit "SLA_OK_MPLS" config rule edit 1 set set-community "65000:100" next end next end ``` ### `router bgp` There are some important BGP parameters that need to be configured on the Branches in order for ADVPN to work. These parameters are: - `set ibgp-multipath enable ` - `set additional-path receive` In FortiOS v6.2.1 a new BGP parameter was introduced to apply a different route-map-out based on the SLA configured in SD-WAN: - `set route-map-out-preferable <name>` - use `<name>` when the SLA is met. - `set route-map-out <name>` - use `<name>` when the SLA is not met (or no SLA is configured). ``` config router bgp set as 65000 set keepalive-timer 5 set holdtime-timer 15 set ibgp-multipath enable set scan-time 20 config neighbor edit "10.0.10.5" set soft-reconfiguration enable set interface "OL_MPLS_0" set remote-as 65000 set route-map-out "SLA_KO_MPLS" set route-map-out-preferable "SLA_OK_MPLS" set additional-path receive next edit "10.0.11.5" set soft-reconfiguration enable set interface "OL_INET_0" set remote-as 65000 set route-map-out "SLA_KO_INET" set route-map-out-preferable "SLA_OK_INET" set additional-path receive next end config network edit 1 set prefix 10.0.1.0 255.255.255.0 next edit 2 set prefix 10.1.1.0 255.255.255.0 next end end ``` ## Apply Dyanmic Routing CLI Template 1. Ensure you have run the **Login** Postman request. Then run the following Postman request: - **22.2 - Add Dynamic Routing CLI Template - Hub** - **22.3 - Add Dynamic Routing CLI Template - Branch** 2. In FMG navigate to **Device Manager** and add the "CLI Template Status" column by clicking on **Column Settings** and selecting it: ![](https://i.imgur.com/JHxtfkM.png) 3. Navigate to **Scripts** and select **CLI Template**: ![](https://i.imgur.com/XT9EjiZ.png) ![](https://i.imgur.com/z9Ozts3.png) ![](https://i.imgur.com/gSDIjp7.png) 4. Install policy (and device) configuration to the Hub and Branches by using the Postman request: - **23 - Install policy - Hub** - **24 - Install policy - Branches** 5. This should fully sync the devices with the assigned CLI templates: ![](https://i.imgur.com/1xqNvpj.png) ## Verify BGP Routing ``` datacenter_fgt # get router info bgp summary BGP router identifier 192.168.250.5, local AS number 65000 BGP table version is 4 1 BGP AS-PATH entries 2 BGP community entries Next peer check timer due in 4 seconds Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.0.10.1 4 65000 157 162 3 0 0 00:11:39 2 10.0.10.2 4 65000 157 164 3 0 0 00:11:39 2 10.0.11.1 4 65000 164 163 3 0 0 00:11:40 2 10.0.11.2 4 65000 164 165 3 0 0 00:11:40 2 Total number of neighbors 4 ``` ``` datacenter_fgt # get router info routing-table bgp Routing table for VRF=0 B 10.0.1.0/24 [200/0] via 10.0.10.1, OL_MPLS_0, 00:17:43 [200/0] via 10.0.11.1, OL_INET_0, 00:17:43 B 10.0.2.0/24 [200/0] via 10.0.10.2, OL_MPLS_0, 00:17:43 [200/0] via 10.0.11.2, OL_INET_0, 00:17:43 B 10.1.1.0/24 [200/0] via 10.0.10.1, OL_MPLS_0, 00:17:43 [200/0] via 10.0.11.1, OL_INET_0, 00:17:43 B 10.1.2.0/24 [200/0] via 10.0.10.2, OL_MPLS_0, 00:17:43 [200/0] via 10.0.11.2, OL_INET_0, 00:17:43 ``` ``` datacenter_fgt # get router info bgp neighbors 10.0.10.1 routes BGP table version is 4, local router ID is 192.168.250.5 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *>i10.0.1.0/24 10.0.10.1 0 100 100 2 i <-/1> *>i10.1.1.0/24 10.0.10.1 0 100 100 2 i <-/1> Total number of prefixes 2 ``` ``` datacenter_fgt # get router info bgp neighbors 10.0.11.1 routes BGP table version is 4, local router ID is 192.168.250.5 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *>i10.0.1.0/24 10.0.11.1 0 100 100 2 i <-/2> *>i10.1.1.0/24 10.0.11.1 0 100 100 2 i <-/2> Total number of prefixes 2 ``` ``` datacenter_fgt # get router info bgp neighbors 10.0.10.2 advertised-routes BGP table version is 4, local router ID is 192.168.250.5 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *>i10.0.1.0/24 10.0.10.1 100 100 2 i <-/1> *>i10.1.1.0/24 10.0.10.1 100 100 2 i <-/1> *>i172.16.0.0/24 10.0.10.5 100 32768 0 i <-/1> Total number of prefixes 3 ``` ``` datacenter_fgt # get router info bgp neighbors 10.0.11.2 advertised-routes BGP table version is 4, local router ID is 192.168.250.5 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *>i10.0.1.0/24 10.0.11.1 100 100 2 i <-/2> *>i10.1.1.0/24 10.0.11.1 100 100 2 i <-/2> *>i172.16.0.0/24 10.0.11.5 100 32768 0 i <-/1> Total number of prefixes 3 ```