---
# System prepended metadata

title: C2C Communication Implementation Notes

---

# C2C Communication Implementation Notes

## 1. Introduction

### 1.1. [App Service Discovery](https://docs.cloudfoundry.org/devguide/deploy-apps/cf-networking.html#discovery)

> With app service discovery, apps pushed to Cloud Foundry Application Runtime can establish container-to-container communications through a known route served by internal BOSH DNS. This allows front end apps to easily connect with back end apps.


### 1.2. [**Reference Implementation**](https://github.com/cloudfoundry-attic/cf-networking-examples/blob/master/docs/c2c-with-service-discovery.md)

| Purpose                                                      | Command                        | Reference Use Case |
| ------------------------------------------------------------ | ------------------------------ | ------------------ |
| Frontend connects to a specific backend.                     | `cf add-network-policy`        | Use case 1         |
| Frontend connects to multiple backends for **load balance**. | `cf create-route`, `map-route` | Use case 2         |

### 1.3. Concept

- [Microservice Architecture](https://microservices.io/)

- Route: An app’s route is the URL that it runs at (or an application address).

- Domain: A group of routes, default internal domain: `apps.internal`


---



## 2. Before pushing applications

### 2.1. Check `domains`, `routes`, `network-policies`

#### Domains

```bash
~$ cf domains
Getting domains in org brilliankingsman-main-org as brilliankingsman@gmail.com...
name            status   type   details
cfapps.io       shared          
cf-tcpapps.io   shared   tcp    
apps.internal   shared          internal
```

#### Routes

```bash
~$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space   host   domain   port   path   type   apps   service
No routes found
```



#### Network Policy

```bash
~$ cf network-policies
Listing network policies in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...

source   destination   protocol   ports   destination space   destination org
```





### 2.2. Default Configuration of 3 Applications:

#### Check `env` and `routes`

| Application Name | `env`                                                        | `routes`                         |
| ---------------- | ------------------------------------------------------------ | -------------------------------- |
| `frontend`       | `GOPACKAGENAME: example-apps/cats-and-dogs/frontend`         |                                  |
| `backend-b`      | `GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-b`<br />`CATS_PORTS: 7007,7008,7009`<br />`UDP_PORTS: 9001,9002,9003` |                                  |
| `backend-a`      | `GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-a`<br />`CATS_PORTS: 7007,7008,7009`<br />`UDP_PORTS: 9001,9002,9003` | `route: backend-a.apps.internal` |

 #### Linked Pictures:

| Application Name | Picture url                                                  |
| ---------------- | ------------------------------------------------------------ |
| `backend-a`      | https://i.imgur.com/1uYroRF.gif                              |
| `backend-b`      | https://media.giphy.com/media/FElR3ylsj6PTy/giphy.gif        |
| `frontend`       | https://i2.kym-cdn.com/photos/images/original/000/234/765/b7e.jpg |



#### In-app routing in `frondend`:

```go
func main() {
	systemPortString := os.Getenv("PORT")
	systemPort, err := strconv.Atoi(systemPortString)
	if err != nil {
		log.Fatal("invalid required env var PORT")
	}

	mux := http.NewServeMux()
	mux.Handle("/proxy/", &HttpDemoHandler{})
	mux.Handle("/udp-test/", &UDPDemoHandler{})
	mux.Handle("/", &HomePageHandler{})
	http.ListenAndServe(fmt.Sprintf("0.0.0.0:%d", systemPort), mux)
}
```



---

## 3. After pushing the applications

### 3.1. Default `env` of 3 applications (focus on key features)

#### backend-b

```bash
System-Provided:


{
  "application_name": "backend-b-bkm",
  "application_uris": [
    "backend-b-bkm.apps.internal"
  ],
  ...
  "uris": [
    "backend-b-bkm.apps.internal"
  ],
  ...
}

User-Provided:
CATS_PORTS: 7007,7008,7009
GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-b
UDP_PORTS: 9001,9002,9003

No running env variables have been set

No staging env variables have been set


```

- if there is no domain specified during the push, the `uri`/`application_uris` would be `backend-b-bkm.cfapps.io`

#### backend-a

```bash
System-Provided:


{
  "application_name": "backend-a-bkm",
  "application_uris": [
    "backend-a.apps.internal"
  ],
  ...
  "uris": [
    "backend-a.apps.internal"
  ],
  ...
}

User-Provided:
CATS_PORTS: 7007,7008,7009
GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-b
UDP_PORTS: 9001,9002,9003

No running env variables have been set

No staging env variables have been set

```

#### frontend

```bash
System-Provided:


{
  "application_name": "frontend-bkm",
  "application_uris": [
    "frontend-bkm.cfapps.io"
  ],
  ...
  "uris": [
    "frontend-bkm.cfapps.io"
  ],
  ...
}

User-Provided:
GOPACKAGENAME: example-apps/cats-and-dogs/frontend

No running env variables have been set

No staging env variables have been set

```

### 3.2. Routes

```bash
~$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space         host            domain          port   path   type   apps            service
development   frontend-bkm    cfapps.io                            frontend-bkm
development   backend-a       apps.internal                        backend-a-bkm
development   backend-b-bkm   apps.internal                        backend-b-bkm
```

### 3.3. Network Policy

```bash
~$ cf network-policies
Listing network policies in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...

source   destination   protocol   ports   destination space   destination org
```

---

## 4. After setting ports in environment variables

Specifying ports for c2c communication based on the allowed ports.

#### backend-a

- Command

  ```bash
  cf set-env backend-a-bkm CATS_PORTS "7007,7008"
  cf set-env backend-a-bkm UDP_PORTS "9003,9004"
  cf restage bakend-a-bkm
  ```

  

- `env`

  ```bash
  User-Provided:
  CATS_PORTS: 7007,7008
  GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-a
  UDP_PORTS: 9003,9004
  ```

#### backend-b

- Command

  ```bash
  cf set-env backend-b-bkm CATS_PORTS "7007,7008"
  cf set-env backend-b-bkm UDP_PORTS "9003,9004"
  cf restage bakend-b-bkm
  ```

- `env`

  ```bash
  User-Provided:
  CATS_PORTS: 7007,7008
  GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-a
  UDP_PORTS: 9003,9004
  ```



#### Trying the first connect

- HTTP

  ```
  request failed: Get http://backend-a.apps.internal:7007: dial tcp 10.248.52.205:7007: connect: connection refused
  request failed: Get http://backend-b-bkm.apps.internal:7007: dial tcp 10.246.253.193:7007: connect: connection refused
  ```

  

- UDP

  ```
  request failed: read udp data: read udp 10.255.127.69:37806->10.248.52.205:9003: recvfrom: connection refused
  request failed: read udp data: read udp 10.255.127.69:50315->10.255.208.229:9003: recvfrom: connection refused
  ```

  

---

## 5. Adding Network Policy: let frontend connect to a specific backend

```bash
~$ cf add-network-policy frontend-bkm --destination-app backend-a-bkm --port 7007 --protocol tcp
Adding network policy from app frontend-bkm to app backend-a-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
~$ cf add-network-policy frontend-bkm --destination-app backend-a-bkm --port 9003 --protocol udp
Adding network policy from app frontend-bkm to app backend-a-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
```

### 5.1. Check Network Policy

```bash
~$ cf network-policies
Listing network policies in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...

source         destination     protocol   ports   destination space   destination org
frontend-bkm   backend-a-bkm   tcp        7007    development         brilliankingsman-main-org
frontend-bkm   backend-a-bkm   udp        9003    development         brilliankingsman-main-org
```

### 5.2. Check Domains

```bash
~$ cf domains
Getting domains in org brilliankingsman-main-org as brilliankingsman@gmail.com...
name            status   type   details
cfapps.io       shared          
cf-tcpapps.io   shared   tcp    
apps.internal   shared          internal
```

### 5.3. Check Routes

```bash
~$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space         host            domain          port   path   type   apps            service
development   frontend-bkm    cfapps.io                            frontend-bkm
development   backend-a       apps.internal                        backend-a-bkm
development   backend-b-bkm   apps.internal                        backend-b-bkm
```

After adding the network policy, the connection should be avaliable.

---

## 6. let frontend connect to multiple backends

### 6.1. Create a route with a new hostname to contain multiple backends

[command reference](https://cli.cloudfoundry.org/en-US/v6/create-route.html): `cf create-route SPACE DOMAIN [--hostname HOSTNAME] [--path PATH]`

```bash
~$ cf create-route development apps.internal --hostname nbackend-bkm
Creating route nbackend-bkm.apps.internal for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
Route nbackend-bkm.apps.internal has been created.
OK
```

**Check Routes:**

```bash
$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space         host            domain          port   path   type   apps            service
development   frontend-bkm    cfapps.io                            frontend-bkm
development   backend-a       apps.internal                        backend-a-bkm
development   backend-b-bkm   apps.internal                        backend-b-bkm
development   nbackend-bkm    apps.internal
```



### 6.2. Map routes to backends

#### Map route to backend-a

```bash
~$ cf map-route backend-a-bkm apps.internal --hostname nbackend-bkm
Creating route nbackend-bkm.apps.internal for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
Route nbackend-bkm.apps.internal already exists
Adding route nbackend-bkm.apps.internal to app backend-a-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
```

- Check after mapping route to backend-a

```bash
~$ cf apps
Getting apps in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK

name            requested state   instances   memory   disk   urls
backend-a-bkm   started           1/1         32M      32M    backend-a.apps.internal, nbackend-bkm.apps.internal
backend-b-bkm   started           1/1         32M      32M    backend-b-bkm.apps.internal
frontend-bkm    started           1/1         32M      32M    frontend-bkm.cfapps.io

~$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space         host            domain          port   path   type   apps            service
development   frontend-bkm    cfapps.io                            frontend-bkm
development   backend-a       apps.internal                        backend-a-bkm
development   backend-b-bkm   apps.internal                        backend-b-bkm
development   nbackend-bkm    apps.internal                        backend-a-bkm

```



#### Map route to backend-b

- before mapping route to `backend-b-bkm`, the network policies for tcp/udp connections should be allowed first:

```bash
~$ cf add-network-policy frontend-bkm --destination-app backend-b-bkm --port 7007 --protocol tcp
Adding network policy from app frontend-bkm to app backend-b-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
~$ cf add-network-policy frontend-bkm --destination-app backend-b-bkm --port 9003 --protocol udp
Adding network policy from app frontend-bkm to app backend-b-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
```



```bash
~$ cf map-route backend-b-bkm apps.internal --hostname nbackend-bkm
Creating route nbackend-bkm.apps.internal for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
Route nbackend-bkm.apps.internal already exists
Adding route nbackend-bkm.apps.internal to app backend-b-bkm in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK
```



- Check after mapping route to backend-a

```bash
~$ cf routes
Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...

space         host            domain          port   path   type   apps                          service
development   frontend-bkm    cfapps.io                            frontend-bkm
development   backend-a       apps.internal                        backend-a-bkm
development   backend-b-bkm   apps.internal                        backend-b-bkm
development   nbackend-bkm    apps.internal                        backend-a-bkm,backend-b-bkm

~$ cf apps
Getting apps in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
OK

name            requested state   instances   memory   disk   urls
backend-a-bkm   started           1/1         32M      32M    backend-a.apps.internal, nbackend-bkm.apps.internal
backend-b-bkm   started           1/1         32M      32M    backend-b-bkm.apps.internal, nbackend-bkm.apps.internal
frontend-bkm    started           1/1         32M      32M    frontend-bkm.cfapps.io
```

(End of reference implemantation)

---

## 7. Keep backend applicaitons only accessible internally 

### 7.1. **Add backend applications to default internal domain `apps.internal`**:

   - By `manifest.yml`: (befor pushing the app)

     ```yaml
     ---
     applications:
       - name: backend-a-bkm
         memory: 32M
         disk_quota: 32M
         buildpack: go_buildpack
         env:
           GOPACKAGENAME: example-apps/cats-and-dogs-with-service-discovery/backend-a
           CATS_PORTS: 7007,7008,7009
           UDP_PORTS: 9001,9002,9003
         routes:
         - route: backend-a.apps.internal
     ```

     

   - By `cf push`: (during pushing the app)

     ```bash
     ~$ cf push -d apps.internal
     ```

     

   - By `cf map-route`: (after pushing the app)
     
     ```
     cf map-route APP_NAME DOMAIN [--hostname HOSTNAME] [--path PATH]
     ```
     
     - Reference:
     	- [Creating a Route on the Internal Domain](https://cfcd-prep.cloudfoundry.org/routing/domains/#internal-domains)
     	- [Configuring Routes and Domains - Internal Domain](https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#internal-domains)

     

### 7.2. **Craete network policy to allow specified connections.**

### 7.3. **Delete** additional routes.

   - Command:

     ```bash
     ~$ cf delete-route apps.internal --hostname backend-a
     
     Really delete the route backend-a.apps.internal?> y
     Deleting route backend-a.apps.internal...
     OK
     ```

     

     ```bash
     ~$ cf delete-route apps.internal --hostname backend-b-bkm
     
     Really delete the route backend-b-bkm.apps.internal?> y
     Deleting route backend-b-bkm.apps.internal...
     OK
     ```

     

   - Check routes:

     ```bash
     ~$ cf routes
     Getting routes for org brilliankingsman-main-org / space development as brilliankingsman@gmail.com ...
     
     space         host           domain          port   path   type   apps                          service
     development   frontend-bkm   cfapps.io                            frontend-bkm
     development   nbackend-bkm   apps.internal                        backend-a-bkm,backend-b-bkm
     
     ~$ cf apps
     Getting apps in org brilliankingsman-main-org / space development as brilliankingsman@gmail.com...
     OK
     
     name            requested state   instances   memory   disk   urls
     backend-a-bkm   started           1/1         32M      32M    nbackend-bkm.apps.internal
     backend-b-bkm   started           1/1         32M      32M    nbackend-bkm.apps.internal
     frontend-bkm    started           1/1         32M      32M    frontend-bkm.cfapps.io
     ```

---

