Egress Internet Access

**Carrier Edge VPC** This VPC will host the VPC Endpoint Interface that connects to the Nginx Load Balancer hosted in the application VPC. Using the VPC Endpoint can increase security by isolating the IoT device network and the application network. All requests coming from the IoT device network/carrier must go through the VPC endpoint, and there is no direct routing between the two networks. The Route 53 Inbound Endpoint should be created on this VPC and the private hosted zone shared with this VPC. Each environment should have their own Carrier Edge VPC. Action Item – confirm any connection initiated from the Application VPC to the IoT device (the diagram shows a connection from Broadcast-SIM and API Server T-Mobile to the carrier networks). # Egress Internet Access Action Item – check how the existing VPC is routing traffic to the Internet. Are they using NAT Gateway? Action Item – check if the customer requires egress traffic inspection. If so, can the customer wait for Vanta? (expected to be released on reinvent) Ingress Internet Access The customer mentioned the need to implement a Web Application Firewall with Imperva. We can use the Edge route table association to route the ingress traffic through a security appliance. If the customer agrees, we can use the AWS WAF instead. My recommendation to prepare for a multi-region approach is to move the inbound traffic to a Global Accelerator and configure the target group to the ALB in each AWS region depending on how they want to balance or failover the ingress traffic. It seems like those requests are low volume and should not cost too much in Global Accelerator data charges, but we should confirm the customer's volume. Action Item – add the Global Accelerator to the architecture diagram for the Ingress traffic from the Internet. Streamer If they are using UDP streaming, then the VPC Endpoint solution will not work because it doesn't support UDP protocol yet. A potential solution would be to move the Streamer to the Edge VPC into a public subnet. We should apply a security group in the VPC Endpoint, only allowing traffic from the carrier network. If an attacker compromises the Streamer, they cannot escalate privilege and access the VPC endpoint. Another potential option is to replace the Streamer with an Elemental service such as MediaLive and MediaPackage. Probably not on this project. Action Item – Check with the customer the protocol used between the IoT device and the Streamer. ## **Route 53 DNS** Research how to implement the Route 53 health check across multi regions. That would be required if they are configuring the DNS on the IoT devices to point to the Route 53 Inbound endpoint, and if one AWS region becomes unavailable, they still can resolve the names. Geolocation and Latency records don't work with private hosted zones. Potential challenges Action Item - Research and test cross-account CloudWatch data sharing to use the metrics from the NLB in the application account to trigger an alarm in the VPC where the Route 53 health check is configured. (https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) Action Item - Test if a client pointing its DNS to the Route 53 Endpoint can perform resolution directly to private hosted zones. (this is not a typical scenario, and I was surprised it worked, but also make sense why it worked). However, if that doesn't work, they need to launch DNS servers in the Edge VPC. **Split-Horizon DNS **Evaluate any potential issue in using split-horizon across different environments. (same domain name in each environment, but with other Route 53 endpoint and hosted zone) IoT device access to S3 through the private network Access to S3 endpoint through a private network is currently not possible by using the S3 Gateway Endpoint. Vimal confirmed with the service team that the VPC Endpoint Interface doesn't have a release date yet. # Carrier Edge VPC This VPC will host the VPC Endpoint Interface that connects to the Nginx Load Balancer hosted in the application VPC. Using the VPC Endpoint can increase security by isolating the IoT device network and the application network. All requests coming from the IoT device network/carrier must go through the VPC endpoint, and there is no direct routing between the two networks. The Route 53 Inbound Endpoint should be created on this VPC and the private hosted zone shared with this VPC. Each environment should have their own Carrier Edge VPC. Action Item – confirm any connection initiated from the Application VPC to the IoT device (the diagram shows a connection from Broadcast-SIM and API Server T-Mobile to the carrier networks). Egress Internet Access Action Item – check how the existing VPC is routing traffic to the Internet. Are they using NAT Gateway? Action Item – check if the customer requires egress traffic inspection. If so, can the customer wait for Vanta? (expected to be released on reinvent) Ingress Internet Access The customer mentioned the need to implement a Web Application Firewall with Imperva. We can use the Edge route table association to route the ingress traffic through a security appliance. If the customer agrees, we can use the AWS WAF instead. My recommendation to prepare for a multi-region approach is to move the inbound traffic to a Global Accelerator and configure the target group to the ALB in each AWS region depending on how they want to balance or failover the ingress traffic. It seems like those requests are low volume and should not cost too much in Global Accelerator data charges, but we should confirm the customer's volume. Action Item – add the Global Accelerator to the architecture diagram for the Ingress traffic from the Internet. Streamer If they are using UDP streaming, then the VPC Endpoint solution will not work because it doesn't support UDP protocol yet. A potential solution would be to move the Streamer to the Edge VPC into a public subnet. We should apply a security group in the VPC Endpoint, only allowing traffic from the carrier network. If an attacker compromises the Streamer, they cannot escalate privilege and access the VPC endpoint. Another potential option is to replace the Streamer with an Elemental service such as MediaLive and MediaPackage. Probably not on this project. Action Item – Check with the customer the protocol used between the IoT device and the Streamer. Route 53 DNS Research how to implement the Route 53 health check across multi regions. That would be required if they are configuring the DNS on the IoT devices to point to the Route 53 Inbound endpoint, and if one AWS region becomes unavailable, they still can resolve the names. Geolocation and Latency records don't work with private hosted zones. Potential challenges Action Item - Research and test cross-account CloudWatch data sharing to use the metrics from the NLB in the application account to trigger an alarm in the VPC where the Route 53 health check is configured. (https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) Action Item - Test if a client pointing its DNS to the Route 53 Endpoint can perform resolution directly to private hosted zones. (this is not a typical scenario, and I was surprised it worked, but also make sense why it worked). However, if that doesn't work, they need to launch DNS servers in the Edge VPC. Split-Horizon DNS Evaluate any potential issue in using split-horizon across different environments. (same domain name in each environment, but with other Route 53 endpoint and hosted zone) IoT device access to S3 through the private network Access to S3 endpoint through a private network is currently not possible by using the S3 Gateway Endpoint. Vimal confirmed with the service team that the VPC Endpoint Interface doesn't have a release date yet. Potential solutions: Implementing an Nginx to proxy requests from the IoT device to S3 Action Item – Evaluate using the API Gateway REST API to proxy requests to S3. (https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html) It can be a temporary solution until the VPC Endpoint Interface for S3 becomes available. Other Network Consideration Three Availability Zones per VPC. Make sure to map the AZ ID in each VPC. (https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html) Transit Gateway versus VPC Peering Initially, the VPC peering would be a more affordable option for the customer, as they only need to establish VPC peering between the Application VPC and the Shared Services VPC. Data processing for Transit Gateway is more expensive than VPC Peering, and they still need to pay for each VPC attachment. They have a single Dev team in charge of the backend applications, so I don't see a need to create multiple application accounts for each environment. I don't see a need to implement Transit Gateway, but let's keep that discussion open. It doesn't seem like this customer will grow that fast to need more than 120 VPC peering connections. Account Structure Here are some suggestions of account structure in addition to the standard Control Tower standard accounts. Each application team should have their own set of AWS accounts (at this point, they only have a single backend team) Create a separate AWS account for the Development tools that will be used for the CI/CD pipeline and other development tools. This is usually created for each App team. Keep the shared VPC/account for the infrastructure servers, such as DNS server, Active Directory, Management servers, etc. Network account Carrier Edge VPC into a private subnet Private Hosted Zones / Route 53 Endpoints Streamer in a separate public subnetImplementing an Nginx to proxy requests from the IoT device to S3 Action Item – Evaluate using the API Gateway REST API to proxy requests to S3. (https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html) It can be a temporary solution until the VPC Endpoint Interface for S3 becomes available. **Other Network Consideration ** Three Availability Zones per VPC. Make sure to map the AZ ID in each VPC. (https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html) **Transit Gateway versus VPC Peering ** Initially, the VPC peering would be a more affordable option for the customer, as they only need to establish VPC peering between the Application VPC and the Shared Services VPC. Data processing for Transit Gateway is more expensive than VPC Peering, and they still need to pay for each VPC attachment. They have a single Dev team in charge of the backend applications, so I don't see a need to create multiple application accounts for each environment. I don't see a need to implement Transit Gateway, but let's keep that discussion open. It doesn't seem like this customer will grow that fast to need more than 120 VPC peering connections. **Account Structure ** Here are some suggestions of account structure in addition to the standard Control Tower standard accounts. Each application team should have their own set of AWS accounts (at this point, they only have a single backend team) Create a separate AWS account for the Development tools that will be used for the CI/CD pipeline and other development tools. This is usually created for each App team. Keep the shared VPC/account for the infrastructure servers, such as DNS server, Active Directory, Management servers, etc. Network account Carrier Edge VPC into a private subnet Private Hosted Zones / Route 53 Endpoints Streamer in a separate public subnet