Skip to main content

Networking

Networking decides who can reach your servers and databases, how fast users get your content, and which name they type to find you.

Decision chart​

Start at the top and answer each question. Click a service box to jump to its setup steps.

If the last answer is no, see Storage for files or Databases for rows. A real app usually needs all three: the VPC holds the servers, CloudFront sits in front of them, and Route 53 gives them a name.

Compare the options​

OptionYou manageScalesCost shapeAWSAzureGCP
Private networkThe IP ranges, subnets (public and private), route tables, gateways, and the security groups and network ACLs that filter trafficYou add subnets, one Availability Zone each. A NAT gateway supports 5 Gbps and scales automatically up to 100 GbpsSecurity groups, network ACLs and internet gateways have no charge. A NAT gateway costs each hour it is available plus each GB it processesVPCVirtual Network (VNet)VPC network
Content delivery (CDN)The origins, cache behaviors (per URL path), the HTTPS certificate and the alternate domain namesAutomatic: content is served from a global network of edge locationsPay for requests and data transferred out to viewers (see the CloudFront pricing page); invalidation paths over the first 1,000 a month cost extraCloudFrontFront DoorCloud CDN
DNSThe hosted zones and the records in themManaged by AWS: a highly available, scalable DNS servicePay for each hosted zone and for the DNS queries answered; alias queries to AWS resources are freeRoute 53Azure DNSCloud DNS

Set it up on AWS​

VPC​

  1. Create the VPC and its subnets. Open the VPC console, choose Create VPC, and under Resources to create choose VPC and more. Keep Name tag auto-generation on and enter the IPv4 CIDR block, for example 10.0.0.0/16. Set Number of Availability Zones (AZs) to 2, then set Number of public subnets and Number of private subnets (2 each). For NAT gateways, choose in how many AZs to create one (see step 3), and optionally add VPC endpoints, S3 Gateway. Leave DNS options alone (both are on by default), then choose Create VPC. The Preview pane shows how the pieces connect, and the Resource map tab shows it again later.

    caution

    A subnet CIDR block can be anywhere from /28 to /16, and the subnets in one VPC cannot overlap. In every subnet, AWS reserves five addresses: the first four and the last one. By default you can have 5 VPCs per Region and 200 subnets per VPC.

    caution

    AWS recommends subnets in at least two Availability Zones for production, so that one zone failing does not take the whole app down. Each subnet lives in a single zone.

  2. Know what makes a subnet public. A subnet is public when its route table has a route that sends 0.0.0.0/0 to an internet gateway, and private when it has no such route. The wizard builds these for you. If you build by hand, choose Internet gateways, Create internet gateway, then Actions, Attach to VPC. Next choose Route tables, select the subnet's table, choose Routes, Edit routes, Add route, enter 0.0.0.0/0 as the Destination, pick the internet gateway as the Target, and choose Save changes.

    caution

    A route to the internet gateway is not enough: an instance also needs a public IPv4 address or an Elastic IP to talk to the Internet. Instances in a private subnet cannot reach the Internet at all, even with a public IP, because there is no route to the gateway.

    caution

    Default VPC and public IPs make things reachable; lock down security groups. Every Region has a default VPC (172.31.0.0/16) with a public /20 subnet in each Availability Zone, an internet gateway and a 0.0.0.0/0 route. An instance launched into a default subnet receives a public IPv4 address, while one launched into a non-default subnet does not. So a quick launch into the default VPC is as reachable as its security group allows: tighten it in step 4, and build your own VPC with private subnets for anything real. The Azure and GCP versions of this gotcha (NSGs and firewall rules) are in the Gotchas on Foundations.

  3. Let private subnets reach the Internet, one way out. Choose NAT gateways, Create NAT gateway, enter an optional name, select a public subnet, leave Connectivity type on Public, and under Elastic IP allocation ID pick an Elastic IP or choose Allocate Elastic IP. When the status changes from Pending to Available, open Route tables, select the private subnet's table, choose Routes, Edit routes, Add route, enter 0.0.0.0/0, choose NAT Gateway as the Target, and select the gateway. Connections must start inside the VPC: a NAT gateway lets private instances call out, and nothing on the Internet can open a connection in. A newer Regional option under Availability mode expands across Availability Zones by itself and needs no public subnet, but it supports public connectivity only. These steps use the zonal one.

    caution

    Egress is not free. NAT gateways add per-GB fees: you pay for each hour a NAT gateway is available and each gigabyte it processes. To lower the bill, keep resources in the same Availability Zone as the NAT gateway they use (or create one gateway per zone that has resources), and send traffic to AWS services that have a gateway or interface endpoint through that endpoint, such as the S3 Gateway option in step 1. The general data-out gotcha for all clouds is in the Gotchas on Foundations.

    caution

    A zonal NAT gateway lives in one Availability Zone. If several zones share it and that zone goes down, the other zones lose Internet access. For resilience, create a NAT gateway in each zone and route each private subnet to the one in its own zone.

    caution

    Deleting a NAT gateway disassociates its Elastic IP but does not release it, so release the address yourself if you no longer need it. The routes that pointed at the gateway stay as blackhole routes until you delete or update them. The gateway's entry stays visible in the console for about an hour after you delete it.

  4. Create the security groups. Choose Security groups, Create security group, enter a name and description, and choose the VPC. Under Inbound rules choose Add rule and set the protocol, port and source. A common chain for a web app uses three groups: the load balancer's group allows HTTP (80) and HTTPS (443) from 0.0.0.0/0, the app's group allows the app port only from the load balancer's group, and the database's group allows the database port only from the app's group (as in Databases).

    caution

    A new security group has no inbound rules, so nothing gets in until you add a rule, and it has one outbound rule that allows all traffic. Rules only allow, there is no deny rule. Security groups are stateful: the reply to an allowed request is allowed back out whatever the outbound rules say. You cannot change a security group's name or description after you create it.

    caution

    When you name another security group as the source, the rule covers every instance in that group, over their private IP addresses, and counts as one rule however large the group is. Prefer that to listing IP ranges.

    caution

    For SSH (22) or RDP (3389), authorize only specific IP address ranges. 0.0.0.0/0 lets anyone on the Internet try to sign in.

    caution

    Every VPC has a default security group named default. You can change its rules but cannot delete it. By default it allows inbound traffic from other members of itself, and a resource launched without a security group gets it. Create dedicated groups instead.

    caution

    Quotas: 60 inbound and 60 outbound rules per security group, and 5 security groups per network interface (up to 16). Both can be raised, but the two numbers multiplied cannot exceed 1,000.

  5. Decide whether you need a network ACL. Every subnet is associated with exactly one, and the default network ACL allows all traffic in and out, so many setups never touch it. Add a custom one only when you want subnet-level rules, including deny rules. Number the rules in steps of 10 or 100 so you can insert more later. The two filters differ like this:

    Security groupNetwork ACL
    Applies toA resource, such as an instanceA whole subnet
    StateStateful: replies are allowed backStateless: allow the replies yourself
    RulesAllow onlyAllow and deny
    EvaluationAll rules are combinedLowest rule number first; the first match wins
    caution

    A custom network ACL ends with a rule * that denies anything no numbered rule allows. Because it is stateless, every rule needs a matching rule for the response traffic, on the ephemeral ports. Open 1024-65535 to cover clients such as a NAT gateway, Elastic Load Balancing and Lambda. A deny rule for all traffic from 0.0.0.0/0 or the subnet's CIDR stops a load balancer from health-checking its instances. The default quota is 20 rules per network ACL for each direction.

  6. Put an Application Load Balancer (ALB) in front of your app. This is the AWS side of the Container web API pattern in Common patterns: the ALB takes the requests and spreads them across your instances or ECS tasks. Open the EC2 console, choose Load Balancers, Create load balancer, and under Application Load Balancer choose Create. Enter a Load balancer name, choose Internet-facing as the Scheme, IPv4 as the IP address type, and the VPC. Under Availability Zones and subnets, select the public subnets from at least two zones. Under Security groups, pick the load balancer's group from step 4. Under Listeners and routing, keep the HTTP listener on port 80 and choose a target group as the Default action (choose Create target group if you have none). Choose Create load balancer. Then open the target group, check that a target is healthy, and open the load balancer's DNS name in a browser.

    caution

    The load balancer name has at most 32 characters (letters, numbers and hyphens) and cannot be changed after you create it. For an internet-facing load balancer, only VPCs that have an internet gateway can be selected.

    caution

    The targets' security groups must allow traffic from the load balancer on the health check port, and on the listener port when targets are registered by IP address, as ECS tasks are (see ECS on Fargate). A load balancer is ready for requests only when each enabled zone has at least one healthy target.

    caution

    For HTTPS, add a second listener and choose From ACM for the certificate. That certificate must be in the same Region as the load balancer. A CloudFront certificate is a different one, and it must be in us-east-1 (see the CloudFront section).

  7. Reach a private instance or database from your laptop. The best way needs no open port: use Session Manager, with port forwarding for a database (see EC2 and step 8 of RDS). The other way is a bastion host: a small instance in a public subnet that you sign in to, and from which you reach the private resources. Give the bastion its own security group that allows SSH (22) only from your own IP address, and let the private resource's security group accept its port from the bastion's group.

    caution

    A bastion is a door into the VPC that is open to the Internet, so every rule above about SSH ranges applies to it. If Session Manager works for you, you do not need one.

CloudFront​

  1. Get the origin ready. For a static site, use a private S3 bucket that holds the files (see step 8 of S3, which also has the bucket policy). For a dynamic app, use an Application Load Balancer from VPC step 6, or a Lambda function URL.

  2. Create the distribution for an S3 origin with origin access control (OAC). Open the CloudFront console, choose Distributions, Create distribution, enter a Distribution name, and choose Single website or app, then Next. For Origin type choose Amazon S3, and for S3 origin choose Browse S3 and select the bucket. Under Settings choose Use recommended origin settings: CloudFront sets up OAC and uses HTTPS redirects, the GET and HEAD methods, and the managed CachingOptimized cache policy with compression on. Choose Next, decide on the AWS WAF protections, choose Next, then Create distribution. The Last modified column shows Deploying until the change reaches the edge locations. Then open https://<distribution domain>/index.html, using the d111111abcdef8.cloudfront.net name shown on the distribution.

    caution

    CloudFront attempts to add the bucket policy for you, so open the bucket's Permissions tab and check that the policy allows cloudfront.amazonaws.com to read, with a condition on your distribution's ARN. If it is missing, paste the policy from S3. Without it every request returns an error.

    caution

    OAC needs Object Ownership on Bucket owner enforced, and it does not work with a bucket set up as an S3 website endpoint. If the objects use SSE-KMS, the KMS key policy must also let the distribution use the key.

    caution

    Delete the distribution before you delete its S3 bucket.

  3. Set the default root object. Open the distribution, choose Edit for the general settings, and enter index.html as the Default root object (the name only, with no leading slash). It is what CloudFront asks the origin for when a viewer requests the root URL.

    caution

    The recommended cache policy keeps objects for 24 hours (a default TTL of 86,400 seconds) when the origin sends no Cache-Control header, so a changed file may not show for a day. Use step 7 or, better, file names that change when the content changes.

  4. Add a dynamic origin and a cache behavior for it. Open the Origins tab and add an origin: choose the Application Load Balancer or paste a Lambda function URL domain such as abc123.lambda-url.ap-southeast-1.on.aws. For an ALB, the preconfigured settings use HTTPS only to the origin on port 443 and TLS 1.2, allow all HTTP methods, and use the UseOriginCacheControlHeaders cache policy and the AllViewer origin request policy. Then open the Behaviors tab and add a behavior with a Path pattern such as /api/*, that origin, Redirect HTTP to HTTPS, and, for API calls that must not be cached, the CachingDisabled cache policy.

    caution

    CloudFront compares each request with the path patterns in the order listed, and the first match wins. The default behavior (*) is always last and cannot be changed. Path patterns are case-sensitive, * matches zero or more characters, and the query string and cookies are ignored when matching. Every origin you want to use needs at least one behavior that points at it.

    caution

    HTTPS from CloudFront to an ALB needs an HTTPS listener whose certificate matches the domain name that is routed to the ALB. CloudFront also forwards the viewer's Host header only if the origin request policy says so (the AllViewer policy does). The ALB's certificate lives in the ALB's Region, while the viewer-facing certificate for CloudFront lives in us-east-1.

    caution

    Direct access to the ALB stays open unless you close it. Either have CloudFront add a custom header (Origin Custom Headers, with a random name and value that you treat like a password) and let the ALB listener forward only requests that carry it, with a default rule that returns a fixed 403, or limit the ALB's security group to the AWS-managed prefix list for CloudFront. A third way is a VPC origin, which lets CloudFront reach an internal ALB in a private subnet.

    caution

    A Lambda function URL used as an origin must be reachable by CloudFront. With OAC, set the function URL's AuthType to AWS_IAM and allow lambda:InvokeFunctionUrl and lambda:InvokeFunction in the function's resource-based policy. Without OAC, use AuthType NONE with a resource-based policy that allows lambda:InvokeFunctionUrl. A secret custom header that the function code checks keeps people from calling the URL directly. For a Lambda origin, use the AllViewerExceptHostHeader origin request policy, because the function URL expects its own domain in the Host header.

  5. Serve HTTPS on your own domain. Open the distribution and, under Settings, Alternate domain names, choose Add a domain. Enter up to five domains and choose Next. For TLS certificate, CloudFront looks for an existing ACM certificate for your domain in your account in us-east-1. If it finds none, you can create one there: with Route 53, CloudFront creates the certificate for you. When the certificate is provisioned, add the DNS records that prove you own the domain (with Route 53, CloudFront updates your DNS records for you), then choose Validate certificate. Choose Next, review the changes, and choose Add domains. If you would rather request the certificate yourself, open the ACM console in us-east-1, choose Request a certificate, enter the domain names, choose DNS validation - recommended as the Validation method, and request it. It shows Pending validation until you add the CNAME record ACM gives you, and then Issued.

    caution

    A certificate for CloudFront must be in the US East (N. Virginia) Region, us-east-1, wherever your bucket and servers are. A wildcard such as *.example.com covers one subdomain level and not the apex name example.com, so add both names to one request. ACM keeps trying to validate for 72 hours, then times out.

  6. Point your domain at the distribution. In Route 53 create an alias record that targets the distribution (see Route 53). With another DNS provider, use a CNAME record for a subdomain such as www. An apex name such as example.com cannot be a CNAME, so use Route 53 alias records for it.

  7. Clear stale content. Open the distribution, choose the Invalidations tab, Create invalidation, enter one path per line (for example /index.html or /*), and choose Create invalidation.

    caution

    The first 1,000 invalidation paths each month are free, counted across all your distributions. A path with a * counts as one path, however many files it clears. Beyond 1,000 you pay for each path. You cannot cancel an invalidation once it starts. For files that change often, versioned file names are cheaper and avoid stale copies in browsers.

Route 53​

  • What it is: Route 53 is AWS's highly available, scalable DNS service. It can register domain names, route Internet traffic to your resources, and check their health.
  • Create a hosted zone: in the Route 53 console choose Hosted zones, Create hosted zone, enter the domain name, keep Public hosted zone as the Type, and choose Create. The zone is not in use until the domain's name servers point at it. For a domain registered elsewhere, follow "Making Amazon Route 53 the DNS service for an existing domain" in the Route 53 guide.
  • Point a name at CloudFront: open the hosted zone, choose Create record, enter the Record name (leave it empty for the apex), choose A - IPv4 address, turn on Alias, and under Route traffic to choose Alias to CloudFront distribution and your distribution. Leave Evaluate target health on No and choose Create records. If the distribution has IPv6 enabled, add a second record of type AAAA - IPv6 address. The distribution must already list the name as an alternate domain name. Changes generally reach all Route 53 servers within 60 seconds.
  • Alias or CNAME: an alias record works at the apex (example.com) and on subdomains, while a CNAME works only on subdomains. Route 53 does not charge for alias queries to CloudFront distributions or other AWS resources. An ALB works the same way, with an A or AAAA alias record.
  • Certificate validation: for DNS validation, ACM gives you a CNAME record to add to the zone. In Terraform, that is the aws_route53_record with for_each over domain_validation_options, as in the CloudFront tab.
  • CLI and Terraform: aws route53 create-hosted-zone --name example.com --caller-reference $(date +%s), then aws route53 change-resource-record-sets with a JSON change batch. In Terraform use aws_route53_zone for the zone and aws_route53_record for records. For a CloudFront alias, set type = "A" and an alias block with name = aws_cloudfront_distribution.site.domain_name, zone_id = aws_cloudfront_distribution.site.hosted_zone_id and evaluate_target_health = false.

Connects to​

  • Compute: Compute (EC2) and ECS on Fargate run in the VPC's subnets, behind its security groups and the ALB.
  • Storage: S3 is the private origin behind CloudFront, and EFS reaches your instances through mount targets in the VPC.
  • Databases: RDS sits in the private subnets and accepts connections only from the app's security group.
  • Identity: Identity (IAM roles) gives instances and functions permissions, while security groups decide which network traffic reaches them.
  • Monitoring: CloudWatch shows NAT gateway metrics such as port allocation errors and dropped packets.
  • Nginx: a reverse proxy you run yourself on an instance, when a managed ALB is more than you need.