Google Cloud networking in depth: Understanding Network Service Tiers

Premium Tier
When you choose Premium Tier, you benefit from the same rock-solid global network that powers Google Search, Gmail, YouTube, and other Google services, and that GCP customers such as The Home Depot, Spotify and Evernote use to power their services. Premium Tier takes advantage of Google’s well-connected, high-bandwidth, low latency, highly reliable global backbone network, consisting of over 100,000 miles of fiber and over 100 points of presence (POPs) across the globe. By this measure, Google’s network is the largest of any public cloud provider.

This network is engineered and provisioned to ensure at least three independent paths (N+2 redundancy) between any two points, ensuring availability even in the case of a fiber cut or other unplanned outages.

When you use the Premium Tier network, your traffic stays on the Google backbone for most of its journey, and is only handed off to the public internet close to the destination user. This maximizes the amount your traffic can benefit from Google’s private network. Compare this to “hot-potato” routing used by other cloud providers and in Standard Tier, which hands off traffic to the public internet early in its journey.

On the ingress path, Global BGP announcements ensure that traffic from a client enters Google’s network as close to the client as possible. On the egress path, we use our Espresso mapping infrastructure to choose a peering location near the destination ISP while avoiding congestion on peering links, then encapsulate the response traffic with a label directing it to this peering connection. This sends outgoing packets along Google’s backbone for the bulk of their journey, and has them egress near the destination, ensuring a fast response path. In many cases, Google is directly connected to the client’s ISP, further helping traffic to avoid delays and congestion on third-party networks.

Many GCP customers extensively use Global Load Balancing (HTTP(S) Load Balancing, SSL Proxy Load Balancing, and TCP Proxy Load Balancing) and Cloud CDN, two services available with Premium Tier. These customers benefit from Premium Tier’s use of dedicated global anycast IP addresses. Compared with using multiple addresses with DNS-based load balancing, dedicated anycast addresses mean that clients anywhere can connect to the same IP address, while still entering Google’s network as fast as possible and connecting to a load balancer at the edge of Google’s network where their traffic entered. This minimizes the network distance between the client and the frontline load balancer. That in turn means that any TCP retransmits, for example due to last-mile packet loss, only have to travel a short distance, even if your instances are located much further away. This improves throughput and minimizes latency for clients around the world. Further, if you also use Cloud CDN, you benefit from caching at these edge locations. Finally, a global anycast IP address enables you to seamlessly change or add regions for deploying application instances and increase capacity as needed.

Standard Tier
In contrast, Standard Tier offers regional networking with performance comparable to that of other cloud providers. In Standard Tier, Google uses hot potato routing to ingress and egress traffic local to your instances. It also reduces costs by using the ISP transit rather than Google’s premium network to bring traffic to your regional instances. Similarly, it egresses traffic from your instances locally, encapsulating it to transit ports near the instance and relying on transit networks to relay it to your clients. This reduces costs while delivering performance comparable to other clouds but lower than Premium Tier.

Because Standard Tier networking is regional, instances behind a Standard Tier load balancer are limited to a single GCP region–you don’t get the benefits of global networking like when you choose Premium Tier. In addition, if you want to use multiple regions with Standard Tier, you need to use one IP address for each region and direct traffic to the appropriate region using another mechanism, such as DNS load balancing.

Standard Tier networking is now available to all cloud customers in asia-northeast1, us-central1, us-east1, us-east4, us-west1, europe-west1, and europe-west3. It is additionally available with approval in asia-east1. For up-to-date information on where you can access Standard Tier, please visit this link.

Performance comparison
For an independent third-party assessment of the performance of Premium Tier vs. Standard Tier networking, we turned to Citrix ITM, an internet performance monitoring and optimization tools company. At time of publication, Citrix ITM found that Premium Tier has almost double the median throughput and 20% lower latency than Standard Tier in us-central1. You can view the live results on Citrix ITM dashboard under “Network Tiers”. Citrix ITM explains their testing methodology on their website.