GKE Sandbox: Bring defense in depth to your pods

Editor’s note: This is one of several posts in a series on the unique capabilities you can find in Google Kubernetes Engine (GKE) Advanced.

There’s a saying among security experts: containers do not contain. Security researchers have demonstrated vulnerabilities that allow an attacker to compromise a container and gain access to the shared host operating system (OS), also known as “container escape.” For applications that use untrusted code, container escape is a critical part of the threat profile.

At Google Cloud Next ’19 we announced GKE Sandbox in beta, a new feature in Google Kubernetes Engine (GKE) that increases the security and isolation of your containers by adding an extra layer between your containers and host OS. At general availability, GKE Sandbox will be available as part of the upcoming GKE Advanced, which offers enhanced features to help you build demanding production applications on top of our managed Kubernetes service.

Let’s look at an example of what could happen with a container escape. Say you have a software as a service (SaaS) application that runs machine learning (ML) workloads for users. Imagine that an attacker uploads malicious code that generates a privilege escalation to the host OS, and from that host OS, the attacker accesses the model and data of the other ML workloads, when the model and data aren’t theirs.

GKE Sandbox is based on gVisor, the open-source container sandbox runtime that we released last year. We originally created gVisor to defend against a host compromise when running arbitrary, untrusted code, while still integrating with our container-based infrastructure. And because we use gVisor to increase the security of Google’s own internal workloads, it continuously benefits from our expertise and experience running containers at scale in a security-first environment. We also use gVisor in Google Cloud Platform (GCP) services like the App Engine standard environment, Cloud Functions, Cloud ML Engine, and most recently Cloud Run.

gVisor works by providing an independent operating system kernel to each container. Applications then interact with the virtualized environment provided by gVisor’s kernel rather than the host kernel. gVisor also manages and places restrictions on file and network operations, ensuring that there are two isolation layers between the containerized application and the host OS. By reducing and restricting the application’s interaction with the host kernel, attackers have a smaller attack surface with which to circumvent the isolating mechanism of the container.

GKE Sandbox takes gVisor, abstracts the internals, and presents it as an easy-to-use service. When you create a pod, simply choose GKE Sandbox and continue to interact with your containers as you normally would–no need to learn a new set of controls or a new mental model.

In addition to limiting potential attacks, GKE Sandbox helps teams running multi-tenant clusters, such as SaaS providers, who often execute unknown or untrusted code. There are many components to multi-tenancy, and technologies like GKE Sandbox take the first step toward delivering more secure multi-tenancy in GKE.

How users are hardening containers with GKE Sandbox
Data refinery creator Descartes Labs applies machine intelligence to massive data sets. “At Descartes Labs, we have a wide range of remote sensing data measuring the Earth and we wanted to enable our users to build unique custom models that deliver value to their organizations,” said Tim Kelton, Co-Founder and Head of SRE, Security, and Cloud Operations at Descartes Labs. “As a multi-tenant SaaS provider, we still wanted to leverage Kubernetes scheduling to achieve cost optimizations, but build additional security layers on top of users’ individual workloads. GKE Sandbox provides an additional layer of isolation that is quick to deploy, scales, and performs well on the ML workloads we execute for our users.”

We also heard from early customer Shopify about how they’re using GKE Sandbox. “Shopify is always looking for more secure ways of running our merchants’ stores,” said Catherine Jones, Infrastructure Security Engineer at Shopify. “Hosting over 800,000 stores and running customer code (such as custom templates and third-party applications) requires substantial work to ensure that a vulnerability in an application cannot be exploited to affect other services running in the same cluster.”

Jones and her team developed proof-of-concept trials to use GKE Sandbox and now plan on upgrading existing clusters and enabling it for all new clusters for developers. “GKE Sandbox’s userland kernel acts as a firewall between applications and the cluster node’s kernel, preventing a compromised application from exploiting other applications through it,” said Jones. “This will allow us to provide more security to our 600+ applications without impacting developers’ workflows or requiring our security team to maintain custom seccomp and apparmor profiles for each individual application. In addition, because GKE Sandbox is based on the open-source gVisor project, we can troubleshoot it more effectively and contribute code to support our use cases as need be.”

Getting started with GKE Sandbox
When we say that running a cluster with GKE Sandbox is easy, we really mean it. The following command creates a node pool with GKE Sandbox enabled, which you can attach to your existing cluster.