大规模安全构建不受信任的容器映像

大规模安全构建不受信任的容器映像
Building untrusted container images safely at scale

原始链接: https://depot.dev/blog/container-security-at-scale-building-untrusted-images-safely

Depot提供了一个API，简化了多对端SaaS平台的容器构建管理，消除了内部容器编排的需要。使用Go和Depot的API，开发人员可以创建独立的项目环境，确保每个客户的安全性和性能隔离。该过程涉及使用Buf生成客户端库，使用作用域令牌创建项目，并通过删除或缓存重置对其进行管理。 API还提供了对容器构建的深入了解，提供了构建持续时间、保存的持续时间和缓存的步骤等指标，促进了客户分析。此外，它允许检索详细的构建步骤，揭示每个阶段的时间和错误状态。通过利用Depot的API，平台可以安全地执行客户代码，通过缓存加速开发工作流程，轻松扩展容器构建，并提供详细的构建分析，为用户生成的工作负载简化容器基础设施管理。

This Hacker News thread discusses the security risks of building and running untrusted container images, especially for executing arbitrary NodeJS code in production. Several commenters suggest alternatives to Docker's container runtime for improved security, including gVisor, Firecracker, and v8 isolates, particularly for multitenancy scenarios. Some argue that Docker alone isn't sufficient for untrusted code, highlighting potential container escapes and the limitations of namespace isolation. They mention tools like AppArmor and Seccomp for added security, but emphasize the need for a solid threat model. The conversation also touches on the inherent risks of the container build process itself, where untrusted code from Dockerfiles is executed. Build environments are often seen as soft targets with access to secrets and testing environments, making them potential attack vectors. Using VMs instead of containers is also suggested as a means to provide better isolation in certain situations.

原文

A lot of our customers run into the same problem: they need to run code on behalf of their customers. Whether you're hosting user-generated Python scripts, processing custom containers, or running code in isolated environments, you end up needing fast, reliable container builds that don't become a bottleneck.

Rather than managing all the container orchestration complexity in-house, many of our customers outsource the container building to us and use our API for the heavy lifting. In this post, we'll walk through how to use the Depot API to set up and administer isolated project cache, report build metrics, and get build logs for your customer workloads.

We'll use Go to build tooling that creates and manages container builds for a multi-tenant SaaS platform.

Depot core API

The Depot core API uses buf.build, so it supports both Connect and gRPC protocols.

Thanks to Buf, we can automatically generate client libraries for many languages. In this example, we'll use Go as the backend language, but Buf can be used in many other languages.

Architecture overview

We'll build some tools to create isolated build environments for users. For a new user, we'll create a new project and a new project-scoped token. Next, we'll get container build metrics including durations. Finally, we'll retrieve the container's steps.

Getting started with the Go Client

First, let's create a new go program.

go mod init github.com/depot/saas

Next, we'll add the Connect Depot API clients.

go get connectrpc.com/connect
go get buf.build/gen/go/depot/api/connectrpc/go
go get buf.build/gen/go/depot/api/protocolbuffers/go

You can find the complete documentation for the Go client at the Buf registry.

Creating projects for customer isolation

We recommend mapping an individual user to a single Depot project. We'll build a simple command-line tool that creates projects.

mkdir -p ./cmd/project

Add this to the file cmd/project/main.go:

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	"buf.build/gen/go/depot/api/connectrpc/go/depot/core/v1/corev1connect"
	buildv1 "buf.build/gen/go/depot/api/protocolbuffers/go/depot/build/v1"
	corev1 "buf.build/gen/go/depot/api/protocolbuffers/go/depot/core/v1"
	"connectrpc.com/connect"
)

func main() {
	var customerName string
	flag.StringVar(&customerName, "name", "", "customer project name")
	flag.Parse()

	if customerName == "" {
		flag.Usage()
		return
	}

	depotToken := os.Getenv("DEPOT_TOKEN")
	if depotToken == "" {
		fmt.Fprintln(os.Stderr, "DEPOT_TOKEN required")
		return
	}

	ctx := context.Background()
	id := createProject(ctx, customerName, depotToken)
	fmt.Printf("Created Project ID %s for %s\n", id, customerName)
}

func createProject(ctx context.Context, customerName, depotToken string) string {
	depotClient := corev1connect.NewProjectServiceClient(
		http.DefaultClient,
		"https://api.depot.dev",
	)

	hardware := corev1.Hardware_HARDWARE_32X64
	req := connect.NewRequest(&corev1.CreateProjectRequest{
		Name:     customerName,
		RegionId: "eu-central-1", // or us-east-1
		CachePolicy: &corev1.CachePolicy{
			KeepGb: 30, // Keep 30 GB of cache
		},
		Hardware: &hardware,
	})

	req.Header().Add("Authorization", "Bearer "+depotToken)
	res, err := depotClient.CreateProject(ctx, req)
	if err != nil {
		log.Fatal(err)
	}

	project := res.Msg.GetProject()
	return project.ProjectId
}

This program creates a named project with a Depot API client. It expects the environment variable, DEPOT_TOKEN to be set to an API token. The token is used as a Bearer token. It sets the project's region to eu-central-1 and gives the project a 30GB cache quota. Additionally, it uses a non-default larger builder sized machine with 32 CPUs and 64GB of RAM. Those values are all configurable to give flexibility in build performance.

Here is how to run the program and its unique project ID output:

go build ./cmd/project && ./project -name my_customer

Created Project ID n9548n2qqx for my_customer

Managing customer projects

Deleting projects removes all project cache and project tokens, preventing any further builds:

func deleteProject(ctx context.Context, projectID, depotToken string) error {
	depotClient := corev1connect.NewProjectServiceClient(
		http.DefaultClient,
		"https://api.depot.dev",
	)

	req := connect.NewRequest(&corev1.DeleteProjectRequest{
		ProjectId: projectID,
	})

	req.Header().Add("Authorization", "Bearer "+depotToken)
	_, err := depotClient.DeleteProject(ctx, req)
	return err
}

Similar to creating a project, we create a client and request with bearer auth.

While managing projects, it is very useful to be able reset a project's build cache in case a customer wishes to start fresh. Here is how to do so:

func resetProject(ctx context.Context, projectID, depotToken string) error {
	depotClient := corev1connect.NewProjectServiceClient(
		http.DefaultClient,
		"https://api.depot.dev",
	)

	req := connect.NewRequest(&corev1.ResetProjectRequest{
		ProjectId: projectID,
	})

	req.Header().Add("Authorization", "Bearer "+depotToken)
	_, err := depotClient.ResetProject(ctx, req)
	return err
}

When a project has been reset, its cache will be reset and all currently running jobs will be canceled.

Getting build metrics for customer analytics

Ok, great, now that we can administer projects we can build containers for our customers. Check out the blog on how to build a container using a project id.

Let's assume several container builds have finished. We can list all those builds using the paginated ListBuilds API request. This example shows how to paginate through all of a project's builds. Likely, you'll need to add log when to stop paging when there are hundreds of builds.

Each build has an ID and some coarse timing metrics. The duration is the time it took the build to complete. The "saved duration" is the estimated time the Depot cache saved the customer for that step.

func listBuilds(ctx context.Context, projectID, depotToken string) {
	depotClient := corev1connect.NewBuildServiceClient(
		http.DefaultClient,
		"https://api.depot.dev",
	)
	listBuilds := &corev1.ListBuildsRequest{
		ProjectId: projectID,
	}

	for {
		req := connect.NewRequest(listBuilds)
		req.Header().Add("Authorization", "Bearer "+depotToken)

		res, err := depotClient.ListBuilds(ctx, req)
		if err != nil {
			log.Fatal(err)
		}

		for _, b := range res.Msg.GetBuilds() {
			fmt.Printf("Build ID: %s\n", b.BuildId)
			fmt.Printf("\tStatus: %s\n", b.Status)
			fmt.Printf("\tCreated At: %s\n", b.CreatedAt.AsTime())
			fmt.Printf("\tBuild Duration: %s\n", time.Duration(b.GetBuildDurationSeconds())*time.Second)
			fmt.Printf("\tSaved Duration: %s\n", time.Duration(b.GetSavedDurationSeconds())*time.Second)
			fmt.Printf("\tCached Steps: %d\n", b.GetCachedSteps())
			fmt.Printf("\tTotal Steps: %d\n", b.GetTotalSteps())
		}

		nextPageToken := res.Msg.GetNextPageToken()
		if nextPageToken == "" {
			break
		}

		listBuilds.PageToken = &nextPageToken
	}
}

Getting detailed build steps

Each container build has multiple steps such as transferring build context and running programs. The Depot API also provides a breakdown of each step including its name, timings, and if the step errored or not. This is useful to visualize the entire container build process.

func buildSteps(ctx context.Context, projectID, buildID, depotToken string) {
	buildClient := buildv1connect.NewBuildServiceClient(http.DefaultClient, "https://api.depot.dev")
	req := connect.NewRequest(&buildv1.GetBuildStepsRequest{
		ProjectId: projectID,
		BuildId:   buildID,
	})
	req.Header().Add("Authorization", "Bearer "+depotToken)

	res, err := buildClient.GetBuildSteps(ctx, req)
	if err != nil {
		log.Fatal(err)
	}

	for _, step := range res.Msg.GetBuildSteps() {
		fmt.Printf("Step Name: %s\n", step.Name)
		fmt.Printf("\tStarted At: %s\n", step.StartedAt.AsTime())
		fmt.Printf("\tFinished At: %s\n", step.GetCompletedAt().AsTime())
		fmt.Printf("\tHad Error: %t\n", step.HasError())
	}
}

Building container infrastructure that scales

With these building blocks, you can create solid container infrastructure for your customers. The isolated projects ensure security and performance isolation, while Depot's caching dramatically reduces build times across your customer base.

This approach works well for platforms that need to:

Execute customer code in isolated environments
Provide fast feedback loops for development workflows
Scale container builds without managing infrastructure complexity
Offer detailed build analytics to customers

Whether you're building a platform that runs customer Python code, or any other service that needs to execute user-generated containers, Depot's API provides the performance and isolation you need without the operational overhead.

Get started today

Ready to build container infrastructure for your customers? Sign up for Depot and start with a 7-day free trial. Our Go client libraries make it easy to integrate Depot into your existing infrastructure.

Have questions about implementing customer container builds? Join our Community Discord to chat with our team and other developers building similar solutions.

Chris Goller

Principal Software Engineer at Depot