Making sense of development containers

In the past few months, I’ve been experimenting with development containers, or dev containers for short: reproducible, containerized development environments that can be shared just like regular container images. The idea behind development containers is that they enable an identical development experience within a team working on a specific project. However, as development container technology is still in its infancy, there isn’t a standard yet that has been adopted across industries. In this post, I’ll walk you through the world of development containers using an open-source format: the Development Container Specification.

What are development containers?

First of all, let’s talk a bit about what development containers actually are. As I mentioned above, development containers aim to reproduce development environments across users. You might ask yourself: didn’t we already solve reproducible environments with container technology?

The answer is yes, and containers are still part of the final solution! However, the way we currently use container images is not suitable for development environments. We mostly use images to create a reproducible environment for the applications that we deploy. Because these images are meant to be used only by our applications, they tend to be as small as possible to speed up build and deployment time. An example is the Alpine Linux distribution.

Small container environments like Alpine Linux are great for applications that need to scale quickly, but not for engineers that want to be productive! As a developer, you want access to all the tools that enable you to do your work properly. This might include utilities such as the AWS CLI and Terraform, productivity tools like GitHub Copilot or CodeWhisperer, or development personalization tweaks like an IDE theme.

So how exactly do we specify such a development environment? File formats like Dockerfile were never meant to describe development environments. Instead, they are used to configure the dependencies and runtime of an application container. As a result, we need a separate specification that can help us to define development environments. If this still sounds a bit, cryptic, no worries! We will get to the specifics soon.

Development container specifications

There is a small issue though: there isn’t an agreed-upon standard within the community - at least not yet. Instead of trying to walk you through all the different standards, I’m going to use a single specification to illustrate how development containers work in practice: the Development Container Specification.

Development Container Specification

The goal of the Development Container Specification is simple: enrich existing file formats like Dockerfile with extra metadata that define development tools, dependencies, and a set of commands that can be run throughout the container lifecycle.

The Development Container Specification is defined in a devcontainer.json file. It can be placed in the root, or in a separate .devcontainer directory. Below is an example of a working devcontainer.json file that spins up a NodeJS development environment with the AWS CLI preinstalled. In addition, it automatically adds GitHub Copilot as a VS Code workspace extension. After the container has been created, it will run npm install so that our project dependencies are installed as well.

 "name": "my-dev-container",
 // Below we extend an existing development container image. 
 // If you want to build one from scratch, you can also use a Dockerfile or a Docker Compose file. 
 // See https://containers.dev/guide/dockerfile
 "image": "mcr.microsoft.com/devcontainers/javascript-node:1-20-bullseye",

 // Features are dependencies that you can add to your container. 
 // They include utility tools and dependencies that are not available in your base image. 
 // See https://containers.dev/features.
"features": {
    "ghcr.io/devcontainers/features/aws-cli:1": {
      "version": "latest",
    }
}

 // Configure tool-specific properties, like IDE extensions.
 "customizations": {
  // Configure properties specific to VS Code.
  "vscode": {
   "settings": {},
   "extensions": [
    "GitHub.copilot"
   ]
  }
 },

 // This is an example of a container lifecycle command. 
 // For more info about the available commands, see https://containers.dev/implementors/spec/ 
 "postCreateCommand": "npm install"
}

The above example covers what I feel are some of the most useful configuration properties:

• image, used to set a base development container image, similar to the FROM command in a DockerFile.
• features, used to define a set of dependencies that are missing in the base image that you use.
• customizations, used to specify any tool-specific properties like IDE extensions.
• postCreateCommand and other commands, which allow us to control the development environment throughout the container lifecycle.

So we have a working configuration now, but how do we actually build a development container and integrate it into our IDE? The VS Code team has open sourced their implementation, called devcontainers-cli. This CLI can be used to build development container images which you can then share with others.

This is roughly what is happening under the hood when you build and start a container with devcontainer up:

1. A docker image template is prepared under the hood.
2. If there are any features defined in devcontainer.json or in the development container base image, it will insert extra layers into the template image (if you’re interested in this, take a look at this article by Ken Muse).
3. The container is built and the metadata specified in the devcontainer.json is added as a Docker label. If you’re extending a base development container image, the metadata from this image is also included!

After a container is up and running, you can execute commands on it with devcontainer exec. This can be used to install your IDE server and any extensions without baking them into the base image. The devcontainers CLI documentation contains some example scripts on how to implement this here.

Ideally we don’t want to orchestrate this whole container setup ourselves. This is where third party tooling like IDE extensions comes into play (or in the case of cloud development environments, the setup is managed for us in the cloud!). For VS Code, Microsoft maintains an official extension called Dev Containers. This extension will create a development container based on your configuration, spin it up, install the VS Code server on the container, and finally connect the local IDE to the IDE server running in the container. The extension also adds some handy features like rebuilding your container, automatically generating devcontainer.json files for your project, or cloning your source code into a Docker volume for better performance.

Not every IDE or platform offers the same level of support for the devcontainer.json file format. In fact, some platforms might support an entirely different specification! Take a look at the JetBrains Dev Containers plugin for example, which implements the same specification. You’ll notice that you won’t be able to use several features that are supported in VS Code. If you don’t want to build your own custom solution to set up development containers, be sure to check if your preferred specification is supported by your IDE or platform!

Overall, my personal experience with the devcontainer.json format has been very positive. UX-wise it has been great - with a simple click you’ve got yourself a full development environment that is accessible from your IDE. What I also like is that you can extend development container images, allowing you to make small adjustments based on your project needs without the need to copy and paste whole sections of the original base image.

This is also a bit of a double-edged sword: how do we know if a base image is not running any malicious extensions? Right now, you’d have to either look into the source code for the original devcontainer.json file or inspect the running container for any labels. As the specification evolves, I imagine this issue will likely have to be addressed.

What standard should you choose?

In this article I showed how you development containers work in practice using the devcontainer.json file format, but there’s other specifications as well. This begs the question: which of these should you choose for your projects? Are we going to end up maintaining multiple formats just to support our dev environments on different platforms?

Right now, I feel that the biggest hurdle to widespread adoption of development containers is the amount of specification standards and the level of their implementation. Because the final implementation of the specification is up to third party tooling, CDE and IDE vendors will inevitably offer different levels of support for different specifications. As we’ve seen, even the implementation devcontainer.json specification is inconsistent between IDE vendors. CDE vendors are facing the exactly same issue: GitHub Codespaces offers support for devcontainer.json, AWS CodeCatalyst uses the devfile.yml files, and GitPod only allows their custom gitpod.yml format.

I personally feel that the Dev Container Specification is most likely to be adopted throughout the community, simply due to the huge popularity of GitHub and VS Code. In addition, the VS Code team has done a pretty good job with open sourcing the specification and its implementation. Finally, I have read through a lot of specifications and Microsoft’s documentation has so far been much more approachable than the others.

In the end, it’s up to you and the platform that you are planning to develop on. Do you work locally or with GitHub Codespaces? Take a look at how to integrate the devcontainer.json specification in your project. Does local development not really matter to you, or do you want to work with AWS CodeCatalyst or GitPod? Then it might be worth it to investigate respectively the devfile.yml or gitpod.yml specification.

I hope this post has been helpful in your own dev container journey. In my next article on development containers, I will talk about the different Cloud Development Environment (CDE) vendors and my experiences with them.