LXC and LXD: a different container story

Ready to give LWN a try?

With a subscription to LWN, you can stay current with what is happening in the Linux and free-software community and take advantage of subscriber-only site features. We are pleased to offer you a free trial subscription, no credit card required, so that you can see for yourself. Please, join us!

OCI containers are the most popular type of Linux container, but they are not the only type, nor were they the first. LXC (short for "LinuX Containers") predates Docker by several years, though it was also not the first. LXC dates back to its first release in 2008; the earliest version of Docker, which was tagged in 2013, was actually a wrapper around LXC. The LXC project is still going strong and shows no signs of winding down; LXC 5.0 was released in July and comes with a promise of support until 2027.

LXC

LXC was initially developed by IBM, and was part of a collaboration between several parties looking to add namespaces to the kernel. Eventually, Canonical took over stewardship of the project, and now hosts its infrastructure and employs many of its maintainers. The project includes a C library called liblxc and a collection of command-line tools built on top of it that can be used to create, interact with, and destroy containers. LXC does not provide or require a daemon to manage containers; the tools it includes act directly on container processes.

LXC was the first container implementation to be built entirely on capabilities found in the mainline kernel; predecessors required out-of-tree patches to work. Like Docker, LXC containers are created using a combination of control groups and namespaces. Because LXC was developed in parallel with the effort to add namespaces to the kernel, it could be considered a sort of reference implementation of using namespaces for containers on Linux.

Unlike Docker, LXC does not presume to espouse an opinion about what kinds of processes should run in a container. By default, it will try to launch an init system inside of the container, which can then launch other processes — something that is notoriously hard to do in a Docker container. With the correct configuration, though, it is even possible to run LXC containers nested within another LXC container, or to run the Docker daemon inside of an LXC container.

LXC containers are defined using a configuration file, which offers a great deal of control over how the container is constructed. The lxc-create utility is used to create containers. LXC does not bundle container configurations and images together; instead, the container configuration specifies a directory or block device to use for the container's root filesystem. LXC can use an existing root filesystem, or lxc-create can construct one on the fly using a template.

An LXC template is a shell script that constructs a root filesystem using a few key variables that lxc-create replaces before the template is run. A handful of templates are included; among them is an OCI template that uses SUSE's umoci utility to download and unpack a container image from an OCI container registry, which gives LXC the ability to run all of the same containers that Docker and other OCI runtimes can.

A separate collection of templates that can build root filesystems for a variety of popular distributions is available, but this approach has fallen out of favor because the tools that these templates use often require root privileges. These days, pre-built images are preferred because they can more easily be used by unprivileged users.

The LXC project has developed a tool called distrobuilder to create these pre-built images, which are made available on an image server hosted by Canonical. The lxc-download template can be used to create a container based on an image from an image server.

In theory, anybody can host their own image server, but in practice, few seem to do so, at least in public. There does not appear to be a large library of pre-packaged applications in this format like a user of Docker or Helm might be accustomed to. Canonical's image server only contains base images for an assortment of distributions; any additional software must be bundled into a custom image, or installed using the package manager inside the container.

Among the various options for running containers on Linux, LXC appears to be the most flexible. It comes with reasonable defaults, but it makes no effort to hide the complexity of creating a container from the user; every detail of the containers that it creates can be customized and adjusted to taste. Docker has found much popularity in papering over these details, but at the cost of flexibility compared to LXC.

LXD

LXD is a more specialized sister (or perhaps daughter) project of LXC; its development is also sponsored by Canonical. LXD was initially released in 2015; version 5.5 came out in August. Like LXC, LXD also has long-term support branches; the most recent long-term support release is LXD 5.0.x, which will be supported until 2027. As might be inferred from the name, LXD includes a daemon, which is built on top of liblxc.

LXD does away with LXC's template system in favor of being purely image-based. Because of this, Docker container images cannot be used with LXD — there is no LXD equivalent to LXC's OCI template. LXD uses the same image servers as the lxc-download template but requires a different image format; distrobuilder contains support for building images of both types (as well as plain .tar.gz images), though, and Canonical's image server carries both LXC and LXD versions of all of the images it hosts.

Like the Docker daemon, LXD is controlled by an API based on HTTP. LXD also comes with a command-line client using this API called lxc (not to be confused with the tools that come with LXC, which are named lxc-*). Also like Docker, LXD can listen on a UNIX socket, and in this mode, authentication is largely-nonexistent; access to the API socket is controlled using filesystem permissions.

As part of its Landscape suite of server-management tools, Canonical offers a role-based access control (RBAC) service that LXD can integrate with for more fine-grained access control. Landscape is only free for personal use or evaluation purposes, though; enterprises that want the additional security controls provided by this feature must subscribe to Canonical's Ubuntu Advantage service.

LXD can also be used to run virtual machines. Working with virtual machines in LXD is more-or-less identical to working with containers, and the same images can be used for both; all that needs to be done to create a VM is to pass the --vm flag to the lxc create command (once again, not to be confused with the lxc-create command from LXC). LXD uses KVM and the QEMU emulator to run its virtual machines.

Several hosts running LXD can be combined into a cluster. LXD cluster nodes coordinate with each other using a protocol based on the Raft consensus algorithm, much like some OCI container orchestrators do. Containers and virtual machines can be launched on a specific cluster node, or jobs can be distributed to arbitrary groups of nodes. Like Swarm and Kubernetes, LXD bridges cluster networks between nodes so that containers or VMs running on different nodes can communicate with each other.

LXD is an interesting project; the set of features it offers would seem to make it a viable alternative to Swarm or Kubernetes, but for the lack of compatibility with OCI containers. This seems like a curious oversight; LXC's OCI template demonstrates that it should be possible, and LXD appears to have everything else it would need to compete in that arena, but its developers are not interested. As it stands, LXD has deliberately limited its audience to the set of people interested in running system containers or virtual machines. The tools that it offers to its chosen audience are powerful; people who are weary of struggling with configuring other virtual-machine managers would be well-advised to have a look at LXD.

Conclusion

The Linux Containers project as a whole seems healthy, with committed maintainers backed by a corporate sponsor, regular releases, and long-term support. LXC offers a mature and stable set of tools, while LXD offers a more "modern" feeling user interface to the same technology, and throws in virtual machines and clustering for good measure. LXC can be made to run OCI containers, but LXD cannot; people who are deeply immersed in the world of OCI might be better-served looking for something more firmly rooted in that ecosystem. For people looking for a different kind of container, though, LXC and LXD are both solid options.

Index entries for this article
GuestArticles	Webb, Jordan

---