I’ve been playing lately with virtualization technology and was struggling to understand how the hypervisors like QEMU/KVM , LXC and Docker related. So once I figured it out I decided I should write it down.
#TL;DR
- Hypervisors provide the most in terms of virtualization, but they are costly in terms of resources
- LXC lets you run Linux guest on your Linux host at a discount compared with Hypervisors. But they are limited to that Linux on Linux
- Docker provide a single process virtualization. LXC-based. Think of it as a new way to distribute and deploy self-contained applications (app+libraries+other tools) that run isolated from the host.
#Hypervisors Probably hypervisors are the best known virtualization technology. It involves emulating the HW so that you can install almost any OS on it. It allows you for example to run a Windows 7 as guest OS on top of a Linux host and that kind of stuff. QEMU/KVM, VirtualBox and VMWare are examples hypervisors.Hypervisors are great but they consume the most resources (compared to LXC and Docker). In essence you are running the guest OS kernel on top of virtualized HW provided by the hypervisor.
#LXC If you happen to run Linux guest on top of a Linux host often then LXC may be a better alternative. LXC doesn’t virtualize HW like an hypervisor. It virtualizes the Linux kernel only.
What do I mean by virtualize the Linux kernel? Well, it’s really like namespaces. All your LXC containers (which emulate independent machines) will share the same kernel as the host. How? After all there are a lot of unique IDs exposed that shouldn’t be shared like PIDs, NICs, the filesystems ,etc. Well LXC provides like namespaces for all those things. So you can think that there are several processes with PID 1 (init) but now it’s 1@host , 1@lxc1 and so on. But in essence all of them are handle by a single kernel, it’s just if you run commands like ps in that namespace that the LXC container represents you only see the kernel structures that belong to that container, so ps will only show the PIDs in the current container. Again there is only one kernel just that a lot of stuff is hidden / isolated
Still you get the feeling of a VM, your LXC container will behave like a completly different machine and can even have a different linux distro. Most of the software doesn’t have strong dependencies to the linux kernel version so you can ever run a LXC with Ubuntu 14.04 on a Ubuntu 12.04 host. Your LXC will have different versions of all the tools like ps, ls,etc but that’s ok those tools can still “talk” to the older kernel. But of course if you install an application that actually tries to make use of some newer kernel feature it won’t work. You are limited by the kernel features of the host.
LXC container are faster to setup than hypervisors, and take less resources.
Docker
Usually LXC container emulate a complete machine like I said earlier. That mean that if you have 3 LXC containers you will have 3 sshd servers running, one for each container.
Docker builds on LXC and the way I see it allows you run just one process on the a separate “namespace”.
Just one process, what is the purpose of that?. Well, seems that it’s a good way to deliver binaries without having to worry to much about dependencies. Nowadays we don’t think too much about the hassle of installing dependencies since it’s usually handled quite will by apt-get and the like. But if you think about all the effort that the community puts into curating the list of software, libraries, etc that apt-get allows you install, it’s not an easy task.
Also apt-get allows you to install whatever versions are available there. So enter Docker it will allow to distribute your software as an image that already contains all the dependencies installed. As a developer you can distribute you app as a Docker container and you can put whatever version of the system libraries that you want, without having to worry about interference with any app. And I think that is in essence the beauty of it. As a developer you can pack you application with all dependencies and you don’t need to worry on testing and providing a .deb for debian and .rpm.
As a user you get application that are fully complete, you don’t need to worry about system requirements in terms of installed libraries, only Linux kernel version requirements (which is usually not a problem). You also get the benefits of full isolations between the dockerized app and your system, it’s like chroot but on steroids.