Libswarm (in a nutshell)

At Dockercon, Docker announced a new project being worked on called libswarm.

I wanted to clarify what exactly libswarm is, what it does, and what it doesn't do.

First, libswarm is not itself an orchestration tool.

It does not and will not replace any orchestration tools.

Libswarm is a library first and foremost and not an end-user tool.

It is a library that helps make it relatively trivial to compose other disparate tools together, including but not limited to orchestration tools.

I see various pieces of Docker core being broken into smaller libswarm services that come together to make Docker.

I see tools that hook into this libswarm API to extend native Docker functionality. No more bind-mounting Docker sockets into containers (which, btw, is super dangerous).

Libswarm is the API you will talk to in order to interact with Docker, and not the traditional REST API (though this will probably be available in one form or another)

As part of libswarm is a reference implementation of it called swarmd.

swarmd is a proof of concept binary you run to chain services together, much like unix pipes.

* swarmd's syntax is incomplete and will probably change

Within the libswarm project (https://www.github.com/docker/libswarm), you will see a dir called "backends" (which I will from here on refer to as "services"). These are the pieces which talk to the libswarm API and interact with some external service, such as AWS, Rackspace, Orchard, Shipyard, or even some other binary (via the "exec" service), or intercept the libswarm message itself to manipulate it in some way.

The most basic swarmd setup would be this:

./swarmd 'dockerserver unix:///var/run/docker.sock' 'dockerclient tcp://1.2.3.4:2375'

This will spin up the dockerserver service, which is an implementation of the Docker REST API. dockerserver takes HTTP requests and makes libswarm messages out of them.

dockerclient receives libswarm messages and forwards them to a docker daemon (currently the normal REST API).

This doesn't really do much, but with it you can run a docker client locally talking to a unix socket at /var/run/docker.sock and have those requests forwarded to a docker daemon at 1.2.3.4:2375.

You can also do something like this:

./swarmd 'dockerserver unix:///var/run/docker.sock' 'aggregate "dockerclient tcp://1.2.3.4:2375" "dockerclient tcp://1.2.3.5:2375" "dockerclient tcp://1.2.3.6:2376"'

This does much like the first example, except it uses the aggregate service to talk to each of the listed dockerclient services and aggregate the results. You could use docker run here and aggregate will pick one of the dockerclient services to create the new container on.

You can docker ps and get a list of all the running containers on all of the listed dockerclient services, in one list.

This is also only generally useful as a demo at this time.

But imagine this:

./swarmd 'dockerserver unix:///var/run/docker.sock' 'mesos "dockerclient tcp://1.2.3.4:2375" "dockerclient tcp://1.2.3.5:2375" "dockerclient tcp://1.2.3.6:2376"'

Now I should note, there is currently no mesos backend (please contribute if you can!), but what would happen here is we could use "docker run" to pipe into Mesos which determines the dockerclient to run on based on availability, or whatever Mesos-ish criteria.

You could swap this out for Kubernetes, or whatever scheduler.

You can put as many services into this chain as you want.

./swarmd 'dockerserver unix://var/run/docker.sock' 'serviceA' 'serviceB' 'serviceC' 'dockerclient tcp://1.2.3.5:2375'

Any of these services can intercept the request, manipulate it on some rule, and forward it on down the chain... or call some external service do some stuff, forward it on down the chain... it really doesn't matter.

As long as the service being called implements the libswarm API and properly sends/receives the libswarm messages it will fit into the chain.

In each of these examples I used dockerserver on the front and dockerclient on the back. This is for convenience and the fact that these are working services.

It technically doesn't even have to be Docker here. Libswarm is completely independent of Docker.

I should also mention, libswarm uses libchan for communication.

Libswarm is also still rough around the edges until the API gets nailed down fully.

TL;DR

libswarm is about providing a common API for composing otherwise completely separate tools/services.

swarmd lets you chain these libswarm services into something sort of like unix pipes.... I prefer the referring to it as a middleware chain.

For more info on libswarm, take a look at the git repo, and join us in #libswarm on freenode.

Thanks @markwrenn for the illustration