DockerInfo design

Motivation

Currently in mesos/Docker branch we integrated Docker directly into Mesos, where we can launch Docker with the new DockerContainerizer. Once the slave is started with docker configured, there are currently two supported docker APIs:

1. Set ContainerInfo's image to a docker image (docker:///busybox) as part of the TaskInfo's CommandInfo, and we will run that docker image with the TaskInfo's command as a task for you.

2. Set ExecutorInfo's ContainerInfo's CommandInfo's image to a docker image and we expect that image to be a Mesos executor that can accept Executor API calls to launch tasks.

However, the current APIs covers very limited ground on what Docker provides and we want to be able to support more docker related options/configurations to enhance our Docker support in Mesos.

We propose to add new proto message called DockerInfo to encapsulate all Docker specific configurations that doesn't fit in all other existing proto fields.

Design

DockerInfo contains Docker specific configurations that we want to support, examples such as mounting directories, or custom login options to private registry, etc.

The following is the proposed fields to support Docker launch:

message ContainerInfo { 
  enum Type {
    DOCKER = 1;
  }

  // DockerInfo schema.
  message DockerInfo {  
    // Docker image to launch.
    required string image = 1;

    message Volume {
      // directory to be mapped in the container.
      required string container_dir = 1;

      // absolute path to the host directory to be mapped.
      optional string host_dir = 2;

      enum Mode {
        RW, // read-write.
        RO // read-only.
      }

      required Mode mode = 3;
    }
  
    // Directories in the host to bind mount into docker.
    repeated Volume volume = 2;
  }

  required Type type = 1;

  optional DockerInfo docker = 100;
}

As we support more Docker options DockerInfo will be added with more explicit optional fields into DockerInfo, such as custom entrypoint, volumes, etc.

One alternative to support all optional fields is to allow a repeated generic key value pair to be passed as part of DockerInfo. However, it becomes harder to limit options and as Docker feature set grows Mesos will also likely to require work to support these options. Therefore, we're proposing to have more explicitness in the options we support to make sure the options we support are working in Mesos.

The two existing API will be changed as DockerInfo will now be added as both optional fields into ExecutorInfo and TaskInfo.

With the new DockerInfo change, there are three paths to launch a task with TaskInfo:

• Set a CommandInfo that launches CommandExecutor to launch the task
• Set a DockerInfo that launches the Docker image as a task
• Set a ExecutorInfo that launches the Docker image in ExecutorInfo as an Executor and passes the TaskInfo to that Executor.

Pods

There has been discussions around Pods (group or nested containers to be launched at the same host) and how DockerInfo can fit into this. We decided to not introduce Pods constructs along with DockerInfo as we see the requirements from other dependencies is not yet clear and will move forward with a DockerInfo that just launches a single image.

For those that like to have Pod constructs it is best to implement a custom executor that launches multiple tasks within a single offer for now.

Future Improvements

• Does Docker Linking work as-is with Mesos? Might want to consider launching Ambassador Docker automatically for linking dockers cross hosts.