»Containerd Task Driver

Name: containerd-driver

Homepage: https://github.com/Roblox/nomad-driver-containerd

Containerd (containerd.io) is a lightweight container daemon for running and managing container lifecycle. Docker daemon also uses containerd.

dockerd (docker daemon) --> containerd --> containerd-shim --> runc

nomad-driver-containerd enables Nomad clients to launch containers directly using containerd, without Docker! The Docker daemon is therefore not required on the host system.

See the project's homepage for more details.

»Client Requirements

The containerd task driver is not built into Nomad. It must be downloaded onto the client host in the configured plugin directory.

»Capabilities

The containerd-driver implements the following capabilities.

FeatureImplementation
send signalstrue
exectrue
filesystem isolationnone
network isolationhost, group, task, none
volume mountingtrue

For sending signals, one can use nomad alloc signal command.
For exec'ing into the container, one can use nomad alloc exec command.

»Task Configuration

Since docker also relies on containerd for managing container lifecycle, the example job created by nomad init -short can easily be adapted to use containerd-driver instead:

job "redis" {
  datacenters = ["dc1"]

  group "redis-group" {
    task "redis-task" {
      driver = "containerd-driver"

      config {
        image = "docker.io/library/redis:alpine"
      }

      resources {
        cpu    = 500
        memory = 256
        network {
          mbits = 10
        }
      }
    }
  }
}

The containerd task driver supports the following parameters:

  • image - (Required) OCI image (Docker is also OCI compatible) for your container.
config {
  image = "docker.io/library/redis:alpine"
}
  • command - (Optional) Command to override command defined in the image.
config {
  command = "some-command"
}
  • args - (Optional) Arguments to the command.
config {
  args = [
    "arg1",
    "arg2",
  ]
}
  • privileged - (Optional) true or false (default) Run container in privileged mode. Your container will have all linux capabilities when running in privileged mode.
config {
  privileged = true
}
  • readonly_rootfs - (Optional) true or false (default) Container root filesystem will be read-only.
config {
  readonly_rootfs = true
}
  • host_network - (Optional) true or false (default) Enable host network. This is equivalent to --net=host in docker.
config {
  host_network = true
}
  • cap_add - (Optional) Add individual capabilities.
config {
  cap_add = [
    "CAP_SYS_ADMIN",
    "CAP_CHOWN",
    "CAP_SYS_CHROOT"
  ]
}
  • cap_drop - (Optional) Drop individual capabilities.
config {
  cap_drop = [
    "CAP_SYS_ADMIN",
    "CAP_CHOWN",
    "CAP_SYS_CHROOT"
  ]
}
  • devices - (Optional) A list of devices to be exposed to the container.
config {
  devices = [
    "/dev/loop0",
    "/dev/loop1"
  ]
}
  • mounts - (Optional) A list of mounts to be mounted in the container. Volume, bind and tmpfs type mounts are supported. fstab style mount options are supported.
    • type - (Optional) Supported values are volume, bind or tmpfs. Default: volume.
    • target - (Required) Target path in the container.
    • source - (Optional) Source path on the host.
    • options - (Optional) fstab style mount options. NOTE: For bind mounts, atleast rbind and ro are required.
config {
  mounts = [
    {
      type = "bind"
      target = "/tmp/t1"
      source = "/tmp/s1"
      options = ["rbind", "ro"]
    }
  ]
}

»Networking

nomad-driver-containerd supports host and bridge networks.

NOTE: host and bridge are mutually exclusive options, and only one of them should be used at a time.

  1. Host network can be enabled by setting host_network to true in task config of the job spec (see host_network under Task Configuration).

  2. Bridge network can be enabled by setting the network stanza in the task group section of the job spec.

network {
  mode = "bridge"
}

You need to install CNI plugins on Nomad client nodes under /opt/cni/bin before you can use bridge networks.

Instructions for installing CNI plugins.

 $ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
 $ sudo mkdir -p /opt/cni/bin
 $ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz

»Plugin Options

  • enabled - (Optional) The containerd driver may be disabled on hosts by setting this option to false (defaults to true).

  • containerd_runtime - (Required) Runtime for containerd e.g. io.containerd.runc.v1 or io.containerd.runc.v2

  • stats_interval - (Optional) This value defines how frequently you want to send TaskStats to nomad client. (defaults to 1 second).

An example of using these plugin options with the new plugin syntax is shown below:

plugin "containerd-driver" {
  config {
    enabled = true
    containerd_runtime = "io.containerd.runc.v2"
    stats_interval = "5s"
  }
}

Please note the plugin name should match whatever name you have specified for the external driver in the plugin_dir directory.