• Overview
    • Batch Processing Workloads
    • Edge Workload Management
    • Non-Containerized Application Orchestration
    • Simple Container Orchestration
  • Enterprise
  • Tutorials
  • Docs
  • API
  • Plugins
  • Tools
  • Community
GitHub—Stars on GitHub
Download
    • v1.3.x (latest)
    • v1.2.x
    • v1.1.x
    • v1.0.x
    • v0.12.x
    • v0.11.x
    • Overview
    • Quickstart
      • Overview
      • Requirements
      • Reference Architecture
      • Deployment Guide
    • Windows Service
    • Overview
    • Specific Version Details
    • Overview
    • Consul
    • Consul Service Mesh
    • Vault Integration

    • Overview
    • Architecture
      • Overview
      • Base
      • Task Drivers
      • Devices
      • Storage
      • Overview
      • Internals
      • Preemption
    • Consensus Protocol
    • Filesystem
    • Gossip Protocol
    • Security Model
    • Overview
    • acl
    • audit
    • autopilot
    • client
    • consul
    • plugin
    • sentinel
    • search
    • server
    • server_join
    • telemetry
    • tls
    • ui
    • vault
    • Overview
      • Overview
      • bootstrap
      • policy apply
      • policy delete
      • policy info
      • policy list
      • token create
      • token delete
      • token info
      • token list
      • token self
      • token update
    • agent
    • agent-info
      • Overview
      • exec
      • fs
      • logs
      • restart
      • signal
      • status
      • stop
      • Overview
      • validate
      • Overview
      • fail
      • list
      • pause
      • promote
      • resume
      • status
      • unblock
      • Overview
      • list
      • status
      • Overview
      • allocs
      • deployments
      • dispatch
      • eval
      • history
      • init
      • inspect
      • plan
      • periodic force
      • promote
      • revert
      • run
      • scale
      • scaling-events
      • status
      • stop
      • validate
      • Overview
      • get
    • monitor
      • Overview
      • apply
      • delete
      • inspect
      • list
      • status
      • Overview
      • config
      • drain
      • eligibility
      • status
      • Overview
      • api
      • autopilot get-config
      • autopilot set-config
      • debug
      • keygen
      • keyring
      • metrics
      • raft info
      • raft list-peers
      • raft logs
      • raft remove-peer
      • raft state
      • snapshot agent
      • snapshot inspect
      • snapshot restore
      • snapshot save
      • snapshot state
      • Overview
      • status
      • Overview
      • apply
      • delete
      • init
      • inspect
      • list
      • status
      • Overview
      • apply
      • dismiss
      • info
      • list
      • Overview
      • policy info
      • policy list
      • Overview
      • apply
      • delete
      • list
      • read
      • Overview
      • force-leave
      • join
      • members
      • Overview
      • service delete
      • service info
      • service list
    • status
      • Overview
      • gc
      • reconcile summaries
    • ui
    • version
      • Overview
      • create
      • delete
      • deregister
      • detach
      • init
      • register
      • snapshot create
      • snapshot delete
      • snapshot list
      • status

    • Overview
      • Overview
      • Expressions
        • Overview
          • chunklist
          • coalesce
          • coalescelist
          • compact
          • concat
          • contains
          • distinct
          • element
          • flatten
          • index
          • keys
          • length
          • lookup
          • merge
          • range
          • reverse
          • setintersection
          • setproduct
          • setunion
          • slice
          • sort
          • values
          • zipmap
          • can
          • convert
          • try
          • bcrypt
          • md5
          • rsadecrypt
          • sha1
          • sha256
          • sha512
          • formatdate
          • timeadd
          • base64decode
          • base64encode
          • csvdecode
          • jsondecode
          • jsonencode
          • urlencode
          • yamldecode
          • yamlencode
          • abspath
          • basename
          • dirname
          • file
          • fileexists
          • fileset
          • pathexpand
          • cidrhost
          • cidrnetmask
          • cidrsubnet
          • cidrsubnets
          • abs
          • ceil
          • floor
          • log
          • max
          • min
          • parseint
          • pow
          • signum
          • chomp
          • format
          • formatlist
          • indent
          • join
          • lower
          • regex_replace
          • replace
          • split
          • strrev
          • substr
          • title
          • trim
          • trimprefix
          • trimspace
          • trimsuffix
          • upper
          • uuidv4
          • uuidv5
      • Locals
      • Syntax
      • Variables
    • artifact
    • affinity
    • check_restart
    • connect
    • constraint
    • csi_plugin
    • device
    • dispatch_payload
    • env
    • ephemeral_disk
    • expose
    • gateway
    • group
    • job
    • lifecycle
    • logs
    • meta
    • migrate
    • multiregion
    • network
    • parameterized
    • periodic
    • proxy
    • reschedule
    • resources
    • restart
    • scaling
    • service
    • sidecar_service
    • sidecar_task
    • spread
    • task
    • template
    • update
    • upstreams
    • vault
    • volume
    • volume_mount
    • Overview
      • Overview
      • capability
      • mount_options
      • topology_request
    • Overview
    • Docker
    • Isolated Fork/Exec
    • Java
    • Podman
    • QEMU
    • Raw Fork/Exec
      • Overview
      • containerd
      • Firecracker driver
      • Jailtask driver
      • Lightrun
      • LXC
      • Pot
      • Rkt Deprecated
      • Rookout
      • Singularity
      • systemd-nspawn
      • Windows IIS
      • Overview
      • ECS
    • Overview
      • Overview
      • Nvidia
      • USB Beta
  • Schedulers
    • Overview
    • Runtime Environment
    • Variable Interpolation
    • Overview
      • Overview
      • apm
      • dynamic_application_sizing
      • http
      • nomad
      • policy
      • policy_eval
      • source
      • strategy
      • target
      • telemetry
    • API
    • CLI
    • Policy
    • Telemetry
      • Overview
        • Overview
        • Datadog
        • Nomad API
        • Prometheus
        • Overview
        • Dynamic Application Sizing Average
        • Dynamic Application Sizing Max
        • Dynamic Application Sizing Percentile
        • Fixed Value
        • Pass-Through
        • Target Value
        • Threshold
        • Overview
        • Amazon Web Services Autoscaling Group
        • Azure Virtual Machine Scale Set
        • Dynamic Application Sizing
        • Google Cloud Engine Managed Instance Group
        • Nomad Task Group
      • Community
      • Overview
      • Checks
      • Node Selector Strategy
        • Overview
        • Base
        • APM
        • Strategy
        • Target
    • Overview
    • Operating Nomad Agents
    • Monitoring Nomad
    • Metrics Reference
    • Cluster Management
    • Transport Security
    • Access Control

    • Overview
    • Alternative to Kubernetes
    • Supplement to Kubernetes
  • Nomad Ecosystem
  • Nomad Partnerships
  • Who Uses Nomad
    • Overview
      • Overview
      • FAQ
  • FAQ
Type '/' to Search

»QEMU Driver

Name: qemu

The qemu driver provides a generic virtual machine runner. QEMU can utilize the KVM kernel module to utilize hardware virtualization features and provide great performance. Currently the qemu driver can map a set of ports from the host machine to the guest virtual machine, and provides configuration for resource allocation.

The qemu driver can execute any regular qemu image (e.g. qcow, img, iso), and is currently invoked with qemu-system-x86_64.

The driver requires the image to be accessible from the Nomad client via the artifact downloader.

»Task Configuration

task "webservice" {
  driver = "qemu"

  config {
    image_path        = "/path/to/my/linux.img"
    accelerator       = "kvm"
    graceful_shutdown = true
    args              = ["-nodefaults", "-nodefconfig"]
  }
}
task "webservice" {
  driver = "qemu"

  config {
    image_path        = "/path/to/my/linux.img"
    accelerator       = "kvm"
    graceful_shutdown = true
    args              = ["-nodefaults", "-nodefconfig"]
  }
}

The qemu driver supports the following configuration in the job spec:

  • image_path - The path to the downloaded image. In most cases this will just be the name of the image. However, if the supplied artifact is an archive that contains the image in a subfolder, the path will need to be the relative path (subdir/from_archive/my.img).

  • accelerator - (Optional) The type of accelerator to use in the invocation. If the host machine has qemu installed with KVM support, users can specify kvm for the accelerator. Default is tcg.

  • graceful_shutdown (bool: false) - Using the qemu monitor, send an ACPI shutdown signal to virtual machines rather than simply terminating them. This emulates a physical power button press, and gives instances a chance to shut down cleanly. If the VM is still running after kill_timeout, it will be forcefully terminated. (Note that prior to qemu 2.10.1, the monitor socket path is limited to 108 characters. Graceful shutdown will be disabled if QEMU is < 2.10.1 and the generated monitor path exceeds this length. You may encounter this issue if you set long data_dir or alloc_dir paths.) This feature is currently not supported on Windows.

  • port_map - (Optional) A key-value map of port labels.

    config {
      # Forward the host port with the label "db" to the guest VM's port 6539.
      port_map {
        db = 6539
      }
    }
    
    config {
      # Forward the host port with the label "db" to the guest VM's port 6539.
      port_map {
        db = 6539
      }
    }
    
  • args - (Optional) A list of strings that is passed to QEMU as command line options.

»Examples

A simple config block to run a qemu image:

task "virtual" {
  driver = "qemu"

  config {
    image_path  = "local/linux.img"
    accelerator = "kvm"
    args        = ["-nodefaults", "-nodefconfig"]
  }

  # Specifying an artifact is required with the "qemu"
  # driver. This is the # mechanism to ship the image to be run.
  artifact {
    source = "https://internal.file.server/linux.img"

    options {
      checksum = "md5:123445555555555"
    }
  }
task "virtual" {
  driver = "qemu"

  config {
    image_path  = "local/linux.img"
    accelerator = "kvm"
    args        = ["-nodefaults", "-nodefconfig"]
  }

  # Specifying an artifact is required with the "qemu"
  # driver. This is the # mechanism to ship the image to be run.
  artifact {
    source = "https://internal.file.server/linux.img"

    options {
      checksum = "md5:123445555555555"
    }
  }

»Capabilities

The qemu driver implements the following capabilities.

FeatureImplementation
nomad alloc signalfalse
nomad alloc execfalse
filesystem isolationimage
network isolationnone
volume mountingnone

»Client Requirements

The qemu driver requires QEMU to be installed and in your system's $PATH. The task must also specify at least one artifact to download, as this is the only way to retrieve the image being run.

»Client Attributes

The qemu driver will set the following client attributes:

  • driver.qemu - Set to 1 if QEMU is found on the host node. Nomad determines this by executing qemu-system-x86_64 -version on the host and parsing the output
  • driver.qemu.version - Version of qemu-system-x86_64, ex: 2.4.0

Here is an example of using these properties in a job file:

job "docs" {
  # Only run this job where the qemu version is higher than 1.2.3.
  constraint {
    attribute = "${driver.qemu.version}"
    operator  = ">"
    value     = "1.2.3"
  }
}
job "docs" {
  # Only run this job where the qemu version is higher than 1.2.3.
  constraint {
    attribute = "${driver.qemu.version}"
    operator  = ">"
    value     = "1.2.3"
  }
}

»Plugin Options

plugin "qemu" {
  config {
    image_paths = ["/mnt/image/paths"]
    args_allowlist = ["-drive", "-usbdevice"]
  }
}
plugin "qemu" {
  config {
    image_paths = ["/mnt/image/paths"]
    args_allowlist = ["-drive", "-usbdevice"]
  }
}
  • image_paths ([]string: []) - Specifies the host paths the QEMU driver is allowed to load images from.
  • args_allowlist ([]string: []) - Specifies the command line flags that the args option is permitted to pass to QEMU. If unset, a job submitter can pass any command line flag into QEMU, including flags that provide the VM with access to host devices such as USB drives. Refer to the QEMU documentation for the available flags.

»Resource Isolation

Nomad uses QEMU to provide full software virtualization for virtual machine workloads. Nomad can use QEMU KVM's hardware-assisted virtualization to deliver better performance.

Virtualization provides the highest level of isolation for workloads that require additional security, and resource use is constrained by the QEMU hypervisor rather than the host kernel. VM network traffic still flows through the host's interface(s).

Note that the strong isolation provided by virtualization only applies to the workload once the VM is started. Operators should use the args_allowlist option to prevent job submitters from accessing devices and resources they are not allowed to access.

github logoEdit this page
DocsAPIResourcesPrivacySecurityPress KitConsent Manager