Skip to content

Commit

Permalink
backport of commit 0f6561b (#24218)
Browse files Browse the repository at this point in the history
Co-authored-by: James Rasell <[email protected]>
  • Loading branch information
hc-github-team-nomad-core and jrasell authored Oct 15, 2024
1 parent d3de99f commit 3d59d52
Show file tree
Hide file tree
Showing 7 changed files with 395 additions and 0 deletions.
3 changes: 3 additions & 0 deletions website/content/partials/virt-beta-callout.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
~> Beta Feature: The Virt task driver is in beta and under active development.
Refer to the [repo README](https://github.com/hashicorp/nomad-driver-virt) for
the latest updates. Do not use this driver in production environments.
97 changes: 97 additions & 0 deletions website/content/plugins/drivers/virt/client.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
layout: docs
page_title: Client
description: Nomad client requirements, capabilities, and configuration options for the Virt task driver.
---

# Client

@include 'virt-beta-callout.mdx'

The `virt` task driver is not built into Nomad. You must [downloaded][releases] onto the client
host in the configured plugin directory. The client host must also be capable of running hardware
accelerated KVM virtual machines, which you can check by using [kvm-ok][] command.
Additional requirements:

- Linux
- Commands `bridge-utils`, `cloudinit`, `dnsmasq`, and `iptables`
- [libvirt daemon][] installed and running
- QEMU system packages installed
- Nomad client running as root

Use the following command to install the package requirements:

```console
$ sudo apt-get update && \
sudo apt-get install -y \
bridge-utils \
cloud-init \
dnsmasq \
iptables \
libvirt-daemon-system \
qemu-system \
qemu-system-x86 \
qemu-utils
```

## Client capabilities

The `virt` task driver implements the following driver [capabilities][capabilities].

| Feature | Implementation |
|----------------------|----------------|
| `nomad alloc signal` | false |
| `nomad alloc exec` | false |
| filesystem isolation | image |
| network isolation | host |
| volume mounting | false |

## Plugin configuration

This example shows the default plugin configuration.

```hcl
plugin "nomad-driver-driver" {
config {
data_dir = "/var/lib/virt"
image_paths = ["/var/lib/virt", "${ALLOC_DIR}"]
emulator {
uri = "qemu:///system"
user = ""
password = ""
}
}
}
```

- `data_dir` - (default: `"/var/lib/virt"`) - the directory for storing VM configuration files and
intermediate files.

- `image_paths` - (default: `["/var/lib/virt", "${ALLOC_DIR}"]`) - an allow-list of paths the driver
is allowed to load an image from.

- `emulator` - (block)
- `uri` - (default: `"qemu:///system"`) - The `uri` specifies which hypervisor to connect to.
- `user` - (default: `""`) - the username to use for authentication.
- `password` - (default: `""`) - the password to use for authentication.

## Client attributes

When installed, the `virt` plugin provides the following node attributes that you can use as
constraints when authoring jobs.

```
driver.virt = true
driver.virt.active = 0
driver.virt.emulator.version = 8002002
driver.virt.inactive = 0bytes
driver.virt.libvirt.version = 10000000
driver.virt.network.default.bridge_name = virbr0
driver.virt.network.default.state = active
```

[releases]: https://releases.hashicorp.com/nomad-driver-virt
[kvm-ok]: https://manpages.ubuntu.com/manpages/bionic/man1/kvm-ok.1.html
[libvirt daemon]: https://libvirt.org/daemons.html
[capabilities]: https://developer.hashicorp.com/nomad/docs/concepts/plugins/task-drivers#task-driver-plugin-api
25 changes: 25 additions & 0 deletions website/content/plugins/drivers/virt/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
layout: docs
page_title: 'Drivers: virt'
description: >-
The virt task driver leverages Linux capabilities and libvirt to run virtual
machines.
---

# Virt task driver

@include 'virt-beta-callout.mdx'

Name: `virt`

The `virt` driver utilises the [libvirt][] API to run and manage virtual machines (VMs) via
hypervisors such as [QEMU][]. It supports a wide variety of Nomad features including
[workload identity][], [task templates][], and [service discovery][].


[libvirt]: https://libvirt.org/
[QEMU]: https://www.qemu.org/
[workload identity]: https://developer.hashicorp.com/nomad/docs/concepts/workload-identity
[task templates]: https://developer.hashicorp.com/nomad/docs/job-specification/template
[service discovery]: https://developer.hashicorp.com/nomad/docs/networking/service-discovery
[github]: https://github.com/hashicorp/nomad-driver-virt
73 changes: 73 additions & 0 deletions website/content/plugins/drivers/virt/installation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
layout: docs
page_title: Installation
description: Installation of the virt task driver.
---

# Installation

@include 'virt-beta-callout.mdx'

Use the following instructions to download the task driver for use on your Nomad client host.

<Tabs>
<Tab heading="Manual installation" group="manual">

Download a [precompiled binary](https://releases.hashicorp.com/nomad-driver-virt/) and
verify the binary using the available SHA-256 sums. Unzip the package. Make sure that the `nomad-driver-virt` binary is on your
[plugin_dir](/nomad/docs/configuration#plugin_dir) path, specified by the client's config file,
before continuing with the other guides.

</Tab>
<Tab heading="Ubuntu/Debian" group="manual">

Install the required packages.

```shell-session
$ sudo apt-get update && \
sudo apt-get install wget gpg coreutils
```

Add the HashiCorp [GPG key][gpg-key].

```shell-session
$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
```

Add the official HashiCorp Linux test repository.

```shell-session
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) test" | sudo tee /etc/apt/sources.list.d/hashicorp.list
```

Update and install.

```shell-session
$ sudo apt-get update && sudo apt-get install nomad-driver-virt
```

</Tab>
<Tab heading="CentOS/RHEL" group="linux">

Install `yum-config-manager` to manage your repositories.

```shell-session
$ sudo yum install -y yum-utils
```

Use `yum-config-manager` to add the official HashiCorp Linux repository.

```shell-session
$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
```

Edit the repo file at `/etc/yum.repos.d/hashicorp.repo` and set `enabled=1` for `[hashicorp-test]`.

Install.

```shell-session
$ sudo yum -y install nomad-driver-virt
```

</Tab>
</Tabs>
155 changes: 155 additions & 0 deletions website/content/plugins/drivers/virt/task-config.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
---
layout: docs
page_title: Task Config
description: Configure the virt task driver job specification parameters.
---

# Task configuration

@include 'virt-beta-callout.mdx'

Task configuration for an virt task includes setting an `image` to provide the base VM image as
well as other parameters required for bootstrapping a VM instance.

- `image` - (string: required) - Path to an `.img` cloud image to base the VM disk on. The image should
be located in an allowed path. The cloud image needs to include cloud-init, otherwise certain
task features are not available. You may also downloaded via the [artifact block][].

- `use_thin_copy` - (bool: `false`) - Make a thin copy of the image using QEMU, and use it as the
backing cloud image for the VM.

- `hostname` - (string: `"nomad-<task-name>"`) - The hostname to assign to the VM. As this value is
used as a network host name, it must be a valid DNS label according to RFC 1123. Nomad generates the default
value using the Nomad task name.

- `cmds` - (list(string): optional) - List of commands to execute on the VM once it is running.
These commands can provide you with a quick way to start a process on the newly created
VM. When used in conjunction with the template, this command list can be a powerful start up tool.

- `default_user_password` - (string: `""`) - Initial password for the default user
on the newly created VM. When set, the user must update the password on first connect.

- `default_user_authorized_ssh_key` - (string: `""`) - SSH public key added to the SSH
configuration for the default user of the cloud image distribution.

- `user_data ` - (string: `""`) - Path to a cloud-init compliant user data file used as the user-data for the cloud-init configuration.

- `primary_disk_size` - (int: required) - Disk space to assign to the VM. Must fit the VM's OS.

Example:
```hcl
config {
image = "local/focal-server-cloudimg-amd64.img"
use_thing_copy = false
hostname = ""
cmds = ["python -m http.server 8000"]
#default_user_password = "CHANGE-ME"
default_user_authorized_ssh_key = ""
user_data = ""
primary_disk_size = 10000
}
```

##### os

The VM architecture and machine can be different from the client host. When this is the case, you must set the
`os` block. By default, this information matches the client host where the driver
plugin is running.

- `arch` - (string: `""`) - The VM architecture.

- `machine` - (string: `""`) - The machine type.

Example:
```hcl
config {
os {
arch = "x86_64"
machine = "pc-i440fx-2.9"
}
}
```

##### network_interface

You need to attach tasks to a libvirt network in order to access the internet and route
from the local network. The `network_interface` block enables this as a list of network interfaces
that the driver should attach to the VM. This feature only supports a single entry.

- `bridge` - (block) - Attach the VM to a bridge network. `virbr0`, the default libvirt
network, is a bridge network.

- `name` - The name of the bridge interface to use. This relates to the output from commands
such as `virsh net-info`.

- `ports` - A list of port labels to expose on the host via mapping to the network
interface. These labels must exist within the job specification [network block][].

Example:
```hcl
config {
network_interface {
bridge {
name = "virbr0"
ports = ["ssh", "http"]
}
}
}
```

### Full example

The following job specification displays use of all the configuration options, as well as other
blocks that are useful when deploying tasks via the virt driver. You need to modify parameters, such as the
`default_user_authorized_ssh_key`, before use depending on your
requirements and Nomad setup.

```hcl
job "virt-example" {
group "virt-group" {
network {
mode = "host"
provider = "nomad"
port "ssh" {
to = 22
}
}
task "virt-task" {
driver = "nomad-driver-virt"
artifact {
source = "http://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img"
destination = "local/focal-server-cloudimg-amd64.img"
mode = "file"
}
config {
image = "local/focal-server-cloudimg-amd64.img"
primary_disk_size = 26000
#default_user_password = "CHANGE-ME"
default_user_authorized_ssh_key = "ssh-ed25519 AAAAC3NzaC1lZDI..."
network_interface {
bridge {
name = "virbr0"
ports = ["ssh"]
}
}
}
resources {
cores = 2
memory = 4096
}
}
}
}
```

[artifact block]: https://developer.hashicorp.com/nomad/docs/job-specification/artifact
[network block]: https://developer.hashicorp.com/nomad/docs/job-specification/network
21 changes: 21 additions & 0 deletions website/data/docs-nav-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -1989,6 +1989,27 @@
"title": "Raw Fork/Exec",
"path": "drivers/raw_exec"
},
{
"title": "Virt <sup>Beta</sup>",
"routes": [
{
"title": "Overview",
"href": "/plugins/drivers/virt"
},
{
"title": "Installation",
"href": "/plugins/drivers/virt/installation"
},
{
"title": "Client",
"href": "/plugins/drivers/virt/client"
},
{
"title": "Task Config",
"href": "/plugins/drivers/virt/task-config"
}
]
},
{
"title": "Community",
"routes": [
Expand Down
Loading

0 comments on commit 3d59d52

Please sign in to comment.