Virtual Machine Orchestration

Cockpit provides a centralized management console to orchestrate the lifecycle of virtual machines (VMs) running across physical Vapor hosts. Administrators can monitor performance, provision new workloads, manage virtual device attachments, execute power actions, and coordinate backup policies from a single control pane.

---

1. Centralized VM Inventory

Cockpit consolidates virtual machines across all connected Vapor hosts into a single inventory.

Key Features

• Unified Inventory: View, search, and organize virtual machines into logical groupings regardless of the physical host they reside on.
• Resource and Metrics Panel: Displays real-time charts of CPU utilization, memory allocation, network traffic (IP/MAC addresses), and disk read/write throughput.
• Console Access: Open direct interactive console sessions (VNC or SPICE protocol) in the web browser. This acts as a virtual monitor and keyboard for direct operating system access and troubleshooting.

---

2. Virtual Machine Provisioning Wizard

The provisioning wizard guides administrators through creating virtual machines and templates. Its steps adapt to the creation type chosen first: a from-scratch build exposes the full set of hardware steps, while clone, deploy, and convert flows collapse to only the fields they need.

Creation types

Type	What it does
Create a New Virtual Machine	Build a VM from a blank slate — placement, OS, firmware, storage, network, PCI, and cloud-init.
Deploy from Template	Instantiate a new VM from an existing template, choosing its name, target storage, and guest identity.
Clone an Existing VM	Create an independent full copy of a VM's configuration and disks, with its own guest identity or as an exact copy.
Clone a VM to Template	Copy a VM and register the copy as a reusable template.
Convert Template to VM	In-place conversion of a template back into a runnable VM.
Clone Template to Template	Create a copy of an existing template.
Convert VM to Template	In-place, destructive conversion of a VM into a template.

Steps for a from-scratch build

1. Creation Type: Pick one of the modes above.

2. Placement Target: Choose the datacenter, cluster, and host. Cockpit shows each host's free CPU and RAM to assist with scheduling.

3. Identity & OS: Set the VM name; the initial and maximum vCPUs and Memory (MiB) — the maximum values reserve headroom for CPU/memory hot-plug; autostart; firmware (UEFI, Secure Boot — requires q35 on x86_64, and TPM); and the guest OS family (Linux, Windows, FreeBSD, Other) and variant, which applies libvirt driver presets.

4. Storage: Add one or more disks. Each disk has an action:

   • Create New — allocate a new disk (qcow2, raw, or vmdk) of a given size on a datastore.
   • Clone From — copy an existing volume.
   • Attach Existing — attach a volume already present on a datastore.
   • Direct Raw Device (DRM) — map a raw host block device (raw format).

   You also choose the disk bus (VirtIO, SCSI, IDE, or SATA), and can Add ISO/CD-ROM media.

5. Network: Add network interfaces. For each, choose the type (Network, Bridge, Direct/macvtap, or User mode), the source virtual switch (a standard bridge or a cluster distributed network), the driver model (VirtIO, e1000, rtl8139, or vmxnet3), and optionally a fixed MAC address.

6. Advanced: Attach PCI passthrough devices such as GPUs or network controllers. Each entry maps a host PCI address, with optional guest address, ROM file, multi-function, and primary-GPU settings.

7. Cloud-Init: Inject cloud-init customizations across five tabs — User Data, Metadata, Network Data, SSH Keys, and Packages — for automatic post-boot provisioning.

8. Review and Create: Verify the configuration and dispatch the asynchronous create request to the target host. An optional Power on VM after creation toggle boots the guest as soon as it is built.

Clone, deploy, and convert flows show a reduced set of steps — typically source selection, a new name, and (where disks are copied) a target storage pool. Cloning a VM and deploying from a template also ask for the Guest identity; see Guest identity of clones.

---

3. Lifecycle and Power Operations

Cockpit exposes the following power controls for a VM:

• Power On: Boots the guest operating system.
• Pause: Suspends execution, freezing the VM's active CPU and memory state in place.
• Resume: Restores CPU execution and memory state for a paused VM.
• Restart Guest OS: Requests an in-guest (ACPI) reboot, letting the operating system restart cleanly.
• Reset (Hard): Immediately resets the VM — the equivalent of a reset button, with no guest cooperation.
• Power Off (Hard): Immediately powers off the VM (similar to disconnecting power), terminating the domain without a graceful guest shutdown.

Each action is offered only when it applies to the VM's current state — for example, Resume appears only for a paused VM, and Restart Guest OS / Reset (Hard) only while it is running.

---

4. Hardware and Spec Modification

The Edit Settings dialog changes a VM's virtual hardware. It is organized into Hardware, Options, and Advanced Parameters tabs, and applies to both running and stopped VMs (some changes take effect on next boot).

• Resources & CPU: Adjust vCPUs and Memory, the CPU topology (sockets / dies / cores / threads), and the CPU model. Max vCPUs and Max Memory ceilings — set above the current values — reserve headroom for CPU and memory hot-plug.
• Device Management:
  • Hard Disks: Add or detach virtual disks. New disks can be created, cloned, attached, or mapped as a direct raw device, with a selectable bus and format; CD-ROM/ISO media is supported.
  • Network Adapters: Add or remove network interfaces (Network, Bridge, Direct/macvtap, or User mode; VirtIO / e1000 / rtl8139 / vmxnet3 models).
  • PCI Device Passthrough: Map host PCI devices (e.g., GPUs, network controllers) directly to the VM, with ROM file, multi-function, and primary-GPU options.
• Firmware & Options: Toggle UEFI, Secure Boot, and TPM, set architecture and machine type, and control autostart with host.
• Advanced Parameters: A read-only view of the VM's current domain definition (autostart and other libvirt parameters), for reference and troubleshooting. This tab reflects the live configuration; it does not edit it.

---

5. Cloning and Template Management

Templates and cloning standardize VM configurations and speed up provisioning. All of these run from the provisioning wizard's creation-type step:

• VM Cloning: Creates an independent copy of a source VM's configuration and virtual disks. The wizard prompts for the source VM, the new VM name, the target storage pool for disk duplication, and the Guest identity. Every disk image is copied in full, so the clone does not depend on the source afterwards and larger disks take longer; linked (copy-on-write) clones are not available. A Direct Raw Device, read-only disks, and ISO media are not copied: the clone uses the same device or file as the source, so do not run both VMs at the same time while they share a writable device.
• Deploy from Template: Provisions a new VM from a template, choosing its name, destination storage, and Guest identity.
• Clone VM to Template: Copies a source VM and registers the copy as a read-only template in the VMs and Templates inventory. The template dialog also captures an optional description and target storage pool. Templates are always exact copies; the identity is reset, if you choose so, when you deploy from the template.
• Clone Template to Template: Duplicates an existing template.
• Convert VM to Template: Permanently converts a VM into a template in place — a status transition that marks the VM as a template.
• Convert Template to VM: Reverts a template back into an independent virtual machine, restoring normal power-cycle and configuration-edit operations.

Guest identity of clones

A plain disk copy also copies everything the guest operating system uses to tell itself apart from other machines. When the source and an exact copy run at the same time, both VMs may get the same IP address from DHCP, a clone of a VM built from a cloud image may start with no network at all, and both VMs show the same hostname and SSH host keys.

The Clone Options step of the wizard (for Clone an Existing VM and Deploy from Template) therefore has a Guest identity choice:

Option	What you get
Reset guest identity (recommended)	A clone with its own identity. The host edits the copied disks before the clone is started for the first time. The source VM or template is never changed.
Keep an exact copy	A byte-for-byte copy of the source's disks. Use it for restore tests, for a copy you keep for investigation, or for appliances whose licence is bound to the machine ID.

Reset guest identity is selected by default when the target host can reset identities. When it cannot, Keep an exact copy is selected, the reset option is disabled, and the step shows the reason (see Host requirement). The step also shows the hostname the clone will get and a short summary of what is and is not reset.

What a reset changes

• Machine ID: a new random ID, so DHCP servers that recognise clients by it give the clone its own address. A machine ID set to uninitialized (a template prepared for first boot) is left as it is.
• SSH host keys: new keys of the same types and sizes. Clients that connected to the source see a different host key on the clone. An old DSA key is removed when the VM also has a newer key type. RSA keys larger than 4096 bits are replaced by 4096-bit keys, and the result includes a note about it.
• Network MAC bindings: the clone gets new MAC addresses, and network configuration that names the source's MAC addresses (netplan, NetworkManager, ifcfg files, systemd-networkd, udev rules, /etc/network/interfaces) is updated so the interfaces come up.
• DHCP leases: leases saved by the source are removed.
• Random seed: replaced with new random data.
• Hostname: set from the clone name, in lowercase. Every character other than ASCII letters, digits, and hyphens becomes a hyphen, repeated hyphens are merged into one, leading and trailing hyphens are removed, and the result is cut to 63 characters. The domain of the source's hostname is kept when it is a valid DNS domain: web01.example.com cloned as web02 becomes web02.example.com. Matching names on the loopback lines of /etc/hosts are updated too. If nothing usable remains of the clone name, or the guest has no /etc/hostname, the hostname is left as it is.

User accounts and passwords, users' SSH authorized keys, installed software and drivers, application data, and static IP addresses stay as they were in the source.

Progress and results

While the host edits the copies, the Tasks panel shows Resetting guest identity…. A clone that was created but needs your attention ends as Completed with warning; open the task to see the details. Typical warnings:

• Static IP address kept: the clone has the same address as the source and is created with autostart turned off. Change the address before both VMs run on the same network, for example: keep the source stopped, start the clone, change its address from the console, then start the source again.
• Joined to an Active Directory or Kerberos domain: the clone still uses the source's machine account, which can lock the source out of the domain. Remove the clone from the domain and join it again under its new name before connecting it to the network.
• cloud-init manages the hostname: cloud-init may set the source's hostname again on every boot. If the clone keeps showing the source's name, change the hostname in the clone's cloud-init configuration.

Registration and management agents (Red Hat Subscription Manager, Ubuntu Pro, Puppet, Salt, Chef) keep the source's registration; register the clone again.

When the identity cannot be reset

If you chose Reset guest identity and it is not possible, the clone fails with the reason, shown on the Clone Options step or in the task, and no VM is created. Most cases are found before anything is copied. Choose Keep an exact copy instead, then change the identity inside the guest before running both VMs:

Situation	What to do
Windows or another non-Linux guest	Keep an exact copy, then generalize the guest (on Windows, with Sysprep).
Encrypted disks (for example LUKS or BitLocker)	Keep an exact copy and change the identity inside the guest.
No operating system found on the disks, or more than one	Keep an exact copy.
A file system the host cannot modify, for example the XFS file system of RHEL 9 and its rebuilds (Rocky Linux, AlmaLinux) on hosts with an older kernel	Keep an exact copy and change the identity inside the guest.
Part of the guest's system, such as /var, is on a disk that is not copied with the clone (a Direct Raw Device or a read-only disk)	Move that part of the system to a disk image in the source VM, then clone again. An exact copy would still share that device with the source.
The VM has more than one network interface, and its network configuration names a MAC address that none of the source's network interfaces has	Remove or correct the stale entry in the source VM, then clone again. With guest_identity set to auto, this is only a warning and the reset still completes.
The only SSH host key is a DSA key	Add a current key type in the source VM (sudo ssh-keygen -A), then clone again.
The host's Vapor version cannot reset identities	Upgrade Vapor on the host, or keep an exact copy.

To change the identity of a Linux clone yourself, run inside the clone:

sudo rm -f /etc/machine-id /var/lib/dbus/machine-id
sudo systemd-machine-id-setup
sudo hostnamectl set-hostname <new-name>
sudo rm -f /etc/ssh/ssh_host_*
sudo ssh-keygen -A
sudo systemctl restart ssh    # the service is named sshd on RHEL-family guests

If something goes wrong while the host is editing the copied disks, the clone fails as a whole: the copies are removed and no VM is created. The source is not affected.

Templates

Templates themselves are always exact copies: Clone VM to Template, Clone Template to Template, and Convert VM to Template never change the guest's identity. The choice is made each time you deploy from the template. A template created by cloning a VM with an earlier Vapor version may cause a reset to fail when the VM has several network interfaces; clone the template again from the original VM to fix this.

Guest agent

A clone keeps the source's QEMU guest agent connection, so features that rely on the agent, such as filesystem quiescing for replication and backups and guest IP reporting, work on the clone as they do on the source.

Host requirement

Resetting the identity needs guestfish (part of libguestfs) on the Vapor host that runs the clone. Install it as root on that host:

# Debian 12 and later, Ubuntu 22.04 and later
sudo apt-get install guestfish libguestfs-xfs

# Debian 11, Ubuntu 20.04
sudo apt-get install libguestfs-tools libguestfs-xfs

# RHEL 9 and rebuilds (Rocky Linux, AlmaLinux)
sudo dnf install libguestfs libguestfs-xfs

# RHEL 8 and rebuilds
sudo dnf install libguestfs-tools-c libguestfs-xfs

Vapor does not need a restart: it notices the tool and checks it with a short self-test in the background. Open the wizard again after a minute. If Reset guest identity is still unavailable, the reason on the step says what the host is missing.

For API clients

POST /api/v1/vms/{id}/clone and POST /api/v1/vms/from-template accept a guest_identity field: reset (reset the identity, or fail with 409 and the code IDENTITY_RESET_UNAVAILABLE or IDENTITY_RESET_UNSUPPORTED_GUEST if that is not possible), auto (reset when possible, otherwise make an exact copy and report a warning), or keep (exact copy). When it is omitted, the host's default applies. GET /api/v1/vms/{id}/clone-capabilities reports whether the VM's host can reset identities. A failure of the reset tool itself (an error or a timeout) is reported with the code IDENTITY_RESET_FAILED (HTTP 500).

---

6. Deletion and Cleanup Options

When deleting a virtual machine, Cockpit exposes options to manage underlying resources:

• Metadata Deletion: Removes the VM registration from Cockpit and the host hypervisor configuration.
• Storage Cleanup: By default, deleting a VM keeps its virtual disks to prevent data loss. Administrators can toggle the remove_disks parameter (executing API DELETE /vms/:id?remove_disks=true) to delete all associated virtual disk volumes from their respective datastores.