Plugin crashes trying to clone a template

[nkamenar]

I have an ubuntu cloudinit template I am trying to create clones of using terraform. I admit I am new to Terraform so it's entirely possible I am making a mistake here and if that is the case I apologize. I have included my terraform file as well as the output from running apply.

Apply output log: terraform_debug.log

I did notice that in the plan, the output memory is set to 512 which is odd because I didn't specify memory in the config so I thought it should just use what is currently set in the template (8192) but maybe I am not understanding how this works? Similarly the plan has bios set as seabios but the template is set up with OVMF (UEFI). Basically I have a cloudinit template already configured with all my settings including user info, SSH tokens, hardware configuration, etc. And I just want to create multiple copies of that template, as is, with unique known IP Addresses. Thanks for any assistance!

terraform {
  required_providers {
    proxmox = {
      source  = "Telmate/proxmox"
      version = "2.9.14"
    }
  }
}

provider "proxmox" {
  pm_api_url      = "https://pve1.mydomain.com:8006/api2/json"
  pm_user         = "root@pam"
  pm_password     = var.proxmox_password
  pm_tls_insecure = false
}

variable "proxmox_password" {}

variable "vm_count" {
  default = 3
}

resource "proxmox_vm_qemu" "docker_nodes" {
  count      = var.vm_count
  name       = "dkr-0${count.index + 4}"
  target_node = "pve1"
  clone      = "ubuntu-2404-cloudinit-template"
  os_type   = "cloud-init"
  # CloudInit settings to assign static IPs
  ipconfig0 = "ip=192.168.1.${count.index + 14}/24,gw=192.168.1.1"
}

output "docker_vm_ips" {
  value = [for vm in proxmox_vm_qemu.docker_nodes : vm.ipconfig0]
}

Proxmox Template Hardware Config:

Template Cloud-Init settings:

[TheGameProfi]

Hey there,
the terraform provider versions 2.x.x are not functional with the newest Proxmox versions.
You would either need to switch to a 3.x release, switch provider or use a working fork.

[Tinyblargon]

@nkamenar the thing you are trying to achieve isn't possible with the Terraform provider. The provider is built with the concept that the Terraform config is the truth of the environment. Therefore, any changes made outside of Terraform will be seen as drift from its state of truth.

[nkamenar]

@TheGameProfi thanks for the info, that got me at least somewhat up and running.
@Tinyblargon Thanks for letting me know that know that the configuration in proxmox doesn't apply to what is created by Terraform. I have updated my configuration as shown below and it is creating VMs but it only creates 1 or 2 and then I get an error for the remainders attached.

I think maybe I need to add some sort of delay between spinning up the VM's since it's all using the same template? Also for the cipassword variable should that be a plain text password in my .tfvars file or does that need to be encrypted like $(openssl passwd -6 'super_secret_pass')? Thanks again for the assistance.

terraform {
  required_providers {
    proxmox = {
      source  = "Telmate/proxmox"
      version = "3.0.1-rc4"
    }
  }
}

provider "proxmox" {
  pm_api_url      = "https://pve1.mydomain.com:8006/api2/json"
  pm_user         = "root@pam"
  pm_password     = var.proxmox_password
  pm_tls_insecure = false
}

variable "proxmox_password" {}

variable "cipassword" {
  type = string
  sensitive = true
}

variable "vm_count" {
  default = 3
}

resource "proxmox_vm_qemu" "docker_nodes" {
  count      = var.vm_count
  name       = "dkr-0${count.index + 4}"
  target_node = "pve1"
  clone      = "ubuntu-2404-cloudinit-template"

  # Hardware config
  memory     = 8192
  cores      = 4
  sockets    = 1
  cpu        = "host"
  scsihw     = "virtio-scsi-pci"
  bios       = "ovmf"
  machine    = "q35"

  network {
    model  = "virtio"
    bridge = "vmbr0"
  }

  ipconfig0 = "ip=192.168.1.${count.index + 14}/24,gw=192.168.1.1"

  ciuser = "nate"
  cipassword = var.cipassword
  sshkeys = file("~/.ssh/authorized_keys")

  cicustom = "user=local:snippets/vendor.yaml"
  os_type   = "cloud-init"
}

output "docker_vm_ips" {
  value = [for vm in proxmox_vm_qemu.docker_nodes : vm.ipconfig0]
}

[Tinyblargon]

Yeah, there is a threading issue causing it to request all IDs at the same time and getting the same ID.

As workaround, you can manually set the ID and auto increment it with count

[nkamenar]

Yeah, there is a threading issue causing it to request all IDs at the same time and getting the same ID.

As workaround, you can manually set the ID and auto increment it with count

Sorry to be a pain but could you give an example of how I might do this? This is literally the first time I am trying to use Terraform to set up some VM's so I'm super new to all of this.

[Tinyblargon]

@nkamenar If I remember correctly this is how you do it. When you statically assign the vmid we won't ask PVE for the next free id. Do keep in mind that it can cause issues if the vmid is already in use by something else, so use a range you know is available.

resource "proxmox_vm_qemu" "docker_nodes" {
    vmid: ${1234 + count.index}
}

[nkamenar]

Ok thanks that helped. I have the VM mostly working and it spins up like it should now however it seems like the cloud-init portion isn't doing what it's supposed to. When I look in the cloud-init section of the cloned VM everything looks right:

But then when the machine boots I am getting an ubuntu login prompt like this:

But I would expect something more like this (which is from a machine I cloned directly in the proxmox UI):

When I try to ssh into the machine it won't connect because it seems like cloud-init didn't set up my user and ssh keys or anything. Am I missing something? Here is my latest config:

terraform {
  required_providers {
    proxmox = {
      source  = "Telmate/proxmox"
      version = "3.0.1-rc4"
    }
  }
}

provider "proxmox" {
  pm_api_url      = "https://pve1.mydomain.com:8006/api2/json"
  pm_user         = "root@pam"
  pm_password     = var.proxmox_password
  pm_tls_insecure = false
}

variable "proxmox_password" {}

variable "cipassword" {
  type = string
  sensitive = true
}

variable "vm_count" {
  default = 3
}

variable "starting_id" {
  default = 1000
}

resource "proxmox_vm_qemu" "docker_nodes" {
  count       = var.vm_count
  name        = "dkr-0${count.index + 4}"
  target_node = "pve1"
  clone       = "ubuntu-2404-cloudinit-template"
  vmid        = "${var.starting_id + count.index}"

  agent = 1
  
  # Hardware config
  memory     = 8192
  cores      = 4
  sockets    = 1
  cpu        = "host"
  scsihw     = "virtio-scsi-pci"
  bios       = "ovmf"
  machine    = "q35"
  bootdisk   = "scsi0"

  serial {
    id   = 0
    type = "socket"
  }

  vga {
    type   = "serial0"
  }

  disks {
    scsi {
      scsi0 {
        disk {
          size = 50
          storage = "local-lvm"
        }
      }
    }
    ide {
      ide2 {
        cloudinit {
          storage = "local-lvm"
        }
      }
    }
  }

  network {
    model  = "virtio"
    bridge = "vmbr0"
  }

  ipconfig0 = "ip=192.168.1.${count.index + 14}/24,gw=192.168.1.1"

  ciuser = "nate"
  cipassword = var.cipassword
  sshkeys = file("~/.ssh/authorized_keys")
  cicustom = "user=local:snippets/vendor.yaml"
  ciupgrade = true
  os_type   = "cloud-init"
}

output "docker_vm_ips" {
  value = [for vm in proxmox_vm_qemu.docker_nodes : vm.ipconfig0]
}

[Tinyblargon]

Never used cloud-init with Ubuntu, but for Debian, I use a `genericcloud' image https://cloud.debian.org/images/cloud/bookworm/latest/ maybe there is something like that for Ubuntu aswell.

Was this vm template created from an image provided by Ubuntu or just a normal Ubuntu installation? In the past, I've seen people post that it should work with a normal installation as long as cloud-init is installed, never verified this.

[nkamenar]

Mine was created similarly. This is the set of commands I used to create the template:

sudo wget -q https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

Configure image:
	sudo apt-get install libguestfs-tools
	export EDITOR=nano

	virt-edit -a ./noble-server-cloudimg-amd64.img /etc/cloud/cloud.cfg
		In cloud_config_modules section:
			change - timezone to - timezone "America/New_York"
		Add to bottom of file:
			bootcmd:
			  - TZ="America/New_York" date >> /etc/birth_cirtificate
		save/quit
sudo qemu-img resize noble-server-cloudimg-amd64.img 50G

sudo qm create 900 --name "ubuntu-2404-cloudinit-template" --ostype l26 \
    --memory 8192 \
    --agent 1 \
    --bios ovmf --machine q35 --efidisk0 local-lvm:0,pre-enrolled-keys=0 \
    --cpu host --socket 1 --cores 4 \
    --vga serial0 --serial0 socket  \
    --net0 virtio,bridge=vmbr0

sudo qm importdisk 900 noble-server-cloudimg-amd64.img local-lvm

sudo qm set 900 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-900-disk-1,discard=on

sudo qm set 900 --boot order=scsi0

sudo qm set 900 --ide2 local-lvm:cloudinit

cat << EOF | sudo tee /var/lib/vz/snippets/vendor.yaml
#cloud-config
runcmd:
    - apt update
    - apt install -y qemu-guest-agent
    - systemctl start qemu-guest-agent
    - reboot
# Taken from https://forum.proxmox.com/threads/combining-custom-cloud-init-with-auto-generated.59008/page-3#post-428772
EOF

sudo qm set 900 --cicustom "vendor=local:snippets/vendor.yaml"

sudo qm set 900 --tags ubuntu-template,24.04,cloudinit

sudo qm set 900 --ciuser nate

sudo qm set 900 --cipassword $(openssl passwd -6 'secret_pass')

sudo qm set 900 --sshkeys ~/.ssh/authorized_keys

sudo qm set 900 --ipconfig0 ip=dhcp

sudo qm template 900

[Tinyblargon]

Just noticed your cloud-init disk is mounted as ide. I've seen some linux distributions give issues with that in the past. Could you change it to a scsi?

[Tinyblargon]

Could be the same issue as #973

[nkamenar]

Just noticed your cloud-init disk is mounted as ide. I've seen some linux distributions give issues with that in the past. Could you change it to a scsi?

The drive was mounted as ide when I cloned some VM's through the proxmox UI and everything worked fine so I wouldn't think that's it but regardless I tried changing my drives section to be like below and still got the same results. Let me know if this isn't what you meant. I also tried changing it to specifically ide3 from ide2 because I noticed the example in the docs seem to use ide3 so I thought maybe there was something special about that slot, but still no change.

disks {
    scsi {
      scsi0 {
        disk {
          size = 50
          storage = "local-lvm"
        }
      }
      scsi1 {
        cloudinit {
          storage = "local-lvm"
        }
      }
    }
  }