Several months ago I used Packer to simplify the creation of VM templates on my organization's vSphere instance. While getting Packer to work was not the most pleasant, the benefits of using it had begun to show and so I decided to begin using Packer in my homelab. In my homelab I use Proxmox as my hypervisor solution, so I figured there would be a few differences but nothing that would cost me too many cycles. I was wrong.
Issue 1: VM Creation Failure
The first issue I encountered was the failure of the VM creation process. Initially, I attempted to authenticate using an API token created by my admin user, but Packer provided no useful feedback in the terminal. After enabling Packer’s debugging mode, I discovered a 501 Not Implemented error being returned from my Proxmox instance.
This error led me to explore the Proxmox API reference documentation. Initially, I suspected that my Proxmox version (v8.0.3) was not compatible with the latest Packer plugin for Proxmox. However, the real issue was with the token permissions, not the API endpoint.
When creating the API token, I had left the Privilege Separation box checked, which required me to manually assign permissions for every resource the token needed to access. I tried adding permissions for each required resource, but I couldn’t find all of them in the UI. Eventually, I created a new token, ensuring that I did not select the Privilege Separation box. This new token inherited all permissions from my admin user, allowing me to create the VM and its resources without issues.
Although diagnosing this issue was challenging due to the misleading 501 error, it could have been avoided if a more appropriate error, such as 401 Unauthorized, had been returned. While this issue originates from the Proxmox API response, an additional warning message in the Packer plugin could help others avoid wasting time on similar problems.
Issue 2: Subiquity Install freezing
Ubuntu uses Subiquity as a framework to drive the installation of the OS, it make it very easy to automate installation of both Ubuntu Desktop and Server. During the Subiquity install I noticed the installer started to hang once additional packages were being installed. This was a pretty easy solve as all I needed to do was add more RAM to my configuration. I modified my configuration to use about 4GB of RAM and the installation started without a hitch!
Issue 3: Packer SSH Connection Failure
After the initial installation was complete Packer was not able to connect to the VM over SSH even though it had the correct credentials (I test the credentials by SSHing in manually while the VM was running). After debugging for a few hours I came across an article that outlines how Packer gets the VM's IP. It relies on the QEMU guest agent to be running on the machine, without it the IP of the machine was not being communicated back to Packer so the SSH connection kept timing out.
All I had to do was add the qemu-guest-agent package to the Subiquity installer so that the service would start and report the IP to Packer for the IP connection.
Issue 4: VM Template Clone Failures
Once Packer was able to create the VM template correctly, I had one last issue when attempting to use it. During the installation a temporary disk was created to expose the files used Subiquity installer. During the teardown by Packer that temporary image is destroyed, but the VM does not know that when it reboots that image will not longer be available so all VMs created from that template would require that drive to be manually removed before boot. To solve this I set the unmount key to true in the image declaration:
Now before Packer finishes the teardown process it makes sure the temporary image is unmounted from the VM. Without this temporary image being permanently mounted in their template all VMs created from it wold no longer throw an error when booting the OS.
Issue 5: Unclear Documentation
This is more a list of gripes rather than a specific issue, but I really do not understand the design behind the Packer documentation (or really Hashicorp's in general). A couple of the issues that stand out to me are:
- Launch point is good, reference is terrible: The base documentation page is actually pretty good, it explains what Packer does and lays out the terminology used in the remainder of the documentation. The tutorials are pretty well defined and provide quite a bit of detail while not completely overloading you along the way. The part I do not like about the documentation is the lack on contrast between section in the "On this page" pane. For example, look at the following screenshot:
The "Required" and "Optional" sections should be nested under the "Configuration Reference" parent section. This would help break things but in that side pane visually, also when with option is selected the remaining instances of that selection are highlighted in that pane. So selecting "Required" highlights the other four instances of "Required" in that pane. I am not sure why you would want that functionality at all.
- Example inconsistency: Inside of the docs the examples used very often use the JSON format over HCL2. Considering the preferred method by Hashicorp is for everyone to use HCL2 I think it would be best if examples are shown in both languages (considering they are both supported) with HCL2 being noted as the primary method to use. I will say this is not really on Hashicorp alone, as community based plugins (like the one I am using) probably do not have to adhere to the same standards BUT considering this documentation is hosted on the official Packer site I think they can share in some of the fault.
TL;DR
Packer is a powerful tool with significant benefits, but using it on new platforms can present challenges. I encountered issues with VM creation, Subiquity installation, SSH connections, and VM template cloning. Additionally, unclear documentation added to the complexity. However, each problem had a solution, and with some persistence, Packer can be effectively used across different environments.
SOCIAL SHARE CARD GENERATOR