The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,531 other followers

Archive for the ‘Proxmox’ Category

Proxmox with Windows VMs and backups: why you want qemu-agent and how to install it

Posted by jpluimers on 2018/01/26

When running Windows VMs on Proxmox and you want to make snapshot backups you really want to run the qemu-agent inside the Windows VMs.

First of all you really want snapshot mode backups as of all backup modes they have the least downtime. By default they have a small inconsistency risk, but on Windows that can be alleviated by running qemu-agent as described in [WayBackBackup and Restore – Proxmox VE: Backup Modes

More on snapshot backup mode

Backup modes for VMs:

snapshot mode
This mode provides the lowest operation downtime, at the cost of a small inconstancy risk. It works by performing a Proxmox VE live backup, in which data blocks are copied while the VM is running. If the guest agent is enabled (agent: 1) and running, it calls guest-fsfreeze-freeze and guest-fsfreeze-thaw to improve consistency.

A technical overview of the Proxmox VE live backup for QemuServer can be found online here (https://git.proxmox.com/?p=pve-qemu-kvm.git;a=blob;f=backup.txt -> https://git.proxmox.com/?p=pve-qemu-kvm.git;a=blob;f=vma_spec.txt;).

Proxmox VE live backup provides snapshot-like semantics on any storage type. It does not require that the underlying storage supports snapshots.

On Windows the trick is that qemu-agent can use VSS to get a frozen state of the filesystem as described in [WayBackQemu-guest-agent – Proxmox VE:

In Proxmox VE, the qemu-guest-agent is used for mainly two things:

  1. To properly shutdown the guest, instead of relying on ACPI commands or windows policies
  2. To freeze the guest file system when making a backup (on windows, use the volume shadow copy service VSS).

So: installing qemu-agent.

Start with the VM options

Start with the VM options

Don’t make the mistake to start at [WayBackQemu-guest-agent – Proxmox VE: Installation; guest; Windows as it will give you a hard time. Always use the full [WayBackQemu-guest-agent Installation steps beginning at the[WayBackHost; these steps worked for me:

  1. Download a recent set of [WayBackWindows VirtIO Drivers – Proxmox VE
  2. Ensure the ISO image is in /var/lib/vz/template/iso on the Proxmox host so they show up as local:iso for mounting.
  3. Shutdown the Windows VM
  4. [WayBackQemu-guest-agent – Proxmox VE: Host; enable guest for VM: on the “Options” page for a VM, ensure the “Qemu Agent” setting is set to “yes”  
    • This will add a PCI serial device to your computer that Windows – after a fresh boot – sees as “PCI Simple Communications Controller”
  5. Mount the ISO image to a CD/DVD drive 
  6. Boot the Windows VM
  7. Start Device Manager (devmgmt.msc)
  8. In the Device Manager, observe a new device “PCI Simple Communications Controller” that doesn’t have drivers installed yet 
  9. Right click the”PCI Simple Communications Controller” device and select “Update Driver Software…” 
  10. Indicate you want to browser for the files (as opposed of Windows finding them on-line: they’re not on-line): 
  11. On the CD/DVD drive letter you mounted the ISO image to, select D:\vioserial\[OS-Version]\ where OS-Version is your Windows Version and ensure “Include subfolder” has a checkmark so it will find the Win32 or Win64 drivers depending on your processor architecture.
    • I used D:\vioserial\w7  
  12. Finish the driver installation 
  13. Observe it now has a driver installed
     
  14. From D:\guest-agent install either qemu-ga-x64.msi for Win64 or qemu-ga-x86.msi for Win32.
  15. During installation, confirm the UAC prompt for sofware by Red Hat, Inc.
  16. Start the Service Manager (services.msc) and execute from any command promt the following statments to verify the check if these services are running: 
  17. Now reboot the VM, logon and start the Service Manager again; now it should look like this 

Note you can obtain the same information from the console by executing these commands.

  • sc queryex "QEMU-GA"
  • sc queryex "QEMU Guest Agent VSS Provider"

Before the reboot, they should show output like this:

C:\Windows\system32>sc queryex "QEMU-GA"

SERVICE_NAME: QEMU-GA
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0
        PID                : 4444
        FLAGS              :

C:\Windows\system32>sc queryex "QEMU Guest Agent VSS Provider"

SERVICE_NAME: QEMU Guest Agent VSS Provider
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0
        PID                : 0
        FLAGS              :

After reboot it should have become this:

C:\Windows\system32>sc queryex "QEMU-GA"

SERVICE_NAME: QEMU-GA
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0
        PID                : 1680
        FLAGS              :

C:\Windows\system32>sc queryex "QEMU Guest Agent VSS Provider"

SERVICE_NAME: QEMU Guest Agent VSS Provider
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0
        PID                : 1624
        FLAGS              :

Verification

Verifying the qemu-agent is reachable from the Proxmox host:

  1. Note the VM ID of the VM as #
  2. Reboot the VM (for instance by typing this on the command prompt: shutdown -r -t 0)
    • I’m not sure this step is needed under all circumstances; if the below steps don’t work then you definitely need it.
  3. Verify the Proxmox host can communicate with the qemu-agent on the VM:
    On the Proxmox host, start a terminal session, then type these commands where # is the ID of the VM (I’ve used 112 as an example here):

# socat /var/run/qemu-server/112.qga -
{"execute":"guest-sync", "arguments":{"id":1234}}

It should get you this result:

{"return": 1234}

Note there’s all sorts of nice stuff you can do with socat /var/run/qemu-server so maybe I’ll put more about it in a future blog post.

–jeroen

Some more links with background information:

Easiest is to download the ISO image to /var/lib/vz/template/iso on the Proxmox host so they show up as local:iso for mounting.

Easiest is to download the ISO image to /var/lib/vz/template/iso on the Proxmox host so they show up as local:iso for mounting.

Posted in Power User, Proxmox, Virtualization, Windows | Leave a Comment »

Provisioning Windows 7 test VMs with know users/passwords

Posted by jpluimers on 2018/01/16

The proxmox side

In Proxmox, ensure you have a named backup of your machine that starts with vzdump-qemu like this:

vzdump-qemu-Win7Sp1UK.vma.lzo

That way, Proxmox knows that it can restore from it.

Don’t forget to assign a new MAC address to the network adapter so it’s unique on the network.

The Windows side

I wanted to provision this with two test accounts: one regular and one with administrator access.

The latter needs to be added to the Administrators group using [WayBacknet localgroup.

Both need passwords that (for now) never expire. This is where [WayBacknet user add fails: even if you set the correct flag, it won’t be reflected, so you need WMIC UserAccount for that.

These two posts helped me a lot with the below batch file fragment:

After restoring, run a batch file like this with an UAC token:

  call :addUser regularTestUser regularTestPassword
  net localgroup "Remote Desktop Users" "regularTestUser" /add
  call :addUser administratorTestUser administratorTestPassword
  :: https://superuser.com/questions/515175/create-admin-user-from-command-line
  net localgroup administrators administratorTestUser /add
  goto :eof
:addUser
  :: https://superuser.com/questions/515175/create-admin-user-from-command-line
  net user /expires:never /add %1 %2 /expires:never
  :: https://serverfault.com/questions/710964/accounts-suddenly-expiring-when-created-with-net-user-add-expiresnever
  WMIC UserAccount where "Name='%1'" set PasswordExpires=FALSE
  goto :eof

The Remote Desktop Users tip is from [WayBackEnable remote desktop from command line (CMD) but that post has “beautified” double quotes in them, so net localgroup by default complains it cannot find the group. The code above should have regular quotes.

Finally the computer needs a new name. Again WMIC to the rescue here as Windows 7 only comes with PowerShell 2.0 which cannot rename a computer.

Again with a UAC token, execute something like this:

WMIC ComputerSystem where Name="%COMPUTERNAME%" call Rename Name=INNOSETUPTEST
%windir%\System32\shutdown.exe -r

This last tip was via [WayBackwindows 7 – Renaming computers via command prompt – Super User.

–jeroen

Posted in Power User, Proxmox, Virtualization, Windows, Windows 7 | Leave a Comment »

Proxmox – recovering a Windows 7 machine having “Missing operating system”

Posted by jpluimers on 2018/01/12

This is not what you like when you reboot a VM in Proxmox:

Booting from Hard disk...
Missing operating system

Booting from Hard disk... Missing operating system

Booting from Hard disk… Missing operating system

This case was a Windows 7 UK Professional x64 SP1 virtual machine.

Luckily the ISO is at https://archive.org/download/en_windows_7_professional_with_sp1_x64_dvd_u_676939_201606/en_windows_7_professional_with_sp1_x64_dvd_u_676939.iso via https://archive.org/details/en_windows_7_professional_with_sp1_x64_dvd_u_676939_201606 (later I found out I had the image in my backup vault as well).

I put that one in /var/lib/vz/template/iso so proxmox will automagically provide it in the local storage of iso images.

Now for some screenshots some based on what I learned at [Archive.isHow to use System Recovery Options for repairing Windows Vista or 7 installations:

Read the rest of this entry »

Posted in Power User, Proxmox, Virtualization, Windows, Windows 7 | Leave a Comment »

Some proxmox notes: adding a new logical volume from a new HDD

Posted by jpluimers on 2017/09/01

–jeroen

Posted in Power User, Proxmox, Virtualization | Leave a Comment »

Some links on converting non KVM VMs to Proxmox

Posted by jpluimers on 2016/10/24

Gut feeling indicates I need these someday:

From VHD to Proxmox you need to convert to RAW not IMG:

# qemu-img convert -f vpc -O raw PATH/to/DISK.vhd PATH/to/DISK.raw

–jeroen

Posted in Power User, Proxmox, Virtualization | Leave a Comment »

 
%d bloggers like this: