Proxmox Virtual Environment
- Bare metal hypervisor
- Discovered it from the post
- Learning Proxmox VE (book)
- Mastering Promox (book)
- Iso image location in proxmox: /var/lib/vz/template/iso
- Proxmox as a home virtualization solution
- Proxmox – a great Hypervisor for running servers
- What are Proxmox advantages vs VMware?
- Comparison of Server Virtualization
- Proxmox VE 6.0 Beginner Tutorial - Installing Proxmox & Creating a virtual machine. Need 3 drives; one for proxmox, one for ISOs and one for VMs.
- Launching a Windows VM in Proxmox 9/28/2021
- $149 Low Cost Virtual Machine Server & installation process
- ProxMox; A VM Server For Your Home
- Proxmox vs Docker for a Home server
- Using a vm give you snapshots, easy backups, live migration and easy swapping between server
- If you have any "security" concerns, there's less chance of a breakout from a VM than from docker. It is also much easier to move between networks / vlans as a VM than a container.
- Install with single SSD? from Reddit
- Before I do anything on Proxmox, I do this first...
- Package Repositories. Comment out the line in /etc/apt/sources.list.d/pve-enterprise.list and add a line deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription to "/etc/apt/sources.list" file.
- How to upgrade Use apt dist-upgrade or the GUI, NOT apt upgrade!
- My NVMe SSD was not detected. It turns out I need to choose ACHI instead of RAID in SATA Operation under System Configuration in BIOS. See AHCI VS RAID: Which One Should I Choose for Better Performance?.
- Proxmox VE 7 Initial Installation Checklist. 2021
- To access it, use https://IP:8006
- Username is 'root'.
- Change to pve-no-subscription Repo. We need to modify package sources before we run apt update and apt dist-upgrade
- My initial setup (disk storage is 240GB). However in web page, the 'Disks' tab will show 240GB for /dev/sda and /dev/sda3 (LVM) shows 93% of usage.
[email protected]:~# df -h Filesystem Size Used Avail Use% Mounted on udev 7.7G 0 7.7G 0% /dev tmpfs 1.6G 1.3M 1.6G 1% /run /dev/mapper/pve-root 55G 3.1G 49G 6% / tmpfs 7.8G 37M 7.7G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda2 511M 328K 511M 1% /boot/efi tmpfs 1.6G 0 1.6G 0% /run/user/0 /dev/fuse 128M 16K 128M 1% /etc/pve
I don't have a separate disk for storing ISOs. I use the local disk. The iso files are saved in /var/lib/vz/template/iso directory. The bridge network works so I can ssh to the new virtual machine (after install openssh-server).
- Proxmox Beginner Tutorial: How to set up your first virtual machine on a secondary hard disk
- You need at least 2 physical drives; one for proxmox and the other for virtual machines.
- Trying out Proxmox VE, an open-source virtualization platform 2021
- How to Unlock a Proxmox VM. Then reboot pve.
- For Windows installation, if we are dropped at the EFI shell, we can use reset -s to shut down the system.
- Windows VirtIO Drivers & Creating Windows virtual machines using virtIO drivers from fedoraproject.org
- (Videos) Launching a Windows VM in Proxmox (Win10), Virtualize Windows 10 with Proxmox VE (Win10)
- In OS tab, be sure to choose the right Guest OS Version; if we choose (10/2016/2019), we will select 2019 in the driver folder locations OR if choose (11/2022), we will select the corresponding driver folders. This affects the later steps when we select the drivers to install.
- There are 3 drivers we shall install.
- vioscsi\win10\amd64\vioscsi.inf (we need it in order to see the virtual disk)
- NetKVM\w10\amd64\netkvm.inf (in order to get network working, this can be installed later from Windows Device Manager)
- Balloon\win10\amd64\balloon.inf (memory balloon driver, this can be installed later)
- Guest agent (Qemu-guest-agent). Make sure we have installed all missing drivers from Device Manager. We can go to virtual machine Summary tab to check if the IPs shows the Windows IP.
- Change network to use a static IP.
- Enable RDP.
- (Related to Audio device). Change Hardware - Display - SPICE (instead of default). W/o doing that, we'll get an error "Can't start vms with audio device(SPICE) in pve 7.3 "
- Might be better to disable audio so we don't need to use SPICE for Windows
- SPICE driver - Proxmox wiki.
- Windows 7 mouse cursor alignment in VNC console when SPICE is enabled
- Spice with Windows 10 - Mouse Does Not Work. Download spice tool does get rid of the 2nd cursor. But the response seems to be getting slower and alignment is not quite correct. So in the end I still disable audio device and switch from SPICE to default.
- Windows 10 guest best practices
- For Audio, we need to add it to the hardware list (driver=spice is OK). PS: audio works when I tested using the Microsoft Remote Desktop app on mac. It does not work when I use Remmina app from Ubuntu initially but if I change the sound setting (from "Off" to "Local" in the "Advanced" tab) it works.
- Windows 11 (10/21/2021). We can still use 10/2016/2019 for the Version selection. When I check the Task Manager (Windows 11 Pro, Version 21H2, OS build 22000.675), it shows 1.8/4GB was used and 101 out of 127GB is free.
- OS: version 10/2016/2019
- System: BIOS: OVMF (UEFI). Machine - q35. Add TPM. Qemu Agent. SCSI: VirtIO SCCI
- HD: Bus-VirtIO Block. Disk size >=32
- CPU: Cores >=2. Type - Host
- Network: VirtIO
- After finish, adding CD/DVD - VirtIO iso
- Install: Browse VirtIO disk. AMD - win10. Next, browse NetKVM - win10.
- After reboot. Continue to answer questions. Reboot
- File manager. VirtIO disk. Run virtio-win-gt-x64.exe.
- Right click on Start. Device Manager. All clean.
- Windows 11 VM / Office 2019 install error 0-2054
- Windows 11 virtual machine on Proxmox 8/19/2022
- AMD/NVIDIA GPU Passthrough in Window 11 - Proxmox Guide 7/22/2022
- Installing macOS 12 “Monterey” on Proxmox 7
- Running a MacOS Monterey VM in ProxMox VE
- Installing macOS 13 Ventura on Proxmox 7.2. It works (tested 2/11/2023).
- I also got an error "macOS Update Assistant”. After I clicked "x" to dismiss it, I can continue the installation.
- MacOS requires at least 4 characters for the password (something like 'asdf' will suffice).
- I followed the instruction up to 'Sleep management'.
- It took about 1 hour.
- The default resolution is 1920x1080. So I need to use noVNC and choose 'Fullscreen' (ESC to return back) from the control bar on LHS.
- On the new Ventura system, the memory usage is 1.7GB. So the default 4GB is still enough.
- How to turn on SSH server on Mac in macOS Ventura 13 or newer.
- The current IP address can be found by ifconfig command or ipconfig getifaddr en0 or System preferences-Network.
- OSX-PROXMOX - Run macOS on ANY Computer - AMD & Intel
Install on Debian
Installing Proxmox VE 7.x on Debian Bullseye for custom partition layout (video)
First time installation of PVE - SSD/HDD choices
My Proxmox Home Server Walk-Through
- By default, Proxmox assigns a standard VGA device for BIOS-based virtual machines and a QXL device for UEFI-based virtual machines.
- For Windows OS, we can use the default (noVNC)
- For Linux OS, SPICE is better. Screen can be scaled as we want. Copy and paste still does not work after I installed spice-vdagent. virt-manager Spice copy paste doesn't work. But the copy-paste function works in Debian VM launched by Virtual Machine Manager (actually the menu bar are different. One is called 'Remote Viewer'/remote-viewer and the other embedded viewer from /usr/bin/qemu-system-x86_64 is called 'QEMU/KVM').
- (Debian11 and antiX VMs) When I use "ps -ef | grep spice", I got /usr/sbin/spice-vdagentd & /usr/bin/spice-vdagent as shown in the screenshot here for the VM launched by QEMU/KVM. But I did not see /usr/bin/spice-vdagent in the VM launched by Proxmox.
- B/C a hint from above, I found a solution here. After I run /usr/bin/spice-vdagent , copy-and-paste works!
- In summary, 1) sudo apt install vdagent 2) /usr/bin/spice-vdagent
- (Fedora 35). Copy-and-paste works out of box (vdagentd & vdagent are automatically running in the background). Maybe it's because Fedora is a Red Hat based Linux OS.
- How to add spice-vdagent to VirtIO VM?
Sound/audio and SPICE
- SPICE (Simple Protocol for Independent Computing Environments) is a communication protocol used for virtual environments. It provides a remote display system, allowing users to view a virtualized desktop on their local machine and interact with it using keyboard and mouse input.
- SPICE is often used in conjunction with virtualization platforms such as QEMU/KVM, and is widely used in enterprise and cloud computing environments.
- SPICE (Simple Protocol for Independent Computing Environments) does not have any direct alternatives as it is a specific communication protocol used for remote display in virtualized environments. However, there are other remote display protocols such as RDP (Remote Desktop Protocol), VNC (Virtual Network Computing), and NX (NoMachine's Remote X protocol) that can be used as alternatives to SPICE in certain situations. Nonetheless, the most suitable protocol for a specific use case depends on various factors such as the nature of the application, the network bandwidth available, the desired level of graphics performance, and more.
- Add sound hardware to VM
- Choose SPICE when launching the VM, it will download a vv file.
- In ubuntu, "remote-viewer" will be used to open the vv file when we double clicked the downloaded file. Proxmox SPICE console apt install virt-viewer
- In macOS, see Proxmox - SPICE client setup for MacOS, or here
Remote Desktop through browser
can't shutdown a VM
Use the command qm unlock XXX
qm stop XXX # can't lock file '/var/lock/qemu-server/lock-996.conf' - got timeout qm unlock XXX qm stop XXX
Now we can go back to proxmox GUI to remove the vm.
But if the "qm unlock" does not work, we can use the kill command. See Proxmox can’t stop VM – How we fix it!
ps aux | grep "/usr/bin/kvm -id VMID" kill -9 PID # VM will stop
Errors and solutions
- Error 0 occurred while receiving the document.: it's a browser problem. Change from Chrome to FF fixed it.
Removing ProxMox Subscription Notice
- Removing ProxMox Subscription Notice.
- How to: Remove “You do not have a valid subscription for this server….” from Proxmox Virtual Environment/Proxmox VE (PVE 6.1 to 7.1 and up) works. I find I need to use Ctrl + F5 to clear cache.
- What is IP Masquerading?
- IP Masquerading, helps machines with non-routable IP addresses to access the Internet via the machine that is masquerading. It is a form of NAT (Network Address Translation).
- What is IP masquerading and when is it of use?
- Beginners Guide to Masquerading and Port Forwarding in CentOS/RHEL 7
- Virtualizing An Internal Network With pfSense In ProxMox (video)
- NAT (default network in Virtualbox)
How many Ethernet ports do I need on my Proxmox?
- LXC vs Docker: Which Container Platform Is Right for You?
- Linux Container and pveam tool
# pveam update # pveam available # pveam available --section system # pveam download local debian-10.0-standard_10.0-1_amd64.tar.gz # pveam list local
- Download http://download.proxmox.com/images/system/
- Debian Appliance Builder, VM Templates and Clones
- Select DHCP in network setting in order to get internet connection (tested on Debian 10 template). ubuntu 20.04 LXC + sshd uses only 37MB memory. It runs 22 processes. Booting is very fast! (cf ubuntu 20.04 server + sshd uses 145MB and runs 99 processes)
- Password needs at least 5 characters.
- To allow ssh log in (it's OK even firewall checked), we need to modify /etc/ssh/sshd_config file (so we can use the root account to log in). See How to SSH to LXC containers?
LoginGraceTime 2m PermitRootLogin yes # prohibit-password StrictModes yes
- This LXC looks very much like a server VM not Docker (we can also install desktop environment in an LXC)
- Convert Docker image to Proxmox lxc
- Proxmox Help Scripts
- Using a Desktop with a LXC Proxmox 7, Is there a GUI for Ubuntu Desktop in a container?
- New users (eg brian)
adduser brian usermod -aG sudo brian mkdir -p /home/brian/.ssh # if we want to copy the ssh key from root to the new user cp ./.ssh/* /home/brian/.ssh chown -R brian:brian /home/brian/.ssh/
Installing Pi-Hole inside a Proxmox LXC Container. 2GB disk, 1 CPU core, and 256MB of memory. The memory usage is pretty flat around 53MB according to Proxmox gui). I am using Debian 11 template.
apt update apt upgrade nano /etc/sysctl.conf # disable IPv6 reboot apt install curl curl -sSL https://install.pi-hole.net | bash pihole -a -p # change to a simpler password
SMART and wearout
I saw the wearout is 99% on my host disk (240GB Kingston SSD). I cannot delete a VM. If I use the command "qm destroy XXX", it shows "Unable to create output file '/var/log/pve/tasks/1/UPID:pvv....:qmdestroy:108:[email protected]:' - Read-only file system". The host disk uses only 37% of storage on root partition. The solution: reboot Proxmox.
- Add additional drive in proxmox | How to?
- How to add storage to Proxmox
- How To Add External USB Storage To Proxmox
- easiest way to mount USB drive Proxmox 7?
- Add a new physical hard drive to Proxmox VE 4x 5x
- Add Storage Drive To Proxmox 7
- Format a disk
Prepare the drive
fdisk /dev/nvme0n1 : p : d : ENTER : p : w : n ENTER ENTER ENTER : p : w : qNow to go GUI, pve -> Disks -> Directory -> Create Dir.
- Storage: LVM Thin
- How to: Delete/Remove local-lvm from Proxmox VE (PVE) (How to Expand/Extend/Grow LVM/Ext3/Ext4/XFS) (and Some LVM basics, commands)
- How to ERASE LVM-THIN and create Legacy LVM
- Need to delete local-lvm and reuse the size
# delete local-lvm storage in gui; pve -> Disks -> LVM-Thin -> More -> Restroy lvdisplay # /dev/pve/swap, /dev/pve/root, /dev/pve/data lvremove /dev/pve/data lvresize -l +100%FREE /dev/pve/root resize2fs /dev/mapper/pve-root vgdisplay pve | grep Free # should be 0 now
- lsblk, df -h and more
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 223.6G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 512M 0 part /boot/efi └─sda3 8:3 0 223.1G 0 part ├─pve-swap 253:0 0 8G 0 lvm [SWAP] └─pve-root 253:1 0 215.1G 0 lvm / nvme0n1 259:0 0 465.8G 0 disk └─nvme0n1p1 259:1 0 465.8G 0 part /mnt/pve/vm1 # df -h Filesystem Size Used Avail Use% Mounted on udev 7.7G 0 7.7G 0% /dev tmpfs 1.6G 1.3M 1.6G 1% /run /dev/mapper/pve-root 214G 30G 176G 15% / tmpfs 7.8G 40M 7.7G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/nvme0n1p1 458G 2.0G 433G 1% /mnt/pve/vm1 /dev/sda2 511M 328K 511M 1% /boot/efi /dev/fuse 128M 16K 128M 1% /etc/pve tmpfs 1.6G 0 1.6G 0% /run/user/0 # fdisk -l Disk /dev/nvme0n1: 465.76 GiB, 500107862016 bytes, 976773168 sectors Disk model: WDC WDS500G2B0C ... Device Start End Sectors Size Type /dev/nvme0n1p1 2048 976773134 976771087 465.8G Linux filesystem Disk /dev/sda: 223.57 GiB, 240057409536 bytes, 468862128 sectors Disk model: KINGSTON SA400S3 ... Device Start End Sectors Size Type /dev/sda1 34 2047 2014 1007K BIOS boot /dev/sda2 2048 1050623 1048576 512M EFI System /dev/sda3 1050624 468862094 467811471 223.1G Linux LVM Disk /dev/mapper/pve-swap: 8 GiB, 8589934592 bytes, 16777216 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/mapper/pve-root: 215.07 GiB, 230925795328 bytes, 451026944 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes # pvdisplay --- Physical volume --- PV Name /dev/sda3 VG Name pve PV Size <223.07 GiB / not usable <3.57 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 57105 Free PE 0 Allocated PE 57105 PV UUID 4taiYE-DDJa-4UdU-v3QY-kd2s-7r5i-shhJ7Q
- Benchmark HD speed
hdparm -t --direct /dev/nvme0n1p1 hdparm -t --direct /dev/sda3
- lvresize vs lvextend. lvextend can only increase the size of a volume, whereas lvresize can increase or reduce it. Increase the size of an LVM logical volume.
- How to Manage and Use LVM (Logical Volume Management) in Ubuntu
Proxmox LXC Intel Quick Sync Transcode for Plex
Clone (full clone vs convert to template)
- Clone a VM in Proxmox – Here’s how to safely duplicate your VM.
- A clone VM works like a regular VM
- Note if a VM is converted to a template, it can't be started (like a regular VM).
Backup and restore VM
- Backup and Restore from proxmox wiki. How to Backup Proxmox? Proxmox Backup and Recovery Methods.
- How to backup and transfer a Proxmox VM to another Proxmox Node
- Backup file has a timestamp on the filename and it is saved to /var/lib/vz/dump if it is saved in local (pve) or /mnt/pve/vm1/dump if it is saved on my vm1 storage.
- The backup file can be seen on the GUI under $STORAGE (pve) -> Backups where $STORAGE is the storage name (e.g. local). From there, it has a 'Restore' button where we can restore it with an option to enter a new VM ID.
- If the backup file is saved in local (pve), the backup file can also be seen under the VM|Backup menu.
- The backup VM has a filename vzdump-qemu-$ID-$Date-$Time.vma.zst (compression by default). If we select the file, we can click the 'Restore' button to restore the VM.
- After restoring, the new VM has a new ID but the VM name is still the same as the original one (so we can only use the ID to distinguish the VMs). Therefore, if we use the static IP in a VM, it is better to shut down the old one before we Start the new VM.
- If we remove/delete the restored VM, the backup file is not affected (not deleted).
- It seems backup + restore = clone.
- If we back up a container, the backup file name has a format vzdump-lxc-$ID-$Date-$Time.tar.zst.
- The backup speed is quick. Using the "top" command it shows it is the command zstd running for the backup.
- Restore Proxmox VM from backup – Here are the steps to recover your VM
- VM ID:
- One biggest problem is I cannot tell what the backup file is from the file name after I back up the files to another location. The backup file however contains VM ID on its filename. That is the only clue we can use to find out what the VM is on the original Proxmox.
- The Backup Notes is useful actually. In the backup folders on Proxmox, it also create *.notes files if we create it on the GUI.
- Maybe it's useful for me to create a text file along with the backup files to show what the files represents.
- Question: why sometimes my backup files are not shown on GUI.
Ans: the default backup storage is "local". We need to toggle that from the GUI. PS: the storage we changed cannot be memorized.
How to mount a Network Share in a Linux Container under Proxmox
Setup a MediaWiki Server
How to Setup a MediaWiki 1.31 Server on a Debian 10 Proxmox container
Multiple node cluster
- How to Passthrough USB Disk to a Virtual Machine in Proxmox 6.2, Adding USB Devices to Proxmox VM (video). This assumes the USB device is on the proxmox host.
- Adding USB Redirection to the Raspberry Pi + Proxmox Thin Client. The USB device is on the client.
- Converting Any USB Device to A Wireless USB, VirtualHere ($49, locked to one device). The server (USB devices are plugged in) can be any where.
Raspberry Pi THIN CLIENT for Proxmox VMs
Android emulator and nested virtualization
Is there a guide to getting Android x86 installed on Proxmox?
Am I compromised? If you need true remote access, set up a VPN that you connect to on your router.
Error 401: no ticket
Empty browser's cache. It works.
add a startup delay to your VMs and CTs if this is the only time you have this issue. Then set up a script in crontab to run @reboot that just waits x seconds, then remounts the share.
Cloud image, Cloud-init
- What are Ubuntu cloud images?
- Using Ubuntu cloud-images without a cloud
- KVM: Testing cloud-init locally using KVM for an Ubuntu cloud image
- Launching Ubuntu Cloud Images with QEMU
Proxmox vs. ESXi
- Comparison of Server Virtualization
Proxmox Backup Server
Remote machine management
Self-Hosted Remote Desktop Connection Alternatives
- Remotely, a free open source Remote Support Alternative to Anydesk GTA LogmeIn TeamViewer and more and notes
- Server + client
- Server is operating on the browser to access the client's computer
- Client (Windows or Linux only, macOS is not supported) needs to install a program
- Remotely Update - an Update on an open source, free alternative to TeamViewer, Anydesk, GoToAssist & notes