Self hosting: Difference between revisions

From 太極
Jump to navigation Jump to search
(38 intermediate revisions by the same user not shown)
Line 32: Line 32:
== Set up ==
== Set up ==
<ul>
<ul>
<li>FQDN
* [https://pmg.proxmox.com/wiki/index.php/Change_FQDN Change FQDN] '''pmgchanged.proxmox.com'''
* [https://easyengine.io/tutorials/mail/fqdn-reverse-dns-ptr-mx-record-checks/ Checking Hostname (FQDN)]: '''hostname -f'''
* [https://forum.proxmox.com/threads/hostname-fqdn-huh.63667/ Hostname (FQDN).......huh?]
* [https://www.techrepublic.com/article/how-to-install-the-proxmox-virtual-environment-virtualization-management-solution/ How to install the Proxmox Virtual Environment virtualization management solution] '''pve.example.invalid'''
<li>My NVMe SSD was not detected. It turns out I need to choose '''ACHI''' instead of '''RAID''' in SATA Operation under System Configuration in BIOS. See [https://www.partitionwizard.com/partitionmanager/ahci-vs-raid.html AHCI VS RAID: Which One Should I Choose for Better Performance?]. </li>
<li>My NVMe SSD was not detected. It turns out I need to choose '''ACHI''' instead of '''RAID''' in SATA Operation under System Configuration in BIOS. See [https://www.partitionwizard.com/partitionmanager/ahci-vs-raid.html AHCI VS RAID: Which One Should I Choose for Better Performance?]. </li>
<li>[https://www.servethehome.com/proxmox-ve-7-initial-installation-checklist/ Proxmox VE 7 Initial Installation Checklist]. 2021
<li>[https://www.servethehome.com/proxmox-ve-7-initial-installation-checklist/ Proxmox VE 7 Initial Installation Checklist]. 2021
Line 71: Line 76:
** '''Balloon\win10\amd64\balloon.inf''' (memory balloon driver, this can be installed later)
** '''Balloon\win10\amd64\balloon.inf''' (memory balloon driver, this can be installed later)
** [https://youtu.be/6c-6xBkD2J4?t=554 Guest agent] ([https://pve.proxmox.com/wiki/Qemu-guest-agent Qemu-guest-agent]). Make sure we have installed all missing drivers from Device Manager. We can go to virtual machine Summary tab to check if the '''IPs''' shows the Windows IP.
** [https://youtu.be/6c-6xBkD2J4?t=554 Guest agent] ([https://pve.proxmox.com/wiki/Qemu-guest-agent Qemu-guest-agent]). Make sure we have installed all missing drivers from Device Manager. We can go to virtual machine Summary tab to check if the '''IPs''' shows the Windows IP.
* Balloon driver
** If balloon driver is not installed on Windows Virtual Machines then memory metrics will not be collected for windows virtual machine. [https://pitstop.manageengine.com/portal/en/kb/articles/steps-to-install-balloon-driver-on-windows-virtual-machines Steps to Install balloon driver on Windows Virtual Machines]
** A '''balloon driver''' is a component of the VMware Tools package of drivers and utilities that help virtual machines (VMs) run better. The purpose of the balloon driver is to take physical RAM from a VM and release it back to the VMkernel. This reclaim usually happens when the ESXi server is short on RAM and this specific VM is the loser in the competition for physical RAM. [https://www.techtarget.com/searchvmware/tip/How-the-balloon-driver-helps-VM-performance How the balloon driver helps VM performance]
* Change network to use a static IP.  
* Change network to use a static IP.  
* Enable [https://youtu.be/6c-6xBkD2J4?t=624 RDP].  
* Enable [https://youtu.be/6c-6xBkD2J4?t=624 RDP].  
* (Related to Audio device). Change Hardware - Display - '''SPICE''' (instead of default). W/o doing that, we'll get an error "Can't start vms with audio device(SPICE) in pve 7.3 "
** Might be better to disable audio so we don't need to use SPICE for Windows
** [https://pve.proxmox.com/wiki/SPICE#Windows_7.2F8.2F10_and_Windows_2012.2F2016 SPICE driver] - Proxmox wiki.
** [https://forum.proxmox.com/threads/windows-7-mouse-cursor-alignment-in-vnc-console-when-spice-is-enabled.16252/ Windows 7 mouse cursor alignment in VNC console when SPICE is enabled]
** [https://forum.proxmox.com/threads/spice-with-windows-10-mouse-does-not-work.69515/ Spice with Windows 10 - Mouse Does Not Work]. Download spice tool does get rid of the 2nd cursor. But the response seems to be getting slower and alignment is not quite correct. So in the end I still disable audio device and switch from SPICE to default.
** [https://pve.proxmox.com/wiki/Windows_10_guest_best_practices Windows 10 guest best practices]
* For Audio, we need to add it to the hardware list (driver=spice is OK). PS: audio works when I tested using the '''Microsoft Remote Desktop''' app on mac. It does not work when I use '''Remmina''' app from Ubuntu initially but if I change the [https://www.digitalcitizen.life/connecting-windows-remote-desktop-ubuntu/ sound setting] (from "Off" to "Local" in the "Advanced" tab) it works.
* For Audio, we need to add it to the hardware list (driver=spice is OK). PS: audio works when I tested using the '''Microsoft Remote Desktop''' app on mac. It does not work when I use '''Remmina''' app from Ubuntu initially but if I change the [https://www.digitalcitizen.life/connecting-windows-remote-desktop-ubuntu/ sound setting] (from "Off" to "Local" in the "Advanced" tab) it works.
* [https://youtu.be/fupuTkkKPDU?t=233 Windows 11]. We can still use 10/2016/2019 for the Version selection. When I check the Task Manager (Windows 11 Pro, Version 21H2, OS build 22000.675), it shows 1.8/4GB was used and 101 out of 127GB is free.
* [https://youtu.be/fupuTkkKPDU?t=233 Windows 11] (10/21/2021). We can still use 10/2016/2019 for the Version selection. When I check the Task Manager (Windows 11 Pro, Version 21H2, OS build 22000.675), it shows 1.8/4GB was used and 101 out of 127GB is free.
** OS: version 10/2016/2019
** OS: version 10/2016/2019
** System: BIOS: OVMF (UEFI). Machine - q35. Add TPM. Qemu Agent. SCSI: VirtIO SCCI
** System: BIOS: OVMF (UEFI). Machine - q35. Add TPM. Qemu Agent. SCSI: VirtIO SCCI
Line 87: Line 101:
** Right click on Start. Device Manager. All clean.
** Right click on Start. Device Manager. All clean.
* [https://www.reddit.com/r/Proxmox/comments/zp45md/comment/j0qnuio/?utm_source=share&utm_medium=web2x&context=3  Windows 11 VM / Office 2019 install error 0-2054]
* [https://www.reddit.com/r/Proxmox/comments/zp45md/comment/j0qnuio/?utm_source=share&utm_medium=web2x&context=3  Windows 11 VM / Office 2019 install error 0-2054]
* [https://youtu.be/z42rCKTuwzE Windows 11 virtual machine on Proxmox] 8/19/2022
* [https://youtu.be/S6jQx4AJlFw AMD/NVIDIA GPU Passthrough in Window 11 - Proxmox Guide] 7/22/2022


=== Mac VM ===
=== Mac VM ===
* [https://www.nicksherlock.com/2021/10/installing-macos-12-monterey-on-proxmox-7/ Installing macOS 12 “Monterey” on Proxmox 7]
* [https://www.nicksherlock.com/2021/10/installing-macos-12-monterey-on-proxmox-7/ Installing macOS 12 “Monterey” on Proxmox 7]
* [https://youtu.be/KLNLRhcQPYU Running a MacOS Monterey VM in ProxMox VE]
* [https://youtu.be/KLNLRhcQPYU Running a MacOS Monterey VM in ProxMox VE]
* [https://www.nicksherlock.com/2022/10/installing-macos-13-ventura-on-proxmox/ Installing macOS 13 Ventura on Proxmox 7.2]. It works.  
* [https://www.nicksherlock.com/2022/10/installing-macos-13-ventura-on-proxmox/ Installing macOS 13 Ventura on Proxmox 7.2]. It works (tested 2/11/2023).  
** I also got an error "macOS Update Assistant”. After I clicked "x" to dismiss it, I can continue the installation.
** I also got an error "macOS Update Assistant”. After I clicked "x" to dismiss it, I can continue the installation.
** MacOS requires at least 4 characters for the password (something like 'asdf' will suffice).
** MacOS requires at least 4 characters for the password (something like 'asdf' will suffice).
Line 98: Line 114:
** The default resolution is 1920x1080. So I need to use noVNC and choose 'Fullscreen' (ESC to return back) from the control bar on LHS.
** The default resolution is 1920x1080. So I need to use noVNC and choose 'Fullscreen' (ESC to return back) from the control bar on LHS.
** On the new Ventura system, the memory usage is 1.7GB. So the default 4GB is still enough.
** On the new Ventura system, the memory usage is 1.7GB. So the default 4GB is still enough.
** [https://iboysoft.com/wiki/ssh-on-mac.html How to turn on SSH server on Mac in macOS Ventura 13 or newer].
** The current IP address can be found by '''ifconfig''' command or ''' ipconfig getifaddr en0''' or System preferences-Network.
* [https://github.com/luchina-gabriel/OSX-PROXMOX OSX-PROXMOX - Run macOS on ANY Computer - AMD & Intel]


=== Install on Debian ===
=== Install on Debian ===
Line 106: Line 125:


=== SSD/HDD choices ===
=== SSD/HDD choices ===
[https://www.reddit.com/r/Proxmox/comments/zku3b5/first_time_installation_of_pve_ssdhdd_choices/ First time installation of PVE - SSD/HDD choices]
* [https://www.reddit.com/r/Proxmox/comments/zku3b5/first_time_installation_of_pve_ssdhdd_choices/ First time installation of PVE - SSD/HDD choices]
* Try these commands to avoid heavy wear on SSD's (only if not clustered)  [https://www.reddit.com/r/Proxmox/comments/13b5wgs/comment/jj9r94e/?utm_source=share&utm_medium=web2x&context=3 Do I need a SSD reserved for Proxmox?]


=== Home Server ===
=== Home Server ===
Line 112: Line 132:
* [https://youtu.be/_sfddZHhOj4 Part 1 (TrueNAS, Portainer, Portainer, Wireguard)],  
* [https://youtu.be/_sfddZHhOj4 Part 1 (TrueNAS, Portainer, Portainer, Wireguard)],  
* [https://youtu.be/BoMlfk397h0 Part 2: Remote Gaming and Streaming]
* [https://youtu.be/BoMlfk397h0 Part 2: Remote Gaming and Streaming]
=== ZFS, NFS ===
* ZFS (Zettabyte File System): A file system developed by Sun Microsystems for use in their Solaris operating system. It is now available on many other operating systems.
* Yes, ZFS can be used without LVM. Even on a workstation, you could use ZFS to pool your disks into a single large pool of storage rather than keep them separate or rely on LVM. [https://www.howtogeek.com/272220/how-to-install-and-use-zfs-on-ubuntu-and-why-youd-want-to/ How to Install and Use ZFS on Ubuntu (and Why You’d Want To)]
* ZFS vs RAID-0
** ZFS is not like RAID-0. RAID-0 is a type of RAID that stripes data across multiple disks without any redundancy. If one disk fails, all data is lost. ZFS, on the other hand, provides '''data redundancy''' and '''checksumming''' to avoid silent data corruption.
* ZFS cons
** [https://blog.servermania.com/xfs-vs-zfs-linux-raid/ XFS vs ZFS vs Linux Raid]. ZFS is a very CPU-intensive filesystem. This can lead to slower performance on systems with limited CPU resources.
* [https://pve.proxmox.com/wiki/Installation Proxmox installation].
** The default file system is ext4. ZFS is an alternative to ext4.
** The Proxmox VE installer, which partitions the local disk(s) with ext4, XFS, BTRFS (technology preview), or ZFS and installs the operating system.
** The main advantage of ZFS over EXT4 is guaranteed data integrity . ZFS protects your data by enabling '''volume management''' on filesystem level. EXT4 does not offer volume management on the filesystem level
* [https://history-computer.com/zfs-vs-ext4/ ZFS vs EXT4: Best File System for Linux and Other Operating Systems]
* RAM
** [https://youtu.be/ipxl3uzWoMI How *I* Reduced Proxmox RAM Usage with NFS Instead of ZFS ]
** [https://www.reddit.com/r/Proxmox/comments/13e9lxh/the_memory_usage_in_proxmox_is_higher_than/ The memory usage in Proxmox is higher than expected]
== Login timeout ==
[https://forum.proxmox.com/threads/proxmox-gui-session-timeout.46583/ Proxmox GUI Session Timeout]. The login ticket is valid for 2 hours and gets refreshed every 15 minutes.
== Login in browser ==
* [https://pve.proxmox.com/wiki/Import_certificate_in_browser Import certificate in browser]
* [https://pve.proxmox.com/wiki/User_Management User Management]. Authentication Realms
** Proxmox VE Authentication Server
** Linux PAM Standard Authentication
* How to save PVE Web Loginpassword on Firefox/Chrome?


== Display, copy-and-paste ==
== Display, copy-and-paste ==
* [https://novnc.com/noVNC/ noVNC] is still a type of VNC.
** noVNC is an open source VNC client that '''runs well in any modern browser''' including mobile browsers (iOS and Android). It is both a VNC client JavaScript library as well as an application built on top of that library. noVNC follows the standard VNC protocol, but unlike other VNC clients it does require WebSockets support.
** [https://www.kali.org/docs/general-use/novnc-kali-in-browser/ Kali In The Browser (noVNC)]
* [https://pve.proxmox.com/wiki/VNC_Client_Access VNC Client Access]
* By default, Proxmox assigns a standard '''VGA''' device for BIOS-based virtual machines and a '''QXL''' device for UEFI-based virtual machines.
* For Windows OS, we can use the default (noVNC)
* For Windows OS, we can use the default (noVNC)
* For Linux OS, SPICE is better. Screen can be scaled as we want. Copy and paste still does not work after I installed '''spice-vdagent'''. [https://serverfault.com/a/874316 virt-manager Spice copy paste doesn't work]. But the copy-paste function works in Debian VM launched by [https://virt-manager.org/ Virtual Machine Manager] (actually the menu bar are different. One is called 'Remote Viewer'/remote-viewer and the other embedded viewer from ''/usr/bin/qemu-system-x86_64'' is called 'QEMU/KVM').
** QXL: [https://linuxhint.com/install_virtio_drivers_kvm_qemu_windows_vm/ How to Install virtio Drivers on KVM-QEMU Windows Virtual Machines]
* For Linux OS, SPICE is better (clipboard in noVNC does not work). Screen can be scaled as we want. Copy and paste still does not work after I installed '''spice-vdagent'''. [https://serverfault.com/a/874316 virt-manager Spice copy paste doesn't work]. But the copy-paste function works in Debian VM launched by [https://virt-manager.org/ Virtual Machine Manager] (actually the menu bar are different. One is called 'Remote Viewer'/remote-viewer and the other embedded viewer from ''/usr/bin/qemu-system-x86_64'' is called 'QEMU/KVM').
** (Debian11 and antiX VMs) When I use "ps -ef | grep spice", I got '''/usr/sbin/spice-vdagentd''' & '''/usr/bin/spice-vdagent''' as shown in the screenshot [https://www.linux-kvm.org/page/SPICE here] for the VM launched by QEMU/KVM. But I did not see '''/usr/bin/spice-vdagent''' in the VM launched by Proxmox.
** (Debian11 and antiX VMs) When I use "ps -ef | grep spice", I got '''/usr/sbin/spice-vdagentd''' & '''/usr/bin/spice-vdagent''' as shown in the screenshot [https://www.linux-kvm.org/page/SPICE here] for the VM launched by QEMU/KVM. But I did not see '''/usr/bin/spice-vdagent''' in the VM launched by Proxmox.
** B/C a hint from above, I found a solution [https://github.com/biglinux/spice-vdagent-autostart-kde/blob/main/spice-vdagent-autostart-kde.desktop here]. After I run '''/usr/bin/spice-vdagent ''', copy-and-paste works!
** B/C a hint from above, I found a solution [https://github.com/biglinux/spice-vdagent-autostart-kde/blob/main/spice-vdagent-autostart-kde.desktop here]. After I run '''/usr/bin/spice-vdagent ''', copy-and-paste works!
Line 121: Line 173:
** (Fedora 35). Copy-and-paste works out of box (vdagentd & vdagent are automatically running in the background). Maybe it's because Fedora is a Red Hat based Linux OS.
** (Fedora 35). Copy-and-paste works out of box (vdagentd & vdagent are automatically running in the background). Maybe it's because Fedora is a Red Hat based Linux OS.
** How to add spice-vdagent to VirtIO VM?
** How to add spice-vdagent to VirtIO VM?
{| class="wikitable"
|-
! Tool
! Pros
! Cons
|-
| [https://novnc.com/info.html noVNC]
| It is a lighter approach, as it has less services required (less overhead), which allows for a quick “one off connection” solution. It is an open source VNC client JavaScript library as well as an application built on top of that library. It runs well in any modern browser including mobile browsers (iOS and Android).
| The clipboard does not work. Audio device?
|-
| [https://en.wikipedia.org/wiki/Simple_Protocol_for_Independent_Computing_Environments Spice]
| presents the guest windowing system with an X driver that captures X protocol operations directly. This means that Spice can provide better performance than VNC.
| It requires more services than noVNC.
|}


== Sound/audio and SPICE ==
== Sound/audio and SPICE ==
* https://en.wikipedia.org/wiki/Simple_Protocol_for_Independent_Computing_Environments
* SPICE (Simple Protocol for Independent Computing Environments) is a communication protocol used for '''virtual environments'''. It provides a remote display system, allowing users to view a virtualized desktop on their '''local machine''' and interact with it using keyboard and mouse input.
** SPICE is often used in conjunction with virtualization platforms such as QEMU/KVM, and is widely used in enterprise and cloud computing environments.
* SPICE (Simple Protocol for Independent Computing Environments) does not have any direct alternatives as it is a specific '''communication protocol''' used for remote display in virtualized environments. However, there are other remote display protocols such as '''RDP''' (Remote Desktop Protocol), '''VNC''' (Virtual Network Computing), and '''NX''' (NoMachine's Remote X protocol) that can be used as alternatives to SPICE in certain situations. Nonetheless, the most suitable protocol for a specific use case depends on various factors such as the nature of the application, the network bandwidth available, the desired level of graphics performance, and more.
* https://pve.proxmox.com/wiki/SPICE
* https://pve.proxmox.com/wiki/SPICE
* Add sound hardware to VM
** Add sound hardware to VM
** Change Display from default to Spice
** ([https://askubuntu.com/a/1229239 For Lubuntu]) '''sudo apt install spice-vdagent spice-webdavd'''
* Choose SPICE when launching the VM, it will download a '''vv''' file.
* Choose SPICE when launching the VM, it will download a '''vv''' file.
* In ubuntu, "remote-viewer" will be used to open the '''vv''' file when we double clicked the downloaded file. [https://www.networkshinobi.com/proxmox-spice-console/ Proxmox SPICE console] '''apt install virt-viewer
* In ubuntu, "remote-viewer" will be used to open the '''vv''' file when we double clicked the downloaded file. [https://www.networkshinobi.com/proxmox-spice-console/ Proxmox SPICE console] '''apt install virt-viewer
Line 133: Line 206:
* [https://dannyda.com/2020/06/12/how-to-create-a-remote-desktop-gateway-server-on-proxmox-ve-pve/ How to: Create a Remote Desktop Gateway server on Proxmox VE (PVE)]
* [https://dannyda.com/2020/06/12/how-to-create-a-remote-desktop-gateway-server-on-proxmox-ve-pve/ How to: Create a Remote Desktop Gateway server on Proxmox VE (PVE)]


== Qemu-guest-agent ==
== Guest agent ==
https://pve.proxmox.com/wiki/Qemu-guest-agent
This affects whether we can see IP in the Summary option.
 
* [https://pve.proxmox.com/wiki/Qemu-guest-agent Qemu-guest-agent]
* [https://pve.proxmox.com/wiki/Windows_10_guest_best_practices Windows 10 guest best practices]
* [https://www.reddit.com/r/Proxmox/comments/sunq97/ram_usage/ RAM usage]
* [https://www.reddit.com/r/Proxmox/comments/t3feip/ram_always_like_this_since_install_any_idea_why/ RAM always like this since install, any idea why?]
 
<ul>
<li>For Ubuntu VM,
* Proxmox -> VM -> Options -> QEMU Guest Agent. Check both options: Use QEMU Guest Agent & Run guest-trim after a disk move or VM migration.
* On Ubuntu VM,
<pre>
apt-get install qemu-guest-agent
</pre>
<li>For Windows VM,
* Proxmox -> VM -> Options -> QEMU Guest Agent. Check the 1st option is enough.
</ul>


== can't shutdown a VM ==
== can't shutdown a VM ==
Line 180: Line 269:
=== Ethernet port ===
=== Ethernet port ===
[https://www.reddit.com/r/Proxmox/comments/zdzgrk/how_many_ethernet_ports_do_i_need_on_my_proxmox/ How many Ethernet ports do I need on my Proxmox?]
[https://www.reddit.com/r/Proxmox/comments/zdzgrk/how_many_ethernet_ports_do_i_need_on_my_proxmox/ How many Ethernet ports do I need on my Proxmox?]
=== Linux bridge commands ===
[https://developers.redhat.com/articles/2022/04/06/introduction-linux-bridging-commands-and-features# An introduction to Linux bridging commands and features]


== LXC ==
== LXC ==
Line 239: Line 331:
== Storage Drive ==
== Storage Drive ==
<ul>
<ul>
<li>[https://nubcakes.net/index.php/2019/03/05/how-to-add-storage-to-proxmox/ How to add storage to Proxmox] </li>
<li>[https://ostechnix.com/add-external-usb-storage-to-proxmox/ How To Add External USB Storage To Proxmox]
<li>[https://www.reddit.com/r/Proxmox/comments/10spjwj/easiest_way_to_mount_usb_drive_proxmox_7/ easiest way to mount USB drive Proxmox 7?]
<li>[https://www.hostfav.com/blog/index.php/2017/02/01/add-a-new-physical-hard-drive-to-proxmox-ve-4x-5x/ Add a new physical hard drive to Proxmox VE 4x 5x] </li>
<li>[https://nodetwelve.com/add-storage-drive-to-proxmox-7/ Add Storage Drive To Proxmox 7] </li>
<li>Format a disk
<li>Format a disk
Prepare the drive
Prepare the drive
Line 259: Line 346:
</pre>
</pre>
Now to go GUI, pve -> Disks -> Directory -> Create Dir. </li>
Now to go GUI, pve -> Disks -> Directory -> Create Dir. </li>
<li>[https://pve.proxmox.com/wiki/Storage:_LVM_Thin Storage: LVM Thin]
<ul>
<li>[https://dannyda.com/2020/05/10/how-to-delete-remove-local-lvm-from-proxmox-ve-pve-and-some-lvm-basics-commands/ How to: Delete/Remove local-lvm from Proxmox VE (PVE) (How to Expand/Extend/Grow LVM/Ext3/Ext4/XFS) (and Some LVM basics, commands)] </li>
<li>[https://forum.proxmox.com/threads/how-to-erase-lvm-thin-and-crete-legacy-lvm.29953/ How to ERASE LVM-THIN and create Legacy LVM] </li>
<li>[https://forum.proxmox.com/threads/need-to-delete-local-lvm-and-reuse-the-size.34087/ Need to delete local-lvm and reuse the size]
<pre>
# delete local-lvm storage in gui; pve -> Disks -> LVM-Thin -> More -> Restroy
lvdisplay # /dev/pve/swap, /dev/pve/root, /dev/pve/data
lvremove /dev/pve/data
lvresize -l +100%FREE /dev/pve/root
resize2fs /dev/mapper/pve-root
vgdisplay pve | grep Free # should be 0 now
</pre>
</li>
</li>
</ul>
<li>'''lsblk''', '''df -h''' and more
<li>'''lsblk''', '''df -h''' and more
<pre>
<pre>
Line 351: Line 420:
</ul>
</ul>


== Guest agent ==
=== Add a new storage ===
* [https://pve.proxmox.com/wiki/Windows_10_guest_best_practices Windows 10 guest best practices]
<ul>
* [https://www.reddit.com/r/Proxmox/comments/sunq97/ram_usage/ RAM usage]
<li>[https://bobcares.com/blog/add-additional-drive-in-proxmox/ Add additional drive in proxmox | How to?]
* [https://www.reddit.com/r/Proxmox/comments/t3feip/ram_always_like_this_since_install_any_idea_why/ RAM always like this since install, any idea why?]
<li>[https://nubcakes.net/index.php/2019/03/05/how-to-add-storage-to-proxmox/ How to add storage to Proxmox] </li>
<li>[https://ostechnix.com/add-external-usb-storage-to-proxmox/ How To Add External USB Storage To Proxmox]
<li>[https://www.reddit.com/r/Proxmox/comments/10spjwj/easiest_way_to_mount_usb_drive_proxmox_7/ easiest way to mount USB drive Proxmox 7?]
<li>[https://www.hostfav.com/blog/index.php/2017/02/01/add-a-new-physical-hard-drive-to-proxmox-ve-4x-5x/ Add a new physical hard drive to Proxmox VE 4x 5x] </li>
<li>[https://nodetwelve.com/add-storage-drive-to-proxmox-7/ Add Storage Drive To Proxmox 7] </li>
 
<li>[https://pve.proxmox.com/wiki/Storage:_LVM_Thin Storage: LVM Thin]
<ul>
<li>[https://dannyda.com/2020/05/10/how-to-delete-remove-local-lvm-from-proxmox-ve-pve-and-some-lvm-basics-commands/ How to: Delete/Remove local-lvm from Proxmox VE (PVE) (How to Expand/Extend/Grow LVM/Ext3/Ext4/XFS) (and Some LVM basics, commands)] </li>
<li>[https://forum.proxmox.com/threads/how-to-erase-lvm-thin-and-crete-legacy-lvm.29953/ How to ERASE LVM-THIN and create Legacy LVM] </li>
<li>[https://forum.proxmox.com/threads/need-to-delete-local-lvm-and-reuse-the-size.34087/ Need to delete local-lvm and reuse the size]
<pre>
# delete local-lvm storage in gui; pve -> Disks -> LVM-Thin -> More -> Restroy
lvdisplay # /dev/pve/swap, /dev/pve/root, /dev/pve/data
 
lvremove /dev/pve/data
lvresize -l +100%FREE /dev/pve/root
resize2fs /dev/mapper/pve-root
 
vgdisplay pve | grep Free # should be 0 now
</pre>
</li>
</li>
</ul>
</ul>
 
=== Upgrade storage ===
[https://www.reddit.com/r/Proxmox/comments/13fndny/resizing_after_copy_to_bigger_ssd_my_experience/ Resizing after copy to bigger SSD my experience]


== Plex ==
== Plex ==
[https://herold.space/proxmox-lxc-intel-quick-sync-for-plex/ Proxmox LXC Intel Quick Sync Transcode for Plex]
[https://herold.space/proxmox-lxc-intel-quick-sync-for-plex/ Proxmox LXC Intel Quick Sync Transcode for Plex]
== Proxmox increase vm disk size ==
* https://pve.proxmox.com/wiki/Resize_disks
* [https://www.wundertech.net/how-to-increase-vm-disk-size-in-proxmox/ How to Increase VM Disk Size in Proxmox]
* [https://dixmata.com/resize-disk-vm-proxmox/ Proxmox Resize Disk VM / Extend Disk VM LVM]


== Clone (full clone vs convert to template) ==
== Clone (full clone vs convert to template) ==
Line 390: Line 491:
<li>Question: why sometimes my backup files are not shown on GUI. </BR>
<li>Question: why sometimes my backup files are not shown on GUI. </BR>
Ans: the default backup storage is "local". We need to toggle that from the GUI. PS: the storage we changed cannot be memorized.
Ans: the default backup storage is "local". We need to toggle that from the GUI. PS: the storage we changed cannot be memorized.
<li>Schedule backup: [https://4sysops.com/archives/backup-in-proxmox-ve/ Backup in Proxmox VE] with screenshots.
</ul>
</ul>


Line 403: Line 505:


== USB passthrough ==
== USB passthrough ==
* [https://pve.proxmox.com/wiki/USB_Devices_in_Virtual_Machines USB Devices in Virtual Machines]
* [https://manjaro.site/how-to-passthrough-usb-disk-to-a-virtual-machine-in-proxmox-6-2/ How to Passthrough USB Disk to a Virtual Machine in Proxmox 6.2], [https://youtu.be/XQOV70JW2zE Adding USB Devices to Proxmox VM] (video). This assumes the USB device is on the proxmox host.
* [https://manjaro.site/how-to-passthrough-usb-disk-to-a-virtual-machine-in-proxmox-6-2/ How to Passthrough USB Disk to a Virtual Machine in Proxmox 6.2], [https://youtu.be/XQOV70JW2zE Adding USB Devices to Proxmox VM] (video). This assumes the USB device is on the proxmox host.
* [https://www.reddit.com/r/Proxmox/comments/13adiiv/share_ext_ssd_for_samba_and_vms/ Share ext SSD for samba and VMs]
* (video)
* [https://youtu.be/VfGnAGT8eRI Adding USB Redirection to the Raspberry Pi + Proxmox Thin Client]. The USB device is on the client.
* [https://youtu.be/VfGnAGT8eRI Adding USB Redirection to the Raspberry Pi + Proxmox Thin Client]. The USB device is on the client.
* [https://youtu.be/I5zA1lU5Tw0 Converting Any USB Device to A Wireless USB], [https://www.virtualhere.com/home VirtualHere] ($49, locked to one device). The server (USB devices are plugged in) can be any where.
** [https://youtu.be/I5zA1lU5Tw0 Converting Any USB Device to A Wireless USB], [https://www.virtualhere.com/home VirtualHere] ($49, locked to one device). The server (USB devices are plugged in) can be any where.


== Thin client ==
== Thin client ==
Line 439: Line 544:
* [https://www.proxmox.com/en/proxmox-ve/comparison Comparison of Server Virtualization]
* [https://www.proxmox.com/en/proxmox-ve/comparison Comparison of Server Virtualization]


= Proxmox Backup Server =
= Proxmox Backup Server/PBS =
[https://ostechnix.com/install-proxmox-backup-server/ How To Install Proxmox Backup Server Step by Step]
* [https://ostechnix.com/install-proxmox-backup-server/ How To Install Proxmox Backup Server Step by Step]
* [https://ostechnix.com/getting-started-with-proxmox-backup-server/ Getting Started With Proxmox Backup Server]
* [https://www.reddit.com/r/Proxmox/comments/12zumo6/dumb_question_can_pbs_be_used_as_standalone/ Can PBS be used as standalone backup server with no integration to proxmox ve?]
 
== proxmox-backup-client ==
* [https://pbs.proxmox.com/docs/backup-client.html Backup Client Usage]


= Remote machine management =
= Remote machine management =

Revision as of 12:43, 20 May 2023

Resource

Proxmox Virtual Environment

Set up

Windows VM

  • How to Unlock a Proxmox VM. Then reboot pve.
  • For Windows installation, if we are dropped at the EFI shell, we can use reset -s to shut down the system.
  • Windows VirtIO Drivers & Creating Windows virtual machines using virtIO drivers from fedoraproject.org
  • (Videos) Launching a Windows VM in Proxmox (Win10), Virtualize Windows 10 with Proxmox VE (Win10)
  • In OS tab, be sure to choose the right Guest OS Version; if we choose (10/2016/2019), we will select 2019 in the driver folder locations OR if choose (11/2022), we will select the corresponding driver folders. This affects the later steps when we select the drivers to install.
  • There are 3 drivers we shall install.
    • vioscsi\win10\amd64\vioscsi.inf (we need it in order to see the virtual disk)
    • NetKVM\w10\amd64\netkvm.inf (in order to get network working, this can be installed later from Windows Device Manager)
    • Balloon\win10\amd64\balloon.inf (memory balloon driver, this can be installed later)
    • Guest agent (Qemu-guest-agent). Make sure we have installed all missing drivers from Device Manager. We can go to virtual machine Summary tab to check if the IPs shows the Windows IP.
  • Balloon driver
    • If balloon driver is not installed on Windows Virtual Machines then memory metrics will not be collected for windows virtual machine. Steps to Install balloon driver on Windows Virtual Machines
    • A balloon driver is a component of the VMware Tools package of drivers and utilities that help virtual machines (VMs) run better. The purpose of the balloon driver is to take physical RAM from a VM and release it back to the VMkernel. This reclaim usually happens when the ESXi server is short on RAM and this specific VM is the loser in the competition for physical RAM. How the balloon driver helps VM performance
  • Change network to use a static IP.
  • Enable RDP.
  • (Related to Audio device). Change Hardware - Display - SPICE (instead of default). W/o doing that, we'll get an error "Can't start vms with audio device(SPICE) in pve 7.3 "
  • For Audio, we need to add it to the hardware list (driver=spice is OK). PS: audio works when I tested using the Microsoft Remote Desktop app on mac. It does not work when I use Remmina app from Ubuntu initially but if I change the sound setting (from "Off" to "Local" in the "Advanced" tab) it works.
  • Windows 11 (10/21/2021). We can still use 10/2016/2019 for the Version selection. When I check the Task Manager (Windows 11 Pro, Version 21H2, OS build 22000.675), it shows 1.8/4GB was used and 101 out of 127GB is free.
    • OS: version 10/2016/2019
    • System: BIOS: OVMF (UEFI). Machine - q35. Add TPM. Qemu Agent. SCSI: VirtIO SCCI
    • HD: Bus-VirtIO Block. Disk size >=32
    • CPU: Cores >=2. Type - Host
    • Memory
    • Network: VirtIO
    • After finish, adding CD/DVD - VirtIO iso
    • Install: Browse VirtIO disk. AMD - win10. Next, browse NetKVM - win10.
    • After reboot. Continue to answer questions. Reboot
    • File manager. VirtIO disk. Run virtio-win-gt-x64.exe.
    • Right click on Start. Device Manager. All clean.
  • Windows 11 VM / Office 2019 install error 0-2054
  • Windows 11 virtual machine on Proxmox 8/19/2022
  • AMD/NVIDIA GPU Passthrough in Window 11 - Proxmox Guide 7/22/2022

Mac VM

Install on Debian

Installing Proxmox VE 7.x on Debian Bullseye for custom partition layout (video)

Cheat sheet

https://github.com/vzamora/Proxmox-Cheatsheet

SSD/HDD choices

Home Server

My Proxmox Home Server Walk-Through

ZFS, NFS

  • ZFS (Zettabyte File System): A file system developed by Sun Microsystems for use in their Solaris operating system. It is now available on many other operating systems.
  • Yes, ZFS can be used without LVM. Even on a workstation, you could use ZFS to pool your disks into a single large pool of storage rather than keep them separate or rely on LVM. How to Install and Use ZFS on Ubuntu (and Why You’d Want To)
  • ZFS vs RAID-0
    • ZFS is not like RAID-0. RAID-0 is a type of RAID that stripes data across multiple disks without any redundancy. If one disk fails, all data is lost. ZFS, on the other hand, provides data redundancy and checksumming to avoid silent data corruption.
  • ZFS cons
    • XFS vs ZFS vs Linux Raid. ZFS is a very CPU-intensive filesystem. This can lead to slower performance on systems with limited CPU resources.
  • Proxmox installation.
    • The default file system is ext4. ZFS is an alternative to ext4.
    • The Proxmox VE installer, which partitions the local disk(s) with ext4, XFS, BTRFS (technology preview), or ZFS and installs the operating system.
    • The main advantage of ZFS over EXT4 is guaranteed data integrity . ZFS protects your data by enabling volume management on filesystem level. EXT4 does not offer volume management on the filesystem level
  • ZFS vs EXT4: Best File System for Linux and Other Operating Systems
  • RAM

Login timeout

Proxmox GUI Session Timeout. The login ticket is valid for 2 hours and gets refreshed every 15 minutes.

Login in browser

Display, copy-and-paste

  • noVNC is still a type of VNC.
    • noVNC is an open source VNC client that runs well in any modern browser including mobile browsers (iOS and Android). It is both a VNC client JavaScript library as well as an application built on top of that library. noVNC follows the standard VNC protocol, but unlike other VNC clients it does require WebSockets support.
    • Kali In The Browser (noVNC)
  • VNC Client Access
  • By default, Proxmox assigns a standard VGA device for BIOS-based virtual machines and a QXL device for UEFI-based virtual machines.
  • For Windows OS, we can use the default (noVNC)
  • For Linux OS, SPICE is better (clipboard in noVNC does not work). Screen can be scaled as we want. Copy and paste still does not work after I installed spice-vdagent. virt-manager Spice copy paste doesn't work. But the copy-paste function works in Debian VM launched by Virtual Machine Manager (actually the menu bar are different. One is called 'Remote Viewer'/remote-viewer and the other embedded viewer from /usr/bin/qemu-system-x86_64 is called 'QEMU/KVM').
    • (Debian11 and antiX VMs) When I use "ps -ef | grep spice", I got /usr/sbin/spice-vdagentd & /usr/bin/spice-vdagent as shown in the screenshot here for the VM launched by QEMU/KVM. But I did not see /usr/bin/spice-vdagent in the VM launched by Proxmox.
    • B/C a hint from above, I found a solution here. After I run /usr/bin/spice-vdagent , copy-and-paste works!
    • In summary, 1) sudo apt install vdagent 2) /usr/bin/spice-vdagent
    • (Fedora 35). Copy-and-paste works out of box (vdagentd & vdagent are automatically running in the background). Maybe it's because Fedora is a Red Hat based Linux OS.
    • How to add spice-vdagent to VirtIO VM?
Tool Pros Cons
noVNC It is a lighter approach, as it has less services required (less overhead), which allows for a quick “one off connection” solution. It is an open source VNC client JavaScript library as well as an application built on top of that library. It runs well in any modern browser including mobile browsers (iOS and Android). The clipboard does not work. Audio device?
Spice presents the guest windowing system with an X driver that captures X protocol operations directly. This means that Spice can provide better performance than VNC. It requires more services than noVNC.

Sound/audio and SPICE

  • https://en.wikipedia.org/wiki/Simple_Protocol_for_Independent_Computing_Environments
  • SPICE (Simple Protocol for Independent Computing Environments) is a communication protocol used for virtual environments. It provides a remote display system, allowing users to view a virtualized desktop on their local machine and interact with it using keyboard and mouse input.
    • SPICE is often used in conjunction with virtualization platforms such as QEMU/KVM, and is widely used in enterprise and cloud computing environments.
  • SPICE (Simple Protocol for Independent Computing Environments) does not have any direct alternatives as it is a specific communication protocol used for remote display in virtualized environments. However, there are other remote display protocols such as RDP (Remote Desktop Protocol), VNC (Virtual Network Computing), and NX (NoMachine's Remote X protocol) that can be used as alternatives to SPICE in certain situations. Nonetheless, the most suitable protocol for a specific use case depends on various factors such as the nature of the application, the network bandwidth available, the desired level of graphics performance, and more.
  • https://pve.proxmox.com/wiki/SPICE
    • Add sound hardware to VM
    • Change Display from default to Spice
    • (For Lubuntu) sudo apt install spice-vdagent spice-webdavd
  • Choose SPICE when launching the VM, it will download a vv file.
  • In ubuntu, "remote-viewer" will be used to open the vv file when we double clicked the downloaded file. Proxmox SPICE console apt install virt-viewer

Remote Desktop through browser

Guest agent

This affects whether we can see IP in the Summary option.

  • For Ubuntu VM,
    • Proxmox -> VM -> Options -> QEMU Guest Agent. Check both options: Use QEMU Guest Agent & Run guest-trim after a disk move or VM migration.
    • On Ubuntu VM,
    apt-get install qemu-guest-agent
    
  • For Windows VM,
    • Proxmox -> VM -> Options -> QEMU Guest Agent. Check the 1st option is enough.

can't shutdown a VM

Use the command qm unlock XXX

qm stop XXX
# can't lock file '/var/lock/qemu-server/lock-996.conf' - got timeout

qm unlock XXX
qm stop XXX

Now we can go back to proxmox GUI to remove the vm.

But if the "qm unlock" does not work, we can use the kill command. See Proxmox can’t stop VM – How we fix it!

ps aux | grep "/usr/bin/kvm -id VMID"
kill -9 PID  # VM will stop

Errors and solutions

Removing ProxMox Subscription Notice

Network

Ethernet port

How many Ethernet ports do I need on my Proxmox?

Linux bridge commands

An introduction to Linux bridging commands and features

LXC

Pi hole

Installing Pi-Hole inside a Proxmox LXC Container. 2GB disk, 1 CPU core, and 256MB of memory. The memory usage is pretty flat around 53MB according to Proxmox gui). I am using Debian 11 template.

apt update
apt upgrade
nano /etc/sysctl.conf # disable IPv6 
reboot
apt install curl
curl -sSL https://install.pi-hole.net | bash
pihole -a -p # change to a simpler password

SMART and wearout

I saw the wearout is 99% on my host disk (240GB Kingston SSD). I cannot delete a VM. If I use the command "qm destroy XXX", it shows "Unable to create output file '/var/log/pve/tasks/1/UPID:pvv....:qmdestroy:108:root@pam:' - Read-only file system". The host disk uses only 37% of storage on root partition. The solution: reboot Proxmox.

Storage Drive

  • Format a disk Prepare the drive
    fdisk /dev/nvme0n1
     : p
     : d
     :   ENTER
     : p
     : w
     : n ENTER ENTER ENTER
     : p
     : w
     : q 
    
    Now to go GUI, pve -> Disks -> Directory -> Create Dir.
  • lsblk, df -h and more
    # lsblk
    NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
    sda            8:0    0 223.6G  0 disk 
    ├─sda1         8:1    0  1007K  0 part 
    ├─sda2         8:2    0   512M  0 part /boot/efi
    └─sda3         8:3    0 223.1G  0 part 
      ├─pve-swap 253:0    0     8G  0 lvm  [SWAP]
      └─pve-root 253:1    0 215.1G  0 lvm  /
    nvme0n1      259:0    0 465.8G  0 disk 
    └─nvme0n1p1  259:1    0 465.8G  0 part /mnt/pve/vm1
    
    # df -h
    Filesystem            Size  Used Avail Use% Mounted on
    udev                  7.7G     0  7.7G   0% /dev
    tmpfs                 1.6G  1.3M  1.6G   1% /run
    /dev/mapper/pve-root  214G   30G  176G  15% /
    tmpfs                 7.8G   40M  7.7G   1% /dev/shm
    tmpfs                 5.0M     0  5.0M   0% /run/lock
    /dev/nvme0n1p1        458G  2.0G  433G   1% /mnt/pve/vm1
    /dev/sda2             511M  328K  511M   1% /boot/efi
    /dev/fuse             128M   16K  128M   1% /etc/pve
    tmpfs                 1.6G     0  1.6G   0% /run/user/0
    
    # fdisk -l
    Disk /dev/nvme0n1: 465.76 GiB, 500107862016 bytes, 976773168 sectors
    Disk model: WDC WDS500G2B0C                         
    ...
    Device         Start       End   Sectors   Size Type
    /dev/nvme0n1p1  2048 976773134 976771087 465.8G Linux filesystem
    
    Disk /dev/sda: 223.57 GiB, 240057409536 bytes, 468862128 sectors
    Disk model: KINGSTON SA400S3
    ...
    Device       Start       End   Sectors   Size Type
    /dev/sda1       34      2047      2014  1007K BIOS boot
    /dev/sda2     2048   1050623   1048576   512M EFI System
    /dev/sda3  1050624 468862094 467811471 223.1G Linux LVM
    
    Disk /dev/mapper/pve-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    Disk /dev/mapper/pve-root: 215.07 GiB, 230925795328 bytes, 451026944 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    # pvdisplay
      --- Physical volume ---
      PV Name               /dev/sda3
      VG Name               pve
      PV Size               <223.07 GiB / not usable <3.57 MiB
      Allocatable           yes (but full)
      PE Size               4.00 MiB
      Total PE              57105
      Free PE               0
      Allocated PE          57105
      PV UUID               4taiYE-DDJa-4UdU-v3QY-kd2s-7r5i-shhJ7Q
    
  • Benchmark HD speed
    hdparm -t --direct /dev/nvme0n1p1
    hdparm -t --direct /dev/sda3
    
  • lvresize vs lvextend. lvextend can only increase the size of a volume, whereas lvresize can increase or reduce it. Increase the size of an LVM logical volume.
  • How to Manage and Use LVM (Logical Volume Management) in Ubuntu

Add a new storage

Upgrade storage

Resizing after copy to bigger SSD my experience

Plex

Proxmox LXC Intel Quick Sync Transcode for Plex

Proxmox increase vm disk size

Clone (full clone vs convert to template)

Backup and restore VM

  • Backup and Restore from proxmox wiki. How to Backup Proxmox? Proxmox Backup and Recovery Methods.
  • How to backup and transfer a Proxmox VM to another Proxmox Node
    • Backup file has a timestamp on the filename and it is saved to /var/lib/vz/dump if it is saved in local (pve) or /mnt/pve/vm1/dump if it is saved on my vm1 storage.
    • The backup file can be seen on the GUI under $STORAGE (pve) -> Backups where $STORAGE is the storage name (e.g. local). From there, it has a 'Restore' button where we can restore it with an option to enter a new VM ID.
    • If the backup file is saved in local (pve), the backup file can also be seen under the VM|Backup menu.
    • The backup VM has a filename vzdump-qemu-$ID-$Date-$Time.vma.zst (compression by default). If we select the file, we can click the 'Restore' button to restore the VM.
    • After restoring, the new VM has a new ID but the VM name is still the same as the original one (so we can only use the ID to distinguish the VMs). Therefore, if we use the static IP in a VM, it is better to shut down the old one before we Start the new VM.
    • If we remove/delete the restored VM, the backup file is not affected (not deleted).
    • It seems backup + restore = clone.
    • If we back up a container, the backup file name has a format vzdump-lxc-$ID-$Date-$Time.tar.zst.
    • The backup speed is quick. Using the "top" command it shows it is the command zstd running for the backup.
  • Restore Proxmox VM from backup – Here are the steps to recover your VM
  • VM ID:
    • One biggest problem is I cannot tell what the backup file is from the file name after I back up the files to another location. The backup file however contains VM ID on its filename. That is the only clue we can use to find out what the VM is on the original Proxmox.
    • The Backup Notes is useful actually. In the backup folders on Proxmox, it also create *.notes files if we create it on the GUI.
    • Maybe it's useful for me to create a text file along with the backup files to show what the files represents.
  • Question: why sometimes my backup files are not shown on GUI.
    Ans: the default backup storage is "local". We need to toggle that from the GUI. PS: the storage we changed cannot be memorized.
  • Schedule backup: Backup in Proxmox VE with screenshots.

Mount a Network Share in a Linux Container

How to mount a Network Share in a Linux Container under Proxmox

Setup a MediaWiki Server

How to Setup a MediaWiki 1.31 Server on a Debian 10 Proxmox container

Multiple node cluster

USB passthrough

Thin client

Raspberry Pi THIN CLIENT for Proxmox VMs

Android app

Proxmox Virtual Environment

Android emulator and nested virtualization

Is there a guide to getting Android x86 installed on Proxmox?

Security

Am I compromised? If you need true remote access, set up a VPN that you connect to on your router.

Error 401: no ticket

Empty browser's cache. It works.

Check SMB Share availability before starting a VM

add a startup delay to your VMs and CTs if this is the only time you have this issue. Then set up a script in crontab to run @reboot that just waits x seconds, then remounts the share.

Cloud image, Cloud-init

Proxmox vs. ESXi

Proxmox Backup Server/PBS

proxmox-backup-client

Remote machine management

Self-Hosted Remote Desktop Connection Alternatives

Remotely

RPort