KVM
Resource
- KVM Kernel-based Virtual Machine
- https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine
- Virtual Machine Manager - virt-manager. Two display types: SPICE or VNC, and uses libvirt to lookup the graphical connection details.
- My notes:
- One advantage over VB for me: No need to go to VirtualBox website for the installation and frequent updates. I can use the command line to install it.
- The desktop screen resolution can be automatically adjusted without a need to install guest addition or vmware tool. Tested Ubuntu and Pop_OS.
- It has two windows. One is virt-manager and the other is the virtual machine.
- Use Left Control + Left Alt to escape the virtual machine.
- https://en.wikipedia.org/wiki/QEMU
- It is probably the most widely deployed open source hypervisor in an open source environment.
- KVM cannot be installed on a Windows system.
- How to Install Linux KVM and Create Guest VM with Examples. Para-virtualization, checking the virtualization support on hardware and others. It includes the usage of virsh and virt-install commands.
- Virtualization solutions on Linux systems - KVM and VirtualBox
- CHAPTER 2. GETTING STARTED WITH VIRTUALIZATION from redhat.com
- How To Use QEMU To Test Operating Systems & Distributions
- Using KVM and VirtualBox side by side and VirtualBox and KVM beside each other
- Comparison
- Fundamental question:
- Can we install KVM without any Operating System installed -Yes because KVM is already a part of Linux kernel. The Linux kernel acts as the virtualization engine.
- Is it possible to install the Linux kernel alone?
- Change amount of Ram and CPU cores in KVM, HowTo Add Memory in KVM Virtual Machine
virsh --version virsh list virsh dominfo ubuntu16.04 | grep mem virsh setmaxmem ubuntu16.04 --size 4194304 --config # in KB virsh setmem ubuntu16.04 --size 2097152 --config # in KB virsh dominfo ubuntu16.04 | grep mem # Or edit the XML configuration file virsh edit ubuntu16.04
libvirt
- https://en.wikipedia.org/wiki/Libvirt. libvirt is an open-source API, daemon and management tool for managing platform virtualization.[3] It can be used to manage KVM, Xen, VMware ESXi, QEMU and other virtualization technologies.
- https://libvirt.org/
- Check libvirt version: sudo head -n2 /var/log/libvirt/qemu/ubuntu2004.log # on Ubuntu 20.04 it shows 6.0.0
virt-install
$ virt-install --help | grep graphics $ osinfo-query os $ sudo virt-install -n ubuntu2004 --os-type=Linux \ --os-variant=ubuntu18.04 --ram=2096 --vcpus=2 \ --disk size=20 \ --network network=br0 \ --graphics vnc,listen=0.0.0.0 \ --location Downloads/ubuntu-20.04.1-desktop-amd64.iso $ sudo virt-install --name=centos7server \ --os-type=Linux \ --os-variant=centos7.0 \ --vcpu=2 --ram=2048 \ --disk size=20 \ --graphics vnc \ --location=/tmp/CentOS-7-x86_64-DVD-2009.iso
Check if running in VM
9 things to do in your first 10 minutes on a Linux server
$ cat /sys/class/dmi/id/sys_vendor QEMU $ sudo dmidecode -s system-manufacturer QEMU
On VirtualBox + CentOS vm, it returns "innotek GmbH".
On VMware + CentOS vm, it returns "VMware, Inc".
Guest additions
It seems QEUM/KVM does not have that to download.
- Share clipboard. How can I copy&paste from the host to a KVM guest? sudo apt install spice-vdagent in the guest virtual machine. had to reboot guest to be able to make it work. It works.
- Display resolution. The display has a resolution 1024x768 (4:3). I can increase to 1600x900 (16:9) or even 1920x1080 (16:9) in the guest machine. In the virtual machine window, I do 'View' -> 'Scale to Display' where I choose 'Always' and check the checkbox 'Auto resize VM with window'. It did increase the resolution. See also the command line method: How to increase the visualized screen resolution on QEMU / KVM?
- Virt-Manager And Full-screen Display. View -> Scale Display -> Auto resize VM with window.
- Share files
- QEMU/KVM on Ubuntu and sharing files between host and guests. It works. Modify/create a file /etc/rc.local file to automount the folder at each login. Note the share folder still belongs to the user's group. So we can actually create a soft link for this folder on the host.
- Is there something like “VirtualBox Guest Additions” for QEMU/KVM?
- Mouse integration. virt-manager KVM mouse pointer integration (client mouse mode) with Ubuntu guest. Any way to release focus on a KVM guest in virt-manager without having to click Ctrl_L + Alt_L?
Bridge network
- 11.2.6. NETWORK BRIDGE from redhat.com. It seems possible to share a NIC with one or more virtual NICs.
- Libvirt: one NIC, multiple VM, all with Bridged mode
- Configuring Guest Networking
- (Not about bridge) KVM libvirt assign static guest IP addresses using DHCP on the virtual machine
- Configure public IP addresses in KVM VMs (using brctl)
- Debian Linux: Configure Network Interfaces As A Bridge / Network Switch (using brctl)
- Ubuntu 20.04 add network bridge (br0) with nmcli command (using nmcli)
- There are different ways to configure the network in Ubuntu.
- If we are using a Ubuntu server, we will use the Netplan method.
- If we are using a Ubuntu desktop, we will use the Network manager command line nmcli method or the GUI tool nm-connection-editor.
- Use nmcli to add a bridge network and a bridge slave interface
sudo nmcli con add ifname br0 type bridge con-name br0 sudo nmcli con add ifname eno1 type bridge-slave master br0
- Use nmcli to bring down a wired network and bring up a bridge network
sudo nmcli con down "Wired connection 1" sudo nmcli con up br0
- Use nmcli to delete unused bridge networks
sudo nmcli device show sudo nmcli con show sudo nmcli con del "br0"
Install
Install on Ubuntu
- Virtualization - The Debian Administrator's Handbook
- How to install KVM on Ubuntu 14.04 LTS Headless Server from cyberciti
- Simple Virtualization With Ubuntu 16.04 Linux and KVM from linuxconfig
- KVM virtualization on a home PC with Linux Mint
- How to Install and Configure KVM on Ubuntu 18.04 LTS Server. I skip step 4: Configure Network Bridge for KVM virtual Machines.
- How to bring down br0 and delete it? After some messing up I need to delete br0.
sudo ip link set br0 down # ifconfig command is not found anymore sudo brctl delbr br0
- Ubuntu 20.04 Essentials (book) → Creating an Ubuntu 20.04 KVM Networked Bridge Interface. It is based on a Ubuntu Desktop environment. The description is quite detailed. Have not tried yet!
- How to Install KVM on Ubuntu 20.04
- qemu: qemu
- qemu-kvm: ovmf, qemu-block-extra, qemu-system-common, qemu-system-data, qemu-system-gui, qemu-system-x86, qemu-utils, seabios, sharutils
- libvritd-daemon: libvirt0, libvirt-daemon, libvirt-daemon-driver-qemu, ...
- virt-manager: qemu-block-extra, qemu-utils, virt-viewer, virtinst, ...
sudo apt install -y qemu qemu-kvm libvirt-daemon \ libvirt-clients bridge-utils virt-manager sudo systemctl status libvirtd # enable it to start on boot sudo systemctl enable --now libvirtd lsmod | grep -i kvm ip a # I got two new: virbr0, virbr0-nic # Click the superkey (Windows) and search/launch "Virtual Machine Manager"
Click the "Create a new virtual machine" button will result in an error: No active connection to install on. Rebooting does not help. But when I use the command line to create a new VM it works (It seems sudo helps).
sudo virt-install --name=ubuntu2004 \ --os-type=Linux \ --os-variant=ubuntu20.04 \ --vcpu=2 --ram=2048 \ --graphics vnc \ --location=ubuntu-20.04.1-desktop-amd64.iso
Add --network bridge:br0 if we have added a new bridge network (br0) in the host (eg using netplan or nmcli or nm-connection-editor GUI Tool in Ubuntu). See some resource below.
- How to install KVM server on Debian 9/10 Headless Server
- How to install KVM on Ubuntu 20.04 LTS Headless Server.
- IP 192.168.2.25 is the new static IP for the bridge network (not virtual machine) and 192.168.2.254 is the gateway current ethernet is using.
- Note according to How to add network bridge with nmcli (NetworkManager) on Linux, WiFi is not possible. It only works with Ethernet. Confirmed! The bridge network can be pinged from another computer.
- How to create wireless bridge connection with nmcli. Ans: This is not possible.
- I follow this How to add network bridge with nmcli (NetworkManager) on Linux disable STP, turn off “Wired connection 1” and turn on br0.
- Question: Do I need to turn off "Wired connection 1". It will lose the original host's static IP. The new bridge IP will become the host's IP. The guest machine still obtain a new IP via DHCP. This is not what I expect?
- Before turn off “Wired connection 1” and turn on br0:
$ nmcli connection show --active NAME UUID TYPE DEVICE Wired connection 1 XXX ethernet eno1 docker0 XXX bridge docker0 virbr0 XXX bridge virbr0
- After turn off “Wired connection 1” and turn on br0:
$ nmcli connection show --active NAME UUID TYPE DEVICE br0 xxx bridge br0 docker0 XXX bridge docker0 virbr0 XXX bridge virbr0 bridge-slave-eno1 XXX ethernet eno1 vnet0 XXX tun vnet0
- I am not able to use remmina to VNC to the bridge VM. Not sure if this is because my host OS is a desktop (not headless).
-
os-variant list in virt-install
$ osinfo-query os
- Where are image files saved: /var/lib/libvirt/images (/var/lib/docker for Docker)
- Troubleshooting: virt-manager can't connect to libvirt. Rebooting computer!
VNC
- KVM/QEMU 以 virt-install 指令建立虛擬機器、VNC 顯示畫面教學
sudo virt-install --virt-type kvm --name ubuntu1804 --ram 2048 --disk /home/seal/kvm/ubuntu1804.qcow2,format=qcow2 --network network=default --graphics vnc,listen=0.0.0.0,password=YOUR_PASSWORD --noautoconsole --os-type=linux --os-variant=ubuntu17.10 --cdrom=/home/seal/Downloads/ubuntu-18.10-desktop-amd64.iso virsh vncdisplay ubuntu1804 # 127.0.0.1:1 # pay attention to the port number # :0 --> 5900 in ssh # :1 --> 5901 in ssh sudo ufw disable # Another machine. # Suppose the host machine has an IP 192.168.0.3 ssh -NL 5901:localhost:5901 [email protected] ssh -NL 5901:127.0.0.1:5901 [email protected] vncviewer 127.0.0.1:1 # from a different machine
NOTE: I have to use local port forwarding. If I try to directly connect to the VM in RealVNC, it will show The connection was refused by the computer. A limit by KVM?
The question is how do we know the client's IP? I guess cockpit method is better.
Install qemu-system-arm in Ubuntu
- sudo apt install qemu-system-arm
- How to emulate Raspberry Pi in QEMU which is based on qemu-rpi-kernel-Qemu kernel for emulating Rpi on QEMU.
mkdir rpi-vm && cd rpi-vm git clone https://github.com/dhruvvyas90/qemu-rpi-kernel.git # Download Raspbian Buster Lite # from https://www.raspberrypi.org/downloads/raspberry-pi-os/ dd if=/dev/zero bs=1M count=4096 >> 2020-08-20-raspios-buster-armhf-lite.img sudo losetup -f -P --show 2020-08-20-raspios-buster-armhf-lite.img sudo mount -o rw /dev/loopXp2 /mnt sudo nano /mnt/etc/fstab sudo nano /etc/ld.so.preload # Comment out the line and save the file. # Then unmount with the following commands: # sudo umount /mnt # sudo losetup -d /dev/loopX qemu-system-arm \ -M versatilepb \ -cpu arm1176 \ -m 256 \ -kernel qemu-rpi-kernel/kernel-qemu-4.19.50-buster \ -hda 2020-08-20-raspios-buster-armhf-lite.img \ -append "dwc_otg.lpm_enable=0 root=/dev/sda2 console=tty1 rootfstype=ext4 elevator=deadline rootwait" \ -dtb qemu-rpi-kernel/versatile-pb-buster.dtb \ -no-reboot \ -serial stdio # We can install a desktop. See # https://www.raspberrypi.org/forums/viewtopic.php?t=133691
Install on CentOS
- How to install KVM on CentOS 8 Headless Server
- How to install KVM on CentOS 7 / RHEL 7 Headless Server
- How to Deploy Multiple Virtual Machines using Network Install (HTTP, FTP and NFS) under KVM Environment
qemu-system-x86_64
- The command can be used to create or rung a VM. See https://www.linux-kvm.org/page/RunningKVM
- How To Use QEMU To Test Operating Systems & Distributions
- virt-install vs qemu-system-x86_64
Install using qcow2 image
- https://cloud.centos.org/centos/7/images/
- Requesting for username and Password for the CentOS7 Qcow image
- KVM: Testing cloud-init locally using KVM for a CentOS cloud image*
- How to add ssh key to qcow2 Linux cloud images using virt-sysprep
Running a virtual machine from a ramdisk
- List of RAM drive software
- How to Create a RAM Drive in Linux
if [ "$(mount | grep -o "/mnt/ram")" != "/mnt/ram" ]; then sudo mkdir -p /mnt/ram sudo mount -t ramfs -o size=1g ramfs /mnt/ram sudo chown -R $(whoami):$(whoami) /mnt/ram fi mount | grep ram sudo umount /mnt/ram
- https://youtu.be/fOaWrhLoVK0
- Libvirt In-Memory Storage
- (Windows only) USE A RAM DISK TO GET EXTREME PERFORMANCE FROM TEMPORARY VIRTUAL MACHINES
- How to Easily Create RAM Disk on Debian, Ubuntu, Linux Mint, CentOS. How to Create a RAM Disk in Any Linux Distro.
sudo mkdir /tmp/ramdisk sudo chmod 777 /tmp/ramdisk htop # check free ram sudo mount -t tmpfs -o size=1G myramdisk /tmp/ramdisk # myramdisk is the device name I gave to it mount | tail -n 1 sudo umount /tmp/ramdisk/
Test RAM speed
# test write speed sudo dd if=/dev/zero of=/tmp/ramdisk/zero bs=4k count=100000 ## 409600000 bytes (410 MB, 391 MiB) copied, 0.734813 s, 557 MB/s # SATA SSD ## 409600000 bytes (410 MB, 391 MiB) copied, 0.18296 s, 2.2 GB/s # NVME # test read speed sudo dd if=/tmp/ramdisk/zero of=/dev/null bs=4k count=100000 ## 409600000 bytes (410 MB, 391 MiB) copied, 0.410809 s, 997 MB/s # SATA SSD ## 409600000 bytes (410 MB, 391 MiB) copied, 0.139487 s, 2.9 GB/s # NVME
Auto-mount on System Boot
sudo nano /etc/fstab ## myramdisk /tmp/ramdisk tmpfs defaults,size=1G,x-gvfs-show 0 0 ## To mount it immediately without reboot, run the following command. sudo mount -a
How to Run VirtualBox VM on RAM Disk. You should set the machine folder to the RAM disk directory (/tmp/ramdisk/).
kvm command
- help kvm. The kvm wrapper script is used to provide compatibility with old qemu-kvm package which has been merged into qemu as of version 1.3. The script executes qemu-system-x86_64 -enable-kvm passing all other command-line arguments to the qemu binary.
- How to Install Manjaro Linux in a QEMU/KVM Virtual Machine
snapshot
- How to Create, Revert and Delete KVM Virtual machine (domain) snapshot with virsh command, How to create snapshot in Linux KVM VM/Domain
$ sudo su # virsh list --all # virsh snapshot-create-as --domain win10 --name win10_snap --description "Windows 10 snapshot" # Create a snapshot # virsh snapshot-list win10 # virsh snapshot-info --domain win10 --snapshotname win10_snap # virsh snapshot-revert win10 win10_snap # Restore # virsh snapshot-delete --domain win10 --snapshotname win10_snap
- To create a snapshot using the GUI, open virtualization manager window. In the VM properties, click on the icon (two displays overlapped) which is on the right of the menu, which says Manage VM snapshots.
- To restore a snapshot using Virtual Machine Manager, right click on a snapshot and select "Start snapshot".
GUI
Create a CentOS homelab in an hour. Network Install.
Virtual Machine Manager/virt-manager, Cockpit, virt-viewer, remote-viewer
sudo apt install cockpit cockpit-machines
- Creating Ubuntu 20.04 KVM Virtual Machines using Cockpit and virt-manager*.
- It shows how to handle the error of cannot opening the ISO file.
- It shows how to use virt-viewer to connect to a VM (virt-viewer)
- remote-viewer will be used if we change the Console type from 'VNC' to 'Graphics Console in Desktop Viewer' and click the 'Launch Remote Viewer' button. A text file called 'download' will be downloaded. If we click it, it will launch remote-viewer. An interesting thing is the file will be automatically deleted once we click on it:)
- Consider running a VM on a remote machine with cockpit version 215.
- For Ubuntu 20.04.1 Desktop. If I increase the display resolution from 1024x768 (4:3) to others without keeping the same ratio, the mouse calibration has a problem:( Maybe it's better not to change the resolution. The VNC method works.
- For Windows 10, we need to unplug ethernet first in order to create a local account. I've not found a way to use virt-viewer or remote-viewer to view the desktop locally. I can still use cockpit web interface (Consoles) to use the Windows desktop. I am able to increase the display resolution to 1280x800 from 1024x768 which makes the interface easy for my eyes. One problem is the machine will freeze for some unknown reason.
- Manage KVM Virtual Machines Using Cockpit Web Console
- How To Manage KVM Virtual Machines With Virt-Manager. View -> Graph -> Memory Usage. However, it seems the reported memory is correct for CentOS, but not Ubuntu or Windows.
- Install KVM && Web Cockpit Console on SparkyLinux 2020.6
- Tip: it may help to use mouse to click the installation screen to refresh the screen if we don't see the progress bar to progress.
- Manage KVM Virtual Machines Using Cockpit Web Console
- Permission error in virtual machine manager. Solution 1: use sudo sudo virt-manager Solution 2: copy iso file to /tmp directory. Solution 3: change the owner of the iso file to libvirt-qemu:kvm .
GNOME Boxes
- https://help.gnome.org/users/gnome-boxes/stable/
- How to Install and Use GNOME Boxes to Create Virtual Machines
Network
- Host only (used by VirtualBox) = Isolated network (used by KVM). Step 1: Select a VM. Click Edit, choose "Connection Details" and add a new network (Isolated mode). This default network has IPv4 192.168.122.0/24 Step 2: Click Edit, choose "Virtual machine Details" and click the Information icon (circle shape). Click "Add Hardware" and choose "Isolated network" for the Network source. Leave the MAC address and Device model as defaults. This network network has IP4 192.168.100.0/24.
- How to Configure Network Bridge in Ubuntu*. Three methods. Excellent instruction!
- The new (slave-type) bridge network should tie to one of existing network adapters for this to work.
- The new adapter will be br0 only if we use the netplan method.
- If we use the nmcli or the nm-connection-editor GUI method, we need to deactivate the Ethernet interface (if we use Ethernet). The final adapters will have br0 and bridge-br0.
- To set a static IP on the host, see Ubuntu -> Set static IP, How to Configure Static IP Address on Ubuntu 20.04.
- How to configure bridged network in virt-manager (CentOS / RHEL 7)
- Install And Set Up KVM On Ubuntu 20.04 Focal Fossa Linux
- KVM: Creating a bridged network with NetPlan on Ubuntu bionic
- VirtualNetworking
- Network XML format
- KVM Networking - NAT & Host-Only
- Home Lab Project: Network Bridges for KVM - NAT, Host-only, Isolated (I got the same error as one reported)
Find out the IP of the guest machine
# Method 1: $ virsh list $ virsh domifaddr centos8 # Method 2: $ sudo apt install net-tools # Debian $ sudo dnf install net-tools # CentOS $ arp -n # find the rows with virbr0 adapter # Method 3: # KVM -> Info -> NIC
Back and restore images
How to KVM Backup and Restore in Linux
# Backup virsh list --all virsh dumpxml win10 > /MyBackup/win10.xml sudo cp /var/lib/libvirt/images/win10.qcow2 /MyBackup/win10.qcow2 sudo chown $USER:$USER /MyBackup/*.qcow2 # Restore Making sudo virsh undefine win10 ls -l /etc/libvirt/qemu virsh list --all sudo rm /var/lib/libvirt/images/win10.qcow2 sudo cp /MyBackup/win10.qcow2 /var/lib/libvirt/images/ sudo virsh define --file /MyBackup/win10.xml # virsh start win10
Cheat sheet
Vagrant
- https://www.vagrantup.com/
- How To Use Vagrant With Libvirt KVM Provider
- Learning vagrant (Lynda.com)
- Managing KVM virtual machines part I – Vagrant and libvirt (it has a nice diagram to show a big picture of things related to libvirt),
- How To Use Vagrant with Libvirt on Linux
How to Use Virtualbox VMs on KVM In Linux
How to Use Virtualbox VMs on KVM In Linux
Run macOS on KVM / QEMU
- How To run macOS on KVM / QEMU
- Install macOS Big Sur Or Catalina In A Virtual Machine Using Docker-OSX
- How To Install macOS in a Virtual Machine on Ubuntu Linux
QuickEMU
- QuickEMU - Quickly create and run optimised Windows, macOS and Linux desktop virtual machines
- How To Run Windows 11 and macOS Virtual Machines in Linux
- Quickly Create And Run Optimized Linux, macOS And Windows Virtual Machines With Quickemu (With Auto ISO Download)
I have a problem with the installation. Maybe caused by my existing QEMU version. Maybe better to test on Fedora or Arch Linux.
$ date # ubuntu 20.04 Fri 10 Dec 2021 11:02:24 AM EST $ quickget windows 11 ... To start your windows 11 virtual machine run: quickemu --vm windows-11.conf $ quickemu --vm windows-11.conf ERROR! Qemu 6.0.0 or newer is required, detected 4.2.1. $ sudo apt-get install qemu-system-x86 # Setting up qemu-system-x86 (1:6.0+dfsg-2expubuntu1~focal1.0) ...
Some lessons:
- Use Ctrl+Alt+g to release the mouse.
- Change boot="legacy" in windows-11.conf helps to show the Windows installation process (See Simple shell script to "manage" Qemu virtual machines.). But it still failed Windows could not create a partition on disk 0. The error occurred while applying the unattend answer file's <DiskConfiguration> setting. Error code: 0x80042565.
- When I tried to install quickemu on another freshly installed xubuntu 20.04 focal fossa (Core2 Quad CPU), the installation went through without any problem.
- I can install macOS Catalina, Windows 11, etc. The trick to install macOS is manually reboot the guest machine a couple times. Also it is useful to play the "Options -> Scale display" option in "spicy" to better see the text when we install macOS. Then it works.
- I need to use Shift + F12 to release mouse (or exit full display) in spicy (the GUI where the guest machine is running) on my xUbuntu. For each time we boot macOS, I can use keyboard (mouse does not work) to select the hdd for booting macOS (or just wait a few seconds & the 1st drive will be used to boot).
- macOS Catalina asks at least 4 characters for the password.
- For macOS Catalina guest machine, it allocates 2 CPUs, 4GB RAM, 100GB storage by default. After installing macOS, 75GB will still be available.
- For Windows 11 guest machine, it allocates 2 CPUs, 4GB RAM, 64GB storage with 43GB free.
- For Lubuntu guest machine (much better speed than macOS/Windows VM, fill full screen, Qt/KDE based), it allocates 2 CPUs, 4GB RAM, 16GB storage with 8.4GB free.
Build QEMU from source