Jump to content

KVM: Difference between revisions

From 太極
← Older editNewer edit →
Brb (talk | contribs)
42,905 edits
Brb (talk | contribs)
42,905 edits
Line 529: Line 529:
* [https://www.debugpoint.com/share-folder-virt-manager/ Share Folder Between Guest and Host in virt-manager (KVM/Qemu/libvirt)]
* [https://www.debugpoint.com/share-folder-virt-manager/ Share Folder Between Guest and Host in virt-manager (KVM/Qemu/libvirt)]
* [https://www.debugpoint.com/kvm-share-folder-windows-guest/ Share Folder Between Windows Guest and Linux Host in KVM using virtiofs]
* [https://www.debugpoint.com/kvm-share-folder-windows-guest/ Share Folder Between Windows Guest and Linux Host in KVM using virtiofs]
=== remote-viewer/SPICE client ===
Example:
<syntaxhighlight lang='sh'>
$ remote-viewer spice://192.168.1.XXX:5900
</syntaxhighlight>


== How to quit full screen ==
== How to quit full screen ==

Revision as of 18:19, 14 September 2025

Resource

Big picture

  • KVM is the acceleration technology in the kernel.
  • QEMU is the software that runs the VMs, using KVM for speed.
  • libvirt is the management layer that abstracts away the complexity of KVM/QEMU.
  • virsh is the command-line tool for libvirt.
  • virt-manager is the GUI tool for libvirt
+-------------------+      +-------------------+
|   Physical Host   |      |     Guest OS      |
|    (Your PC)      |      |   (e.g., Windows/Linux) |
|                   |      |                   |
| +-----------------+-----+ |  +---------------+--+
| |     Hardware    |     | |  | VM Applications  |
| |   (CPU, RAM,    |     | |  |   (Browser, IDE) |
| |    Disks, NIC)  |     | |  +------------------+
| +-------+---------+     | |
|         |               | |
|         |               | |
| +-------v---------+     | |
| |   Linux Kernel  |     | |
| | (incl. KVM module)|<----+------> Disk (Virtual)
| +-------+---------+     | |  +---------------+--+
|         ^               | |  |    Guest OS     |
|         |  (Hardware   | |  |   Kernel/Drivers|
|         |   Acceleration)| |  +------------------+
|         |               | |
| +-------+---------+     | |
| |     QEMU        |     | |
| | (User-space VM  |     | |
| |   Emulator)     |     | |
| +-------^---------+     | |
|         |               | |
|         | (Manages VMs, | |
|         |  Networks, etc.)| |
| +-------+---------+     | |
| |   libvirtd      |     | |
| |   (Daemon)      |     | |
| +-------^---------+     | |
|         |               | |
|         | (API/Calls)   | |
| +-------+---------+     | |
| |     virsh       |     | |
| |   (CLI Tool)    |     | |
| +-----------------+-----+ |
|         ^               | |
|         |               | |
| +-------+---------+     | |
| | virt-manager    |     | |
| |   (GUI Tool)    |     | |
| +-----------------+-----+ |
|                   |       |
+-------------------+       +-------------------+

QEMU

How to Use QEMU to Boot Another Operating System

QEMU vs. KVM

QEMU vs. KVM: Which Should You Use for Virtualization? QEMU uses full system emulation, allowing it to emulate a complete computer system, including the processor, memory, storage, and peripheral devices. This flexibility makes QEMU an excellent choice for cross-platform virtualization since it supports a wide range of architectures.

libvirt

  • https://en.wikipedia.org/wiki/Libvirt. libvirt is an open-source API, daemon and management tool for managing platform virtualization.[3] It can be used to manage KVM, Xen, VMware ESXi, QEMU and other virtualization technologies.
  • https://libvirt.org/
  • Check libvirt version: sudo head -n2 /var/log/libvirt/qemu/ubuntu2004.log # on Ubuntu 20.04 it shows 6.0.0

"virt-install" to install or delete a VM

$ virt-install --help | grep graphics
$ osinfo-query os 

$ sudo virt-install  -n ubuntu2004 --os-type=Linux  \
  --os-variant=ubuntu18.04  --ram=2096  --vcpus=2  \
  --disk size=20  \
  --network network=br0 \
  --graphics vnc,listen=0.0.0.0  \
  --location Downloads/ubuntu-20.04.1-desktop-amd64.iso

$ sudo virt-install --name=centos7server \
  --os-type=Linux \
  --os-variant=centos7.0 \
  --vcpu=2 --ram=2048 \
  --disk size=20 \
  --graphics vnc \
  --location=/tmp/CentOS-7-x86_64-DVD-2009.iso

# Running locally
$ sudo virt-install  -n ubuntu20043 --os-type=Linux  \
  --os-variant=ubuntu20.04  \
  --ram=4096  --vcpus=2  \
  --disk size=20  \
  --network network=default  \
  --cdrom ~/Downloads/ubuntu-20.04.3-desktop-amd64.iso

NB

  • If we just run a live session and power off the VM, it will automatically reject the installation medium (iso file) and reboot. So eventually it showed a message "No bootable device."
  • Using the Virtual Machine Manager GUI I found after I tried to delete a VM it won't actually delete the image file created in /var/lib/libvirt/images. Be careful when deleting files there. See How to delete KVM VM guest using virsh command & Removing and deleting a VM from RedHat. 
    $ virsh list --all
 Id   Name                State
------------------------------------
 39   ubuntu20043         running

$ virsh dumpxml --domain ubuntu20043 | grep 'source file' 
      <source file='/var/lib/libvirt/images/ubuntu20043.qcow2' index='2'/>

$ sudo virsh destroy ubuntu20043  # SHUT DOWN

# "sudo virsh undefine ubuntu20043" WILL LEAVE qcow2

$ sudo virsh undefine ubuntu20043 --remove-all-storage 
Domain 'ubuntu20043' has been undefined
Volume 'vda'(/var/lib/libvirt/images/ubuntu20043.qcow2) removed.

$ sudo ls -lt /var/lib/libvirt/images/

Check if running in VM

9 things to do in your first 10 minutes on a Linux server 

$ cat /sys/class/dmi/id/sys_vendor 
QEMU

$ sudo dmidecode -s system-manufacturer 
QEMU

On VirtualBox + CentOS vm, it returns "innotek GmbH".

On VMware + CentOS vm, it returns "VMware, Inc".

Guest additions

It seems QEUM/KVM does not have that to download.

Bridge network

  • There are different ways to configure the network in Ubuntu.
    • If we are using a Ubuntu server, we will use the Netplan method.
    • If we are using a Ubuntu desktop, we will use the Network manager command line nmcli method or the GUI tool nm-connection-editor.
  • Use nmcli to add a bridge network and a bridge slave interface 
    sudo nmcli con add ifname br0 type bridge con-name br0
sudo nmcli con add ifname eno1 type bridge-slave master br0
  • Use nmcli to bring down a wired network and bring up a bridge network 
    sudo nmcli con down "Wired connection 1"
sudo nmcli con up br0
  • Use nmcli to delete unused bridge networks 
    sudo nmcli device show
sudo nmcli con show
sudo nmcli con del "br0"

Install

Install on Ubuntu

Where are image files saved

Install on Arch/Manjaro

VNC

  • KVM/QEMU 以 virt-install 指令建立虛擬機器、VNC 顯示畫面教學 
    sudo virt-install --virt-type kvm --name ubuntu1804 --ram 2048 
  --disk /home/seal/kvm/ubuntu1804.qcow2,format=qcow2 
  --network network=default 
  --graphics vnc,listen=0.0.0.0,password=YOUR_PASSWORD 
  --noautoconsole 
  --os-type=linux --os-variant=ubuntu17.10 
  --cdrom=/home/seal/Downloads/ubuntu-18.10-desktop-amd64.iso

virsh vncdisplay ubuntu1804 #  127.0.0.1:1
                            # pay attention to the port number
                            # :0  --> 5900 in ssh
                            # :1  --> 5901 in ssh
sudo ufw disable

# Another machine. 
# Suppose the host machine has an IP 192.168.0.3
ssh -NL 5901:localhost:5901 [email protected]
ssh -NL 5901:127.0.0.1:5901 [email protected]

vncviewer 127.0.0.1:1 # from a different machine

    NOTE: I have to use local port forwarding. If I try to directly connect to the VM in RealVNC, it will show The connection was refused by the computer. A limit by KVM?

The question is how do we know the client's IP? I guess cockpit method is better.

Install qemu-system-arm in Ubuntu

  • sudo apt install qemu-system-arm
  • How to emulate Raspberry Pi in QEMU which is based on qemu-rpi-kernel-Qemu kernel for emulating Rpi on QEMU. 
    mkdir rpi-vm && cd rpi-vm
git clone https://github.com/dhruvvyas90/qemu-rpi-kernel.git

# Download Raspbian Buster Lite
# from https://www.raspberrypi.org/downloads/raspberry-pi-os/
dd if=/dev/zero bs=1M count=4096 >> 2020-08-20-raspios-buster-armhf-lite.img

sudo losetup -f -P --show 2020-08-20-raspios-buster-armhf-lite.img

sudo mount -o rw /dev/loopXp2 /mnt

sudo nano /mnt/etc/fstab

sudo nano /etc/ld.so.preload # Comment out the line and save the file. 
# Then unmount with the following commands:
# sudo umount /mnt
# sudo losetup -d /dev/loopX

qemu-system-arm \
  -M versatilepb \
  -cpu arm1176 \
  -m 256 \
  -kernel qemu-rpi-kernel/kernel-qemu-4.19.50-buster \
  -hda 2020-08-20-raspios-buster-armhf-lite.img \
  -append "dwc_otg.lpm_enable=0 root=/dev/sda2 console=tty1 rootfstype=ext4 elevator=deadline rootwait" \
  -dtb qemu-rpi-kernel/versatile-pb-buster.dtb \
  -no-reboot \
  -serial stdio

# We can install a desktop. See 
# https://www.raspberrypi.org/forums/viewtopic.php?t=133691

Install on CentOS

qemu-system-x86_64

qemu-system-x86_64 -name debian \
  -vnc 146.20.141.254:0 -m 1024 \
  -drive format=raw,index=2,file=debian.img \
  -enable-kvm -daemonize
If we use Virtual Machine Manager program, you need to have KVM (lsmod | grep kvm) installed on your system for this to work.

Install using qcow2 image

Running a virtual machine from a ramdisk

  • List of RAM drive software
  • How to Create a RAM Drive in Linux 
    if [ "$(mount | grep -o "/mnt/ram")" != "/mnt/ram" ]; then
  sudo mkdir -p /mnt/ram
  sudo mount -t ramfs -o size=1g ramfs /mnt/ram
  sudo chown -R $(whoami):$(whoami) /mnt/ram
fi
mount | grep ram

sudo umount /mnt/ram
  • https://youtu.be/fOaWrhLoVK0
  • Libvirt In-Memory Storage
  • (Windows only) USE A RAM DISK TO GET EXTREME PERFORMANCE FROM TEMPORARY VIRTUAL MACHINES
  • How to Easily Create RAM Disk on Debian, Ubuntu, Linux Mint, CentOS. How to Create a RAM Disk in Any Linux Distro. 
    sudo mkdir /tmp/ramdisk
sudo chmod 777 /tmp/ramdisk
htop  # check free ram
sudo mount -t tmpfs -o size=1G myramdisk /tmp/ramdisk
            # myramdisk is the device name I gave to it
mount | tail -n 1
sudo umount /tmp/ramdisk/

    Test RAM speed 

    # test write speed
sudo dd if=/dev/zero of=/tmp/ramdisk/zero bs=4k count=100000
## 409600000 bytes (410 MB, 391 MiB) copied, 0.734813 s, 557 MB/s  # SATA SSD
## 409600000 bytes (410 MB, 391 MiB) copied, 0.18296 s, 2.2 GB/s   # NVME

# test read speed
sudo dd if=/tmp/ramdisk/zero of=/dev/null bs=4k count=100000
## 409600000 bytes (410 MB, 391 MiB) copied, 0.410809 s, 997 MB/s  # SATA SSD
## 409600000 bytes (410 MB, 391 MiB) copied, 0.139487 s, 2.9 GB/s  # NVME

    Auto-mount on System Boot 

    sudo nano /etc/fstab
## myramdisk  /tmp/ramdisk  tmpfs  defaults,size=1G,x-gvfs-show  0  0

## To mount it immediately without reboot, run the following command.
sudo mount -a

    How to Run VirtualBox VM on RAM Disk. You should set the machine folder to the RAM disk directory (/tmp/ramdisk/).

kvm command

snapshot

  • How to Create, Revert and Delete KVM Virtual machine (domain) snapshot with virsh command, How to create snapshot in Linux KVM VM/Domain 
    $ sudo su
# virsh list --all

# virsh snapshot-create-as --domain win10 --name win10_snap --description "Windows 10 snapshot"    # Create a snapshot
# virsh snapshot-list win10
# virsh snapshot-info --domain win10 --snapshotname win10_snap

# virsh snapshot-revert win10 win10_snap    # Restore

# virsh snapshot-delete --domain win10 --snapshotname win10_snap
  • To create a snapshot using the GUI, open virtualization manager window. In the VM properties, click on the icon (two displays overlapped) which is on the right of the menu, which says Manage VM snapshots.
  • To restore a snapshot using Virtual Machine Manager, right click on a snapshot and select "Start snapshot".

GUI

Virtual Machine Manager/virt-manager, Cockpit, virt-viewer, remote-viewer

sudo apt install cockpit cockpit-machines
  • Creating Ubuntu 20.04 KVM Virtual Machines using Cockpit and virt-manager*.
    • It shows how to handle the error of cannot opening the ISO file.
    • It shows how to use virt-viewer to connect to a VM (virt-viewer)
    • remote-viewer will be used if we change the Console type from 'VNC' to 'Graphics Console in Desktop Viewer' and click the 'Launch Remote Viewer' button. A text file called 'download' will be downloaded. If we click it, it will launch remote-viewer. An interesting thing is the file will be automatically deleted once we click on it:)
  • Consider running a VM on a remote machine with cockpit version 215.
    • For Ubuntu 20.04.1 Desktop. If I increase the display resolution from 1024x768 (4:3) to others without keeping the same ratio, the mouse calibration has a problem:( Maybe it's better not to change the resolution. The VNC method works.
    • For Windows 10, we need to unplug ethernet first in order to create a local account. I've not found a way to use virt-viewer or remote-viewer to view the desktop locally. I can still use cockpit web interface (Consoles) to use the Windows desktop. I am able to increase the display resolution to 1280x800 from 1024x768 which makes the interface easy for my eyes. One problem is the machine will freeze for some unknown reason.

Network Install in virtual machine manager

Share folders

remote-viewer/SPICE client

Example: 

$ remote-viewer spice://192.168.1.XXX:5900

How to quit full screen

How to exit full screen mode in qemu virt-manager ( RHEL / CentOS 7). Hint: click the first of two icons shown at the top-center screen.

Windows VM

USB host device

  • USB Host Device Assigned to Guest
  • Rufus + USB Passthrough in Virt-manager
    • Create the Bootable USB with Rufus. Making sure to select the options to "Remove requirement for 4GB+ RAM, Secure Boot and TPM 2.0" and "Remove requirement for an online Microsoft account."
    • Plug the USB Drive into your Host Machine
    • Add the USB Drive to your VM in Virt-Manager. Add Hardware -> USB Host Device. Select your USB flash drive from this list.
    • Configure Boot Order by prioritizing the USB device.

Windows VM by docker

Running Windows in a Docker Container!

GNOME Boxes

Network

Find out the IP of the guest machine

# Method 1:
$ virsh list
$ virsh domifaddr centos8

# Method 2:
$ sudo apt install net-tools   # Debian
$ sudo dnf install net-tools   # CentOS
$ arp -n               # find the rows with virbr0 adapter

# Method 3:
# KVM -> Info -> NIC

Backup and restore images

How to KVM Backup and Restore in Linux

  • /etc/libvirt/qemu # location of xml files
  • /var/lib/libvirt/images # location of qcow2 files
# Backup 
sudo virsh list --all
sudo virsh dumpxml win10 > /MyBackup/win10.xml
sudo cp /var/lib/libvirt/images/win10.qcow2 /MyBackup/win10.qcow2
sudo chown $USER:$USER /MyBackup/*.qcow2

# Restore Making
sudo virsh undefine win10 # delete xml (VM configuration) file 
sudo virsh list --all
sudo rm /var/lib/libvirt/images/win10.qcow2 # delete qcow2 image file

sudo cp /MyBackup/win10.qcow2 /var/lib/libvirt/images/
sudo virsh define --file /MyBackup/win10.xml
# virsh start win10

Free pre-built images

  • Notes
    1. We only need the image filew (like qcow2). We don't need xml files since virt-manager can recreate it.
    2. .img is a generic disk image extension: The .img extension simply indicates that the file is a raw, sector-by-sector copy of a disk. It's a very generic term that doesn't necessarily imply a specific format like QCOW2 or RAW.
    3. The actual format is QCOW2: Despite the .img extension, the Ubuntu cloud images that are intended for QEMU/KVM (the *-cloudimg-amd64.img files) are indeed internally formatted as QCOW2 (QEMU Copy On Write 2).
    4. You can verify this yourself using the qemu-img command-line tool, which is part of the QEMU package on your Linux host: qemu-img info noble-server-cloudimg-amd64.img
    5. There is no default username/password. They are designed to be provisioned using SSH keys and a service called cloud-init. Automating Virtual Machine Installation Using libvirt, virsh And cloud-init, Quickly create guest VMs using virsh, cloud image files, and cloud-init.

Cheat sheet

KVM Cheatsheet

Vagrant

How to Use Virtualbox VMs on KVM In Linux

How to Use Virtualbox VMs on KVM In Linux

Run macOS on KVM / QEMU

Run SteamOS on QEMU

Running the Steam Deck’s OS in a virtual machine using QEMU

QuickEMU

Some lessons: 

quickget -h  # list of all OSs
quickget ubuntu 24.04
quickemu --vm ubuntu-24.04.conf --display spice
du -sh ubuntu-24.04 # 6.0G
# Install the SPICE agent (spice-vdagent) in the guest to enable copy/paste and USB redirection
sudo apt install spice-vdagent
ps aux | grep spice-vdagent
sudo systemctl start spice-vdagent
sudo systemctl enable spice-vdagent

# Install the SPICE WebDAV agent (spice-webdavd) in the guest to enable file sharing.
sudo apt install spice-webdavd
  • By default new folders are created when we create/download guest machine in the home folder. We can create a new folder for managing these guest machines.
  • Use Ctrl+Alt+g to release the mouse. The hint is at the top of window.
  • Change boot="legacy" in windows-11.conf helps to show the Windows installation process (See Simple shell script to "manage" Qemu virtual machines.). But it still failed Windows could not create a partition on disk 0. The error occurred while applying the unattend answer file's <DiskConfiguration> setting. Error code: 0x80042565.
  • When I tried to install quickemu on another freshly installed xubuntu 20.04 focal fossa (Core2 Quad CPU), the installation went through without any problem.
  • The app running the guest machine is spicy.
  • I can install macOS Catalina, Windows 11, etc. The trick to install macOS is manually reboot the guest machine a couple times. Also it is useful to play the "Options -> Scale display" option in "spicy" to better see the text when we install macOS. Then it works.
  • I need to use Shift + F12 (see the hint at the bottom of spicy window) to release mouse (or exit full display) in spicy (the GUI where the guest machine is running) on my Ubuntu. When installing macOS, I need to use my keyboard (mouse does not work) to select the hdd for booting macOS (or just wait a few seconds & the 1st drive will be used to boot).
  • It helps to drag and enlarge the guest machine window (by default 'Scale display' option is checked).
  • When installing macOS, the screen says "About X minutes remaining". But it seems it stuck there (the minutes is not changed) though the CPU is busy. If I closed the window, shutdown the guest machine and reboot it, installation continues. Doing this a few times and eventually a new window ("Select Your Country or Region") will come up. In order to monitor the download, I open another terminal and run something like watch du -s ~/macos-big-sur. This will help me to decide when to restart the VM. For big-sur, the folder size stops at 26655436 and after I reboot the VM, the folder size keep increasing and stops at 32565600. For the 3rd reboot, it still downloads something and needs 2 internal reboots. So I think the problem is the data needs to be downloaded from Apple server is TOO large.
  • macos-big-sur & macos-monterey took about 32G & 34G space.
  • Every time I boot macOS, I shall use the right arrow to select the 2nd icon and press the Enter button to start.
  • macOS Catalina asks at least 4 characters for the password (I choose 6)
  • For macOS Catalina/Monterey guest machine, it automatically allocates 2 CPUs(?), 4/8GB RAM (depending on the host total RAM?), 100GB storage by default. After installing macOS, 75GB will still be available.
  • (Maybe related to Ubuntu 22.04 host using wayland). Whenever I start macOS vm, it pops up a window "Allow inhibiting shortcuts. The application spicy wants to inhibit shortcuts. You can restore shortcuts by pressing Super+Escape." I choose the default "Allow" and it seems to be OK. PS: I need to do this action quickly or the macOS window will go to black and not responds to my keyboard/mouse.
  • Windows 11 bug and solution
  • For Windows 11 guest machine, it allocates 2 CPUs, 4/8GB RAM, 64GB storage with 43GB free.
  • For Lubuntu guest machine (much better speed than macOS/Windows VM, fill full screen, Qt/KDE based), it allocates 2 CPUs, 4GB RAM, 16GB storage with 8.4GB free.

Build QEMU from source

quickget

quickget os release edition 

quickget -h
quickget fedora 42 COSMIC

quickemu with own iso

Consider the example of Diet Pi

  • Download UEFI x86 dietpi iso file and move it to a folder "DietPi".
  • Create a file dietpi.conf 
    guest_os="linux" 
iso="DietPi/DietPi_NativePC-UEFI-x86_64-Bookworm_Installer.iso" # Adjust filename
disk_img="DietPi/dietpi_vm.qcow2"
disk_size="10G"
ram="2G" # Adjust as needed
cpu_cores="2" # Adjust as needed
# You won't typically need an 'iso=' line if you're booting from a pre-made image
# Add other Quickemu options as desired (e.g., network, port_forwards)
  • Run 
    quickemu --vm dietpi.conf

    During the installation, you'll see the Clonezilla interface, which helps flash the DietPi system to your virtual disk. Ensure you select the correct target virtual disk (e.g., /dev/vda or similar, usually the only one presented apart from the ISO).

    After diet pi shut down, run 'quickemu --vm dietpi.conf' again.

    The installation took a while and required user interaction.

    The default username/password is root/dietpi.

ssh to VM

  • Suppose my VM is Ubuntu 24.04
sudo apt install openssh-server
sudo systemctl start ssh
sudo systemctl enable ssh
  • On my host, it shows ssh port 22221. It means this port on your Manjaro host is forwarded to port 22 (the standard SSH port) on your Ubuntu VM.
ls ubuntu-24.04
cat ubuntu-24.04/ubuntu-24.04.ports
  • On my host
ssh -p 22221 username@localhost
  • To ssh from VM to host
ssh [email protected]

Adjust VM storage

If I just want to delete the old VM and start a new VM, I can just run 

quickemu --vm ubuntu-24.04.conf --delete-disk
nano ubuntu-24.04.conf # Add a line disk_size="50G"
quickemu --vm ubuntu-24.04.conf

Adjust RAM

Add a line ram="4G" to the .conf file.

Sound

It works automatically (tested on MX Linux VM and Ubuntu VM).

DistroSea

Nested KVM

How to Enable Nested KVM Virtualization

Retrieved from "https://wiki.taichimd.us/index.php?title=KVM&oldid=42412"