I learnt that Kamil Rytarowski has taken up a challenge of porting HAXM to NetBSD. I found this to be a great learning opportunity to learn about OS development. I decided to help him and check whether his demo of Windows 7 really works. Below are the steps I have taken to get it running.
This post is meant to be a guide on setting up HAXM accelerated QEMU on NetBSD 8.0. The
HAXM-capable qemu is currently provided in pkgsrc as
wip/qemu-haxm.
Apart from that, a kernel patch provided by Kamil
is necesary. [Not anymore. The FPUDNA fix has been integrated into HAXM.] Below are the steps needed to get all this running on your NetBSD/8.0 system.
Command lines prefixed by # are executed as root, $ as regular user
You can check that in bios in most cases, or look for vmx flag in cat /proc/cpuinfo. lscpu(1) on Linux will also list that
Install src xsrc syssrc sets. This can be done during initial system installation or later. In the latter case:
$ wget ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-8.0/source/sets/{src,xsrc,syssrc}.tgz
$ su
# for set in src syssrc xsrc; do tar -zxvf $set.tgz -C / ; done
Now patch the kernel source. You will need to patch one file, located in sys/arch/x86/x86/fpu.c
$ cd /usr/src
$ su
# curl http://www.netbsd.org/~kamil/patch-00083-fpu-dna-netbsd8.txt | patch -p0
Now, if that patch fails for some reason, just apply it manually, its a very small change. For the record, the patch contents are below
--- sys/arch/x86/x86/fpu.c10 Jul 2018 15:35:26 -00001.12.8.3
+++ sys/arch/x86/x86/fpu.c8 Feb 2019 15:01:40 -0000
@@ -440,9 +440,12 @@
struct pcb *pcb;
int s;
-if (!USERMODE(frame->tf_cs, frame->tf_eflags))
-panic("fpudna from kernel, ip %p, trapframe %p\n",
- (void *)X86_TF_RIP(frame), frame);
+if (!USERMODE(frame->tf_cs, frame->tf_eflags)){
+ //panic("fpudna from kernel, ip %p, trapframe %p\n",
+ // (void *)X86_TF_RIP(frame), frame);
+ clts();
+ return;
+}
if (i386_fpu_present == 0) {
send_sigill((void *)X86_TF_RIP(frame));
Now that you have patched the kernel, go ahead and build it.
# cd /usr/src
# ./build.sh -m amd64 tools
# ./build.sh -m amd64 kernel=GENERIC
# cp /netbsd /netbsd.old
# cp sys/arch/amd64/compile/obj/GENERIC/netbsd /
# reboot
Should this fail, you have your old kernel backed up, strike the 5 key at the booloader and boot that instead.
Install the Netwide Assembler (nasm) -- necesary to compile HAXM
# pkg_add nasm
Get HAXM sources and build it
$ git clone https://github.com/intel/haxm
$ cd platforms/netbsd
$ make
Load the HAXM module
$ cd haxm/platforms/netbsd
$ uname -a # check uname. kernel MUST be 8.0
$ su
# modload ./haxm.kmod
You can run dmesg to verify the module was loaded correctly:
Its necesary to use wip/qemu-haxm, so prepare pkgsrc-wip. There is another patch to be applied (for the time being. I asked for git access to pkgsrc-wip and I'll push it as soon as I get my account).
The following commands will install pkgsrc-wip on your system
# cd /usr/pkgsrc
# git clone --depth 1 git://wip.pkgsrc.org/pkgsrc-wip.git wip
Apply patch (this might not be necessary)
# cd qemu-haxm
# curl http://polprog.net/papiery/netbsd-haxm/pkgsrc-qemu-haxm.diff | patch
Build qemu-haxm with proper options. If you have previously installed qemu, use pkg_delete qemu to remove it
# make PKG_OPTIONS.qemu="sdl gtk3"
# make install
NOTICE: Major numbers have changed!
You need to recreate the device nodes. Delete the device nodes with;
# rm /dev/HAX
# rm -r /dev/hax_vm*
And re-run the script!
Use this script to create HAXM devices and VCPU nodes
It's contents are below
#!/bin/sh
# Makes NetBSD HAXM devices
if [ $(id -u) != 0 ]; then echo "You need to be root" 1>&2; exit 1; fi
echo "*** Loading HAXM kmod ***"
modload ./haxm.kmod
echo "*** Making HAX devices ***"
mknod /dev/HAX c 348 0
mkdir /dev/hax_vm
for i in `seq 0 7`; do mkdir /dev/hax_vm0$i; done
for i in `seq 0 7`; do mknod /dev/hax_vm/vm0$i c 349 $i; done
echo "*** Making VCPU Nodes ***"
i=0
while [ $i -lt 128 ]; do
vm=$(echo "$i/16"|bc)
vcpu=$(echo "$i%16"|bc)
echo "i=$i vm=$vm vcpu=$vcpu"
cmd=$(printf "mknod /dev/hax_vm%02d/vcpu%02d c 350 %d" $vm $vcpu $i)
$cmd
let "i+=1"
done
Windows will run only as 32 bit guest for now (due to HAXM not emulating CR8 (TPR) register that Windows uses. CR8 is a new control register added in amd64 used for task prioritization. Currently, a hack ignores CR8 changes in guest). Other amd64 operating systems should be OK. Testing is necessary!!! CR8 emulation should be in HAXM soon, since some people are actively working on it.
Grab a Win 7 x86 image from Microsoft. Select any x86 Win7 and VirtualBox (any platform should do, commands below are for VirtualBox image).
This test image works very well on my quad core, 3.3GHz i3 (!) with 8 GB of RAM test box. Windows is not laggy, mouse pointer is OK, everything works like it should!
Use the commands below to get a qcow2 image
$ unzip [zip file you downloaded]
$ tar -zvf [content].ova
(ignore tar error about corrupted archive, files are intact!)
$ qemu-img convert -f vmdk -O qcow2 [vmdk file] win7live.qcow2
OVA file is just a tar archive, so you have to untar it to get the vmdk file. You can remove the vmdk, ovf and zip after you make sure the image works to free up space
# qemu-system-i386 -hda win7live.qcow2 -netdev user,id=nd0 -device rtl8139,netdev=nd0 -accel hax
The above command line will launch the windows image with working network. Enjoy
Below there are screenshots of operating systems that boot successfully for now :)
FreeBSD 12.0
Minix 3
Plan 9
Haiku, ToaruOS and KolibriOS - thanks to nia @ !
Sytems that currently fail to run under HAXM:
Win 10 - hangs at the loading screen, run with -m 4G, othwerise errors out on ramdisk creation. 64-bit OS
OpenBSD 6.4 - #GP ? (see src/trap.c) - github issue
Solaris 10 - trap in bootloader - github issue - partially solved. Boots
ReactOS 0.4.6 - BSOD after selecting live mode (device detection dialog runs OK)
Icaros - thanks nia - github issue
Windows XP, Vista - hang host
Windows NT 4.0 - installer broken on normal QEMU
Windows 98 - Issues a VCPU shutdown request (?), emulation stops
QNX 6 - Reported to cause VCPU shutdown (nia)
We were enlightened by a different OS user, that some HAXM guest systems hanging the host kernel is a DoS attack vector :D