Using QEMU with Olive to emulate Juniper Routers
From Internetworkpro
This guide is to help you run Olive (JunOS running on a standard PC not attached to a PFE) inside QEMU to create a virtual Juniper router lab. You should also be able to attach your QEMU processes to dynamips.
Other sites recommend you download a binary called JEMU.exe from a Chinese forum and use it to run your Olive on. First of all it's a Windows only binary. Windows is not the ideal environment for running QEMU (kqemu comes to mind.) Secondly there is no source provided! Like hell i am going to run a binary i got from a Chinese forum. So i went out and searched for an alternative.
You are asking yourself: 'Why don't I just run my Olive inside VMWare?'. Yes you can do this but multicast, is not supported. This means no OSPF, no LDP, no multicast, etc. This is ok if you are just trying out the JunOS cli for the first time but hardly good enough to do any real lab work on.
Contents |
[edit] Updates
Started from new svn trunk (20080910) and applied latest changes from upstream sources. This also gets rid of the PEMU nic cards which didn't work very well. The eepro100 nic cards work perfectly so nothing is needed.
[edit] Getting and Installing QEMU
Currently QEMU does not support multicast or dynamips compatible udp connections so we will need to patch and compile the source. The instructions below are for Debian/Ubuntu but should be adaptable to any linux distribution (or perhaps BSD?)
I have a patch from that gathers code from many sources but mostly pemu which uses the same nic and allows you to connect it to dynamips.
[edit] Dependencies
First we need to get your PC installed with the dependencies to run and compile QEMU.
Install build dependancies (your distribution may require you to manually do this. Consult the QEMU documentation for what is required to build QEMU)
sudo apt-get install build-essential sudo apt-get build-dep qemu sudo apt-get install libpcap-dev
Also, on Ubuntu 9.04 you should install additional packages:
sudo apt-get install gcc-3.4 sudo apt-get install subversion
Install kemu module and package (We need 1.4.0pre1 for trunk qemu)
wget http://www.nongnu.org/qemu/kqemu-1.4.0pre1.tar.gz tar zxfv kqemu-1.4.0pre1.tar.gz cd kqemu-1.4.0pre1 ./configure make sudo make install
Next we need to load the module into the running configuration
sudo modprobe kqemu
Fetch the QEMU code from SVN into a temporary directory
mkdir ~/src cd ~/src svn co svn://svn.savannah.nongnu.org/qemu/trunk -r 5193
Fetch and install the olive patch. This patch enables multicast on the emulated intel chipset and also turns on udp network connections for connecting to dynamips (can also be used for interconnecting QEMU proccesses at well).
Apparently, this link isn't valid anymore...
http://www.internetworkpro.org/nemith/qemu-olive/qemu-cvs20080910-brb_01-olive.patch
Or you can try to get patch from there:
http://shef.net.ru/misc/qemu-cvs20080910-brb_01-olive.patch
cd trunk wget http://www.internetworkpro.org/nemith/qemu-olive/qemu-cvs20080910-brb_01-olive.patch patch -p1 < qemu-cvs20080910-brb_01-olive.patch
[edit] Compiling and Installing
Now we have our build dependencies out of the way and we have a patched source tree we to compile it an install it. If you are familiar with any GNU package this will all be similar.
First run configure to and set a prefix (default is /usr/local/) and turn on options provided in the patch
./configure --prefix=/usr/local/ \
--target-list=i386-softmmu \
After the configure script completes successfully Just issue make and make install.
make make install
Now your patched QEMU is in the prefix you specified above.
[edit] Compiling and Installing under FreeBSD
As root go to the qemu-devel port's directory.
cd /usr/ports/emulators/qemu-devel
Download the FreeBSD's adopted patch (big thanx to nox!). OUTDATED!
wget or fetch http://people.freebsd.org/~nox/qemu/qemu-devel-jemu-20080302.patch
Then:
make fetch make extract patch < ./qemu-devel-jemu-20080302.patch make config /* DO NOT forget JEMU option! */ make install clean
N.B. QEMU depends on aio kld-module!
[edit] Installing FreeBSD as a guest
JunOS is based on FreeBSD and installing FreeBSD is required when setting up an Olive box.
Download FreeBSD 4.11 mini iso from FreeBSD ftp site
Create a new qcow2 disk image to use for our olive hard drive
qemu-img create olive-base.img -f qcow2 4G
This creats a new image 4 gig in size which seems to be enough to install and run olive just fine
Now you need to start our modified QEMU and boot from the ISO you downloaded. We'll attach a NIC now and tell it to use the internal natted 'user' mode. This will allow us to scp the jinstall image later on.
qemu -m 256 -hda olive-base.img -cdrom 4.11-RELEASE-i386-miniinst.iso -boot d \ -curses -localtime -net nic,macaddr=00:aa:00:00:01:01,model=i82559er -net user
You will now see qemu come up in curses mode which doesn't require X.
Skip the kernel configuration and choose the standard installation.
When you get to partitioning, allocate first the whole disk to BSD, and after that create slices like this:
ad0s1a / 512M ad0s1b swap 1024M ad0s1e /config 12M ad0s1f /var rest
Choose 'User' installation type and skip installing ports.
After the base is installed it will ask you if you want to use DHCP to configure your NIC. Do this now to allow us to scp your jinstall after we reboot.
Except for the DHCP, choose "no" for everything else (Linux compatibility, NFS, FTP, Inetd etc..)
After the installer completes it will reboot. Right after it reboots close or kill your QEMU process or it will boot into the installer cd again.
[edit] Installing Olive
Now we are ready to install Olive into our QEMU host. You will need to obtain a jinstall version. Use something less than 8.5 to get it to properly work. (8.5 is based on FreeBSD 6 and not FreeBSD 4.11).
Now we need to boot our guest again but this time boot from the image. Again we will attach one nic to the 'user' proccess.
qemu -m 256 -hda olive-base.img -boot c -localtime -curses \ -net nic,macaddr=00:aa:00:00:01:01,model=i82559er -net user
After FreeBSD fully boots login as root. Issue a netstat -r | grep default to see what our default gateway (our host machine is). You need SSH running on your host machine for this to work.
After you find out the default route ip address go ahead and scp the image onto your guest.
scp bbennett@10.0.2.2:~/Desktop/jinstall-8.3R1.5-domestic-signed.tgz /var/tmp
The following is a step that was missing from this guide, if the way I did it is inefficient to you, feel free to edit :)
[edit] Modify jinstall file
JunOS image after 7.4 version has a binary called checkpic. This binary will fail and the image cannot be installed. Replacing this binary with /usr/bin/true fixes the issue.
cd /var/tmp mkdir jinst-signed cd jinst-signed tar zxvf ../jinstall-8.3R1.5-domestic-signed.tgz mkdir jinst cd jinst tar zxvf ../jinstall-8.3R1.5-domestic.tgz mkdir pkgtools cd pkgtools tar zxvf ../pkgtools.tgz cd bin cp /usr/bin/true ./checkpic cd .. tar zcvf ../pkgtools.tgz * cd .. rm -rf pkgtools tar zcfv /var/tmp/jinstall-8.3R1.5-domestic-olive.tgz *
You might need to recalculate MD5 and SHA1 checksums before archiving them back. I have had to do it when i was fixing this version: jinstall-9.0R1.10-domestic-signed.tgz I did it on same freebsd install before installing jinstall. This can be done like this:
md5 -q jinstall-9.0R1.10-domestic-signed.tgz > jinstall-9.0R1.10-domestic-signed.tgz.md5 openssh sha1 jinstall-9.0R1.10-domestic-signed.tgz > jinstall-9.0R1.10-domestic-signed.tgz.sha1
just edit *.sha1 file and remove file name and leaving only checksum inside.
[edit] Installing Olive
pkg_add -f /var/tmp/jinstall-8.3R1.5-domestic-olive.tgz
After this finishes issue the halt command and kill your QEMU session window.
[edit] First Boot
The jinstall above really just installed a bootstrap enviroment so you need to boot up the guest OS one more time to finish the installation (depending on your version). If you just restarted your guest above you will notice that you will get no output on your screen. This is because a real Juniper router has no VGA out and redirects everything to the serial port. No worries for us since QEMU will redirect the serial port to either stdio or a telnet port.
Start up our guest one more time this time getting rid of the gui and redirecting the console to a telnet port. Please leave the memory at 256 for this proccess. After this first boot you can get by with a lot less memory, but we need it now.
qemu -m 256 -hda olive-base.img -boot c -localtime -nographic \ -serial stdio
Now wait while the bootstrap process completes. The virtual olive will reboot itself automatically and nothing is needed. At the end of this process we will be sitting at a login prompt. Login as root and issue the halt command and kill your QEMU proccess.
Now you have a base olive hard drive image. QEMU allows you to use this as a base for other harddrive issues and only writing the changes to your 'slave' images saving on disk space!
[edit] Running your router
[edit] Creating a router instance
Now we will create a new hard drive image off of your base image above. Repeat for all your routers you want to emulate
qemu-img create -b olive-base.img -f qcow2 R1.img
Now you can start your router with the following
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,macaddr=00:aa:00:60:00:01,model=i82559er -net socket,listen=:6000
Telnet to localhost port 2001 to connect to your new router and start configuring!
[edit] Networking your routers
[edit] QEMU Olive to QEMU Olive
[edit] socket
Socket mode creates a tcp stream between two qemu instances with one a client and the other a server. More testing is needed, but sometimes it creates duplicate packets
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er -net socket,vlan=1,listen=:6000 qemu R2.img -m 96 -nographic -daemonize -serial telnet::2002,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er -net socket,vlan=1,connect=127.0.0.1:6000
[edit] udp
UDP mode is mainly used to connect to dynamips emulated routers but can also be used as a more reliable way to connect two QEMU olives together. UDP mode can result in much lower latency connections as well.
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er -net udp,vlan=1,sport=10000,dport=10001,daddr=127.0.0.1 qemu R2.img -m 96 -nographic -daemonize -serial telnet::2002,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:02,model=i82559er -net udp,vlan=1,sport=10001,dport=10000,daddr=127.0.0.1
[edit] QEMU Olive to Real port
[edit] tun/tap
One way to connect an interface on your Olive with a real Ethernet port is to use a bridge and the net=tap option. This requires you to have the generic TUN/TAP driver either built-in to your kernel, or available as a module. To check the availability of this module do the following:
% ls -la /dev/net/tun crw-rw-rw- 1 root root 10, 200 2007-12-13 17:05 /dev/net/tun
If you get no such file or directory, try doing a modprobe tun. It should then appear in the lsmod output.
Let's say you started the emulator with the following:
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er -net tap,vlan=1,script=no
Note the vlan=1 and -net tap options. This basically connects your Olive's fxp0 to a virtual tap interface (usually tap0) on your host system. Start up Qemu, and once the emulator is running, proceed to the next step. You'll need to start up the emulator as root, or change the ownership or permissions on /dev/net/tun.
We're going to need to now bridge the tap0 interface to another Ethernet interface on the host system. To do this, we'll utilize the bridge capabilities of Linux. First, get the bridge utilities:
% sudo apt-get install bridge-utils
Let's assume you want to connect fxp0 on the Olive to eth1 on the host system. Do the following as root:
brctl addbr br0 brctl addif br0 eth1 brctl addif br0 tap0
This creates a bridge device, br0, and binds the two interfaces to it. Right now everything is still down. Let's bring it up...
ifconfig eth1 up ifconfig tap0 up ifconfig br0 up
Now the bridge and member interfaces should be up. Don't assign any IP addresses to either of the member interfaces. If you want to, you can assign something to br0 if needed. (br0 is analogous an SVI in the Cisco world) To see the status of the bridge, do the following:
% brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000cf19ce06c no eth1
tap0
Now, assign an IP address to the fxp0 interface on your Olive, and it should be online. You are also free to tcpdump on the bridge or member interfaces, for debugging. All of this can probably be put into a QEMU interface script, so you can remove the script=no option, and make it a little more automatic.
[edit] pcap/lcap
In the latest patch provided on this website there is an option to use pcap or lcap libraries to bridge directly to a physical nic. To do this run
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,macaddr=00:aa:00:60:00:01,model=i82559er -net pcap,ifname=eth1
or for lcap
qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,macaddr=00:aa:00:60:00:01,model=i82559er -net lcap,ifname=eth1
Note: this will stop any other traffic on the physical nic
[edit] QEMU Olive to dynamips
The lastest patch also added a udp option to allow networking to a dynamips or pemu.
qemu Juniper1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \ -net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er -net udp,vlan=1,sport=10000,dport=10001,daddr=127.0.0.1
Then in your dynagen .net file
[[router Cisco1]] fa0/0=NIO_udp:10001:127.0.0.1:10000
