Configuring the network

There are a few different ways to allow a virtual machine access to the external network. The default virtual network configuration is usermode networking, which uses the SLIRP protocol and traffic is NATed through the host interface to the outside network. If you do not want to access network services on your virtual machine then you can skip this next step.

However to enable external hosts to directly access services on virtual machines a bridge needs to be configured. This allows the virtual interfaces to connect to the outside network through the physical interface, making them appear as normal hosts to the rest of the network.

Warning: Network bridging will not work when the physical network device (eg eth1, ath0) used for bridging is a wireless device (eg ipw3945), as most wireless device drivers do not support bridging!

Warning 2: Bridged networking does not work by default. Since the release of kernel 2.6.18 in Sep 2006, the CAP_NET_ADMIN capability is required to use TUN/TAP networking (bug #103010).

• QEMU does not need to be suid root. Since Intrepid it has been possible to give the qemu binaries this capability using the "setcap" tool. First install the Linux capabilities tools (sudo apt-get install libcap2-bin) and choose:

  • Safer method (can be done in Lucid and later). This allows specific users the ability to disrupt all networking on the system. Use cautiously.

    • Give qemu the inheritable CAP_NET_ADMIN capability:

      sudo setcap cap_net_admin=ei /usr/bin/qemu-system-*

    • Install the inheritable PAM capability module:

      sudo apt-get install libcap2-bin

    • Allow specific users to gain the inheritable CAP_NET_ADMIN capability by editing /etc/security/capability.conf:

      cap_net_admin        USER-NAME-HERE

  • Less safe method (must be used prior to Lucid). This allows all users the ability to potentially disrupt all networking on the system. Use with extreme caution.

    • Give qemu the forced CAP_NET_ADMIN capability:

      sudo setcap cap_net_admin=ep /usr/bin/qemu-system-*

• Note that the filesystem capabilities above will be lost on every qemu upgrade, since setting of the file system capability is not supported by Ubuntu packaging (see FilesystemCapabilities for details on the blockers). For a good overview of Linux capabilities and QEMU see this writeup.

Warning 3: A number of people are having problems with the network bridge losing connection with the client after large amounts of data transfer (eg. during rsync) For a Hardy or Intrepid host/client see below.

Warning 4: A network bridge configured as described here will not appear in virt-manager when using a remote management session, see bug #520386.

Simple file sharing

By default, if you enable the usermode networking using the '-net user' command-line option, the guest OS will get an IP address in the 10.0.2.0/24 address space and the host OS will be reachable at 10.0.2.2.

Thus, you should be able to ssh into the host OS (at 10.0.2.2) from inside the guest OS and thus use scp to copy files back and forth.

Creating a network bridge on the host

Install the bridge-utils package:

sudo apt-get install bridge-utils

We are going to change the network configuration¹. To do it properly, we should first stop networking²:

sudo invoke-rc.d networking stop

If you are on a remote connection, and so cannot stop networking, go ahead with the following commands, and use sudo invoke-rc.d networking restart at the end. If you made a mistake, it won't come back up, though.

To setup a bridge interface, edit /etc/network/interfaces and either comment or replace the existing config with (replace with the values for your network):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address 192.168.0.10
        network 192.168.0.0
        netmask 255.255.255.0
        broadcast 192.168.0.255
        gateway 192.168.0.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

or to use DHCP

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet dhcp
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

This will create a virtual interface br0.

Now restart networking:

sudo /etc/init.d/networking restart

Configuring ubuntu-vm-builder to create bridged guests by default

Virtual machines are defined in XML files; ubuntu-vm-builder, the tool we will use to create VMs, bases them on the template file /etc/vmbuilder/libvirt/libvirtxml.tmpl (before Ubuntu 8.10 /usr/share/ubuntu-vm-builder/templates/libvirt.tmpl)

Open that file, and change:

    <interface type='network'>
      <source network='default'/>
    </interface>

To:

    <interface type='bridge'>
      <source bridge='br0'/>
    </interface>

Generating a KVM MAC

If you are managing your guests via command line, the following script might be helpful to generate a randomized MAC. If you are getting an error for 'rl', install the package 'randomize-lines'.

echo -n "54:52:00"
for i in 1 2 3; do
        echo -n ":"
        for j in 1 2; do
                for k in 0 1 2 3 4 5 6 7 8 9 A B C D E F; do
                        echo $k
                done|rl|sed -n 1p
        done|while read m; do
                echo -n $m
        done
done
echo

Converting an existing guest

If you have already created VMs before, you can make them use bridged networking if you change the XML definition (in /etc/libvirt/qemu/) for the network interface, adjusting the mac address as desired from:

    <interface type='network'>
      <mac address='00:11:22:33:44:55'/>
      <source network='default'/>
    </interface>

to:

    <interface type='bridge'>
      <mac address='00:11:22:33:44:55'/>
      <source bridge='br0'/>
    </interface>

Note: Make sure the first octet in your MAC address is EVEN (eg. 00:) as MAC addresses with ODD first-bytes (eg. 01:) are reserved for multicast communication and can cause confusing problems for you. For instance, the guest will be able to receive ARP packets and reply to them, but the reply will confuse other machines. This is not a KVM issue, but just the way Ethernet works.

You do not need to restart libvirtd to reload the changes; the easiest way is to log into virsh (a command line tool to manage VMs), stop the VM, reread its configuration file, and restart the VM:

yhamon@paris:/etc/libvirt/qemu$ ls
mirror.xml  networks  vm2.xml
yhamon@paris:/etc/libvirt/qemu$ virsh --connect qemu:///system
Connecting to uri: qemu:///system
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list
 Id Name                 State
----------------------------------
 10 vm2                  running
 15 mirror               running

virsh # shutdown mirror
Domain mirror is being shutdown

virsh # define mirror.xml
Domain mirror defined from mirror.xml

virsh # start mirror
Domain mirror started

The VM "mirror" is now using bridged networking.

DNS and DHCP Guests

libvirt uses dnsmasq to hand out IP addresses to guests which are configured to use dhcp. If on your host machine, you add 192.168.122.1 (the default IP of your host in libvirt) as your first nameserver in /etc/resolv.conf, then you can do name resolution for your guests. dnsmasq is smart enough to use the other 'nameserver' entries in your /etc/resolv.conf for resolving non-libvirt addresses. For example, if your current /etc/resolv.conf is:

search example.com
nameserver 10.0.0.1

Change this to be:

search example.com
nameserver 192.168.122.1
nameserver 10.0.0.1

Now, if you have a virtual machine named 'hardy-amd64', after starting it, you can do:

$ host hardy-amd64
hardy-amd64 has address <IP address given by dnsmasq>

Note that when using ssh you may need to use a trailing '.' after the hostname:

$ ssh hardy-amd64.

Finally, for this to work, your guest must send its hostname as part of the dhcp request. This is done automatically on many operating systems. For systems that do not send this automatically and use dhcp3, you can adjust the dhclient.conf file. For example, on Ubuntu 6.06 LTS (Dapper), adjust /etc/dhcp3/dhclient.conf to have:

send host-name "<your guest hostname here>";

IMPORTANT: Depending on your network configuration, your host's /etc/resolv.conf file might be periodically overwritten. You will have to either adjust the dhcp server on your network to hand out the additional libvirt name server for your libvirt hosts, or adjust each host machine accordingly. As there are many possible configurations for host machines, user's are encouraged to look at resolvconf and/or man interfaces.

Booting Over the Network Using PXE

The current Ubuntu release does not ship pxe binary ROM images because the source code is not included to recreate the images in the upstream tarball. There may be a way to automate the creation of these files as part of the package. In order to use boot -n, you will need to download or create the appropriate ROM images from http://etherboot.org

KVM and QEMU can emulate a number of network cards. Here is the current ROM files

Copy the respective file to /usr/share/kvm and/or /usr/share/qemu.

Use virtio for Ubuntu Hardy/Intrepid or Windows guests

For Windows guests follow this instruction.

You may find the performances of the network relatively poor (approx. 100/120mbits on my servers, which are quite fast). If you are running Ubuntu Hardy or Intrepid, you can enable virtio. Go to the definition file of your VM, and add the virtio line to the definition of your network interface:

    <interface type='bridge'>
      <mac address='52:54:00:a0:41:92'/>
      <source bridge='br0'/>
      <model type='virtio'/>   <-- add this line, leave the rest
    </interface>

Or, if you're using KVM on the command line, add the options:

-net nic,model=virtio -net user

This improves the network performances by a lot (factor 10, nearly). But this works only with Ubuntu Hardy or Intrepid guests for the moment, which is why it is not by default.

Note that this also corrects the issue some are reporting with their network connections going away after a period of time or data transfer.

Using multiple nics with multiple subnets i.e. vlans

You may experience some KVM host connectivity issues when using multiple nics, each on their own subnet/vlan (multiple default routes?). In my case SSH logins (to the KVM host) would take a long time and connectivity would be cut when I restarted the network interfaces making ssh sessions and virt-manager connections crash.

I needed multiple nics, each to be on a separate subnet (vlan). Each nic is then dedicated to a specific VM on the KVM host. The VM's then connect directly to the network using a bridge device.

I never experienced problems with KVM guest connectivity. Only the KVM Host.

I fixed the problem using the following configuration in /etc/network/interfaces on the KVM host. Please note the use of "manual" and "metric". YMMV.

Note: first make sure that the guestOS loads the right network drivers, this worked for me: remove network modules 8139cp and 8139too , then modprobe 8139cp

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
        metric 0
###################

auto eth1
iface eth1 inet manual

auto br1
iface br1 inet dhcp
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0
        metric 1
###################

auto eth2
iface eth2 inet manual
        
auto br2
iface br2 inet dhcp
        bridge_ports eth2
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0
        metric 1
###################

# add more ethN and brN as needed

CategoryHomepage