Accessing services on KVM guests behind a NAT

I have a small web service running on a Fedora 17 VM. The VM lives on the default virtual network provided by libvirt, which allows outbound connections to the external world. But because the IPv4 space for that network (192.168.122.0/24) is private, the host's iptables rules will NAT all packets on their way out.

That means "no way in" for external connections. The virtual network addresses are unseen (unroutable) to the outside world.

Luckily, libvirtd provides hooks so that you can insert custom operations at key points in the start-up/shut-down phases of a guest.   The following procedure shows how to create a hook to modify iptables when a guest is starting (or stopping).

1. On the host, create a new file called /etc/libvirt/hooks/qemu containing this sample script. (There's a link to a more comprehensive script if you need it. I also wrote a Python replacement below.)
2. Replace the placeholder variables in the sample script with the guest name, addresses, and ports that are to be forwarded.
3. Restart the libvirt service.
4. Verify new DNAT rules with iptables -nvL -t nat.

Make sure that your guest doesn't have any iptables rules that will interfere.  For instance, you'll need to open up TCP port 80 if you want your web service to be accessible.

As an exercise, I rewrote the sample script in Python, adding some logging and removing some redundancy along the way.

#!/usr/bin/env python
#!/usr/bin/env python
import argparse
import logging
import subprocess
import sys

guestAddr = { 'f17-base':'192.168.122.200' }
hostPort = { '80':'8000', '22':'222' }
logFile = '/var/log/qemu-hook.log'
logLevel = logging.DEBUG

logging.basicConfig(filename=logFile, level=logLevel, format='%(asctime)s : %(message)s')

parser = argparse.ArgumentParser()
for arg in [ 'guest', 'op', 'subop', 'extra' ]:
    parser.add_argument(arg)
args = parser.parse_args()

logging.debug('qemu hook: guest %s, op %s' % (args.guest, args.op))
if args.guest not in guestAddr:
    logging.debug('Nothing to do for guest %s' % args.guest)
    sys.exit()

if args.op == 'start':
    natOp = '-A'
    filterOp = '-I'
    opStr = 'Adding'
elif args.op == 'stopped':
    natOp = filerOp = '-D'
    opStr = 'Removing'
else:
    logging.debug('Nothing to do for op %s' % args.op)
    sys.exit()

for gport in hostPort.keys():
    # build the nat command
    natcmd = ['iptables', '-t', 'nat', natOp, 'PREROUTING']
    gAddrPort = "{}:{}".format(guestAddr[args.guest], gport)
    natcmd += ['-p', 'tcp', '--dport', hostPort[gport], '-j', 'DNAT', '--to', gAddrPort]  
    logging.info('%s nat rules for %s' % (opStr, gAddrPort))
    subprocess.call(natcmd)
    # build the filter command
    filtercmd = ['iptables', filterOp, 'FORWARD']
    filtercmd += ['-d', '{}/32'.format(guestAddr[args.guest]), '-p', 'tcp']
    filtercmd += ['-m', 'state', '--state', 'NEW']
    filtercmd += ['-m', 'tcp', '--dport', gport, '-j', 'ACCEPT']
    logging.info('%s forwarding rules for %s' % (opStr, gport))
    subprocess.call(filtercmd)

logging.info('iptables update complete')
sys.exit()

Note that, depending on the version of libvirtd you are running, the hook may not process "reconnect" operations -- i.e., when libvirtd is restarted.  You need 0.9.13 or later for that.  If you have an earlier version, you may have an issue with duplicate rules.

I left those operations out of the Python script because my Fedora 17 system is running 0.9.11.

Cheers!