Introduction

By default, your containers are accessible only from the host. For serious use you will want to expose some containers to the outside world. There are various ways of doing this. Currently I have settled on the following,

Port Forwarding using Static IPs with IP Tables - allows you to leverage your host's IP address (assuming it is public).

macvlan with Additional IP - allows you to have, a dedicated network interfaces (to the outside world) but actually only use one real physical network card. Unlike using a bridge this will not have the cpu overhead and need for your network card to work in promiscuous mode. This article builds on the work done in introductory LXC article.

I actually use both techniques together.

Port Forwarding using IP Tables

You might want to use one IP Address on the host and then map specific ports out from the containers. As a pre-requisite you will need to setup Static LXC Assigned IP address.

There are a number of ways to do this but I favour iptables.

At the moment, it seems to conflict with my favourite firewall utility ufw, but I do not know for sure yet.

In this example we setup Apache which runs on port 80 in the container which has been assigned the static IP 10.0.3.10,

sudo iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to-destination 10.0.3.10:80

# save your changes (at this point I think it starts working, does not persists after reboot)
sudo iptables-save
 
# save changes to files,
https://help.ubuntu.com/community/IptablesHowTo#Saving_iptables # this just looks like so much work must be better way
 
# ok looks better to save changes using this tool (still to try)
sudo apt-get install iptables-persistent
sudo /etc/init.d/iptables-persistent save
sudo /etc/init.d/iptables-persistent reload

Now traffic on port 80 on the host will be forwarded to port 80 in the container IP specified.

macvlan with Additional IP

For further isolation you may have purchased an additional IP address. In most hosting services this is not expensive, but they will likely not give you a dedicated network card. In this example we purchase an additional static IP from the hosting company and use the same network card as the host.

Hopefully this will get better with LXD, but this is quite cumbersome compared to Solaris where you just tell the container to use a public IP address.

The most viable options, I understand are using bridge or a dedicated vlan.

With macvlan you configure the container to directly use the public IP address without the overhead of changing the network card to promiscuous mode. Once setup the macvlan gets it's own MAC address. This only works if there are no restrictions on the network which set's static IPs based on the hosts' MAC address. Usually this is only the case with the initial primary IP provided by the hosting company.

The containers can reach the network and each other, but not the host. Even though the host may be on the same network. I am not sure why this is the case (maybe security?) but do not see a need to solve with any use case. macvlan has many modes, but from my readings bridge mode is most appropriate.

If you had previously assigned a static IP to the container using /etc/lxc/dnsmasq.conf make sure to remove the entry (I believe you also need to restart the host).

macvlan mac address

The first thing to do is to create a mac address for the macvlan interface to be created on the host.

According to this article the Second bit of the most significant Byte indicates if the MAC address is Locally or Universally administered. (I think locally is better on for the public facing so you do not collide with other mac addresses). When generating MAC addresses for macvlan, this bit should be set to '1', indicating it is Locally administered. As such translated to a mac address any of these numbers are locally managed and you will not need to worry about colliding with the larger Internet,

x2-xx-xx-xx-xx-xx
x6-xx-xx-xx-xx-xx
xA-xx-xx-xx-xx-xx
xE-xx-xx-xx-xx-xx

Right now I have not read enough to describe the math, but for now, use the hellion website to generate a Random Locally Administered Unicast MAC Address. In this example, I selected 8a:38:2a:cc:d7:aa because the last aa is easy to search on.

You can not use the same MAC interface on multiple containers on the same host. Otherwise, you will not be able to start you container and receive the error message about your interface already being in use.

Command Line macvlan

(move this section out as a subpage)

You can use the command line to quickly test, but it will disappear after reboot,

ip link add mvlan0 link eth0 address 8a:38:2a:cc:d7:aa type macvlan mode bridge
ifconfig mvlan0 up

I believe you must have 1 macvlan mapped to 1 container interface and they all must have different static IP addresses. Need to test. I tried replicating my steps in a new container and though it got the IP address, I could not connect.

Create a Permanent macvlan on the Host

Add to the bottom of the /etc/network/interfaces file,

# Creates a macvlan interface called macvlan0 without an IP address
iface mvlan0 inet manual
   pre-up ip link add mvlan0 link eth0 address 8a:38:2a:cc:d7:aa type macvlan mode bridge
   post-down ip link del macvlan0
auto mvlan0

Connect Container to macvlan on Host

Now one ore more containers may connect to the mvlan0 interface on the host and they will get their IPs directly from the same network connected to the host (if DHCP) or you can assign static IPs inside the container that are reserved for you.

Make the container aware of the mvlan0 by modifying the config file located in /var/lib/lxc/[container]/config. In this example, I named the container web so the location will be /var/lib/lxc/web/config,

# The directory itself is root only so for ease of browsing you might want to switch to root
sudo su -

Modify the config file,

# Template used to create this container: /usr/share/lxc/templates/lxc-ubuntu
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Common configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
# Container specific configuration
lxc.rootfs = /var/lib/lxc/web/rootfs
lxc.mount = /var/lib/lxc/web/fstab
lxc.utsname = web
lxc.arch = amd64

# Network configuration
lxc.network.type = macvlan	
lxc.network.macvlan.mode = bridge
lxc.network.flags = up
lxc.network.link = mvlan0
# Hardware address was generated during container creation so leave whatever is there alone.
lxc.network.hwaddr = 00:16:3e:91:7b:4f
lxc.network.name = eth0

This tells tells the container to use the macvlan network interface mvlan0 which we created in the host and in turn map it to the container's eth0 interface.

There is probably a way to setup the static ip address here in this config file for the container. I am pretty sure I saw an option. However, in the vein of individual container control, I would rather set that at the container level using steps outlined below.

The last step is to modify the interfaces file within the container to your selected static IP address. I find the fastest is from within the Host OS using root, /var/lib/lxc/<container name>/rootfs/etc/network/interfaces which in this case would be, /var/lib/lxc/web/rootfs/etc/network/interfaces,

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 64.73.220.117
gateway 64.73.220.1
netmask 255.255.255.0
dns-nameservers 216.15.129.205 216.15.129.206

Multiple Interfaces

In most cases you will want multiple interfaces. In this example, we built a front-end container called "web", gave it a public IP address using mavlan. In addition, we create an "app" container which has an lxc internal IP address. In order for web and app to communicate, web must have a second interface that also uses an lxc provided internal IP address.

(...draw a diagram here...)

macvtap

This looks promising... The most prominent user of macvtap interfaces seems to be libvirt/KVM, which allows guests to be connected to macvtap interfaces. Doing so allows for (almost) bridged-like behaviour of guests but without the need to have a real bridge on the host, as a regular ethernet interface can be used as the macvtap's lower device.