Introduction
By default, your containers are accessible only from the host. For serious use you will want to expose some containers to the outside world. There are various ways of doing this. Currently I have settled on the following,
Port Forwarding using IP Tables - allows you to leverage your host's IP address (assuming it is public).
macvlan with Additional IP - allows you to have, a dedicated network interfaces (to the outside world) but actually only use one real physical network card. Unlike using a bridge this will not have the cpu overhead and need for your network card to work in promiscuous mode. This article builds on the work done in introductory LXC article.
I actually use both techniques together.
Port Forwarding using IP Tables
You might want to use one IP Address on the host and then map specific ports out from the containers. There are a number of ways to do this but I favour iptables.
In this example we setup apache which runs on port 80 in the container which has been assigned the static IP 10.0.3.10,
sudo iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to-destination 10.0.3.10:80 # save your changes (at this point I think it starts working, does not persists after reboot) sudo iptables-save # save changes to files, https://help.ubuntu.com/community/IptablesHowTo#Saving_iptables # this just looks like so much work must be better way # ok looks better to save changes using this tool (still to try) sudo apt-get install iptables-persistent sudo /etc/init.d/iptables-persistent save sudo /etc/init.d/iptables-persistent reload
Now traffic on port 80 on the host will be forwarded to port 80 in the container IP specified.
macvlan with Additional IP
For further isolation you may have purchased an additional IP address. In most hosting services this is not expensive, but they will likely not give you a dedicated network card. In this example we purchase an additional static IP from the hosting company and use the same network card as the host.
The most viable options, I understand are using bridge or a dedicated vlan.
With MACVLAN you configure the container to directly use the public IP address without the overhead of changing the network card to promiscuous mode. Once setup the macvlan gets it's own MAC address. This only works if there are no restrictions on the network which set's static IPs based on the hosts' MAC address. Usually this is only the case with the initial primary IP provided by the hosting company.
With macvlan, the containers can reach the network and each other, but not the host. Even though the host may be on the same network. I am not sure why this is the case (maybe security?) but do not see a need to solve with any use case. macvlan has many modes, but from my readings bridge mode is most appropriate.
macvlan mac address
The first thing to do is to create a mac address for the macvlan interface to be created on the host.
According to this article the Second bit of the most significant Byte indicates if the MAC address is Locally or Universally administered. When generating MAC addresses for macvlan, this bit should be set to '1', indicating it is Locally administered. As such translated to a mac address any of these numbers are locally managed and you will not need to worry about colliding with the larger Internet,
x2-xx-xx-xx-xx-xx x6-xx-xx-xx-xx-xx xA-xx-xx-xx-xx-xx xE-xx-xx-xx-xx-xx
Right now I have not read enough to describe the math, but for now, use the hellion website to generate a Random Locally Administered Unicast MAC Address. I selected 8a:38:2a:cc:d7:aa because the last aa is easy to search on.
Command Line macvlan
(move this section out as a subpage)
You can use the command line to quickly test, but it will disappear after reboot,
ip link add mvlan0 link eth0 address 8a:38:2a:cc:d7:aa type macvlan mode bridge ifconfig mvlan0 up
Create a Permanent macvlan on the Host
Add to the bottom of the /etc/network/interfaces file,
# Creates a macvlan interface called macvlan0 without an IP address iface mvlan0 inet manual pre-up ip link add mvlan0 link eth0 address 8a:38:2a:cc:d7:aa type macvlan mode bridge post-down ip link del macvlan0 auto mvlan0
Connect Container to macvlan on Host
Now one ore more containers may connect to the mvlan0 interface on the host and they will get their IPs directly from the same network connected to the host (if DHCP) or you can assign static IPs inside the container that are reserved for you.
Make the container aware of the mvlan0 by modifying the config file located in /var/lib/lxc/[container]/config. In this example, I named the container web so the location will be /var/lib/lxc/web/config,
# The directory itself is root only so for ease of browsing you might want to switch to root sudo su -
Modify the config file,
# Template used to create this container: /usr/share/lxc/templates/lxc-ubuntu # Parameters passed to the template: # For additional config options, please look at lxc.container.conf(5) # Common configuration lxc.include = /usr/share/lxc/config/ubuntu.common.conf # Container specific configuration lxc.rootfs = /var/lib/lxc/web/rootfs lxc.mount = /var/lib/lxc/web/fstab lxc.utsname = web lxc.arch = amd64 # Network configuration lxc.network.type = macvlan lxc.network.macvlan.mode = bridge lxc.network.flags = up lxc.network.link = mvlan0 # Hardware address was generated during container creation so leave whatever is there alone. lxc.network.hwaddr = 00:16:3e:91:7b:4f lxc.network.name = eth0
This tells tells the container to use the macvlan network interface mvlan0 which we created in the host and in turn map it to the container's eth0 interface.
There is probably a way to setup the static ip address here in this config file for the container. I am pretty sure I saw an option. However, in the vein of individual container control, I would rather set that at the container level using steps outlined below.
The last step is to modify the interfaces file within the container to your selected static IP address. I find the fastest is from within the Host OS using root, /var/lib/lxc/<container name>/rootfs/etc/network/interfaces which in this case would be, /var/lib/lxc/web/rootfs/etc/network/interfaces,
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 64.73.220.117 gateway 64.73.220.1 netmask 255.255.255.0 dns-nameservers 216.15.129.205 216.15.129.206
macvtap
This looks promising... The most prominent user of macvtap interfaces seems to be libvirt/KVM, which allows guests to be connected to macvtap interfaces. Doing so allows for (almost) bridged-like behaviour of guests but without the need to have a real bridge on the host, as a regular ethernet interface can be used as the macvtap's lower device.
References
Networking - https://help.ubuntu.com/lts/serverguide/lxc.html#lxc-network
Networking LXD More Detail - https://www.flockport.com/lxc-networking-guide/
INotes issue about bridge mode promiscuous mode - http://wiki.alpinelinux.org/wiki/LXC#Creating_a_LXC_container_without_modifying_your_network_interfaces
Getting macvlan working - https://www.flockport.com/lxc-macvlan-networking/
Learning macvlan and good background info - http://backreference.org/2014/03/20/some-notes-on-macvlanmacvtap/
Oracle article on details of container creation - http://docs.oracle.com/cd/E37670_01/E37355/html/ol_config_os_containers.html#ol_setup_fs_containers
Looks to be most comprehensive yet of what I want (multiple network cards with macvlan) - http://containerops.org/2013/11/19/lxc-networking/
How I figured out to create a macvlan - http://cyberiantiger.livejournal.com/24104.html