How to Create Docker Macvlan Networks & Make it Persistent

The most common question that crops up regularly on #docker is “How do I attach a container directly to my local network?” One possible answer to that question is the macvlan network type, which lets you create “clones” of a physical interface on your host and use that to attach containers directly to your local network. For the most part it works great, but it does come with some minor caveats and limitations. I would like to explore those here.

For the purpose of this example, let’s say we have a host interface eno1 that looks like this:

2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 64:00:6a:7d:06:6a brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.24/24 brd 192.168.10.255 scope global dynamic ens3
       valid_lft 73303sec preferred_lft 73303sec
    inet6 fe80::b2c9:3793:303:2a55/64 scope link 
       valid_lft forever preferred_lft forever

To create a macvlan network named 'my-docker-net‘ attached to that interface, you might run something like this:

docker network create -d macvlan -o parent=ens3 \
  --subnet 192.168.10.0/24 \
  --gateway 192.168.10.254 \
  my-docker-net

…but don’t do that.…

Address assignment

When you create a container attached to your macvlan network, Docker will select an address from the subnet range and assign it to your container. This leads to the potential for conflicts: if Docker picks an address that has already been assigned to another host on your network, you have a problem!

You can avoid this by reserving a portion of the subnet range for use by Docker. There are two parts to this solution:

• You must configure any DHCP service on your network such that it will not assign addresses in a given range.
• You must tell Docker about that reserved range of addresses.

How you accomplish the former depends entirely on your local network infrastructure and is beyond the scope of this document. The latter task is accomplished with the --ip-range option to docker network create.

On my local network, my DHCP server will not assign any addresses above 192.168.10.190. I have decided to assign to Docker the subset 192.168.10.192/27, which is a range of 32 address starting at 192.168.10.192 and ending at 192.168.10.223. The corresponding docker network create command would be:

docker network create -d macvlan -o parent=ens3 \
  --subnet 192.168.10.0/24 \
  --gateway 192.168.10.254 \
  --ip-range 192.168.10.192/27 \
  my-docker-net

Now it is possible to create containers attached to my local network without worrying about the possibility of ip address conflicts.

Host access

With a container attached to a macvlan network, you will find that while it can contact other systems on your local network without a problem, the container will not be able to connect to your host (and your host will not be able to connect to your container). This is a limitation of macvlan interfaces: without special support from a network switch, your host is unable to send packets to its own macvlan interfaces.

Fortunately, there is a workaround for this problem: you can create another macvlan interface on your host, and use that to communicate with containers on the macvlan network.

First, I’m going to reserve an address from our network range for use by the host interface by using the --aux-address option to docker network create. That makes our final command line look like:

docker network create -d macvlan -o parent=ens3 \
  --subnet 192.168.10.0/24 \
  --gateway 192.168.10.254 \
  --ip-range 192.168.10.192/27 \
  --aux-address 'host=192.168.10.223' \
   my-docker-net

This will prevent Docker from assigning that address to a container.

Next, we create a new macvlan interface on the host. You can call it whatever you want, but I’m calling this one my-docker-route:

ip link add  my-docker-route link ens3 type macvlan  mode bridge

Now we need to configure the interface with the address we reserved and bring it up:

ip addr add 192.168.10.223/32 dev mynet-shim
ip link set  my-docker-route up

The last thing we need to do is to tell our host to use that interface when communicating with the containers. This is relatively easy because we have restricted our containers to a particular CIDR subset of the local network; we just add a route to that range like this:

ip route add 192.168.10.192/27 dev  my-docker-route

With that route in place, your host will automatically use this my-docker-route interface when communicating with containers on the my-docker-net network.

Note that the interface and routing configuration presented here is not persistent – you will lose if if you were to reboot your host. How to make it persistent is distribution dependent.

Persistent IP settings on Ubuntu/Linux

In my case, I used the rc.local script. This guide explains how to enable /etc/rc.local script to run on system startup.

First try ‘sudo systemctl status rc-local‘

nextc@nextk:~$ sudo systemctl status rc-local
[sudo] password for nextk:
○ rc-local.service - /etc/rc.local Compatibility
     Loaded: loaded (/lib/systemd/system/rc-local.service; static)
    Drop-In: /usr/lib/systemd/system/rc-local.service.d
             └─debian.conf
     Active: inactive (dead)
       Docs: man:systemd-rc-local-generator(8)

As you can see from above, The unit file have no [Install] section. As such Systemd can not enable it. First we need to create a file

"sudo nano /etc/systemd/system/rc-local.service

Then add the following content to it.

[Unit]
 Description=/etc/rc.local Compatibility
 ConditionPathExists=/etc/rc.local

[Service]
 Type=forking
 ExecStart=/etc/rc.local start
 TimeoutSec=0
 StandardOutput=tty
 RemainAfterExit=yes
 SysVStartPriority=99

[Install]
 WantedBy=multi-user.target

Save and close the file. To save a file in Nano text editor, press Ctrl+O, then press Enter to confirm. To exit the file, Press Ctrl+X.  Next, run the following command to make sure /etc/rc.local file is executable.

sudo chmod +x /etc/rc.local

Note: Starting with 16.10, Ubuntu doesn’t ship with /etc/rc.local file anymore. You can create the file by executing this command.

printf '%s\n' '#!/bin/bash' 'exit 0' | sudo tee -a /etc/rc.local

Then add execute permission to /etc/rc.local file.

in the /etc/rc.local file, you will find:

#!/bin/sh -e

exit 0

Here is the most important part: Insert the script below into the spaces between '#!/bin/sh -e' and 'exit 0' above. The result will appear like this:

#!/bin/sh -e
ip link add my-docker-route link ens3 type macvlan mode bridge
ip addr add 192.168.10.223/32 dev my-docker-route
ip link set my-docker-route up
ip route add 192.168.10.197/27 dev my-docker-route
exit 0

then apply:

sudo chmod +x /etc/rc.local

After that, enable the service on system boot:

sudo systemctl enable rc-local

Output:

Created symlink from /etc/systemd/system/multi-user.target.wants/rc-local.service to /etc/systemd/system/rc-local.service.

Now start the service and check its status:

sudo systemctl start rc-local.service
sudo systemctl status rc-local.service

Output:
nextc@nextk:~$ sudo systemctl status rc-local.service
[sudo] password for ubuntu:
● rc-local.service - /etc/rc.local Compatibility
     Loaded: loaded (/etc/systemd/system/rc-local.service; enabled; vendor preset: enabled)
    Drop-In: /usr/lib/systemd/system/rc-local.service.d
             └─debian.conf
     Active: active (exited) since Mon 2024-04-01 11:40:44 CEST; 1h 1min ago
    Process: 689 ExecStart=/etc/rc.local start (code=exited, status=0/SUCCESS)
        CPU: 166ms

Now reboot and test & enjoy!