Running unprivileged LXC containers on Fedora

LXC containers are great - they are an easy way to set up a clean development or testing environment, lightweight, and simple to maintain. Unfortunately, running unprivileged containers (i.e. not running them as root, which is inconvenient and generally not a good idea) doesn’t work out-of-the box on Fedora (but to be fair, the same goes for Debian and friends).

At the end of this guide you should be able to run unprivileged containers without needing to mess with SELinux (which strangely the Wiki suggests you need to disable) with the option to add bind mounts in your container to share files between your home folder and the container’s filesystem.

Some basics first

Before we continue we just need to clear up some basics. By running containers in unprivileged mode, it means that you will never be making use of root privileges on your host machine. They will be started as your normal user account - for this to work LXC will make use of something called UID namespaces. Basically, inside the container, UID 0 (which is root) will be mapped to some arbitrary UID on the host (usually way up in the thousands). This has the cool benefit that if something were to break out of the container, it will be running as some arbitrary, unprivileged UID on the host and probably won’t be able to do much. The other cool thing is that this allows your unprivileged normal user account to start, stop and manage containers without ever touching sudo.

Installation

I’m writing this guide on Fedora 34, but it should also work on other versions. Firstly, install LXC and its templates (templates makes creating containers much easier):

sudo dnf install lxc lxc-templates

Setting up the networking

Next you’ll need to allow your user access to the network bridge created by the LXC networking service. To do so, run the following:

# allow user to access network bridge
# see the note regarding F35 above
sudo tee /etc/lxc/lxc-usernet <<EOF
$USER veth virbr0 10
EOF

# restart the lxc network service
sudo systemctl restart lxc-net

With these two commands you’ve allowed your user account to create 10 network bridges on the default LXC network device virbr0. Without this your containers won’t be able to attach to the network bridge and they won’t have network connectivity.

Updating the firewall rules on F35

This step is only required if you’re running Fedora 35 or newer. By default, firewalld will block your containers from obtaining DHCP addresses. To fix this you’ll need to add the LXC bridge to firewalld’s trusted zone.

sudo firewall-cmd --zone=trusted --change-interface=lxcbr0 --permanent
sudo firewall-cmd --reload

Configuring LXC

Finally we also need to tell LXC to use the correct network interface, and to set up the correct UID and GID mappings. You can add the following to the system-wide LXC config (/etc/lxc/default.conf) however I prefer using the one in my home folder so that I can check it into my dotfiles.

Add the following to ~/.config/lxc/default.conf (you might need to create the folders):

# default networking
lxc.net.0.type = veth
# remember to update this to lxcbr0 on F35 and above
lxc.net.0.link = virbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx

# uid and gid mapping - all the ids, except 1000, are mapped to an
# unpriviledged range on the host. 1000 is kept so that filesharing through
# bind mounts is possible
lxc.idmap = u 0 100000 1000
lxc.idmap = u 1000 1000 1
lxc.idmap = u 1001 101000 64536

lxc.idmap = g 0 100000 1000
lxc.idmap = g 1000 1000 1
lxc.idmap = g 1001 101000 64536

One thing to note is the UID mapping ranges. All the container’s UIDs, except for 1000, is mapped to UIDs >100000. However UID 1000 is mapped to 1000 on the host - this is to allow read-write bind mounts between your home folder and the container’s default user (not root! A normal user account that you created - see the next section). Your account should have an UID of 1000 by default. If not, you’ll need to slightly adjust the above to fit your account’s UID (man lxc.container.conf).

Creating the container

At this point you should be able to create and run a container. To do so run the following:

# here because the keyserver used by LXC no longer exists
export DOWNLOAD_KEYSERVER=keys.openpgp.org
# create the container - doesn't need to be fedora
lxc-create -t download -n my_container -- -d fedora -r 34 -a amd64

If everything went well you should be presented with something like the following:

Setting up the GPG keyring
Downloading the image index
Downloading the rootfs
Downloading the metadata
The image cache is now ready
Unpacking the rootfs

---
You just created a Fedora 34 x86_64 (20210805_20:33) container.

Start the container, and create a normal user account to use:

# start the container
lxc-start my_container
# attach to it
lxc-attach my_container

## we're now inside the container as the root user

# create a normal user and set a password
useradd -mG wheel -s /usr/bin/bash my_user
passwd my_user
# log in as the normal user
su my_user
# create a folder in their home folder
mkdir ~/test
# log out of the normal user
exit
# exit the container
exit

## we're now back on the host

# stop the container
lxc-stop my_container

We created the test folder so that we can set up a bind-mount, as explained in the next section.

Bind-mounting folders from the host to the container

Using bind-mounts comes in very handy, for example mounting your code directory (which is on the host) to inside the container. Since the UID mappings have already been set up you just need to add the following to your container config file in ~/.local/share/lxc/my_container/config:

lxc.mount.entry = /home/kobus/test home/my_user/test none bind 0 0

A few things to note here:

• The first part of the line is the absolute path to the folder on the host system (my username is kobus, be sure to adjust it accordingly).
• The second part is the path relative to the container’s rootfs - i.e. just make sure to omit the leading slash /.
• You can add mounting options instead of the none. For instance, this can be used to make the mount read-only.

Close and save the file, and that’s it! If you start your container again the bind mount should be up, allowing you to easily share files between your host and container.