gotchas with IPs and Jails

Through some problems with installing postfix and opensmtpd at the same time, I again had the need to invest some time into FreeBSD Jails.

As I had some problems with the IP allocation, I document what I found out here.

First and foremost, I think I could have had it easier using VIMAGE/vnet, but that still isn’t enabled per default on 10.2 and 10.3, the versions I tested.

The following settings are for the jail.conf system, but can also be used on the command line.

configure an IP

The easiest setup is to define an IP on any interface and tell the jail system to use a specific one.

For this example, I use the prestart command to define the IP on any interface.

# define on a public interface
jail1 {
  exec.prestart = "ifconfig em0 192.168.1.2 alias";
  ip4.addr = 192.168.1.2;
}

# define on loopback
jail2 {
  exec.prestart = "ifconfig lo0 192.168.1.3 alias";
  ip4.addr = 192.168.1.3;
}

# reuse 127.0.0.1 from the host
jail3 {
  ip4.addr = 127.0.0.1;
}

Using this mechanism, the IP is left alone when starting or stopping the jail.

configure an IP on an interface

When specifing an IP together with an interface, jails will take over the life management. When the jail is started the IP is created and when stopping the jail, the IP is removed.

The following will show some ways how to do that:

# set an IP on a public interface
jail1 {
  ip4.addr = em0|192.168.1.2;
}

# define a loopback address
jail2 {
  ip4.addr = lo0|192.168.1.3;
}

There is also the interface option, which can be used to pin every IP to that specific interface.

# define two addresses on the same interface, maintained by the jail system
jail1 {
  interface = em0;
  ip4.addr = 192.168.1.10, 127.0.0.1;
}

This should be used when an IP is not used by the host or another jail. The following example would destroy the loopback address on shutdown:

# removes localhost at jail shutdown
jail1 {
  ip4.addr = lo0|127.0.0.1;
}

mixing both options

In the case of poudriere, you have to mix both options. Poudriere wants to put 127.0.0.1 and ::1 into the child jails, so that these have to be defined in the poudriere jail too.

If other IPs are also needed, this can be done with the ip4.addr and ip6.addr options.

# manage 192.168.1.11 using jails and use localhost unmanaged
jail1 {
  ip4.addr = em0|192.168.1.11, 127.0.0.1;
}

Hope that helps to clarify, what exactly each option does.