Trace: systemd

systemd

Let services start after network-online.target

On systems with systemd it can happen, that network related services cannot start due to Cannot bind to IP address.

This is, because systemd sometimes tries to start up certain services when the network-online.target hasn't finished yet.

Click for more details

This is because they assumed during systemd development that network daemons would normally just use local addresses and generally not rely on routable addresses.

The following two quotes may clarify this:

network.target has very little meaning during start-up. It only indicates that the network management stack is up after it has been reached. Whether any network interfaces are already configured when it is reached is undefined. Its primary purpose is for ordering things properly at shutdown: since the shutdown ordering of units in systemd is the reverse of the startup ordering, any unit that is order After=network.target can be sure that it is stopped before the network is shut down if the system is powered off. This allows services to cleanly terminate connections before going down, instead of abruptly losing connectivity for ongoing connections, leaving them in an undefined state. Note that network.target is a passive unit: you cannot start it directly and it is not pulled in by any services that want to make use of the network. Instead, it is pulled in by the network management service itself. Services using the network should hence simply place an After=network.target dependency in their unit files, and avoid any Wants=network.target or even Requires=network.target.
network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind. Its primary purpose is to actively delay activation of services until the network is set up. It is an active target, meaning that is may be pulled in by the services requiring the network to be up, but is not pulled in by the network management service itself. By default all remote mounts defined in /etc/fstab pull this service in, in order to make sure the network is up before it is attempted to connect to a network share. Note that normally, if no service requires it, and if no remote mount point is configured this target is not pulled into the boot, thus avoiding any delays during boot should the network not be available. It is strongly recommended not to pull in this target too liberally: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up), its primary purpose is network client software that cannot operate without network.

Source: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

This means that IP addresses aren't assigned to the interfaces and if the daemons are configured to listen on ports with specific IP addresses, then those daemons fail to start. There are several ways to fix this. Two of them are: set net.ipv4.ip_nonlocal_bind = 1 in sysctl, or let the daemons bind to any ip.

But you can also make systemd work the way you want and let it start the daemons after the network is fully online.

Let's take nginx as example:

# mkdir /etc/systemd/system/nginx.service.d

And create a network-online.conf file in that directory with the following setting:

[Unit]
After=network-online.target

Source: https://unix.stackexchange.com/questions/442181/sshd-failed-due-to-network-not-yet-available

pub/tech/linux/systemd.txt · Last modified: 2024/02/12 13:36