Skip to content

Instantly share code, notes, and snippets.

@plembo
Last active June 5, 2019 19:50
Show Gist options
  • Save plembo/84af060175fec0bc9113ba18db525296 to your computer and use it in GitHub Desktop.
Save plembo/84af060175fec0bc9113ba18db525296 to your computer and use it in GitHub Desktop.
Mongod fails on reboot with specific IP

Mongod fails on reboot with specific IP

See note below on upstream change over a year ago that fixes the problem described here. Until the current release (4.0) server package is fixed the below change will need to be re-applied after each update of the server package.

The mongod service fails to restart after reboot when configured to listen on a specific IP. This only happens on my PCs that run NetworkManager. Servers using the traditional ifupdown/networking stack do not exhibit it.

My /etc/mongod.conf:

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1,10.1.0.101

I am using the official (mongodb-org) deb packages, installed from repo.mongodb.org (for bionic).

My (educated) guess is that the problem occurs because NM has not yet brought up all interfaces when mongod is starting.

If you set bindIp in /etc/mongod.conf to 0.0.0.0, the service will start.

Comparing the mongod.service file in /lib/systemd/system to other services dependent on networking, I found that the Unit section of some have:

After=network-online.target

while mongod.service only has:

After=network.target

The documentation on NetworkTarget states that it "has very little meaning during start-up", so "network.target" is clearly insufficient in this use case. The better choice would be "network-online.target", which actually has the service wait until all network interfaces are up. I've tested changing "After=network.target" to "After=network-online.target" in my mongod.service file on the NetworkManager-enabled machine, and it works.

This problem was the subject of SERVER-36043. It was fixed by a August 8, 2018 commit against the 4.1 release, with a request it be backported. The fix changed "After-network.target" to "After=multi-user.target". As a result, systemd will not attempt to start mongod until all other services essential for multi-user sessions are up (which would include all network interfaces). This change has still not been backported as of this latest writing (June 5, 2019).

I've now modified my /lib/systemd/system/mongod.service to use "After=multi-user.target" instead of "After=network.service".

[Unit]
Description=MongoDB Database Server
After=multi-user.target
Documentation=https://docs.mongodb.org/manual
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment