Mosquitto config file can't be renamed on SUSE Linux Enterprise Server

I have to install a Mosquitto MQTT broker on a SUSE Linux Enterprise Server (SLES 15 SP3). On a bare installation, Mosquitto 2.0.10 is installed from the SUSE-PackageHub-15-SP3-Backports-Pool repository.

Installing it, enabling it and rebooting, the service works fine.

According to the manual (mosquitto.conf man page | Eclipse Mosquitto),

[the configuration] file can reside anywhere as long as mosquitto can read it.

However, if I simply rename the configuration file (not altering it) and point mosquitto to the new configuration file location in the systemd service file, it fails to start. The configuration file is the SLES default (empty) configuration file. See the contents from the terminal session below to illustrate what I’m trying to say.

What I’m ultimately trying to achieve is having a symbolic link for mosquitto.conf to some other location. Debugging why this didn’t work, it turned out even this basic sanity check fails. I’m pretty stuck with this one, any help pointing me to the right direction appreciated!

Thanks a lot and cheers,

Fabian

Additional things tested:

  • I get exactly the same behaviour when executing mosquitto manually (e. g. running /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto_renamed.conf)
  • I had also tried with a manually installed 1.6.9 version, same behaviour (though not extensively tested)
  • The mosquitto user should have access to the file (see the namei output at the end of the terminal output below)
  • On a Ubuntu 20.04 system, building mosquitto 2.0.10 from source, it works as expected, you can rename/move/readonly the config file

Terminal session to illustrate my problem: First, run mosquitto service with the config file at its default location, everything is fine. Rename the same file, restart the service, it complains, although having the very same permissions and all.

localhost:/etc/mosquitto # ls -l
total 64
-rw-r----- 1 root mosquitto   230 Apr  3  2021 aclfile.example
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 ca_certificates
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 certs
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 conf.d
-rw-r----- 1 root mosquitto 39217 Apr 23  2021 mosquitto.conf
-rw-r----- 1 root mosquitto    23 Apr  3  2021 pskfile.example
-rw-r----- 1 root mosquitto   355 Apr  3  2021 pwfile.example
localhost:/etc/mosquitto # systemctl status mosquitto.service
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
     Loaded: loaded (/usr/lib/systemd/system/mosquitto.service; enabled; vendor preset: disabled)
     Active: active (running) since Mon 2022-07-04 17:50:33 CEST; 13s ago
       Docs: man:mosquitto.conf(5)
             man:mosquitto(8)
   Main PID: 2375 (mosquitto)
      Tasks: 1
     CGroup: /system.slice/mosquitto.service
             └─2375 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Jul 04 17:50:33 localhost systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: mosquitto version 2.0.10 starting
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: Config loaded from /etc/mosquitto/mosquitto.conf.
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: Starting in local only mode. Connections will only be possible f>
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: Create a configuration file which defines a listener to allow re>
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: For more details see https://mosquitto.org/documentation/authent>
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: Opening ipv4 listen socket on port 1883.
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: Opening ipv6 listen socket on port 1883.
Jul 04 17:50:33 localhost mosquitto[2375]: 1656949833: mosquitto version 2.0.10 running
localhost:/etc/mosquitto # mv mosquitto.conf mosquitto_renamed.conf
localhost:/etc/mosquitto # ls -l
total 64
-rw-r----- 1 root mosquitto   230 Apr  3  2021 aclfile.example
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 ca_certificates
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 certs
drwxr-xr-x 2 root mosquitto  4096 Jul  4 16:06 conf.d
-rw-r----- 1 root mosquitto 39217 Apr 23  2021 mosquitto_renamed.conf
-rw-r----- 1 root mosquitto    23 Apr  3  2021 pskfile.example
-rw-r----- 1 root mosquitto   355 Apr  3  2021 pwfile.example
localhost:/etc/mosquitto # sed -i 's%mosquitto.conf%mosquitto_renamed.conf%' /usr/lib/systemd/system/mosquitto.service
localhost:/etc/mosquitto # systemctl daemon-reload
localhost:/etc/mosquitto # systemctl restart mosquitto.service
localhost:/etc/mosquitto # systemctl status mosquitto.service
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
     Loaded: loaded (/usr/lib/systemd/system/mosquitto.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Mon 2022-07-04 17:55:36 CEST; 3s ago
       Docs: man:mosquitto_renamed.conf(5)
             man:mosquitto(8)
    Process: 2418 ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto_renamed.conf (code=exited, status=3)
   Main PID: 2418 (code=exited, status=3)

Jul 04 17:55:36 localhost systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.
Jul 04 17:55:36 localhost mosquitto[2418]: 1656950136: Error: Unable to open config file /etc/mosquitto/mosquitto_renam>
Jul 04 17:55:36 localhost systemd[1]: mosquitto.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Jul 04 17:55:36 localhost systemd[1]: mosquitto.service: Failed with result 'exit-code'.
localhost:/etc/mosquitto # cat mosquitto_renamed.conf
# Config file for mosquitto
#
# See mosquitto.conf(5) for more information.
#
# Default values are shown, uncomment to change.
#
# Use the # character to indicate a comment, but only if it is the
# very first character on the line.

[removed all the commented sections for readability]
include_dir /etc/mosquitto/conf.d
localhost:/etc/mosquitto # ls -l conf.d/
total 4
-rw-r--r-- 1 root mosquitto 142 Apr 23  2021 README
localhost:/etc/mosquitto # cd /
localhost:/ # namei -l /etc/mosquitto/mosquitto_renamed.conf
f: /etc/mosquitto/mosquitto_renamed.conf
drwxr-xr-x root root      /
drwxr-xr-x root root      etc
drwxr-x--- root mosquitto mosquitto
-rw-r----- root mosquitto mosquitto_renamed.conf

Hi Fabian,

would you please check the apparmor settings for the mosquitto program. When I remember correctly the AppAmor is configured very restrictive in SLES and allows access to the default config file only.

As services are started with root permission the mosquitto executable reads the config file first and drops root permission by switching to the mosquitto user after the config file is read.

Best,
Norbert

Double checked it now by taking a look into the SLES mosquitto package you use. This package installs a profile for apparmor in the file /etc/apparmor.d/usr.sbin.mosquitto. And in this profile access to the /etc/mosquitto/ is limited to the certs, the conf.d subfolder and the default mosquitto.conf file.

profile mosquitto /usr/sbin/mosquitto {
        #include <abstractions/base>
        #include <abstractions/nameservice>

        /usr/sbin/mosquitto r,
        /etc/mosquitto/mosquitto.conf r,
        /etc/mosquitto/ca_certificates/* r,
        /etc/mosquitto/certs/* r,
        /etc/mosquitto/conf.d/ r,
        /etc/mosquitto/conf.d/* r,
        /var/lib/mosquitto/ r,
        /var/lib/mosquitto/mosquitto.db rwk,
        /var/lib/mosquitto/mosquitto.db.new rwk,
        /var/run/mosquitto.pid rw,

        network inet stream,
        network inet6 stream,
        network inet dgram,
        network inet6 dgram,

        # For drop privileges
        capability setgid,
        capability setuid,

        # For tcp-wrappers
        /lib{,32,64}/libwrap.so*  rm,
        /etc/hosts.allow r,
        /etc/hosts.deny r,
        #include <local/usr.sbin.mosquitto>
}

So you may put your config files into the conf.d subfolder (which will be included by the mosquitto.conf provided by the package). This ways it will work on all other SLES systems as well. If you want to use a different config file you would need to modify the apparmor profile for the mosquitto.

Best,
Norbert

1 Like

Hi Norbert,

I was not aware of AppArmor I must admit - that is probably the link I was missing for SLES. Thanks a lot!!!
I will try disabling AppArmor/modifying the policy and report back.

Best,
Fabian

Hi Norbert,

you were right, when disabling the AppArmor profile for Mosquitto (with ln -s /etc/apparmor.d/usr.sbin.mosquitto /etc/apparmor.d/disable and restart) everything works as expected. I can move/rename/symlink the config file however I want to :slight_smile:
So the difference between SLES and other systems, e. g. Ubuntu, is simply, that a (restrictive) AppArmor profile for Mosquitto is installed by default.

Thanks again for pointing that out and best,
Fabian