TLS works, but breaks ACL

Mosquitto MQTT Broker
1

This is a summary of my problem. What should I do please?

I can subscribe and/or publish locally using a pwfile and aclfile.
I can subscribe and/or publish remotely using TLS with no aclfile.
I cannot publish remotely using TLS with the same certs & aclfile, but I can still subscribe.

client1 has a username of user1, client 2 user2 etc

To be certain that it wasn’t my TLS, I recreated every single key and cert from scratch. I am certain that it isn’t a problem with the certs/keys because the clients securely connect properly.

Example Error:

1673354561: New client connected from 192.168.1.10:57530 as client2 (p2, c1, k60, u'CN=user2').
1673354561: No will message specified.
1673354561: Sending CONNACK to client2 (0, 0)
1673354562: Denied PUBLISH from client2 (d0, q2, r0, m1, 'test/topic01', ... (24 bytes))
1673354562: Sending PUBREC to client2 (m1, rc135)
1673354562: Received PUBREL from client2 (Mid: 1)
1673354562: Sending PUBCOMP to client2 (m1)
1673354562: Received DISCONNECT from client2
1673354562: Client client2 disconnected.

I have been struggling with this for quite a few days now, & I have finally worked out how to replicate & validate my problem.

I have mosquitto (version 2.0.15-2) installed on a Raspberry Pi running Arch Linux. I have Arch Linux running on my main workstation, & have installed mosquitto there too so that I have local access to the CLI clients. I use SSH and Screen to open multiplex terminals on the Pi 4. I have several terminal tabs open on my main workstation.

For test purposes, I have create a password file called pwfile, containing user1, user2, user3 and user4, with passwords set to password1, password2, password3 and password4. I have created TLS keys and certificates for each user, with corresponding CN’s of user1, user2, user3, user4. Finally, I created the following ACL file based on the originally supplied example file. I am also using the following mosquitto.conf file.

aclfile

topic read $SYS/#

# user section
user user1
topic #

user user2
topic test/topic01

user user3
topic read test/+

user user4
topic write test/topic01

# This affects all clients.
pattern write $SYS/broker/connection/%c/state

mosquitto.conf file

per_listener_settings true
persistence true
persistence_file mosquitto.db
persistence_location /var/lib/mosquitto
autosave_interval 300
retain_available true

# Local Listener with PW and ACL
#
listener 1883 127.0.0.1
socket_domain ipv4
allow_anonymous false
allow_zero_length_clientid false
password_file /etc/mosquitto/pwfile
acl_file /etc/mosquitto/aclfile

# TLS Listener with acl
#
listener 8884
socket_domain ipv4
allow_anonymous false
allow_zero_length_clientid false
require_certificate true
use_subject_as_username true
acl_file /etc/mosquitto/aclfile
cafile /etc/mosquitto/certs/ed_ca.crt
certfile /etc/mosquitto/certs/mqtt-broker.crt
keyfile /etc/mosquitto/certs/mqtt-broker.key

# TLS Listener without acl
#
listener 8883
socket_domain ipv4
allow_anonymous false
allow_zero_length_clientid false
require_certificate true
use_subject_as_username true
#acl_file /etc/mosquitto/aclfile
cafile /etc/mosquitto/certs/ed_ca.crt
certfile /etc/mosquitto/certs/mqtt-broker.crt
keyfile /etc/mosquitto/certs/mqtt-broker.key

Test publish & subscribe commands that demonstrate the problem

Using Screen on the Pi to multiplex terminals

Terminal 1 runs mosquitto

mosquitto -v -c /etc/mosquitto/mosquitto.conf

Terminal 2 - subscribes to test/topic01 (works as expected)

mosquitto_sub 	-h 127.0.0.1 \
		-d \
		-i client3 \
		-u user3 \
		-P password3 \
		-q 2 \
		-t test/topic01

Terminal 3 - sends these 3 publish commands (works as expected)

mosquitto_pub	-h 127.0.0.1 \
		-d \
		-i client4 \
		-u user4 \
		-P password4 \
		-q 2 \
		-t 'test/topic01' \
		-m 'user4 localhost helloWorld01'

mosquitto_pub	-h 127.0.0.1 \
		-d \
		-i client1 \
		-u user1 \
		-P password1 \
		-q 2 \
		-t 'test/topic01' \
		-m 'user1 localhost helloWorld02'

mosquitto_pub	-h 127.0.0.1 \
		-d \
		-i client2 \
		-u user2 \
		-P password2 \
		-q 2 \
		-t 'test/topic01' \
		-m 'user2 localhost helloWorld03'

Port 8883 subscribe and publish remotely

Port 8883 subscribe on terminal 1 (works as expected)

mosquitto_sub 	-h 192.168.1.1 \
		-p 8883 \
		-d \
		-i client3 \
		--cafile ed_ca.crt \
		--cert mqtt-user3.crt \
		--key mqtt-user3.key  \
		-q 2 \
		-t 'test/topic01'

Port 8883 publish: terminal 2 sends these 3 commands (works as expected)

mosquitto_pub	-h 192.168.1.1 \
		-p 8883 \
		-d \
		-i client4 \
		--cafile ed_ca.crt \
		--cert mqtt-user4.crt \
		--key mqtt-user4.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_02 from user4'

mosquitto_pub	-h 192.168.1.1 \
		-p 8883 \
		-d \
		-i client1 \
		--cafile ed_ca.crt \
		--cert mqtt-user1.crt \
		--key mqtt-user1.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_02 from user1'

mosquitto_pub	-h 192.168.1.1 \
		-p 8883 \
		-d \
		-i client2 \
		--cafile ed_ca.crt \
		--cert mqtt-user2.crt \
		--key mqtt-user2.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_02 from user2'

Port 8884 subscribe and publish remotely

Port 8884 subscribe on terminal 1 (works as expected)

mosquitto_sub 	-h 192.168.1.1 \
		-p 8884 \
		-d \
		-i client3 \
		--cafile ed_ca.crt \
		--cert mqtt-user3.crt \
		--key mqtt-user3.key  \
		-q 2 \
		-t 'test/topic01'

Port 8884 publish: terminal 2 sends these 3 commands (publish denied on broker)


mosquitto_pub	-h 192.168.1.1 \
		-p 8884 \
		-d \
		-i client4 \
		--cafile ed_ca.crt \
		--cert mqtt-user4.crt \
		--key mqtt-user4.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_03 from user4'

mosquitto_pub	-h 192.168.1.1 \
		-p 8884 \
		-d \
		-i client1 \
		--cafile ed_ca.crt \
		--cert mqtt-user1.crt \
		--key mqtt-user1.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_03 from user1'

mosquitto_pub	-h 192.168.1.1 \
		-p 8884 \
		-d \
		-i client2 \
		--cafile ed_ca.crt \
		--cert mqtt-user2.crt \
		--key mqtt-user2.key  \
		-q 2 \
		-t 'test/topic01' \
		-m 'helloWorld_03 from user2'
2

Hi Justin,

Thanks for the great summary, it made it very easy to find the problem. There are three important lines:

What you said:

client1 has a username of user1, client 2 user2 etc

Config file entry:

use_subject_as_username true

And log entry:

New client connected from 192.168.1.10:57530 as client2 (p2, c1, k60, u'CN=user2').

The use_subject_as_username option uses the whole certificate subject as the username, hence your username becomes CN=user2 rather than just user2. If you use use_identity_as_username true instead, then just the commonName part will be extracted from the certificate to give you user2.

I hope that helps,

Roger

1 Like
3

Thanks Roger, I shall make that change & retest everything later today when I get home. I’ll post the results back as soon as possible. That looks to be an interesting gotcha.

4 
1673384548: New client connected from 192.168.1.10:52834 as client4 (p2, c1, k60, u'user4').
1673384548: No will message specified.
1673384548: Sending CONNACK to client4 (0, 0)
1673384548: Received PUBLISH from client4 (d0, q2, r0, m1, 'test/topic01', ... (24 bytes))
1673384548: Sending PUBREC to client4 (m1, rc0)
1673384548: Received PUBREL from client4 (Mid: 1)
1673384548: Sending PUBLISH to client3 (d0, q2, r0, m2, 'test/topic01', ... (24 bytes))
1673384548: Sending PUBCOMP to client4 (m1)
1673384548: Received PUBREC from client3 (Mid: 2)
1673384548: Sending PUBREL to client3 (m2)
1673384548: Received DISCONNECT from client4
1673384548: Client client4 disconnected.
1673384548: Received PUBCOMP from client3 (Mid: 2, RC:0)

Thank you Roger, all is now sweetness & light.

I swapped use_subject_as_username for use_identity_as_username as you advised. The logs now have u'user4'instead of u'CN=user4' and everything has suddenly started working.

1 Like