Planning a DIY Smart Home Setup With MQTT

Tech Talk MQTT
1

Hey everyone,

I’m currently planning out a DIY smart home project and I’d love to get some advice before I actually start building anything. This is my first time doing a full IoT setup from scratch, so I want to make sure I’m going in the right direction.

My idea is to take several IoT devices (mainly sensors, switches, plugs, etc.) and flash them, so I can run custom firmware and get rid of the cloud-based ecosystem. I want all devices to communicate locally with my Raspberry Pi 5, without using the Tuya cloud or any external services.

Since I want fast and reliable communication between devices, I was thinking about using MQTT with Mosquitto as the central message broker. The plan is:

  • Raspberry Pi 5 running Mosquitto
  • My custom Python backend handling API calls and device logic
  • IoT devices flashed, publishing/receiving MQTT messages
  • Light switches, sensors, etc. reacting as instantly as possible

I’m aiming for very low latency. For example, if I toggle a light switch through my backend, the response should feel immediate.

I haven’t started building this yet, I’m still collecting information and figuring out the best way to structure everything. Since this is my first time doing this, I’d really appreciate suggestions, recommendations, or potential pitfalls I should watch out for.

Some things I’m especially curious about:

  • Is Mosquitto a good choice for this kind of setup?
  • Any best practices for structuring topics and payloads?
  • Tips for making communication as fast and stable as possible
  • Good ways to secure the system while keeping it simple

Thanks a lot for any insight you can share! I want to set this up properly from the beginning.

2

Hello,

I did a very similar journey recently and that journey continues.

All your points make sense but here are a few observations that you might want to think about.

Mosquitto is absolutely fine. A robust and performant piece of software. You’ll have no problems with it and, if you do, it will be a configuration issue or another connected piece of hardware or software.

Use a MQTT channel outside of normal WiFi: change the default channel 11 to 15, 20 or 25.

Consider a mini-pc rather than the Raspberry Pi 5. I use a Pi for collecting stats, etc., about the home and for monitoring (temperatures, boiler pressures, weather and so on, using DIY ESP32 devices and sensors) but I have come to use - and rather like - Home Assistant for controlling the smart home. It just makes life so much easier.

I use Zigbee2mqtt for the coordinator. What do you intend to use? I find Mosquitto MQTT + Zigbee2MQTT a great combination; the latter has thousands of devices in its database, so you may not need to flash anything at all.

Or do you intend to ignore Zigbee completely and go for Matter over Thread? (It’s the THREAD bit that’s the alternative to ZIGBEE). What, then, are the cost implications?

Will you install direct to the RPI 5 or will you use docker containers? Or maybe ProxMox? Lots to think about there.

And will you want local voice control? If so, another nail in the coffin for the RPI 5, perhaps.

I will continue to use the RPI 5 I already have for the information aspect of IOT (Influxdb, Grafana, Node-Red, etc): it’s a great fit. But I shall transfer the home automation stuff to a fast mini-pc with NVME SSD to control the house via sensors. The aim is to replace Alexa at some point, but few things cut it as well just yet, especially voice control.

A Smart Home needs rock-steady hardware. A RPI 5 + SD card is OK but useless long-term. A RPI 5 + NVME is better. A good spec mini-pc for local fast voice processing would be better still. All can run the software you’re considering. How robust/fast do you want it to be?

If you’re not sure about Home Assistant for your Smart Home AND you already own a RPI 5 you can always trial it using docker containers - with zero consequence if you don’t like it.

3

Hi,

Thank you so much for the detailed response! Really appreciate you taking the time to share your experience. You’ve given me a lot to think about, especially around hardware choices and the Zigbee vs WiFi decision.

Let me tell you what i intent to do:

This is primarily a learning project for my university thesis (but I also want to build a smart home for me). I want to understand IoT systems from the ground up - firmware, protocols, backend architecture, basicaly everything. That’s why I’m choosing to flash devices with custom firmware rather than using off the shelf Zigbee devices.

I’m targeting Tuya devices with BK7231N chips and flashing them with OpenBeken firmware. I’ve already build a pogo pin flash station (3D printed jig with needles), which has been working great for quick flashing without soldering each time.

Starting with RPi5 + NVMe - I’ve already bought nvme hat. I tried two m.2 SSDs and neither worked. First was a SATA m.2 (wrong interface, learned that lesson). Then i tried a Samsung 980 EVO NVMe, but it doesn’t recognize it. I’ve read some NVMe drives have compatibility issues with the RPi5. Do you have any SSD recommendations that you know work reliably with RPi5 and this NVMe hat: X1002 PCIe M.2 2280 to NVMe SSD Bottom for Raspberry Pi 5

For learning porpuses, I’m sticking with WiFi + MQTT via OpenBeken, but maybe I will use Zigbee in the future (after I am done with my project). I also plan to design and build some custom sensors (ESP-32 based) to really undersand the hardware side.

I’ve studied HA quite a lot. Really great architecture. Not trying to compete, just building a focused system for specific use case as a learning exercise (I have time, since I am a student.)

Also I am planning to containerize from the start (Mosquitto, FastAPI, PostgreSQL). I’ve already set up everything on my RPI5, when it reboots, the whole backend gets put back up. I also set up cloudflare tunnel, so I can safely connect to my RPI from anywhere in the world and control lights,…

I have a few questions for you:

  1. MQTT topic strucutre you recommend? I am considering something like: homeassistant/devices/{device_id}/command/{action}
  2. Docker networking: Bridge vs Host for MQTT performance
  3. Is InfluxDB + Grafana worth for less then 20 devices? I am interested in tracking temperature trends, energy usage over time, stats of how many times does certain door open, how many times does certain light gets turned on,…

Also really appreciate the MQTT channel tip.

If you encountered any challenges along your journey or have any more tips, I’d be very happy to hear about them. Always better to learn from others’ mistakes than make them all yourself!

Thank you!

4

Hello,

I think you are perhaps asking on the wrong forum for all you need. This (Cedalo) forum appears to have extremely low activity and it is unlikely someone with the knowledge you require would come across your post. I think you may have better luck on other fora.

I can answer only a few questions but your use case is highly specific and, since it is a research project, then you should be using your results to determine your future direction. Does your tutor have no input/guidance? It all seems a little odd for a thesis.

  • I have had success with a Crucial P310
  • Samsung EVO appear not to to be compatible

As for messaging, the existing protocols have been honed by many including those with commercial interests and therefore are highly reliable. If you want to explore different ones then why not try them? It is a research project?

Bridge v Host? Absolutely nothing to do with MQTT and I cannot understand why you are asking that. This is all explained in the Docker documentation. That is where you should direct your gaze. Besides: Surely you are measuring performance if it is a stated aim? The measure will, of themselves, answer your question.

Influx + Grafana: Both are free for personal use. So is it worth it? Of course. Or do you mean worth the effort? It’s no greater for one device than several hundred. But beware, should you choose InfluxDB3, the limit on the number of Parquet files (~ 3 days) for queries for the single-use version. Enterprise does not have this restriction.

Finally: Your last comment raises many suspicions about your motives or abilities or commitment. A thesis involves research, mistakes, data analysis and mid-course corrections. Asking “If you encountered any challenges along your journey or have any more tips, I’d be very happy to hear about them. Always better to learn from others’ mistakes than make them all yourself!” has all the hallmarks of a newbie wanting others to supply all the answers. A thesis has absolutely no relation to “tips”.

I’m sorry if you haven’t liked the reply but I have delayed responding until I felt sure that much in your post simply didn’t add up. A thesis is an extremely high level of work (or should be) and your questions sound as if you’ve no plan.

Nonetheless, I wish you success.

Regards,

Ric