MQTT Cluster - shower thoughs

notes.md

multiple layers. No layer is a unique solution, just an algamation of existing solutions.

Problem:

Scaleability + Remote

• lots of individual sensors/hardware connecting to singular services
  • hard to scale/manage at any moderate size
  • creates lots of different interfaces to have to deal with
  • may connect up to some central brand service: AWS, Nest, Apple, Google etc.
    • but not universal across all your sensors:
    • a reduction but still lots to manage/interface with
    • also all are remote, susptable to all the elements inbetween you and that service going down.

Solution:

Centrailization

• central hubs/gateways
  • single place to funnel and manage sensors.
  • easier to optimize groups of stream/messages than indiviually
    • reduce outbound requests if the response can be handled locally.
  • central hub with common stream normilization -> greater potential for interoperability

Problem

Single Point of Failure

• Good start, but this also suffers now from being a single point of failure

Solution

Central Gateway Cluster

• The solution is actually Reddis' solution. Master/Slave node cluster

  • A group of identacle gateways
    • 1 master, the rest slaves that report to the master
    • if master falls over, slaves vote on who gets promoted to new master
      • vote can be random at first just to get up and runing
        • ultimatly use ping/transfer rates, log/traffic volume or other qualifiers to determine best candidate
    • once old master comes back up, it is a slave to new master.
  • No worry about dupe messages, MQTT already has that handled

• Side benifts:

  • can place nodes throughout a large space.
    • some would be closer to particular groups of sensors
    • hardware with shorter range can report to a closer node
      • better hardware of local node (PI) deals with range issue/data tranfer to "central hub"
      • can be treated as local "sub station"
      • things like Zigbee and Z-Wave suffer from this "short/impeaded range issue"
  • hardward can pick which node to report to based on transfer rate
    • ocationally ping other node to check transfer rates
      • then ping more offten if it's faster/similar to current primary node