Description

Introduction:
• This assignment is worth 15% of the final mark
Assignment 3
MQTT is the most common open IoT protocol being deployed today. It provides a publisher/subscriber model, using a broker or server. It allows for an almost-unbounded number of sources to publish information, each at their own rate, and subscribers to receive information (and act on it). As such, it is designed to provide high-performance communication mechanisms, with minimal delays and excellent scaling performance. We’ll use it to monitor the performance of some imaginary system: say counting the total kilograms of minerals rushing by on a conveyor belt, that you can control. This assignment will look at the functionality and performance of the publishers, brokers, the network and the subscribers.

Submitting
You will be submitting your code and your analysis report. Note that there will be two submission links on the course-site in wattle:
1. Your code must be submitted to wattle as a zip file, with instructions on how to compile/run the components as appropriate.
Each question has an indication of how much (rough maximum) you should write.
Outcomes
We’re assessing your understanding of MQTT, your code’s functionality in subscribing/publishing to a broker, dealing with high message rates, and your insight to interpret what you’re seeing. You will be exploring MQTT functionality, the quality-of-service (QoS) levels, describing how they work, why you would use different levels, and how they perform in real-world deployments given various publishers.
Resources
You will need to set up your own MQTT server (broker) for you to connect to as per the specifications below. The Mosquitto broker is perhaps the best choice to run on your own computer, it has binaries and/or packages for most operating systems, but if you find a better one you can use that. It must be at least MQTT v3.1.1 compliant (i.e. MQTT v5 is not necessary).
A key consideration is that you will need to allow your assignment partner to connect to it over the Internet for part of the assignment. If you have a home modem/router that does NAT you will need to work out how to configure it to allow remote connections (think about the port(s) you need to forward, and how to manage any changing IP addresses). There are a few other ways to solve this, e.g. a VPN service on your modem or a local computer. If it proves too difficult you might consider setting up a cloud server with a public IP address on e.g. Amazon, Azure, Alibaba, etc. and installing a broker there. Yet another option is to set up a (free) fully-hosted broker through a provider such as HiveMQ.com.
Your broker needs to be configured for some basic authentication and access control:
• username student and password 33102021
• require the client-id to start with 3310-
You can test the broker works by subscribing to the $SYS topics, which describe the server, and help get you familiar with the information presented there – you will be using them for your analysis later.

Assignment requirements and questions
You need to write three small programs, the first two of which could be combined into one:
• A Publisher: It will send a simple message to the broker, an incrementing counter (e.g. 0, 1, 2, 3, …). It will send those messages at a specific QoS level (0, 1 or 2), and with a specific delay between
messages (0ms, 10ms, 20ms, 50ms, 100ms, 500ms). It needs to publish to the topic
‘counter/<qos>/<delay>’, so e.g. ‘counter/1/100’ implies qos=1 and delay=100. It should only publish one stream at any given time, based on input from the Controller.

• A Controller: This will listen to the broker on two topics ‘request/qos’ and ‘request/delay’. Whenever it gets a message on either of them it needs to modify the Publisher’s behaviour accordingly. One way is to stop any current publisher process (e.g. send a KILL signal, or at worst put a 2.5-minute time limit on it running) and start a new one with the new parameters, say via command line arguments. Another approach might be through event-handling and multi-threading, but it is up to you. Please explain in a README file or comments what approach you are taking.

• An Analyser: This will listen to the counter on the broker and take measurements (below) to report statistics on the performance of the broker/publisher/network combination. Those measurements should be taken across the range of qos and delay values as above, so that you can compare them. Your Analyser will publish to the ‘request/qos’ and ‘request/delay’ topics to get the Publisher to deliver what you need at that time. For the QoS, subscribe at the same QoS as requested (i.e. if publisher->broker qos=2 then Analyser->broker should have qos=2 as well). The Analyser does not need to take all (6 delay * 3 qos) measurements in one run, but the closer in time the measurements occur the better, as network conditions do change all the time.
You can test all of these at home on your local broker. It is only in the final measurements that you will allow another student’s Analyser to access your broker, and vice-versa.
In your report, please address the following questions:
1. Subscribe to a broker, with the matching QoS level on your client (you can use any mqtt client you like for this). Wireshark the handshake for one example of each of the differing QoS-level messages coming from the counter, capture screenshots that show it, and briefly explain in your report how each handshake works, and what it implies for message duplication and message order. Discuss briefly in your report in which circumstances would you choose each QoS level, and offer an
example of a sensor that would use each of them. [around 1 page]

2. Run your Analyser against your partner’s broker, tell it what you want the publisher to send and listen for 2 minutes. Tip: only print individual messages to screen for debugging, otherwise it will slow your code down a lot. Check with the broker owner what delay there might be between your request and their publisher responding.

a. Collect statistics, for each delay and QoS level, and measure over the 2-minute period:
i. The overall average rate of messages you actually receive across the period [messages/second].
ii. The rate of message loss you see [percentage].
(i.e. how many messages should you have seen, minus how many you saw)
iii. The rate of any out-of-order messages you see [percentage]
(i.e. how often do you get a smaller number after a larger number)
iv. The mean and median inter-message-gap you see [milliseconds].
Only measure for actually consecutive messages, ignore the gap if you lose any messages in between.

b. While measuring the above also
i. Subscribe to and record the $SYS topics, and identify what, if anything, on the broker do the loss/misordered-rates correlate with. (Look at measurements under e.g. ‘load’, as well as the ‘heap’ and ‘active clients’, anything that seems relevant.
See https://mosquitto.org/man/mosquitto-8.html ) ii. Record a traceroute between the Analyser and the broker, and between the
Publisher and the broker (you’ll need the broker owner to provide the second one)

c. In your report [around 2-3 pages of text, plus any charts]
i. Summarise your measurements, and note when you took them. Explain what you expected to see, especially in relation to the different QoS levels, and whether your expectations were matched.
ii. Describe what correlations of measured rates with $SYS topics you expect to see and why, and whether you do, or do not.

3. Consider the broader end-to-end (internet-wide maybe) network environment, in a situation with millions of sensors publishing regularly/irregularly to thousands of subscribers. Explain in your report [around 1-2 pages]
a. what (cpu/memory/network) performance challenges there might be, from the source publishing its messages, all the way through the network to your subscribing client,
c. how it compares (or not) with the actual quantified differences between QoS levels you measured as part of this assignment.

Assessment
Your assignment will be assessed on
• Your code clarity, efficiency, and documentation/comments (20%)
• your code subscribing and properly publishing to the broker (10%), and
• your analysis report addressing the questions above (70%)