Peter M. Groen / FlashMQ

This needed a separation: getting the current thread, and getting the
thread of the client you're queueing a command for.

This also resolves a circular reference between Client and ThreadData.

Instead of the thread data, which didn't make sense.

This required adding a global stats object.

It also contains a bit of refactor to make a type out of the derived
counters.

This makes much more sense than returning the amount of messages sent
all the way up the call stack.

This fixes clients being disconnected after reducing the max value and
reloading the settings.

Check events are placed in a sorted map based on the last activity and
keep-alive interval of the client.

This makes it more accurate and reduces system load because it saves
unnecessary checking.

It merely drops packets when they exceed it. The specs are unclear about
whether you're supposed to delay transmission until the quota is
non-negative again. I decided against it because of increased
complexity, and because on a continously overloaded client, this makes
no sense.

Effectively, this formalizes the 'max qos pending' mechanism that was
already in place.

It also includes PUBACK/PUBREL/PUBCOMP error handling, because that
needed to be done for proper quota control.

This also contains some fixes/unifications for traditional
authentication, error handling specifically.

On server shutdown and when taking over a session.

On disconnect, wills are queued first, we wait for the queueing to be done, then
initiate disconnect.

When TCP buffers are full and fds are not reported by epoll, the thread
loop still exits and clients are just closed on exit.

This required a special type WillPublish to make this easier and more
logical.

This includes some logic to have threads finish their work before
quitting.

- Delay is properly counted from moment of disconnect.
- When a session is picked up again, the will is not sent.
- An actual fix to what I tried a few commits ago: fix sending will
  twice.

This logic should make storing wills also easier.

I think it's very hard to distinguish between protocol error and
malformed packet. It's kind of arbitrary...

Because clients can now also exist as dummy objects, I had to add some
extra checks.

Also split up handlePublish() and the new parsePublishData().

I think it's correct, but mosquitto_sub doesn't seem to support it, so I
can't test.

Also some other stuff I happen to see.

This replaces the old style checking of expired sessions once every 10
minutes or so.

which also has implications on how I think to handle 'queuePacketAtSub'

The behavior for MQTT3 clients in the same, but I replaced the term
'clean session' and described the behavior in MQTT5 terms, of 'clean
start' and an expiry interval.

Most of it is limits we already implemented non-standard compliant.

This is a preparation for MQTT5, because when there are receivers and
publishers with different protocols, you can't always just write out the
same packet. You can sometimes though, so that's what the copy factory
determines.

This entails making copies of the original packet when necessary,
because QoS 0 doesn't have a packet id. I tried to keep it to an
absolute minimum and do some precarious optmizations for it. There are
tests though.

It caused really funky behavior. When a destructed client closed fd 0,
eventfd() would give 0 back as fd. This would then later give errors.

This makes more sense, logically, and also helps in tests I'm about to
write.

One fix is client destruction happening on the correct thread (when
kicking another one off with existing client ID). This caused deadlocks
on the subscriptions lock during a race condition when
doKeepAliveCheck() also ran.

A related deadlock was that the queued functions were executed while
holding the lock taskQueueMutex. Together with the subscriptions lock,
that was executed in the $SYS topic function, this also caused
deadlocks.

One part is a fix, where the frame was advertised to be bigger than it
was.

The other change is making it possible to send chunks larger than the
initial buffer size, by resizing the buffer (and resetting it later).

Mandatory normative statement MQTT-3.1.2-6.