PZQ Sockets & Formats

PZQ has 3 sockets that can be accessed: a 'receive' socket where messages are added to the queue, a 'send' socket where they are sent on to the message receivers, and a 'monitor' socket that can be queried for information about the status of the PZQ instance itself.

PZQ messages in general will separate protocol control information from the user data with a blank message part - message headers may be added, and doing so should not break most client implementations.

Producers

PZQ refers to clients that add messages to the queue as 'producers'. They connect on the 'receive' PZQ socket with a ZeroMQ DEALER type socket. Messages are multipart ZeroMQ messages which consist of the following parts:

[0] - The identity of the message - this will be used to acknowledge PZQ has received it
[...] - There may be more protocol message parts, which should be ignored. 
[] - A blank message part (length 0) - this is the delimiter between protocol control and the data units
[Delimiter + 1] - The body of the message

There may be more parts, which will be passed through PZQ unmodified. After sending, the producer should listen for a response from PZQ, which will consist of the following parts:

[0] - The identity of the message being acknowledged 
[1] - A 1 or 0 - 1 for success, 0 for failure
[...] - There may be more protocol message parts, which should be ignored. 
[] - A blank delimiter message part
[Delimiter + 1] - Any descriptive messages - for example the failure message in the case of a 0 status

The producer should maintain a list of messages sent (if more than one) and mark off acknowledged IDs. If the second message part is 'NO' or any string other than OK, the producer should assume the message has not been registered successfully.

Consumers

Clients that accept messages from PZQ are referred to as 'consumers', and connect on the 'send' PZQ socket with a ROUTER type ZeroMQ socket. Consumers will receive a multipart ZeroMQ message with the following structure:

[0] - The identity of the PZQ instance
[1] - The identity of the message 
[2] - The time the message was sent as a microsecond timestamp
[3] - The acknowledgement timeout, in microseconds
[...] - There may be other header message parts here, so the following numbers are as an example.
[] - A blank delimiter message part
[Delimiter + 1] - The first part of the message body. 
[Delimiter + N] - The n'th part of the message body

More parts may be available, and should be consumed as normal and treated as part of the passed message. There may be extra parts before the blank message part - these should be ignore

Part 0 is used to allow replying to PZQ through the ROUTER socket used, and also to allow a client to consumer from multiple PZQ instances simply by connecting to each one.

Part 1 is a unique identity for the message, and should be used in the acknowledgement, and to ensure duplicate messages have not been received, if required.

Parts 2 and 3 are used to allow detection of expired messages. If the current time as a microsecond timestamp is greater than the sent time + the ack timeout, then the client can assume the message will be resent, and discard it. This may reduce instances of duplicate messages received.

On processing a message, the consumer should acknowledge it, to prevent resending and allow PZQ to remove the message from it's store. The acknowledgement should be sent back through the socket, as a ZeroMQ message with the following structure:

[0] - The PZQ identity
[1] - The identity of the message
[2] - ACK value - 0 (negative acknowledgment) or 1 (positive acknowledgment).

On receiving an ACK with a value of 1, PZQ will delete the message from the queue. If the value is 0, the message will be immediately requeued for a resend (for example if processing failed for a temporary reason, a client may wish to send this negative acknowledgment to trigger a explicit resend). If PZQ does not receive a response within the ack-timeout, it will automatically requeue the message for resending.

Monitoring

Monitoring code must create a REQ socket, and connect to the monitor PZQ socket. The monitor responds to a request containing the ASCII string "MONITOR" with a single part message. This will include multiple lines of monitoring information, which can be exploded by splitting field on the \n newline character, and then key from value on the colon ':'. An example response:

messages: 0
messages_in_flight: 0
db_size: 1438720
in_flightdb_size: 8390432
syncs: 16694
expired_messages: 6