RabbitMQ > Basic Understanding

Basic Understanding

RabbitMQ is a message broker that was originally built on AMQP. Besides AMQP, it supports STOMP ( Simple Text Oriented Messaging Protocol ), MQTT ( Message Queuing Telemetry Transport ), HTTP protocols. RabbitMQ is a well-known for its documentation. 

In fact, RabbitMQ newbies will be able to deploy RabbitMQ very easily with its excellent documentation and sample source codes. They are right there for you as if RabbitMQ developers had already expected that newbies would be lost at certain points.

---

Glossary

Message Broker

Literally, message broker. It receives messages and sends them back.

Producer

The one who sends (publishes) messages.

Exchange

It's like a physical network router on Layer7.

It gets a routing key and sends the message to a matched queue that expects to have messages with the right key.

Publish / Subscribe

It's Publish / Subscribe pattern. It's also called Observer pattern.

Queue

It's a queue, literally.

Bindings

It's a relationship between a queue and an exchange ( or exchanges and queues )

Consumer

It consumes messages from a queue.

Connection

It's an abstract socket connection.

Channel

It means a channel that has been made out of a connection. Ofcourse you can create several channels more in a single connection.

In code terms, Channel class got the whole RabbitMQ APIs.

---

Basics

What queue is for?

To deliver messages to different kind of systems. Messages themselves can be various things. It could be a HTTP request header form, just some text combinations, or some operation code for a system. Messages can be asynchronous. In asynchronous environment, a consumer who consumed a message from a queue dies with an unfortunate incidence, then the message can be re-queued again and the queue immediately sends it back to the consumer right after online status turns on. And this is especially effective in a web application that has only a HTTP request window.

Round-Robin Dispatching

RabbitMQ delivers message in round-robin way. Let's see the example below.

Assume that we have two queues.

1. Producer has published "1", "2", "3", "4", "5", "6", "7" messages.

2. As a result of round-robin dispatching, Queue_A and Queue_B are going to get messages respectively.

Queue_A got message "1"

Queue_A got message "3"

Queue_A got message "5"

Queue_A got message "7"

Queue_B got message "2"

Queue_B got message "4"

Queue_B got message "6"

Message Acknowledgement

Basically a queue marks a message for deletion that has been consumed by a customer. If the consumer dies before it processes the messages properly, then the message will be gone, nowhere either in the queue or the customer. 

So here message acknowledgement option goes. If this option is turned on, a consumed message won't get a deletion mark, instead it gets an unack mark. This unack mark never fades away until the consumer fires BasicAck event. If the consumer dies, then the unack message will be requeued to the queue and be ready to be redelivered.

This way, you can prevent any message losses in this case.

No message timeout

RabbitMQ doesn't have any message timeout. It just redelivers them.

Message Durability

A similar backup behavior like message acknowledgement, but durability is for RabbitMQ server itself. Message acknowledgement options is for consumers, it doesn't guarantee the message backup when RabbitMQ server crashes. You have to set true to durable when you declare a queue and have to make sure to set true to persistent property in a channel. If not, it's going to recover the queue only, not the messages in it.

Fair dispatch

Literally it means dispatching messages faily. As I mentioned before, RabbitMQ delivers messages in round robin way. in below circumstances, round robin dispatching can be painful.

1. Producer publishes "1", "2", "3", "4", "5" ······ messages every 5 seconds.

2. As a result of round-robin dispatching, Queue_A and Queue_B are going to get messages respectively.

3. But it takes 10 seconds for Queue_A to process a message.

4. It takes 3 seconds for Queue_B to process one.

5. Eventually and gradually, unprocessed messages ( Unack messages ) will be queued up high in Queue_A over and over again.

	5 secs passed	10 secs passed	20 secs passed	30 secs passed	40 secs passed	······
Queue_A	Unack MSG 1 Acked MSG 0	Unack MSG 1  Acked MSG 1	Unack MSG 2 Acked MSG 2	Unack MSG 3 Acked MSG 3	Unack MSG 4 Acked MSG 4	······
Queue_B	Unack MSG 0 Acked MSG 1	Unack  MSG 0 Acked  MSG 2	Unack MSG 0 Acked MSG 4	Unack MSG 0 Acked MSG 6	Unack MSG 0 Acked MSG 8	······

To prevent this, you can set prefetchCount using basicQos. Setting prefetchCount will block additional messages coming into the slow queue once it reaches the limit. Then the blocked messages will be delivered to another available queue.

---

Basics+

Exchange

The core idea is to never send message directly to a queue.

An exchange exists as a middle layer between a producer and queues. Below is the diagram.

An exchange could have four types

- direct

- topic

- header

- fanout

Binding

You have to specify which exchanges you will use in order to route a message. This is called "binding".

A routing key must be set when you configure binding and this is the most important feature you should know to make the best of use of the four exchange types above.

Direct Exchange

In this scenario, a message will be delivered to a queue that exactly, 100% matches the routing key.

As the diagram says, a message with a routing key "orange" goes into Q1, and the others go into Q2.

You can set N routing keys to a queue by invoking QueueBind.

You can also set a same routing key to different queues. This is called "multi bindings"

Fanout Exchange

It's a mindless broadcasting, completely ignoring routing keys. When you google "fan out" you will get to see some of this.

Topic Exchange

In this scenario, you can have a topic as a routing key. Some specific titles or topics can be your routing key.

You can have more flexibility with topic exchange as opposed to direct exchange where you have to match the whole word to deliver a message, because topic exchange has two wild cards.

* : replace one word.

# : replace more than one word.

genius.orange.damon routing key goes to Q1.

you.orange.farm routing key also goes to Q1.

apple.orange.mango routing key goes to Q1.

topic.exchange.rabbit routing key goes to Q2.

sample.example.rabbit routing key goes to Q2.

.

.

.

But * wildcard can replace only one word, genius.dm.tistory.orange.post will not be able to go to Q1.

Here # wildcard comes out.

lazy.rabbit routing key goes Q2.

lazy.exchange.queue routing key also goes Q2.

lazy.company.department.employer routing key also can go to Q2.

lazy.company.department.employer.sex.height ~~ ( whatever ) routing key goes to Q2.

Header Exchange

You can use this when you want to use a routing key as { Key : Value } pair, just like a HTTP request header. In this scenario, the routing key will be completely ignored and exchanges will only inspect X-match property in the header and route the message accordingly.

You can set two options in the header.

- any : OR condition. One of the key-value pairs in X-match prop matches? then it's okay to send the message to the target queue.

- all : AND condition. All of the key-value pairs in X-match prop match? then it's okay to send the message to the target queue.

IMAO, topic exchange will be prevalent, who's going to use header exchange?

RPC

The core idea is to set a response queue name to reply_to property. You can set reply_to property, publishing a message. Then a consumer gets the msg and sends it back to the destination if reply_to exists.

Correlation Id

When you get a response message from the reply_to queue, you just don't know which request the response belongs. That's when this correlation id is used. Everytime you publish a message, you make a GUID value and set it to the correlation id. Then the property will live to the end of the message life cycle.