If you have a company, whether technological or more traditional, you're sure to ever be forced to use message queues, whether physical (order forms in a restaurant, for example) or technological (RabbitMQ, Amazon SQS, etc.). These queues are very important assets for the operation of the company, as without them there would be no way to make any communication between the different areas (kitchen and tables in the case of the restaurant) or different programs (in the case of the technology company).

As this is a technology blog, from now we will always talk about technological message queues.

We'll start by talking a little about these message queues. At the operational level, basically, these message queues would work like a pile of papers. The programs fill the stack with new messages, while other programs collect them. These messages can contain anything from simple text to files.

Advantages

Let's start by commenting above the advantages of this technology:

• Efficient distribution of messages between the different processes or programs within your company..
• Automatic storage. Messages are stored until they are read, making it possible to avoid the storage part of them and being able to focus on the use we make of them.
• Great scalability. As the number of programs that make use of them increases, there is no noticeable drop in performance.
• Decoupling between programs. By only having to interpret the messages of the queues, the programs are not so dependent on each other.

Importance of monitoring

To show the importance of monitoring this technology. Let's give an example of a company that uses these queues but does not monitor them.

As an example, we have a luxury car manufacturing company. In order not to have any cars in stock due to the expense involved, they use a production organization system based on the Japanese "Just in time" system.

This company has several factories, and different parts suppliers. They use queue technology for communication between different factories and company departments. Each time they are commissioned to manufacture an automobile, a message is created which is picked up by a factory, which will then assemble the parts. Messages are also created for the purchasing department, with the different orders that have to be made to the suppliers for the manufacture of the vehicle.

With the pass of time, they realize that there are some parts sent by suppliers that are left over. At first they think they can be spare parts sent by the supplier, in case there is one that fails during the automobile manufacturing process.

When they have an important stock of these parts, they talk to the supplier so that he does not send them any more spare parts, since the quantity of these parts is already becoming a major storage problem. From the supplier company they confirm that they send the right parts and do not send spare parts.

From the company they check how it can happen that there are pieces left over. By doing calculations, they come to the conclusion that suppliers send them twice as many parts as they need, so the fault has to come from being asked for twice as many parts needed for manufacturing.

Due to this worrying error, the company decides to investigate why orders to this supplier are duplicated. In order to do this, it has several people following the process of ordering parts from suppliers from the moment they are commissioned to manufacture a car.

First they go to check if this is due to a human error and they think about the purchasing department, but they think that if it was a human error it would only have happened once and not always, so it shouldn't be there. Subsequently, they will look at how many times the request reaches this department. There they notice that they receive an order of more than the due parts for each order of cars that are made. Therefore the error has to come from the software in charge of distributing the requests to the suppliers.

When they review this software, they realize that they are right, and it is in charge of ordering twice as many parts as necessary for each car they make.

Possible solutions

If this company would have worked with RabbitMQ, through a monitoring tool, it could have been easily detected as there are more purchase orders than there should be. Therefore, this problem would have been detected much faster, as we can see in the following image.

We observe that the number of messages in the "Buy" queue goes up by 5, when it should be 4.

This company would have detected this anomaly much more quickly if they had hired a monitoring service right from the start. By only having to observe it to obtain the necessary data to find the error. This would have reduced costs both in the purchase of parts and in the storage of them.

If, in addition to that, the monitoring system had an early warning system, it would be possible to have detected this anomaly before it even posed a problem for the company. Even at the time the software was tested, there would have been no failure.

From Muutech, thanks to Zabbix, these message queues can be monitored. And with it, know how to manage your message queues. Whether to see the servers that are active, the amount of messages that there are or the space they occupy, among other things. With this it is much easier to make an early detection of possible problems.

From this monitoring, we can have a much clearer idea of how to improve the efficiency of these message queues, to make better use of them and get the full potential possible.

In our online demo, in which we use Grafana, you can see how useful it is to see the data in an orderly and concise way. And as this way of visualizing the data is a help to see if your company is performing one hundred percent.