Needs of Message Broker
What is Message Broker
“Message broker is an inter-application communication technology to help build a common integration mechanism to support cloud-native, microservices-based, serverless, and hybrid cloud architectures.”
Sounds confusing? Let’s simplify it. Actually, The message broker is a technology that helps to communicate and exchange information between applications, systems, and services. The job of the message broker is translating messages between formal messaging protocols. The message broker is really helpful when you want to ensure the process is neither lost or duplicated. Let’s see some examples to understand it.
Transactional systems — If you are developing a video streaming website, it should have a way to publish new videos. Let’s assume, there are four different services to complete this process. Such as validation, compression, thumbnail generation, and publishing the video. Every service can have the role of sender and receiver at the same time. After consuming its task it can send a message to the broker that the job’s finished.
Controlling data feed — If you want to control the number of some entities in your system. This is a very simple task with message brokers that needs to control the limit of registered users.
Essential process — If you develop a payment processing system, you need to ensure the payment is sent only once. You can handle these transactions using a message broker and avoid the process is not lost and not duplicated.
Basic concepts in Message Broker
Let’s see how it works and basic concepts under the message broker.
The producer is an endpoint that sends any kind of data stored inside the message broker to distribute.
The consumer is an endpoint that asks the message broker for data(messages).
The queue is a data type that the message broker uses to store messages inside, with the logic of FIFO(First in First out).
The exchanger is a logical configuration or even entity, on top of the queues. It tells the message broker to create a group that consumers can listen to receive messages or producers can write to send messages.
Types of Message Broker
Point-to-point messaging
In point-to-point messaging, the message producer is called the sender, and the consumer is called the receiver. Communication happens between one sender and one receiver at a time. The message is stored in the queue until it starts the execution. Each message in the queue is sent to only one receiver and is consumed only once. For example, the payroll and financial transaction process can be identified as a point-to-point message. Because both senders and receivers need a guarantee that each payment will be sent once and once only.
Publish/subscribe messaging
In publish/subscribe messaging, message producers are called publishers, and consumers are called subscribers. The message is stored in the topic until it the subscriber ready to receive it. This message distribution can be considered as a broadcast-style distribution. Because one or many publishers can publish on the same topic and one or many subscribers can receive the same message. The publisher doesn't know about the subscribers. For example, an airline (publisher) disseminates updates about the landing times or delay status of their flights, and many parties (subscribers) could receive the information.
Advantages of using Message Brokers
- Provided communication between services that may not be running at the same time. For example, the producer can send messages even though the consumer is active or not.
- Perform on any service even it runs independent from the platform and is implemented by different languages.
- Improved reliability by guaranteeing the transmission of messages. Because message brokers offer a redelivering mechanism.
- Increased system performance by introducing asynchronous processing. High-consuming tasks can be distributed to separate processes.
Examples for Message Brokers
There are many message brokers used in applications. For example, RabbitMQ, Apache Kafka, Redis, Amazon SQS can consider as few popular message brokers.
Apache Kafka
Apache Kafka is an open-source message broker originally developed by LinkedIn in early 2011. It is capable to handle millions of events. Because it utilizes stream-processing to provide a low-latency and high-throughput platform that handles real-time data feeds. It supports all major programming languages like Java, C/C++, PHP, Python, .NET, Ruby, JavaScript, Swift, Go.
Kafka is used as a scalable, publish-subscriber messaging platform. Therefore, it supports building real-time data pipelines and streaming apps. When comparing the performance, this is on top of messaging brokers.
For more information:-
RabbitMQ
RabbitMQ is an open-source message queue software that is written in Erlang. It was released in 2007. It can consider as lightweight and easy to deploy on-premises and in the cloud. RabbitMQ supports most of the common programming languages like Java, PHP, Python, .NET, Ruby, JavaScript, Swift, Go.
RabbitMQ supports both point-to-point and publish/subscribe messaging methods. It implemented Advanced Message Queuing Protocols (AMQP) and it is designed to support complex routing logic.
For more information:-
Amazon SQS
SQS is a fully managed, distributed messaging service provided by Amazon. It was introduced in 2006. SQS supports both point-to-point and publish/subscribe messaging methods. It enables to work with decouple and scale microservices, distributed systems, and serverless applications.
For more information:-
Java Messaging Service (JMS)
JMS is a Java-based API. It supports both Point-to-Point messaging and publish/subscribe messaging methods. It is a messaging standard containing Java components that are designed to create, send, receive, and read messages. The message distribution can consider as loosely coupled, asynchronous, and reliable. It is supported by a wide range of enterprise messaging products.
For more information:-
Message brokers vs APIs
RESTful APIs are commonly used for communications between the client and the server. It defines a set of principles and constraints that developers can follow when building web services. RESTful APIs use the HTTP protocol to communicate. The communication between client and server is provided by requests and responses. So the RESTful API is best used in situations that call for synchronous requests and responses. This means that services making requests via RESTful APIs must be designed to expect an immediate response.
Message brokers enable asynchronous communications between services. So, the sending service doesn't need to wait for the receiving service’s reply. In addition, the message broker has a proper scaling system since a publish/subscribe messaging pattern can easily support changing numbers of services. Message brokers also keep track of consumers’ states.
I hope you learn something from this article. If I missed any points, let me know your suggestion in the comment section.
