Apache Kafka - 01

 What is Kafka?

• Event and Message streaming platform
• Yes, It is open-source and freely available. There are some commercial availabilities also. 
• Producers( AKA publishers) push messages/events to Kafka
• Consumers( AKA subscribers) listen & receive messages/events

Publisher ?? Subscriber ??  Don't worry. Let's take a look at it before digging deep. 😊

Publisher Subscriber Pattern.

Consider below case. A sender sends message to the receiver. 

• Sender needs to know the receiver address ( means receiver address should configure in Sender side)
• Both parties should agreed upon common protocol/format for data sending and receiving. 
  

Now see this case, 

Now 3 receiver addresses should configure in sender side. 
And if you need to add a new one, then sender side also has a change to be done. 

Imagine multiple sender sending multiple receivers , 
Yes. the configuration is a mess now. It is not easy to add a new receiver because there can be many senders that needs changes. 😟

Now see how it is solved in the top most diagram with Kafka. 

Senders only needs to know about the queue. And receivers only needs to know about the queue. So Easy to manage.  And Easy to add new receivers and senders ( Easy to scale )

Now, There are few key points that needs to be highlighted like , how Kafka messages looks like , How it is persisted in the Kafka , Concepts like Broker, Partitioning, Topics , Logs. etc. 

But first let's set up Kafka first in our machine using docker and Let's discuss those things while practicing. Let's meet in a new post , Bye.