Kafka is a message streaming platform that uses publish and subscribe mechanism to stream the records. Kafka was developed by linkedIn which later was donated to apache. It is a open source platform widely used for building real-time data pipelines and streaming applications.
Before the introduction of messaging systems like Kafka, most applications communicated directly in a tightly coupled manner.
The connection was supposed to be built from each source node to destination. This caused rigid and fragile interactions between the source and destination. To overcome these difficulties messaging system was developed which significantly reduced the rigid interactions.
Messaging System:
A messaging system is a middleware component responsible for transferring data between applications. It helps applications exchange information efficiently without getting burdened by the complexities of data transmission, timing, or direct system dependencies. This allows developers to focus on business logic while the messaging system handles reliable data delivery.
- Point to point communication
Message persisted in queue.
Message can be consumed by one receiver.
No time dependency laid for the receiver to receive message.
After the message is sent an acknowledgement to sender.
- Publish-Subscribe communication
Message persisted in topic.
Message can be consumed by any number of receivers.
Time dependency exist for consumers to consume.
After message is sent no acknowledgement to sender.
Apache Kafka has transformed the way modern systems handle data exchange, moving from tightly coupled, direct communication to a more scalable, decoupled, and resilient messaging approach. Its ability to handle real-time data streams, ensure durability, and support both point-to-point and publish-subscribe patterns makes it a powerful backbone for event-driven architectures.
SOCIAL SHARE CARD GENERATOR