# Kafka
###### tags: `Kafka` `GNU` `Linux` `Message Queue` `Event Driven` `Distributed System` `Server` `System Design` `Software` `API`

A data pipeline system that is designed for "distributed" & "scalible" producer/consumer models, leveraging sequential reads, effective partitioning, batching, compression, & zero-copy operations.

## Resources

[What is Event Driven Architecture (EDA)? - IBM Technology](https://www.youtube.com/watch?v=o2HJCGcYwoU)
[What is Kafka? - IBM Technology](https://www.youtube.com/watch?v=aj9CDZm0Glc)
[Apache Kafka in 6 minutes - James Cutajar](https://www.youtube.com/watch?v=Ch5VhJzaoaI)
[System Design: Why is Kafka fast? - ByteByteGo](https://www.youtube.com/watch?v=UNUz1-msbOM)

## Event-Driven Message System

~ scalable / distributed / resillient

### Example

microservices / event log storage backup, analysis <-> event backbone <-> user-ends sub to pub

:::info
**feasible functions such as:**
- event logging & analysis
- streaming processing
- comunication with pub-sub instead of connection to EVERY microservices / end-points
:::

### Message Queue


[為什麼需要 Message Queue?](https://medium.com/starbugs/讓任務排隊吧-message-queue-1-de949e274c43)

### Streaming Sequentially

producers -> brokers in partitions -> comsumers

comsumers follow their record in order to tracing & blocks expired in time