### Step 4: Introducing Kafka (Traffic Shaping + Async + Decoupling)

Redis successfully deducts inventory
→ Send Kafka message "order_created"
→ Kafka consumer asynchronously writes to MySQL (orders / order_items)
→ After payment completion, send "payment_completed"

Kafka solves:
- Traffic shaping: Redis responds quickly, slow operations go to queue
- Decoupling: Orders, payments, and inventory synchronization are independent
- Reliable delivery: Messages are not lost

### Step 5: Kafka Duplicate Consumption Problem

First, Kafka does have a "Consumer Group" mechanism:
A message will only be received by one consumer in a group.
For example, if you have a group with three consumers (C1, C2, C3),
Kafka will assign messages from different partitions to different consumers,
and messages from the same partition will not be consumed by two consumers simultaneously.

Sounds fine, right?
But — in actual operation, "duplicate processing" situations occur for several typical reasons 👇

| Scenario | Description | What it causes |
|---|---|---|
| ✅ Kafka guarantees "at least once" | After processing a message, the consumer must send an "ack" (acknowledgment) to Kafka | If the ack fails to send (network jitter, program crash), Kafka will resend the message |
| 🔄 Rebalance | When a new consumer joins, leaves, or crashes in the group, Kafka will reassign partitions | During rebalance, old consumers may not have committed offsets, and new consumers will consume the same messages again |
| 💥 Program crash and restart | Consumer crashes mid-processing, offset not yet committed | After restart, it will consume the same message again from the old offset |

> So **Kafka does not guarantee "consume exactly once" (Exactly Once)**,
> Its default mode is **"At-Least Once"**.
> This means: the same message may be **processed repeatedly**!

### Step 6: Idempotency & Lock

To prevent duplicate consumption from causing "duplicate orders", "duplicate deductions", "inventory errors" and other issues,
we need to add another layer of protection at the consumer logic layer.

That is:
✅ Messages can be received repeatedly,
❌ But core business logic must never be executed repeatedly.

This relies on "idempotency" and "locks".

#### 6-1: Unique ID Deduplication (Idempotency)
For example, the message contains an orderId:
Before execution, first check if this order has already been processed.

Approach:
1. Use database unique constraint:
`INSERT IGNORE INTO orders (order_id, user_id, ...) VALUES (...)`
- If duplicate insertion occurs, the database automatically ignores it.

2. Or use Redis:

	if (await redis.setnx(`order:${orderId}`, 1, 'EX', 3600)) {
	    // First processing successful, execute
	} else {
	    // Duplicate message, ignore
	}

This way, even if Kafka sends the same message again, it won't be executed repeatedly.

#### 6-2: Distributed Lock (Prevent Concurrent Execution)
For example, two consumers **almost simultaneously** receive the same message (possibly due to Kafka retry, Rebalance instant concurrency),
We use a Redis lock to ensure only one can proceed:

const lockKey = `lock:order:${orderId}`;

// SETNX only succeeds when key doesn't exist
if (await redis.setnx(lockKey, 1, 'EX', 10)) {  
  await processOrder(orderId);  // ✅ Process order logic
  await redis.del(lockKey);     // Release lock after completion
} else {
  // 🚫 Another consumer is already processing this order
  // Skip this consumption
}

##### 6-2-1: Lock Auto-Renewal (Watchdog Mechanism)
Redis distributed lock (using `SETNX + EX`) has an expiration time. What if the lock hasn't finished executing but the key expires? The lock is automatically released by Redis. Could this cause **another consumer to acquire the lock and execute repeatedly**?

After a task acquires the lock, start a background "watchdog thread".
Every interval (e.g., every 5 seconds), refresh the lock's expiration time.
As long as the task is still running, the lock won't expire.
Once the task crashes or the process hangs, the watchdog also stops, and the lock naturally expires and releases.

// Pseudocode: Redisson-style
const lock = await redisson.getLock(lockKey);
await lock.lock();              // Automatically includes watchdog
try {
    await processOrder(orderId);
} finally {
    await lock.unlock();
}

This avoids the problem of "task execution taking too long causing lock auto-expiration".

### Step 7: Kafka Consumption Flow (Including Idempotency & Lock)

Summary logic relationship diagram

① Producer sends message
    ↓
② Kafka Broker receives and stores message
    ↓
③ Consumer pulls message from Kafka (orderId=123)
    ↓
④ [Redis Distributed Lock] 
       ├─ Try to acquire lock with key=order:123
       ├─ If lock is already held → indicates someone else is processing the same order → skip current consumption
       └─ If lock acquisition succeeds → continue to next step
    ↓
⑤ [Idempotency Check]
       ├─ Query database (or idempotency record table / Redis)
       ├─ If orderId=123 has been processed → directly release lock and skip
       └─ Otherwise record "processing orderId=123"
    ↓
⑥ processOrder()
       ├─ Create order, deduct inventory, send payment request and other core business logic
       ├─ Ensure any exceptions during the process can rollback or mark failure
    ↓
⑦ [Commit Offset (ACK)]
       ├─ Confirm to Kafka that message has been successfully processed
       └─ Broker deletes/marks this message as consumed
    ↓
⑧ [Release Redis Lock]
       └─ Delete lock:order:123

| Step | Protection Mechanism | Purpose |
|---|---|---|
| Kafka retry mechanism | at least once | May cause duplicate consumption |
| Idempotency check | Redis SETNX / DB unique | Avoid duplicate execution |
| Distributed lock | Redis SETNX EX | Avoid multiple processes processing simultaneously |
| offset commit | ack mechanism | Tell Kafka consumption is complete |

> Idempotency ensures "duplicate messages don't execute repeatedly"
> Lock ensures "not executed by two consumers simultaneously"

Step 5: Introducing Async, Message Callbacks, and Eventual Consistency

Add complete chain of payment, callbacks, and inventory rollback.

- Async task queue (e.g., Node.js background worker or Kafka consumer)
- Idempotent design (payment duplicate notifications don't affect results)
- Data consistency (compensation mechanism / Outbox pattern)
- State machine design: `PENDING → PAID → CONFIRMED → SHIPPED → REFUNDED`

The system is not instantaneously consistent, but _eventually consistent_, and must be traceable and repairable.

###Next Chapter
Ticket-Craze: Eventual Consistency - https://hackmd.io/@chaodotcom/ry5ZJRi1Wx