---
# System prepended metadata

title: Kafka MirrorMaker

---

# Kafka MirrorMaker

- Kafka Mirror is used for replication of messages from one kafka cluster to another.


## Working
- In simple terms kafka mirror is nothing but a producer and consumer combined together in one.
- When a message is produced to a topic in source cluster, it is consumed by kafka mirror and it produces the same message to the target cluster.


## Setup
- First we need to two config files to start kafka mirror maker.
- One provides information for consuming data from source cluster and the other to produce data to target cluster

- You need to point the consumer to the source cluster.
- For a new cluster we can use offset as earliest. But for an existing cluster use it as latest to avoid producer errors due to bad metadata.

**File - consumer.config**
```
bootstrap.servers=<source-kafka-cluster-ip:port>
exclude.internal.topics = true # To skip internal topics such as _consumer.offsets
auto.offset.reset=earliest # Change based on required offset
group.id=test-kafkamirror-consumer-group # Change the consumer group id as required
```

- And the producer to the targer cluster.

**File - producer.config**
```
bootstrap.servers=<target-kafka-cluster-ip:port>
acks=1
batch.size=100
max.request.size=524288000
```

### Start the Kafka Mirror

- To consume all the topics from kafka cluster use

```
/usr/bin/kafka-mirror-maker --new.consumer --consumer.config /path-to/consumer.config --producer.config /path-to/producer.config --whitelist ".*"
```

- To consume only to specific topic

```
/usr/bin/kafka-mirror-maker --new.consumer --consumer.config /path-to/consumer.config --producer.config /path-to/producer.config --whitelist "kafka-mirror-maker-topic"
```

- There are also options for black list of certain topics. For which please ref to official documentation.