---
title: "Real time communication of microservices with Apache Kafka"
disqus: JBeanny
---

By: Yim Sotharoth

Real time communication of Nodejs Application with Kafkajs
<img
src="https://kafka.js.org/img/kafkajs_circle.png"
alt="kafka-png"
width="30"
/>
===

## Table of Contents

[TOC]

## Installation

_I create 2 folders one is called `kafka-producer` and another one is called `kafka-consumer` after that I go inside each of the 2 folders and do the following:_

:::info
_firstly create a node project_

$ npm init --y

_I named folder as `kafkajs` ( or anything you want ) so I cd in my folder_

> In `kafkajs` folder install a few libraries with the following command

$ npm install kafkajs

:::

:::warning
Before Getting started, make sure you have the kafka server up and running.
If not, then you can go the our repository to grab the `docker-compose.yaml`
file and then run the following command:

docker compose up -d

[Project Repository](#Demo1)
:::

## First we are going to build the message sender service (Producer)

### kafka producer

#### Producer configuration

_I create `kafka-config` folder in `kafka-producer` service and then create a `producer.js` file with following code_:

const { Kafka } = require("kafkajs");

const clientId = "nodejs-kafka-metaphorlism";
// kafka broker
const brokers = ["localhost:9092"];
// define topic name
const topic = "messages-topic";

const kafka = new Kafka({ clientId, brokers });
const producer = kafka.producer();

const produce = async () => {
  await producer.connect();
  let i = 0;

  // send a message to a kafka topic every 1000ms(1s)
  setInterval(async () => {
    try {
      await producer.send({
        topic,
        messages: [
          {
            key: String(i),
            value: "Hello from Metaphorlism: " + i,
          },
        ],
      });
      console.log("message produced: ", i);
      i++;
    } catch (err) {
      console.error("error: " + err);
    }
  }, 1000);
};

module.exports = produce;

_After done creating producer configuration, I create a `index.js` file within the main directory of `kafka-producer` folder with the following code in order to invoke the producer to use:_

const produce = require("./kafka-config/producer");

produce().catch((err) => {
  console.error("Producer error: ", err);
});

***after implementing all of this , you will be able to send the message to the `messages-topic` topic. But we want another service to work as a message consumer from the `messages-topic` and that service is going to be named as `kafka-consumer`***

## Now we are going to build the message receiver service

### kafka consumer

#### consumer configuration

_I create `kafka-config` folder in `kafka-consumer` service and then create a `consumer.js` file with following code_:

const { Kafka } = require("kafkajs");

const clientId = "nodejs-kafka-metaphorlism";
// kafka broker
const brokers = ["localhost:9092"];
// define topic name
const topic = "messages-topic";

const kafka = new Kafka({ clientId, brokers });

const consumer = kafka.consumer({ groupId: clientId });

// listening to events of messages-topic
const consume = async () => {
  await consumer.connect();
  await consumer.subscribe({ topic });
  await consumer.run({
    eachMessage: ({ message }) => {
      console.log(`message received : ${message.value}`);
    },
  });
};

module.exports = consume;

_After done creating producer configuration, I create a `index.js` file within the main directory of `kafka-consumer` folder with the following code in order to invoke the producer to use:_

const consume = require("./kafka-config/consumer");

consume().catch((err) => {
  console.error("Consumer error: ", err);
});

:::success
After implementing both of the services, we will be able to see the magic of KafkaJS with NodeJS 😉😉
:::

Project Repository: https://github.com/metaphorlism/nodejs-with-kafkajs

## Contact Us

- :mailbox: [e-mail]
- [GitHub](https://github.com/metaphorlism)
- [Facebook Page](https://www.facebook.com/Metaphorlism)
- [Instagram: Metaphorlism](https://www.instagram.com/metaphorlism/)