# AWS EC2 access AWS MSK **經過上篇設定完 AWS MSK後(https://hackmd.io/@kafka-exchange/H14x_N602),我們接著使用 AWS EC2 來創建與消費 MSK Topic** ## 1. 請在與 AWS MSK 相同 VPC 下創建 AWS EC2  ## 2. 進入EC2 開啟兩個網頁版的 Terminal **(兩個 Terminal 分別做為 AWS Producer,AWS Consumer)**  ## 3. 安裝 Kafka on EC2 ``` sudo yum -y install java-11 wget https://archive.apache.org/dist/kafka/{YOUR MSK VERSION}/kafka_2.13-{YOUR MSK VERSION}.tgz tar -xzf kafka_2.13-{YOUR MSK VERSION}.tgz wget https://github.com/aws/aws-msk-iam-auth/releases/download/v1.1.1/aws-msk-iam-auth-1.1.1-all.jar ``` ## 4. 建立 AWS MSK Topic (ZookeeperConnectString 換成 Plaintext address) ``` <path-to-your-kafka-installation>/bin/kafka-topics.sh --create --zookeeper ZookeeperConnectString --replication-factor 3 --partitions 1 --topic ExampleTopicName ``` ## 5. 在 ./kafka_2.13-2.8.1/bin 底下創建 users_jaas.conf 檔案 ``` vim users_jaas.conf ``` **users_jaas.conf 內容(username,password)換成自己的ASM secret:** ``` KafkaClient { org.apache.kafka.common.security.scram.ScramLoginModule required username="alice" password="alice-secret"; }; ``` ## 6. 設置環境變量 ``` export KAFKA_OPTS=-Djava.security.auth.login.config=<path-to-jaas-file>/users_jaas.conf ``` ## 7. 複製 truststore.jks 至 ./tmp(需要自己找一下檔案路徑) ``` cp /usr/lib/jvm/JDKFolder/jre/lib/security/cacerts /tmp/kafka.client.truststore.jks ``` ## 8. 在 ./kafka_2.13-2.8.1/bin 底下創建 client_sasl.properties 檔案 ```vim client_sasl.properties``` **client_sasl.properties內容:** ``` security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-512 ssl.truststore.location=<path-to-keystore-file>/kafka.client.truststore.jks ``` ## 9. 建立 Producer 輸出數據至 AWS MSK Topic **若想使用Kafka Client 請參考(https://hackmd.io/@kafka-exchange/rkFi8gXA3)** ``` <path-to-your-kafka-installation>/bin/kafka-console-producer.sh --broker-list BootstrapBrokerStringSaslScram --topic ExampleTopicName --producer.config client_sasl.properties ``` ## 10. 建立 Consumer 訂閱 AWS MSK Topic 數據 ``` <path-to-your-kafka-installation>/bin/kafka-console-consumer.sh --bootstrap-server BootstrapBrokerStringSaslScram --topic ExampleTopicName --from-beginning --consumer.config client_sasl.properties ```
×
Sign in
Email
Password
Forgot password
or
By clicking below, you agree to our
terms of service
.
Sign in via Facebook
Sign in via Twitter
Sign in via GitHub
Sign in via Dropbox
Sign in with Wallet
Wallet (
)
Connect another wallet
New to HackMD?
Sign up