Oct 8, 2018

Textual description of firstImageUrl

Setup Kafka and Zookeeper with brew(homebrew) on Mac operating system: Use Kafka tool(UI) to Publish message and monitor Kafka topic message.

                No comments   
Kafka is publish subscribe messaging system which are most commonly used in asynchronous work flow. Homebrew is a software package management system that simplifies the installation of software on Apple's macOS operating system. Using brew Kafka and Zookeeper can installed & setup with ease.

Install zookeeper using brew(homebrew):
➜ brew install zookeeper
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core, homebrew/cask).
==> New Formulae
bundletool         geant4             i2pd               mallet             opensubdiv         opentracing-cpp    stanford-corenlp   tdlib
==> Updated Formulae
....
.....
==> Downloading https://homebrew.bintray.com/bottles/zookeeper-3.4.12.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring zookeeper-3.4.12.high_sierra.bottle.tar.gz
==> Caveats
To have launchd start zookeeper now and restart at login:
  brew services start zookeeper
Or, if you don't want/need a background service you can just run:
  zkServer start
==> Summary
🍺  /usr/local/Cellar/zookeeper/3.4.12: 242 files, 32.9MB
Where is installation directory of zookeeper : /usr/local/Cellar/zookeeper

Install Kafka Using brew(hombrew): 
➜  brew install kafka
Updating Homebrew...
==> Downloading https://homebrew.bintray.com/bottles/kafka-2.0.0.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring kafka-2.0.0.high_sierra.bottle.tar.gz
==> Caveats
To have launchd start kafka now and restart at login:
  brew services start kafka
Or, if you don't want/need a background service you can just run:
  zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties
==> Summary
🍺  /usr/local/Cellar/kafka/2.0.0: 160 files, 46.8MB
Where is installation directory of Kafka : /usr/local/Cellar/kafka

Kafka Producer and Consumer(Command line tool)

Create Kafka topic: Create topic "topic-devinline-1" with replication factor 1 and partition 1
➜  bin pwd
/usr/local/Cellar/kafka/2.0.0/bin
➜  bin sh kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic-devinline-1 
Created topic "topic-devinline-1".

Start producer: 
➜  bin kafka-console-producer --broker-list localhost:9092 --topic topic-devinline-1                       
>Hello, Fist message
>Bye, Message
>
Start consumer: 
➜  bin kafka-console-consumer --bootstrap-server localhost:9092 --topic  topic-devinline-1 --from-beginning
Hello, Fist message
Bye, Message

Kafka UI tool(Kafka Tool)

An UI tool to analyse and post/consume messages in topic. Download Kafka tool
As we have setup Zookeeper and Kafka locally, we can use following configuration to setup Kafka tool.

Analyse Kafka Topic through UI:
Open Topic "topic-devinline-1" and partition-0. Click green play button and find both message posted earlier.

Post Message from UI tool in Kafka topic:
1. Select Partition-0, Select Data tab and Click + 
2. Post message in text-box and click Add.
3. Click on green-play button and message posted is available in topic.


Note: If you find text box for Key also, provide a key along with message. It can be configured in preferences.
Kafka tool -> Preferences -> topics -> Default content type (Key = No Key)
Location: Bengaluru, Karnataka, India