Kafka

Data Analytics with Kafka - KAFKA#IOT

In this project a data analysis was performed on IoT data with Kafka (https://kafka.apache.org/).

The data to be monitored are provided by a Filippetti Device Simulator.

Prerequisites

• Ubuntu 18.04 64 bit
• Java 8

Installation

• JavaFX
• MQTT broker
• Mosquitto
• Simulator
• Confluent Platform

If there are problems with JavaFX you can see a possible solution on this post https://stackoverflow.com/questions/56166267/how-do-i-get-java-fx-running-with-openjdk-8-on-ubuntu-18-04-2-lts.

Usage

• Download and activate Mosquitto to running services
• Start mqtt data simulator

  java -jar simulator.jar safe_Simulator.json

  (make sure that you are in the simulator folder)
• Launch Confluent from any directory with the underneath command:

  confluent local services start

• Navigate to the Control Center web interface at http://localhost:9021/ and select your cluster
  
  
  
  
• Create the topics you need
• Install source and sink connectors you need with the command

  confluent-hub install confluentinc/kafka-connect-mqtt:latest

• Add source and sink connector
  
  
  
  
• Create and Write to a Stream and Table using KSQL

Authors

• Diego Diomedi - diegodiome
• Lucia Passeri - LuciaPasseri