A Real-time Stream Processing App Example Exploring Twitter Sentiment and Weather Condition Correlation

Authors : Babak Badkoubeh, Brian White, Sebastian Alcaino, Vignesh Perumal

In this article, we will go through the process of building a distributed, scalable, fault-tolerant, microservice-oriented data pipeline using Kafka, Docker, and Cassandra. We utilize this architecture to explore correlation between Twitter sentiment analysis and the weather condition.

Example application preview

At a high level, the figure below depicts the application we will be implementing, which involves Kafka at its core and various microservices attached.

Microservice architecture with Kafka as a message queue
Microservice architecture with Kafka as a message queue
Microservice architecture with Kafka as a message queue

Feel free to skip to the second half of this article to find the step…

Vignesh Perumal

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store