Twitter Streaming

Real-time Twitter hashtag trending and sentiment analysis using Kafka, Spark Streaming and Flask.

Setup

• Install Zookeeper for Kafka
• Install Kafka on all servers
• Install Spark pip install pyspark==2.2.3
• Install Twitter API pip install tweepy
• Get Twitter API keys

Run in local machine

• Run Zookeeper in Standalone mode

  bin/zkServer.sh start

• Start Kafka in your server

  kafka-server-start /usr/local/etc/kafka/server.properties

• Create a Kafka topic

  kafka-topics --create --zookeeper localhost:9092 --replication-factor 2 --partitions 4 --topic china

• Put your API key in twitter_app.py

• Fetch real-time Twitter feeds

  python twitter_app.py

• Get top trending hashtags

  python hashtag_app.py

• Run real-time sentiment analysis

  python sentiment_analysis/classifier.py

• Run dashboard

  python dashboard/dashboard_app.py

Demo

track='china'