Sentiment-EMR

Sentiment analysis for keywords over time using Pig and Elastic MapReduce

Team member: Fan Su (fs2488), Jingyi Guo (jg3421), Wei Cao (wc2467), Shijie Hu (sh3251)

Features

1. Acquire Tweet data for topic ‘news’ using Twitter Live Streaming API with timestamp and content
2. Sentiment filtering based on positive sentiment words dictionary
3. Get frequency of positive keywords using Pig running on Elastic Map Reduce for each timestamp data storing in AWS S3
4. Extra point: Generate a diagram in how word frequency involving over time

Tools used

1. Programming tools: Pig, AWS S3
2. API: Twitter Live and Streaming API, Elastic MapReduce