Skip to content

README.md

Latest commit

 

History

History
30 lines (24 loc) · 1.25 KB

README.md

File metadata and controls

30 lines (24 loc) · 1.25 KB
Raw

Elon Musk in the eyes of twitter users -Sentiment Analysis Project using (NLP).

Sep 5, 2023

Project/Goals

This is a Sentiment Analysis Project using Natural Language Processing (NLP) Techniques, PySpark. The topic is about Elon Musk. I felt it would be a good idea to measure the people’s opinion and to obtain insights into how Twitter users felt about Elon Musk. The data is collected (328,766) for only two days (Nov 21 - Nov 22), 2022.

Tools

Python Packages:

  • Tweepy: to create a stream and listen to the live tweets;
  • Flair: to do simple sentiment analysis on tweets.

Data Processing Engine:

  • Apache Spark (pyspark - databricks);

Cloud Platform: AWS; AWS Services:

  • S3: Amazon Simple Storage Service
  • Athena: a serverless, interactive query service to query data and analyze big data in Amazon S3 using standard SQL
  • QuickSight: a cloud-scale business intelligence (BI) service that you can use to deliver easy-to-understand insights.

Process

  • Data Processing
  • Creating Sentiment
  • Exploratory Analysis of the Data
  • Data Modeling
  • Interpreting the Data

Results

  • Flair is the best library that can be able to guess the meaning
  • Logesstic Regression is the best model that can predict the daa with Accuracy Score: 0.9395 ROC-AUC: 0.9734