General Overview

To start program, type make

Note: Check Python dependencies in twitter_miner, twitter_main, twitter_nlp, twitter_processing, twitter_datavis are all installed.

You'll then be presented with two options: mine or visuals.

mine: this selection will direct you to a place to help you mine tweets.

visuals: this selection will direct you to a place to generate visualizations based on mined data.

All terminal UI is contained in twitter_main.py for clarity.

Twitter Miner

This program helps you compile large Twitter datasets. It is written in Python 3.6.

Requirements: Python 3.6

1. Activate the Python environment source env/bin/activate

2. The following is the terminal command to run the program /...Current Working Directory.../env/bin/python3.6 /...Current Working Directory.../twitter_main.py

3. Follow the directions stated in the std. output in the terminal. It will walk you through the steps of running the program.

4. There are a multiple features that are available as of now: obtaining tweets from a single user, obtaining tweets from a list of users, and performing search queries

Twitter Watson

File: twitter_watson.py

This file generates emotional scores with IBM Watson. After generating a file with Twitter Miner above, change the file variable in the script to the name of the generated file from Twitter Miner. Remember to edit the input and output location in the read_csv(), to_csv() lines as well.

Then, run program: python3 twitter_watson.py

Note: To run this program, an IBM Watson free account is required. Set up your own account here: link. Then, set up a natural language understanding service. Paste the api key and url from the service into the script in the "Authentication" section.

Twitter Datavis and Twitter Processing

File: twitter_datavis.py, twitter_procsesing.py

The datavis file provides the backend for the visuals selection. The processing file stores some of the filtering/cleaning functions of the datavis script.

The datavis file contains a class which holds all the visualization functions. The __init__ function initializes parameters, sorts the entries by date, cleans the data (function is in processing), and then calls all the requested visualizations in a loop. editDataframe() is called within __init__ to allow the user to constrain the given data by sentiment, subjectivity, and date range.

Current Visualization Options: Bar, Stacked Bar, Barh, Pie, Radar Chart, Time Series, Word Cloud, Bar Chart Race, among more.

It's easy to create a new visualization. Create a new function within the class. Use self.df and a Python library (i.e. matplotlib, Plotly) to generate a desired visualizations. Remember to update the __init__ function and twitter_main.py whenever a new function is added.