This project demonstrates a complete Natural Language Processing (NLP) pipeline for detecting sarcasm in text headlines. It compares traditional machine learning approaches using different text vectorization techniques: TF-IDF with unigrams vs. N-grams (bigrams and trigrams) with CountVectorizer.
sarcasm_detection.ipynb: The main Jupyter Notebook containing the code, analysis, and visualizations.sarcasm.json: A JSON dataset containing headlines and their sarcasm labels (0 for non-sarcastic, 1 for sarcastic).
To run this notebook, you need Python installed along with the following libraries:
pip install pandas numpy scikit-learn matplotlib seabornAll required libraries are part of the standard data science stack - no additional dependencies needed!
- Clone this repository.
- Ensure
sarcasm.jsonis in the same directory as the notebook (or update the path in the notebook). - Open
sarcasm_detection.ipynbin Jupyter Notebook, JupyterLab, or VS Code. - Run all cells to execute the pipeline.
- Data Exploration: Distribution of classes, class balance analysis.
- Preprocessing: Text cleaning, lowercasing, punctuation removal.
- Vectorization:
- TF-IDF (Term Frequency - Inverse Document Frequency) with unigrams
- CountVectorizer with N-grams (unigrams, bigrams, and trigrams)
- Modeling:
- Logistic Regression
- Support Vector Machine (SVM)
- Random Forest
- Evaluation: Accuracy score, Confusion Matrix, Classification Report.
- N-grams capture word sequences and phrase patterns (e.g., "yeah right", "oh great")
- Simple methods can be very effective for sarcasm detection
- No need for complex deep learning models for good baseline performance
- Add character n-grams to capture spelling patterns and informal language
- Implement feature engineering (sentiment scores, punctuation patterns, capitalization)
- Use pre-trained embeddings (GloVe, FastText) for semantic understanding
- Try ensemble methods combining multiple models
- Experiment with deep learning approaches (LSTM, Transformers) for comparison