Add high revenue, low profit analysis notebook

[Copilot]

Implements analysis pipeline to identify products and sub-categories generating high revenue but low/negative profit—a critical signal for pricing, discounting, or COGS issues.

Implementation

• Auto-detection: Uses Path.glob() to discover CSV files in data/ directory
• Flexible column mapping: Handles variations (Product/Product Name, Sub-Category/SubCategory, etc.)
• Dual aggregation: Analyzes both product and sub-category levels with total_sales, total_profit, n_orders
• Configurable thresholds: Flags items in top 20% sales + (bottom 20% profit OR negative profit)
• Visualization: Scatter plots with red highlighting and annotations for flagged items
• Export: Saves flagged items as CSVs to notebooks/outputs/
• Actionable guidance: Markdown sections with interpretation help and short/medium/long-term remediation steps

Example

# Configurable thresholds at top of analysis
HIGH_SALES_PERCENTILE = 80  # Top 20%
LOW_PROFIT_PERCENTILE = 20  # Bottom 20%

# Creates composite flag
flagged_strong = (
    high_sales_flag & 
    (low_or_negative_profit_flag | low_profit_bottom_pct_flag)
)

Test execution identified 1 product (Tables: $4,370 sales, -$212 profit) and 1 sub-category requiring immediate attention.

Files

• notebooks/high_revenue_low_profit_analysis.ipynb - Main analysis notebook (23 cells)
• .gitignore - Excludes notebooks/outputs/ directory

Original prompt

Create a new Jupyter notebook file at notebooks/high_revenue_low_profit_analysis.ipynb in the repository WebCraftPhil/superstore-sales-analysis. The notebook should implement an analysis to identify "High Revenue, Low Profit" products and sub-categories and include explanatory markdown. Requirements:

1. Notebook purpose and metadata

   • Title: "High Revenue, Low Profit Analysis (Very Strong Signal)"
   • Brief description of goal and steps.

2. Data loading

   • Auto-detect a CSV file in the repository's data/ directory (use glob) and load it with pandas.
   • Provide helpful error message if no CSV is found.

3. Column guidance and flexible mapping

   • Detect common column names (Product/Product Name, Sub-Category/Sub Category/SubCategory, Sales, Profit) and map them to variables.
   • Convert Sales and Profit to numeric and drop rows missing those values.

4. Aggregation

   • Aggregate by Product (if available) and by Sub-Category (if available) computing total_sales, total_profit, and n_orders.
   • Sort results by total_sales descending and show top rows.

5. Flagging logic (configurable)

   • Compute high sales threshold as top 20% by total_sales (configurable variable) and low profit threshold as bottom 20% by profit (configurable).
   • Create flags: high_sales_flag, low_or_negative_profit_flag (profit <= 0), low_profit_bottom_pct_flag, and flagged_strong (high_sales_flag AND (low_or_negative_profit OR low_profit_bottom_pct_flag)).

6. Visualizations

   • Scatter plot total_sales vs total_profit with flagged_strong highlighted and annotations for top flagged items (use seaborn/matplotlib).
   • Optionally create bar chart for flagged items.

7. Export

   • Save flagged product and sub-category CSVs to notebooks/outputs/ (create directory if necessary).

8. Actionable recommendations and next steps in markdown cells

   • Explain how to interpret flags and suggested next actions (investigate discounts, COGS, pricing; consider time windows; margin calculation, etc.).

9. Code/style

   • Use pandas, numpy, matplotlib, seaborn.
   • Include %matplotlib inline or appropriate plotting setup.
   • Set pandas display options to show enough columns.

10. Files to create in PR

• notebooks/high_revenue_low_profit_analysis.ipynb (the notebook implementing above with clear markdown and runnable code)

11. Pull request

• Create a branch named feature/high-revenue-low-profit-notebook
• Add the notebook file to that branch
• Open a pull request from feature/high-revenue-low-profit-notebook into main with title: "Add notebook for High Revenue, Low Profit analysis" and a description that summarizes the notebook purpose and what files were added.

Please implement the notebook with the cells and code consistent with the earlier conversation (include the full code cells and markdown described). Do not modify other files in the repository. Use main as the base branch.

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.