feat: add dataframe duolicated issue - #667

[RahulDas-dev]

This merge request adds a new [duplicated()] method to the DataFrame class that identifies duplicate rows within a DataFrame. This functionality is essential for data cleaning and exploration workflows.

Resolve the issue - #667

Features

• Identifies duplicate rows in a DataFrame based on specified columns
• Returns a Series of boolean values marking duplicate entries
• Supports flexible options for handling duplicates:
  • keep: 'first' - Mark duplicates except for the first occurrence (default)
  • keep: 'last'- Mark duplicates except for the last occurrence
  • keep: false - Mark all duplicates
    Allows focusing on specific columns with the subset option

Implementation Details

• Optimized to handle large datasets efficiently with a hash-based approach
• Comprehensive input validation for better error handling
• Well-documented with JSDoc comments and examples

// Create a DataFrame with duplicate rows
const df = new DataFrame({
  'A': [1, 2, 2, 3, 3],
  'B': ['a', 'b', 'b', 'c', 'c']
});

// Find duplicates keeping first occurrence (default)
const dups = df.duplicated();
// Returns: [false, false, true, false, true]

// Find duplicates keeping last occurrence
const dupsLast = df.duplicated({ keep: 'last' });
// Returns: [false, true, false, true, false]

// Find duplicates based on specific columns
const dupsSubset = df.duplicated({ subset: ['B'] });
// Returns: [false, false, true, false, true]