Skip to content

Commit cd1b9c4

Browse files
committed
Update validate.py
1 parent 1a6adc1 commit cd1b9c4

1 file changed

Lines changed: 7 additions & 34 deletions

File tree

pointblank/validate.py

Lines changed: 7 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -735,11 +735,6 @@ def get_data_path(
735735
return tmp_file.name
736736

737737

738-
# =============================================================================
739-
# Utility functions for processing input data (shared by preview() and Validate class)
740-
# =============================================================================
741-
742-
743738
def _process_data(data: FrameT | Any) -> FrameT | Any:
744739
"""
745740
Centralized data processing pipeline that handles all supported input types.
@@ -749,6 +744,7 @@ def _process_data(data: FrameT | Any) -> FrameT | Any:
749744
sequence of transformations to handle different data source types.
750745

751746
The processing order is important:
747+
752748
1. GitHub URLs (must come before connection string processing)
753749
2. Database connection strings
754750
3. CSV file paths
@@ -758,41 +754,18 @@ def _process_data(data: FrameT | Any) -> FrameT | Any:
758754
----------
759755
data : FrameT | Any
760756
The input data which could be:
761-
- A DataFrame object (Polars, Pandas, Ibis, etc.)
762-
- A GitHub URL pointing to a CSV or Parquet file
763-
- A database connection string (e.g., "duckdb:///path/to/file.ddb::table_name")
764-
- A CSV file path (string or Path object with .csv extension)
765-
- A Parquet file path, glob pattern, directory, or partitioned dataset
766-
- Any other data type (returned unchanged)
757+
- a DataFrame object (Polars, Pandas, Ibis, etc.)
758+
- a GitHub URL pointing to a CSV or Parquet file
759+
- a database connection string (e.g., "duckdb:///path/to/file.ddb::table_name")
760+
- a CSV file path (string or Path object with .csv extension)
761+
- a Parquet file path, glob pattern, directory, or partitioned dataset
762+
- any other data type (returned unchanged)
767763

768764
Returns
769765
-------
770766
FrameT | Any
771767
Processed data as a DataFrame if input was a supported data source type,
772768
otherwise the original data unchanged.
773-
774-
Examples
775-
--------
776-
>>> # GitHub URL
777-
>>> data = "https://github.com/user/repo/blob/main/data.csv"
778-
>>> df = _process_data(data)
779-
780-
>>> # Database connection string
781-
>>> data = "duckdb:///path/to/database.ddb::table_name"
782-
>>> df = _process_data(data)
783-
784-
>>> # CSV file
785-
>>> data = "data.csv"
786-
>>> df = _process_data(data)
787-
788-
>>> # Parquet file/pattern
789-
>>> data = "data/*.parquet"
790-
>>> df = _process_data(data)
791-
792-
>>> # Already a DataFrame - returned unchanged
793-
>>> import polars as pl
794-
>>> df = pl.DataFrame({"a": [1, 2, 3]})
795-
>>> result = _process_data(df) # Same as input
796769
"""
797770
# Handle GitHub URL input (e.g., "https://github.com/user/repo/blob/main/data.csv")
798771
data = _process_github_url(data)

0 commit comments

Comments
 (0)