Skip to content

Possible data leakage: transformations fitted on test data #75

Description

@Shraddha6211

Hi! Thanks for the great tutorial, it was very helpful.

I noticed that during column transformations in Aam Zindagi portion, the test data is also being fitted
(e.g., using fit_transform on test data instead of only transform).

This could lead to data leakage, since test data should only be transformed using
parameters learned from the training https://github.com/campusx-official/100-days-of-machine-learning/blob/main/day28-column-transformer/day28.ipynb

Would it be better to:

  • fit transformations on training data only, and
  • apply transform on test data
    (possibly using a Pipeline / ColumnTransformer)?

Just wanted to point this out in case it helps future learners

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions