Open
Description
Search before asking
- I searched the issues and found no similar issues.
Component
Transforms/universal/fdedup
What happened + What you expected to happen
From here : https://github.com/IBM/data-prep-kit/blob/dev/transforms/universal/fdedup/fdedup_python.ipynb
result = Fdedup(input_folder='test-data/input',
output_folder='output',
contents_column= "contents",
document_id_column= "int_id_column",
num_permutations= 112,
num_bands= 14,
num_minhashes_per_band= 8,
operation_mode="filter_duplicates").transform()
transform
should return 0 for success or any error code
Reproduction script
https://github.com/IBM/data-prep-kit/blob/dev/transforms/universal/fdedup/fdedup_python.ipynb
Anything else
No response
OS
Ubuntu
Python
3.11.x
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Activity