Skip to content

[Bug] Fdedup (simpler API) transform does not return a success/error code #957

Open
@sujee

Description

@sujee

Search before asking

  • I searched the issues and found no similar issues.

Component

Transforms/universal/fdedup

What happened + What you expected to happen

From here : https://github.com/IBM/data-prep-kit/blob/dev/transforms/universal/fdedup/fdedup_python.ipynb

result = Fdedup(input_folder='test-data/input',
  output_folder='output',
  contents_column= "contents",
  document_id_column= "int_id_column",
  num_permutations= 112,
  num_bands= 14,
  num_minhashes_per_band= 8,
  operation_mode="filter_duplicates").transform()

transform should return 0 for success or any error code

Reproduction script

https://github.com/IBM/data-prep-kit/blob/dev/transforms/universal/fdedup/fdedup_python.ipynb

Anything else

No response

OS

Ubuntu

Python

3.11.x

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions