Open
Description
Is there an existing issue for this?
- I have searched the existing issues
Problem statement
Some loose ideas on how to improve and provide additional syntax.
Proposed Solution
Monkey patching dataframe:
df = df.withColumnDQ(“errors”, F.col(“col1”)), dq=[is_not_null()])
df = df.withNotNullColumn(“errors”, F.col(“col1”))
df.withColumnRule(“errors”, *args)
Additional Context
No response