This repository was archived by the owner on Aug 17, 2024. It is now read-only.
This repository was archived by the owner on Aug 17, 2024. It is now read-only.
[FEATURE] Merge dataframes with different columns #112
Open
Description
Is your feature request related to a problem? Please describe.
I'd like to merge DataFrames with different columns.
Describe the solution you'd like
I'd like to have a df1.merge(df2)
way to automatically merge two dataframes, even if a column is in df1 but not in df2, filling it with
Describe alternatives you've considered
Here is a snippet from @lmeyerov I found (and completed) on issue #15, that makes just what I want :
function unionDFs(a, b, fill='n/a') {
// Merge two dataframes with different columns
const aCols = a.listColumns(); // this line was missing on lmeyerov's original snippet
const bCols = b.listColumns(); // this line was missing on lmeyerov's original snippet
const aNeeds = b.listColumns().filter((v) => aCols.indexOf(v) === -1);
const bNeeds = a.listColumns().filter((v) => bCols.indexOf(v) === -1);
const a2 = aNeeds.reduce((df, name) => df.withColumn(name, () => fill), a);
const b2 = bNeeds.reduce((df, name) => df.withColumn(name, () => fill), b);
return a2.union(b2);
}