Merge Mercury Graph with GraphFrames

[rjurney]

Dearest peeps,

I wanted to open a ticket to discuss merging this library with graphframes/graphframes. It seems like we have common goals and this could combine our communities to give us super-powers :)

So GraphFrames was all but dead, GraphX was being deprecated and so several of us stepped up to work on it. Recently we've had 3-4 new regular contributors join the project. @SemyonSinchenko just added Spark Connect Suport, for example. I did a new Motif Finding Tutorial and the project is moving. Sem is so productive it is amazing.

So good things... at the same time we lack many features that Mercury Graph has. Your Node2Vec implementation is GREATLY appreciated. I want this and struc2vec in GraphFrames and it feels best if it wasn't rewritten.

Some issues:

• GraphFrames is a Scala first project, with Python wrapping Scala features. This improves performance but may not match the skills of Mercury Graph. We do have these skills on the GraphFrames side and could support the Scala side of additions. We could also support Python only features until we port them to Scala.
• Efforts are underway to add similar features like Node2Vec
• Thinking...

[sbasaldua]

Hi again,

Thanks for reaching out and your PRs! We’re excited to see the momentum behind GraphFrames and really appreciate the idea of working together. There’s definitely a lot of synergy, and we see real value in bringing complementary strengths to the table.

That said, mercury-graph is the result of years of in-house development, with a lot of functionality refactored and designed with specific platform goals in mind. We recently (end of 2024) dedicated some time to modernize everything and release it as an open-source project. Our team has many other projects (inner-source and open-source) too, but we are very enthusiastic about the potential of Mercury Graph.

As part of a bank, we also need to consider our branding and strategic priorities—Mercury Graph is something we want to keep growing under its own name.

We’re still brainstorming how best to approach this, but we’d love to continue the conversation and see if there’s a way to work together that benefits both projects. Let’s keep the discussion going!

[SauronShepherd]

Looks like a quite interesting project to check out and get some ideas from. Thanks for open-sourcing it!