🎉 New Destination: NebulaGraph

[GrayFluex]

What

NebulaGraph is a high-performance, open-source distributed graph database. It excels at storing and querying large-scale graph data with millisecond latency. Airbyte lacks native support for syncing data to NebulaGraph, which prevents users within the graph database ecosystem from leveraging Airbyte's extensive library of sources to populate their graphs.

This PR introduces a new destination connector for NebulaGraph to bridge this gap, enabling seamless data integration from any Airbyte source into a NebulaGraph instance.

How

This PR implements the destination-nebulagraph, a new destination connector that writes data to NebulaGraph.
Key features of this implementation include:

Connection: Establishes a connection to a NebulaGraph instance using the provided host, port, username, and password.

Schema Mapping: Maps each Airbyte stream to a corresponding NebulaGraph Tag or Vertex. The schema of the Tag is automatically inferred from the stream's properties.

Data Writing: Converts Airbyte records into NebulaGraph vertices and inserts them using nGQL INSERT VERTEX statements. It handles data type conversions between Airbyte's JSON schema types and NebulaGraph's native data types.

Core Operations: Implements the required spec, check, and write operations.

Acceptance Tests: Passes all relevant Connector Acceptance Tests (CAT), ensuring reliability and correctness.

Review guide

For an efficient review, I recommend starting with the following files:

airbyte-integrations/connectors/destination-nebulagraph/src/main/resources/spec.json:
To understand the user-facing configuration.

airbyte-integrations/connectors/destination-nebulagraph/src/main/java/io/airbyte/integrations/destination/nebulagraph/NebulaGraphDestination.java:
The main entry point for the connector.

airbyte-integrations/connectors/destination-nebulagraph/src/main/java/io/airbyte/integrations/destination/nebulagraph/NebulaGraphRecordConsumer.java:
Contains the core logic for building and executing nGQL queries to write data.

airbyte-integrations/connectors/destination-nebulagraph/src/test-integration/java/io/airbyte/integrations/destination/nebulagraph/NebulaGraphDestinationAcceptanceTest.java:
To see the integration test setup and confirm it passes the standard test suite.

User Impact

Users can now seamlessly sync data from any supported Airbyte source into their NebulaGraph databases.
They can configure the connection to their NebulaGraph instance (specifying host, port, user, password, and graph space) and map data streams to be written as Vertices (Tags) within their graph.
There are no known negative side effects, as this is a new, self-contained connector.
-->

Can this PR be safely reverted and rolled back?

• [√] YES 💚
• NO ❌

[CLAassistant]

All committers have signed the CLA.

[github-actions]

👋 Welcome to Airbyte!

Thank you for your contribution from GrayFluex/airbyte! We're excited to have you in the Airbyte community.

Helpful Resources

• PR Guidelines: Check our guidelines for contributions.
• Developing Connectors Locally: Learn how to set up your environment and develop connectors locally.
• If you enable BYO Connector Credentials within your fork, you can view your test results here:
  

PR Slash Commands

As needed or by request, Airbyte Maintainers can execute the following slash commands on your PR:

• /format-fix - Fixes most formatting issues.
• /bump-version - Bumps connector versions.
• /run-connector-tests - Runs connector tests.
• /run-cat-tests - Runs CAT tests.
• /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).

If you have any questions, feel free to ask in the PR comments or join our Slack community.

Tips for Working with CI

1. Pre-Release Checks. Please pay attention to these, as they contain standard checks on the metadata.yaml file, docs requirements, etc. If you need help resolving a pre-release check, please ask a maintainer.
   • Note: If you are creating a new connector, please be sure to replace the default logo.svg file with a suitable icon.
2. Connector CI Tests. Some failures here may be expected if your tests require credentials. Please review these results to ensure (1) unit tests are passing, if applicable, and (2) integration tests pass to the degree possible and expected.
3. (Optional.) BYO Connector Credentials for tests in your fork. You can optionally set up your fork with BYO credentials for your connector. This can significantly speed up your review, ensuring your changes are fully tested before the maintainers begin their review.

📝 Edit this welcome message.

[DanyloGL]

@GrayFluex hi, thanks for your contribution! Could you please sign CLA before we review your PR?

[GrayFluex]

@GrayFluex hi, thanks for your contribution! Could you please sign CLA before we review your PR?

@DanyloGL hi , thanks for the reminder. I've just signed the CLA.👌

[DanyloGL]

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (c1f63cd)