Skip to content

CrateDB Fivetran Destination connector, for loading data into CrateDB.

License

Notifications You must be signed in to change notification settings

crate/cratedb-fivetran-destination

Repository files navigation

CrateDB Fivetran Destination

Bluesky Release Notes CI Downloads per month

Package version License Status Supported Python versions

» Documentation | Changelog | PyPI | Issues | Source code | License | CrateDB | Community Forum

The cratedb-fivetran-destination package implements the [CrateDB destination adapter for Fivetran]. It works with both CrateDB and CrateDB Cloud.

Feel free to use the adapter as provided or else modify / extend it as appropriate for your own applications. We appreciate contributions of any kind.

Introduction

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

Fivetran is an automated data movement platform. Automatically, reliably and securely move data from 650+ sources including SaaS applications, databases, ERPs, and files to data warehouses, data lakes, and more.

Requirements

The package is based on CrateDB's SQLAlchemy dialect package. It will be automatically installed when installing the Fivetran adapter.

You can run CrateDB Self-Managed or start using CrateDB Cloud, see CrateDB Installation, or CrateDB Cloud Console.

Installation

pip install --upgrade cratedb-fivetran-destination

Usage

Start gRPC destination server.

python -m cratedb_fivetran_destination.main

Status

The adapter is still in its infancy and needs exposure to remedy teething problems.

On its next iterations, we will evaluate if the encoding (column mapping) style needs to be adjusted, and if the type mapping needs improvements. We will also take care about a few more other details around easing deployment.

Also, the adapter still lacks implementations for the AlterTable and DescribeTable methods, which will be submitted on subsequent iterations. Contributions are welcome.

Project Information

Acknowledgements

Kudos to the authors of all the many software components this library is inheriting from and building upon.

Contributing

The cratedb-fivetran-destination package is an open source project, and is managed on GitHub. We appreciate contributions of any kind.

License

The project uses the Apache license, like CrateDB itself.