Skip to content

development.md

Latest commit

 

History

History
175 lines (125 loc) · 4.34 KB

development.md

File metadata and controls

175 lines (125 loc) · 4.34 KB

Development

This document details steps to running this connector with hasura ddn for local testing during connector development

This document in now oudated

Prerequisites

Clone the repository

Clone this repository to the directory of your choice

git clone https://github.com/hasura/ndc-clickhouse.git

All subsequent commands will assume they are running from the directory the repository was cloned into

cd ndc-clickhouse

Generating a configuration directory

Create a config directory in the project root

mkdir config

Initialize the directory with a configuration based on your database schema

Change the placeholder auth details to your actual database connection information

cargo run --package ndc-clickhouse-cli -- --connector-context-path ./config --clickhouse-url "URL" --clickhouse-username "USERNAME" --clickhouse-password "PASSWORD" update

You can run this command again if your database schema has changed and you'd like the config to reflect that

Any configuration customization should not be overwritten

See also: editing the configuration

Running the connector in docker

Create a .env file in the project root

CLICKHOUSE_URL=<URL>
CLICKHOUSE_USERNAME=<USERNAME>
CLICKHOUSE_PASSWORD=<PASSWORD>

Start the connector

docker compose up -d

The first build may take a while. Subsequent builds should be faster thanks to layer caching

To restart the connector (required for changes to configuration to take effect)

docker compose restart

To rebuild the connector (required for changes to connector source code to take effect)

docker compose up -d --build

Exposing the running connector to the web

We use ngrok to expose our connector to the web. You could use an alternative.

ngrok http http://localhost:4000

Take note of the resulting address, we'll need it.

See also: ngrok documenation

Adding the connector to a ddn project

If you don't yet have a ddn project, you'll need to create one

The following instructions assume a default, empty project. We will be adding a new datasource clickhouse. Change the name as needed

  1. create a directory for the data source app/clickhouse
  2. create the data source definition file app/clickhouse/clickhouse.hml with content:
kind: DataConnectorLink
version: v1
definition:
    name: clickhouse
    url:
        Fn::ManifestRef: clickhouse
  1. create the data source connector directory app/clickhouse/connector
  2. create the data source connector file app/clickhouse/connector/clickhouse.build.hml with content:
kind: ConnectorManifest
version: v1
spec:
  supergraphManifests:
  - base
definition:
  name: clickhouse
  type: endpoints
  deployments:
    - endpoint:
        valueFromEnv: CLICKHOUSE_CONNECTOR_ENDPOINT
  1. add the CLICKHOUSE_CONNECTOR_ENDPOINT env var to base.env.yaml
supergraph: {}
subgraphs:
  app:
    CLICKHOUSE_CONNECTOR_ENDPOINT: <endpoint>

The endpoint should be the one exposed by NGROK

Building the ddn project

Using ddn dev

This command will

  • watch for changes in the connector schema
  • track and update models whenever the schema changes
  • create ddn builds whenever the metadata changes
ddn dev

You should now be able to navigate to your api

Using ddn build

You can also replicate ddn dev step by step

Updating the connector schema

ddn update data-connector-link clickhouse

note here clickhouse is our source name, change it if needed

Tracking models

To explicitly track a model

ddn add model --data-connector-link clickhouse --name <model name>

The model name should be one of the collections exposed by the connector. Check the app/clickhouse/clickhouse.hml file for a list.

note here clickhouse is our source name, change it if needed

Creating a build

ddn build connector-manifest

See also: ddn documentation