Skip to content

Commit 9b14665

Browse files
committed
add tests and create ci workflow
1 parent 1476f98 commit 9b14665

39 files changed

Lines changed: 686 additions & 388 deletions

.github/workflows/ci.yml

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
env:
10+
CARGO_TERM_COLOR: always
11+
12+
jobs:
13+
build:
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- uses: dtolnay/rust-toolchain@stable
20+
with:
21+
components: rustfmt, clippy
22+
23+
- uses: actions/cache@v4
24+
with:
25+
path: |
26+
~/.cargo/registry/index
27+
~/.cargo/registry/cache
28+
~/.cargo/git/db
29+
target/
30+
key: cargo-${{ runner.os }}-${{ hashFiles('**/Cargo.lock') }}
31+
restore-keys: cargo-${{ runner.os }}-
32+
33+
- run: cargo fmt --check
34+
- run: cargo clippy -- -D warnings
35+
- run: cargo test
36+
- run: cargo build

CONTRIBUTING.md

Lines changed: 52 additions & 103 deletions
Original file line numberDiff line numberDiff line change
@@ -1,156 +1,105 @@
1-
** replace `dbt-oss-template` with your repository name in all docs
2-
3-
4-
# Contributing to `dbt-oss-template`
5-
6-
`dbt-oss-template` is a template for open source software projects at dbt Labs.
1+
# Contributing to `dbtp`
72

3+
`dbtp` is a Rust CLI for the dbt Cloud Platform APIs.
84

95
1. [About this document](#about-this-document)
106
2. [Getting the code](#getting-the-code)
117
3. [Setting up an environment](#setting-up-an-environment)
12-
4. [Running in development](#running-dbt-oss-template-in-development)
8+
4. [Running in development](#running-in-development)
139
5. [Testing](#testing)
14-
6. [Debugging](#debugging)
15-
7. [Adding or modifying a changelog entry](#adding-or-modifying-a-changelog-entry)
16-
8. [Submitting a Pull Request](#submitting-a-pull-request)
17-
9. [Troubleshooting Tips](#troubleshooting-tips)
10+
6. [Adding a changelog entry](#adding-a-changelog-entry)
11+
7. [Submitting a Pull Request](#submitting-a-pull-request)
1812

1913
## About this document
2014

21-
There are many ways to contribute to the ongoing development of `dbt-oss-template`, such as by participating in discussions and issues. We encourage you to first read our higher-level document: ["Expectations for Open Source Contributors"](https://docs.getdbt.com/docs/contributing/oss-expectations).
22-
23-
The rest of this document serves as a more granular guide for contributing code changes to `dbt-oss-template` (this repository). It is not intended as a guide for using `dbt-oss-template`, and some pieces assume a level of familiarity with Python development (virtualenvs, `pip`, etc). Specific code snippets in this guide assume you are using macOS or Linux and are comfortable with the command line.
15+
There are many ways to contribute — opening issues, joining discussions, or submitting code changes. This document focuses on the latter. It assumes familiarity with Rust and the command line.
2416

25-
If you get stuck, we're happy to help! Drop us a line in the `#dbt-oss-template-development` channel in the [dbt Community Slack](https://community.getdbt.com).
17+
If you get stuck, open a [GitHub Discussion](https://github.com/trouze/dbtp/discussions) or file an issue.
2618

2719
### Notes
2820

29-
- **CLA:** Please note that anyone contributing code to `dbt-oss-template` must sign the [Contributor License Agreement](https://docs.getdbt.com/docs/contributor-license-agreements). If you are unable to sign the CLA, the `dbt-oss-template` maintainers will unfortunately be unable to merge any of your Pull Requests. We welcome you to participate in discussions, open issues, and comment on existing ones.
30-
- **Branches:** All pull requests from community contributors should target the `main` branch (default).
31-
- **Releases**: This repository is never released.
21+
- **Branches:** All pull requests should target the `main` branch.
22+
- **CLA:** Contributions are accepted under the [Apache-2.0](LICENSE) license.
3223

3324
## Getting the code
3425

35-
### Installing git
26+
### Prerequisites
3627

37-
You will need `git` in order to download and modify the source code.
28+
- [Rust](https://rustup.rs/) (stable toolchain)
29+
- `git`
3830

39-
### External contributors
31+
### Fork and clone
4032

41-
If you are not a member of the `dbt-labs` GitHub organization, you can contribute to `dbt-oss-template` by forking the `dbt-oss-template` repository. For a detailed overview on forking, check out the [GitHub docs on forking](https://help.github.com/en/articles/fork-a-repo). In short, you will need to:
33+
External contributors should fork the repository, then clone their fork:
4234

43-
1. Fork the `dbt-oss-template` repository
44-
2. Clone your fork locally
45-
3. Check out a new branch for your proposed changes
46-
4. Push changes to your fork
47-
5. Open a pull request against `dbt-labs/dbt-oss-template` from your forked repository
48-
49-
### dbt Labs contributors
35+
```sh
36+
git clone https://github.com/<your-username>/dbtp.git
37+
cd dbtp
38+
```
5039

51-
If you are a member of the `dbt-labs` GitHub organization, you will have push access to the `dbt-oss-template` repo. Rather than forking `dbt-oss-template` to make your changes, just clone the repository, check out a new branch, and push directly to that branch.
40+
dbt Labs contributors can clone directly and push to a feature branch.
5241

5342
## Setting up an environment
5443

55-
There are some tools that will be helpful to you in developing locally. While this is the list relevant for `dbt-oss-template` development, many of these tools are used commonly across open-source python projects.
56-
57-
### Tools
44+
No additional setup is required beyond a working Rust toolchain. Install or update via:
5845

59-
These are the tools used in `dbt-oss-template` development and testing:
60-
61-
- [`flake8`](https://flake8.pycqa.org/en/latest/) for code linting
62-
- [`black`](https://github.com/psf/black) for code formatting
63-
- [`mypy`](https://mypy.readthedocs.io/en/stable/) for static type checking
64-
- [`pre-commit`](https://pre-commit.com) to easily run those checks
65-
- [`changie`](https://changie.dev/) to create changelog entries, without merge conflicts
66-
67-
A deep understanding of these tools in not required to effectively contribute to `dbt-oss-template`, but we recommend checking out the attached documentation if you're interested in learning more about each one.
68-
69-
#### Virtual environments
70-
71-
We strongly recommend using virtual environments when developing code in `dbt-oss-template`. We recommend creating this virtualenv
72-
in the root of the `dbt-oss-template` repository. To create a new virtualenv, run:
7346
```sh
74-
python3 -m venv env
75-
source env/bin/activate
47+
rustup update stable
7648
```
7749

78-
This will create and activate a new Python virtual environment.
50+
## Running in development
7951

80-
## Running `dbt-oss-template` in development
81-
82-
### Installation
83-
84-
First make sure that you set up your `virtualenv` as described in [Setting up an environment](#setting-up-an-environment). Also ensure you have the latest version of pip installed with `pip install --upgrade pip`. Next, install `dbt-oss-template` (and its dependencies):
52+
Build and run directly with `cargo`:
8553

8654
```sh
87-
git
88-
pre-commit install
55+
cargo build # debug build
56+
cargo run -- projects list # run a command
57+
cargo run -- --help # top-level help
8958
```
9059

91-
### Running `dbt-oss-template`
60+
Set environment variables to avoid passing flags every run:
9261

93-
This repository is just a template and cannot be run.
62+
```sh
63+
export DBTP_TOKEN=<your-token>
64+
export DBTP_ACCOUNT_ID=<your-account-id>
65+
```
9466

9567
## Testing
9668

97-
Once you're able to manually test that your code change is working as expected, it's important to run existing automated tests, as well as adding some new ones. These tests will ensure that:
98-
- Your code changes do not unexpectedly break other established functionality
99-
- Your code changes can handle all known edge cases
100-
- The functionality you're adding will _keep_ working in the future
69+
Currently there is no automated test suite — commands are tested manually against a live dbt Cloud instance with the environment variables above.
10170

71+
When adding tests in the future, prefer unit tests for pure logic (URL construction, argument parsing, response formatting) that do not require network access.
10272

103-
### Initial setup
104-
105-
None needed.
106-
107-
### Test commands
108-
109-
No tests included.
110-
111-
112-
### Unit, Integration, Functional?
113-
114-
Here are some general rules for adding tests:
115-
* unit tests (`tests/unit`) don’t need to access a database; "pure Python" tests should be written as unit tests
116-
* functional tests (`tests/functional`) cover anything that interacts with a database, namely adapter
117-
118-
## Debugging
73+
```sh
74+
cargo test # runs any tests in the repo
75+
```
11976

120-
1. The logs for a `dbt run` have stack traces and other information for debugging errors (in `logs/dbt.log` in your project directory).
121-
2. Try using a debugger, like `ipdb`. For pytest: `--pdb --pdbcls=IPython.terminal.debugger:pdb`
122-
3.
77+
## Code quality
12378

124-
### Assorted development tips
125-
* Append `# type: ignore` to the end of a line if you need to disable `mypy` on that line.
126-
* Sometimes flake8 complains about lines that are actually fine, in which case you can put a comment on the line such as: # noqa or # noqa: ANNN, where ANNN is the error code that flake8 issues.
127-
* To collect output for `CProfile`, run dbt with the `-r` option and the name of an output file, i.e. `dbt -r dbt.cprof run`. If you just want to profile parsing, you can do: `dbt -r dbt.cprof parse`. `pip` install `snakeviz` to view the output. Run `snakeviz dbt.cprof` and output will be rendered in a browser window.
79+
CI enforces formatting and lints. Run these locally before pushing:
12880

129-
## Adding or modifying a CHANGELOG Entry
81+
```sh
82+
cargo fmt --check # check formatting (run `cargo fmt` to fix)
83+
cargo clippy -- -D warnings # lints
84+
```
13085

131-
We use [changie](https://changie.dev) to generate `CHANGELOG` entries. **Note:** Do not edit the `CHANGELOG.md` directly. Your modifications will be lost.
86+
## Adding a changelog entry
13287

133-
Follow the steps to [install `changie`](https://changie.dev/guide/installation/) for your system.
88+
We use [changie](https://changie.dev) to manage the `CHANGELOG.md`. Do not edit `CHANGELOG.md` directly.
13489

135-
Once changie is installed and your PR is created for a new feature, simply run the following command and changie will walk you through the process of creating a changelog entry:
90+
Install changie, then run:
13691

137-
```shell
92+
```sh
13893
changie new
13994
```
14095

141-
Commit the file that's created and your changelog entry is complete!
142-
143-
If you are contributing to a feature already in progress, you will modify the changie yaml file in dbt/.changes/unreleased/ related to your change. If you need help finding this file, please ask within the discussion for the pull request!
144-
145-
You don't need to worry about which `dbt-oss-template` version your change will go into. Just create the changelog entry with `changie`, and open your PR against the `main` branch. All merged changes will be included in the next minor version of `dbt-oss-template`. The Core maintainers _may_ choose to "backport" specific changes in order to patch older minor versions. In that case, a maintainer will take care of that backport after merging your PR, before releasing the new version of `dbt-oss-template`.
96+
Commit the generated file in `.changes/unreleased/` alongside your PR.
14697

14798
## Submitting a Pull Request
14899

149-
Code can be merged into the current development branch `main` by opening a pull request. A `dbt-oss-template` maintainer will review your PR. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code.
150-
151-
Automated tests run via GitHub Actions. If you're a first-time contributor, all tests (including code checks and unit tests) will require a maintainer to approve. Changes in the `dbt-oss-template` repository trigger integration tests against Postgres. dbt Labs also provides CI environments in which to test changes to other adapters, triggered by PRs in those adapters' repositories, as well as periodic maintenance checks of each adapter in concert with the latest `dbt-oss-template` code changes.
152-
153-
Once all tests are passing and your PR has been approved, a `dbt-oss-template` maintainer will merge your changes into the active development branch. And that's it! Happy developing :tada:
100+
1. Create a branch off `main`.
101+
2. Make your changes and ensure `cargo build`, `cargo fmt --check`, and `cargo clippy -- -D warnings` all pass.
102+
3. Add a changelog entry with `changie new`.
103+
4. Open a PR against `main`. A maintainer will review and merge.
154104

155-
## Troubleshooting Tips
156-
- Sometimes, the content license agreement auto-check bot doesn't find a user's entry in its roster. If you need to force a rerun, add `@cla-bot check` in a comment on the pull request.
105+
CI runs automatically on all PRs. Once checks pass and the PR is approved, a maintainer will merge it.

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# dbtp — dbt Cloud Platform CLI
22

3+
[![CI](https://github.com/trouze/dbtp/actions/workflows/ci.yml/badge.svg)](https://github.com/trouze/dbtp/actions/workflows/ci.yml)
4+
[![Latest Release](https://img.shields.io/github/v/release/trouze/dbtp)](https://github.com/trouze/dbtp/releases/latest)
5+
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
6+
37
A fast, ergonomic command-line interface for the [dbt Cloud](https://www.getdbt.com/product/dbt-cloud) platform APIs. Manage accounts, projects, environments, jobs, and runs. Browse your dbt DAG through the Discovery API. Query metrics through the Semantic Layer. All from your terminal.
48

59
```

src/api/admin/audit_logs.rs

Lines changed: 56 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,29 +4,71 @@ use serde_json::Value;
44
use crate::core::error::Result;
55
use crate::core::rest_client::RestClient;
66

7-
pub async fn list(
8-
client: &RestClient,
9-
params: &[(String, String)],
10-
) -> Result<Vec<Value>> {
7+
pub async fn list(client: &RestClient, params: &[(String, String)]) -> Result<Vec<Value>> {
118
client.paginate_v3("audit-logs/", params, None).await
129
}
1310

1411
/// Paginate all audit log events at or after `since`, filtering client-side.
15-
pub async fn export_since(
16-
client: &RestClient,
17-
since: &DateTime<Utc>,
18-
) -> Result<Vec<Value>> {
12+
pub async fn export_since(client: &RestClient, since: &DateTime<Utc>) -> Result<Vec<Value>> {
1913
let events = client.paginate_v3("audit-logs/", &[], None).await?;
2014

2115
let filtered = events
2216
.into_iter()
23-
.filter(|e| {
24-
e["created_at"]
25-
.as_str()
26-
.and_then(|s| s.parse::<DateTime<Utc>>().ok())
27-
.map_or(true, |dt| dt >= *since)
28-
})
17+
.filter(|e| event_at_or_after(e, since))
2918
.collect();
3019

3120
Ok(filtered)
3221
}
22+
23+
/// Returns true if the event has no parseable `created_at`, or if it is >= `since`.
24+
fn event_at_or_after(event: &Value, since: &DateTime<Utc>) -> bool {
25+
event["created_at"]
26+
.as_str()
27+
.and_then(|s| s.parse::<DateTime<Utc>>().ok())
28+
.is_none_or(|dt| dt >= *since)
29+
}
30+
31+
#[cfg(test)]
32+
mod tests {
33+
use super::*;
34+
use serde_json::json;
35+
36+
fn ts(s: &str) -> DateTime<Utc> {
37+
s.parse::<DateTime<Utc>>().unwrap()
38+
}
39+
40+
#[test]
41+
fn event_after_cutoff_is_included() {
42+
let since = ts("2024-01-10T00:00:00Z");
43+
let event = json!({ "created_at": "2024-01-15T12:00:00Z" });
44+
assert!(event_at_or_after(&event, &since));
45+
}
46+
47+
#[test]
48+
fn event_at_cutoff_is_included() {
49+
let since = ts("2024-01-10T00:00:00Z");
50+
let event = json!({ "created_at": "2024-01-10T00:00:00Z" });
51+
assert!(event_at_or_after(&event, &since));
52+
}
53+
54+
#[test]
55+
fn event_before_cutoff_is_excluded() {
56+
let since = ts("2024-01-10T00:00:00Z");
57+
let event = json!({ "created_at": "2024-01-05T00:00:00Z" });
58+
assert!(!event_at_or_after(&event, &since));
59+
}
60+
61+
#[test]
62+
fn event_with_no_created_at_is_included() {
63+
let since = ts("2024-01-10T00:00:00Z");
64+
let event = json!({ "type": "login" });
65+
assert!(event_at_or_after(&event, &since));
66+
}
67+
68+
#[test]
69+
fn event_with_unparseable_created_at_is_included() {
70+
let since = ts("2024-01-10T00:00:00Z");
71+
let event = json!({ "created_at": "not-a-date" });
72+
assert!(event_at_or_after(&event, &since));
73+
}
74+
}

src/api/admin/environments.rs

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,7 @@ pub async fn list(
1818
.await
1919
}
2020

21-
pub async fn get(
22-
client: &RestClient,
23-
project_id: u64,
24-
environment_id: u64,
25-
) -> Result<Value> {
21+
pub async fn get(client: &RestClient, project_id: u64, environment_id: u64) -> Result<Value> {
2622
client
2723
.get_v3(
2824
&format!("projects/{project_id}/environments/{environment_id}/"),
@@ -31,16 +27,9 @@ pub async fn get(
3127
.await
3228
}
3329

34-
pub async fn create(
35-
client: &RestClient,
36-
project_id: u64,
37-
body: &Value,
38-
) -> Result<Value> {
30+
pub async fn create(client: &RestClient, project_id: u64, body: &Value) -> Result<Value> {
3931
client
40-
.post_v3(
41-
&format!("projects/{project_id}/environments/"),
42-
body,
43-
)
32+
.post_v3(&format!("projects/{project_id}/environments/"), body)
4433
.await
4534
}
4635

@@ -58,11 +47,7 @@ pub async fn update(
5847
.await
5948
}
6049

61-
pub async fn delete(
62-
client: &RestClient,
63-
project_id: u64,
64-
environment_id: u64,
65-
) -> Result<Value> {
50+
pub async fn delete(client: &RestClient, project_id: u64, environment_id: u64) -> Result<Value> {
6651
client
6752
.delete_v3(&format!(
6853
"projects/{project_id}/environments/{environment_id}/"

0 commit comments

Comments
 (0)