Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 179 additions & 31 deletions Cargo.lock

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[workspace]
resolver = "2"
members = ["crates/bulloak", "crates/foundry", "crates/syntax"]
members = ["crates/bulloak", "crates/foundry", "crates/rust", "crates/syntax", "crates/noir"]

[workspace.package]
authors = ["Alexander Gonzalez <[email protected]>"]
Expand Down Expand Up @@ -35,6 +35,8 @@ all = "warn"
[workspace.dependencies]
bulloak-syntax = { path = "crates/syntax", version = "0.9.0" }
bulloak-foundry = { path = "crates/foundry", version = "0.9.0" }
bulloak-rust = { path = "crates/rust", version = "0.9.0" }
bulloak-noir = { path = "crates/noir", version = "0.9.0" }

anyhow = "1.0.75"
clap = { version = "4.3.19", features = ["derive"] }
Expand Down
90 changes: 82 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@

# bulloak

A Solidity test generator based on the
A test generator based on the
[Branching Tree Technique](https://twitter.com/PaulRBerg/status/1682346315806539776).

**Supported languages:**

- **Solidity (Foundry)**: Generates `.t.sol` files with modifiers for conditions
- **Rust**: Generates `_test.rs` files with helper functions for conditions
- **Noir**: Generates `_test.nr` files with helper functions for conditions

- [Installation](#installation)
- [VSCode](#vscode)
- [Usage](#usage)
Expand Down Expand Up @@ -60,7 +66,11 @@ better user experience:
- `bulloak scaffold`
- `bulloak check`

### Scaffold Solidity Files
### Scaffold Test Files

By default, `bulloak scaffold` generates Solidity test files. You can specify a different language using the `--lang` (or `-l`) flag.

#### Solidity (default)

Say you have a `foo.tree` file with the following contents:

Expand Down Expand Up @@ -98,7 +108,7 @@ Say we have a bunch of `.tree` files in the current working directory. If we run
the following:

```text
$ bulloak scaffold -w ./**/*.tree
bulloak scaffold -w ./**/*.tree
```

`bulloak` will create a `.t.sol` file per `.tree` file and write the generated
Expand All @@ -110,7 +120,7 @@ behavior with the `-f` flag. This will force `bulloak` to overwrite the contents
of the file.

```text
$ bulloak scaffold -wf ./**/*.tree
bulloak scaffold -wf ./**/*.tree
```

Note all tests are showing as passing when their body is empty. To prevent this,
Expand All @@ -130,10 +140,74 @@ When enabled, bulloak capitalizes the first letter of each branch description
and ensures it ends with a dot, so you don't need to touch the `.tree` file to
get consistent sentence casing in the scaffolded test bodies.

#### Rust

To generate Rust test files, use `--lang rust`:

```bash
bulloak scaffold --lang rust foo.tree
```

This will generate a `foo_test.rs` file with helper functions for conditions and `#[test]` functions for actions. The generated file will use `#[should_panic]` for actions containing panic keywords like "panic", "revert", "error", or "fail".

```rust
// $ bulloak scaffold --lang rust foo.tree
// Generated by bulloak

/// Helper function for condition
fn stuff_is_called() {
}

/// Helper function for condition
fn a_condition_is_met() {
}

#[test]
#[should_panic]
fn test_when_a_condition_is_met() {
stuff_is_called();
a_condition_is_met();
// It should revert.
}
```

#### Noir

To generate Noir test files, use `--lang noir`:

```bash
bulloak scaffold --lang noir foo.tree
```

This will generate a `foo_test.nr` file with helper functions for conditions and `#[test]` functions for actions. The generated file will use `#[test(should_fail)]` for actions containing panic keywords.

```rust
// $ bulloak scaffold --lang noir foo.tree
// Generated by bulloak

/// Helper function for condition
fn stuff_is_called() {
}

/// Helper function for condition
fn a_condition_is_met() {
}

#[test(should_fail)]
unconstrained fn test_when_a_condition_is_met() {
stuff_is_called();
a_condition_is_met();
// It should revert.
}
```

**Note:** The `-m` (skip helpers) and `-F` (format descriptions) flags work for all languages.

### Check That Your Code And Spec Match

You can use `bulloak check` to make sure that your Solidity files match your
spec. For example, any missing tests will be reported to you.
You can use `bulloak check` to make sure that your test files match your
spec. For example, any missing tests will be reported to you. The `--lang`
flag works the same way as in `scaffold`.

Say you have the following spec:

Expand Down Expand Up @@ -453,6 +527,6 @@ This project has been possible thanks to the support of:
This project is licensed under either of:

- Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
https://www.apache.org/licenses/LICENSE-2.0).
<https://www.apache.org/licenses/LICENSE-2.0>).
- MIT license ([LICENSE-MIT](LICENSE-MIT) or
https://opensource.org/licenses/MIT).
<https://opensource.org/licenses/MIT>).
11 changes: 11 additions & 0 deletions crates/bulloak/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ categories.workspace = true
[dependencies]
bulloak-syntax.workspace = true
bulloak-foundry.workspace = true
bulloak-rust.workspace = true
bulloak-noir.workspace = true

anyhow.workspace = true
clap.workspace = true
Expand All @@ -27,10 +29,19 @@ glob = "0.3.2"
[dev-dependencies]
pretty_assertions.workspace = true
criterion.workspace = true
assert_cmd = "2.0"

[[bench]]
name = "emit"
harness = false

[[bench]]
name = "emit_rust"
harness = false

[[bench]]
name = "emit_noir"
harness = false

[lints]
workspace = true
19 changes: 19 additions & 0 deletions crates/bulloak/benches/emit_noir.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#![allow(missing_docs)]
use bulloak_noir::scaffold;
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn emit_big_tree_noir(c: &mut Criterion) {
let tree =
std::fs::read_to_string("benches/bench_data/cancel.tree").unwrap();
let ast = bulloak_syntax::parse_one(&tree).unwrap();

let cfg = Default::default();
let mut group = c.benchmark_group("sample-size-10");
group.bench_function("emit-big-tree-noir", |b| {
b.iter(|| scaffold::generate(black_box(&ast), &cfg))
});
group.finish();
}

criterion_group!(benches, emit_big_tree_noir);
criterion_main!(benches);
19 changes: 19 additions & 0 deletions crates/bulloak/benches/emit_rust.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#![allow(missing_docs)]
use bulloak_rust::scaffold;
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn emit_big_tree_rust(c: &mut Criterion) {
let tree =
std::fs::read_to_string("benches/bench_data/cancel.tree").unwrap();
let ast = bulloak_syntax::parse_one(&tree).unwrap();

let cfg = Default::default();
let mut group = c.benchmark_group("sample-size-10");
group.bench_function("emit-big-tree-rust", |b| {
b.iter(|| scaffold::scaffold(black_box(&ast), &cfg))
});
group.finish();
}

criterion_group!(benches, emit_big_tree_rust);
criterion_main!(benches);
113 changes: 112 additions & 1 deletion crates/bulloak/src/check.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ use clap::Parser;
use owo_colors::OwoColorize;
use serde::{Deserialize, Serialize};

use crate::{cli::Cli, glob::expand_glob};
use crate::{
cli::{Backend, Cli},
glob::expand_glob,
};

/// Check that the tests match the spec.
#[doc(hidden)]
Expand All @@ -41,6 +44,9 @@ pub struct Check {
/// Whether to capitalize and punctuate branch descriptions.
#[arg(long = "format-descriptions", default_value_t = false)]
pub format_descriptions: bool,
/// The target language for checking.
#[arg(short = 'l', long = "lang", value_enum, default_value_t = Backend::Solidity)]
pub backend: Backend,
}

impl Default for Check {
Expand All @@ -54,6 +60,15 @@ impl Check {
///
/// Note that we don't deal with `solang_parser` errors at all.
pub(crate) fn run(&self, cfg: &Cli) {
if self.backend == Backend::Rust {
return self.run_rust_check();
}

if self.backend == Backend::Noir {
return self.run_noir_check();
}

// Solidity check
let mut specs = Vec::new();
for pattern in &self.files {
match expand_glob(pattern.clone()) {
Expand Down Expand Up @@ -163,6 +178,102 @@ impl Check {
eprintln!("{}: {e}", "warn".yellow());
}
}

/// Expand glob patterns into file paths.
fn expand_specs(&self) -> Vec<PathBuf> {
let mut specs = Vec::new();
for pattern in &self.files {
match expand_glob(pattern.clone()) {
Ok(iter) => specs.extend(iter),
Err(e) => eprintln!(
"{}: could not expand {}: {}",
"warn".yellow(),
pattern.display(),
e
),
}
}
specs
}

/// Run check for Rust tests.
fn run_rust_check(&self) {
let specs = self.expand_specs();
let cfg = bulloak_rust::Config {
files: self.files.iter().map(|p| p.display().to_string()).collect(),
skip_helpers: self.skip_modifiers,
format_descriptions: self.format_descriptions,
};

let violations = self.collect_violations(&specs, |path| {
bulloak_rust::check::check(path, &cfg)
});

self.report_violations(&violations);
}

/// Run check for Noir tests.
fn run_noir_check(&self) {
let specs = self.expand_specs();
let cfg = bulloak_noir::Config {
files: self.files.iter().map(|p| p.display().to_string()).collect(),
skip_helpers: self.skip_modifiers,
format_descriptions: self.format_descriptions,
};

let violations = self.collect_violations(&specs, |path| {
bulloak_noir::check::check(path, &cfg)
});

self.report_violations(&violations);
}

/// Collect violations from checking multiple tree files.
fn collect_violations<F, V>(&self, specs: &[PathBuf], check_fn: F) -> Vec<V>
where
F: Fn(&PathBuf) -> anyhow::Result<Vec<V>>,
V: std::fmt::Display,
{
let mut all_violations = Vec::new();
for tree_path in specs {
match check_fn(tree_path) {
Ok(violations) => {
for violation in &violations {
eprintln!("{}", violation);
}
all_violations.extend(violations);
}
Err(e) => {
eprintln!(
"{}: Failed to check {}: {}",
"error".red(),
tree_path.display(),
e
);
}
}
}
all_violations
}

/// Report violations and exit if necessary.
fn report_violations<V: std::fmt::Display>(&self, violations: &[V]) {
if violations.is_empty() {
println!(
"{}",
"All checks completed successfully! No issues found.".green()
);
} else {
let check_literal = pluralize(violations.len(), "check", "checks");
eprintln!(
"\n{}: {} {} failed",
"warn".bold().yellow(),
violations.len(),
check_literal
);
std::process::exit(1);
}
}
}

fn exit(violations: &[Violation]) {
Expand Down
Loading