Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: lock on target directories not Nargo.toml #7538

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions tooling/nargo_cli/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -188,9 +188,12 @@ fn generate_test_cases(
{test_cases}
fn test_{test_name}(force_brillig: ForceBrillig, inliner_aggressiveness: Inliner) {{
let test_program_dir = PathBuf::from("{test_dir}");
let temp_dir = tempfile::tempdir().unwrap();
let target_dir = temp_dir.path();

let mut nargo = Command::cargo_bin("nargo").unwrap();
nargo.arg("--program-dir").arg(test_program_dir);
nargo.arg("--target-dir").arg(target_dir);
nargo.arg("{test_command}").arg("--force");
nargo.arg("--inliner-aggressiveness").arg(inliner_aggressiveness.0.to_string());
// Check whether the test case is non-deterministic
Expand Down
58 changes: 29 additions & 29 deletions tooling/nargo_cli/src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use nargo_toml::{
};
use noirc_driver::{CrateName, NOIR_ARTIFACT_VERSION_STRING};
use std::{
fs::File,
fs::{File, OpenOptions},
path::{Path, PathBuf},
};

Expand Down Expand Up @@ -203,18 +203,17 @@ where
// Lock manifests if the command needs it.
let _locks = match cmd.lock_type() {
LockType::None => None,
typ => Some(lock_workspace(&workspace, typ == LockType::Exclusive)?),
typ => {
Some(lock_target_dir(&workspace.target_directory_path(), typ == LockType::Exclusive)?)
}
};
run(cmd, workspace)
}

/// Lock the (selected) packages in the workspace.
/// The lock taken can be shared for commands that only read the artifacts,
/// or exclusive for the ones that (might) write artifacts as well.
fn lock_workspace(
workspace: &Workspace,
exclusive: bool,
) -> Result<Vec<impl Drop + use<>>, CliError> {
fn lock_target_dir(target_dir: &Path, exclusive: bool) -> Result<impl Drop + use<>, CliError> {
struct LockedFile(File);

impl Drop for LockedFile {
Expand All @@ -223,31 +222,32 @@ fn lock_workspace(
}
}

let mut locks = Vec::new();
for pkg in workspace.into_iter() {
let toml_path = get_package_manifest(&pkg.root_dir)?;
let path_display = toml_path.display();

let file = File::open(&toml_path)
.unwrap_or_else(|e| panic!("Expected {path_display} to exist: {e}"));

if exclusive {
if fs2::FileExt::try_lock_exclusive(&file).is_err() {
eprintln!("Waiting for lock on {path_display}...");
}
fs2::FileExt::lock_exclusive(&file)
.unwrap_or_else(|e| panic!("Failed to lock {path_display}: {e}"));
} else {
if fs2::FileExt::try_lock_shared(&file).is_err() {
eprintln!("Waiting for lock on {path_display}...",);
}
fs2::FileExt::lock_shared(&file)
.unwrap_or_else(|e| panic!("Failed to lock {path_display}: {e}"));
}
let lock_path = target_dir.join(".nargo-lock");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest locking on a package specific lock file in the output directory, to allow the compilations in aztec-packages to go concurrently without all waiting on this single .nargo-lock to be free.

let path_display = lock_path.display();

std::fs::create_dir_all(target_dir).unwrap();
let file = OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(&lock_path)
.unwrap_or_else(|e| panic!("Expected {path_display} to exist: {e}"));

locks.push(LockedFile(file));
if exclusive {
if fs2::FileExt::try_lock_exclusive(&file).is_err() {
eprintln!("Waiting for lock on {path_display}...");
}
fs2::FileExt::lock_exclusive(&file)
.unwrap_or_else(|e| panic!("Failed to lock {path_display}: {e}"));
} else {
if fs2::FileExt::try_lock_shared(&file).is_err() {
eprintln!("Waiting for lock on {path_display}...",);
}
fs2::FileExt::lock_shared(&file)
.unwrap_or_else(|e| panic!("Failed to lock {path_display}: {e}"));
}
Ok(locks)

Ok(LockedFile(file))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also suggest removing the lock files to not leave around hundreds of them, but maybe that's not important. Or maybe they can all be in a target/locks directory, so nobody is bothered.

}

/// Parses a path and turns it into an absolute one by joining to the current directory.
Expand Down
Loading