Skip to content

test(cubestore): Fix decimal_partition_pruning test in release mode #9378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

test(cubestore): Fix decimal_partition_pruning test in release mode #9378

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 26 additions & 13 deletions rust/cubestore/cubestore/src/sql/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2812,21 +2812,34 @@ mod tests {

println!("All partitions: {:#?}", partitions);

let plans = service
.plan_query("SELECT sum(num) from foo.numbers where num = 50")
.await
.unwrap();
// Semi-busy-wait for, or, seemingly, induce, compaction for 2000 ms.
let num_attempts = 100;
for i in 0..num_attempts {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's better to use JobResultListener and wait_for_job_results here or metastore listener.

cC @waralexrom @waralex

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not clear to me what job to wait for. Based on logs, some sort of PartitionCompaction job. But I don't know the right way to see which ones I want to wait on.

tokio::time::sleep(Duration::from_millis(20)).await;

let plans = service
.plan_query("SELECT sum(num) from foo.numbers where num = 50")
.await
.unwrap();

let worker_plan = pp_phys_plan(plans.worker.as_ref());
let parquet_regex = Regex::new(r"\d+-[a-z0-9]+\.parquet").unwrap();
let matches = parquet_regex.captures_iter(&worker_plan).count();
let chunk_parquet_regex = Regex::new(r"\d+-[a-z0-9]+\.chunk\.parquet").unwrap();
let chunk_matches = chunk_parquet_regex.captures_iter(&worker_plan).count();

let worker_plan = pp_phys_plan(plans.worker.as_ref());
println!("Worker Plan: {}", worker_plan);
let parquet_regex = Regex::new(r"\d+-[a-z0-9]+.parquet").unwrap();
let matches = parquet_regex.captures_iter(&worker_plan).count();
assert!(
// TODO 2 because partition pruning doesn't respect half open intervals yet
matches < 3 && matches > 0,
"{}\nshould have 2 and less partition scan nodes",
worker_plan
);
if matches < 3 && matches > 0 && chunk_matches == 0 {
break;
} else if i == num_attempts - 1 {
panic!(
"{}\nshould have 2 and less partition scan nodes, matches = {}, chunk_matches = {}",
worker_plan,
matches,
chunk_matches,
);
}
}
})
.await;
}
Expand Down
Loading