feat: tracing spans across threadpool

[ion-elgreco]

Description

The description of the main changes of your pull request

Related Issue(s)

Documentation

[codecov]

Codecov Report

❌ Patch coverage is 91.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.74%. Comparing base (acfaee5) to head (af21bd5).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/core/src/kernel/snapshot/mod.rs	70.00%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3894      +/-   ##
==========================================
- Coverage   73.75%   73.74%   -0.01%     
==========================================
  Files         151      151              
  Lines       39374    39374              
  Branches    39374    39374              
==========================================
- Hits        29040    29038       -2     
- Misses       9021     9025       +4     
+ Partials     1313     1311       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chiragjn]

I tried this PR, I still see disconnection after Snapshot::try_new_with_engine

I have forked kernel and added decorators on for_snapshot and read_metadata

I am mostly fumbling around at this point and my observation is for spawn_blocking, I need to wrap like this to make it work

let dispatch = tracing::dispatcher::get_default(|d| d.clone());
let span = tracing::Span::current();
let result = spawn_blocking(move || {
    tracing::dispatcher::with_default(&dispatch, || {
        let _enter = span.enter();
        // actual logic

[ion-elgreco]

@chiragjn hmm I see, I've updated the code to reflect that now, can you try again? I guess the key thing is that .instrument only enters the current span when polling the future but the dispatcher also needs to be re-used which doesn't happen with tracing-futures :(

[chiragjn]

Tried the PR again, works fine 🎉

I think a few places are missing - replay_for_scan_metadata from kernel shows up disconnected

I believe these are from scan.rs

delta-rs/crates/core/src/kernel/snapshot/scan.rs

Lines 136 to 145 in acfaee5

	let blocking_iter = move || {
	for res in inner.scan_metadata(engine.as_ref())? {
	if tx.blocking_send(Ok(res?)).is_err() {
	break;
	}
	}
	Ok(())
	};
	builder.spawn_blocking(blocking_iter);

I am able to nest correctly on my fork where I have changed all spawn_blocking in scan.rs

[ion-elgreco]

@chiragjn have added that as well, should be good to go now

[ion-elgreco]

@roeap can you stamp it again? :)

[rtyler]

I want to incorporate this in 0.29.2 which I'll be releasing today