Async compliant

[bnjjj]

I'm digging a little bit into this codebase because I think this project is pretty promising. I'm wondering if there's a reason on why the metadata provider is not async ?
Another feedback is that the discord link in your readme doesn't work anymore. I was looking for a place where we can chat. If you have some please share it with me

[zfarrell]

Hey @bnjjj! Thanks for checking out the project!

I'm wondering if there's a reason on why the metadata provider is not async ?

This is driven by two factors:

1. DataFusion's CatalogProvider and SchemaProvider traits have sync methods for schema_names(), schema(), table_names(), and table_exist(), so most of the call sites where we query metadata would still need block_on even with an async trait. There are a few call sites that are async (like SchemaProvider::table() and TableProvider::scan()), but not enough to justify the refactor today.
2. The DuckDB client is inherently sync, and it was the first backend we integrated with. That said, our sqlx-based providers (Postgres, MySQL, SQLite) are async under the hood and bridge to sync via block_in_place, so if DataFusion's upstream traits go async in the future, it'd be a natural refactor.

Thanks for letting me know about the Discord link. Let me check with the team and get back to you!

[eddietejeda]

Hey @bnjjj, we're not using the Discord channel for the time being. Thanks for letting us know. I will remove the link.

If you want to be to be in regular contact with our team, we can setup a Slack Connect. Send me an email at team-ducklake@hotdata.dev, and we can find ways to connect.