-
Notifications
You must be signed in to change notification settings - Fork 55
Express the wallet changeset over the FFI layer #756
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Express the wallet changeset over the FFI layer #756
Conversation
5c7d850
to
1af0175
Compare
a2027f3
to
4940e39
Compare
Best left for after #764 |
505a992
to
2c359be
Compare
Rebased and ready for discussion next call |
Sounds good I'll start reviewing leading up to next call |
To clarify, when trying to compile /// A reference to an unspent output by TXID and output index.
#[derive(Debug, Clone, Eq, PartialEq, std::hash::Hash, uniffi:: Record)]
#[uniffi::export(Eq, Hash)]
pub struct OutPoint {
/// The transaction.
pub txid: Arc<Txid>,
/// The index of the output in the transaction.
pub vout: u32,
} UniFFI does not allow the |
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
#771 demonstrates how this is used |
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
2c359be
to
c10721a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry about the Swift specific Equatable and Hashable stuff needed, but yeah.
Whatever things you thought might be kind-of-gross still look like the right thing to me after review, the only alternate ideas I had ended up with a worse tradeoff balance than the ones in the PR.
Tested locally, also build swift bindings and tested on bdk iOS app, built and ran fine, no changes needed in iOS app for me since I wasn't using ChangeSet directly anywhere.
ACK c10721a
Thanks for that thorough review and testing! I will continue to poke around the UniFFI docs to see if there is a way to get around the HashableOutPoint, but with my latest solution for persistence, this type doesn't matter unless you need custom persistence. |
c10721a
to
afc9fb8
Compare
afc9fb8 rebased |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK afc9fb8.
Question for @rustaceanrob: given the approach of #771 (which I have not reviewed yet), it's the job of the application layer to serialize the |
3854590
to
07a2363
Compare
Yeah, I think it makes sense to spell this out explicitly for future reference: Primitive objects that implement to/from bytes: The records in the change sets that are made up of the primitives: By representing each of the smallest sub-components as serializable, the user can determine how they will store each one of these fields. For instance, maybe the Some last minute clean-up:
|
07a2363
to
c015197
Compare
c015197
to
30675df
Compare
They are aware of this bug for Swift, so hopefully we can remove Looking for re-ACK on 30675df |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 30675df.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
With the following approach we can accomplish two things at once: 1. Make it easy to add new Rust backends without sacrifing performance 2. Allow for arbitrary persistence accross the FFI To accomplish this we can differentiate between a native backend and a foreign backend. The foreign backend interacts with the FFI `ChangeSet`, whereas the native backend can just use the Rust changeset directly. Whenever a new backend is introduced in Rust, a new enum variant may simply be added to the `PersistenceType`. To build a Sqlite backend, or a foreign backend, the user will use the `Persister` structure. Abitrary persistence is allowed by implementing the `Persistence` trait. This was accomplished with no changes to bitcoindevkit#756. I hope 1. motivates this change thoroughly, as we expect BDK will add support for new backends in the future. I am also interested in the applications of 2., where a user might be able to do encrypted and cloud storage.
Description
Builds on #754 to create a fully expressive and serializable
ChangeSet
type. The goal is for this to enable arbitrary persistence backends across the FFI layer. Ultimately that would close #739 and any further database requests that are outside the immediately supported options.For context: The iterators in the BDK changesets are
BTree
based to beno_std
compatible and have deterministic ordering of duplicate data across all targets. UniFFI exposes only two iterators across the boundary,Vec
andHashMap
, but I can try to lay out why I think using these is acceptable.The goal here is to convert between
bdk_wallet::ChangeSet
and FFIChangeSet
, and the easiest way to go about that is to just make each field convertible. For theLocalChain::ChangeSet
, I iterate over theBTreeMap
, which will return the key-value pairs in order. That means the vector will be in order over the FFI layer. Everything else coming from BDK could be ordered by key, but is not required in my opinion to implement the persistence. For instance any mapping fromDescriptorId
to last revealed index is okay, as theDescirptorId
do not necessarily have to be in any particular order. Same goes forTxid
orOutpoint
mapping to some value. While it is nice to be reproducible, these keys are suitable for aHashMap
.Can elaborate on the next call
Notes to the reviewers
In Swift, and key of a
HashMap
must implementEquitable
andHashable
. We can exportEq
andHash
from Rust for Objects, but not for Records from what I can tell. As a result, I made a kind-of-gross wrapper calledHashableOutPoint
that is anObject
so it can be used as a key, but it is convertible to the underlyingRecord
if required. Open to ideas on that one.Also, I cannot for the life of me figure out the best way to go from a
Descriptor<DescriptorPublicKey>
intoDescriptorPublicKey
and vise versa without introducing anunwrap
. Feels like we can improve that.Changelog notice
Checklists
All Submissions:
cargo fmt
andcargo clippy
before committingNew Features:
Bugfixes: