Skip to content

feat: a more lightweight approach #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 69 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
8478eb7
move all the things to old
rklaehn Mar 16, 2025
44a8840
new, more low level approach to rpc
rklaehn Mar 16, 2025
3bcdc86
Add first version of the macro crate
rklaehn Mar 16, 2025
1607a02
keep the first example macro free
rklaehn Mar 16, 2025
c28ae8b
WIP macros
rklaehn Mar 16, 2025
97d53a3
WIP macros
rklaehn Mar 16, 2025
37c1144
WIP rename
rklaehn Mar 16, 2025
09a77f1
works!
rklaehn Mar 16, 2025
7cd6c33
fix comments
rklaehn Mar 16, 2025
0bfac3c
one attribute
rklaehn Mar 16, 2025
7567a66
Add more tests and derive example
rklaehn Mar 16, 2025
2b55b7b
rename rpc feature
rklaehn Mar 16, 2025
36d0a81
Minimize deps even more
rklaehn Mar 16, 2025
206daba
enough for today
rklaehn Mar 16, 2025
9b74bec
Fix a dependency
rklaehn Mar 16, 2025
0006ac4
Add compute example
rklaehn Mar 16, 2025
b6ddade
WIP
rklaehn Mar 16, 2025
6bde683
adapt channel size to same value as in old quic-rpc bench
rklaehn Mar 16, 2025
02d7823
rename Msg to WithChannels again
rklaehn Mar 17, 2025
fada3c4
Add the weird stuff to get the feature flags in docs
rklaehn Mar 17, 2025
57ec9a3
Remove old stuff
rklaehn Mar 17, 2025
6b02fc8
Remove some stuff and do the wasm part in a different way
rklaehn Mar 17, 2025
bc25ba8
clippy
rklaehn Mar 17, 2025
aacb6db
disable compile fail tests for now - they have tiny stupid diffs depe…
rklaehn Mar 17, 2025
ce1abd2
try to fix minimal crates
rklaehn Mar 17, 2025
342c253
another minimal crates fix
rklaehn Mar 17, 2025
902277e
increase version number by a lot and fix postcard minimal version
rklaehn Mar 17, 2025
e282b19
fix tokio-util as well
rklaehn Mar 17, 2025
6965776
Don't require derive_more in the macro
rklaehn Mar 17, 2025
d6ecf83
fix features
rklaehn Mar 17, 2025
2f73a82
Fuse the damn oneshot receiver
rklaehn Mar 17, 2025
2250980
Add ability to send if not busy
rklaehn Mar 18, 2025
90195f7
comment
rklaehn Mar 18, 2025
1f6561d
fixes
rklaehn Mar 18, 2025
39b7172
Add generic enum for local/remote
rklaehn Mar 18, 2025
9fd3f35
fix mismatch with blobs
rklaehn Mar 18, 2025
30e28b8
add spans to compute example
rklaehn Mar 20, 2025
f6d5069
Pass though get_parent_span from the WithChannels impl
rklaehn Mar 20, 2025
ec252c0
pub duh!
rklaehn Mar 20, 2025
15249c2
skip the span when debugging
rklaehn Mar 20, 2025
34c4192
Add a way to find out if a sender is rpc
rklaehn Mar 20, 2025
170f57b
Nicer debug, and don't log at warn level for a normal application close
rklaehn Mar 20, 2025
cf8f4fd
fix bug in try_send
rklaehn Mar 20, 2025
a07b87b
swap order of tx and rx in write
rklaehn Mar 26, 2025
58c4b6f
Common SendError
rklaehn Mar 26, 2025
4d48639
Add RecvError
rklaehn Mar 26, 2025
cea1d03
use thiserror after all
rklaehn Mar 26, 2025
c8fdc3c
More io::Error conversions
rklaehn Mar 26, 2025
bd2026b
more flagging
rklaehn Mar 26, 2025
69a62ea
Add comments for the errors.
rklaehn Mar 27, 2025
5973798
Merge branch 'main' into tabula-rasa
rklaehn Mar 27, 2025
8166a2c
clippy
rklaehn Mar 27, 2025
e8305ba
attempt to get rid of minimal crates error
rklaehn Mar 27, 2025
4c877a3
- rename local and remote service senders
rklaehn Mar 31, 2025
2bd55c9
shuffle the phantomdata around a bit
rklaehn Mar 31, 2025
545d18a
more phantom data shuffling
rklaehn Mar 31, 2025
d70b068
Add a trait to abstract over the remote connection
rklaehn Mar 31, 2025
b70acb6
Add quic-rpc-iroh crate
rklaehn Mar 31, 2025
6b4924a
Some renaming
rklaehn Mar 31, 2025
6cb4565
some renaming
rklaehn Mar 31, 2025
47f9e52
Disable minimal crates
rklaehn Apr 2, 2025
42083a2
Eliminate RemoteSender and RemoteReceiver newtypes
rklaehn Apr 3, 2025
c1dac1a
Docs
rklaehn Apr 3, 2025
5a78709
Add a long motivational section to the module level docs
rklaehn Apr 3, 2025
45e53d4
clippy the docs
rklaehn Apr 3, 2025
5a09a77
Re-enable minimal versions check, but only for quic-rpc and quic-rpc-…
rklaehn Apr 3, 2025
b602f86
Just refer to the module docs from the readme
rklaehn Apr 3, 2025
f9cb191
Rename BoxedSender to DynSender, likewise with receiver.
rklaehn Apr 3, 2025
3852b52
Change comment about features
rklaehn Apr 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,22 @@ impl<I: Channels<S>, S: Service> Deref for WithChannels<I, S> {
}
}

/// A client to the service `S` using the local message type `M` and the remote
/// message type `R`.
///
/// `R` is typically a serializable enum with a case for each possible message
/// type. It can be thought of as the definition of the protocol.
///
/// `M` is typically an enum with a case for each possible message type, where
/// each case is a `WithChannels` struct that extends the inner protocol message
/// with a local tx and rx channel as well as a tracing span to allow for
/// keeping tracing context across async boundaries.
///
/// In some cases, `M` and `R` can be enums for a subset of the protocol. E.g.
/// if you have a subsystem that only handles a part of the messages.
///
/// The service type `S` provides a scope for the protocol messages. It exists
/// so you can use the same message with multiple services.
#[derive(Debug)]
pub struct Client<M, R, S>(ClientInner<M>, PhantomData<(R, S)>);

Expand All @@ -528,11 +544,16 @@ impl<M, R, S> From<tokio::sync::mpsc::Sender<M>> for Client<M, R, S> {
}

impl<M, R, S> Client<M, R, S> {
/// Create a new client to a remote service using the given quinn `endpoint`
/// and a socket `addr` of the remote service.
#[cfg(feature = "rpc")]
pub fn quinn(endpoint: quinn::Endpoint, addr: std::net::SocketAddr) -> Self {
Self::boxed(rpc::QuinnRemoteConnection::new(endpoint, addr))
}

/// Create a new client from a `rpc::RemoteConnection` trait object.
/// This is used from crates that want to provide other transports than quinn,
/// such as the iroh transport.
#[cfg(feature = "rpc")]
pub fn boxed(remote: impl rpc::RemoteConnection) -> Self {
Self(ClientInner::Remote(Box::new(remote)), PhantomData)
Expand Down Expand Up @@ -725,12 +746,17 @@ pub mod rpc {
BoxedFuture, RequestError, RpcMessage,
};

/// Error that can occur when writing the initial message when doing a
/// cross-process RPC.
#[derive(Debug, thiserror::Error)]
pub enum WriteError {
#[error("error serializing: {0}")]
Io(#[from] io::Error),
/// Error writing to the stream with quinn
#[error("error writing to stream: {0}")]
Quinn(#[from] quinn::WriteError),
/// Generic IO error, e.g. when serializing the message or when using
/// other transports.
#[error("error serializing: {0}")]
Io(#[from] io::Error),
}

impl From<WriteError> for io::Error {
Expand All @@ -742,9 +768,21 @@ pub mod rpc {
}
}

/// Trait to abstract over a client connection to a remote service.
///
/// This isn't really that much abstracted, since the result of open_bi must
/// still be a quinn::SendStream and quinn::RecvStream. This is just so we
/// can have different connection implementations for normal quinn connections,
/// iroh connections, and possibly quinn connections with disabled encryption
/// for performance.
///
/// This is done as a trait instead of an enum, so we don't need an iroh
/// dependency in the main crate.
pub trait RemoteConnection: Send + Sync + Debug + 'static {
/// Boxed clone so the trait is dynable.
fn clone_boxed(&self) -> Box<dyn RemoteConnection>;

/// Open a bidirectional stream to the remote service.
fn open_bi(
&self,
) -> BoxedFuture<std::result::Result<(quinn::SendStream, quinn::RecvStream), RequestError>>;
Expand Down Expand Up @@ -1058,9 +1096,12 @@ pub mod rpc {
}
}

/// A request to a service. This can be either local or remote.
#[derive(Debug)]
pub enum Request<L, R> {
/// Local request
Local(L),
/// Remote request
Remote(R),
}

Expand Down
12 changes: 12 additions & 0 deletions src/util.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
//! Utilities
//!
//! This module contains utilities to read and write varints, as well as
//! functions to set up quinn endpoints for local rpc and testing.
#[cfg(feature = "test")]
#[cfg_attr(quicrpc_docsrs, doc(cfg(feature = "test")))]
mod quinn_setup_utils {
Expand Down Expand Up @@ -261,7 +265,11 @@ mod varint_util {
Ok(())
}

/// Provides a fn to read a varint from an AsyncRead source.
pub trait AsyncReadVarintExt: AsyncRead + Unpin {
/// Reads a u64 varint from an AsyncRead source, using the Postcard/LEB128 format.
///
/// If the stream is at the end, this returns `Ok(None)`.
fn read_varint_u64(&mut self) -> impl Future<Output = io::Result<Option<u64>>>;
}

Expand All @@ -271,9 +279,13 @@ mod varint_util {
}
}

/// Provides a fn to write a varint to an [`io::Write`] target, as well as a
/// helper to write a length-prefixed value.
pub trait WriteVarintExt: std::io::Write {
/// Write a varint
#[allow(dead_code)]
fn write_varint_u64(&mut self, value: u64) -> io::Result<usize>;
/// Write a value with a varint enoded length prefix.
fn write_length_prefixed<T: Serialize>(&mut self, value: T) -> io::Result<()>;
}

Expand Down
Loading