docs(iroh): improve readme

README.md

@@ -48,7 +48,6 @@ Use pre-existing protocols built on iroh instead of writing your own:
 - [iroh-blobs] for [BLAKE3]-based content-addressed blob transfer scaling from kilobytes to terabytes
 - [iroh-gossip] for establishing publish-subscribe overlay networks that scale, requiring only resources that your average phone can handle
 - [iroh-docs] for an eventually-consistent key-value store of [iroh-blobs] blobs
-- [iroh-willow] for an (in-construction) implementation of the [willow protocol]
 
 ## Getting Started
 
@@ -130,10 +129,9 @@ If you want to use iroh from other languages, make sure to check out [iroh-ffi],
 
 This repository contains a workspace of crates:
 - `iroh`: The core library for hole-punching & communicating with relays.
-- `iroh-relay`: The relay server implementation. This is the code we run in production (and you can, too!).
-- `iroh-base`: Common types like `Hash`, key types or `RelayUrl`.
-- `iroh-dns-server`: DNS server implementation powering the `n0_discovery` for EndpointIds, running at dns.iroh.link.
-- `iroh-net-report`: Analyzes your host's networking ability & NAT.
+- `iroh-relay`: The relay client and server implementation. This is the code we run in production for the public relays (and you can, too!).
+- `iroh-base`: Common types like `EndpointId` or `RelayUrl`.
+- `iroh-dns-server`: DNS server implementation powering the DNS/Pkarr address lookup for EndpointIds, running at dns.iroh.link.
 
 ## License
 
@@ -158,7 +156,6 @@ Unless you explicitly state otherwise, any contribution intentionally submitted
 [iroh-blobs]: https://github.com/n0-computer/iroh-blobs
 [iroh-gossip]: https://github.com/n0-computer/iroh-gossip
 [iroh-docs]: https://github.com/n0-computer/iroh-docs
-[iroh-willow]: https://github.com/n0-computer/iroh-willow
 [iroh-doctor]: https://github.com/n0-computer/iroh-doctor
 [willow protocol]: https://willowprotocol.org
 [iroh-ffi]: https://github.com/n0-computer/iroh-ffi
@@ -167,4 +164,4 @@ Unless you explicitly state otherwise, any contribution intentionally submitted
 [Iroh Experiments]: https://github.com/n0-computer/iroh-experiments
 [echo-rs]: /iroh/examples/echo.rs
 [iroh-perf]: https://perf.iroh.computer
-[docs]: https://iroh.computer/docs
+[docs]: https://docs.iroh.computer

iroh/DEVELOPMENT.md

@@ -0,0 +1,49 @@
+# Developing iroh
+
+## Structured events
+
+The library uses [tracing] both for logging and for *structured events*.
+Events differ from normal logging by convention:
+
+- The [target] is prefixed with `iroh::_events::`, with `::`-separated names.
+- There is **no message**; the unique target indicates the meaning.
+- The [fields] carry exclusively structured data.
+- The [Level] is always `DEBUG`.
+
+This lets automated tooling process events through custom subscribers while
+still producing distinct output under the default tracing formatters, and makes
+them unlikely to conflict with real modules.
+
+An application can subscribe to the `iroh::_events` target separately. With the
+default file logging it is also easy to grep for all events:
+
+```sh
+rg 'events::[a-z_\-:]+' path/to/iroh/logs/iroh.YYYY-MM-DD-NN.log
+```
+
+When adding events, aim for a high signal-to-noise ratio and design them to be
+extracted automatically. To keep them distinct from normal logging, write them
+with the `event!()` macro:
+
+```rust,ignore
+event!(
+    target: "iroh::_events::subject",
+    Level::DEBUG,
+    field = value,
+);
+```
+
+## Building documentation
+
+Building the documentation is only supported with `--all-features`. To also
+document the cargo features required for certain APIs, pass the `iroh_docsrs`
+cfg to rustdoc, which requires nightly Rust:
+
+```sh
+RUSTDOCFLAGS="--cfg iroh_docsrs" cargo +nightly doc --workspace --no-deps --all-features
+```
+
+[target]: https://docs.rs/tracing/latest/tracing/struct.Metadata.html#method.target
+[fields]: https://docs.rs/tracing/latest/tracing/#recording-fields
+[Level]: https://docs.rs/tracing/latest/tracing/struct.Level.html
+[tracing]: https://docs.rs/tracing

iroh/README.md

@@ -1,91 +1,153 @@
-# iroh
+<h1 align="center">
+<a href="https://iroh.computer">
+<img alt="iroh" src="https://raw.githubusercontent.com/n0-computer/iroh/main/.img/iroh_wordmark.svg" width="100" />
+</a>
+</h1>
+
+<h3 align="center">
+less net work for networks
+</h3>
+
+[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg?style=flat-square)](https://docs.rs/iroh/)
+[![Crates.io](https://img.shields.io/crates/v/iroh.svg?style=flat-square)](https://crates.io/crates/iroh)
+[![Chat](https://img.shields.io/discord/1161119546170687619?logo=discord&style=flat-square)](https://discord.com/invite/DpmJgtU7cW)
+[![Youtube](https://img.shields.io/badge/YouTube-red?logo=youtube&logoColor=white&style=flat-square)](https://www.youtube.com/@n0computer)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE-MIT)
+[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=flat-square)](LICENSE-APACHE)
+
+<div align="center">
+  <h3>
+    <a href="https://iroh.computer/docs">
+      Docs Site
+    </a>
+    <span> | </span>
+    <a href="https://docs.rs/iroh">
+      Rust Docs
+    </a>
+  </h3>
+</div>
+<br/>
+
+Iroh is a Rust library to establish direct connections between endpoints.
+It gives you an API for dialing by public key. You say "connect to that
+endpoint", and iroh finds and maintains the best connection for you.
+
+Under the hood iroh establishes peer-to-peer [QUIC] connections between
+endpoints. The fastest route is a direct connection, so iroh tries to
+[hole-punch] one whenever it can. If that fails it falls back to using
+relay servers.
+
+Because iroh is built on [QUIC], all connections are end-to-end encrypted and may
+carry any number of concurrent streams. Dialing by public key also makes them mutually
+authenticated, because each endpoint's public key is its TLS identity.
+
+## Overview
+
+An iroh endpoint is created and controlled by the [`Endpoint`]. Each endpoint
+has a unique [`SecretKey`], whose public key is the endpoint's identity, the
+[`EndpointId`]. Connections are authenticated against this key, which means an
+[`EndpointId`] can't be impersonated.
+
+A connection is usually established with the help of a *relay server*. When an
+endpoint is created it connects to the closest relay and designates it as its
+*home relay*. Other endpoints reach it first through this relay, then both
+sides use QUIC NAT traversal to establish a direct connection. In the rare
+cases where a direct connection is not possible, traffic keeps flowing over the
+relay.
+
+Relay servers only forward encrypted packets addressed to Endpoint IDs, they
+cannot read any traffic between endpoints.
+
+Endpoints can also connect directly without a relay, as long as the accepting
+endpoint is directly reachable at one of its addresses.
+
+To discover addressing information for an endpoint, iroh uses
+[address lookup services]. With address lookup, you can connect to other
+endpoints with only their [`EndpointId`]. Addressing information will then
+be resolved on-demand.
+
+The [`N0` preset] installs the DNS/Pkarr address lookup service, which uses
+servers hosted by [n0] to provide global lookup for endpoints.
+
+## Example
+
+This is an echo protocol: the accepting side copies back whatever it receives.
+The full, commented version is in [`echo.rs`](examples/echo.rs).
 
-Iroh is a library to establish direct connectivity between peers.
-It's built on peer-to-peer [QUIC](https://en.wikipedia.org/wiki/QUIC) using both relays and holepunching.
-The main structure for connection is the `Endpoint` entrypoint.
-
-Peer to peer connectivity is established with the help of a _relay server_. The relay server provides [QUIC Address Discovery](https://www.ietf.org/archive/id/draft-ietf-quic-address-discovery-00.html) (QAD) and hole-punching assistance for the peers. If no direct connection can be established, the connection is relayed via the server.
-
-Peers must know and do verify the PeerID of each other before they can connect. When using a relay server to aid the connection establishment they will register with a home relay server using their PublicKey.  Other peers which can not establish a direct connection can then establish connection via this relay server.  This will try to assist establishing a direct connection using QAD and holepunching but continue relaying if not possible.
-
-Peers can also connect directly without using a relay server. For this, however the listening peer must be directly reachable by the connecting peer via one of it's addresses.
-
-## Examples
-
-Examples for `iroh` are in `iroh/examples`, run them with `cargo run --example $NAME`. Details for each example are in the file/directory itself.
-
-## Structured Events
-
-The library uses [tracing](https://docs.rs/tracing) to for logging as
-well as for **structured events**.  Events are different from normal
-logging by convention:
-
-- The [target] has a prefix of `$crate_name::_events` and target names
-  are `::` separated.
-
-  For this library the target will always start with `iroh::_events::`.
-
-- There is **no message**.
-
-  Each event has a unique [target] which indicates the meaning.
-
-- The event [fields] are exclusively used for structured data.
-
-- The [Level] is always `DEBUG`.
-
-This is a compromise between being able to process events using
-automated tooling using custom subscribers and them still producing
-distinguishing output in logs when using the default tracing
-subscriber formatters.  While still being unlikely to conflict with
-real modules.
-
-[target]: https://docs.rs/tracing/latest/tracing/struct.Metadata.html#method.target
-[fields]: https://docs.rs/tracing/latest/tracing/#recording-fields
-[Level]: https://docs.rs/tracing/latest/tracing/struct.Level.html
-
-### Using events
-
-If desired an application can use the `$crate_name::_events` target to
-handle events by a different subscriber.  However with the default
-file logging it is already easy to search for all events, e.g. using
-ripgrep:
-
-`rg 'events::[a-z_\-:]+' path/to/iroh/logs/iroh.YYYY-MM-DD-NN.log`
-
-Which will also highlight the full target name by default on a colour
-supporting terminal.
-
-### Development
+```rust
+use iroh::{
+    Endpoint,
+    endpoint::{Connection, presets},
+    protocol::{AcceptError, ProtocolHandler, Router},
+};
+
+/// Each protocol is identified by its ALPN, exchanged during the handshake.
+const ALPN: &[u8] = b"iroh-example/echo/0";
+
+#[tokio::main]
+async fn main() -> anyhow::Result<()> {
+    // The accepting side: bind an endpoint and route the ALPN to a handler.
+    let endpoint = Endpoint::bind(presets::N0).await?;
+    let router = Router::builder(endpoint.clone()).accept(ALPN, Echo).spawn();
+    endpoint.online().await;
+    // Get the endpoint's address so that we can connect to it.
+    let addr = endpoint.addr();
+
+    // The connecting side: dial the accepting endpoint by its address.
+    {
+        let other_endpoint = Endpoint::bind(presets::N0).await?;
+
+        let conn = other_endpoint.connect(addr, ALPN).await?;
+        let (mut send, mut recv) = conn.open_bi().await?;
+        send.write_all(b"Hello, world!").await?;
+        send.finish()?;
+        let response = recv.read_to_end(1000).await?;
+        assert_eq!(&response, b"Hello, world!");
+        conn.close(0u32.into(), b"bye!");
+
+        other_endpoint.close().await;
+    }
+
+    router.shutdown().await?;
+    Ok(())
+}
+
+#[derive(Debug, Clone)]
+struct Echo;
+
+impl ProtocolHandler for Echo {
+    async fn accept(&self, connection: Connection) -> Result<(), AcceptError> {
+        let (mut send, mut recv) = connection.accept_bi().await?;
+        // Echo bytes back until the sender signals the end of data.
+        tokio::io::copy(&mut recv, &mut send).await?;
+        send.finish()?;
+        // Wait until the other endpoint closes the connection.
+        connection.closed().await;
+        Ok(())
+    }
+}
+```
 
-Be cautious about adding new events.  Events aim for a high
-signal-to-noise ratio.  Events should be designed to be able to
-extract in an automated way.  If multiple events need to be related,
-fields with special values can be used.
+More examples live in [`iroh/examples`](examples). Run them with
+`cargo run --example NAME`. Details for each are in the file itself.
 
-To make events distinct from normal logging in the code it is
-recommended to write them using the `event!()` macro:
+## Compose protocols
 
-```rust
-event!(
-    target: "iroh::_event::subject",
-    Level::DEBUG,
-    field = value,
-);
-```
+Instead of writing your own, you can build on protocols that already exist on
+top of iroh:
 
-## Building documentation
+- [iroh-blobs] for [BLAKE3]-based content-addressed blob transfer, scaling from
+  kilobytes to terabytes.
+- [iroh-gossip] for publish-subscribe overlay networks that scale down to what
+  an average phone can handle.
+- and many more.
 
-Building the documentation is only supported when using
-`--all-features`.
+To use iroh from other languages, see [iroh-ffi].
 
-Additionally you might want to enable documenting the cargo features
-required for certain APIs, which is done by also passing the `--cfg
-iroh_docsrs` flag to rustdoc when building the documentation.  This
-also requires using nightly rust, e.g.:
+## Development
 
-```sh
-RUSTDOCFLAGS="--cfg iroh_docsrs" cargo +nightly doc --workspace --no-deps --all-features
-```
+For notes on iroh's structured events and how to build the documentation, see
+[DEVELOPMENT.md](DEVELOPMENT.md).
 
 # License
 
@@ -100,6 +162,22 @@ at your option.
 
 ### Contribution
 
+See [CONTRIBUTING.md](https://github.com/n0-computer/iroh/blob/main/CONTRIBUTING.md)
+for how to get involved.
+
 Unless you explicitly state otherwise, any contribution intentionally submitted
 for inclusion in this project by you, as defined in the Apache-2.0 license,
 shall be dual licensed as above, without any additional terms or conditions.
+
+[QUIC]: https://en.wikipedia.org/wiki/QUIC
+[hole-punch]: https://en.wikipedia.org/wiki/Hole_punching_(networking)
+[BLAKE3]: https://github.com/BLAKE3-team/BLAKE3
+[`Endpoint`]: https://docs.rs/iroh/latest/iroh/struct.Endpoint.html
+[`SecretKey`]: https://docs.rs/iroh/latest/iroh/struct.SecretKey.html
+[`EndpointId`]: https://docs.rs/iroh/latest/iroh/struct.EndpointId.html
+[address lookup services]: https://docs.rs/iroh/latest/iroh/address_lookup/index.html
+[`N0` preset]: https://docs.rs/iroh/latest/iroh/endpoint/presets/struct.N0.html
+[iroh-blobs]: https://github.com/n0-computer/iroh-blobs
+[iroh-gossip]: https://github.com/n0-computer/iroh-gossip
+[iroh-ffi]: https://github.com/n0-computer/iroh-ffi
+[n0]: https://n0.computer

iroh/examples/echo-no-router.rs

@@ -5,7 +5,7 @@
 //!
 //! ## Running the Example
 //!
-//!     cargo run --example echo-no-router --features=examples
+//!     cargo run --example echo-no-router
 
 use iroh::{Endpoint, EndpointAddr, endpoint::presets};
 use n0_error::{AnyError as Error, Result, StdResultExt};

iroh/examples/echo.rs

@@ -4,7 +4,7 @@
 //!
 //! ## Usage
 //!
-//!     cargo run --example echo --features=examples
+//!     cargo run --example echo
 
 use iroh::{
     Endpoint, EndpointAddr,

iroh/examples/screening-connection.rs

@@ -6,7 +6,7 @@
 //!
 //! ## Usage
 //!
-//!     cargo run --example screening-connection --features=examples
+//!     cargo run --example screening-connection
 use std::sync::{
     Arc,
     atomic::{AtomicU64, Ordering},

iroh/examples/search.rs

@@ -11,13 +11,13 @@
 //!
 //! In one terminal, run
 //!
-//!     cargo run --example search --features=examples  -- listen "hello-world" "foo-bar" "hello-moon"
+//!     cargo run --example search -- listen "hello-world" "foo-bar" "hello-moon"
 //!
 //! This spawns an iroh endpoint with three blobs. It will print the endpoint's endpoint id.
 //!
 //! In another terminal, run
 //!
-//!     cargo run --example search --features=examples  -- query <endpoint-id> hello
+//!     cargo run --example search -- query <endpoint-id> hello
 //!
 //! Replace <endpoint-id> with the endpoint id from above. This will connect to the listening endpoint with our
 //! protocol and query for the string `hello`. The listening endpoint will return a number of how many