Skip to content

Commit 229c114

Browse files
authored
Rust updates, better doc testing (#598)
Capture a bunch of changes I had sitting around for a while. Fixes a bug in the mdbook, but otherwise no functional changes. Signed-off-by: Moritz Hoffmann <[email protected]>
1 parent 5323fd6 commit 229c114

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+89
-160
lines changed

README.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,14 @@ Be sure to read the [documentation for timely dataflow](https://docs.rs/timely).
1010

1111
To use timely dataflow, add the following to the dependencies section of your project's `Cargo.toml` file:
1212

13-
```
13+
```toml
1414
[dependencies]
1515
timely="*"
1616
```
1717

1818
This will bring in the [`timely` crate](https://crates.io/crates/timely) from [crates.io](http://crates.io), which should allow you to start writing timely dataflow programs like this one (also available in [timely/examples/simple.rs](https://github.com/timelydataflow/timely-dataflow/blob/master/timely/examples/simple.rs)):
1919

2020
```rust
21-
extern crate timely;
22-
2321
use timely::dataflow::operators::*;
2422

2523
fn main() {
@@ -32,7 +30,7 @@ fn main() {
3230

3331
You can run this example from the root directory of the `timely-dataflow` repository by typing
3432

35-
```
33+
```text
3634
% cargo run --example simple
3735
Running `target/debug/examples/simple`
3836
seen: 0
@@ -54,8 +52,6 @@ This is a very simple example (it's in the name), which only just suggests at ho
5452
For a more involved example, consider the very similar (but more explicit) [examples/hello.rs](https://github.com/timelydataflow/timely-dataflow/blob/master/timely/examples/hello.rs), which creates and drives the dataflow separately:
5553

5654
```rust
57-
extern crate timely;
58-
5955
use timely::dataflow::{InputHandle, ProbeHandle};
6056
use timely::dataflow::operators::{Input, Exchange, Inspect, Probe};
6157

@@ -96,7 +92,7 @@ We first build a dataflow graph creating an input stream (with `input_from`), wh
9692
We then drive the computation by repeatedly introducing rounds of data, where the `round` itself is used as the data. In each round, each worker introduces the same data, and then repeatedly takes dataflow steps until the `probe` reveals that all workers have processed all work for that epoch, at which point the computation proceeds.
9793

9894
With two workers, the output looks like
99-
```
95+
```text
10096
% cargo run --example hello -- -w2
10197
Running `target/debug/examples/hello -w2`
10298
worker 0: hello 0
@@ -120,7 +116,7 @@ The `hello.rs` program above will by default use a single worker thread. To use
120116
To use multiple processes, you will need to use the `-h` or `--hostfile` option to specify a text file whose lines are `hostname:port` entries corresponding to the locations you plan on spawning the processes. You will need to use the `-n` or `--processes` argument to indicate how many processes you will spawn (a prefix of the host file), and each process must use the `-p` or `--process` argument to indicate their index out of this number.
121117

122118
Said differently, you want a hostfile that looks like so,
123-
```
119+
```text
124120
% cat hostfile.txt
125121
host0:port
126122
host1:port
@@ -129,7 +125,7 @@ host3:port
129125
...
130126
```
131127
and then to launch the processes like so:
132-
```
128+
```text
133129
host0% cargo run -- -w 2 -h hostfile.txt -n 4 -p 0
134130
host1% cargo run -- -w 2 -h hostfile.txt -n 4 -p 1
135131
host2% cargo run -- -w 2 -h hostfile.txt -n 4 -p 2
@@ -187,7 +183,7 @@ The communication layer is based on a type `Content<T>` which can be backed by t
187183

188184
**NOTE**: Differential dataflow demonstrates how to do this at the user level in its `operators/arrange.rs`, if somewhat sketchily (with a wrapper that lies about the properties of the type it transports).
189185

190-
This would allow us to safely pass Rc<T> types around, as long as we use the `Pipeline` parallelization contract.
186+
This would allow us to safely pass `Rc<T>` types around, as long as we use the `Pipeline` parallelization contract.
191187

192188
## Coarse- vs fine-grained timestamps
193189

communication/Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,10 @@ license = "MIT"
1717
default = ["getopts"]
1818

1919
[dependencies]
20-
getopts = { version = "0.2.14", optional = true }
20+
getopts = { version = "0.2.21", optional = true }
2121
bincode = { version = "1.0" }
2222
byteorder = "1.5"
2323
serde = { version = "1.0", features = ["derive"] }
2424
timely_bytes = { path = "../bytes", version = "0.12" }
2525
timely_logging = { path = "../logging", version = "0.12" }
26-
crossbeam-channel = "0.5.0"
26+
crossbeam-channel = "0.5"

communication/examples/comm_hello.rs

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
extern crate timely_communication;
2-
31
use std::ops::Deref;
42
use timely_communication::{Message, Allocate};
53

communication/src/allocator/zero_copy/allocator.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ use std::cell::RefCell;
44
use std::collections::{VecDeque, HashMap, hash_map::Entry};
55
use crossbeam_channel::{Sender, Receiver};
66

7-
use bytes::arc::Bytes;
7+
use timely_bytes::arc::Bytes;
88

99
use crate::networking::MessageHeader;
1010

@@ -274,4 +274,4 @@ impl<A: Allocate> Allocate for TcpAllocator<A> {
274274
fn await_events(&self, duration: Option<std::time::Duration>) {
275275
self.inner.await_events(duration);
276276
}
277-
}
277+
}

communication/src/allocator/zero_copy/allocator_process.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ use std::cell::RefCell;
55
use std::collections::{VecDeque, HashMap, hash_map::Entry};
66
use crossbeam_channel::{Sender, Receiver};
77

8-
use bytes::arc::Bytes;
8+
use timely_bytes::arc::Bytes;
99

1010
use crate::networking::MessageHeader;
1111

@@ -250,4 +250,4 @@ impl Allocate for ProcessAllocator {
250250
}
251251
}
252252
}
253-
}
253+
}

communication/src/allocator/zero_copy/bytes_exchange.rs

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
use std::sync::{Arc, Mutex};
44
use std::collections::VecDeque;
55

6-
use bytes::arc::Bytes;
6+
use timely_bytes::arc::Bytes;
77
use super::bytes_slab::BytesSlab;
88

99
/// A target for `Bytes`.
@@ -177,4 +177,3 @@ impl<P: BytesPush> Drop for SendEndpoint<P> {
177177
self.send_buffer();
178178
}
179179
}
180-

communication/src/allocator/zero_copy/bytes_slab.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
//! A large binary allocation for writing and sharing.
22
3-
use bytes::arc::Bytes;
3+
use timely_bytes::arc::Bytes;
44

55
/// A large binary allocation for writing and sharing.
66
///
@@ -91,4 +91,4 @@ impl BytesSlab {
9191
}
9292
}
9393
}
94-
}
94+
}

communication/src/allocator/zero_copy/initialize.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ impl Drop for CommsGuard {
3131
}
3232

3333
use crate::logging::{CommunicationSetup, CommunicationEvent};
34-
use logging_core::Logger;
34+
use timely_logging::Logger;
3535

3636
/// Initializes network connections
3737
pub fn initialize_networking(

communication/src/allocator/zero_copy/push_pull.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ use std::rc::Rc;
44
use std::cell::RefCell;
55
use std::collections::VecDeque;
66

7-
use bytes::arc::Bytes;
7+
use timely_bytes::arc::Bytes;
88

99
use crate::allocator::canary::Canary;
1010
use crate::networking::MessageHeader;
@@ -139,4 +139,4 @@ impl<T:Data> Pull<Message<T>> for PullerInner<T> {
139139
&mut self.current
140140
}
141141
}
142-
}
142+
}

communication/src/allocator/zero_copy/tcp.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ use super::bytes_slab::BytesSlab;
99
use super::bytes_exchange::MergeQueue;
1010
use super::stream::Stream;
1111

12-
use logging_core::Logger;
12+
use timely_logging::Logger;
1313

1414
use crate::logging::{CommunicationEvent, CommunicationSetup, MessageEvent, StateEvent};
1515

0 commit comments

Comments
 (0)