Skip to content

Use the new R decorator wrappers in the docs #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: r
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ metaflow("HelloAWSFlow") %>%
r_function = start,
next_step = "hello") %>%
step(step = "hello",
decorator("retry", times=2),
decorator("batch", cpu=2, memory=2048),
retry(times=2),
batch(cpu=2, memory=2048),
r_function = hello,
next_step = "end") %>%
step(step = "end",
Expand Down
2 changes: 1 addition & 1 deletion metaflow-on-aws/metaflow-sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Only a limited number of sandboxes are available. When you sign up, you are adde
Here are some ideas that you can try with the sandbox:

* [The season 2 of tutorials](../getting-started/tutorials/season-2-scaling-out-and-up/) focuses on scaling out. This is a good way to get started. Note that the Season 1 tutorials work with the Sandbox too, when executed using [the `batch` decorator](../metaflow/scaling.md).
* You have up to 64 CPU cores at your disposal using [the `batch` decorator](../metaflow/scaling.md). Test some number crunching! You can run everything in the cloud simply by or you can mix local and remote steps by adding `decorator("batch",...)` to select steps.
* You have up to 64 CPU cores at your disposal using [the `batch` decorator](../metaflow/scaling.md). Test some number crunching! You can run everything in the cloud simply by or you can mix local and remote steps by adding `batch(...)` to select steps.
* Test your favorite ML libraries in the cloud using [`batch`](../metaflow/scaling.md) decorator. For instance, try a basic hyper-parameter search using [a custom parameter grid and foreach](../metaflow/basics.md#foreach).
* Evaluate Metaflow's [experiment tracking and versioning](../metaflow/tagging.md) using local runs and the [Client API](../metaflow/client.md) in a local notebook. In contrast to the local mode, all runs are registered globally in the Metaflow Service regardless of the directory where you run them.
* Test how you can [`resume` tasks locally](../metaflow/debugging.md#how-to-use-the-resume-command) which were originally run remotely using [the `batch` decorator](../metaflow/scaling.md).
Expand Down
10 changes: 5 additions & 5 deletions metaflow/failures.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ end <- function(self){

metaflow("RetryFlow") %>%
step(step="start",
decorator("retry"),
retry(),
r_function=start,
next_step="end") %>%
step(step="end",
Expand Down Expand Up @@ -115,7 +115,7 @@ you may end up withdrawing up to $4000 instead of the intended $1000. To make su
metaflow("MoneyFlow") %>%
...
step(step="withdraw",
decorator("retry", times=0),
retry(times=0),
r_function=withdraw_money_from_account,
next_step="end") %>%
...
Expand Down Expand Up @@ -195,7 +195,7 @@ metaflow("CatchFlow") %>%
next_step = "sanity_check",
foreach = "params") %>%
step(step = "sanity_check",
decorator("catch", var="compute_failed", print_exception=FALSE),
catch(var="compute_failed", print_exception=FALSE),
r_function = sanity_check,
next_step = "join") %>%
step(step = "join",
Expand Down Expand Up @@ -234,8 +234,8 @@ end <- function(self) {

metaflow("SuicidalFlowR") %>%
step(
decorator("catch", var = "start_failed"),
decorator("retry"),
catch(var = "start_failed"),
retry(),
step = "start",
r_function = start,
next_step = "end"
Expand Down
4 changes: 2 additions & 2 deletions metaflow/scaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ end <- function(self) {

metaflow("BigSumFlowR") %>%
step(
decorator("resources", memory=60000, cpu=1),
resources(memory=60000, cpu=1),
step = "start",
r_function = start,
next_step = "end"
Expand All @@ -57,7 +57,7 @@ This example creates a huge 80000x80000 random matrix, `big_matrix`. The matrix
If you attempt to run this on your local machine, it is likely that the following will happen:

```bash
Evaluation error: vector memory exhausted (limit reached?).
Evaluation error: vector memory exhausted (limit reached?).
```

This fails quickly due to a `MemoryError` on most laptops as we are unable to allocate 48GB of memory.
Expand Down