-
Notifications
You must be signed in to change notification settings - Fork 22
Refactor the fre make readme and move quickstart #792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 7 commits
cb7be27
c85602f
dc35b65
3d216bf
af16acf
54ffe3d
e739212
4b410d3
bed9d85
594504d
b33cbd1
be74ca5
7813ee1
3c5d3f2
c9ac4ae
010ab34
2aa357d
73eaea9
1cbb8d2
3cc619f
d5387e9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,51 +7,111 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, | |
| - container creation | ||
| - parallel checkouts for bare-metal build** | ||
|
|
||
| ** **Note: Users will not be able to create containers without access to podman** | ||
| **Note: Users will not be able to create containers without access to podman** | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Guide | ||
| ## Quickstart | ||
|
|
||
| ### **Bare-metal Build:** | ||
| The quickstart instructions can be used with the null model example located in the fre-cli repository: https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
|
|
||
| Use fre make subtools to checkout code, create a Makefile, and compile code: | ||
| Users can clone the fre-cli repository and invoke the commands below from the root of the repository. | ||
| ### Bare-metal Build: | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```bash | ||
| # Create and run checkout script | ||
| fre make checkout-script -y [model yaml file] -p [platform] -t [target] --execute | ||
| # Create and run checkout script: checkout script will check out source code as defined in the compile.yaml | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
| fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute | ||
|
|
||
| # Create Makefile | ||
| fre make makefile -y [model yaml file] -p [platform] -t [target] | ||
| fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod | ||
|
|
||
| # Create and run the compile script | ||
| fre make compile-script -y [model yaml file] -p [platform] -t [target] --execute | ||
| # Create and run the compile script to generate a model executable | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
| fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute | ||
| ``` | ||
|
|
||
| Or use `fre make all` to do the job of all 3 subtools in one step: | ||
| ### Bare-metal Build (Multi-target): | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```bash | ||
| # Run fre make checkout-script, fre make makefile, and fre make compile-script in order | ||
| fre make all -y [model yaml file] -p [platform] -t [target] [other options...] | ||
| ``` | ||
| # Create and run checkout script: checkout script will check out source code as defined in the compile.yaml | ||
|
singhd789 marked this conversation as resolved.
Outdated
|
||
| fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute | ||
|
|
||
| # Create Makefile | ||
| fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug | ||
|
|
||
| ### **Container Build:** | ||
| ***To reiterate, users will not be able to create containers unless they have podman access on gaea.*** | ||
| # Create and run a compile script for each target specified; generates model executables | ||
| fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute | ||
| ``` | ||
|
|
||
| Use fre make subtools to checkout code, create a Makefile, and build a container: | ||
| ### Container Build: | ||
| In order for the container to build successfully, the parallel checkout feature is disabled. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hmm, maybe parallel checkout example should be given above for the bare-metal build?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the bare-metal build, the parallel checkout is the default behavior. Do you think users would want to see a non-parallel checkout example for the bare-metal build? |
||
|
|
||
| ```bash | ||
| # Create and run checkout script | ||
| fre make checkout-script -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute | ||
| # Create checkout script | ||
| fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod | ||
|
|
||
| # Create Makefile | ||
| fre make makefile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] | ||
| fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod | ||
|
|
||
| # Create and run the Dockerfile | ||
| fre make dockerfile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute | ||
| # Create the Dockerfile and container build script: the container build script (createContainer.sh) uses the Dockerfile to build a model container | ||
| fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute | ||
| ``` | ||
|
|
||
| Use `fre make all` to do the job of all 3 subtools in one step: | ||
| ### Run all of fremake: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "blackbox" fre make? maybe this should be at the top
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. made it "all-in-one"? Is this better? 😅 |
||
|
|
||
| `all` kicks off the compilation automatically | ||
|
|
||
| ```bash | ||
| # Run fre make checkout-script, fre make makefile, and fre make dockerfile in order | ||
| fre make all -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute | ||
| # Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable | ||
| fre make all -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute | ||
|
|
||
| # Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container | ||
| fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute | ||
| ``` | ||
|
|
||
| ## Subtools | ||
|
singhd789 marked this conversation as resolved.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. wondering if this section is needed...
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oook, hm, maybe I can re-format some things? I do kind of like the list but I also forgot (and just re-found) that a |
||
| - `fre make checkout-script [options]` | ||
| - Purpose: Create and run a checkout script. | ||
| - Options: | ||
| - `-y, --yamlfile [model yaml] (required)` | ||
| - `-p, --platform [platform] (required)` | ||
| - `-t, --target [target] (required)` | ||
| - `-gj, --gitjobs` | ||
| - `-npc, --no-parallel-checkout` | ||
| - `--execute` | ||
| - `--force-checkout` | ||
| - `fre make makefile [options]` | ||
| - Purpose: Create a Makefile. | ||
| - Options: | ||
| - `-y, --yamlfile [model yaml] (required)` | ||
| - `-p, --platform [platform] (required)` | ||
| - `-t, --target [target] (required)` | ||
| - `fre make compile-script [options]` | ||
| - Purpose: Create and run a compile script to generate a model executable. | ||
| - Options: | ||
| - `-y, --yamlfile [model yaml] (required)` | ||
| - `-p, --platform [platform] (required)` | ||
| - `-t, --target [target] (required)` | ||
| - `-n --nparallel` | ||
| - `-mj --makejobs` | ||
| - `-e, --execute` | ||
| - `-v, --verbose` | ||
| - `fre make dockerfile [options]` | ||
| - Purpose: Create and run a Dockerfile to generate a model container. | ||
| - Options: | ||
| - `-y, --yamlfile [model yaml] (required)` | ||
| - `-p, --platform [platform] (required)` | ||
| - `-t, --target [target] (required)` | ||
| - `-nft, --no-format-transfer` | ||
| - `-e, --execute` | ||
| - `fre make all [options]` | ||
| - Purpose: | ||
| - For a bare-metal build: Create a checkout script, Makefile, and compile script to generate a model executable | ||
| - For a container build: Create a checkout script, Makefile, and Dockerfile to generate a model container. | ||
| - Options: | ||
| - `-y, --yamlfile [model yaml] (required)` | ||
| - `-p, --platform [platform] (required)` | ||
| - `-t, --target [target] (required)` | ||
| - `-n --nparallel` | ||
| - `-mj --makejobs` | ||
| - `gj, --gitjobs` | ||
| - `-npc, --no-parallel-checkout` | ||
| - `-nft, --no-format-transfer` | ||
| - `-e, --execute` | ||
| - `-v, --verbose` | ||
Uh oh!
There was an error while loading. Please reload this page.