Skip to content

Commit f0e804a

Browse files
committed
Update README
1 parent ce18814 commit f0e804a

File tree

1 file changed

+39
-45
lines changed
  • tests/integration/bash_tests/run_from_chrysalis/blocking_test_scripts

1 file changed

+39
-45
lines changed
Lines changed: 39 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -3,93 +3,87 @@ This document outlines the procedures conducted to test the zstash bloclking
33
and non-blocking behavior.
44

55
Note: As it was intended to test blocking with regard to archive tar-creations
6-
vs Globus transfers, it wsa convenient to have both source snd destination be
7-
the same Globus endpoint. Effectively, we are employing Globus merely to move
6+
vs Globus transfers, it was convenient to have both source and destination be
7+
the same Globus endpoint. Effectively, we are employing Globus merely to move
88
tar archive files from one directory to another on the same file system.
99

1010
The core intent in implementing zstash blocking is to address a potential
1111
"low-disk" condition, where tar-files created to archive source files could
12-
add substantially to the disk load. To avoid disk exhaustion, "blocking"
13-
("--non-blocking" is absent on the command line), tar-file creation will
12+
add substantially to the disk load. To avoid disk exhaustion, when "blocking"
13+
(`--non-blocking` is absent on the command line), tar-file creation will
1414
pause to wait for the previous tarfile globus transfer to complete, so that
1515
the local copy can be deleted before the next tar-file is created.
1616

1717
I. File System Setup
1818
====================
1919

2020
As one may want, or need to re-conduct testing under varied conditions, the
21-
test script:
22-
23-
test_zstash_blocking.sh
24-
25-
will establish the following directory structure in the operator's current
21+
test script `test_zstash_blocking.sh` will establish the following directory structure in the operator's current
2622
working directory:
2723

28-
[CWD]/src_data/
29-
30-
- contains files to be tar-archived. One can experiment
31-
with different sizes of files to trigger behaviors.
32-
33-
[CWD]/src_data/zstash/
34-
35-
- default location of tarfiles produced. This directory is
36-
created automatically by zstash unless "--cache" indicates
37-
an alternate location.
38-
39-
[CWD]/dst_data/
24+
```
25+
[CWD]/src_data/
26+
# Contains files to be tar-archived.
27+
# One can experiment with different sizes of files to trigger behaviors.
4028
41-
- destination for Globus transfer of archives.
29+
[CWD]/src_data/zstash/
30+
# Default location of tarfiles produced.
31+
# This directory is created automatically by zstash unless
32+
# "--cache" indicates an alternate location.
4233
43-
[CWD]/tmp_cache/
34+
[CWD]/dst_data/
35+
# Destination for Globus transfer of archives.
4436
45-
- [Optional] alternative location for tar-file generation.
37+
[CWD]/tmp_cache/
38+
# [Optional] alternative location for tar-file generation.
39+
```
4640

4741
Note: It may be convenient to create a "hold" directory to store files of
4842
various sizes that can be easily produced by running the supplied scripts.
4943

50-
gen_data.sh
51-
gen_data_runner.sh
44+
```
45+
gen_data.sh
46+
gen_data_runner.sh
47+
```
5248

5349
The files to be used for a given test must be moved or copied to the src_data
5450
directory before a test is initiated.
5551

56-
Note: It never hurts to run the supplied script:
57-
58-
reset_test.sh
59-
60-
before a test run. This will delete any archives in the src_data/zstash
52+
Note: It never hurts to run the supplied script `reset_test.sh` before a test run. This will delete any archives in the src_data/zstash
6153
cache and the receiving dst_data directories, and delete the src_data/zstash
62-
directory itself if it exists. This ensures a clean restart for testing.
54+
directory itself if it exists. This ensures a clean restart for testing.
6355
The raw data files placed into src_data are not affected.
6456

6557
II. Running the Test Script
6658
===========================
6759

6860
The test script "test_zstash_blocking.sh" accepts two positional parameters:
6961

70-
test_zstash_blocking.sh (BLOCKING|NON_BLOCKING)
62+
```
63+
test_zstash_blocking.sh (BLOCKING|NON_BLOCKING)
64+
```
7165

72-
If "BLOCKING" is selected, zstash will run in default mode, waiting for
66+
If `BLOCKING` is selected, zstash will run in default mode, waiting for
7367
each tar file to complete transfer before generating another tar file.
7468

75-
If "NON_BLOCKING" is selected, the zstash flag "--non-blocking" is supplied
69+
If `NON_BLOCKING` is selected, the zstash flag `--non-blocking` is supplied
7670
to the zstash command line, and tar files continue to be created in parallel
7771
to running Globus transfers.
7872

7973
It is suggested that you run the test script with
8074

81-
test_zstash_blocking.sh (BLOCKING|NON_BLOCKING) > your_logfile 2>&1
82-
83-
so that your command prompt returns and you can monitor progress with
75+
```
76+
test_zstash_blocking.sh (BLOCKING|NON_BLOCKING) 2>&1 | tee your_logfile
77+
```
8478

85-
snapshot.sh
86-
87-
which will provide a view of both the tarfile cache and the destination
79+
so that your command prompt returns and you can monitor progress with `snapshot.sh`, which will provide a view of both the tarfile cache and the destination
8880
directory for delivered tar files. It is also suggested that you name your
89-
logfile to reflect the date, and whether BLOCKING or not was specified.
90-
81+
logfile to reflect the date, and whether BLOCKING or not was specified. Example:
82+
```bash
83+
test_zstash_blocking.sh BLOCKING 2>&1 | tee test_zstash_blocking_20251020.log
84+
```
9185

92-
FINAL NOTE: In the zstash code, the tar file "MINSIZE" parameter is taken
93-
to be (int) multiples of 1 GB. During testing, this had been changed to
86+
FINAL NOTE: In the zstash code, the tar file `MINSIZE` parameter is taken
87+
to be (int) multiples of 1 GB. During testing, this had been changed to
9488
"multiple of 100K" for rapid testing. It may be useful to expose this as
9589
a command line parameter for debugging purposes.

0 commit comments

Comments
 (0)