-
Notifications
You must be signed in to change notification settings - Fork 3
Run Remote Test Scenario
In this page, we will run a test of the Remote Test Scenarios testing the Target Site availability, data search and download. We assume you have followed the first part of this guide and the Test Site Preparation
We are going to run the following Test Scenarios:
Tip
All Test Scenarios and their Test Cases are described in the Test Scenarios Description page.
The Test Scenarios are remotely executed on a virtual machines within the service providers' cloud infrastructure and ran using cdab-remote-client command line tool.
We will start the Test Site docker container we will use for this benchmark session.
docker run --detach --name testsite-1 esacdab/testsuite:latestYour Test Site Container should be ready. You can control it with the following command and yoiu should have a similar output
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a7c0ab615fe esacdab/testsuite:latest "/usr/bin/supervisor…" About a minute ago Up About a minute 22/tcp testsite-1Let's now copy the config.yaml file previously prepared to the container
docker cp config.yaml testsite-1:/home/jenkins/config.yamlFor the processing scenario running on a virtual machine in the T-System cloud, we also have to copy the private key file for the SSH access and the docker configuration file for the download of the docker image that is used for the processing.
docker cp cdab-key-cdse-ts.pem testsite-1:/home/jenkins/cdab-key-cdse-ts.pem
docker cp config.yaml testsite-1:/home/jenkins/config.yamlNote
The container is running as a non-root user. The config.yaml file is copied to the /home/jenkins directory which is the workspace directory where the user has write access.
In this section, we will run the Test Scenarios TS13 on Copernicus Data Space Ecosystem using the T-SYSTEMS cloud infrastructure.
You have to configure the config.yaml file and set the values in the compute section. They regard cloud authentication and VM parameters.
Make sure the private key file (private_key_file setting) for SSH connections is in the correct location.
Make also sure the docker authentication file is configured correctly under the global section. This is necessary for downloading the docker image used for the processing.
This Test Scenario will measure the computing performance of a typical Earth Observation Processing Algorithm on the Target Site cloud infrastructure. Most of the load shall be done on the virtual machine started on the remote infrastructure. The local container will simply wait for the completion and compile the metrics.
docker exec -it testsite-1 cdab-remote-client -v -conf=config.yaml -v -vm=1 -sp=cdse-ts -ts=cdse-ts -n=Local TS13The output of the console will show the progress of the Test Cases:
...
2020-11-06T15:33:16.331845Z [INFO] cdab-remote-client version 1.28
2020-11-06T15:33:16.331875Z [INFO] Checking for old resources to delete ...
2020-11-06T15:33:16.332142Z [DEBUG] Command: openstack server list -f json --os-auth-url https://iam.eu-de.otc.t-systems.com:443/v3 ...
2020-11-06T15:33:19.522224Z [DEBUG] Command: openstack volume list -f json --os-auth-url https://iam.eu-de.otc.t-systems.com:443/v3 ...
2020-11-06T15:33:22.744039Z [INFO] Done
2020-11-06T15:33:22.744171Z [INFO] Obtaining list of available floating IP addresses ...
2020-11-06T15:33:22.744480Z [DEBUG] Command: openstack floating\ ip list -f json --os-auth-url https://iam.eu-de.otc.t-systems.com:443/v3 ...
2020-11-06T15:33:25.131797Z [INFO] Available floating IP addresses: 80.158.58.122, 80.158.5.179, 80.158.7.134, 80.158.44.63, 80.158.47.173, 80.158.2.136
2020-11-06T15:33:25.132001Z [INFO] Start of execution
2020-11-06T15:33:25.132699Z [INFO] Creating virtual machine ...
2020-11-06T15:33:25.133040Z [DEBUG] Command: openstack server create --wait -f json --os-auth-url https://iam.eu-de.otc.t-systems.com:443/v3 ... --image Standard_CentOS_7_latest --flavor s2.xlarge.4 --security-group default --key-name cdab-key cdab-test-mundi-48ca4482
2020-11-06T15:33:48.463922Z [DEBUG] {...}
2020-11-06T15:33:48.464096Z [INFO] Virtual machine '506586c1-64f5-406b-8033-4673e22b6bb6' created
2020-11-06T15:33:48.464145Z [INFO] Assigning floating IP address ...
2020-11-06T15:33:48.464360Z [DEBUG] Command: openstack server add floating\ ip --os-auth-url https://iam.eu-de.otc.t-systems.com:443/v3 ... 506586c1-64f5-406b-8033-4673e22b6bb6 80.158.58.122
2020-11-06T15:33:53.511692Z [INFO] IP address 80.158.58.122 assigned explicitly
2020-11-06T15:33:53.511854Z [INFO] Awaiting SSH availability ...
2020-11-06T15:33:53.512372Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "ls"
...
2020-11-06T15:34:44.568503Z [INFO] Virtual machine available
...
2020-11-06T15:35:08.762503Z [INFO] Installing and starting docker ...
2020-11-06T15:35:08.762794Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sudo yum install -y yum-utils device-mapper-persistent-data lvm2"
2020-11-06T15:35:12.930048Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo"
2020-11-06T15:35:14.201981Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sudo yum install docker-ce docker-ce-cli containerd.io -y"
2020-11-06T15:35:55.694565Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sudo systemctl start docker"
...
2020-11-06T15:36:06.749615Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sudo usermod -a -G docker $USER"
2020-11-06T15:36:07.664600Z [INFO] Docker service started
2020-11-06T15:36:07.665044Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "mkdir .docker"
2020-11-06T15:36:08.546214Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... docker-config.json [email protected]:.docker/config.json
2020-11-06T15:36:09.552809Z [INFO] Installing docker image for test ...
2020-11-06T15:36:09.553309Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "docker pull esacdab/geohazards-tep/ewf-s3-olci-composites:0.41"
2020-11-06T15:51:17.480898Z [INFO] Docker image installed
2020-11-06T15:51:19.364119Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... config-mundi.yaml [email protected]:config.yaml
2020-11-06T15:51:20.460051Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... ts-scripts/s3-olci-composites.py [email protected]:./s3-olci-composites.py
2020-11-06T15:51:21.539316Z [INFO] Running processing test scenario TS13 ...
2020-11-06T15:51:21.539754Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... ts-scripts/TS13-remote.sh [email protected]:TS13-remote.sh
2020-11-06T15:51:22.558491Z [DEBUG] Command: ssh -i cdab-key-mundi.pem ... [email protected] "sh TS13-remote.sh . \"esacdab/geohazards-tep/ewf-s3-olci-composites:0.41\" MUNDI MUNDI ..."
This processing may take some time, due to the large size of the docker image it uses. After a while, once completed, the output will continue as follows:
2020-11-06T16:04:04.919408Z [INFO] Test completed
2020-11-06T16:04:04.921933Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... [email protected]:/mnt/cdab-volume/test/TS13Results.json TestResult-remote-48ca4482.json
2020-11-06T16:04:05.947749Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... [email protected]:/mnt/cdab-volume/test/junit.xml junit-remote--48ca4482.xml
2020-11-06T16:04:06.920356Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... [email protected]:/mnt/cdab-volume/test/cdab.stdout cdab-48ca4482.stdout
2020-11-06T16:04:07.897211Z [DEBUG] Command: scp -i cdab-key-mundi.pem ... [email protected]:/mnt/cdab-volume/test/cdab.stderr cdab-48ca4482.stderr
2020-11-06T16:04:08.888813Z [INFO] Test result files received
2020-11-06T16:04:08.888995Z [INFO] stdout and stderr from cdab-client execution on virtual machine below
2020-11-06T16:04:08.889055Z [INFO] --------------------------------
2020-11-06T16:04:08.889084Z [INFO] remote execution stdout (START)
...
('Executing processing graph\n....10%....20%....30%....40%....50%....60%....70%....80%....90% done.\n', 'INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters\nWARNING: org.esa.snap.core.util.EngineVersionCheckActivator: A new SNAP version is available for download.\nCurrently installed 6.0, available is 8.0.0.\nPlease visit http://step.esa.int\n\nINFO: org.hsqldb.persist.Logger: dataFileCache open start\nINFO: java.util.prefs.FileSystemPreferences$1: Created user preferences directory.\n')
Done.
k = S3 OLCI Natural Colors
BANDS = ['s3_olci.data/Oa08_reflectance.img', 's3_olci.data/Oa06_reflectance.img', 's3_olci.data/Oa04_reflectance.img', 's3_olci.data/pixel_classif_flags.img']
2020-11-06 16:03:29 (INFO): Default RAM limit for OTB is 128 MB
2020-11-06 16:03:29 (INFO): GDAL maximum cache size is 802 MB
2020-11-06 16:03:29 (INFO): OTB will use at most 4 threads
2020-11-06 16:03:29 (INFO): Estimated memory for full processing: 566.619MB (avail.: 128 MB), optimal image partitioning: 5 blocks
2020-11-06 16:03:29 (INFO): Estimation will be performed in 6 blocks of 2112x2112 pixels
2020-11-06T16:04:08.894477Z [INFO] remote execution stdout (END)
2020-11-06T16:04:08.894572Z [INFO] remote execution stderr (START)
...
WARNING:Fiona:PROJ data files not located, PROJ_LIB not set
EXIT CODE = 0
-rw-r--r--. 1 root root 30456665 Nov 6 16:04 S3-OLCI-NATURAL-COLORS-20201031T231058-20201031T231058.png
-rw-r--r--. 1 root root 891 Nov 6 16:04 S3-OLCI-NATURAL-COLORS-20201031T231058-20201031T231058.png.properties
-rw-r--r--. 1 root root 88060234 Nov 6 16:03 S3-OLCI-NATURAL-COLORS-20201031T231058-20201031T231058.tif
-rw-r--r--. 1 root root 879 Nov 6 16:03 S3-OLCI-NATURAL-COLORS-20201031T231058-20201031T231058.tif.properties
-rw-r--r--. 1 root root 88060234 Nov 6 16:03 S3-OLCI-NATURAL-COLORS-20201031T231058-20201031T231058.tif
2020-11-06T16:04:08.898047Z [INFO] remote execution stderr (END)
2020-11-06T16:04:08.898149Z [INFO] --------------------------------
...
2020-11-06T16:04:27.734130Z [INFO] Deleting virtual machine '506586c1-64f5-406b-8033-4673e22b6bb6' ...
2020-11-06T16:04:27.734348Z [DEBUG] Command: openstack server delete ... 506586c1-64f5-406b-8033-4673e22b6bb6
2020-11-06T16:04:30.956052Z [INFO] Virtual machine deleted
2020-11-06T16:04:30.958353Z [INFO] Test run finished
--------------------------------------------------------------------
Timing summary for Test run
* VM creation request: 2020-11-06T15:33:25.132777Z
* VM ready to use: 2020-11-06T15:34:44.568306Z
* Docker and image installation started: 2020-11-06T15:35:08.762622Z
* Test started: 2020-11-06T15:51:17.481430Z
* Test finished: 2020-11-06T16:04:04.918552Z
* Test results downloaded: 2020-11-06T16:04:08.888691Z
* VM deleted: 2020-11-06T16:04:30.955966Z
--------------------------------------------------------------------
Obtained metrics
* Error rate (%): 0.0
* Total duration (ms): 1865823
* Average provisioning latency (ms): 79436
* Average concurrency (#): 1
* Peak concurrency (#): 1
* Run 'Test run'
- Cost per hour (EUR): 0.0
- Cost per month (EUR): 0.0
- Duration (ms): 1865823
- Process duration (ms): 729043
- Provisioning latency (ms): 79436
--------------------------------------------------------------------
2020-11-06T16:04:30.984282Z [INFO] Output file written: TS13Results.json
2020-11-06T16:04:30.994425Z [DEBUG] Command: xmllint --format junit.xml.tmp
2020-11-06T16:04:31.002329Z [INFO] Output file written: junit.xml
2020-11-06T16:04:31.003467Z [INFO] End of execution... we copy the results
docker cp testsite-1:/home/jenkins/TS13Results.json testsite-1-TS13-cdse-ts-results.jsonAt the end of the benchmark session, when all reaults have been copied, we can stop and remove the the container.
docker stop testsite-1
docker rm testsite-1Warning
It is important to clean container after a benchmark session. Indeed, it contains its own image and all the data downloaded during the tests. Severall gigabytes of disk space may be used!
⏭️ You can now Perform an End to End Use Case