Fog Service Orchestration

Dynamic Deployment of the services on fog nodes (Raspberry Pi) on the fly with TOSCA Service Template and xOpera a lightweight orchestrator.

Tech Stack	Links
Standard	OASIS TOSCA
Orchestrator	xOpera
Implementation	Ansible

Table of Contents

• Environment Setup
  • System Native
  • Docker
• Orchestration
• Orchestration Behind NAT
• Service Templates
• Node Types
• Relationship Types
• Supporting Repositories
• License
• Citation

Environment Setup

---

System Native

---

Install dependencies

sudo apt install python3-venv python3-wheel python-wheel-common

Create Virtual Environment

python3 -m venv .venv

Activate

Activate python virtual environment

source .venv/bin/activate

Install Orchestrator

The xOpera orchestrator tool is available on PyPI.

pip install opera==0.6.8

Generate SSH Key Pair

The system running orchestrator should be able to login into each fog device through SSH without a password. So, generate SSH key pair and copy the public key to all other nodes.

ssh-keygen

Copy the Public Key

ssh-copy-id [email protected]

Set Environment Variable

By default xOpera login as centos username. To change the login username set the OPERA_SSH_USER environment variable. All the fog node's usernames should be the same.

export OPERA_SSH_USER=root

Deactivate

Deactivate python virtual environment

deactivate

---

Docker

---

• Install Docker
• Start orchestrator container (ARM64/AMD64 only)

docker container run \
-v ~/.ssh:/root/.ssh \
-it --rm suvambasak/orchestrator:latest \
"bash"

Orchestration

TOSCA Sevice Template validation, deployment, and undeployment with xOpera. Check out xOpera CLI documentation here.

Deploying demo template service-0.yaml developed for realizing Dynamic Deployment without fog devices (works with Personal Computers/Virtual Machine running Linux).

Validate

Validate TOSCA Service Templates
-e: executors (Ansible Playbooks) behind them
-i: Input file

opera validate -e -i inputs.yaml service-0.yaml 

Validation should look like this if nothing is wrong.

(.venv) dex@Suvams-Air tosca % opera validate -e -i inputs-0.yaml service-0.yaml 
Validating service template...
[Worker_0]   Validating fog-node-1_0
[Worker_0]   Validation of fog-node-1_0 complete
[Worker_0]   Validating fog-node-2_0
[Worker_0]   Validation of fog-node-2_0 complete
[Worker_0]   Validating docker-swarm-leader_0
[Worker_0]     Executing create on docker-swarm-leader_0
[Worker_0]     Executing delete on docker-swarm-leader_0
[Worker_0]   Validation of docker-swarm-leader_0 complete
[Worker_0]   Validating docker-swarm-worker-1_0
[Worker_0]     Executing create on docker-swarm-worker-1_0
[Worker_0]     Executing pre_configure_source on docker-swarm-worker-1_0--docker-swarm-leader_0
[Worker_0]     Executing delete on docker-swarm-worker-1_0
[Worker_0]   Validation of docker-swarm-worker-1_0 complete
[Worker_0]   Validating docker-service-1_0
[Worker_0]     Executing create on docker-service-1_0
[Worker_0]     Executing delete on docker-service-1_0
[Worker_0]   Validation of docker-service-1_0 complete
Done.

Deploy

Deploy TOSCA Service Templates
-i: Input file
-w: Number of concurrent threads

opera deploy -w 2 -i inputs.yaml service-0.yaml

If the deployment of the services is successful.

(.venv) dex@Suvams-Air tosca % opera deploy -w 2 -i inputs-0.yaml service-0.yaml
[Worker_0]   Deploying fog-node-1_0
[Worker_1]   Deploying fog-node-2_0
[Worker_1]   Deployment of fog-node-2_0 complete
[Worker_0]   Deployment of fog-node-1_0 complete
[Worker_0]   Deploying docker-swarm-leader_0
[Worker_0]     Executing create on docker-swarm-leader_0
[Worker_0]   Deployment of docker-swarm-leader_0 complete
[Worker_1]   Deploying docker-swarm-worker-1_0
[Worker_1]     Executing create on docker-swarm-worker-1_0
[Worker_1]     Executing pre_configure_source on docker-swarm-worker-1_0--docker-swarm-leader_0
[Worker_1]   Deployment of docker-swarm-worker-1_0 complete
[Worker_0]   Deploying docker-service-1_0
[Worker_0]     Executing create on docker-service-1_0
[Worker_0]   Deployment of docker-service-1_0 complete

  

Undeploy

Undeploy TOSCA Service Templates
-w: Number of concurrent threads

opera undeploy -w 2

If the undeployment of the services is successful.

(.venv) dex@Suvams-Air tosca % opera undeploy -w 2                              
[Worker_0]   Undeploying docker-service-1_0
[Worker_0]     Executing delete on docker-service-1_0
[Worker_0]   Undeployment of docker-service-1_0 complete
[Worker_0]   Undeploying docker-swarm-worker-1_0
[Worker_0]     Executing delete on docker-swarm-worker-1_0
[Worker_0]   Undeployment of docker-swarm-worker-1_0 complete
[Worker_0]   Undeploying fog-node-2_0
[Worker_1]   Undeploying docker-swarm-leader_0
[Worker_1]     Executing delete on docker-swarm-leader_0
[Worker_0]   Undeployment of fog-node-2_0 complete
[Worker_1]   Undeployment of docker-swarm-leader_0 complete
[Worker_0]   Undeploying fog-node-1_0
[Worker_0]   Undeployment of fog-node-1_0 complete

Orchestration Behind NAT

Orchestration Manager Node

• Start orchestration manager container

docker container run -p 80:7890 -it --rm suvambasak/master:latest

Orchestrator Node

• Pick one node under each NAT for the orchestration. The node must log in to each device in that NAT through SSH without a password (~/.ssh)
• Create a file HOST and list all the IP addresses of the device in that NAT

root@raspberrypi:~# cat /tmp/opera_config/HOST 
192.168.0.184
192.168.0.103

• Start the orchestrator container (IP=52.140.51.137 IP of the Orchestration Manager Node)

docker container run \
-v ~/.ssh:/root/.ssh \
-v /tmp/opera_config:/root/.config \
-e PORT=80 \
-e IP=52.140.51.137 \
-t suvambasak/orchestrator:latest

Service Templates

Template Name	Description
service 0	Demo Service Template for realizing Dynamic Deployment on VMs/PCs running Linux without IoT hardware.
service 1	Deploy remote LED service with privileged containers (actuation). On/Off LEDs from a Webpage.
service 2	Deploy remote LED service with System Service (actuation). On/Off LEDs from a Webpage.

Node Types

Node Types	Description
Docker Containers	Deploy/Undeploy Docker Containers from a docker-compose.yaml file.
Docker Services	Deploy/Undeploy Docker Stack from a docker-compose.yaml file.
Swarm Leader	Node type for Docker Swarm Leader.
Swarm Worker	Node type for Docker Swarm Worker.
System Service	Node type for systemctl service.

Relationship Types

Relationship Types	Description
Token Transfer	Swarm Worker nodes dependency on Swarm Manager node.

Supporting Repositories

Repository	Description
Docker	Dockerfile, Docker-Compose, Python Scripts, and Webpages
Pi System Services	Service files, Python Scripts, and Configuration files
System setup and utility tools	System setup and Monitoring tools

License

This work is licensed under the Apache License 2.0.

Citation

• Cite this work

@article{https://doi.org/10.1002/nem.2246,
author = {Basak, Suvam and Srirama, Satish Narayana},
title = {Fog computing out of the box: Dynamic deployment of fog service containers with TOSCA},
journal = {International Journal of Network Management},
volume = {n/a},
number = {n/a},
pages = {e2246},
keywords = {docker, dynamic deployment, fog computing, Internet of Things, TOSCA},
doi = {https://doi.org/10.1002/nem.2246},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/nem.2246},
eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1002/nem.2246}
}

• Cite the case study

@INPROCEEDINGS{9861005,
  author={Srirama, Satish Narayana and Basak, Suvam},
  booktitle={2022 IEEE 15th International Conference on Cloud Computing (CLOUD)}, 
  title={Fog Computing out of the Box with FogDEFT Framework: A Case Study}, 
  year={2022},
  volume={},
  number={},
  pages={342-350},
  keywords={Performance evaluation;Cloud computing;Computational modeling;Prototypes;Real-time systems;Topology;Internet of Things;Fog computing;Internet of Things;Dynamic deployment;TOSCA;Docker},
  doi={10.1109/CLOUD55607.2022.00057}}