Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
4a39560
first adjustments for cloudevent support
linakrisztian Apr 17, 2025
a99d5ca
first version for cloudevent endpoint + update setup
linakrisztian Apr 22, 2025
00c5bcb
add test files
linakrisztian Apr 23, 2025
ccca3b6
setup as standalone app
linakrisztian Apr 23, 2025
25b6374
update cloudevent processing: receiver url within config + added desc…
linakrisztian Apr 23, 2025
b493177
update setup: only standalone
linakrisztian Apr 24, 2025
544cd7d
fix standalone setup
linakrisztian Apr 24, 2025
563af7d
remove not needed files
linakrisztian Apr 24, 2025
5d36068
Update __init__.py
linakrisztian Jun 2, 2025
71280db
Update __init__.py
linakrisztian Jun 2, 2025
5f7587b
remove not needed parts
linakrisztian Jun 2, 2025
9383695
Merge branch 'cloudevent_support' of github.com:actinia-org/actinia-c…
linakrisztian Jun 2, 2025
93f81e8
black linting
linakrisztian Jun 2, 2025
fc61eab
linting flake8
linakrisztian Jun 2, 2025
f108f1f
lint black
linakrisztian Jun 2, 2025
72f17fd
lint hadolint
linakrisztian Jun 2, 2025
f83becd
remove exmplae plugin tests
linakrisztian Jun 2, 2025
eea12c7
linting_pylint
linakrisztian Jun 2, 2025
6d54e3b
lint black
linakrisztian Jun 2, 2025
527a1fc
linting
linakrisztian Jun 2, 2025
24018e7
update readme
linakrisztian Jun 2, 2025
35fcc9f
add pylint exceptions
linakrisztian Jun 2, 2025
991b904
fix pytlint exception
linakrisztian Jun 2, 2025
9444d62
pylint
linakrisztian Jun 2, 2025
20978ba
pylint
linakrisztian Jun 2, 2025
e290192
fix hadolint exceptions
linakrisztian Jun 2, 2025
2221209
ruff
linakrisztian Jun 2, 2025
f1e3d44
ruff
linakrisztian Jun 2, 2025
1373bf3
ruff
linakrisztian Jun 2, 2025
f6e3e79
black
linakrisztian Jun 2, 2025
5e03ba7
Update src/actinia_cloudevent_plugin/resources/config.py
linakrisztian Jun 2, 2025
532b352
black
linakrisztian Jun 2, 2025
3c8e69a
linting
linakrisztian Jun 2, 2025
ba5392a
update docker setup
linakrisztian Jun 2, 2025
d09680c
update documentation
linakrisztian Jun 2, 2025
f63f33d
update authors
linakrisztian Jun 2, 2025
1ad5dd5
Update response_models.py
linakrisztian Jun 2, 2025
1950fb5
update test setup
linakrisztian Jun 2, 2025
aef15f2
Merge branch 'cloudevent_support' of github.com:actinia-org/actinia-c…
linakrisztian Jun 2, 2025
3e1d18e
add integrationtest
linakrisztian Jun 3, 2025
1c1a867
update reademe for tests + add examples + remove unittest
linakrisztian Jun 3, 2025
3551bb2
json linting
linakrisztian Jun 3, 2025
74a257d
fix prod setup + improve error message
linakrisztian Jun 3, 2025
a284be8
update readme/comments
linakrisztian Jun 3, 2025
f9ae552
stop server within tests
linakrisztian Jun 3, 2025
b4adb25
Update src/actinia_cloudevent_plugin/resources/config.py
linakrisztian Jun 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 52 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,62 @@

This is an plugin for [actinia-core](https://github.com/mundialis/actinia_core) which adds cloudevent endpoints to actinia-core.

You can run actinia-cloudevent-plugin as an actinia-core plugin.
You can run actinia-cloudevent-plugin as an actinia-core plugin or as standalone app.

## Installation
## As standalone app
### With docker
TODO

### Without docker - TODO: remove
#### Requirements
```
sudo apt install \
python3-venv\
python3\
```
#### Installation
For local developments outside of docker, it is preferred to run actinia-cloudevent-plugin in a virtual python environment.

Clone repository, create virtual environment and activate it, and install actinia-cloudevent-plugin:
```bash
git clone git@github.com:mundialis/actinia-cloudevent-plugin.git
cd actinia-cloudevent-plugin
virtualenv -p python3 venv
. venv/bin/activate
pip3 install .
```
Run tests:
```
python3 -m pytest
```
Run the server for development:
```bash
python3 -m actinia_cloudevent_plugin.main
```
Or for production use actinia_gdi.wsgi as WSGI callable:
```bash
gunicorn -b :5000 -w 1 --access-logfile=- -k gthread actinia_cloudevent_plugin.wsgi
```
If all done, leave environment
```
deactivate
```

## As actinia-core plugin
### Installation with docker
Use docker-compose for installation:
```bash
docker compose -f docker/docker-compose.yml build
docker compose -f docker/docker-compose.yml up -d
docker compose -f docker/docker-compose-plugin.yml build
docker compose -f docker/docker-compose-plugin.yml up -d
```

### Installation hints
* If you get an error like: `ERROR: for docker_kvdb_1 Cannot start service valkey: network xxx not found` you can try the following:
```bash
docker compose -f docker/docker-compose.yml down
docker compose -f docker/docker-compose-plugin.yml down
# remove all custom networks not used by a container
docker network prune
docker compose -f docker/docker-compose.yml up -d
docker compose -f docker/docker-compose-plugin.yml up -d
```

### Requesting helloworld endpoint
Expand All @@ -28,11 +68,13 @@ curl -u actinia-gdi:actinia-gdi -X GET http://localhost:8088/api/v3/helloworld |
curl -u actinia-gdi:actinia-gdi -H 'accept: application/json' -H 'Content-Type: application/json' -X POST http://localhost:8088/api/v3/helloworld -d '{"name": "test"}' | jq
```

## DEV setup
For a DEV setup you can use the docker/docker-compose.yml:
## DEV setup - as actinia-core plugin
For a DEV setup you can use the docker/docker-compose-plugin.yml:
```bash
docker compose -f docker/docker-compose.yml build
docker compose -f docker/docker-compose.yml run --rm --service-ports --entrypoint sh actinia
docker compose -f docker/docker-compose-plugin.yml build
docker compose -f docker/docker-compose-plugin.yml run --rm --service-ports --entrypoint sh actinia
# OR with mounted source code
docker compose -f docker/docker-compose-plugin.yml run --volume `pwd`:/src/actinia_cloudevent-plugin/ --rm --service-ports --entrypoint sh actinia

# install the plugin
(cd /src/actinia-cloudevent-plugin && python3 setup.py install)
Expand Down
4 changes: 1 addition & 3 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,8 @@ COPY docker/actinia.cfg /etc/default/actinia
COPY src /src/actinia-cloudevent-plugin/src/
COPY setup.cfg /src/actinia-cloudevent-plugin/
COPY setup.py /src/actinia-cloudevent-plugin/
COPY requirements.txt /src/actinia-cloudevent-plugin/

RUN pip3 install --no-cache-dir -r /src/actinia-cloudevent-plugin/requirements.txt && \
pip3 uninstall actinia-cloudevent-plugin.wsgi -y
RUN pip3 uninstall actinia-cloudevent-plugin.wsgi -y
# SETUPTOOLS_SCM_PRETEND_VERSION is only needed if in the plugin folder is no
# .git folder
ENV SETUPTOOLS_SCM_PRETEND_VERSION=0.0
Expand Down
2 changes: 1 addition & 1 deletion docker/actinia-cloudevent-plugin-test/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ ENV DEFAULT_CONFIG_PATH=/etc/default/actinia-cloudevent-plugin-test
# DL3018: Pin versions in apk add
# hadolint ignore=DL3018
RUN apk add --no-cache valkey && \
pip3 install --no-cache-dir iniconfig==2.0.0 colorlog==6.8.2 pwgen==0.8.2.post0 pytest==8.3.5 pytest-cov==6.0.0
pip3 install --no-cache-dir pwgen==0.8.2.post0 pytest==8.3.5 pytest-cov==6.0.0

# COPY docker/actinia-cloudevent-plugin-test/start.sh /src/start.sh

Expand Down
2 changes: 1 addition & 1 deletion docker/actinia.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ plugins = ["actinia_cloudevent_plugin"]
force_https_urls = False

[KVDB]
kvdb_server_url = valkey
kvdb_server_url = localhost
kvdb_server_pw = pass
kvdb_resource_expire_time = 864001
worker_logfile = /actinia_core/workspace/tmp/actinia_worker.log
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ services:
- valkey
cap_add:
- SYS_PTRACE
networks:
- actinia
network_mode: "host"

valkey:
image: valkey/valkey:8.1-alpine
Expand All @@ -32,8 +31,7 @@ services:
]
ports:
- "6379:6379"
networks:
- actinia
network_mode: "host"

networks:
actinia:
Expand Down
9 changes: 9 additions & 0 deletions docker/docker-compose-standalone.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
version: "3"
services:

actinia-cloudevent:
image: actinia-cloudevent
ports:
- "5000:5000"
volumes:
- ./actinia-cloudevent-data:/opt/app-root/src/config/mount
9 changes: 7 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,13 @@ keywords = [
"example",
]
dependencies = [
"colorlog>=4.2.1",
"xmltodict",
"Flask",
"Flask-Cors>=3.0.3",
"flask-restful-swagger-2>=0.35",
"colorlog>=3.1.4",
"python-json-logger",
"requests>=2.20.0",
"cloudevents",
]

[project.optional-dependencies]
Expand Down
17 changes: 0 additions & 17 deletions requirements.txt

This file was deleted.

2 changes: 1 addition & 1 deletion src/actinia_cloudevent_plugin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

__license__ = "GPLv3"
__author__ = "Carmen Tawalika, Anika Weinmann"
__copyright__ = "Copyright 2022 mundialis GmbH & Co. KG"
__copyright__ = "Copyright 2018-2025 mundialis GmbH & Co. KG"
__maintainer__ = "mundialis GmbH & Co. KG"


Expand Down
2 changes: 1 addition & 1 deletion src/actinia_cloudevent_plugin/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""actinia-example-plguin API part of package.
"""actinia-cloudevent-plugin API part of package.

This part provides the API part of the actinia-cloudevent-plugin.
"""
54 changes: 54 additions & 0 deletions src/actinia_cloudevent_plugin/api/cloudevent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/usr/bin/env python
"""Copyright (c) 2025 mundialis GmbH & Co. KG.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

Hello World class
"""

__license__ = "GPLv3"
__author__ = "Lina Krisztian"
__copyright__ = "Copyright 2025 mundialis GmbH & Co. KG"
__maintainer__ = "mundialis GmbH & Co. KG"


from flask import make_response, request

Check failure on line 26 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (F401)

src/actinia_cloudevent_plugin/api/cloudevent.py:26:34: F401 `flask.request` imported but unused

Check failure on line 26 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (F401)

src/actinia_cloudevent_plugin/api/cloudevent.py:26:19: F401 `flask.make_response` imported but unused
from flask_restful_swagger_2 import Resource, swagger

from actinia_cloudevent_plugin.apidocs import cloudevent
from actinia_cloudevent_plugin.core.processing import receive_cloud_event, cloud_event_to_process_chain, send_binary_cloud_event #, send_structured_cloud_event

Check failure on line 30 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (E262)

src/actinia_cloudevent_plugin/api/cloudevent.py:30:130: E262 Inline comment should start with `# `

Check failure on line 30 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (E261)

src/actinia_cloudevent_plugin/api/cloudevent.py:30:129: E261 Insert at least two spaces before an inline comment

Check failure on line 30 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (E501)

src/actinia_cloudevent_plugin/api/cloudevent.py:30:80: E501 Line too long (159 > 79)
from actinia_cloudevent_plugin.model.response_models import (
SimpleStatusCodeResponseModel,
)


class Cloudevent(Resource):

Check failure on line 36 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (I001)

src/actinia_cloudevent_plugin/api/cloudevent.py:26:1: I001 Import block is un-sorted or un-formatted
"""Receives cloudevent, transorms to process chain,
and returns cloudevent with queue name"""

Check failure on line 38 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (D415)

src/actinia_cloudevent_plugin/api/cloudevent.py:37:5: D415 First line should end with a period, question mark, or exclamation point

Check failure on line 38 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (D400)

src/actinia_cloudevent_plugin/api/cloudevent.py:37:5: D400 First line should end with a period

Check failure on line 38 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (D209)

src/actinia_cloudevent_plugin/api/cloudevent.py:37:5: D209 Multi-line docstring closing quotes should be on a separate line

Check failure on line 38 in src/actinia_cloudevent_plugin/api/cloudevent.py

View workflow job for this annotation

GitHub Actions / lint / ruff

Ruff (D205)

src/actinia_cloudevent_plugin/api/cloudevent.py:37:5: D205 1 blank line required between summary line and description

def __init__(self) -> None:
"""Cloudevent class initialisation."""
self.msg = "Received event <EVENT1> and returned event <EVENT2> with queue <QUEUE>."

@swagger.doc(cloudevent.describe_cloudevent_post_docs)
def post(self) -> SimpleStatusCodeResponseModel:
"""Cloudevent post method with cloudevent from postbody."""
# Transform postbody to cloudevent
event_received = receive_cloud_event()
# Received process chain to queue name
queue_name = cloud_event_to_process_chain(event_received)
# TODO: binary or structured cloud event?
event_returned = send_binary_cloud_event(event_received, queue_name, event_received["cloudeventreceiver"])
return SimpleStatusCodeResponseModel(status=204, message=self.msg.replace("<EVENT1>",event_received["id"]).replace("<EVENT2>", event_returned["id"]).replace("<QUEUE>" ,queue_name))

16 changes: 5 additions & 11 deletions src/actinia_cloudevent_plugin/api/project_helloworld.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,7 @@

from typing import ClassVar

from actinia_core.models.response_models import SimpleResponseModel
from actinia_core.rest.base.deprecated_locations import (
location_deprecated_decorator,
)
from actinia_cloudevent_plugin.model.response_models import SimpleResponseModel, SimpleStatusCodeResponseModel
from flask import jsonify, make_response, request
from flask.wrappers import Response
from flask_restful_swagger_2 import Resource, swagger
Expand All @@ -40,19 +37,16 @@
class ProjectHelloWorld(Resource):
"""Returns 'Hello world with project/location!'."""

decorators: ClassVar[list] = []

# Add decorators for deprecated GRASS GIS locations
decorators.append(location_deprecated_decorator)

def __init__(self) -> None:
"""Project hello world class initialisation."""
self.msg = "Project: Hello world!"

@swagger.doc(project_helloworld.describe_project_hello_world_get_docs)
def get(self, project_name: str) -> Response:
def get(self) -> Response:
"""Get 'Hello world!' as answer string."""
msg = f"{self.msg} {project_name}"
msg = f"{self.msg}"
return make_response(
jsonify(
SimpleResponseModel(
Expand All @@ -64,13 +58,13 @@ def get(self, project_name: str) -> Response:
)

@swagger.doc(project_helloworld.describe_project_hello_world_post_docs)
def post(self, project_name: str) -> Response:
def post(self):
"""Hello World post method with name from postbody."""
req_data = request.get_json(force=True)
if isinstance(req_data, dict) is False or "name" not in req_data:
return make_response("Missing name in JSON content", 400)
name = req_data["name"]
msg = f"{self.msg} {transform_input(name)} {project_name}"
msg = f"{self.msg} {transform_input(name)}"

return make_response(
jsonify(
Expand Down
2 changes: 1 addition & 1 deletion src/actinia_cloudevent_plugin/apidocs/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""actinia-example-plguin API DOCs part of package.
"""actinia-cloudevent-plugin API DOCs part of package.

This part provides the API DOCs part of the actinia-cloudevent-plugin.
"""
45 changes: 45 additions & 0 deletions src/actinia_cloudevent_plugin/apidocs/cloudevent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/usr/bin/env python
"""Copyright (c) 2025 mundialis GmbH & Co. KG.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

Hello World class
"""

__license__ = "GPLv3"
__author__ = "Lina Krisztian"
__copyright__ = "Copyright 2025 mundialis GmbH & Co. KG"
__maintainer__ = "mundialis GmbH & Co. KG"


from actinia_cloudevent_plugin.model.response_models import (
SimpleStatusCodeResponseModel,
)

describe_cloudevent_post_docs = {
# "summary" is taken from the description of the get method
"tags": ["cloudevent"],
"description": "Receives cloudevent, generates queue name and returns cloudevent.",
"responses": {
"200": {
"description": ("This response returns received, and returned events, "
"generated queue name and the status"),
"schema": SimpleStatusCodeResponseModel,
},
"400": {
"description": "This response returns an error message",
"schema": SimpleStatusCodeResponseModel,
},
},
}
2 changes: 1 addition & 1 deletion src/actinia_cloudevent_plugin/core/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""actinia-example-plguin core part of package.
"""actinia-cloudevent-plugin core part of package.

This part provides the core part of the actinia-cloudevent-plugin.
"""
Loading
Loading