Enhance documentation and setup scripts for Dynatrace Snowflake Obser…

.github/workflows/release.yml

@@ -0,0 +1,100 @@
+name: Build and Release
+
+on:
+  push:
+    branches:
+      - main
+      - devel
+    tags:
+      - "v*"
+  workflow_dispatch:
+    inputs:
+      create_release:
+        description: "Create a draft release (for testing)"
+        required: false
+        type: boolean
+        default: false
+
+jobs:
+  build-and-release:
+    name: Build and Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write # Needed to create releases
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # Required for git history access
+
+      - name: Install build dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y libcairo2 libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 libffi-dev pandoc
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+
+      - name: Get Version
+        id: get_version
+        run: |
+          if [[ "${{ github.ref_type }}" == "tag" ]]; then
+            VERSION_NUMBER="${{ github.ref_name }}"
+            VERSION_NUMBER="${VERSION_NUMBER#v}"
+          else
+            VERSION_NUMBER=$(grep '^VERSION' src/dtagent/version.py | cut -d '"' -f 2)
+            if [[ -z "$VERSION_NUMBER" ]]; then
+              echo "Error: Failed to extract version from src/dtagent/version.py. File may be missing or format may be incorrect."
+              exit 1
+            fi
+          fi
+          echo "VERSION=${VERSION_NUMBER}" >> $GITHUB_OUTPUT
+
+      - name: Clean up previous build artifacts
+        run: |
+          rm -f dynatrace_snowflake_observability_agent-*.zip
+          rm -f Dynatrace-Snowflake-Observability-Agent-*.pdf
+
+      - name: Run package script
+        id: package
+        run: |
+          chmod +x *.sh
+          ./package.sh full
+          BUILD_NUMBER=$(grep 'BUILD =' build/_version.py | awk '{print $3}')
+          echo "ZIP_NAME=dynatrace_snowflake_observability_agent-${{ steps.get_version.outputs.VERSION }}.${BUILD_NUMBER}.zip" >> $GITHUB_OUTPUT
+          echo "PDF_NAME=Dynatrace-Snowflake-Observability-Agent-${{ steps.get_version.outputs.VERSION }}.pdf" >> $GITHUB_OUTPUT
+
+      - name: Upload build artifacts
+        if: github.ref_type != 'tag' && inputs.create_release == false
+        uses: actions/upload-artifact@v4
+        with:
+          name: dsoa-package-${{ steps.get_version.outputs.VERSION }}
+          path: |
+            ${{ steps.package.outputs.ZIP_NAME }}
+            ${{ steps.package.outputs.PDF_NAME }}
+
+      - name: Extract Release Notes from CHANGELOG
+        id: extract_release_notes
+        if: github.ref_type == 'tag' || inputs.create_release == true
+        run: |
+          VERSION_HEADER="## Dynatrace Snowflake Observability Agent ${{ steps.get_version.outputs.VERSION }}"
+          # Use awk to find the section for the current version and print until the next ## heading
+          awk -v header="$VERSION_HEADER" '
+            $0 == header { in_section=1; next }
+            /^## / { if (in_section) exit }
+            in_section { print }
+          ' CHANGELOG.md > release_notes.md
+
+      - name: Create GitHub Release
+        if: github.ref_type == 'tag' || inputs.create_release == true
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: ${{ github.ref_name }}
+          name: Dynatrace Snowflake Observability Agent ${{ steps.get_version.outputs.VERSION }}
+          body_path: release_notes.md
+          draft: ${{ inputs.create_release == true }}
+          files: |
+            ${{ steps.package.outputs.ZIP_NAME }}
+            ${{ steps.package.outputs.PDF_NAME }}
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.vscode/settings.json

@@ -97,6 +97,7 @@
         "snowflakedb",
         "snowpark",
         "SNOWPIPE",
+        "softprops",
         "specversion",
         "spendings",
         "SQLERRM",

CONTRIBUTING.md

@@ -140,11 +140,65 @@ The build process for the Dynatrace Snowflake Observability Agent package involv
 
 ## Setting up development environment
 
-You will need [Python 3.9](https://www.python.org/) or newer and [git](https://git-scm.com/).
-If you are on Windows, you will need to install WSL2. Please refer to the `Prerequisites` in [the installation documentation](INSTALL.md) for more details.
+This guide was created for developers who want to contribute to the Dynatrace Snowflake Observability Agent. If you only want to install and use the agent, please refer to the [INSTALL.md](INSTALL.md) guide.
 
-The recommended setup is to use [VS Code](https://code.visualstudio.com/) with [Snowflake plugin](https://marketplace.visualstudio.com/items?itemName=snowflake.snowflake-vsc). Though neither of those are necessary, and any IDE that supports python and SQL should do.
-On Windows, after installing WSL, it is necessary to open your IDE of choice using WSL (see [Setting up WSL on VS Code guide)](https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-vscode).
+### Prerequisites
+
+You will need the following software installed:
+
+* [Python](https://www.python.org/) (3.9 or newer)
+* [Git](https://git-scm.com/)
+* On Windows, [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) is required.
+
+The recommended setup is to use [VS Code](https://code.visualstudio.com/) with the [Snowflake plugin](https://marketplace.visualstudio.com/items?itemName=snowflake.snowflake-vsc).
+
+### Environment Setup
+
+1. **Clone the repository:**
+
+    ```bash
+    git clone https://github.com/dynatrace-oss/dynatrace-snowflake-observability-agent.git
+    cd dynatrace-snowflake-observability-agent
+    ```
+
+1. **Create and activate a virtual environment:**
+
+    ```bash
+    python -m venv .venv
+    source .venv/bin/activate
+    ```
+
+1. **Install dependencies:**
+
+    The `setup.sh` script can help install most of the required tools.
+
+    ```bash
+    ./setup.sh
+    ```
+
+    Alternatively, you can install them manually. You will need the dependencies for running the agent (see `INSTALL.md`) plus the development dependencies.
+
+### System Dependencies
+
+For **Ubuntu/Debian**:
+
+  ```bash
+  sudo apt-get update
+  sudo apt-get install -y pango cairo gdk-pixbuf libffi pandoc
+  ```
+
+For **macOS** (using [Homebrew](https://brew.sh/)):
+
+  ```bash
+  brew install pango cairo gdk-pixbuf libffi pandoc
+  ```
+
+Additional **Python packages** for all platforms are listed in `requirements.txt`.
+Install them using pip:
+
+  ```bash
+  pip install -r requirements.txt
+  ```
 
 ## Building Dynatrace Snowflake Observability Agent
 

INSTALL.md

@@ -2,10 +2,65 @@
 
 Dynatrace Snowflake Observability Agent comes in the form of a series of SQL scripts (accompanied with a few configuration files), which need to be deployed at Snowflake by executing them in the correct order.
 
+This document assumes you are installing from the distribution package (`dynatrace_snowflake_observability_agent-*.zip`). If you are a developer and want to build from source, please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) guide.
+
+## Prerequisites
+
+Before you can deploy the agent, you need to ensure the following tools are installed on your system.
+
+### Windows Users
+On Windows, it is necessary to install Windows Subsystem for Linux (WSL) version 2.0 or higher. The deployment scripts must be run through WSL. See [Install WSL guide](https://learn.microsoft.com/en-us/windows/wsl/install) for more details.
+
+### All Users
+You will need the following command-line tools:
+
+* **bash**: The deployment scripts are written in bash.
+* **Snowflake CLI**: For connecting to and deploying objects in Snowflake.
+* **jq**: For processing JSON files.
+* **gawk**: For text processing.
+
+You can run the included `./setup.sh` script, which will attempt to install these dependencies for you.
+
+Alternatively, you can install them manually:
+
+#### Snowflake CLI
+Install using `pipx` (recommended):
+
+```bash
+# If you do not have pipx installed, run:
+# on Ubuntu/Debian
+sudo apt install pipx
+# on macOS
+brew install pipx
+
+# With pipx installed, run:
+pipx install snowflake-cli-labs
+```
+
+Or on macOS with Homebrew:
+
+```bash
+brew tap snowflakedb/snowflake-cli
+brew install snowflake-cli
+```
+
+#### jq and gawk
+
+On **Ubuntu/Debian**:
+
+```bash
+sudo apt install jq gawk
+```
+
+On **macOS** (with Homebrew):
+
+```bash
+brew install jq gawk
+```
+
 ## Deploying Dynatrace Snowflake Observability Agent
 
 The default option to install Dynatrace Snowflake Observability Agent is from the distribution package.
-Should you require to install Dynatrace Snowflake Observability Agent from the sources, please check the [Installing Dynatrace Snowflake Observability Agent from sources](#installing-dynatrace-snowflake-observability-agent-from-sources) section before continuing here.
 To deploy Dynatrace Snowflake Observability Agent, run the `./deploy.sh` command:
 
 ```bash
@@ -148,7 +203,8 @@ To list your currently defined connections run:
 snow connection list
 ```
 
-Here is an example of how to fill in the form to configure connection based on external browser authentication, which is a recommended way for users authenticating with external SSO:
+Here is an example of how to fill in the form to configure connection based on external browser authentication,
+which is a recommended way for users authenticating with external SSO:
 
 ```bash
 Snowflake account name: ${YOUR_SNOWFLAKE_ACCOUNT_NAME.REGION_NAME}
@@ -168,111 +224,10 @@ Path to private key file [optional]:
 You can also run this command to fill in the required and recommended parts:
 
 ```bash
-snow connection add --connection-name snow_agent_$config_name --account ${YOUR_SNOWFLAKE_ACCOUNT_NAME.REGION_NAME} --user ${YOUR_USERNAME} --authenticator externalbrowser
+snow connection add --connection-name snow_agent_$config_name \
+                    --account ${YOUR_SNOWFLAKE_ACCOUNT_NAME.REGION_NAME} \
+                    --user ${YOUR_USERNAME} \
+                    --authenticator externalbrowser
 ```
 
 If you have any issues setting up the connection check [the SnowCli documentation](https://docs.snowflake.com/en/user-guide/snowsql).
-
-## Prerequisites
-
-### Windows Subsystem for Linux
-
-On Windows it is necessary to install Windows Subsystem for Linux before proceeding with the following steps. Dynatrace Snowflake Observability Agent must be executed through WSL to guarantee proper functioning: WSL version 2.0 or higher is recommended (see [Install WSL guide)](https://learn.microsoft.com/en-us/windows/wsl/install):
-
-### MacOS
-
-You will need `brew` installed and up to date (XCode) Command Line Tools.
-
-To install `brew` go to [Brew installation](https://docs.brew.sh/Installation)
-To update Command Line Tools you need to run
-
-```bash
-xcode-select --install
-```
-
-or go to Software Update in System Settings
-
-## Installing Dynatrace Snowflake Observability Agent from sources
-
-For the development purposes you can install Dynatrace Snowflake Observability Agent from sources.
-
-### Getting local copy of Dynatrace Snowflake Observability Agent
-
-To get the source code of Dynatrace Snowflake Observability Agent you will need `git` installed.
-Run the following command:
-
-```bash
-git clone https://github.com/dynatrace-oss/dynatrace-snowflake-observability-agent.git
-```
-
-Otherwise you should receive a package with Dynatrace Snowflake Observability Agent named `dynatrace_snowflake_observability_agent-$version.zip` which you should unpack.
-
-### Installing necessary software
-
-You can simply run `./setup.sh` to install all necessary software on your MacOS or Ubuntu.
-
-#### Creating virtual environment
-
-Once you have Python (only versions 3.9 through 3.11 are supported) installed, it is a good practice to operate with Python code within a virtual environment (see [Virtual Environment Guide](https://docs.python.org/3/library/venv.html)). To create and active one run:
-
-```bash
-python -m venv .venv
-source .venv/bin/activate
-```
-
-#### Installing required packages
-
-Packages necessary to run the program should be installed using following commands.
-
-We need [Snowflake CLI](https://github.com/snowflakedb/snowflake-cli) installed either via pipx
-
-```bash
-# if You do not have pipx installed, run
-sudo apt install pipx
-# with pipx installed, run
-pipx install snowflake-cli-labs
-```
-
-or Homebrew (macOS only):
-
-```bash
-brew tap snowflakedb/snowflake-cli
-brew install snowflake-cli
-snow --help
-```
-
-Once this is done install the remaining Python packages:
-
-```bash
-pip install -r requirements.txt
-```
-
-Finally, you need to install few more development tools:
-
-* [jq](https://github.com/jqlang/jq)
-
-In Windows WSL / Ubuntu linux run:
-
-```bash
-sudo apt install jq
-```
-
-On macOS run:
-
-```bash
-brew install jq
-```
-
-* gawk
-
-In Windows WSL / Ubuntu linux run:
-
-```bash
-sudo apt install gawk
-```
-
-On macOS run:
-
-```bash
-brew install gawk
-```

package.sh

@@ -30,6 +30,20 @@
 #              =               - which will remove :DEV tags from deploy.sh
 #
 
+# Function to check if a command exists
+command_exists() {
+  command -v "$1" >/dev/null 2>&1
+}
+
+# Check for required commands
+required_commands=("pylint" "python" "pandoc" "zip")
+for cmd in "${required_commands[@]}"; do
+  if ! command_exists "$cmd"; then
+    echo "Error: Required command '$cmd' is not installed. Please install it before running this script."
+    exit 1
+  fi
+done
+
 PARAM=$1
 
 # Resetting the package directory