Contributing

Welcome! This document is intended for developers wishing to contribute to the Dynatrace Snowflake Observability Agent (DSOA).

Table of Contents:

Setting up Development Environment
Development Workflow
Testing
Writing Plugins
Semantic Conventions
Source Code Overview
Pull Request Checklist

Setting up Development Environment

This guide helps developers who want to contribute to DSOA. If you only want to install and use it, see the installation guide.

Prerequisites

Required tools:

Python (3.9 or newer)
Git
Windows Users: WSL2 is required.
Recommended IDE: VS Code with the Snowflake extension.

Environment Setup

Clone the repository:

git clone https://github.com/dynatrace-oss/dynatrace-snowflake-observability-agent.git
cd dynatrace-snowflake-observability-agent

Run the setup script: This helper script installs system dependencies and sets up the Python virtual environment.
```
./scripts/deploy/setup.sh
source .venv/bin/activate
```
Install dependencies manually (if needed):
```
pip install -r requirements.txt
```

System Dependencies

For Ubuntu/Debian:

sudo apt-get update
sudo apt-get install -y pango cairo gdk-pixbuf libffi pandoc
npm install -g prettier

For macOS (using Homebrew):

brew install pango cairo gdk-pixbuf libffi pandoc prettier

Development Workflow

The source code is split into Python, SQL templates, and configuration files. You must build the agent to combine these into deployable artifacts.

Building the Agent

Before deploying changes or running integration tests, run the build script. This compiles the Python code into Snowpark-compatible formats and assembles the SQL templates.

./scripts/dev/build.sh

What this does:

Compilation (compile.sh):
- Creates _version.py
- Pre-compiles dtagent.otel.semantics.Semantics to include metric semantics dictionary
- Creates single files for both main stored procedures (_dtagent.py and _send_telemetry.py)
- The ##INSERT directive controls assembly order
Building and embedding (build.sh):
- Creates default configuration file (build/config-default.yml)
- Copies SQL files from all *.sql folders
- Embeds compiled Python files into procedure templates
Prepares all files for deployment in the build/ directory

Note: When Snowflake reports issues in stored procedures, line numbers correspond to _dtagent.py and _send_telemetry.py files.

Updating Documentation

If you change info.md, configuration files, instruments-def.yml, or add a new plugin, you must rebuild the documentation (PDFs and READMEs).

./scripts/dev/build_docs.sh

This command will:

Rebuild the agent
Refresh README.md
Generate Dynatrace-Snowflake-Observability-Agent-$VERSION.pdf

Note: This requires pango, cairo, libffi, and prettier installed on your system. On macOS, you may need to set:

export WEASYPRINT_DLL_DIRECTORIES=/opt/homebrew/lib

Packaging for Distribution

To create a distributable zip file (containing SQL scripts and docs) for sharing with other users:

./scripts/dev/package.sh

This creates dynatrace_snowflake_observability_agent-$VERSION.$BUILD.zip containing everything necessary for distribution.

Testing

We use pytest for Python tests and bats for Bash script tests.

Test Suites Overview

Core Tests (test/core/): Configuration, utilities, and view structure
OTel Tests (test/otel/): OpenTelemetry integration
Plugin Tests (test/plugins/): Individual plugin logic
Bash Tests (test/bash/): Deployment and build scripts, including:
- Custom object name replacement (test_custom_object_names.bats)
- Optional object filtering (test_optional_objects.bats)
- Configuration conversion (test_convert_config_to_yaml.bats)
- Deployment script utilities (test_list_options_to_exclude.bats)

For detailed information about each test suite, see:

Test Execution Modes

Local Mode (Mocked):
- Runs without test/credentials.yml
- Does not connect to Snowflake/Dynatrace
- Useful for rapid logic testing
- Uses mocked APIs and example test data from test/test_data/
Live Mode:
- Runs if test/credentials.yml exists
- Connects to real Snowflake/Dynatrace endpoints
- Sends data to actual Dynatrace APIs

Running Tests

Run all Python tests:

pytest

Run all bash tests:

./test/bash/run_tests.sh

Run specific test suites:

# Core tests
pytest test/core/

# OTel tests
pytest test/otel/

# Plugin tests
pytest test/plugins/

# Individual bash test
./test/bash/run_tests.sh test_custom_object_names.bats

Run a specific plugin test: (e.g., for the 'budgets' plugin)

./scripts/dev/test.sh test_budgets

Regenerate test data (Pickles): If you modify a plugin's SQL logic, you may need to update the test data.

./scripts/dev/test.sh test_budgets -p

Test Data

Tests use example test data from the test/test_data folder:

Pickle (*.pkl) files are used for test execution
ndJSON files are provided for reference only
Test results are validated against expected data in test_results

Setting Up Test Environment

To run tests in live mode:

Create a test deployment with configuration in conf/config-test.yml:

core:
  dynatrace_tenant_address: abc12345.live.dynatrace.com
  deployment_environment: TEST
  log_level: DEBUG
  tag: ""
  procedure_timeout: 3600
  snowflake:
    account_name: 'your_snowflake_account.us-east-1'
    host_name: 'your_snowflake_account.us-east-1.snowflakecomputing.com'
    resource_monitor:
      credit_quota: 1
otel: {}
plugins:
  disabled_by_default: true

Create test/credentials.yaml from the test/credentials.template.yml template.

Generate test/conf/config-download.yml by running:

PYTHONPATH="./src" pytest -s -v "test/core/test_config.py::TestConfig::test_init" --pickle_conf y

Running Tests in Local Mode

For local mode testing (mocked APIs), ensure test/conf/config-download.yml does NOT exist. A good practice is to temporarily disable these files by prefixing them with an underscore (e.g., _config-download.yml and _credentials.yml). The gitignore ensures files prefixed with underscore are not tracked.

VS Code Test Explorer

Open Test Explorer view (Ctrl+Shift+P → "Test: Focus on Test Explorer")
Click "Run All Tests" to execute all Python and bash tests
Individual test suites can be run by expanding the test tree

Writing Plugins

For a comprehensive guide on how to write a plugin, including step-by-step instructions, complete examples, configuration details, and debugging tips, please refer to the Plugin Development Guide.

The Plugin Development Guide covers:

Complete plugin structure and file organization
Step-by-step plugin creation with working examples
Python class implementation patterns
SQL views, procedures, and task definitions
Configuration and semantic dictionary setup
Bill of Materials (BOM) documentation
Comprehensive testing strategies
Common patterns and advanced topics
Troubleshooting and debugging tips

Semantic Conventions

IMPORTANT: Before contributing code, ensure you understand these naming rules.

Field and Metric Naming Rules

Case: ALWAYS use snake_case
Prefix: Custom fields SHOULD start with snowflake.
Units: AVOID measurement units in names (e.g., use duration, not duration_ms)
Boolean: Must use is_ or has_ prefix
No Suffix: DO NOT use .count suffix (it is implied for counters)
Structure: Use dots . to denote object hierarchy (e.g., snowflake.table.name)
No Objects: Avoid reporting raw OBJECT fields; expand them into specific metrics/attributes
Existing Semantics: Use existing OpenTelemetry or Dynatrace semantics when they express the same field
Dimensionality: DO NOT provide dimensionality information in field names
Consistency: DO NOT produce different names of the same metric depending on the dimension set
Extension Name: DO NOT add extension name or technology as part of field name
Singular/Plural: Use singular and plural properly to reflect the field content

SQL Object Rules

Names: All Snowflake objects in DTAGENT_DB must be UPPERCASE (e.g., DTAGENT_DB.APP.MY_VIEW). Lowercase names will not be caught during custom tag deployment, causing objects to initialize in the default DTAGENT_DB.
Permissions:
- Tables/Views: Grant SELECT to DTAGENT_VIEWER
- Procedures: Grant USAGE to DTAGENT_VIEWER
- Tasks: Grant OWNERSHIP to DTAGENT_VIEWER
Safety: Procedures should include EXCEPTION handling blocks
Boolean Returns: Avoid returning boolean values from stored procedures. Make them more descriptive (e.g., by specifying affected tables)
Table Initialization: Avoid using create or replace table in stored procedures. Initialize tables before procedures and truncate them inside
Log Messages: For views reported as logs, include _MESSAGE column (automatically mapped to content field)

Metric Types

OpenTelemetry defines 3 metric types:

Counters: Values that accumulate (e.g., total requests)
Gauges: Point-in-time measurements (e.g., current temperature)
Histograms: Distributions for aggregation (e.g., request duration)

Important: Since Dynatrace API only recognizes counters and gauges, and there are no examples of counter metrics in current implementation, all metrics are currently sent as gauge until further improvements.

Source Code Overview

src/dtagent: Python source code
src/dtagent.sql: Core SQL init scripts (Roles, DBs, Warehouses)
src/dtagent.conf: Default configuration and core semantics
src/dtagent/plugins: Source code for all plugins (Python + SQL + Config)
src/dtagent/otel: Telemetry API Python code
scripts/dev: Tools for building, compiling, and testing
scripts/deploy: Tools for deploying the agent to a Snowflake account
scripts/tools: Utility scripts for config and dashboard format conversion

SQL File Prefixes

SQL files use three-digit prefixes to enforce execution order:

0xx - Core initialization + plugin-specific procedures, views
70x - Core procedures
80x - Task definitions
90x - Plugin-specific update procedures

Conditional Code Blocks

SQL scripts support annotation blocks for conditional inclusion:

--%PLUGIN:plugin_name:
-- Code included only when plugin is enabled
--%:PLUGIN:plugin_name

--%OPTION:option_name:
-- Code included only when optional component is enabled
--%:OPTION:option_name

Supported options:

dtagent_admin - Admin role code (excluded when core.snowflake.roles.admin is "-")
resource_monitor - Resource monitor code (excluded when core.snowflake.resource_monitor.name is "-")

Pull Request Checklist

Before submitting a PR, please ensure:

You have run ./scripts/dev/build.sh successfully
You have added tests for any new functionality
All tests pass locally (pytest and ./test/bash/run_tests.sh)
Documentation (README.md, PLUGIN_DEVELOPMENT.md, etc.) is updated if needed
If adding a plugin, instruments-def.yml is defined and valid
Code follows the Semantic Conventions
If changing SQL objects, all names are UPPERCASE
If adding new semantic fields, they follow naming rules
Documentation has been rebuilt (./scripts/dev/build_docs.sh) if needed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Contributing

Setting up Development Environment

Prerequisites

Environment Setup

System Dependencies

Development Workflow

Building the Agent

Updating Documentation

Packaging for Distribution

Testing

Test Suites Overview

Test Execution Modes

Running Tests

Test Data

Setting Up Test Environment

Running Tests in Local Mode

VS Code Test Explorer

Writing Plugins

Semantic Conventions

Field and Metric Naming Rules

SQL Object Rules

Metric Types

Source Code Overview

SQL File Prefixes

Conditional Code Blocks

Pull Request Checklist

FilesExpand file tree

CONTRIBUTING.md

Latest commit

History

CONTRIBUTING.md

File metadata and controls

Contributing

Setting up Development Environment

Prerequisites

Environment Setup

System Dependencies

Development Workflow

Building the Agent

Updating Documentation

Packaging for Distribution

Testing

Test Suites Overview

Test Execution Modes

Running Tests

Test Data

Setting Up Test Environment

Running Tests in Local Mode

VS Code Test Explorer

Writing Plugins

Semantic Conventions

Field and Metric Naming Rules

SQL Object Rules

Metric Types

Source Code Overview

SQL File Prefixes

Conditional Code Blocks

Pull Request Checklist