Version new

[gmartinez-dbai]

This pull request introduces several improvements to the shell script automation and CI workflow, focusing on more robust environment detection, improved test reliability, and enhanced code quality. The primary changes are grouped into CI/test automation enhancements and shell scripting style improvements.

CI and Test Automation Enhancements:

• Added environment capability detection to automation/test_pgtools.sh to skip SQL syntax tests that require unavailable superuser privileges or missing extensions, avoiding false failures in restricted CI environments. (automation/test_pgtools.sh) [1] [2] [3]
• Expanded SQL syntax validation to cover all relevant directories, with allowlists for scripts that require superuser, extensions, or TimescaleDB, and improved logging for skipped scripts. (automation/test_pgtools.sh)
• Improved the CI workflow to enable the pg_stat_statements extension and run a broader set of ShellCheck validations and test suites, including HOT checklist validations. (.github/workflows/ci.yml)
• Enhanced test reporting to handle cases where no tests are run, displaying "n/a" for success rate instead of a division by zero. (automation/test_pgtools.sh)

Shell Scripting Style and Safety Improvements:

• Refactored variable assignments to use separate declaration and assignment for better POSIX compliance and readability across multiple scripts. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
• Added targeted shellcheck disables and improved quoting/word-splitting to prevent false positives and ensure robust handling of user input in maintenance scripts. (maintenance/auto_maintenance.sh) [1] [2]
• Improved user prompts and command safety (e.g., using read -rp for confirmation prompts). (maintenance/auto_maintenance.sh)
• Refined duration and size calculations in maintenance scripts for clarity and correctness, including improved use of pg_size_pretty. (maintenance/auto_maintenance.sh) [1] [2] [3] [4] [5]

These updates collectively make the automation and maintenance scripts more reliable, maintainable, and CI-friendly.

[Copilot]

Pull request overview

This PR expands CI/test automation for the repo’s shell/SQL toolchain (broader ShellCheck + more reliable SQL validation in restricted environments) and adds a new set of TimescaleDB-focused diagnostic SQL scripts.

Changes:

• Add TimescaleDB diagnostics scripts + timescaledb/README.md quick reference.
• Refactor automation/test_pgtools.sh SQL syntax sweep to scan multiple directories and skip unsupported scripts based on detected DB capabilities.
• Expand CI + local precommit checks to ShellCheck more directories and run HOT checklist validations.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
.github/workflows/ci.yml	Runs broader ShellCheck, enables pg_stat_statements, and adds HOT checklist validations in CI.
automation/test_pgtools.sh	Adds environment capability detection and expands SQL syntax validation scope/skip logic.
scripts/precommit_checks.sh	Aligns local ShellCheck coverage with CI (more directories).
maintenance/auto_maintenance.sh	Shell style tweaks + targeted ShellCheck disables + safer prompts/quoting changes.
integration/prometheus_exporter.sh	Shell style refactor (separate declaration/assignment).
integration/grafana_dashboard_generator.sh	Shell style refactor (separate declaration/assignment).
configuration/parameter_tuner.sh	Shell style refactor (separate declaration/assignment).
timescaledb/chunk_health.sql	New TimescaleDB chunk sizing/count diagnostics.
timescaledb/compression_diagnostics.sql	New TimescaleDB compression configuration/effectiveness/backlog diagnostics.
timescaledb/continuous_aggregates.sql	New TimescaleDB CAGG health + refresh policy diagnostics.
timescaledb/background_jobs.sql	New TimescaleDB background job failure/saturation diagnostics.
timescaledb/replication_tiering.sql	New Tiger Lake / tiered storage replication slot lag diagnostics.
timescaledb/README.md	Documents TimescaleDB scripts and a triage flow.
timescaledb.txt	Adds a TimescaleDB “cheatsheet” document (currently written as conversational transcript).

Comments suppressed due to low confidence (1)

maintenance/auto_maintenance.sh:264

• SCHEMA_PATTERN and TARGET_TABLES patterns are interpolated directly into the SQL string (schemaname = '$SCHEMA_PATTERN' and tablename LIKE '${pattern}'). If either value contains a single quote (or other special characters), the query can break and also enables SQL injection against the connected database. Prefer passing values via psql -v and using psql’s SQL-literal substitution (:'var'), or at minimum escape single quotes in user-supplied patterns before embedding them.

    else
        schema_filter="AND schemaname NOT IN ('information_schema', 'pg_catalog')"
    fi
    
    local table_filter=""
    if [[ -n "$TARGET_TABLES" ]]; then
        # Convert comma-separated patterns to SQL LIKE conditions
        # shellcheck disable=SC2206  # intentional word-split of comma-separated list
        local patterns=(${TARGET_TABLES//,/ })
        local conditions=()
        for pattern in "${patterns[@]}"; do
            conditions+=("tablename LIKE '${pattern}'")
        done
        # shellcheck disable=SC2124  # join-with-IFS idiom uses [*] by design
        table_filter="AND ($(IFS=' OR '; echo "${conditions[*]}"))"
    fi
    
    psql -t -c "

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

CI runs run_hot_update_report.sh --stdout without --quiet, which causes the script to print local “Next steps” instructions (paths/tooling) into CI logs. Consider adding --quiet in CI to keep logs focused on validation output.

[Copilot]

SQL_REQUIRES_EXTENSIONS is not comprehensive for pg_stat_statements-dependent scripts. For example, optimization/missing_indexes.sql and multiple troubleshooting/* packs query pg_stat_statements unconditionally, so when pg_stat_statements isn’t installed the new syntax sweep will fail even though SKIP_EXTENSION_SCRIPTS is true. Either add all extension-dependent scripts to the allowlist, or make the skip logic data-driven (e.g., skip files containing pg_stat_statements/pg_buffercache references when the corresponding extension isn’t installed, unless the script self-guards).