Feature/jira issue status category flow analysis

CHANGELOG.md

@@ -24,7 +24,8 @@
 - Added team-related test data to `issue.csv`, `issue_field_history.csv`, and `field.csv` seed files.
 - Created `get_team_columns` macro for consistent team column definitions.
 - Updated package variables to include team source reference and team identifier configuration.
-- Created consistency tests for new end models. 
+- Created consistency tests for new end models.
+- Created new analysis folder with `jira__issue_cumulative_flow_analysis` model. See analysis [README.md](https://github.com/fivetran/dbt_jira/blob/main/analysis/README.md) for more details on how to use this model for your Jira reporting.
 
 # dbt_jira v1.0.0
 

analysis/README.md

@@ -0,0 +1,38 @@
+# Jira Analysis
+> Note: The compiled sql within the analysis folder references the final models [jira__issue_status_transitions](https://github.com/fivetran/dbt_jira/blob/master/models/jira__issue_status_transitions.sql) and [jira__timestamp_issue_field_history](https://github.com/fivetran/dbt_jira/blob/master/models/jira__timestamp_issue_field_history). As such, prior to
+compiling the provided sql to analyze issue status category metrics, you must first execute `dbt run`.
+
+
+## Analysis SQL
+| **sql**                | **description**                                                                                                                                |
+| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| [jira__issue_cumulative_flow_analysis](https://github.com/fivetran/dbt_jira/blob/master/analysis/jira__issue_cumulative_flow_analysis.sql) | The output of the compiled sql will generate daily metrics for issue status categories by joining status transitions with field history data. The analysis aggregates data by date, project, team, and status category to provide: count of distinct issues transitioning into a new status category, average days spent in status category, and counts of issues that started work, completed work, or reopened work. The SQL references the `jira__issue_status_transitions` and `jira__timestamp_issue_field_history` models. Aggregation granularity can be adjusted by adding/removing field names in the `issue_field_history_columns` var in your dbt project.yml. `status` can be used in place of `status_category` if desired by modifying the model accordingly. |
+
+
+
+
+## SQL Compile Instructions
+Leveraging the above sql is made possible by the [analysis functionality of dbt](https://docs.getdbt.com/docs/building-a-dbt-project/analyses/). In order to
+compile the sql, you will perform the following steps:
+- Execute `dbt run` to create the package models.
+- Execute `dbt compile` to generate the target specific sql.
+- Navigate to your project's `/target/compiled/jira/analysis` directory.
+- Copy the `jira__issue_cumulative_flow_analysis` code and run in your data warehouse.
+- Confirm the issue status category metrics match your expected workflow patterns.
+- Analyze the daily counts, cumulative flow, and completion metrics to identify trends and bottlenecks in your development process.
+
+
+## Contributions
+Don't see a compiled sql statement you would have liked to be included? Notice any bugs when compiling
+and running the analysis sql? If so, we highly encourage and welcome contributions to this package!
+Please create issues or open PRs against `master`. Check out [this post](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657) on the best workflow for contributing to a package.
+
+
+## Database Support
+This package has been tested on BigQuery, Snowflake, Redshift, Postgres, and Databricks.
+
+
+## Are there any resources available?
+- If you have questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_jira/issues/new/choose) section to find the right avenue of support for you.
+- If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).
+

analysis/jira__issue_cumulative_flow_analysis.sql

@@ -0,0 +1,62 @@
+with field_history as (
+ select *
+ from {{ ref('jira__timestamp_issue_field_history') }}
+),
+
+status_transitions as (
+ select *
+ from {{ ref('jira__issue_status_transitions') }}
+),
+
+---if you prefer to keep analytics at the status level, substitute 'status' for 'status_category' throughout this model
+joined as (
+    select 
+    transitions.transition_at,
+    transitions.issue_id,
+    transitions.status_category_name,
+    transitions.minutes_in_status / 1440.0 as days_in_status,
+    transitions.started_work,
+    transitions.completed_work,
+    transitions.reopened_work,
+     -- edit field_history columns as desired based on the issue_field_history_columns var for your dbt project
+     -- project and team have been chosen for this example
+    field_history.team,
+    field_history.project
+  from status_transitions as transitions
+    join field_history 
+        on transitions.issue_id = field_history.issue_id
+        and transitions.transition_at = field_history.valid_from
+),
+
+---roll up to daily for reporting
+daily_rollup as (
+ select
+    cast(transition_at as date) as status_category_date,
+    project,
+    team,
+    status_category_name as status_category,
+    count(distinct issue_id) as issues_in_new_status_category,
+    round(avg(days_in_status),2) as avg_days_in_status_category,
+    sum(started_work) as issues_started_work,
+    sum(completed_work) as issues_completed_work,
+    sum(reopened_work) as issues_reopened_work   
+ from joined
+  group by 1,2,3,4
+)
+--add cumulative flow calculation
+select
+  status_category_date,
+  project,
+  team,
+  status_category,
+  issues_in_new_status_category,
+  sum(issues_in_status_category) over (
+    partition by project, team, status_category
+    order by status_category_date
+    rows unbounded preceding
+  ) as cumulative_issues_in_status_category,
+  avg_days_in_status_category,
+  issues_started_work,
+  issues_completed_work,
+  issues_reopened_work
+from daily_rollup

dbt_project.yml

@@ -25,6 +25,12 @@ vars:
     version: "{{ source('jira', 'version') }}"
     jira_issue_history_buffer: 1
 
+analysis-paths: ["analysis"]
+clean-targets:
+  - target
+  - dbt_modules
+  - dbt_packages
+
 
 models:
   jira:
@@ -36,3 +42,5 @@ models:
     staging:
       +materialized: view
       +schema: jira_source
+
+

integration_tests/dbt_project.yml

@@ -3,9 +3,11 @@ version: '1.1.0'
 config-version: 2
 profile: 'integration_tests'
 
-vars: 
-  # Comment out the below when generating docs
-  issue_field_history_columns: ['summary', 'components', 'team'] # @docs-ignore
+
+vars:
+
+ # Comment out the below when generating docs
+  issue_field_history_columns: ['summary', 'components', 'team' ,'project']  # @docs-ignore
 
   jira:
     jira_schema: jira_integrations_tests_42
@@ -32,6 +34,7 @@ vars:
     jira_version_identifier: "version"
     jira_team_identifier: "team"
 
+
 models:
   jira:
     +schema: "{{ 'jira_integrations_tests_sqlw' if target.name == 'databricks-sql' else 'jira' }}"