Feature/jira issue status category flow analysis

CHANGELOG.md

@@ -19,6 +19,11 @@
   - When teams are enabled, team data will populate in the `jira__daily_issue_field_history` model to track team assignments over time and the `jira__issue_enhanced` model to provide current team assignment for each issue.
 
 ## Under the Hood
+- Added new seed file `team.csv` for integration testing.
+- Added team-related test data to `issue.csv`, `issue_field_history.csv`, and `field.csv` seed files.
+- Created `get_team_columns` macro for consistent team column definitions.
+- Updated package variables to include team source reference and team identifier configuration.
+- Created new analysis folder with `jira__issue_transition_cumulative_flow_analysis` and `jira_daily_issue_status_category_analysis` models. See analysis [README.md](https://github.com/fivetran/dbt_jira/blob/main/analysis/README.md) for more details on how to use this model for your Jira reporting.
 - Created new ephemeral intermediate models that perform necessary operations to build the new `jira__timestamp_issue_field_history` models.
   - `int_jira__pivot_timestamp_field_history`: Table pivoting out the fields in field history into columns at the timestamp level to capture all field changes rather than just daily snapshots. 
   - `int_jira__timestamp_field_history_scd`: Slowly-changing-dimension model that backfills field values at the timestamp level.

README.md

@@ -30,7 +30,7 @@ The following table provides a detailed list of all tables materialized within t
 | **Table**                | **Description**                                                                                                                                |
 | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
 | [jira__daily_issue_field_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__daily_issue_field_history)             | History table with one row for each day an issue remained open, with additional details about the issue sprint, status, and story points (if enabled). <br><br>**Example Analytics Questions:**<br>• How many issues, by sprint, were Closed or Blocked each week in Q1, 2025? <br>• How many days, on average, does it take an issue to go from 'Accepted' to either 'Closed' or 'Blocked'?|
-| [jira__timestamp_issue_field_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__timestamp_issue_field_history)             | Table tracking field changes at timestamp level with validity periods. Each record shows complete field state during a time period with valid_from/valid_until timestamps. <br><br>**Example Analytics Questions:**<br>• What was the exact sequence of field changes for a specific issue? <br>• How long did an issue spend in each status with precise timing?|
+| [jira__timestamp_issue_field_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__timestamp_issue_field_history)             | Table tracking field changes at timestamp level with validity periods. Each record shows complete field state during a time period with `valid_from`/`valid_until` timestamps. <br><br>**Example Analytics Questions:**<br>• What was the exact sequence of field changes for a specific issue? <br>• How long did an issue spend in each status with precise timing?|
 | [jira__issue_status_transitions](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__issue_status_transitions)             | Issue status transition tracking with workflow analysis. Provides chronological view of status changes with timing metrics, transition direction analysis, and lifecycle indicators. <br><br>**Example Analytics Questions:**<br>• What is the average time spent in each status across all issues? <br>• Which is the lead time and cycle time for issues based on when work is added/started/completed? <br>• What are the most common workflow transition paths?|
 | [jira__issue_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__issue_enhanced)            | One row per Jira issue with enriched details about assignee, reporter, sprint, project, and current status, plus metrics on assignments and re-openings. <br><br>**Example Analytics Questions:** <br>• How many issues are currently blocked and who owns them? <br>• What's the average time to resolution for high-priority bugs by assignee? |
 | [jira__project_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__project_enhanced)            | One row per project with team member details, issue counts, work velocity metrics, and project scope information. <br><br>**Example Analytics Questions:**<br>• Which projects have the highest velocity in terms of issues closed per sprint? <br>• What is the ratio of unassigned open tickets to assigned open tickets by project? |

analysis/README.md

@@ -0,0 +1,39 @@
+# Jira Analysis
+> Note: The compiled sql within the analysis folder references the final models [jira__issue_status_transitions](https://github.com/fivetran/dbt_jira/blob/master/models/jira__issue_status_transitions.sql), [jira__timestamp_issue_field_history](https://github.com/fivetran/dbt_jira/blob/master/models/jira__timestamp_issue_field_history), and [jira__daily_issue_field_history](https://github.com/fivetran/dbt_jira/blob/master/models/jira__daily_issue_field_history). As such, prior to
+compiling the provided sql to analyze issue status category metrics, you must first execute `dbt run`.
+
+
+## Analysis SQL
+| **sql**                | **description**                                                                                                                                |
+| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| [jira__daily_issue_status_category_analysis](https://github.com/fivetran/dbt_jira/blob/master/analysis/jira__daily_issue_status_category_analysis.sql) | The output of the compiled sql will generate daily counts of issues in each status category by date, project, and team. The analysis provides a snapshot view of how many issues are in each status category (To Do, In Progress, Done) on any given day. The SQL references the `jira__daily_issue_field_history` model and can be customized by adjusting the `issue_field_history_columns` var in your dbt_project.yml. `status` can be used in place of `status_category` if desired by modifying the model accordingly. |
+| [jira__issue_transition_cumulative_flow_analysis](https://github.com/fivetran/dbt_jira/blob/master/analysis/jira__issue_transition_cumulative_flow_analysis.sql) | The output of the compiled sql will generate daily transition metrics for issue status categories with cumulative flow calculations. The analysis joins status transitions with field history data to provide: count of distinct issues transitioning into a new status category, cumulative count of issues in each status category over time, and counts of issues that started work, completed work, or reopened work. The SQL references the `jira__issue_status_transitions` and `jira__timestamp_issue_field_history` models. Aggregation granularity can be adjusted by adding/removing field names in the `issue_field_history_columns` var in your dbt_project.yml. `status` can be used in place of `status_category` if desired by modifying the model accordingly. |
+
+
+
+
+## SQL Compile Instructions
+Leveraging the above sql is made possible by the [analysis functionality of dbt](https://docs.getdbt.com/docs/building-a-dbt-project/analyses/). In order to
+compile the sql, you will perform the following steps:
+- Execute `dbt run` to create the package models.
+- Execute `dbt compile` to generate the target specific sql.
+- Navigate to your project's `/target/compiled/jira/analysis` directory.
+- Copy the desired analysis code (`jira__daily_issue_status_category_analysis` or `jira__issue_transition_cumulative_flow_analysis`) and run in your data warehouse.
+- Confirm the issue status category metrics match your expected workflow patterns.
+- Analyze the daily counts, transition metrics, and cumulative flow data to identify trends and bottlenecks in your development process.
+
+
+## Contributions
+Don't see a compiled sql statement you would have liked to be included? Notice any bugs when compiling
+and running the analysis sql? If so, we highly encourage and welcome contributions to this package!
+Please create issues or open PRs against `master`. Check out [this post](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657) on the best workflow for contributing to a package.
+
+
+## Database Support
+This package has been tested on BigQuery, Snowflake, Redshift, Postgres, and Databricks.
+
+
+## Are there any resources available?
+- If you have questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_jira/issues/new/choose) section to find the right avenue of support for you.
+- If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).
+

analysis/jira__daily_issue_status_category_analysis.sql

@@ -0,0 +1,63 @@
+with field_history as ( 
+
+  select * 
+  from {{ ref('jira__daily_issue_field_history') }} 
+), 
+
+statuses as ( 
+
+  select * 
+  from {{ ref('stg_jira__status') }} 
+), 
+
+status_category as ( 
+
+  select * 
+  from {{ ref('stg_jira__status_category') }} 
+), 
+
+status_mapping as ( 
+
+  select 
+    statuses.status_id, 
+    statuses.status_name, 
+    status_category.status_category_id, 
+    case when lower(status_category.status_category_name) = 'to do' then 'To Do' 
+      when lower(status_category.status_category_name) = 'in progress' then 'In Progress' 
+      when lower(status_category.status_category_name) = 'done' then 'Done' 
+      else 'Other' 
+    end as status_category_name 
+  from statuses left join status_category 
+    on statuses.status_category_id = status_category.status_category_id
+), 
+
+--if you prefer to keep analytics at the status level, substitute 'status' for 'status_category' throughout this model and skip the status_mapping step
+daily_issue_status_category as ( 
+  select 
+    field_history.date_day as report_date, 
+    -- edit field_history columns as desired based on the issue_field_history_columns var for your dbt project
+    -- project and team have been chosen for this example
+    field_history.project,
+    field_history.team,
+    field_history.issue_id, 
+    field_history.status,
+    status_mapping.status_category_name as status_category, 
+  from field_history 
+    left join status_mapping 
+    on lower(field_history.status) = lower(status_mapping.status_name) 
+), 
+
+--count of issues in each status on report date 
+daily_counts as ( 
+  select 
+    report_date, 
+    project, 
+    team, 
+    status_category, 
+    count(distinct issue_id) as issues_in_status 
+  from daily_issue_status_category 
+  group by 1,2,3,4 
+)
+
+select *
+from daily_counts

analysis/jira__issue_transition_cumulative_flow_analysis.sql

@@ -0,0 +1,61 @@
+with field_history as ( 
+
+  select * 
+  from {{ ref('jira__timestamp_issue_field_history') }}
+
+),
+
+status_transitions as (
+
+  select *
+  from {{ ref('jira__issue_status_transitions') }}
+),
+
+---if you prefer to keep analytics at the status level, substitute 'status' for 'status_category' throughout this model
+joined as (
+
+  select 
+    transitions.transition_at,
+    transitions.issue_id,
+    transitions.status_category_name,
+    -- edit field_history columns as desired based on the issue_field_history_columns var for your dbt project
+    -- project and team have been chosen for this example
+    field_history.team,
+    field_history.project
+  from status_transitions as transitions
+    join field_history 
+      on transitions.issue_id = field_history.issue_id
+      and transitions.transition_at = field_history.valid_from
+),
+
+---roll up to daily for reporting
+daily_rollup as (
+
+  select
+    cast(transition_at as date) as status_category_transition_date,
+    project,
+    team,
+    status_category_name as status_category,
+    count(distinct issue_id) as issues_in_new_status_category,
+  from joined
+  group by 1,2,3,4
+),
+
+--add cumulative flow calculation
+cumulative_flow as (
+  select
+    status_category_transition_date,
+    project,
+    team,
+    status_category,
+    issues_in_new_status_category,
+    sum(issues_in_new_status_category) over (
+      partition by project, team, status_category
+      order by status_category_transition_date
+      rows unbounded preceding
+    ) as cumulative_issues_in_status_category
+  from daily_rollup
+)
+
+select *
+from cumulative_flow

dbt_project.yml

@@ -25,6 +25,12 @@ vars:
     version: "{{ source('jira', 'version') }}"
     jira_issue_history_buffer: 1
 
+analysis-paths: ["analysis"]
+clean-targets:
+  - target
+  - dbt_modules
+  - dbt_packages
+
 
 models:
   jira:
@@ -36,3 +42,5 @@ models:
     staging:
       +materialized: view
       +schema: jira_source
+
+

integration_tests/dbt_project.yml

@@ -3,9 +3,11 @@ version: '1.1.0'
 config-version: 2
 profile: 'integration_tests'
 
-vars: 
-  # Comment out the below when generating docs
-  issue_field_history_columns: ['summary', 'components', 'team'] # @docs-ignore
+
+vars:
+
+ # Comment out the below when generating docs
+  issue_field_history_columns: ['summary', 'components', 'team' ,'project']  # @docs-ignore
 
   jira:
     jira_schema: jira_integrations_tests_42
@@ -32,6 +34,7 @@ vars:
     jira_version_identifier: "version"
     jira_team_identifier: "team"
 
+
 models:
   jira:
     +schema: "{{ 'jira_integrations_tests_sqlw' if target.name == 'databricks-sql' else 'jira' }}"