feat(api-service): use new workflow run count table in usage page#10074
Open
djabarovgeorge wants to merge 4 commits intonextfrom
Open
feat(api-service): use new workflow run count table in usage page#10074djabarovgeorge wants to merge 4 commits intonextfrom
djabarovgeorge wants to merge 4 commits intonextfrom
Conversation
… data instead of traces - Replaced TraceLogRepository with WorkflowRunCountRepository in build-workflow-runs-trend-chart.usecase.ts. - Updated feature flag from IS_WORKFLOW_RUN_TREND_FROM_TRACES_ENABLED to IS_WORKFLOW_RUN_TREND_FROM_ROLLUP_ENABLED. - Modified chart building logic to utilize workflow run counts for trend data. - Added new method getWorkflowRunsTrendData in WorkflowRunCountRepository for fetching trend data from the database.
…port - Introduced new method getWorkflowVolumeData in WorkflowRunCountRepository to fetch workflow volume data. - Updated BuildWorkflowByVolumeChart to utilize workflow run counts based on feature flag for rollup data. - Integrated notification template fetching to associate workflow names with their identifiers in the chart data. - Refactored chart building logic to accommodate both rollup and standard workflow run data.
❌ Deploy Preview for dashboard-v2-novu-staging failed. Why did it fail? →
|
djabarovgeorge
temporarily deployed
to
Linting
GitHub Actions
Inactive
djabarovgeorge
temporarily deployed
to
Linting
GitHub Actions
Inactive
djabarovgeorge
temporarily deployed
to
Linting
GitHub Actions
Inactive
Contributor
|
Hey there and thank you for opening this pull request! 👋 We require pull request titles to follow specific formatting rules and it looks like your proposed title needs to be adjusted. Your PR title is: Requirements:
Expected format: Details: PR title must end with 'fixes TICKET-ID' (e.g., 'fixes NOV-123') or include ticket ID in branch name |
github-actions
bot
added
@novu/dal
@novu/shared
@novu/dashboard
@novu/api-service
labels
Contributor
LaunchDarkly flag references❌ 1 flag removed
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughFeature-flagged switch to use rollup-based workflow run counts: adds WorkflowRunCountRepository query methods, registers the repository, introduces IS_WORKFLOW_RUN_COUNT_ENABLED flag, updates two analytics use cases and the dashboard to prefer count-based data when enabled, and adds overloads to NotificationTemplateRepository.findByTriggerIdentifierBulk. Changes
Sequence Diagram(s)sequenceDiagram
participant Dashboard
participant API as Novu API
participant UseCase as BuildChart UseCase
participant FF as FeatureFlagsService
participant CountRepo as WorkflowRunCountRepository
participant TemplateRepo as NotificationTemplateRepository
participant CH as ClickHouse
Dashboard->>API: Request chart data
API->>UseCase: execute(startDate,endDate,env,org)
UseCase->>FF: isEnabled(IS_WORKFLOW_RUN_COUNT_ENABLED)
alt flag ON
UseCase->>CountRepo: getWorkflowVolumeData(env,org,start,end)
CountRepo->>CH: execute volume query
CH-->>CountRepo: rows [{workflow_run_id,count},...]
CountRepo-->>UseCase: rows
UseCase->>TemplateRepo: findByTriggerIdentifierBulk(env, identifiers, select:['identifier','name'])
TemplateRepo->>DB: query templates
DB-->>TemplateRepo: templates with names
TemplateRepo-->>UseCase: mapped names
UseCase-->>API: chart payload (workflowName,count)
else flag OFF
UseCase->>UseCase: buildChartFromWorkflowRuns(...) (legacy flow)
UseCase-->>API: chart payload (workflowName,count)
end
API-->>Dashboard: return chart payload
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts (1)
44-81: The rollup chart-building logic is well-structured. Good use of selective field projection (['name', 'triggers']) and the fallback toworkflow_run_idwhen no template name is found.One consideration:
workflow_run_idin the rollup table stores a trigger identifier, not an actual workflow run ID. This naming mismatch could confuse future maintainers reading this code. A clarifying inline comment at line 61 would help, e.g., noting thatworkflow_run_idis actually a trigger identifier in this context.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts` around lines 44 - 81, Add an inline clarifying comment in buildChartFromWorkflowRunCount explaining that the rollup column workflow_run_id actually contains a trigger identifier (not a real workflow run ID) to avoid confusion for future maintainers; place the comment near where workflowVolumes is mapped to triggerIdentifiers / when reading row.workflow_run_id (references: buildChartFromWorkflowRunCount, workflowVolumes, triggerIdentifiers, nameByIdentifier) so it's obvious why we use workflow_run_id as the lookup key for template names.libs/dal/src/repositories/notification-template/notification-template.repository.ts (1)
66-98: Clean overload design for field projection support.The conditional
steps.templatepopulation (line 93) is a good optimization. One subtle note: whenselectis provided without_id, MongoDB still returns_idby default, so consumers will get_ideven if not listed inselect. This is likely desirable but worth being aware of — thePick<NotificationTemplateEntity, K>return type won't include_idin its type signature unless explicitly selected, creating a minor type/runtime mismatch.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/dal/src/repositories/notification-template/notification-template.repository.ts` around lines 66 - 98, The runtime projection currently omits explicit _id handling so MongoDB will still include _id by default while the TypeScript return type Pick<NotificationTemplateEntity, K> may not include it; update findByTriggerIdentifierBulk to ensure projection matches the types by adding _id: 1 whenever a select array is provided (i.e., when building projection from select ensure projection._id = 1 if not already present), and keep the rest of the logic (projection variable, baseQuery, query population, and the returned mapEntities) unchanged so runtime and types remain consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`:
- Around line 30-41: The code silently ignores workflowIds when the rollup
feature flag is enabled; update the call-site and implementation so workflowIds
are not dropped: change buildChartFromWorkflowRunCount to accept an optional
workflowIds parameter and apply that filter inside its logic (or if filtering by
workflow is not supported, at minimum add a clear log via processLogger.warn
inside build-workflow-by-volume-chart.usecase.ts before returning to state that
non-empty workflowIds are being ignored when isRollupEnabled is true),
referencing the isRollupEnabled check and the functions
buildChartFromWorkflowRunCount and buildChartFromWorkflowRuns so the behavior is
explicit.
In
`@apps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.ts`:
- Around line 35-39: When isRollupEnabled is true the code calls
buildChartFromWorkflowRunCount and silently drops the workflowIds filter; update
the branch that currently returns buildChartFromWorkflowRunCount(startDate,
endDate, environmentId, organizationId) to check if workflowIds is non-empty and
emit a warning via the use case's logger (e.g., this.logger.warn or existing
logging utility) stating that workflowIds are ignored for rollup mode, or
alternatively make buildChartFromWorkflowRunCount accept and apply workflowIds
if the rollup data supports it; reference the isRollupEnabled flag and the
buildChartFromWorkflowRunCount / buildChartFromWorkflowRuns call sites when
implementing the change.
---
Nitpick comments:
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`:
- Around line 44-81: Add an inline clarifying comment in
buildChartFromWorkflowRunCount explaining that the rollup column workflow_run_id
actually contains a trigger identifier (not a real workflow run ID) to avoid
confusion for future maintainers; place the comment near where workflowVolumes
is mapped to triggerIdentifiers / when reading row.workflow_run_id (references:
buildChartFromWorkflowRunCount, workflowVolumes, triggerIdentifiers,
nameByIdentifier) so it's obvious why we use workflow_run_id as the lookup key
for template names.
In
`@libs/dal/src/repositories/notification-template/notification-template.repository.ts`:
- Around line 66-98: The runtime projection currently omits explicit _id
handling so MongoDB will still include _id by default while the TypeScript
return type Pick<NotificationTemplateEntity, K> may not include it; update
findByTriggerIdentifierBulk to ensure projection matches the types by adding
_id: 1 whenever a select array is provided (i.e., when building projection from
select ensure projection._id = 1 if not already present), and keep the rest of
the logic (projection variable, baseQuery, query population, and the returned
mapEntities) unchanged so runtime and types remain consistent.
ℹ️ Review info
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.tsapps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.tsapps/api/src/app/shared/shared.module.tsapps/dashboard/src/components/analytics/charts/workflow-runs-trend-chart.tsxlibs/application-generic/src/services/analytic-logs/workflow-run-count/workflow-run-count.repository.tslibs/dal/src/repositories/notification-template/notification-template.repository.tspackages/shared/src/types/feature-flags.ts
Comment on lines
+30
to
+41
| const isRollupEnabled = await this.featureFlagsService.getFlag({ | ||
| key: FeatureFlagsKeysEnum.IS_WORKFLOW_RUN_TREND_FROM_ROLLUP_ENABLED, | ||
| defaultValue: false, | ||
| organization: { _id: organizationId }, | ||
| environment: { _id: environmentId }, | ||
| }); | ||
|
|
||
| if (isRollupEnabled) { | ||
| return this.buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId); | ||
| } | ||
|
|
||
| return this.buildChartFromWorkflowRuns(startDate, endDate, environmentId, organizationId, workflowIds); |
Contributor
There was a problem hiding this comment.
Same observation as the trend chart: workflowIds is silently ignored when rollup is enabled.
When the rollup flag is on, buildChartFromWorkflowRunCount ignores the workflowIds parameter. This is the same pattern as in BuildWorkflowRunsTrendChart. If this is expected, consider at minimum adding a log when workflowIds is non-empty but unused.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`
around lines 30 - 41, The code silently ignores workflowIds when the rollup
feature flag is enabled; update the call-site and implementation so workflowIds
are not dropped: change buildChartFromWorkflowRunCount to accept an optional
workflowIds parameter and apply that filter inside its logic (or if filtering by
workflow is not supported, at minimum add a clear log via processLogger.warn
inside build-workflow-by-volume-chart.usecase.ts before returning to state that
non-empty workflowIds are being ignored when isRollupEnabled is true),
referencing the isRollupEnabled check and the functions
buildChartFromWorkflowRunCount and buildChartFromWorkflowRuns so the behavior is
explicit.
Comment on lines
35
to
39
| if (isRollupEnabled) { | ||
| return this.buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId); | ||
| } | ||
|
|
||
| return this.buildChartFromWorkflowRuns(startDate, endDate, environmentId, organizationId, workflowIds); |
Contributor
There was a problem hiding this comment.
Note: workflowIds filter is silently ignored when rollup is enabled.
When isRollupEnabled is true, buildChartFromWorkflowRunCount doesn't accept or use the workflowIds parameter from the command, meaning any workflow-specific filtering the caller requested is silently dropped. If this is intentional (rollup table doesn't support per-workflow filtering), consider logging a warning when workflowIds is provided but ignored, so callers aren't surprised by unfiltered results.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@apps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.ts`
around lines 35 - 39, When isRollupEnabled is true the code calls
buildChartFromWorkflowRunCount and silently drops the workflowIds filter; update
the branch that currently returns buildChartFromWorkflowRunCount(startDate,
endDate, environmentId, organizationId) to check if workflowIds is non-empty and
emit a warning via the use case's logger (e.g., this.logger.warn or existing
logging utility) stating that workflowIds are ignored for rollup mode, or
alternatively make buildChartFromWorkflowRunCount accept and apply workflowIds
if the rollup data supports it; reference the isRollupEnabled flag and the
buildChartFromWorkflowRunCount / buildChartFromWorkflowRuns call sites when
implementing the change.
…charts - Changed feature flag from IS_WORKFLOW_RUN_TREND_FROM_ROLLUP_ENABLED to IS_WORKFLOW_RUN_COUNT_ENABLED in build-workflow-by-volume-chart and build-workflow-runs-trend-chart use cases. - Updated WorkflowRunsTrendChart component to reflect the new feature flag. - Removed deprecated feature flag from the shared types.
commit: |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
apps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.ts (1)
35-37:⚠️ Potential issue | 🟡 Minor
workflowIdsis ignored when count path is enabledLine 36 switches to
buildChartFromWorkflowRunCount(...)without preservingworkflowIds. This silently changes API behavior under the feature flag.Suggested minimal fix
if (isWorkflowRunCountEnabled) { + if (workflowIds?.length) { + this.logger.warn( + { workflowIdsCount: workflowIds.length }, + 'workflowIds filter is ignored when IS_WORKFLOW_RUN_COUNT_ENABLED is true' + ); + } + return this.buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.ts` around lines 35 - 37, The feature-flag branch ignores the workflowIds filter: when isWorkflowRunCountEnabled is true the code returns buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId) and drops workflowIds, changing API behavior; update buildChartFromWorkflowRunCount (and any callers) to accept a workflowIds parameter and pass the existing workflowIds through (e.g., call buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId, workflowIds)), and ensure the new parameter is applied inside buildChartFromWorkflowRunCount to filter counts by the provided workflowIds.apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts (1)
37-39:⚠️ Potential issue | 🟡 Minor
workflowIdsfilter is dropped in count-enabled modeLine 38 returns count-based data without applying or acknowledging
workflowIds, so requested filtering is silently ignored when the flag is on.Suggested minimal fix
if (isWorkflowRunCountEnabled) { + if (workflowIds?.length) { + this.logger.warn( + { workflowIdsCount: workflowIds.length }, + 'workflowIds filter is ignored when IS_WORKFLOW_RUN_COUNT_ENABLED is true' + ); + } + return this.buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts` around lines 37 - 39, The count-path currently ignores the workflowIds filter: when isWorkflowRunCountEnabled is true the code returns buildChartFromWorkflowRunCount(startDate, endDate, environmentId, organizationId) without honoring workflowIds; update the call or implementation so workflowIds is passed and applied (e.g., change the call to include workflowIds or modify buildChartFromWorkflowRunCount to accept and filter by workflowIds) ensuring the workflowIds variable is used in the count calculation inside buildChartFromWorkflowRunCount.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`:
- Around line 70-74: The current mapping only reads
template.triggers?.[0]?.identifier so it misses identifiers not at index 0;
update the logic in the loop over templates to iterate all trigger entries
(e.g., for each template in templates, loop through template.triggers) and call
nameByIdentifier.set(identifier, template.name) for each trigger.identifier
found so every trigger identifier is mapped to the template name (use the
existing templates, template.triggers and nameByIdentifier symbols to locate and
fix the code).
---
Duplicate comments:
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`:
- Around line 37-39: The count-path currently ignores the workflowIds filter:
when isWorkflowRunCountEnabled is true the code returns
buildChartFromWorkflowRunCount(startDate, endDate, environmentId,
organizationId) without honoring workflowIds; update the call or implementation
so workflowIds is passed and applied (e.g., change the call to include
workflowIds or modify buildChartFromWorkflowRunCount to accept and filter by
workflowIds) ensuring the workflowIds variable is used in the count calculation
inside buildChartFromWorkflowRunCount.
In
`@apps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.ts`:
- Around line 35-37: The feature-flag branch ignores the workflowIds filter:
when isWorkflowRunCountEnabled is true the code returns
buildChartFromWorkflowRunCount(startDate, endDate, environmentId,
organizationId) and drops workflowIds, changing API behavior; update
buildChartFromWorkflowRunCount (and any callers) to accept a workflowIds
parameter and pass the existing workflowIds through (e.g., call
buildChartFromWorkflowRunCount(startDate, endDate, environmentId,
organizationId, workflowIds)), and ensure the new parameter is applied inside
buildChartFromWorkflowRunCount to filter counts by the provided workflowIds.
ℹ️ Review info
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.tsapps/api/src/app/activity/usecases/build-workflow-runs-trend-chart/build-workflow-runs-trend-chart.usecase.tsapps/dashboard/src/components/analytics/charts/workflow-runs-trend-chart.tsxpackages/shared/src/types/feature-flags.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- apps/dashboard/src/components/analytics/charts/workflow-runs-trend-chart.tsx
- packages/shared/src/types/feature-flags.ts
Comment on lines
+70
to
+74
| for (const template of templates) { | ||
| const identifier = template.triggers?.[0]?.identifier; | ||
| if (identifier) { | ||
| nameByIdentifier.set(identifier, template.name); | ||
| } |
Contributor
There was a problem hiding this comment.
Template name mapping assumes the matching trigger is always first
Line 71 only reads template.triggers?.[0]?.identifier. If the matched identifier is in another trigger entry, the chart falls back to raw workflow_run_id even though a template name exists.
Suggested fix
-const nameByIdentifier = new Map<string, string>();
+const nameByIdentifier = new Map<string, string>();
+const requestedIdentifierSet = new Set(triggerIdentifiers);
for (const template of templates) {
- const identifier = template.triggers?.[0]?.identifier;
- if (identifier) {
- nameByIdentifier.set(identifier, template.name);
+ for (const trigger of template.triggers ?? []) {
+ if (trigger?.identifier && requestedIdentifierSet.has(trigger.identifier)) {
+ nameByIdentifier.set(trigger.identifier, template.name);
+ }
}
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@apps/api/src/app/activity/usecases/build-workflow-by-volume-chart/build-workflow-by-volume-chart.usecase.ts`
around lines 70 - 74, The current mapping only reads
template.triggers?.[0]?.identifier so it misses identifiers not at index 0;
update the logic in the loop over templates to iterate all trigger entries
(e.g., for each template in templates, loop through template.triggers) and call
nameByIdentifier.set(identifier, template.name) for each trigger.identifier
found so every trigger identifier is mapped to the template name (use the
existing templates, template.triggers and nameByIdentifier symbols to locate and
fix the code).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed? Why was the change needed?
Screenshots
Expand for optional sections
Related enterprise PR
Special notes for your reviewer
What changed
This PR transitions workflow run analytics from trace-based data to a count-based rollup table approach. It introduces a new feature flag (
IS_WORKFLOW_RUN_COUNT_ENABLED) to toggle between data sources while maintaining backward compatibility. The change affects both API use cases that build analytics charts and the dashboard component that renders them, with new repository methods added to query aggregated workflow run counts.Affected areas
build-workflow-by-volume-chartandbuild-workflow-runs-trend-chartuse cases to read fromWorkflowRunCountRepositorywith feature-flagged conditional logic that falls back to the legacy workflow runs data path.WorkflowRunsTrendChartcomponent now checks the newIS_WORKFLOW_RUN_COUNT_ENABLEDflag instead of the deprecated traces-based flag.getWorkflowVolumeData()andgetWorkflowRunsTrendData()methods toWorkflowRunCountRepositoryto query aggregated workflow run counts with date range and environment/organization filtering.NotificationTemplateRepository.findByTriggerIdentifierBulk()with overloaded signatures supporting optional field projection via aselectparameter.WorkflowRunCountRepositoryinANALYTICS_PROVIDERSand replaced the deprecatedIS_WORKFLOW_RUN_TREND_FROM_TRACES_ENABLEDflag withIS_WORKFLOW_RUN_COUNT_ENABLEDin the feature flags enum.Key technical decisions
IS_WORKFLOW_RUN_COUNT_ENABLEDto conditionally route between the new count-based queries and legacy workflow runs path, enabling safe gradual migration.workflow_run_id(for volume) anddate/event_type(for trends).findByTriggerIdentifierBulkwith conditional population of thesteps.templaterelation to optimize query performance.Testing
No new tests were added. The feature is behind a feature flag allowing manual verification and gradual rollout before full migration.