feat(aci): Add new GroupOpenPeriod table #87813

snigdhas · 2025-03-24T22:20:33Z

Adds a new table GroupOpenPeriod to track the times an issue is open, i.e. having a status that is not resolved. This will be used for detector-based issues in place of events. For example, instead of relying on incidents, open periods will be used to determine the start and end of a metric alert firing for metric issues.

spec

github-actions · 2025-03-24T22:23:48Z

This PR has a migration; here is the generated SQL for src/sentry/migrations/0846_add_group_open_periods.py ()

--
-- Create model GroupOpenPeriod
--
CREATE TABLE "sentry_groupopenperiod" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "user_id" bigint NULL, "date_started" timestamp with time zone NOT NULL, "date_ended" timestamp with time zone NULL, "data" text NULL, "data_condition_id" bigint NULL, "group_id" bigint NOT NULL, "project_id" bigint NOT NULL, "resolution_activity_id" bigint NULL);
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_data_condition_id_ac18ad82_fk_workflow_" FOREIGN KEY ("data_condition_id") REFERENCES "workflow_engine_datacondition" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_data_condition_id_ac18ad82_fk_workflow_";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr" FOREIGN KEY ("group_id") REFERENCES "sentry_groupedmessage" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id" FOREIGN KEY ("project_id") REFERENCES "sentry_project" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac" FOREIGN KEY ("resolution_activity_id") REFERENCES "sentry_activity" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac";
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_user_id_807f0152" ON "sentry_groupopenperiod" ("user_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_data_condition_id_ac18ad82" ON "sentry_groupopenperiod" ("data_condition_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_group_id_f4411413" ON "sentry_groupopenperiod" ("group_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_project_id_5ef8bacc" ON "sentry_groupopenperiod" ("project_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_resolution_activity_id_cc5cc897" ON "sentry_groupopenperiod" ("resolution_activity_id");

codecov · 2025-03-24T22:45:51Z

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #87813       +/-   ##
===========================================
+ Coverage   33.09%   87.77%   +54.67%     
===========================================
  Files        8433     9982     +1549     
  Lines      470146   564343    +94197     
  Branches    22274    22166      -108     
===========================================
+ Hits       155608   495328   +339720     
+ Misses     314103    68599   -245504     
+ Partials      435      416       -19

wedamija

I thought the plan here was to use activity to figure this all out?

Will this end up creating an extra row for each group that is created?

wedamija · 2025-03-25T20:00:42Z

src/sentry/models/groupopenperiod.py

+    data_condition = FlexibleForeignKey(
+        "workflow_engine.DataCondition", null=True, on_delete=models.SET_NULL
+    )


What will this be used for?

we'll need this for the short-term to continue support existing Incident-related endpoints while we give users a heads up and time to make any changes. We can remove the column when we remove those endpoints.

I don't really follow what we're using it for though, how are we joining here?

The longer writeup is here but basically we will be updating the existing rule, alert rule, and incident serializers to read from the new ACI tables as much as possible but the main sticking point is that we need to continue returning the correct IDs for the aforementioned models.

For this specifically, the one additional piece of tricky data is the incident status returned in the serializer response here. The only way I've been able to come up with us recreating this (outside of temporarily duplicating the column on this model) is to have a FK to DataCondition so we can infer the status based on the data condition (for example, a data condition with a condition result of DetectorPriorityLevel.HIGH is the equivalent to a critical trigger today).

I'm completely open to other ideas btw

Discussed in slack and we should be able to use GroupOpenPeriod -> Group -> Group.priority or GroupOpenPeriod -> Activity (type=SET_PRIORITY) instead of the FK

markstory · 2025-03-25T20:12:38Z

src/sentry/models/groupopenperiod.py

+    date_started = models.DateTimeField(default=timezone.now)
+    date_ended = models.DateTimeField(null=True)
+
+    data: models.Field[dict[str, Any] | None, dict[str, Any]] = GzippedDictField(null=True)


Do you need to store values that are pickled? Generally I wouldn't recommend adding more fields like this. Pickled objects in the database is fragile. You could use models.JsonField instead.

gotcha, updated to models.JsonField

github-actions · 2025-03-25T21:12:28Z

This PR has a migration; here is the generated SQL for src/sentry/migrations/0849_add_group_open_periods.py ()

--
-- Create model GroupOpenPeriod
--
CREATE TABLE "sentry_groupopenperiod" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "user_id" bigint NULL, "date_started" timestamp with time zone NOT NULL, "date_ended" timestamp with time zone NULL, "data" jsonb NOT NULL, "data_condition_id" bigint NULL, "group_id" bigint NOT NULL, "project_id" bigint NOT NULL, "resolution_activity_id" bigint NULL);
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_data_condition_id_ac18ad82_fk_workflow_" FOREIGN KEY ("data_condition_id") REFERENCES "workflow_engine_datacondition" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_data_condition_id_ac18ad82_fk_workflow_";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr" FOREIGN KEY ("group_id") REFERENCES "sentry_groupedmessage" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id" FOREIGN KEY ("project_id") REFERENCES "sentry_project" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac" FOREIGN KEY ("resolution_activity_id") REFERENCES "sentry_activity" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac";
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_user_id_807f0152" ON "sentry_groupopenperiod" ("user_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_data_condition_id_ac18ad82" ON "sentry_groupopenperiod" ("data_condition_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_group_id_f4411413" ON "sentry_groupopenperiod" ("group_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_project_id_5ef8bacc" ON "sentry_groupopenperiod" ("project_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_resolution_activity_id_cc5cc897" ON "sentry_groupopenperiod" ("resolution_activity_id");

snigdhas · 2025-03-25T21:14:24Z

I thought the plan here was to use activity to figure this all out?

It was, but we'll be needing open periods in multiple places and as something we can easily query for the issue stream, the issue details graph and the datepicker in issue details. We'll also need the open periods to support existing Incident APIs. With all of that, it makes more sense to be able to query the lastest open period and update it as needed, and this should give us more nuanced info for each open period as we built it out.

Will this end up creating an extra row for each group that is created?

Yea it will -- it's not ideal given we have grouphistory and activity which both serve similar purposes but this feels like the cleanest way to support open periods more natively without having to query and recalculate open periods multiple times.

src/sentry/models/groupopenperiod.py

github-actions · 2025-03-27T18:47:39Z

This PR has a migration; here is the generated SQL for src/sentry/migrations/0852_add_group_open_periods.py ()

--
-- Create model GroupOpenPeriod
--
CREATE TABLE "sentry_groupopenperiod" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "user_id" bigint NULL, "date_started" timestamp with time zone NOT NULL, "date_ended" timestamp with time zone NULL, "data" jsonb NOT NULL, "group_id" bigint NOT NULL, "project_id" bigint NOT NULL, "resolution_activity_id" bigint NULL);
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr" FOREIGN KEY ("group_id") REFERENCES "sentry_groupedmessage" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id" FOREIGN KEY ("project_id") REFERENCES "sentry_project" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac" FOREIGN KEY ("resolution_activity_id") REFERENCES "sentry_activity" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac";
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_user_id_807f0152" ON "sentry_groupopenperiod" ("user_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_group_id_f4411413" ON "sentry_groupopenperiod" ("group_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_project_id_5ef8bacc" ON "sentry_groupopenperiod" ("project_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_resolution_activity_id_cc5cc897" ON "sentry_groupopenperiod" ("resolution_activity_id");

github-actions · 2025-03-27T18:51:55Z

This PR has a migration; here is the generated SQL for src/sentry/migrations/0853_add_group_open_periods.py ()

--
-- Creates extension btree_gist
--
CREATE EXTENSION IF NOT EXISTS "btree_gist";
--
-- Create model GroupOpenPeriod
--
CREATE TABLE "sentry_groupopenperiod" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NOT NULL, "user_id" bigint NULL, "date_started" timestamp with time zone NOT NULL, "date_ended" timestamp with time zone NULL, "data" jsonb NOT NULL, "group_id" bigint NOT NULL, "project_id" bigint NOT NULL, "resolution_activity_id" bigint NULL, CONSTRAINT "exclude_open_period_overlap" EXCLUDE USING GIST ((TSTZRANGE("date_started", "date_ended", '[)')) WITH &&));
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr" FOREIGN KEY ("group_id") REFERENCES "sentry_groupedmessage" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_group_id_f4411413_fk_sentry_gr";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id" FOREIGN KEY ("project_id") REFERENCES "sentry_project" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperiod_project_id_5ef8bacc_fk_sentry_project_id";
ALTER TABLE "sentry_groupopenperiod" ADD CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac" FOREIGN KEY ("resolution_activity_id") REFERENCES "sentry_activity" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupopenperiod" VALIDATE CONSTRAINT "sentry_groupopenperi_resolution_activity__cc5cc897_fk_sentry_ac";
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_user_id_807f0152" ON "sentry_groupopenperiod" ("user_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_group_id_f4411413" ON "sentry_groupopenperiod" ("group_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_project_id_5ef8bacc" ON "sentry_groupopenperiod" ("project_id");
CREATE INDEX CONCURRENTLY "sentry_groupopenperiod_resolution_activity_id_cc5cc897" ON "sentry_groupopenperiod" ("resolution_activity_id");
CREATE INDEX CONCURRENTLY "sentry_grou_group_i_4bffd0_idx" ON "sentry_groupopenperiod" ("group_id", "date_started");

src/sentry/models/groupopenperiod.py

wedamija · 2025-03-27T20:16:33Z

src/sentry/models/groupopenperiod.py

+    date_started = models.DateTimeField(default=timezone.now)
+    date_ended = models.DateTimeField(null=True)


Do we want to guard against overlapping open periods?

TIL about ExclusionConstraint - not quite sure if I've set it up right

src/sentry/models/groupopenperiod.py

wedamija · 2025-03-27T23:35:07Z

src/sentry/models/groupopenperiod.py

+            models.Index(fields=("group", "date_started")),
+        )
+        constraints = (
+            ExclusionConstraint(


I'm not sure if this extension is actually installed, so we should do https://docs.djangoproject.com/en/5.1/ref/contrib/postgres/operations/#django.contrib.postgres.operations.BtreeGistExtension

wedamija

lgtm, thanks!

Add new GroupOpenPeriod table

73fd6c2

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 24, 2025

Update region

761a8b1

vercel bot deployed to Preview March 24, 2025 23:25 View deployment

snigdhas marked this pull request as ready for review March 25, 2025 18:14

snigdhas requested a review from a team as a code owner March 25, 2025 18:14

snigdhas requested a review from a team March 25, 2025 18:14

wedamija reviewed Mar 25, 2025

View reviewed changes

markstory reviewed Mar 25, 2025

View reviewed changes

snigdhas added 2 commits March 25, 2025 14:06

Merge branch 'master' into snigdha/add-open-periods

2fc5ec4

Update data field and regenerate migration

97297c6

snigdhas requested review from wedamija and markstory March 25, 2025 21:14

vercel bot deployed to Preview March 25, 2025 21:18 View deployment

Merge branch 'master' into snigdha/add-open-periods

e172fab

vercel bot deployed to Preview March 27, 2025 17:52 View deployment

markstory reviewed Mar 27, 2025

View reviewed changes

src/sentry/models/groupopenperiod.py Show resolved Hide resolved

Remove column and update migration

2e3176b

vercel bot deployed to Preview March 27, 2025 18:38 View deployment

Merge branch 'master' into snigdha/add-open-periods

b2fd3e6

Add index and regenerate migration

fa69870

vercel bot deployed to Preview March 27, 2025 18:48 View deployment

vercel bot deployed to Preview March 27, 2025 18:53 View deployment

wedamija reviewed Mar 27, 2025

View reviewed changes

Add exclusionconstraint

d62f0c8

vercel bot deployed to Preview March 27, 2025 23:05 View deployment

wedamija reviewed Mar 27, 2025

View reviewed changes

snigdhas added 2 commits March 27, 2025 16:44

Add BtreeGistExtension

6cf162f

Update field names

f776b7c

vercel bot deployed to Preview March 27, 2025 23:48 View deployment

Merge branch 'master' into snigdha/add-open-periods

6d21690

vercel bot deployed to Preview March 28, 2025 16:20 View deployment

Merge branch 'master' into snigdha/add-open-periods

1c03728

vercel bot deployed to Preview March 28, 2025 18:51 View deployment

wedamija approved these changes Mar 28, 2025

View reviewed changes

snigdhas merged commit 7111b1d into master Mar 28, 2025
48 checks passed

snigdhas deleted the snigdha/add-open-periods branch March 28, 2025 20:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(aci): Add new GroupOpenPeriod table #87813

feat(aci): Add new GroupOpenPeriod table #87813

snigdhas commented Mar 24, 2025 •

edited

Loading

github-actions bot commented Mar 24, 2025

codecov bot commented Mar 24, 2025 •

edited

Loading

wedamija left a comment

wedamija Mar 25, 2025

snigdhas Mar 25, 2025

wedamija Mar 25, 2025

ceorourke Mar 26, 2025

snigdhas Mar 27, 2025

markstory Mar 25, 2025

snigdhas Mar 25, 2025

github-actions bot commented Mar 25, 2025

snigdhas commented Mar 25, 2025

github-actions bot commented Mar 27, 2025

github-actions bot commented Mar 27, 2025 •

edited

Loading

wedamija Mar 27, 2025

snigdhas Mar 27, 2025

wedamija Mar 27, 2025

wedamija left a comment

		date_started = models.DateTimeField(default=timezone.now)
		date_ended = models.DateTimeField(null=True)

feat(aci): Add new GroupOpenPeriod table #87813

feat(aci): Add new GroupOpenPeriod table #87813

Conversation

snigdhas commented Mar 24, 2025 • edited Loading

github-actions bot commented Mar 24, 2025

codecov bot commented Mar 24, 2025 • edited Loading

Codecov Report

wedamija left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 25, 2025

snigdhas commented Mar 25, 2025

github-actions bot commented Mar 27, 2025

github-actions bot commented Mar 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wedamija left a comment

Choose a reason for hiding this comment

snigdhas commented Mar 24, 2025 •

edited

Loading

codecov bot commented Mar 24, 2025 •

edited

Loading

github-actions bot commented Mar 27, 2025 •

edited

Loading