♻️ preparation for osparc migration (merging deployments) (🗃️)

[matusdrobuliak66]

What do these changes do?

• ♻️ Remove project comments resource (🗃️)
• 🗃️ Add missing foreign keys
• 🗃️ Add zzz_resource_tracker_service_runs__osparc_io_archive_202508 where we will archive osparc service runs data.zzz prefix is so it is in the end of all tables.

PLAN for index increment:

table	increment column	serial sequence name
api_keys	id	api_keys_id_seq
comp_runs	run_id	comp_runs_run_id_seq
comp_tasks	task_id	comp_tasks_task_id_seq
folders_v2	folder_id	folders_v2_folder_id_seq
group_classifiers	id	group_classifiers_id_seq
groups	gid	groups_gid_seq
payments_autorecharge	id	payments_autorecharge_id_seq
projects_nodes	project_node_id	projects_nodes_project_node_id_seq
tags	id	tags_id_seq
tokens	token_id	tokens_token_id_seq
user_to_projects	id	user_to_projects_id_seq
users	id	users_id_seq
wallets	wallet_id	wallets_wallet_id_seq
workspaces	workspace_id	workspaces_workspace_id_seq

NOTE! resource_tracker_service_runs user ID needs to be manually incremented (foreign key can not be introduced, because all anonymous users are always deleted)

For migration:

-- users
UPDATE users SET id = id + 2000000;
SELECT setval('users_id_seq', (SELECT MAX(id) FROM users));
-- user_to_projects
UPDATE user_to_projects SET id = id + 2000000;
SELECT setval('user_to_projects_id_seq', (SELECT MAX(id) FROM user_to_projects));
-- EXPLICITELY resource_usage_tracker_service_runs (It is without the foreign key)
UPDATE resource_tracker_service_runs SET user_id = user_id + 2000000;

-- wallets
UPDATE wallets SET wallet_id = wallet_id + 2000000;
SELECT setval('wallets_wallet_id_seq', (SELECT MAX(wallet_id) FROM wallets));
-- workspaces
UPDATE workspaces SET workspace_id = workspace_id + 2000000;
SELECT setval('workspaces_workspace_id_seq', (SELECT MAX(workspace_id) FROM workspaces));
-- tokens
UPDATE tokens SET token_id = token_id + 2000000;
SELECT setval('tokens_token_id_seq', (SELECT MAX(token_id) FROM tokens));
-- tags
UPDATE tags SET id = id + 2000000;
SELECT setval('tags_id_seq', (SELECT MAX(id) FROM tags));
-- projects_nodes
UPDATE projects_nodes SET project_node_id = project_node_id + 2000000;
SELECT setval('projects_nodes_project_node_id_seq', (SELECT MAX(project_node_id) FROM projects_nodes));
-- payments_autorecharge
UPDATE payments_autorecharge SET id = id + 2000000;
SELECT setval('payments_autorecharge_id_seq', (SELECT MAX(id) FROM payments_autorecharge));
-- groups
UPDATE groups SET gid = gid + 2000000 where gid != 1;
SELECT setval('groups_gid_seq', (SELECT MAX(gid) FROM groups));
-- group_classifiers
UPDATE group_classifiers SET id = id + 2000000;
SELECT setval('group_classifiers_id_seq', (SELECT MAX(id) FROM group_classifiers));
-- folders_v2
UPDATE folders_v2 SET folder_id = folder_id + 2000000;
SELECT setval('folders_v2_folder_id_seq', (SELECT MAX(folder_id) FROM folders_v2));
-- comp_tasks
UPDATE comp_tasks SET task_id = task_id + 7000000;
SELECT setval('comp_tasks_task_id_seq', (SELECT MAX(task_id) FROM comp_tasks));
-- comp_runs
UPDATE comp_runs SET run_id = run_id + 2000000;
SELECT setval('comp_runs_run_id_seq', (SELECT MAX(run_id) FROM comp_runs));
-- api_keys
UPDATE api_keys SET id = id + 2000000;
SELECT setval('api_keys_id_seq', (SELECT MAX(id) FROM api_keys));

For archiving the runs table:

INSERT INTO zzz_resource_tracker_service_runs__osparc_io_archive_202508 (
    product_name,
    service_run_id,
    wallet_id,
    wallet_name,
    pricing_plan_id,
    pricing_unit_id,
    pricing_unit_cost_id,
    pricing_unit_cost,
    simcore_user_agent,
    user_id,
    user_email,
    project_id,
    project_name,
    node_id,
    node_name,
    parent_project_id,
    root_parent_project_id,
    root_parent_project_name,
    parent_node_id,
    root_parent_node_id,
    service_key,
    service_version,
    service_type,
    service_resources,
    service_additional_metadata,
    started_at,
    stopped_at,
    service_run_status,
    modified,
    last_heartbeat_at,
    service_run_status_msg,
    missed_heartbeat_counter
)
SELECT
    product_name,
    service_run_id,
    wallet_id,
    wallet_name,
    pricing_plan_id,
    pricing_unit_id,
    pricing_unit_cost_id,
    pricing_unit_cost,
    simcore_user_agent,
    user_id,
    user_email,
    project_id,
    project_name,
    node_id,
    node_name,
    parent_project_id,
    root_parent_project_id,
    root_parent_project_name,
    parent_node_id,
    root_parent_node_id,
    service_key,
    service_version,
    service_type::text::resourcetrackerservicetypeosparciohistory,
    service_resources,
    service_additional_metadata,
    started_at,
    stopped_at,
    service_run_status::text::resourcetrackerservicerunstatusosparciohistory,
    modified,
    last_heartbeat_at,
    service_run_status_msg,
    missed_heartbeat_counter
FROM resource_tracker_service_runs;

Related issue/s

• relates move of osparc.io: make db tables ready for merging osparc-issues#1867
• relates Reducing Overhead Costs by Merging Deployments (Aug. -> Feb) osparc-issues#1652

How to test

Dev-ops

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.42%. Comparing base (87a16bc) to head (02f707c).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7700      +/-   ##
==========================================
- Coverage   87.13%   84.42%   -2.72%     
==========================================
  Files        2010     1419     -591     
  Lines       79007    58787   -20220     
  Branches     1377      438     -939     
==========================================
- Hits        68844    49629   -19215     
+ Misses       9764     9042     -722     
+ Partials      399      116     -283     

Flag	Coverage Δ		*Carryforward flag
integrationtests	63.97% <ø> (+<0.01%)	⬆️	Carriedforward from ae7c575
unittests	82.43% <ø> (-3.77%)	⬇️

*This pull request uses carry forward flags. Click here to find out more.

Components	Coverage Δ
pkg_aws_library	∅ <ø> (∅)
pkg_celery_library	∅ <ø> (∅)
pkg_dask_task_models_library	∅ <ø> (∅)
pkg_models_library	92.83% <ø> (ø)
pkg_notifications_library	85.20% <ø> (ø)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	∅ <ø> (∅)
pkg_service_library	∅ <ø> (∅)
pkg_settings_library	∅ <ø> (∅)
pkg_simcore_sdk	84.45% <ø> (-0.14%)	⬇️
agent	93.44% <ø> (ø)
api_server	91.37% <ø> (ø)
autoscaling	95.83% <ø> (ø)
catalog	92.06% <ø> (ø)
clusters_keeper	99.14% <ø> (ø)
dask_sidecar	91.72% <ø> (ø)
datcore_adapter	97.95% <ø> (ø)
director	75.68% <ø> (ø)
director_v2	91.23% <ø> (-0.09%)	⬇️
dynamic_scheduler	96.66% <ø> (ø)
dynamic_sidecar	82.00% <ø> (ø)
efs_guardian	89.83% <ø> (ø)
invitations	90.90% <ø> (ø)
payments	∅ <ø> (∅)
resource_usage_tracker	∅ <ø> (∅)
storage	∅ <ø> (∅)
webclient	∅ <ø> (∅)
webserver	74.54% <ø> (-12.17%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87a16bc...02f707c. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sanderegg]

did you try with the DBs already?
thanks!

[mrnicegyu11]

I dont fully get what happens in this PR (what is CASCADE?) or the introduction of the zzz_resource_tracker_service_runs__osparc_io_archive_202508 table, but I also dont see blockers. :--)

[pcrespov]

thx. Please double check mycomments. Specially theone of user_id

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[mergify]

🧪 CI Insights

Here's what we observed from your CI run for 02f707c.

❌ Job Failures

Pipeline	Job	Health on master	Retries	🔍 CI Insights	📄 Logs
CI	integration-tests		0	View	View
	system-tests		0	View	View
	unit-tests		0	View	View

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud