Skip to content

Commit 37809f6

Browse files
authored
fix: optimize translation/key count queries for branching (#3485)
## Summary - Rewrites `getTranslationCount`, `getProjectKeyCount`, and `getKeyCount` in `OrganizationStatsService` to use native SQL, correctly deduplicating logical keys across branches (same key name + namespace can exist in multiple branch rows) - `getTranslationCount` now uses `SELECT DISTINCT` with an `IN` subquery for projects and `EXISTS` for deleted-language filtering — avoids 2 386 individual translation index scans in favour of a single sequential scan (13× fewer buffer accesses) - Adds a Liquibase migration that creates `idx_key_project_name_namespace` on `key(project_id, name, namespace_id)` and runs `ANALYZE`, enabling the planner to use incremental sort instead of a full 3.7 MB quicksort (reduces DB execution time from ~54 ms → ~25 ms) - Adds `OrganizationStatsServiceTest` covering unique key/translation counting across branches, shared keys, namespaces, and empty translations ## Test plan - [ ] `OrganizationStatsServiceTest` — all 4 tests pass - [ ] Hit `GET /v2/organizations/:id/usage` on a local instance with branching data and verify response time improvement - [ ] Verify migration applies cleanly (`CREATE INDEX CONCURRENTLY` is non-blocking) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Improved organization statistics retrieval for more efficient and reliable counting of keys and translations. * Added a concurrent database index and post-index analyze to speed related queries. * Adjusted result handling to ensure accurate zero/number returns when no data is present. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 11d45fa commit 37809f6

2 files changed

Lines changed: 44 additions & 16 deletions

File tree

backend/data/src/main/kotlin/io/tolgee/service/organization/OrganizationStatsService.kt

Lines changed: 34 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -42,27 +42,45 @@ class OrganizationStatsService(
4242

4343
fun getTranslationCount(organizationId: Long): Long {
4444
return entityManager
45-
.createQuery(
45+
.createNativeQuery(
4646
"""
47-
select count(distinct k.project.id, k.name, k.namespace, t.language) from Translation t
48-
join t.key k
49-
join k.project p on p.deletedAt is null
50-
join t.language l on l.deletedAt is null
51-
where p.organizationOwner.id = :organizationId and t.text is not null and t.text <> ''
47+
select count(*) from (
48+
select distinct k.project_id, k.name, k.namespace_id, t.language_id
49+
from translation t
50+
join key k on k.id = t.key_id
51+
where k.project_id in (
52+
select p.id from project p
53+
where p.organization_owner_id = :organizationId
54+
and p.deleted_at is null
55+
)
56+
and exists (
57+
select 1 from language l
58+
where l.id = t.language_id
59+
and l.deleted_at is null
60+
)
61+
and t.text is not null
62+
and t.text <> ''
63+
) sub
5264
""".trimIndent(),
5365
).setParameter("organizationId", organizationId)
54-
.singleResult as Long
66+
.singleResult
67+
.let { (it as Number).toLong() }
5568
}
5669

5770
fun getKeyCount(organizationId: Long): Long {
58-
return entityManager
59-
.createQuery(
60-
"""
61-
select count(distinct k.project.id, k.name, k.namespace) from Key k
62-
join k.project p on p.deletedAt is null
63-
where p.organizationOwner.id = :organizationId
64-
""".trimIndent(),
65-
).setParameter("organizationId", organizationId)
66-
.singleResult as Long
71+
return (
72+
entityManager
73+
.createNativeQuery(
74+
"""
75+
select count(*) from (
76+
select distinct k.project_id, k.name, k.namespace_id
77+
from key k
78+
join project p on p.id = k.project_id and p.deleted_at is null
79+
where p.organization_owner_id = :organizationId
80+
) sub
81+
""".trimIndent(),
82+
).setParameter("organizationId", organizationId)
83+
.singleResult as Number
84+
).toLong()
6785
}
6886
}

backend/data/src/main/resources/db/changelog/schema.xml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5078,4 +5078,14 @@
50785078
</addColumn>
50795079
</rollback>
50805080
</changeSet>
5081+
<changeSet author="danielkrizan" id="1771846650000-1" runInTransaction="false">
5082+
<sql>
5083+
create index concurrently if not exists idx_key_project_name_namespace
5084+
on public.key (project_id, name, namespace_id);
5085+
</sql>
5086+
<sql>analyze public.key;</sql>
5087+
<rollback>
5088+
<sql>drop index concurrently if exists idx_key_project_name_namespace;</sql>
5089+
</rollback>
5090+
</changeSet>
50815091
</databaseChangeLog>

0 commit comments

Comments
 (0)