Skip to content

Commit 3c7db22

Browse files
docs: Overview dashboard (#9417)
Co-authored-by: caitlinwheeless <caitlinwheeless@users.noreply.github.com>
1 parent d60b7c0 commit 3c7db22

File tree

7 files changed

+130
-10
lines changed

7 files changed

+130
-10
lines changed

docs/source/guide/dashboard_annotator.md

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Member performance dashboard
3-
short: Performance dashboard
3+
short: Member performance
44
tier: enterprise
55
type: guide
66
order: 0
@@ -45,7 +45,7 @@ Depending on your user role, you can access this dashboard in several ways:
4545
| User role | Access restrictions |
4646
|-----------------------|------------------------------------------------------------------------------------------------------|
4747
| **Owner** <br>and<br> **Admin** | Can access from the navigation menu, the Organization page, the Home page, and the Members dashboard. <br> Can filter for all users. <br> Can filter for all workspaces. |
48-
| **Manager** | Can access from the navigation menu, the Home page and the Members dashboard. <br> Can filter for all users. <br> Can only filter for workspaces or projects in which they are a member. <br><br> Note that this means Managers can see the full organization user list. However, they can only see annotation and review metrics for work completed within their own projects/workspaces. |
48+
| **Manager** | Can access from the navigation menu, the Home page and the Members dashboard. <br> Can only filter for users who are members of workspaces or projects in which they are also a member. <br> Can only filter for workspaces or projects in which they are a member. |
4949
| **Reviewer** <br>and<br> **Annotator** | Can access from navigation menu and the Home page. <br> Can only see their own annotation and reviewer history. <br> Can only filter for workspaces or projects in which they are a member. |
5050

5151

@@ -76,10 +76,13 @@ The **Annotations** tab shows information related to a user's annotation perform
7676

7777
The metrics are calculated from the following data:
7878

79-
* `last_action` -- The last action taken on an annotation. This can can be submitted, updated, fixed and accepted, accepted, rejected.
79+
* `last_action` -- The last action taken on an annotation. This can be submitted, updated, fixed and accepted, accepted, rejected.
8080
* `submitted_or_reviewed` -- Annotations that have a last action matching those listed above.
8181
* `lead_times` -- The time spent with annotations that have a last action matching those listed above. Lead time includes idle time and is cumulative across multiple sessions.
8282

83+
!!! note
84+
A dash (`--`) appears when there is no data. A zero (`0`) appears when there is data available and the value is zero.
85+
8386
### Annotation performance summaries
8487

8588
| Metric | Calculation | Description |
@@ -88,11 +91,11 @@ The metrics are calculated from the following data:
8891
| **Time Annotating** | Sum of `lead_times` | The total time spent annotating during the selected time frame. This is calculated based on annotations that meet the criteria for `submitted_or_reviewed`. <br /><br />All annotations have a lead time. The lead time reflects how much time a user spent labeling from the moment the task was opened until they click **Submit** or **Update**. This includes idle time and time spent updating previously submitted annotations. <br /><br />The total time does not include time spent on annotations that have not been submitted and/or updated. For example, it does not include time spent on drafts or time spent on skipped annotations. <br /><br />Lead time is cumulative across sessions. That means if they leave or log out, and then return to an annotation draft or a previously skipped annotation, then their earlier time spent on the annotation is included when calculating their lead time. |
8992
| **Performance Score** | Calculated from reviewer actions | The Performance Score reflects the overall performance of annotators in terms of review actions (**Accept**, **Reject**, **Fix+Accept**). <br /><br />The calculation is as follows:<ul><li>Each annotation review action (**Accept**, **Reject**, **Fix+Accept**) contributes to the score.</li><li>The score is calculated by summing the scores of all review actions and dividing by the total number of review actions. For example: </li><ul><li>If an annotation is rejected twice and then accepted once, the Performance Score would be (0 + 0 + 1) / 3 = 33%.</li><li>If an annotation is rejected once and then fixed+accepted with an agreement score of 42%, the Performance Score would be (0 + 0.42) / 2 = 21%. The agreement here is measured between what the annotator submitted and what the reviewer submitted when fixing it.</li></ul></ul> |
9093

91-
### Annotation graphs
94+
### Annotation charts
9295

9396
![Screenshot of annotator dashboard graphs](/images/analytics/annotator-graphs.png)
9497

95-
| Graph | Description |
98+
| Chart | Description |
9699
| --- | --- |
97100
| **Annotation States Over Time** | A summary of all annotations completed by the user, segmented by date over the selected time period. |
98101
| **Time Spent Annotating** | The time spent annotating (the lead time), segmented by date over the selected time period. <br /><br />This includes two metrics: The total time spent annotating in hours, and the average time per annotation in minutes. |
@@ -128,6 +131,9 @@ Keep in mind that all metrics in the table, including task assignments, are [fil
128131

129132
The **Reviews** tab shows information related to a user's activity reviewing other users' annotations.
130133

134+
!!! note
135+
A dash (`--`) appears when there is no data. A zero (`0`) appears when there is data available and the value is zero.
136+
131137
### Review performance summaries
132138

133139
| Metric | Description |
@@ -138,11 +144,11 @@ The **Reviews** tab shows information related to a user's activity reviewing oth
138144
!!! note
139145
Data collection for review time began on September 25, 2025 for cloud environments, or for 2.30 for on-prem deployments. If you filter for earlier dates, review time will not be calculated.
140146

141-
### Review graphs
147+
### Review charts
142148

143149
![Screenshot of review dashboard graphs](/images/analytics/reviewer-graphs.png)
144150

145-
| Graph | Description |
151+
| Chart | Description |
146152
| --- | --- |
147153
| **Review States Over Time** | A summary of all reviews completed by the user, segmented by date over the selected time period. |
148154
| **Time Spent Reviewing** | The time spent reviewing, segmented by date over the selected time period. <br /><br />This includes two metrics: The total time spent reviewing in hours, and the average review time per annotation in minutes. |
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
title: Projects overview dashboard
3+
short: Projects overview
4+
tier: enterprise
5+
type: guide
6+
order: 0
7+
order_enterprise: 172
8+
meta_title: Projects Overview Dashboard
9+
meta_description: Use the analytics overview page to explore progress across multiple projects
10+
section: "Analytics"
11+
date: 2026-01-10 11:50:41
12+
---
13+
14+
The **Analytics > Overview** page is a centralized starting point to explore performance insights across multiple projects.
15+
16+
![Screenshot Overview dashboard](/images/analytics/overview.png)
17+
18+
#### Access by user role
19+
20+
| User role | Access restrictions |
21+
|-----------|---------------------|
22+
| **Owner** and **Admin** | Can view the **Overview** dashboard and filter for all workspaces and projects. |
23+
| **Manager** | Can view the **Overview** dashboard, but can only filter for and view workspaces or projects in which they are a member. |
24+
| **Reviewer** and **Annotator** | Cannot access the **Overview** dashboard. <br/><br/>When they open **Analytics**, they are redirected to the [Member Performance dashboard](dashboard_annotator), where they can only see their own annotation and review history. |
25+
26+
## Filtering
27+
28+
### Filter by project and workspace
29+
30+
Select one or more workspaces or individual projects. To see all projects, select **All Workspaces** at the top of the workspace filter.
31+
32+
This does not include Personal Sandbox projects, and is limited to 50 projects at a time.
33+
34+
35+
### Filter by date range
36+
37+
You can select a date range to apply to the project metrics. If you want to be precise, you can also include time.
38+
39+
Note that annotation and acceptance metrics are calculated on when they are **completed**, not when they **begin**.
40+
41+
For example, say you select a date range of 2 days: Monday and Tuesday. If an annotator opens an annotation on Tuesday, begins working on it, and then submits it on Wednesday, then that annotation and the time they spent on it will **not** be included in the metrics.
42+
43+
44+
## Tasks and project states
45+
46+
For more information about the states referenced in the Overview dashboard, see [Project and task state management](project_states).
47+
48+
!!! note
49+
50+
Project and task state history tracking did not start until state management was implemented for your organization. For most Label Studio Cloud organizations, state management was implemented in February 2026.
51+
52+
This means that the **Overview** dashboard will not reflect when a project or task changed states if that change happened before state management was implemented.
53+
54+
55+
## Project progress summaries
56+
57+
At the top of the Overview dashboard, summary cards show aggregate metrics for the selected date range and workspaces or projects.
58+
59+
!!! note
60+
A dash (`--`) appears when there is no data. A zero (`0`) appears when there is data available and the value is zero.
61+
62+
63+
| Metric | Description |
64+
|--------|--------------|
65+
| **Total Done Tasks** | Total number of tasks in the [**Done** state](project_states#Task-states) across the selected projects. |
66+
| **Total Tasks Annotated** | Total number of tasks that have the required annotations completed across all selected projects. This includes tasks in the [**Done** state, **Needs Review** state, and the **In Review** state](project_states#Task-states). |
67+
| **Total Annotations Submitted** | Total number of annotations submitted across all selected projects. This includes annotations that have been submitted or updated, whether or not they have been reviewed. |
68+
| **Total Time Spent** | Total time spent annotating (lead time) and reviewing across all selected projects. <br/><br/> The lead time is the time spent on annotations in which the last action was one of the following: **Submit**, **Update**, **Fix + Accept**, **Accept**, **Reject**. <br/><br/>Lead time includes idle time and is cumulative across multiple sessions. |
69+
70+
71+
## Project progress charts
72+
73+
The Overview dashboard includes charts that break down metrics over the selected time period.
74+
75+
Dates for annotations and reviews are based on when they were completed, not when they were started.
76+
77+
![Screenshot Overview dashboard](/images/analytics/overview-charts.png)
78+
79+
80+
| Chart | Description |
81+
|-------|--------------|
82+
| **Task Progress** | Shows how tasks move through [states](project_states#Task-states) over the selected period of time.|
83+
| **Annotations Submitted vs Accepted** | Shows submitted annotations and accepted annotations over time, with an acceptance score line (the acceptance score is the number of accepted annotations divided by the number of annotations reviewed). Note that accepted annotations includes annotations that were fixed and then accepted. |
84+
| **Time Spent** | Shows time spent annotating and reviewing over time. Includes total time in the period and average time per submitted annotation. |
85+
86+
87+
## Project progress table
88+
89+
The **Project Progress** table display detailed metrics for each project.
90+
91+
![Screenshot Overview dashboard](/images/analytics/overview-table.png)
92+
93+
!!! note
94+
A dash (`--`) appears when there is no data. A zero (`0`) appears when there is data available and the value is zero.
95+
96+
| Column | Description |
97+
|--------|--------------|
98+
| **Workspace** | The workspace that contains the project. |
99+
| **Project** | The project name. Click to open the project dashboard. |
100+
| **Task Count** | Number of tasks in the project. This includes every task and is not date filtered. |
101+
| **Tasks (Initial)** | Number of tasks in the [**Initial** state](project_states#Task-states). |
102+
| **Tasks (Annotating)** | Number of tasks in the [**Annotating** state](project_states#Task-states). |
103+
| **Tasks (Needs Review)** | Number of tasks in the [**Needs Review** state](project_states#Task-states). |
104+
| **Tasks (In Review)** | Number of tasks in the [**In Review** state](project_states#Task-states). |
105+
| **Tasks (Done)** | Number of tasks in the [**Done** state](project_states#Task-states). |
106+
| **Tasks Annotated** | Total number of tasks that have each received the required number of annotations during the selected date range. This includes tasks in the **Done** state, **Needs Review** state, and the **In Review** state. |
107+
| **Annotations Submitted** | Number of annotations submitted (includes those updated and/or reviewed). |
108+
| **Annotations Accepted** | Annotations accepted or fixed and then accepted. |
109+
| **Total Time Spent** | Total time spent annotating and reviewing annotations. |
110+
| **Avg Time / Annotation** | Average time spent per submitted annotation. Includes both annotation and review times. |
111+
| **Avg Agreement Score** | Average agreement Score across all tasks which have received the required number of annotations. |
112+
| **Acceptance Score** | Total number of accepted annotations divided by the number of annotations that have been reviewed. |

docs/source/guide/manage_projects.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,11 +64,11 @@ When you duplicate a project, you can copy the following:
6464

6565
* Labeling configuration
6666
* Most project settings
67+
* Project membership settings
6768
* Project tasks or uploaded data (optionally)
6869

6970
Duplication does not include:
7071

71-
* Project membership settings
7272
* Task assignments
7373
* Completed annotations
7474
* External storage settings

docs/source/guide/project_states.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Project and Task State Management
2+
title: Project and task state management
33
short: Project and task states
44
type: guide
55
tier: enterprise
@@ -10,6 +10,9 @@ section: "Create & Manage Projects"
1010

1111
Project and tasks move through a series of states as they progress from their initial created state to completion.
1212

13+
!!! note
14+
Note that state change history tracking did not start until state management was implemented for your organization. For most Label Studio Cloud organizations, state management was implemented in February 2026.
15+
1316
## Project states
1417

1518
![Screenshot of project states](/images/project/project-states.png)
@@ -98,7 +101,6 @@ stateDiagram-v2
98101

99102
<img src="/images/project/state-history.png" style="max-width: 450px" alt="Screenshot">
100103

101-
Note that state change history tracking did not start until state management was implemented for your organization. For most Label Studio Cloud organizations, state management was implemented in February 2026.
102104

103105
## Additional notes
104106

103 KB
Loading
99.9 KB
Loading
130 KB
Loading

0 commit comments

Comments
 (0)