feat: expose active_workers from semaphore for accurate idle researcher count (#4644)

## Summary

- **Expose semaphore entry counts** as `active_workers` (per-task) and
`user_active_workers` (per-user) on the `TaskStatusResponse` API, giving
an accurate view of real worker concurrency
- **Fix idle researcher calculation** — the old `idle = pool_size -
running` was broken for most operation types because artifact RUNNING
status doesn't track actual semaphore-held concurrency:
- **Classify**: `running` counted all rows in active batches (e.g. 30),
but only 3 semaphore slots used
- **Forecast**: all rows marked RUNNING upfront before agents even
started
- **Dedupe**: artifacts never marked RUNNING during execution; `running
≈ 0` while workers are active
- **New formula**: `idle = pool_size - (user_active_workers ?? running)`
— falls back to `running` for backward compat
- **Remove forecast upfront RUNNING hack** — the block that marked all
row artifacts RUNNING before research agents were queued is no longer
needed since `active_workers` from the semaphore shows real concurrency

## Changes (14 files)

**Engine (4 files)**
- `semaphore.py` — add `get_task_entry_count()` method +
`get_task_active_workers()` convenience function
- `results.py` — add `active_workers` and `user_active_workers` fields
to `TaskStatusResponse`
- `handlers/tasks.py` — populate new fields from semaphore in
`get_task_status`
- `forecast.py` — remove upfront RUNNING artifact status hack

**MCP (1 file)**
- `tool_helpers.py` — add `active_workers` / `user_active_workers`
computed properties on `TaskState`, include in `progress_message()` text

**Frontend (9 files)**
- `types.ts` — add fields to `TaskStatusResponse` interface
- `artifact-data.ts` — parse `active_workers` and `user_active_workers`
from MCP progress text
- `useEveryrowPolling.ts` — thread new fields through the hook
- `StreamStatsBar.tsx`, `LiveAnimation.tsx`, `LiveUtilityAnimation.tsx`,
`ResearcherStreamVizPane.tsx` — use `userActiveWorkers` prop for idle
calc
- `TeamToolCall.tsx`, `TaskGroup.tsx` — use parsed `userActiveWorkers`
for idle calc

## Test plan

- [x] TypeScript compiles clean (`tsc --noEmit`)
- [x] Python syntax valid for all modified files
- [x] Pre-commit hooks pass (format, lint, typecheck for engine, MCP,
and frontend)
- [ ] Engine API tests (need env vars — verify in CI)
- [ ] E2E: `pnpm cypress:run:replay`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Sourced from commit 879e62bc2bd40d7862e9356cd3ba281d9a4a2008

Author: jackwildman

futuresearch-mcp/src/futuresearch_mcp/tool_helpers.py

@@ -395,6 +395,21 @@ def total(self) -> int:
         p = self._response.progress
         return p.total if p else 0
 
+    @computed_field
+    @property
+    def pool_size(self) -> int | None:
+        return self._response.additional_properties.get("pool_size")
+
+    @computed_field
+    @property
+    def active_workers(self) -> int | None:
+        return self._response.additional_properties.get("active_workers")
+
+    @computed_field
+    @property
+    def user_active_workers(self) -> int | None:
+        return self._response.additional_properties.get("user_active_workers")
+
     @computed_field
     @property
     def artifact_id(self) -> str:
@@ -462,9 +477,22 @@ def progress_message(
             return f"Task {self.status.value}. Report the error to the user."
 
         fail_part = f", {self.failed} failed" if self.failed else ""
+        pool_part = (
+            f", pool_size {self.pool_size}" if self.pool_size is not None else ""
+        )
+        aw_part = (
+            f", active_workers {self.active_workers}"
+            if self.active_workers is not None
+            else ""
+        )
+        uaw_part = (
+            f", user_active_workers {self.user_active_workers}"
+            if self.user_active_workers is not None
+            else ""
+        )
         cursor_arg = f", cursor='{cursor}'" if cursor else ""
         msg = dedent(f"""\
-            Running: {self.completed}/{self.total} complete, {self.running} running{fail_part} ({self.elapsed_s}s elapsed)""")
+            Running: {self.completed}/{self.total} complete, {self.running} running{fail_part}{pool_part}{aw_part}{uaw_part} ({self.elapsed_s}s elapsed)""")
 
         if summaries:
             msg += "\n\nAgent activity:" + _format_summary_lines(summaries)

futuresearch-mcp/tests/test_tool_helpers.py

@@ -25,6 +25,7 @@ def _make_status_response(
     progress: TaskProgressInfo | None = None,
     created_at: datetime.datetime | None = None,
     updated_at: datetime.datetime | None = None,
+    additional_properties: dict | None = None,
 ):
     """Build a mock TaskStatusResponse."""
     resp = MagicMock()
@@ -35,6 +36,7 @@ def _make_status_response(
     resp.error = error
     resp.created_at = created_at
     resp.updated_at = updated_at
+    resp.additional_properties = additional_properties or {}
 
     if progress is None:
         p = MagicMock()
@@ -160,3 +162,69 @@ def test_mixed_with_and_without_row_index(self):
         result = _format_summary_lines(summaries)
         assert "[Row 3] Has index" in result
         assert "No index" in result
+
+
+class TestTaskStateActiveWorkers:
+    def test_active_workers_from_additional_properties(self):
+        resp = _make_status_response(
+            additional_properties={"active_workers": 7, "user_active_workers": 12},
+        )
+        ts = TaskState(resp)
+        assert ts.active_workers == 7
+        assert ts.user_active_workers == 12
+
+    def test_active_workers_none_when_absent(self):
+        resp = _make_status_response(additional_properties={})
+        ts = TaskState(resp)
+        assert ts.active_workers is None
+        assert ts.user_active_workers is None
+
+    def test_pool_size_still_works(self):
+        resp = _make_status_response(
+            additional_properties={"pool_size": 30, "active_workers": 5},
+        )
+        ts = TaskState(resp)
+        assert ts.pool_size == 30
+        assert ts.active_workers == 5
+
+
+class TestProgressMessageActiveWorkers:
+    def test_running_message_includes_active_workers(self):
+        p = MagicMock()
+        p.completed = 10
+        p.failed = 1
+        p.running = 5
+        p.total = 50
+        resp = _make_status_response(
+            status=TaskStatus.RUNNING,
+            progress=p,
+            created_at=datetime.datetime.now(datetime.UTC),
+            additional_properties={
+                "pool_size": 30,
+                "active_workers": 8,
+                "user_active_workers": 15,
+            },
+        )
+        ts = TaskState(resp)
+        msg = ts.progress_message("task-aw")
+        assert "active_workers 8" in msg
+        assert "user_active_workers 15" in msg
+        assert "pool_size 30" in msg
+
+    def test_running_message_omits_active_workers_when_absent(self):
+        p = MagicMock()
+        p.completed = 10
+        p.failed = 0
+        p.running = 5
+        p.total = 50
+        resp = _make_status_response(
+            status=TaskStatus.RUNNING,
+            progress=p,
+            created_at=datetime.datetime.now(datetime.UTC),
+            additional_properties={"pool_size": 30},
+        )
+        ts = TaskState(resp)
+        msg = ts.progress_message("task-no-aw")
+        assert "active_workers" not in msg
+        assert "user_active_workers" not in msg
+        assert "pool_size 30" in msg