fix(gateway): return ISO 8601 timestamps from threads endpoints by fancyboi999 · Pull Request #2599 · bytedance/deer-flow

fancyboi999 · 2026-04-27T03:38:56Z

What

Threads endpoints under /api/threads return created_at / updated_at as unix-second strings ("1777252410.411327") instead of the ISO 8601 format that ThreadResponse documents and that the LangGraph Platform schema (langgraph_sdk.schema.Thread) treats as datetime. This PR fixes the wire format.

Closes #2594.

Why

Three concrete fallouts of the mismatch:

The frontend's formatTimeAgo (frontend/src/core/utils/datetime.ts) ends up calling new Date("1777252410.411327"), which returns Invalid Date. The chats list (frontend/src/app/workspace/chats/page.tsx:58) and the markdown export (frontend/src/core/threads/export.ts:32) render "Invalid Date" today.
search_threads returns mixed formats from the same endpoint. Phase 1 reads the store (unix-float strings written by the gateway) while Phase 2 reads the checkpointer (ISO strings written by LangGraph Server). The results.sort(key=lambda r: r.updated_at, reverse=True) is a lexical string sort, so "1777..." always lands after "2026..." regardless of actual time.
Every other timestamp source in the repo (thread_runs.py, assistants_compat.py, agents/memory/storage.py, guardrails/middleware.py, skills/manager.py, sandbox_audit_middleware.py) already uses datetime.now(UTC).isoformat(), and tests/test_client.py:1007 asserts ISO. threads.py was the only file producing the legacy format.

How

New helper at backend/packages/harness/deerflow/utils/time.py:

now_iso() returns datetime.now(UTC).isoformat().
coerce_iso(value) converts legacy unix-second strings/numbers to ISO; ISO strings, empty values, and unrecognised inputs pass through. The legacy match is anchored to 10 digits (^\d{10}(?:\.\d+)?$) so a 4-digit ISO year like "2026" cannot be misread as a unix timestamp, and the 10-digit shape stays valid until year 2286.

The helper sits in harness/deerflow/utils/ so both the harness layer (runtime/runs/manager.py) and the app layer (app/gateway/routers/threads.py) can import it without crossing the harness→app boundary that tests/test_harness_boundary.py enforces.

Changes in threads.py:

6 call sites of time.time() → now_iso(). The issue lists 5; update_thread_state at line 606 follows the same pattern, so it's included.
All read paths run through coerce_iso(): create_thread idempotent return, search_threads Phase 1 and Phase 2, patch_thread, get_thread, and the legacy synthesis branch.
_store_upsert rewrites legacy created_at to ISO on the update path, so the store converges to ISO without a migration script.
Removed the now-unused import time.

thread_runs.py swaps its private _now_iso for the shared now_iso to keep the two call sites from drifting apart later.

Modules that already emit correct ISO via inline datetime.now(UTC).isoformat() are left alone in this PR. Consolidating them into the helper is a mechanical follow-up outside the issue's scope.

Testing

$ make lint
All checks passed!
311 files already formatted

$ uv run pytest
=========== 2144 passed, 15 skipped, 4 warnings in 129.98s (0:02:09) ===========

tests/test_utils_time.py (9 cases) covers now_iso and the coerce_iso branches: ISO passthrough, unix-second string and numeric forms, None, empty string, the short-numeric guard against "2026", unparseable input, and the bool subclass-of-int trap.

tests/test_threads_router.py adds 5 cases using real langgraph.checkpoint.memory.InMemorySaver and langgraph.store.memory.InMemoryStore (the components the gateway boots in dev when no checkpointer is configured):

create_thread returns ISO created_at / updated_at.
get_thread returns ISO for a pre-seeded legacy unix-timestamp record.
patch_thread returns ISO and updated_at advances past created_at.
search_threads over a legacy + modern mix normalizes everything to ISO and sorts by real time.
_store_upsert writes ISO on create and heals legacy created_at on update.

Live HTTP verification

The integration tests already drive the real router through FastAPI's TestClient. To rule out anything that sits above the router, I also booted uvicorn app.gateway.app:app on port 8001 and ran the issue's exact repro against the live process.

$ curl -s -X POST http://127.0.0.1:8001/api/threads \
    -H "Content-Type: application/json" -d '{"metadata": {"source": "verify"}}' | jq
{
  "thread_id": "67fc738b-3fbf-4aed-b5c6-de1d0a4c0da6",
  "status": "idle",
  "created_at": "2026-04-27T03:50:16.102445+00:00",
  "updated_at": "2026-04-27T03:50:16.102445+00:00",
  "metadata": {"source": "verify"},
  ...
}

PATCH advances updated_at correctly:

$ curl -s -X PATCH http://127.0.0.1:8001/api/threads/$TID \
    -H "Content-Type: application/json" -d '{"metadata": {"patched": true}}' | jq
{
  ...
  "created_at": "2026-04-27T03:50:16.102445+00:00",
  "updated_at": "2026-04-27T03:50:27.037832+00:00"
}

GET /api/threads/{id} and POST /api/threads/search return ISO in the same session.

To exercise the coerce_iso healing path, I pre-seeded the live store with created_at = "1777252410.411327" (the literal value from the issue body) plus a separate ISO record:

GET /api/threads/legacy-thread →
  "created_at": "2026-04-27T01:13:30.411327+00:00"

POST /api/threads/search →
  [0] modern-thread   updated_at=2026-04-27T03:00:00+00:00
  [1] legacy-thread   updated_at=2026-04-27T01:13:30.411327+00:00

The healed value matches the Expected format example in the issue body to the microsecond, so the unix↔ISO round-trip preserves information. The mixed-format sort now reflects real time order.

The affected endpoints only touch the store and checkpointer, so no model/LLM call was involved in any of this verification.

Compatibility

ThreadResponse.created_at and updated_at keep the str type. Only the semantic format changes, no breaking schema change. Legacy unix-string store records are normalized on read, so no data migration script is required.

…dance#2594) ThreadResponse documents created_at / updated_at as ISO timestamps, matching the LangGraph Platform schema (langgraph_sdk.schema.Thread exposes them as datetime, JSON-encoded as ISO 8601). The gateway threads router was instead emitting str(time.time()) — unix-second floats — breaking frontend new Date() parsing and producing a mixed ISO/unix wire format that also corrupted the search sort order. Centralize timestamp generation in deerflow.utils.time: - now_iso() — datetime.now(UTC).isoformat() - coerce_iso(x) — heals legacy unix-timestamp strings on read so the store converges to ISO without a one-shot migration threads.py: replace 6 time.time() call sites with now_iso(); wrap all read paths and Phase-2 checkpoint metadata with coerce_iso(); _store_upsert opportunistically heals legacy created_at on update; drop unused time import. thread_runs.py: reuse now_iso() instead of a private duplicate _now_iso(), preventing future drift between the two timestamp call sites. Tests: 9 unit tests for the helper; 5 integration tests pinning the ISO contract for create/get/patch/search and the legacy-healing path on the internal store upsert. Full suite: 2144 passed, 15 skipped, 0 failed. Closes bytedance#2594

WillemJiang · 2026-05-01T08:17:54Z

@fancyboi999 Please resolve the conflicts with the main.

Copilot

Pull request overview

Fixes the Gateway /api/threads timestamp wire format so created_at / updated_at are consistently emitted as ISO 8601 strings (matching ThreadResponse docs and LangGraph Platform expectations), while preserving backward compatibility by normalizing legacy unix-second records on read.

Changes:

Added shared timestamp helpers now_iso() and coerce_iso() in deerflow.utils.time and adopted them across threads endpoints.
Updated thread store/checkpointer read paths to normalize legacy unix-second timestamps to ISO, and updated write paths to emit ISO directly (including opportunistic healing on update).
Added focused unit + router integration tests to pin the ISO contract and mixed-format normalization behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
backend/packages/harness/deerflow/utils/time.py	Introduces `now_iso()` and `coerce_iso()` for ISO generation and legacy timestamp normalization.
backend/app/gateway/routers/threads.py	Switches thread endpoints from unix-second strings to ISO; normalizes legacy read paths and heals store records on update.
backend/packages/harness/deerflow/runtime/runs/manager.py	Reuses the shared `now_iso()` helper instead of a local implementation.
backend/tests/test_utils_time.py	Adds unit tests covering `now_iso()` and `coerce_iso()` conversion/edge cases.
backend/tests/test_threads_router.py	Adds end-to-end router tests verifying ISO output and mixed legacy/ISO normalization + ordering.

Copilot · 2026-05-01T08:22:06Z

+def coerce_iso(value: object) -> str:
+    """Best-effort coerce a stored timestamp to an ISO 8601 string.
+
+    Translates legacy unix-timestamp floats / strings written by older
+    DeerFlow versions into ISO without a one-shot migration. ISO strings
+    pass through unchanged; empty values become ``""``; unrecognised
+    values are stringified as a last resort.
+    """
+    if value is None or value == "":
+        return ""
+    if isinstance(value, bool):
+        # ``bool`` is a subclass of ``int`` — treat as garbage, not 0/1.
+        return str(value)
+    if isinstance(value, (int, float)):
+        try:
+            return datetime.fromtimestamp(float(value), UTC).isoformat()
+        except (ValueError, OverflowError, OSError):
+            return str(value)
+    if isinstance(value, str):
+        if _UNIX_TIMESTAMP_PATTERN.match(value):
+            try:
+                return datetime.fromtimestamp(float(value), UTC).isoformat()
+            except (ValueError, OverflowError, OSError):
+                return value
+        return value
+    return str(value)


coerce_iso() currently stringifies non-str/non-numeric inputs (including datetime objects). If created_at / updated_at ever come through as datetime (e.g., from LangGraph internals or in-memory stores), str(datetime) produces a space-separated format (YYYY-MM-DD HH:MM:SS+00:00) rather than ISO-8601 with T, which can break consumers expecting strict ISO. Consider handling datetime explicitly (and normalizing to UTC if tz-naive) by returning value.astimezone(UTC).isoformat() / value.replace(tzinfo=UTC).isoformat() as appropriate.

Addressed in ed9026f. coerce_iso now branches on datetime before the int/float check and routes through astimezone(UTC).isoformat() (or replace(tzinfo=UTC) when tz-naive), so the output always uses the T separator regardless of how an upstream component handed us the value.

Three new test cases cover the contract:

test_coerce_iso_handles_tz_aware_datetime — explicit assertion that T is in the output and a space is not.

test_coerce_iso_handles_tz_naive_datetime_as_utc — tz-naive input is treated as UTC.

test_coerce_iso_normalises_non_utc_datetime_to_utc — +08:00 value gets normalised so the wire format stays UTC.

Verification: uv run pytest tests/test_utils_time.py -v → 12 passed (3 new).

WillemJiang · 2026-05-01T12:15:31Z

@fancyboi999, please check out the review message and resolve the conflicts with the main branch.

Reconciles the ISO 8601 timestamp fix with main's ThreadMetaStore refactor: legacy ``_store_*`` helpers in threads.py are gone, replaced by ``MemoryThreadMetaStore`` / ``SqlThreadMetaStore`` reached via ``get_thread_store(request)``. Carries forward in the merged tree: - ``coerce_iso`` now handles ``datetime`` instances explicitly so ``str(datetime)`` (which uses a space separator) cannot leak through; tz-naive values are assumed UTC. Addresses the Copilot review note. - Router timestamp reads continue to flow through ``coerce_iso`` to heal legacy unix-second values still sitting in older stores; writes go through ``now_iso``. - ``MemoryThreadMetaStore`` now writes ``now_iso()`` and exposes ISO via ``coerce_iso`` in ``_item_to_dict`` so the in-memory backend matches the SQL backend's wire format. - New tests pin the datetime branch of ``coerce_iso`` and re-pin the end-to-end ISO contract on the new ThreadMetaStore-backed router.

fancyboi999 · 2026-05-02T04:29:14Z

@WillemJiang Thanks for the ping. Both items are addressed in ed9026f:

1. Copilot review (coerce_iso + datetime inputs)

coerce_iso now has an explicit datetime branch ahead of the int/float check and emits value.astimezone(UTC).isoformat() (or value.replace(tzinfo=UTC).isoformat() for tz-naive inputs), so the output always uses the T separator instead of falling through to str(datetime) (which would space-separate). 3 new cases in test_utils_time.py pin this (tz-aware, tz-naive, non-UTC).

2. Conflicts with main

main refactored threads.py away from the inline LangGraph BaseStore helpers (_store_get/_store_put/_store_upsert) into the new ThreadMetaStore abstraction (MemoryThreadMetaStore + SqlThreadMetaStore, reached via get_thread_store(request)). I rebased the ISO contract onto that new architecture:

app/gateway/routers/threads.py: drops the now-redundant _store_* helpers; reads still flow through coerce_iso for defense-in-depth on legacy unix-second values, writes use now_iso.
packages/harness/deerflow/persistence/thread_meta/memory.py: MemoryThreadMetaStore now writes now_iso() (was time.time()) and exposes ISO via coerce_iso in _item_to_dict, so the in-memory backend matches the SQL backend's wire format.
packages/harness/deerflow/runtime/runs/manager.py: clean import resolution, kept from deerflow.utils.time import now_iso as _now_iso over main's inline datetime.now(UTC).isoformat() to keep the helper as the single source of truth.
tests/test_threads_router.py: re-pinned the end-to-end ISO contract on the new ThreadMetaStore-backed router (pre-seeded legacy time.time() floats heal to ISO on read; mixed legacy + ISO records normalise correctly through /search).

Verification

$ uv run ruff check .
All checks passed!

$ PYTHONPATH=. uv run pytest --tb=short -q
2904 passed, 14 skipped in 147.03s

(The 19 errors in the full run are all in tests/test_client_live.py and require OPENAI_API_KEY to be set — pre-existing, unrelated to this change.)

After the merge with main, three additional read paths in ``threads.py`` were still emitting raw ``str(metadata.get("created_at", ""))`` — ``get_thread_state``, ``update_thread_state``, and ``get_thread_history``. Same root cause as bytedance#2594: when the checkpoint metadata's ``created_at`` is a unix-second float (legacy data, or a checkpoint written by an older Gateway version), ``str(float)`` produces ``"1777252410.411327"`` and the frontend's ``new Date(...)`` returns ``Invalid Date``. The fix on the ``/threads/{id}`` GET path was already in place; these three sibling endpoints needed the same treatment. All four call sites now flow through ``coerce_iso``, so: - legacy float metadata heals to ISO on the way out, - ISO metadata passes through unchanged, - ``datetime`` instances (which the new ``coerce_iso`` branch handles explicitly) emit with the ``T`` separator instead of falling through to the space-separated ``str(datetime)`` form. Coverage added for the two endpoints not already pinned by the merge: - ``test_get_thread_state_returns_iso_for_legacy_checkpoint_metadata`` - ``test_get_thread_history_returns_iso_for_legacy_checkpoint_metadata`` Both pre-seed a checkpoint whose metadata carries the literal float from the issue body and assert the wire format is ISO.

fancyboi999 · 2026-05-02T04:37:01Z

Follow-up: while scanning the merged tree I found three sibling endpoints that were still emitting raw str(metadata.get("created_at", "")) — same root cause as the original issue, just on the checkpoint-metadata side instead of the thread-record side. Pushed bd5c6456 to fix them in the same PR.

Affected endpoints:

GET /api/threads/{id}/state — created_at and checkpoint.ts
POST /api/threads/{id}/state — created_at
POST /api/threads/{id}/history — every entry's created_at

When the checkpoint metadata's created_at is a unix-second float (legacy data, or a checkpoint written by an older Gateway), str(float) produces "1777252410.411327" and the frontend's new Date(...) returns Invalid Date. Same prod symptom as #2594, just on these three routes.

All four call sites now flow through coerce_iso, so:

legacy float metadata heals to ISO on the way out,
ISO metadata passes through unchanged,
datetime instances (the new branch added in ed9026ff) emit with the T separator.

Coverage added for the two endpoints that weren't already pinned:

test_get_thread_state_returns_iso_for_legacy_checkpoint_metadata
test_get_thread_history_returns_iso_for_legacy_checkpoint_metadata

Both pre-seed a checkpoint whose metadata carries the literal float from the issue body (1777252410.411327) and assert the wire format is ISO.

Verification:

$ uv run ruff check .
All checks passed!

$ PYTHONPATH=. uv run pytest tests/test_threads_router.py tests/test_utils_time.py tests/test_memory_thread_meta_isolation.py
38 passed in 0.92s

Rationale for bundling vs. separate PR: this is the same wire-format contract as #2594 — splitting would just leave three known-broken routes in the tree until a follow-up gets prioritized. Happy to split if you'd prefer to keep this PR's diff minimal.

ninesunsabiu approved these changes Apr 27, 2026

View reviewed changes

WillemJiang added the reviewing The PR is in reviewing status label May 1, 2026

WillemJiang requested a review from Copilot May 1, 2026 08:18

Copilot started reviewing on behalf of WillemJiang May 1, 2026 08:18 View session

Copilot AI reviewed May 1, 2026

View reviewed changes

WillemJiang added the question Further information is requested label May 1, 2026

WillemJiang added this to the 2.0-m1 milestone May 2, 2026

WillemJiang approved these changes May 2, 2026

View reviewed changes

WillemJiang merged commit ca3332f into bytedance:main May 2, 2026
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(gateway): return ISO 8601 timestamps from threads endpoints#2599

fix(gateway): return ISO 8601 timestamps from threads endpoints#2599
WillemJiang merged 3 commits intobytedance:mainfrom
fancyboi999:fix/2594-thread-iso-timestamps

fancyboi999 commented Apr 27, 2026 •

edited

Loading

Uh oh!

WillemJiang commented May 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 1, 2026

Uh oh!

fancyboi999 May 2, 2026

Uh oh!

WillemJiang commented May 1, 2026

Uh oh!

fancyboi999 commented May 2, 2026

Uh oh!

fancyboi999 commented May 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

fancyboi999 commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Testing

Live HTTP verification

Compatibility

Uh oh!

WillemJiang commented May 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI May 1, 2026

Choose a reason for hiding this comment

Uh oh!

fancyboi999 May 2, 2026

Choose a reason for hiding this comment

Uh oh!

WillemJiang commented May 1, 2026

Uh oh!

fancyboi999 commented May 2, 2026

Uh oh!

fancyboi999 commented May 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fancyboi999 commented Apr 27, 2026 •

edited

Loading