about workflow_node_executions

[zhouji1026]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

workflow_node_executions
May I ask, this table has a large number of entries, and the front end business query logs are very slow. This table is for executing actions, but I don't think it does much. Can I delete data that only retains half a year?

2. Additional context or comments

help me

[dosubot[bot]]

Yes, you can safely delete old data from the workflow_node_executions table. Dify actually has built-in cleanup mechanisms for this — you just need to enable them.

Option 1: Enable the built-in cleanup task

Set these environment variables to enable the scheduled daily cleanup (runs at midnight):

• ENABLE_WORKFLOW_RUN_CLEANUP_TASK=true
• SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS=180 (set to 180 for half a year)

[1]

Option 2: Enable the workflow log cleanup task

There's also a second cleanup task (runs at 2:00 AM daily):

• WORKFLOW_LOG_CLEANUP_ENABLED=true
• WORKFLOW_LOG_RETENTION_DAYS=180

[2]

Option 3: Manual database deletion

If you prefer to do it manually, you can run a SQL query like:

DELETE FROM workflow_node_executions WHERE created_at < NOW() - INTERVAL '180 days';

Be aware this could be slow on a very large table. Consider deleting in batches (e.g., 1000–5000 rows at a time) to avoid locking issues.

Note: The built-in cleanup tasks use batch processing with random sleep intervals between batches to avoid overwhelming the database [3], so enabling them is the recommended approach.

Regarding query performance: The table has indexes including one on (tenant_id, workflow_id, node_id, created_at DESC) [4] which should help with the frontend log queries. If queries are still slow after reducing data volume, check that your database has up-to-date statistics (ANALYZE workflow_node_executions for PostgreSQL).

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[dosubot[bot]]

Good questions — let me clarify both options:

Both options clean more than just workflow_node_executions. They perform cascade deletions across multiple related tables including workflow_runs, workflow_node_executions, workflow_app_log, workflow_pause, and others [1].

Option 1 (ENABLE_WORKFLOW_RUN_CLEANUP_TASK):

• Runs daily at midnight (not just once). Each day it recalculates the cutoff as now() - 180 days and deletes everything older than that [2]. So on day 181, it will delete records from day 1; on day 182, records from day 2, etc. — it continuously maintains a 180-day rolling window.
• ⚠️ Important caveat: This task is designed for sandbox/free-tier tenants only. If billing is enabled, it skips paid tenants [3]. For a self-hosted deployment without billing, it should work for all tenants, but please verify this in your environment.

Option 2 (WORKFLOW_LOG_CLEANUP_ENABLED):

• Runs daily at 2:00 AM. Same concept — each day it deletes records older than WORKFLOW_LOG_RETENTION_DAYS [4]. It maintains a continuous rolling window of 180 days if you set that value.
• This one applies to all tenants regardless of billing status, and also cleans additional tables like messages, conversations, and related message data [5].
• Supports optional filtering to specific workflows via WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS [6].

For a self-hosted deployment where you want to retain 180 days of data, Option 2 is likely the better fit since it's not restricted to sandbox tenants. Set:

WORKFLOW_LOG_CLEANUP_ENABLED=true
WORKFLOW_LOG_RETENTION_DAYS=180

Just be aware that Option 2 also deletes associated messages and conversations older than 180 days, not just workflow execution data.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[zhouji1026]

WORKFLOW_LOG_CLEANUP_ENABLED=true
WORKFLOW_LOG_RETENTION_DAYS=180
Okay, if I enable this parameter in env, do I need to reference it in the docker-compose.yaml container, or should I just enter it directly in .env and then restart all containers to take effect?

[dosubot[bot]]

These variables are already defined in the Dify Docker Compose configuration, so you don't need to add any new references to docker-compose.yaml.

Dify uses an env_file approach where environment variables are loaded from layered files [1]. The workflow cleanup variables are already present in ./envs/core-services/shared.env.example with defaults [2]:

WORKFLOW_LOG_CLEANUP_ENABLED=false
WORKFLOW_LOG_RETENTION_DAYS=30
WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100

To enable, you have two options:

1. Edit ./envs/core-services/shared.env and set:

   WORKFLOW_LOG_CLEANUP_ENABLED=true
   WORKFLOW_LOG_RETENTION_DAYS=180
   

2. Or add them to your root ./.env file (which has the highest override priority).

Then restart the containers (specifically the worker_beat service is the one that runs the scheduled cleanup task). A full docker compose down && docker compose up -d will work.

No changes to docker-compose.yaml are needed.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[zhouji1026]

workflow_node_executions
If I enable these two options and my dify version is v1.14.2, please list all my affected tables?
Also, may I ask about workflow_node_executions, workflow_runs, workflow_app_logs, workflow_pauses, etc.? Are records in these tables judged to be deleted based on created_at fields for more than 180 days?

[dosubot[bot]]

Here's the complete picture for the WORKFLOW_LOG_CLEANUP_ENABLED task:

How deletion is determined:

The task uses only the created_at field on the workflow_runs table to identify which records to delete [1]. It finds workflow runs where created_at < (today - 180 days). All related records in other tables are then deleted by foreign key relationships (e.g., matching workflow_run_id), not by their own created_at fields.

All affected tables (18 total):

Workflow tables:

1. workflow_runs (primary — filtered by its created_at)
2. workflow_node_executions (by workflow_run_id)
3. workflow_node_execution_offloads (by node_execution_id)
4. workflow_app_logs (by workflow_run_id)
5. workflow_pauses (by workflow_run_id)
6. workflow_pause_reasons (by pause_id)
7. workflow_trigger_logs (by workflow_run_id)

Message & conversation tables:
8. messages (by workflow_run_id)
9. conversations (by conversation_id from matched messages)
10. conversation_variables (by conversation_id)
11. app_annotation_hit_histories (by message_id)
12. dataset_retriever_resources (by message_id)
13. message_agent_thoughts (by message_id)
14. message_chains (by message_id)
15. message_files (by message_id)
16. message_annotations (by message_id)
17. message_feedbacks (by message_id)
18. saved_messages (by message_id)

[2] [3]

In summary: The created_at on workflow_runs is the sole time-based criterion. Everything else is cascade-deleted by relationship. So if a workflow run is older than 180 days, all its node executions, messages, conversations, and associated data get removed together regardless of those child records' own timestamps.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[omni-front]

You can implement a cleanup job to delete entries older than six months in the workflow_node_executions table.

DELETE FROM workflow_node_executions
WHERE execution_date < NOW() - INTERVAL '6 months';

This SQL command will remove entries older than six months, helping improve query performance on the table.