feat(backend/executor): Avoid full table scan on AgentNodeExecutionInputOutput table

[majdyz]

This query is currently the most time consuming

select
  "platform"."AgentNodeExecution"."id",
  "platform"."AgentNodeExecution"."agentGraphExecutionId",
  "platform"."AgentNodeExecution"."agentNodeId",
  "platform"."AgentNodeExecution"."executionStatus"::text,
  "platform"."AgentNodeExecution"."executionData",
  "platform"."AgentNodeExecution"."addedTime",
  "platform"."AgentNodeExecution"."queuedTime",
  "platform"."AgentNodeExecution"."startedTime",
  "platform"."AgentNodeExecution"."endedTime",
  "platform"."AgentNodeExecution"."stats"
from
  "platform"."AgentNodeExecution"
where
  (
    "platform"."AgentNodeExecution"."agentNodeId" = $1
    and "platform"."AgentNodeExecution"."agentGraphExecutionId" = $2
    and "platform"."AgentNodeExecution"."executionStatus" = CAST($3::text as "platform"."AgentExecutionStatus")
    and ("platform"."AgentNodeExecution"."id") not in (
      select
        "t1"."referencedByInputExecId"
      from
        "platform"."AgentNodeExecutionInputOutput" as "t1"
      where
        (
          (not "t1"."name" <> $4)
          and "t1"."referencedByInputExecId" is not null
        )
    )
  )
order by
  "platform"."AgentNodeExecution"."addedTime" asc
limit
  $5
offset
  $6 /* traceparent='00-00000000000000000000000000000000-0000000000000000-01' */

which is likely caused by the inner select query that do full table scan most of the time

(
      select
        "t1"."referencedByInputExecId"
      from
        "platform"."AgentNodeExecutionInputOutput" as "t1"
      where
        (
          (not "t1"."name" <> $4)
          and "t1"."referencedByInputExecId" is not null
        )
    )

Changes 🏗️

The scope of the change is to avoid the full table call, by trying to load everything without filtering and filters it on the application level.

Checklist 📋

For code changes:

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • CI, manual agent run

[netlify]

✅ Deploy Preview for auto-gpt-docs-dev canceled.

Name	Link
🔨 Latest commit	cab1299
🔍 Latest deploy log	https://app.netlify.com/projects/auto-gpt-docs-dev/deploys/6834c7b93dc1430008391ded

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Memory Usage The new implementation loads all executions into memory and filters them in Python code instead of letting the database handle the filtering. This could lead to excessive memory usage if there are many executions. existing_execution = next( ( execution for execution in await AgentNodeExecution.prisma().find_many( where=existing_exec_query_filter, order={"addedTime": "asc"}, include={"Input": True}, ) if input_name not in [d.name for d in execution.Input or []] ), None, Performance Edge Case While this change avoids a full table scan on AgentNodeExecutionInputOutput, it might be less efficient if there are many AgentNodeExecution records but few inputs with the specified name. The database query optimization should be validated with real-world data volumes. existing_execution = next( ( execution for execution in await AgentNodeExecution.prisma().find_many( where=existing_exec_query_filter, order={"addedTime": "asc"}, include={"Input": True}, ) if input_name not in [d.name for d in execution.Input or []] ), None,

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	cab1299
🔍 Latest deploy log	https://app.netlify.com/projects/auto-gpt-docs/deploys/6834c7b94255670008eaa939

[deepsource-io]

Here's the code health analysis summary for commits 8e2fb2d..cab1299. View details on DeepSource ↗.

Analysis Summary

Analyzer	Status	Summary	Link
JavaScript	✅ Success		View Check ↗
Python	✅ Success	❗ 2 occurences introduced 🎯 2 occurences resolved	View Check ↗

---

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

[majdyz]

I'll hold it for now, we will fetch more data over the wire which could be concerning

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.