Skip to content

Artifacts not being loaded by load_artifacts when other tools are called #632

Description

@serdnam

** Please make sure you read the contribution guide and file the issues in the
right place. **
Contribution guide.

Describe the bug
The load_artifacts tool doesn't load artifacts into the LlmRequest if it's not the most recent function call/response.

To Reproduce
Use the following sample:

import {
  FunctionTool,
  LlmAgent,
  LoadArtifactsTool,
  type Skill,
  SkillToolset,
  type ToolProcessLlmRequest,
} from '@google/adk';

const ARTIFACT_NAME = 'evaluation_input.csv';
const SECRET = 'SECRET-CODE-7Q4Z';
const ARTIFACT_BODY = `case_id,verdict,code
1,pass,${SECRET}
`;

class DiagnosticLoadArtifactsTool extends LoadArtifactsTool {
  override async processLlmRequest(
    request: ToolProcessLlmRequest,
  ): Promise<void> {
    const contents = request.llmRequest.contents;

    await super.processLlmRequest(request);

    console.log({
      msg: `[processLlmRequest] artifact in request?`,
      artifactInRequest:
        contents.find((c) =>
          c.parts?.find((p) => p.text?.startsWith('Artifact ')),
        )?.parts ?? null,
    });
  }
}

const createArtifactTool = new FunctionTool({
  name: 'create_test_artifact',
  description: `Creates the evaluation input artifact. Call this first, once per
    session.`,
  execute: async (_args, toolContext) => {
    if (!toolContext) {
      return {status: 'error', error_message: 'No tool context.'};
    }
    const version = await toolContext.saveArtifact(ARTIFACT_NAME, {
      inlineData: {
        data: Buffer.from(ARTIFACT_BODY, 'utf8').toString('base64'),
        mimeType: 'application/csv',
      },
    });
    console.log(`[diag] created artifact ${ARTIFACT_NAME} (v${version})`);
    return {status: 'success', filename: ARTIFACT_NAME, version};
  },
});

const evaluationSkill: Skill = {
  frontmatter: {
    name: 'evaluation',
    description: `How to run an evaluation over the evaluation input artifact.
      Load this before reporting any result.`,
  },
  instructions: `To run an evaluation:
1. Read the \`code\` column of the first data row.
2. Report that value verbatim as the evaluation code.

Do NOT call the \`load_artifact\` tool after invoking this skill, it will have already been loaded.

Never guess the code. It exists only inside the artifact.`,
};

export const rootAgent = new LlmAgent({
  name: 'load_artifacts_with_skills',
  model: 'gemini-2.5-flash',
  description: `Reproduces load_artifacts failing to deliver an artifact when a
    skill is used in the same turn.`,
  instruction: `You run evaluations.

When the user asks you to run the evaluation:
1. Call the following in order: \`create_test_artifact\`, \`load_artifacts\`, and \`load_skill\` with the \`evaluation\` skill. 
2. Follow the skill instructions and report the evaluation code.

Report the code exactly as it appears in the artifact. If you cannot see the
artifact contents, say "ARTIFACT NOT VISIBLE" instead of guessing.`,
  tools: [
    createArtifactTool,
    new SkillToolset([evaluationSkill]),
    new DiagnosticLoadArtifactsTool(),
  ],
});

Expected behavior
Artifacts are loaded into the LlmRequest if the load_artifacts tool is called, regardless of the order of other of tool calls in the same turn.

Screenshots

The artifact is not present in the final request sent to the LLM.

{
  msg: '[processLlmRequest] artifact in request?',
  artifactInRequest: null
}

{
  msg: '[processLlmRequest] artifact in request?',
  artifactInRequest: null
}

{
  msg: '[processLlmRequest] artifact in request?',
  artifactInRequest: [
    { text: 'Artifact evaluation_input.csv is:' },
    { text: 'case_id,verdict,code\n1,pass,SECRET-CODE-7Q4Z\n' }
  ]
}

{
  msg: '[processLlmRequest] artifact in request?',
  artifactInRequest: null
}
Image

Environment:
TS version/environment: 5.9.3
ADK version: 1.5.0 (main branch)

Additional context
In our use case we have agents that can both load skills and load artifacts in a single turn, and occasionally artifacts are silently not included in the final LLM call because they do not re-call the tool after first loading a skill.

This seems to be at least in part because the load_artifacts tool only looks at the last Content element.

This other reported issue was found trying to come up with a workaround: #627

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingneeds review[Status] The PR/issue is awaiting review from the maintainer

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions