Using logging_plugin in Agent Engine to Log Custom Fields

[rohan-kavari]

I am building an Agent Engine application using ADK and deploying it via Agent Engine. A simplified version of the setup looks like this:

class AgentEngineApp(AdkApp):
    def set_up(self) -> None:
        """Initialize the agent engine app with logging and telemetry."""
        vertexai.init()
        setup_telemetry()
        super().set_up()

        logging.basicConfig(level=logging.INFO)
        logging_client = google_cloud_logging.Client()
        self.logger = logging_client.logger(__name__)

        if gemini_location:
            os.environ["GOOGLE_CLOUD_LOCATION"] = gemini_location

    def register_feedback(self, feedback: dict[str, Any]) -> None:
        """Collect and log feedback."""
        feedback_obj = Feedback.model_validate(feedback)
        self.logger.log_struct(feedback_obj.model_dump(), severity="INFO")

    def register_operations(self) -> dict[str, list[str]]:
        """Registers the operations of the Agent."""
        operations = super().register_operations()
        operations[""] = operations.get("", []) + ["register_feedback"]
        return operations


gemini_location = os.environ.get("GOOGLE_CLOUD_LOCATION")
logs_bucket_name = os.environ.get("LOGS_BUCKET_NAME")

agent_engine = AgentEngineApp(
    app=adk_app,
    artifact_service_builder=lambda: GcsArtifactService(bucket_name=logs_bucket_name)
    if logs_bucket_name
    else InMemoryArtifactService(),
)

Question

I want to understand how to properly use the logging_plugin within an agent that is deployed on Agent Engine.

Specifically:

• How can I attach or configure logging_plugin for agents running inside Agent Engine?

• What is the recommended way to log custom / contextual fields, such as:

  • user_email
  • country
  • any other per-request metadata

Goal

Any guidance, examples, or best practices for doing this with logging_plugin + Agent Engine would be very helpful.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: To log custom fields like user_email in Agent Engine, the recommended approach is to store this data in the session state. You can then create a custom plugin that uses a callback (e.g., before_tool_callback) to access this data from the ToolContext and log it. For production-level structured logging, consider using the BigQueryAgentAnalyticsPlugin instead of the basic logging_plugin.

---

Hello! As an AI agent, I can provide some guidance on how to best approach logging custom fields with ADK in Agent Engine.

The standard logging_plugin is designed for basic debugging in a terminal and does not have a direct way to include rich, per-request contextual data. [4]

For your use case, the recommended pattern is to leverage ADK's session state and plugin system. Here’s how you can do it:

1. Store Per-Request Data in Session State

Your application should add the contextual information (like user_email and country) to the session state when a conversation begins. The state is designed to hold information that needs to persist across multiple turns of a conversation. [5]

2. Create a Custom Logging Plugin

You can create your own plugin to inspect the state at various points in the agent's lifecycle and log the custom data. A plugin is a class that inherits from BasePlugin and implements one or more callback methods. [1]

Here is an example of a simple plugin that logs custom fields before a tool is executed:

import logging
from google.adk.plugins import BasePlugin, ToolContext

class ContextualLoggingPlugin(BasePlugin):
    """A plugin to log contextual data from the session state."""

    def __init__(self, logger):
        self.logger = logger

    def before_tool_callback(self, tool_context: ToolContext) -> None:
        """Called before a tool is executed."""
        custom_fields = {
            "user_email": tool_context.state.get("user_email"),
            "country": tool_context.state.get("country"),
            "tool_name": tool_context.tool_name,
            "tool_parameters": tool_context.tool_parameters,
        }

        # Remove fields that are not set
        custom_fields = {k: v for k, v in custom_fields.items() if v is not None}

        if custom_fields:
            self.logger.info("Executing tool with context: %s", custom_fields)

# In your main application setup:
# logger = ... (your configured logger)
# contextual_logger_plugin = ContextualLoggingPlugin(logger)
#
# agent_engine = AgentEngineApp(
#     app=adk_app,
#     plugins=[contextual_logger_plugin],
#     ...
# )

In this example, the before_tool_callback accesses the session state through the ToolContext to retrieve and log your custom data. [3]

3. For Production: Use BigQueryAgentAnalyticsPlugin

While the custom plugin above works well, for a robust production deployment, we highly recommend using the BigQueryAgentAnalyticsPlugin. [2] This plugin provides structured logging to BigQuery, which is ideal for analytics, monitoring, and debugging. It is highly configurable and can be extended to include custom data fields in the BigQuery schema. You could adapt the logic from the ContextualLoggingPlugin to integrate with the event structure of the BigQuery plugin.

By combining session state with a custom plugin or the BigQueryAgentAnalyticsPlugin, you can achieve the detailed, contextual logging you're looking for.

I hope this helps you build your Agent Engine application!

---

[1] base_plugin.py source code: https://github.com/google/adk-python/blob/main/src/google/adk/plugins/base_plugin.py
[2] BigQuery Agent Analytics Documentation: https://google.github.io/adk-docs/observability/bigquery-agent-analytics/
[3] Custom Tools Documentation (ToolContext): https://google.github.io/adk-docs/tools-custom/
[4] logging_plugin.py source code: https://github.com/google/adk-python/blob/main/src/google/adk/plugins/logging_plugin.py
[5] Context and State Management Documentation: https://google.github.io/adk-docs/context/

[haiyuan-eng-google]

ttps://google.github.io/adk-docs/observability/bigquery-agent-analytics/ BigQuery Agent Analytics Plugin: the ADK offers a plugin to log detailed agent analytics to BigQuery. This allows you to run complex queries on your agent's behavior and performance, get out of box observability for both token usage, tool call/LLM call error rate and the latency for both tool call and LLM call immediately. The plugin just need one line code change to add plugin into your APP. With this plugin, you can monitor which tool/subagent would cost your token most in the whole agent life-cycle