Add Code Interpreter AI Agent Tools with Unity Catalog Integration

[nehmetohmedb]

Summary

Add code interpreter capabilities to AI agents using Databricks' Mosaic AI Agent Framework, specifically leveraging the system.ai.python_exec Unity Catalog function.

Context

• Databricks provides built-in Unity Catalog function for Python code execution
• Current implementation only imports CodeInterpreterTool from crewai_tools
• CrewAI's CodeInterpreterTool provides safe isolation but requires Docker, which doesn't exist in our Databricks environment
• Databricks' code execution approach provides better security and isolation for our use case

Proposed Implementation

1. Create DatabricksCodeInterpreterTool that uses python_exec function
2. Extend UCClient to support Unity Catalog function execution
3. Add security controls (timeouts, resource limits, audit logging)
4. Register new tools in tool factory

Key Features

• Execute Python code via SELECT python_exec('code')
• Support for Unity Catalog function discovery and execution
• User context tracking and authorization
• Resource limits and execution timeouts
• Comprehensive error handling
• Native Databricks isolation without Docker dependency

References

• Databricks Code Interpreter Documentation
• Existing implementation: src/backend/src/engines/crewai/tools/

Acceptance Criteria

• Agents can execute Python code snippets
• Unity Catalog functions can be executed
• Security controls are in place
• Tests provide 80%+ coverage