Fix #1069 #1073 #1078 #190 + RetryLlmService, CachingLlmService, LoggingObservabilityProvider + 58 new tests

[mrigankad]

Summary

This PR addresses several bugs reported in the issue tracker, adds three new utility integrations, improves the evaluation runner, and ships 58 new offline tests.

---

Bug Fixes

#1069 — FastAPI TypeError: multiple values for keyword argument

• servers/fastapi/app.py: copy the app_config dict, then .pop() title/description/version before spreading with **app_config so user-provided keys never collide with positional arguments.

#1073 — Gemini 3 thought_signature support

• core/tool/models.py: added thought_signature: Optional[bytes] = None to ToolCall.
• integrations/google/gemini.py: _parse_response() captures part.thought_signature; stream_request() scans all chunks (not just the last) for tool calls; _build_payload() pre-builds a thought_sig_by_call_id lookup and passes the signature through both function-call and function-response Part kwargs.

#1078 — SQL injection via prompt injection (legacy API)

• legacy/base/base.py: ask() calls is_sql_valid() before displaying or executing SQL; non-SELECT statements are blocked and the raw LLM text is printed instead.
• legacy/flask/__init__.py: /api/v0/run_sql endpoint validates SQL before running, returns a JSON error for disallowed statements.

#190 — Graceful handling when LLM returns no SQL

• Covered by the same is_sql_valid() guard in ask() — plain-text LLM responses fail validation cleanly and are printed as informational messages rather than crashing.

Pinecone embedding — _create_embedding() was producing MD5 hashes

• integrations/pinecone/agent_memory.py: replaced the MD5 placeholder with real sentence-transformers encoding (all-MiniLM-L6-v2 by default, configurable via embedding_model parameter). Similarity search now works correctly.

---

New Features

Thinking indicator

• core/agent/agent.py: the TODO stub now yields a real StatusIndicatorComponent(status="loading", message="Thinking...", pulse=True) before the first LLM call when config.include_thinking_indicators is enabled.

Evaluation runner — trajectory recording

• core/evaluation/runner.py: added _RecordingLlmMiddleware (one instance per test case, no shared state across concurrent runs). _execute_agent() now creates a per-run agent copy with a fresh MemoryConversationStore and the recorder appended to llm_middlewares, so AgentResult.tool_calls, llm_requests, and total_tokens are fully populated instead of empty.

RetryLlmService (integrations/local/retry_llm.py)

• LlmService decorator with exponential-backoff retry for transient failures (httpx timeouts, connection errors, HTTP 429/500/502/503/504).
• stream_request only retries if zero chunks have been yielded — prevents duplicate content in the caller's accumulator.
• Parameters: max_retries=3, base_delay=1.0s, max_delay=30.0s, jitter=True.

CachingLlmService (integrations/local/caching_llm.py)

• LlmService decorator with a SHA-256 keyed, insertion-order LRU in-memory cache.
• Cache key covers system_prompt, messages, tools, temperature, max_tokens — deliberately excludes user and metadata so different users asking the same question share the cached answer.
• Never caches tool-call responses (provider-specific state) or streaming responses.
• Exposes stats (hits/misses/size) and invalidate().

LoggingObservabilityProvider (integrations/local/logging_observability.py)

• ObservabilityProvider implementation that writes span lifecycle events (start, end with duration) and metric measurements to Python's standard logging module.
• Zero external dependencies. Configurable log levels for spans and metrics separately.

---

Tests Added

File	Coverage
tests/test_regression_fixes.py	14 tests — FastAPI keyword-arg fix, SQL injection guard, no-SQL handling
tests/test_openai_responses.py	16 tests — OpenAIResponsesService: tool serialization, payload building, response extraction, streaming
tests/test_local_wrappers.py	28 tests — RetryLlmService, CachingLlmService, LoggingObservabilityProvider

58 new tests total, all passing offline (no real API keys required).

---

tox.ini

• Added py311-openai-responses environment for the new OpenAIResponsesService test suite.
• Re-enabled the py311-gemini environment (was previously commented out).

---

Files Changed

Path	Change
src/vanna/servers/fastapi/app.py	Fix keyword-arg collision (#1069)
src/vanna/core/tool/models.py	Add thought_signature field (#1073)
src/vanna/integrations/google/gemini.py	Full thought_signature plumbing (#1073)
src/vanna/legacy/base/base.py	SQL validation in ask() (#1078, #190)
src/vanna/legacy/flask/__init__.py	SQL validation in /api/v0/run_sql (#1078)
src/vanna/integrations/pinecone/agent_memory.py	Real sentence-transformer embeddings
src/vanna/core/agent/agent.py	Thinking indicator
src/vanna/core/evaluation/runner.py	Trajectory recording middleware
src/vanna/integrations/local/retry_llm.py	New — RetryLlmService
src/vanna/integrations/local/caching_llm.py	New — CachingLlmService
src/vanna/integrations/local/logging_observability.py	New — LoggingObservabilityProvider
src/vanna/integrations/local/__init__.py	Export new integrations
tests/test_regression_fixes.py	New — 14 regression tests
tests/test_openai_responses.py	New — 16 OpenAI Responses tests
tests/test_local_wrappers.py	New — 28 wrapper tests
tox.ini	New test env + re-enable Gemini env