Ai agent#3
Open
yunks128 wants to merge 31 commits into
Open
Conversation
…end) Incorporates two AI-agent plugin repos into the MMGIS-Plugins structure: - tools/AgentChat: natural-language chat panel that drives map operations via the Agent backend REST API - tools/Analysis: updated analysis tool (ECharts + D3) extended for AI-agent workflows - tools/Animation: animated map sequence export tool (GIF/MP4) - backend/Agent: LLM backend supporting Azure AI Foundry and Gemini, with a generic tool registry and REST routes at /api/agent Each plugin includes a plugin.json conforming to MMGIS-Plugins conventions. Source repos: yunks128/MMGIS-Plugin-Tools-AI-Agent and yunks128/MMGIS-Plugin-Backend-AI-Agent.
- Rename backend/Agent/setup.js → plugin.js (required lifecycle entry point) - Move AgentChat/__tests__/ → tests/ and rename *.test.js → *.spec.js - Add tests/ placeholder dirs to Analysis, Animation, and backend/Agent - Add pluginDependencies: ["backend/Agent"] to AgentChat plugin.json - Add routes, envs, and priority fields to backend/Agent/plugin.json
…rsion The version copied from MMGIS-Plugin-Tools-AI-Agent had regressions: - Replaced @basics webpack aliases with broken relative paths - Hardcoded a mission-specific apiBaseUrl - Removed mmgisToolHeader/mmgisToolTitle standard CSS classes - Stripped proper L_.getToolVars() initialization logic
No changes were made to the Analysis tool itself, so the plugin.json id, version, author and repository should remain as upstream.
Frontend tools (AgentChat, Animation):
- Replace ../../Basics/... imports with @basics/... webpack aliases
- Replace ../../Tools/... imports with @essence/Tools/... webpack aliases
These relative paths only work when files live inside src/essence/; the
aliases are resolved by MMGIS webpack config regardless of install location.
Backend Agent:
- Replace require('../../../API/logger') with process.cwd()-relative path,
with console.info fallback when logger is unavailable
- Replace require('../../../Backend/Config/models/config') and
require('../../../websocket') with process.cwd()-relative paths
These deep relative paths only work when the plugin sits inside API/;
process.cwd() resolves to the MMGIS root regardless of plugin install path.
Backend models (agentConversation.js, agentTool.js): - Replace ../../../connection with process.cwd()-relative API/connection (../../../connection only worked when plugin lived inside API/) Backend routes (agent.js): - Correct earlier fix: API/Backend/Config not Backend/Config - Correct earlier fix: API/websocket not websocket (process.cwd() = MMGIS root, all internals live under API/) AgentConversation model: - Remove hardcoded defaultValue: 'frozon' for missionName field Test: - Fix rendererSelection.spec.js import: ../../AgentChat/renderers -> ../renderers (file lives in tests/, so ../renderers is the correct relative path)
…hardening New tools: - open_animation_tool: LLM-callable tool to open Animation Tool pre-configured with layer, time range, region, and format (gif/mp4) - run_analysis: LLM-callable tool to open Analysis Tool with chart type (timeseries/histogram/scatterplot), date range, and region - Both registered in tool-registry.json with uiProfiles and renderers in renderers.js (render_open_animation_tool, render_run_analysis) Mission-agnostic hardening: - plugin.js: remove frozon hardcodes; graceful degradation when no mission config available; empty cache entry instead of crash - provider.js: remove SWOT/frozon-specific examples; add result summarization guideline (2-3 sentence plain-language reply after analysis/animation tools) - AgentChatTool.js: replace SWOT/ICESat-2 hardcoded suggestions with generic visible-layer logic; refactor hideConflictingLayersForTarget to be driven by window.mmgisAgentLayerConflicts (no-op by default, mission-injectable) - renderers.js: AREA_PRESETS configurable via window.mmgisAgentAreaPresets proxy; getAnalyticsBaseUrl() removes FROZON_ANALYTICS_BASE_URL fallback; simulated stats comment de-branded - tool schemas: set additionalProperties:true on parameters so LLM synonym args (time_start/time_end, time_range, etc.) pass validation
- Fix DEMO_QUERIES_CONFIG_PATH depth (was resolving one level too high) - Use process.cwd() for REPO_ROOT to match Missions-lookup convention used elsewhere in the plugin - Remove duplicate valid_count key that silently discarded the intended value - Declare node-fetch as an npm dependency (used by regionResolver.js but undeclared) - Align plugin.json's documented Azure env vars with what azureService.js actually reads
- Delete backend/Agent/tools/*.json: unused duplicates of data already embedded in tool-registry.json (nothing in the codebase reads this directory) - Delete generate_threshold_contours.py: empty stub, never referenced - Delete tools/AgentChat/tests/README.md: documented a browser-console test runner (run-copilot-tests.js, copilot-capability-test.js) that was never committed - Remove unused threadReused variable in azureService.js - Strip leftover narration console.log calls in rasterDifference.js and AnimationTool.js (kept console.error calls)
…commit The prior cleanup commit's git add included a stale pathspec that made the whole add fail silently, so these edits (unused threadReused variable, leftover narration console.log calls) never actually landed despite being reported as committed.
crossSectionAnalysis.js fabricated profile values with Math.random() instead of sampling real raster data, so it wasn't ready to ship. Drop its wiring from renderers.js (import, render_cross_section, RENDERERS map entry) and disable the cross_section tool definition in tool-registry.json so the agent doesn't invoke a feature with no working renderer.
MMGIS's real test runner is Playwright (npm run test:unit), which globs
plugins/**/tests/*.spec.js and requires `import { test, expect } from
'@playwright/test'` in every spec file - confirmed against every existing
spec file in the MMGIS host repo. layerResolver.spec.js, timeUtils.spec.js,
and rendererSelection.spec.js used bare Jest-style describe/test/expect
globals with no import, which throw ReferenceError under Playwright.
agentChat.spec.js and agentCapability.spec.js were not tests at all - they
were manual browser-console QA scripts (assigning to window.* at module
scope), which crash Playwright's Node-side test collection entirely since
window is undefined outside a browser page context. Removed.
layerResolver.spec.js, rendererSelection.spec.js, and timeUtils.spec.js
used bare Jest-style describe/test/expect globals with no import. MMGIS's
real runner (Playwright, via plugins/**/tests/*.spec.js) provides no such
globals - every real spec in the host imports { test, expect } from
'@playwright/test'. Add that import and use test.describe(...) so these
specs actually execute instead of throwing ReferenceError.
…Dependencies pluginDependencies: ["backend/Agent"] can never resolve - MMGIS's real dependency matcher (API/pluginDiscovery.js) checks the full <container>/<type>/<name> ID (3 segments), and container is derived at install time from the git org/repo, so a 2-segment value never matches regardless of which URL installs this repo. No single hardcoded value is correct across both the canonical repo and forks, so remove it rather than ship a permanently-false "dependency not enabled" startup warning. The dependency is already documented via the Agent API URL config field.
The table only listed Analysis/Chemistry/Isochrone/Segment, missing the three plugins this branch adds.
…uite Confirmed by actually running `npm run test:unit` from a live MMGIS install: this spec imports RENDERERS from renderers.js, which pulls in @basics/Layers_/Layers_ -> Map_ -> TimeControl -> LayerConstructors.js, and the last of those reads `window.L` at module scope. Playwright loads every matched spec file in Node during test discovery, before any test body runs, so this throws "window is not defined" and aborts the entire test:unit run for the whole MMGIS host, not just this plugin. Adding the @playwright/test import (prior commit) fixed the syntax but not this — the crash happens on import, before any test code executes. There's no lightweight fix: renderers.js can't be imported outside a real browser. The coverage this test provided (checking ~10 exported functions are typeof 'function') is weak enough that a webpack build failure would also catch a real regression here, so removing it is a better trade than leaving a test that breaks the host's shared CI.
Surfaced by actually running timeUtils.spec.js for the first time (it
was previously silently broken - see earlier commit). Two real bugs:
1. fuzzyMatchMonth's 0.7 acceptance threshold rejected a legitimate
single-word typo ("jnaury" -> "january" scores 0.571), so typo
tolerance never actually worked. Lowering the threshold alone isn't
safe though - it also admits unrelated month confusion (e.g.
"september" vs "december" scores ~0.67 on edit distance alone).
Fix: restrict fuzzy candidates to the same leading letter (real
typos preserve the first character; cross-month confusion mostly
doesn't) and lower the threshold to 0.55, which sits above the
worst same-letter false positive (june/july ~0.5) and below the
real typo score.
2. The fallback precision-detection regex /[T\s]\d{2}/ matched the
leading two digits of any bare 4-digit year (e.g. " 2023"), so any
snippet reaching this fallback got mislabeled 'hour' regardless of
its actual content. Added a (?!\d) guard so it only matches an
actual 2-digit token, not the prefix of a longer number.
3. extractDateSnippet had no pattern for "YYYY MonthName Dth" (year
first), so a query like "...on 2024 March 15th..." extracted the
wrong/incomplete snippet ("on 2024") instead of the full date.
Added the missing extraction pattern (checked ahead of the shorter
word+year patterns) and a matching parse rule in extractTimeParts.
- getLayerSearchRoots now falls back to sibling Missions/* directories when the requested mission's own folder doesn't exist, since a mission's config can reference STAC collections whose raw rasters physically live under a differently-named mission (e.g. "frozon_ai_forecast" sharing data with "frozon"). - The layer-difference tool now sends the frontend's active mission (L_.mission) to /analytics/difference, so the backend resolves layers against the correct mission config instead of silently falling back to the server default, which lacked the SFNO layers.
tariqksoliman
requested changes
Jul 10, 2026
Harden the AI Agent plugin per PR review feedback: - Scrub all server internals from client responses: generic messages for 5xx (sendError helper), no stack traces, no absolute paths (drop layer_path from statistics, return only /Missions URL from resolve-cog, drop path_a/path_b from difference, generic provider registry errors). - Never walk the Missions directory: scope filesystem lookups to the single requesting mission, drop sibling/root scans, replace the config-dir scan with one existsSync, and hard-bound searchRasterByName (depth + budget). - Remove arbitrary-code-execution vector in /analytics/difference by moving the inline Python to calculate_raster_difference.py and passing all inputs as argv. - Validate client path segments (isSafePathSegment) to prevent traversal in STAC/COG directory resolution; cap free-text params (readLayerNameParam) to stop levenshtein DoS. - Apply computeLimiter rate limiting to the compute-heavy routes. - Require mission from the request (no FROZON_DEFAULT_MISSION env / hardcoded default); thread the active mission through AgentChat analytics calls. - Remove unused, unauthenticated /tools/register|PUT|DELETE endpoints. - Escape agent-supplied layer names before innerHTML (XSS) in the animation control panel.
…tput - geminiService: send GEMINI_API_KEY in the x-goog-api-key header instead of the URL query string so it can't leak into logs/proxies/referrers. - calculate_raster_stats.py: stop emitting the absolute raster path in the JSON output.
Drop the Animation tool plugin per review feedback — it will be proposed separately. AgentChat's open_animation_tool renderer already no-ops when the AnimationTool module isn't present, so nothing breaks in its absence.
Remove the deployment-specific `window.frozonAnalyticsBase` and `FROZON_ANALYTICS_BASE_URL` override hooks; the generic `mmgisglobal.ANALYTICS_BASE_URL` override already covers this.
Author
|
tariqksoliman
requested changes
Jul 21, 2026
| "@azure/identity": "^4.4.0", | ||
| "@terraformer/wkt": "^2.2.0", | ||
| "ajv": "^8.17.1", | ||
| "node-fetch": "^2.7.0" |
Member
There was a problem hiding this comment.
I'd drop node-fetch as a dependency because node now ships with it by default.
From Devin:
backend/Agent/regionResolver.js is the only place that requires it (const fetch = require("node-fetch")), and plugin.json declares "node-fetch": "^2.7.0".
So it is a real runtime dep as written (regionResolver would throw without it), but it isn't truly needed: global fetch is stable since Node 18 and MMGIS 5.x runs on Node ≥18, so regionResolver could use the global like geminiService already does and the dependency could be removed.
Caveat if dropped: regionResolver passes { timeout: 10000/15000 }, which is a node-fetch v2-specific option — built-in/undici fetch ignores it. They'd need to switch to AbortSignal.timeout(ms) via the signal option to preserve the timeout.
Author
There was a problem hiding this comment.
Dropped node-fetch; regionResolver.js now uses global fetch. Kept the timeouts by switching to AbortSignal.timeout() (global fetch ignores node-fetch's timeout option).
| { | ||
| "components": [ | ||
| { | ||
| "field": "separatedTool", |
Member
There was a problem hiding this comment.
Toggling between separated and unseparated tools should have no effect now. Please remove.
| "defaultChecked": true | ||
| }, | ||
| { | ||
| "field": "variables.justification", |
Member
There was a problem hiding this comment.
Separated tool justification should have no effect now. Please remove.
Author
There was a problem hiding this comment.
Removed. (Confirmed unused — the tool never read justification.)
| "hasVars": true, | ||
| "name": "AgentChat", | ||
| "toolbarPriority": 3, | ||
| "separatedTool": true, |
Member
There was a problem hiding this comment.
Can you set "separatedTool": "custom"
I haven't built this into core yet but am working on it right now. Basically when "custom" it won't try to render a a little panel for the tool to fit in -- you won't see that empty "AGENTCHAT" panel in the top-left.
Author
There was a problem hiding this comment.
Set to "custom". Note it's set ahead of the core feature landing, so untested against real core behavior for now.
| @@ -0,0 +1,12 @@ | |||
| { | |||
| "queries": [ | |||
Member
There was a problem hiding this comment.
Not a required change, but it would be cool if, instead of hardcoding these demo queries, site admins can configure them in the tool in the /configure page for each mission.
Author
There was a problem hiding this comment.
Good idea — tracking as a follow-up (per-mission demo queries in /configure), leaving hardcoded for this PR.
tariqksoliman
requested changes
Jul 21, 2026
| "AgentChatTool": "./AgentChatTool" | ||
| }, | ||
| "expandable": false, | ||
| "config": { |
Member
There was a problem hiding this comment.
The AgentChat is hardcoded to always be open by default. Please add an option in the configuration for whether it should be open on page load or not. The LegendTool has a sample of this is done with its "Display On Start" checkbox.
Author
There was a problem hiding this comment.
Done — added a "Display on Start" checkbox (default off), matching LegendTool. Since this is now a "custom" separated tool it self-handles the auto-open in initialize() rather than relying on core's displayOnStart loop.
Signed-off-by: Aashish Panta <aashishpanta0@gmail.com>
- AgentChat: set separatedTool to "custom" - AgentChat: add "Display on Start" config option (default off) - AgentChat: remove obsolete Separated Tool and Justification config - Agent: drop node-fetch dependency; use global fetch with AbortSignal timeouts
- Close (X) button now calls ToolController_.closeTool('AgentChat') so core
updates activeSeparatedTools / UI store / toggle event, not just DOM teardown
- Topbar launcher opens via ToolController_.openTool('AgentChat') for symmetry
- Drop redundant self-open in initialize(); core's displayOnStart loop now
auto-opens "custom" separated tools
- Both open/close paths fall back to the old make/destroy API on older core
replace deprecated ai-agent w/ ai-projects
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an AI Agent copilot suite for MMGIS: natural-language chat that can list/toggle layers, adjust opacity, zoom, run analysis, export data, and create animations — backed by Azure AI Foundry or Google Gemini.
backend/Agent— LLM provider abstraction (Azure AI Foundry / Gemini), a tool registry + REST routes at/api/agent, region resolution (MarineRegions.org), and Python/rasterio-based raster statistics.tools/AgentChat— the chat panel, natural-language layer resolution (typo-tolerant fuzzy matching), time-query parsing, and renderers that turn tool calls into map actions.tools/Animation— create animated GIF/MP4 sequences of map data over a custom time range and bounding box.All three are
tier: experimentaland requireengines.mmgis >= 5.0.0. The backend is opt-in viaWITH_AGENT=trueand needs eitherPROJECT_ENDPOINT+AZURE_AI_FOUNDRY_AGENT_ID(Azure) orGEMINI_API_KEY(Gemini).Tests
npm run plugins -- validate— all 33 plugins validnpm run test:unit— 765 passing (2 remaining failures are unrelated: a pre-existing MMGIS core test bug, and a missing Chromium binary -- unrelated local environment issue)