chore: Upgrade Docusaurus from 3.6.3 to 3.9.2

.github/workflows/website-deploy-prod.yml

@@ -71,11 +71,14 @@ jobs:
           aws s3 sync . s3://${{ env.S3_BUCKET_NAME }} --delete
           aws s3 ls s3://${{ env.S3_BUCKET_NAME }} --recursive --human-readable --summarize
 
-      - name: ReIndex with Algolia
-        env:
-          ALGOLIA_SCRAPER_API_KEY: ${{ secrets.ALGOLIA_SCRAPER_API_KEY }}
-        run: |
-          ./website/algolia/reindex.sh
+      - name: Trigger Algolia Crawler
+        uses: algolia/algoliasearch-crawler-github-actions@v1
+        with:
+          crawler-user-id: ${{ secrets.ALGOLIA_CRAWLER_USER_ID }}
+          crawler-api-key: ${{ secrets.ALGOLIA_CRAWLER_API_KEY }}
+          algolia-app-id: ${{ env.ALGOLIA_APP_ID }}
+          algolia-api-key: ${{ secrets.ALGOLIA_API_KEY }}
+          site-url: 'https://${{ env.DEPLOYMENT_HOST }}'
 
       - name: Trigger ReIndexing of atmos.tools and docs.cloudposse.com Docs
         run: |

docs/algolia-search-indexing.md

@@ -0,0 +1,153 @@
+# Algolia Search Indexing
+
+This document describes how Algolia search indexing is configured for the atmos.tools documentation site.
+
+## Overview
+
+The atmos.tools documentation uses [Algolia DocSearch](https://docsearch.algolia.com/) for search functionality. Search indexing is managed via the **Algolia Crawler**, which is triggered automatically on every deployment.
+
+## Architecture
+
+```
+┌─────────────────────┐     ┌──────────────────────┐     ┌─────────────────┐
+│  GitHub Actions     │────▶│   Algolia Crawler    │────▶│  Algolia Index  │
+│  (website-deploy)   │     │   (cloud-hosted)     │     │  (atmos.tools)  │
+└─────────────────────┘     └──────────────────────┘     └─────────────────┘
+```
+
+### Components
+
+1. **GitHub Actions Workflow** (`.github/workflows/website-deploy-prod.yml`)
+   - Triggers the Algolia Crawler after deploying the website to S3.
+   - Uses the official `algolia/algoliasearch-crawler-github-actions` action.
+
+2. **Algolia Crawler** (dashboard.algolia.com)
+   - Cloud-hosted crawler that fetches and indexes the documentation.
+   - Uses the official Docusaurus v3 template for optimal indexing.
+   - Runs on-demand (CI-triggered) and weekly (scheduled backup).
+
+3. **Docusaurus Frontend** (`website/docusaurus.config.js`)
+   - Integrates with Algolia via the DocSearch plugin.
+   - Supports DocSearch v4 with Ask AI integration.
+
+## Configuration
+
+### GitHub Secrets Required
+
+The following secrets must be configured in the GitHub repository:
+
+| Secret | Description | Source |
+|--------|-------------|--------|
+| `ALGOLIA_CRAWLER_USER_ID` | Crawler authentication user ID | Algolia Crawler > Account Settings |
+| `ALGOLIA_CRAWLER_API_KEY` | Crawler authentication API key | Algolia Crawler > Account Settings |
+| `ALGOLIA_API_KEY` | API key for writing to index | Algolia Dashboard > API Keys |
+
+### API Key Permissions (ACL)
+
+The `ALGOLIA_API_KEY` must have the following ACL permissions:
+
+| Permission | Purpose |
+|------------|---------|
+| `addObject` | Add records to the index |
+| `editSettings` | Modify index settings |
+| `deleteIndex` | Delete/recreate the index during full reindex |
+| `browse` | Optional - needed for partial updates |
+
+To create or edit an API key with these permissions:
+
+1. Go to **Algolia Dashboard** → **Settings** → **API Keys** → **All API Keys**.
+2. Click **New API Key** (or edit an existing key).
+3. Under **ACL**, select the permissions listed above.
+4. Under **Indices**, restrict to `atmos.tools` or use `*` for all indices.
+5. Save and copy the key to GitHub secrets.
+
+### Algolia Crawler Configuration
+
+The crawler is configured in the Algolia dashboard (dashboard.algolia.com) with these settings:
+
+- **Template**: Docusaurus v3
+- **Start URL**: `https://atmos.tools/`
+- **Sitemap URL**: `https://atmos.tools/sitemap.xml`
+- **Index Name**: `atmos.tools`
+- **Schedule**: Weekly (as backup to CI-triggered crawls)
+
+### Frontend Configuration
+
+The Docusaurus frontend is configured in `website/docusaurus.config.js`:
+
+```javascript
+algolia: {
+    appId: process.env.ALGOLIA_APP_ID || '32YOERUX83',
+    apiKey: process.env.ALGOLIA_SEARCH_API_KEY || '557985309adf0e4df9dcf3cb29c61928',
+    indexName: process.env.ALGOLIA_INDEX_NAME || 'atmos.tools',
+    contextualSearch: false,
+    askAi: {
+        assistantId: process.env.ALGOLIA_ASKAI_ASSISTANT_ID || 'xzgtsIXZSf7V',
+        // ... additional Ask AI config
+    }
+}
+```
+
+## Triggering a Crawl
+
+### Automatic (CI/CD) - Primary Method
+
+Crawls are automatically triggered via the `algolia/algoliasearch-crawler-github-actions` GitHub Action on:
+- Push to `main` branch
+- Release published
+- Manual workflow dispatch
+
+This is the primary indexing method. The GitHub Action is configured in `.github/workflows/website-deploy-prod.yml`.
+
+### Manual (Dashboard)
+
+For debugging or one-off reindexing:
+
+1. Log into dashboard.algolia.com.
+2. Navigate to the Crawler section.
+3. Select the `atmos.tools` crawler.
+4. Click "Start crawl" or "Restart crawling".
+
+### Manual (API)
+
+For scripting or automation outside of GitHub Actions:
+
+```bash
+curl -X POST "https://crawler.algolia.com/api/1/crawlers/{CRAWLER_ID}/reindex" \
+  -H "Authorization: Basic $(echo -n '{USER_ID}:{API_KEY}' | base64)"
+```
+
+## Troubleshooting
+
+### Search Not Returning Results
+
+1. **Check index status**: Log into Algolia dashboard and verify the index has records.
+2. **Check crawler logs**: Review the crawler run logs for errors.
+3. **Verify sitemap**: Ensure `https://atmos.tools/sitemap.xml` is accessible and complete.
+4. **Test selectors**: Use the URL Tester in the crawler dashboard to verify content extraction.
+
+### CI Workflow Failing
+
+1. **Verify secrets**: Ensure all required GitHub secrets are configured.
+2. **Check credentials**: Verify the Crawler User ID and API Key are correct.
+3. **Review action logs**: Check the GitHub Actions logs for specific error messages.
+
+### Low Record Count
+
+If the index has significantly fewer records than expected:
+
+1. **Check sitemap**: Verify all pages are included in the sitemap.
+2. **Review crawler config**: Ensure the start URL and sitemap URL are correct.
+3. **Check for blocked pages**: Review robots.txt for any blocked paths.
+4. **Verify page linking**: Ensure all pages are linked from the sitemap or other indexed pages.
+
+## Migration History
+
+**January 2026**: Migrated from deprecated `algolia/docsearch-scraper` Docker image to the official Algolia Crawler with GitHub Actions integration. The legacy scraper was deprecated in February 2022 and was causing indexing failures.
+
+## References
+
+- [Algolia Crawler Documentation](https://www.algolia.com/doc/tools/crawler/getting-started/overview/)
+- [Algolia Crawler GitHub Action](https://github.com/algolia/algoliasearch-crawler-github-actions)
+- [DocSearch Documentation](https://docsearch.algolia.com/)
+- [Docusaurus Search Documentation](https://docusaurus.io/docs/search)

website/blog/2026-01-07-ask-ai-search.mdx

@@ -0,0 +1,57 @@
+---
+slug: ask-ai-search
+title: "Ask AI: Conversational Search for Atmos Documentation"
+authors:
+  - osterman
+tags:
+  - feature
+  - documentation
+---
+
+Finding information in documentation shouldn't require knowing the exact terminology or page structure. With **Ask AI**, you can now ask natural language questions about Atmos and get intelligent, contextual answers—powered by [Algolia DocSearch v4](https://docsearch.algolia.com/docs/v4/askai/) and ChatGPT.
+
+<!--truncate-->
+
+## How It Works
+
+When you open the search dialog (press `/` or `Cmd+K`), you can seamlessly switch between traditional keyword search and a chat-style assistant that understands your questions.
+
+**Key features:**
+
+- **Natural language queries** - Ask questions like "How do I configure AWS SSO authentication?" or "What's the difference between stacks and components?"
+- **Contextual answers** - Responses are generated from the actual documentation content, not generic AI knowledge
+- **Source citations** - Every answer includes links to the specific documentation pages it drew from
+- **Seamless experience** - Switch between keyword search and AI assistant without leaving the search dialog
+
+## How It Works
+
+Ask AI uses Algolia's integration with large language models (LLMs) to understand your questions and find relevant information from the indexed documentation. The assistant is specifically trained on Atmos documentation, so it understands:
+
+- Atmos CLI commands and their flags
+- Stack configuration patterns and YAML syntax
+- Authentication providers and identity management
+- Workflows, custom commands, and extensibility patterns
+- Design patterns and best practices
+
+## Try It Now
+
+Visit [atmos.tools](https://atmos.tools) and press `/` or `Cmd+K` to open the search dialog. Try asking questions like:
+
+- "How do I set up AWS SSO with Atmos?"
+- "What is the difference between imports and inheritance?"
+- "How do I create a custom workflow?"
+- "What YAML functions are available?"
+
+The AI assistant will provide detailed answers with direct links to the relevant documentation pages.
+
+## Technical Details
+
+This feature is enabled by upgrading to Docusaurus 3.9.2 and DocSearch v4, which adds native support for Algolia's Ask AI feature. The assistant is configured with an understanding of Atmos as a DevOps and cloud automation tool, ensuring responses are relevant and technically accurate.
+
+## Get Involved
+
+Have feedback on Ask AI? We'd love to hear how it's helping you navigate the documentation:
+
+- [Open an issue](https://github.com/cloudposse/atmos/issues) for bugs or feature requests
+- [Join our Slack](https://slack.cloudposse.com/) to discuss with the community
+- [Star the repo](https://github.com/cloudposse/atmos) if you find Atmos helpful

website/docusaurus.config.js

@@ -337,7 +337,7 @@ const config = {
         [
             'docusaurus-plugin-sentry',
             {
-              DSN: 'b022344b0e7cc96f803033fff3b377ee@o56155.ingest.us.sentry.io/4507472203087872',
+              DSN: 'https://b022344b0e7cc96f803033fff3b377ee@o56155.ingest.us.sentry.io/4507472203087872',
             },
         ],
         [
@@ -496,7 +496,15 @@ const config = {
                 appId: process.env.ALGOLIA_APP_ID || '32YOERUX83',
                 apiKey: process.env.ALGOLIA_SEARCH_API_KEY || '557985309adf0e4df9dcf3cb29c61928', // this is SEARCH ONLY API key and is not sensitive information
                 indexName: process.env.ALGOLIA_INDEX_NAME || 'atmos.tools',
-                contextualSearch: false
+                contextualSearch: false,
+                // DocSearch v4 Ask AI integration
+                // https://docsearch.algolia.com/docs/v4/askai/
+                askAi: {
+                    assistantId: process.env.ALGOLIA_ASKAI_ASSISTANT_ID || 'xzgtsIXZSf7V',
+                    appId: process.env.ALGOLIA_APP_ID || '32YOERUX83',
+                    apiKey: process.env.ALGOLIA_SEARCH_API_KEY || '557985309adf0e4df9dcf3cb29c61928',
+                    indexName: process.env.ALGOLIA_INDEX_NAME || 'atmos.tools',
+                }
             },
             zoom: {
                 selector: '.markdown :not(em) > img',

website/package.json

@@ -17,13 +17,14 @@
     "build:site": "npm run build"
   },
   "dependencies": {
-    "@docusaurus/core": "^3.6.3",
-    "@docusaurus/plugin-client-redirects": "^3.6.3",
-    "@docusaurus/plugin-content-blog": "^3.6.3",
-    "@docusaurus/plugin-google-tag-manager": "^3.6.3",
-    "@docusaurus/preset-classic": "^3.6.3",
-    "@docusaurus/theme-common": "^3.6.3",
-    "@docusaurus/theme-mermaid": "^3.6.3",
+    "@docsearch/react": "^4.2.0",
+    "@docusaurus/core": "^3.9.2",
+    "@docusaurus/plugin-client-redirects": "^3.9.2",
+    "@docusaurus/plugin-content-blog": "^3.9.2",
+    "@docusaurus/plugin-google-tag-manager": "^3.9.2",
+    "@docusaurus/preset-classic": "^3.9.2",
+    "@docusaurus/theme-common": "^3.9.2",
+    "@docusaurus/theme-mermaid": "^3.9.2",
     "@excalidraw/excalidraw": "^0.17.6",
     "@fortawesome/fontawesome-svg-core": "^6.7.2",
     "@fortawesome/free-brands-svg-icons": "^6.7.2",
@@ -58,7 +59,7 @@
     "unified": "^11.0.5"
   },
   "devDependencies": {
-    "@docusaurus/module-type-aliases": "^3.6.3",
+    "@docusaurus/module-type-aliases": "^3.9.2",
     "prettier": "^3.4.2"
   },
   "browserslist": {