Merge branch 'trunk' into dependabot/npm_and_yarn/website/npm_and_yarn-b99aa93e97

Author: lukekim

.github/workflows/enforce-pulls-with-spice.yml

@@ -13,10 +13,10 @@ jobs:
       contents: read
       pull-requests: write
     steps:
-      - uses: spiceai/pulls-with-spice-action@v1.0.3
+      - uses: spiceai/pulls-with-spice-action@v1.0.6
         if: github.event_name == 'pull_request'
         with:
-          require_title_min_length: '5'
+          require_title_min_length: '10'
           required_labels_any: 'area/blog,area/docs,area/cookbook'
           banned_labels: 'invalid,wontfix,nomerge,duplicate'
           require_assignee: 'true'

website/blog/releases/v1.1.1.md

@@ -0,0 +1,164 @@
+---
+date: 2025-04-08
+title: 'Spice v1.1.1 (Apr 7, 2025)'
+type: blog
+authors: [phillipleblanc]
+tags: [release, caching, mcp, spice.js, node.js]
+---
+
+Announcing the release of Spice v1.1.1! 📊  
+
+Spice v1.1.1 introduces several key updates, including a new Component Metrics System, improved Delta Data Connector performance, improved MCP tool descriptions, and expanded runtime results caching options. This release also adds detailed MySQL connection pool metrics for better observability. Component Metrics are Prometheus-compatible and accessible via the metrics endpoint.
+
+## Highlights v1.1.1
+
+- **Component Metrics System**: A new system for monitoring components, starting with MySQL connection pool metrics. These metrics provide insights into MySQL connection performance and can be selectively enabled in the dataset configuration. Metrics are exposed in Prometheus format via the metrics endpoint.
+
+For more details, see the [Component Metrics documentation](https://spiceai.org/docs/features/observability/component_metrics).
+
+- **Results Caching Enhancements**: Added a `cache_key_type` option for runtime results caching. Options include:
+  - **`plan` (Default):** Uses the query's logical plan as the cache key. Matches semantically equivalent queries but requires query parsing.
+  - **`sql`:** Uses the raw SQL string as the cache key. Provides faster lookups but requires exact string matches. Use `sql` for predictable queries without dynamic functions like `NOW()`.
+
+Example `spicepod.yaml` configuration:
+
+```yaml
+runtime:
+  results_cache:
+    enabled: true
+    cache_max_size: 128MiB
+    cache_key_type: sql # Use SQL for the results cache key
+    item_ttl: 1s
+```
+
+For more details, see the [runtime configuration documentation](https://spiceai.org/docs/reference/spicepod/runtime.md).
+
+- **Delta Data Connector**: Improved scan performance for faster query performance.
+
+- **MCP Tools**: Improved descriptions for built-in MCP tools to improve usability.
+
+- **MySQL Component Metrics**: Added detailed metrics for monitoring MySQL connections, such as connection count and pool activity.
+
+Example `spicepod.yaml` configuration:
+
+```yaml
+datasets:
+  - from: mysql:my_table
+    name: my_dataset
+    metrics:
+      - name: connection_count
+        enabled: true
+      - name: connections_in_pool
+        enabled: true
+      - name: active_wait_requests
+        enabled: true
+    params:
+      mysql_host: localhost
+      mysql_tcp_port: 3306
+      mysql_user: root
+      mysql_pass: ${secrets:MYSQL_PASS}
+```
+
+For more details, see the [MySQL Data Connector documentation](https://spiceai.org/docs/components/data-connectors/mysql).
+
+- **spice.js SDK:** The [`spice.js` SDK](https://github.com/spiceai/spice.js/releases/v2.0.1) has been updated to v2.0.1 and includes several important security updates.
+
+## New Contributors 🎉
+
+- [@kczimm](https://github.com/kczimm) made their first contribution in [#5243](https://github.com/spiceai/spiceai/pull/5243)
+
+## Contributors
+
+- [@peasee](https://github.com/peasee)
+- [@lukekim](https://github.com/lukekim)
+- [@phillipleblanc](https://github.com/phillipleblanc)
+- [@Sevenannn](https://github.com/Sevenannn)
+- [@sgrebnov](https://github.com/sgrebnov)
+- [@ewgenius](https://github.com/ewgenius)
+- [@kczimm](https://github.com/kczimm)
+
+## Breaking Changes
+
+No breaking changes in this release.
+
+## Cookbook Updates
+
+The [Spice Cookbook](https://spiceai.org/cookbook) now includes 65 recipes to help you get started with Spice quickly and easily.
+
+## Upgrading
+
+To upgrade to v1.1.1, use one of the following methods:
+
+**CLI**:
+
+```console
+spice upgrade
+```
+
+**Homebrew**:
+
+```console
+brew upgrade spiceai/spiceai/spice
+```
+
+**Docker**:
+
+Pull the `spiceai/spiceai:1.1.1` image:
+
+```console
+docker pull spiceai/spiceai:1.1.1
+```
+
+For available tags, see [DockerHub](https://hub.docker.com/r/spiceai/spiceai/tags).
+
+**Helm**:
+
+```console
+helm repo update
+helm upgrade spiceai spiceai/spiceai
+```
+
+## What's Changed
+
+### Dependencies
+
+- No major dependency changes.
+
+### Changelog
+
+```text
+- fix: Testoperator DuckDB, SQLite, Postgres, Spicecloud by [@peasee](https://github.com/peasee) in [#5190](https://github.com/spiceai/spiceai/pull/5190)
+- Update Helm Chart and SECURITY.md to v1.1.0 by [@lukekim](https://github.com/lukekim) in [#5223](https://github.com/spiceai/spiceai/pull/5223)
+- Update version.txt to v1.1.1-unstable by [@lukekim](https://github.com/lukekim) in [#5224](https://github.com/spiceai/spiceai/pull/5224)
+- Update Cargo.lock to v1.1.1-unstable by [@lukekim](https://github.com/lukekim) in [#5225](https://github.com/spiceai/spiceai/pull/5225)
+- Add tests for `verify_schema_source_path` in `ListingTableConnector` by [@phillipleblanc](https://github.com/phillipleblanc) in [#5221](https://github.com/spiceai/spiceai/pull/5221)
+- Reduce noise from debug logging by [@phillipleblanc](https://github.com/phillipleblanc) in [#5227](https://github.com/spiceai/spiceai/pull/5227)
+- Improve `openai_test_chat_messages` integration test reliability by [@Sevenannn](https://github.com/Sevenannn) in [#5222](https://github.com/spiceai/spiceai/pull/5222)
+- Verify the checkpoints existence before shutting down runtime in integration tests directly querying checkpoint by [@Sevenannn](https://github.com/Sevenannn) in [#5232](https://github.com/spiceai/spiceai/pull/5232)
+- Fix CORS support for json content-type api by [@sgrebnov](https://github.com/sgrebnov) in [#5241](https://github.com/spiceai/spiceai/pull/5241)
+- Fix ModelGradedScorer error: The 'metadata' parameter is only allowed when 'store' is enabled. by [@sgrebnov](https://github.com/sgrebnov) in [#5231](https://github.com/spiceai/spiceai/pull/5231)
+- fix: Use `pulls-with-spice-action` and switch to `spiceai-macos` runners by [@peasee](https://github.com/peasee) in [#5238](https://github.com/spiceai/spiceai/pull/5238)
+- Use v1.0.3 pulls with spice action by [@lukekim](https://github.com/lukekim) in [#5244](https://github.com/spiceai/spiceai/pull/5244)
+- feat: Build ODBC binaries, run testoperator on ODBC by [@peasee](https://github.com/peasee) in [#5237](https://github.com/spiceai/spiceai/pull/5237)
+- Bump timeout for several integration test runtime load_components & readiness check by [@Sevenannn](https://github.com/Sevenannn) in [#5229](https://github.com/spiceai/spiceai/pull/5229)
+- Validate port is available before binding port for docker container in integration tests by [@Sevenannn](https://github.com/Sevenannn) in [#5248](https://github.com/spiceai/spiceai/pull/5248)
+- Update datafusion-table-providers to fix the schema for PostgreSQL materialized views by [@ewgenius](https://github.com/ewgenius) in [#5259](https://github.com/spiceai/spiceai/pull/5259)
+- Verify flight server is ready for flight integration tests by [@Sevenannn](https://github.com/Sevenannn) in [#5240](https://github.com/spiceai/spiceai/pull/5240)
+- fix: Publish to MinIO inside of matrix on build_and_release by [@peasee](https://github.com/peasee) in [#5258](https://github.com/spiceai/spiceai/pull/5258)
+- fix: TPCDS on zero results benchmarks by [@peasee](https://github.com/peasee) in [#5263](https://github.com/spiceai/spiceai/pull/5263)
+- Use model as a judge scorer for Financebench by [@sgrebnov](https://github.com/sgrebnov) in [#5264](https://github.com/spiceai/spiceai/pull/5264)
+- Fix FinanceBench llm scorer secret name by [@sgrebnov](https://github.com/sgrebnov) in [#5276](https://github.com/spiceai/spiceai/pull/5276)
+- Implements support for `runtime.results_cache.cache_key_type` by [@phillipleblanc](https://github.com/phillipleblanc) in [#5265](https://github.com/spiceai/spiceai/pull/5265)
+- fix: Testoperator MS SQL, query overrides, dispatcher by [@peasee](https://github.com/peasee) in [#5279](https://github.com/spiceai/spiceai/pull/5279)
+- refactor: Delete old benchmarks by [@peasee](https://github.com/peasee) in [#5283](https://github.com/spiceai/spiceai/pull/5283)
+- Imporve embedding column parsing performance test by [@Sevenannn](https://github.com/Sevenannn) in [#5268](https://github.com/spiceai/spiceai/pull/5268)
+- Add Support for AWS Session Token in S3 Data Connector by [@kczimm](https://github.com/kczimm) in [#5243](https://github.com/spiceai/spiceai/pull/5243)
+- Implement Component Metrics system + MySQL connection pool metrics by [@phillipleblanc](https://github.com/phillipleblanc) in [#5290](https://github.com/spiceai/spiceai/pull/5290)
+- Add default descriptions to built-in MCP tools by [@lukekim](https://github.com/lukekim) in [#5293](https://github.com/spiceai/spiceai/pull/5293)
+- fix: Vector search with cased columns by [@peasee](https://github.com/peasee) in [#5295](https://github.com/spiceai/spiceai/pull/5295)
+- Run delta kernel scan in a blocking Tokio thread. by [@phillipleblanc](https://github.com/phillipleblanc) in [#5296](https://github.com/spiceai/spiceai/pull/5296)
+- Expose the `mysql_pool_min` and `mysql_pool_max` connection pool parameters by [@phillipleblanc](https://github.com/phillipleblanc) in [#5297](https://github.com/spiceai/spiceai/pull/5297)
+- use patched pdf-extract by [@kczimm](https://github.com/kczimm) in [#5270](https://github.com/spiceai/spiceai/pull/5270)
+```
+
+**Full Changelog**: [v1.1.0...v1.1.1](https://github.com/spiceai/spiceai/compare/v1.1.0...v1.1.1)

website/blog/releases/v1.1.2.md

@@ -0,0 +1,141 @@
+---
+date: 2025-04-15
+title: 'Spice v1.1.2 (Apr 14, 2025)'
+type: blog
+authors: [phillipleblanc]
+tags: [release, delta, tableau, snowflake, sql, nsql]
+---
+
+Announcing the release of Spice v1.1.2! 🔗  
+
+Spice v1.1.2 improves Delta Lake Data Connector performance, introduces new Accept headers for the `/v1/sql` and `/v1/nsql` endpoints to include query metadata with results, and resolves an issue with the Snowflake Data Connector when handling wide tables (>600 columns).
+
+The official Tableau Connector for Spice.ai v0.1 has been released, making it easy to connect to both self-hosted Spice.ai and Spice Cloud instances using Tableau.
+
+## What's New in v1.1.2
+
+- **Tableau Connector for Spice.ai**: Released the initial version (v0.1) of the official Tableau Taco Connector (fully open-source), enabling data visualization and analytics in Tableau with self-hosted Spice.ai and Spice Cloud deployments.
+
+  - Official Release: [github.com/spicehq/tableau-connector/releases/tag/v0.1.0](https://github.com/spicehq/tableau-connector/releases/tag/v0.1.0)
+  - Docs: [spiceai.org/docs/clients/tableau](https://spiceai.org/docs/clients/tableau)
+  - Open Source Repository: [github.com/spiceai/tableau-connector](https://github.com/spiceai/tableau-connector)
+
+![Query Spice.ai using Tableau](/img/tableau/tableau-spice-example.png)
+
+- **Delta Lake Data Connector**: Upgraded `delta_kernel` to v0.9, and optimized scan operations, reducing query execution time by up to 20% on large datasets.
+
+- **Snowflake Data Connector**: Fixed a bug that caused failures when loading tables with more than 600 columns.
+
+- **Query Metadata (SQL and NSQL)**: Added support for the `application/vnd.spiceai.sql.v1+json` Accept header on the `/v1/sql` endpoint, and the `application/vnd.spiceai.nsql.v1+json` Accept header on the `/v1/nsql` endpoint, enabling responses to include metadata such as the executed SQL query and schema alongside results.
+
+Example:
+
+```bash
+curl -XPOST "http://localhost:8090/v1/nsql" \
+  -H "Content-Type: application/json" \
+  -H "Accept: application/vnd.spiceai.nsql.v1+json" \
+  -d '{
+    "query": "What’s the highest tip any passenger gave?"
+  }' | jq
+```
+
+Example response:
+
+```json
+{
+  "row_count": 1,
+  "schema": {
+    "fields": [
+      {
+        "name": "highest_tip",
+        "data_type": "Float64",
+        "nullable": true,
+        "dict_id": 0,
+        "dict_is_ordered": false,
+        "metadata": {}
+      }
+    ],
+    "metadata": {}
+  },
+  "data": [
+    {
+      "highest_tip": 428.0
+    }
+  ],
+  "sql": "SELECT MAX(\"tip_amount\") AS \"highest_tip\"\nFROM \"spice\".\"public\".\"taxi_trips\""
+}
+```
+
+For details, see the [SQL Query API](https://spiceai.org/docs/api/HTTP/post-sql) and [NSQL API](https://spiceai.org/docs/api/HTTP/post-nsql) documentation.
+
+## Contributors
+
+- [@ewgenius](https://github.com/ewgenius)
+- [@peasee](https://github.com/peasee)
+- [@lukekim](https://github.com/lukekim)
+- [@phillipleblanc](https://github.com/phillipleblanc)
+- [@Sevenannn](https://github.com/Sevenannn)
+- [@sgrebnov](https://github.com/sgrebnov)
+- [@kczimm](https://github.com/kczimm)
+
+## Breaking Changes
+
+No breaking changes in this release.
+
+## Cookbook Updates
+
+No major cookbook additions.
+
+The [Spice Cookbook](https://spiceai.org/cookbook) now includes 65 recipes to help you get started with Spice quickly and easily.
+
+## Upgrading
+
+To upgrade to v1.1.2, use one of the following methods:
+
+**CLI**:
+
+```console
+spice upgrade
+```
+
+**Homebrew**:
+
+```console
+brew upgrade spiceai/spiceai/spice
+```
+
+**Docker**:
+
+Pull the `spiceai/spiceai:1.1.2` image:
+
+```console
+docker pull spiceai/spiceai:1.1.2
+```
+
+For available tags, see [DockerHub](https://hub.docker.com/r/spiceai/spiceai/tags).
+
+**Helm**:
+
+```console
+helm repo update
+helm upgrade spiceai spiceai/spiceai
+```
+
+## What's Changed
+
+### Dependencies
+
+- **[delta_kernel](https://crates.io/crates/delta_kernel/0.9.0):** updated to v0.9.0.
+
+### Changelog
+
+- Backport - Fix schema inference for Snowflake tables with large number of columns #5348 by [@ewgenius](https://github.com/ewgenius) in [#5350](https://github.com/spiceai/spiceai/pull/5350)
+- Upgrade `delta_kernel` to 0.9 (#5343) by [@phillipleblanc](https://github.com/phillipleblanc) in [#5356](https://github.com/spiceai/spiceai/pull/5356)
+- Add basic support for `application/vnd.spiceai.sql.v1+json` format (#5333) by [@sgrebnov](https://github.com/sgrebnov) in [#5333](https://github.com/spiceai/spiceai/pull/5333)
+- Convert DataFusion filters to Delta Kernel predicates by [@phillipleblanc](https://github.com/phillipleblanc) in [#5362](https://github.com/spiceai/spiceai/pull/5362)
+- revert to previous pdf-extract; remove test for encrypted pdf support by [@kczimm](https://github.com/kczimm) in [#5355](https://github.com/spiceai/spiceai/pull/5355)
+- Turn off `delta_kernel::log_segment` logging and refactor log filtering by [@phillipleblanc](https://github.com/phillipleblanc) in [#5367](https://github.com/spiceai/spiceai/pull/5367)
+- Extend `application/vnd.spiceai.sql.v1+json` with `schema` and `row_count` fields by [@sgrebnov](https://github.com/sgrebnov) in [#5365](https://github.com/spiceai/spiceai/pull/5365)
+- Make separate `vnd.spiceai.sql.v1+json` and `vnd.spiceai.nsql.v1+json` MIME types by [@sgrebnov](https://github.com/sgrebnov) in [#5382](https://github.com/spiceai/spiceai/pull/5382)
+
+**Full Changelog**: [v1.1.1...v1.1.2](https://github.com/spiceai/spiceai/compare/v1.1.1...v1.1.2)

website/blog/tags.yml

@@ -26,6 +26,10 @@ azure:
   label: 'azure'
   permalink: '/azure'
   description: 'Microsoft Azure related topics and usage'
+caching:
+  label: 'caching'
+  permalink: '/caching'
+  description: 'Results Cache related topics and usage'
 cli:
   label: 'cli'
   permalink: '/cli'
@@ -58,6 +62,10 @@ deepseek:
   label: 'deepseek'
   permalink: '/deepseek'
   description: 'Deepseek AI firm related topics and usage' 
+delta:
+  label: 'delta'
+  permalink: '/delta'
+  description: 'Delta Lake Data Connector related topics and usage'
 delta-lake:
   label: 'delta-lake'
   permalink: '/delta-lake'
@@ -122,6 +130,10 @@ machine-learning:
   label: 'machine-learning'
   permalink: '/machine-learning'
   description: 'Machine learning techniques and tools'
+mcp:
+  label: 'mcp'
+  permalink: '/mcp'
+  description: 'Model-Context-Protocol (MCP) related topics and usage'
 metal:
   label: 'metal'
   permalink: '/metal'
@@ -134,6 +146,14 @@ mysql:
   label: 'mysql'
   permalink: '/mysql'
   description: 'MySQL database topics and usage'
+node.js:
+  label: 'node.js'
+  permalink: '/node.js'
+  description: 'Node.js related topics and usage'
+nsql:
+  label: 'nsql'
+  permalink: '/nsql'
+  description: 'Natural Language SQL (NSQL) topics and usage'
 on-prem:
   label: 'on-prem'
   permalink: '/on-prem'
@@ -170,6 +190,14 @@ security:
   label: 'security'
   permalink: '/security'
   description: 'Security practices and tools'
+snowflake:
+  label: 'snowflake'
+  permalink: '/snowflake'
+  description: 'Snowflake Data Connector related topics and usage'
+spice.js:
+  label: 'spice.js'
+  permalink: '/spice.js'
+  description: 'spice.js (Node.js SDK) related topics and usage'
 spicepod:
   label: 'spicepod'
   permalink: '/spicepod'
@@ -186,3 +214,7 @@ task-history:
   label: 'task-history'
   permalink: '/task-history'
   description: 'Task history tracking and management'
+tableau:
+  label: 'tableau'
+  permalink: '/tableau'
+  description: 'Tableau Connector related topics and usage'