Add cluster-sidecar hybrid architecture (#1387)

* Refactor deployment architecture documentation to use MDX components for better modularity and maintainability

- Updated microservice, sharded, sidecar, tiered, cluster, hosted, and hybrid architecture documentation to import content from MDX partials.
- Removed inline content in favor of centralized MDX files for each architecture type.
- Added new hybrid architecture documentation to support sidecar caching backed by a centralized cluster.

* Update image formatting in deployment architecture documentation for consistency

Author: lukekim

website/docs/deployment/architectures/cluster.md

@@ -2,31 +2,11 @@
 title: 'Cluster-Based Deployment (Spice.ai Enterprise)'
 sidebar_label: 'Cluster'
 description: 'Deploying Spice as a cluster'
-sidebar_position: 6
+sidebar_position: 7
 pagination_prev: null
 pagination_next: null
 ---
 
-A full cluster-based deployment leveraging **Spice.ai Enterprise**, which includes advanced services and integrations for Kubernetes. This method is ideal for organizations requiring large-scale or complex deployments, including specialized clustering capabilities.
+import Content from '@site/src/partials/deployment/architectures/_cluster.mdx';
 
-<img width="740" alt="cluster" src="https://github.com/user-attachments/assets/643e0a5c-6745-40c0-8695-0955c795179b" />
-
-**Benefits**
-
-- Provides **enterprise-grade features**: advanced security, monitoring, and support.
-- Simplifies **managing multiple nodes** for high availability and large workloads.
-- Offers **direct integration** with Spice Cloud or on-prem Kubernetes clusters.
-
-**Considerations**
-
-- **Requires a commercial license** or subscription to Spice Enterprise.
-- More **complex initial setup**, typically involving specialized DevOps expertise.
-
-**Use This Approach When**
-
-- You operate at **significant scale** or have stringent availability requirements.
-- You need **enterprise-level support** and advanced monitoring, security, or compliance features.
-- Your team can manage a **robust Kubernetes environment** or you plan to integrate with the Spice Cloud at scale.
-
-**Example Use Case**  
-A large financial services firm requiring a highly available, secure environment. They run Spice.ai across multiple clusters using Spice Enterprise for advanced monitoring, role-based access control, and dedicated support.
+<Content />

website/docs/deployment/architectures/hosted.md

@@ -2,33 +2,11 @@
 title: 'Cloud Hosted'
 sidebar_label: 'Hosted'
 description: 'Deploying Spice cloud hosted in the Spice Cloud Platform'
-sidebar_position: 5
+sidebar_position: 6
 pagination_prev: null
 pagination_next: null
 ---
 
-The Spice Runtime is deployed on a fully managed service within the Spice Cloud Platform, minimizing the operational burden of managing clusters, upgrades, and infrastructure.
+import Content from '@site/src/partials/deployment/architectures/_hosted.mdx';
 
-<img width="740" alt="hosted" src="https://github.com/user-attachments/assets/a985527b-3481-40f4-a689-f784c893b314" />
-
-**Benefits**
-
-- Reduced overhead for deployment, scaling, and maintenance.
-- Access to specialized hosting features and quick setup.
-- Helps reduce operational complexity and cost.
-
-**Considerations**
-
-- Reliance on external hosting and associated terms or limits.
-- Potential compliance or data residency considerations for certain industries.
-- May introduce latency depending on the cloud provider's infrastructure.
-
-**Use This Approach When**
-
-- Limited DevOps resources are available, or focus on application logic over infrastructure is preferred.
-- A fully managed environment with minimal setup time is desired.
-- A single, managed solution is prioritized over running own clusters.
-- Minimizing operational complexity and cost is the goal.
-
-**Example Use Case**  
-A startup or team with limited DevOps support that needs a reliable, managed environment. Quick deployment and minimal in-house infrastructure responsibilities are priorities.
+<Content />

website/docs/deployment/architectures/hybrid.md

@@ -0,0 +1,12 @@
+---
+title: 'Hybrid Deployment'
+sidebar_label: 'Hybrid'
+description: 'Deploying Spice with sidecar caching backed by a centralized cluster for acceleration, distributed query, and ingestion.'
+sidebar_position: 4
+pagination_prev: null
+pagination_next: null
+---
+
+import Content from '@site/src/partials/deployment/architectures/_hybrid.mdx';
+
+<Content />

website/docs/deployment/architectures/index.md

@@ -14,6 +14,7 @@ Spice supports multiple deployment architectures:
 - [Sidecar Deployment](architectures/sidecar) - Deploy alongside applications
 - [Microservice Deployment (Single or Multiple Replicas)](architectures/microservice) - Standalone service deployment
 - [Tiered Deployment](architectures/tiered) - Edge, application, and cloud tiers
+- [Hybrid Deployment](architectures/hybrid) - Sidecar caching backed by a centralized cluster
 - [Cloud-Hosted in the Spice Cloud Platform](architectures/hosted) - Managed cloud deployment
 - [Sharded Deployment](architectures/sharded) - Horizontal data partitioning
 - [Cluster Deployment (Spice.ai Enterprise)](architectures/cluster) - Distributed cluster architecture

website/docs/deployment/architectures/microservice.md

@@ -7,29 +7,6 @@ pagination_prev: null
 pagination_next: null
 ---
 
-The Spice Runtime operates as an independent microservice. Multiple replicas may be deployed behind a load balancer to achieve high availability and handle spikes in demand.
+import Content from '@site/src/partials/deployment/architectures/_microservice.mdx';
 
-<img width="740" alt="microservice" src="https://github.com/user-attachments/assets/b46f050b-e500-4d53-b354-24f0ab30cad3" />
-
-**Benefits**
-
-- Loose coupling between the application and the Spice Runtime.
-- Independent scaling and upgrades.
-- Can serve multiple applications or services within an organization.
-- Helps achieve high availability and redundancy.
-
-**Considerations**
-
-- Additional network hop introduces latency compared to sidecar.
-- More complex infrastructure, requiring service discovery and load balancing.
-- Potentially higher cost due to additional infrastructure components.
-
-**Use This Approach When**
-
-- A loosely coupled architecture and the ability to independently scale the AI service are desired.
-- Multiple services or teams need to share the same AI engine.
-- Heavy or varying traffic is anticipated, requiring independent scaling of the Spice Runtime.
-- Resiliency and redundancy are prioritized over simplicity.
-
-**Example Use Case**  
-A large organization where multiple services (recommendations, analytics, etc.) need to share AI insights. A centralized Spice Runtime microservice cluster helps separate teams consume AI outputs without duplicating efforts.
+<Content />

website/docs/deployment/architectures/sharded.md

@@ -2,33 +2,11 @@
 title: 'Sharded'
 sidebar_label: 'Sharded'
 description: 'Deploying Spice with shards'
-sidebar_position: 4
+sidebar_position: 5
 pagination_prev: null
 pagination_next: null
 ---
 
-The Spice Runtime instances can be sharded based on specific criteria, such as by customer, state, or other logical partitions. Each shard operates independently, with a 1:N Application to Spice instances ratio.
+import Content from '@site/src/partials/deployment/architectures/_sharded.mdx';
 
-<img width="740" alt="sharded" src="https://github.com/user-attachments/assets/5730d108-6d22-4ea4-8c14-8e87ad6d0079" />
-
-**Benefits**
-
-- Helps distribute load across multiple instances, improving performance and scalability.
-- Isolates failures to specific shards, enhancing resiliency.
-- Allows tailored configurations and optimizations for different shards.
-
-**Considerations**
-
-- More complex deployment and management due to multiple instances.
-- Requires effective sharding strategy to balance load and avoid hotspots.
-- Potentially higher cost due to multiple instances.
-
-**Use This Approach When**
-
-- Distributing load across multiple instances for better performance is needed.
-- Isolating failures to specific shards to improve resiliency is desired.
-- The application can benefit from tailored configurations for different logical partitions.
-- The complexity of managing multiple instances can be handled.
-
-**Example Use Case**  
-A multi-tenant application where each customer has a dedicated Spice Runtime instance. This helps ensure that heavy usage by one customer does not impact others, and allows for customer-specific optimizations.
+<Content />

website/docs/deployment/architectures/sidecar.md

@@ -7,30 +7,6 @@ pagination_prev: null
 pagination_next: null
 ---
 
-Run the Spice Runtime in a separate container or process on the same machine as the main application. For example, in Kubernetes as a [Sidecar Container](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/). This approach minimizes communication overhead as requests to the Spice Runtime are transported over local loopback.
+import Content from '@site/src/partials/deployment/architectures/_sidecar.mdx';
 
-<img width="740" alt="sidecar" src="https://github.com/user-attachments/assets/716f7c23-1939-4947-85f5-b0ee2bbd63fc" />
-
-**Benefits**
-
-- Low-latency communication between the application and the Spice Runtime.
-- Simplified lifecycle management (same pod).
-- Isolated environment without needing a separate microservice.
-- Helps ensure resiliency and redundancy by replicating data across sidecars.
-
-**Considerations**
-
-- Each application pod includes a copy of the Spice Runtime, increasing resource usage.
-- Updating the Spice Runtime independently requires updating each pod.
-- Accelerated data is replicated to each sidecar, adding resiliency and redundancy but increasing resource usage and requests to data sources.
-- May increase overall cost due to resource duplication.
-
-**Use This Approach When**
-
-- Fast, low-latency interactions between the application and the Spice Runtime are needed (e.g., real-time decision-making).
-- Scaling needs are small or moderate, making duplication of the Spice Runtime in each pod acceptable.
-- Keeping the architecture simple without additional services or load balancers is preferred.
-- Performance and latency are prioritized over cost and complexity.
-
-**Example Use Case**  
-A real-time trading bot or a data-intensive application that relies on immediate feedback, where minimal latency is critical. Both containers in the same pod ensure very fast data exchange.
+<Content />

website/docs/deployment/architectures/tiered.md

@@ -7,29 +7,6 @@ pagination_prev: null
 pagination_next: null
 ---
 
-A hybrid approach combining sidecar deployments for performance-critical tasks and a shared microservice for batch processing or less time-sensitive workloads.
+import Content from '@site/src/partials/deployment/architectures/_tiered.mdx';
 
-<img width="740" alt="tiered" src="https://github.com/user-attachments/assets/e602bad4-bd0d-4069-bc91-5b5678a10710" />
-
-**Benefits**
-
-- Real-time responsiveness where needed (sidecar).
-- Centralized microservice handles broader or shared tasks.
-- Balances resource usage by limiting sidecar instances to high-priority operations.
-- Helps balance performance and latency with cost and complexity.
-
-**Considerations**
-
-- More complex deployment structure, mixing two patterns.
-- Must ensure consistent versioning between sidecar and microservice instances.
-- Potentially higher operational complexity and cost.
-
-**Use This Approach When**
-
-- Certain application components require ultra-low-latency responses, while others do not.
-- Centralized AI or analytics is needed, but localized real-time decision-making is also required.
-- The system can handle the operational complexity of running multiple deployment patterns.
-- Balancing performance and latency with cost and complexity is the goal.
-
-**Example Use Case**  
-A logistics application that calculates routing decisions in real time (sidecar) while a microservice component processes aggregated data for periodic analysis or re-training models.
+<Content />

website/docs/deployment/index.md

@@ -15,6 +15,7 @@ Spice supports flexible deployment options ranging from a single binary to fully
 - [Sidecar Deployment](deployment/architectures/sidecar)
 - [Microservice Deployment (Single or Multiple Replicas)](deployment/architectures/microservice)
 - [Tiered Deployment](deployment/architectures/tiered)
+- [Hybrid Deployment](deployment/architectures/hybrid)
 - [Cloud-Hosted in the Spice Cloud Platform](deployment/architectures/hosted)
 - [Sharded Deployment](deployment/architectures/sharded)
 - [Cluster Deployment (Spice.ai Enterprise)](deployment/architectures/cluster)

website/src/partials/deployment/architectures/_cluster.mdx

@@ -0,0 +1,27 @@
+A full cluster-based deployment leveraging **Spice.ai Enterprise**, which includes advanced services and integrations for Kubernetes. This method is ideal for organizations requiring large-scale or complex deployments, including specialized clustering capabilities.
+
+<img
+  width='740'
+  alt='cluster'
+  src='https://github.com/user-attachments/assets/643e0a5c-6745-40c0-8695-0955c795179b'
+/>
+
+**Benefits**
+
+- Provides **enterprise-grade features**: advanced security, monitoring, and support.
+- Simplifies **managing multiple nodes** for high availability and large workloads.
+- Offers **direct integration** with Spice Cloud or on-prem Kubernetes clusters.
+
+**Considerations**
+
+- **Requires a commercial license** or subscription to Spice Enterprise.
+- More **complex initial setup**, typically involving specialized DevOps expertise.
+
+**Use This Approach When**
+
+- You operate at **significant scale** or have stringent availability requirements.
+- You need **enterprise-level support** and advanced monitoring, security, or compliance features.
+- Your team can manage a **robust Kubernetes environment** or you plan to integrate with the Spice Cloud at scale.
+
+**Example Use Case**
+A large financial services firm requiring a highly available, secure environment. They run Spice.ai across multiple clusters using Spice Enterprise for advanced monitoring, role-based access control, and dedicated support.