Add Compatibility Report documentation (#974)

Author: avelanarius

README.md

@@ -18,7 +18,7 @@
 <a href="https://docs.quesma.com/eap-docs">Docs</a>
 </p>
 
-Quesma is an actively developed database gateway currently in pre-alpha Early Access. Route your queries through a translation layer that seamlessly connects your apps with the modern database platforms of your choice. No more query refactoring during migrations.
+Quesma is an actively developed database gateway. Route your queries through a translation layer that seamlessly connects your apps with the modern database platforms of your choice. No more query refactoring during migrations.
 
 How? Isn't migrating to a new database hard? Quesma does it without touching your application/client stack - keeping backward compatibility and moving safely, step-by-step. We translate your queries so you don’t have to rewrite your database query code during migrations.
 

docs/public/docs/.vitepress/config.mts

@@ -3,7 +3,7 @@ import { withMermaid } from "vitepress-plugin-mermaid";
 
 // https://vitepress.dev/reference/site-config
 export default defineConfig({
-  title: "Quesma EAP",
+  title: "Quesma",
   description: "Quesma Database Gateway Early Access Program",
   head: [['link', { rel: 'icon', href: 'favicon.ico' }]],
   themeConfig: {
@@ -12,7 +12,7 @@ export default defineConfig({
       light: '/logo/quesma-logo-black-full-svg.svg',
       dark: '/logo/quesma-logo-white-full-svg.svg'
     },
-    siteTitle: 'EAP',
+    siteTitle: 'Docs',
     nav: [
       { text: 'Home', link: '/' },
       { text: 'Getting started', link: '/eap-docs' },
@@ -47,6 +47,7 @@ export default defineConfig({
             items: [
               { text: 'Configuration primer', link: '/config-primer'},
               { text: 'Ingest', link: '/ingest' },
+              { text: 'Compatiblity Report', link: '/compatibility-report' },
             ],
           },
           { text: 'Known limitations or unsupported functionalities', link: '/limitations' },

docs/public/docs/compatibility-report.md

@@ -0,0 +1,61 @@
+# Compatibility Report
+
+Quesma can help validate your migration by sending queries to both Elasticsearch and ClickHouse simultaneously and comparing the results. This allows you to verify that queries return equivalent results from both systems before fully switching over.
+
+You can enable Compatibility Report on an individual index level or for all (unconfigured) indexes via `*`. The configuration supports specifying which source (Elasticsearch or ClickHouse) should be the primary source to return results from and compare to.
+
+Apart from validating correctness, Compatibility Report also measures the execution time difference (speedups or slowdowns) and generates a report per each dashboard and index.
+
+## Configuration
+
+Any index that has two targets (one Elastic type and one ClickHouse/Hydrolix type) will have Compatibility Report enabled:
+
+```yaml
+processors:
+  - name: my-query-processor
+    type: quesma-v1-processor-query
+    config:
+      indexes:
+        kibana_sample_data_ecommerce:
+          target: [ backend-elastic, backend-clickhouse ]
+        ab_testing_logs:
+          target: [ backend-clickhouse ]
+  - name: my-ingest-processor
+    type: quesma-v1-processor-ingest
+    config:
+      indexes:
+        ab_testing_logs:
+          target: [ backend-clickhouse ]
+```
+
+This configuration turns on Compatibility Report for `kibana_sample_data_ecommerce` index. The `ab_testing_logs` is an internal Quesma index which is required in the configuration for Compatibility Report to work properly.
+
+The order of targets matters in the configuration - the first target will be the primary target that Kibana (or other applications) receives results from. In the example above, Quesma will:
+
+1. Receive an incoming query from Kibana/OpenSearch Dashboards
+2. Send the same query to both Elasticsearch and ClickHouse 
+3. Compare the responses to ensure they match
+4. Return the response from the Elastic to the client
+5. Log any discrepancies for analysis
+
+## Accessing Compatibility Report
+
+In the Quesma managment UI (default port `9999`) the "CR" tab shows a compatibility report based on the collected data:
+
+![Kibana dashboards compatibility report](./public/quesma-cr/cr-1.png)
+
+Upon clicking on the "Details" link, you can see a more detailed information about the discovered mismatch between sources.
+
+![Compatibility Report - Panel Details](./public/quesma-cr/cr-2.png)
+
+### Analyzing Compatibility Report
+
+The Compatibility Report helps identify potential issues with performance and correctness between data sources.
+
+The "performance gain" column shows the relative difference in query execution time between the two data sources. A positive percentage indicates that the second source (e.g. ClickHouse) is faster than the primary source (e.g. Elasticsearch), while a negative percentage means it's slower. We recommend starting your analysis by focusing on the dashboard panels most important for you. 
+
+If you notice slower performance for certain panels, you can analyze the specific queries by clicking "Details". Based on that information, consider manually optimizing the ClickHouse schema accordingly - for example by adjusting the table's `ORDER BY` clause or converting string columns to `LowCardinality(String)` type to improve query efficiency.
+
+The "response similarity" column indicates whether there are any discrepancies between query results from different sources. Similar to performance analysis, we recommend prioritizing the validation of your most critical dashboard panels.
+
+A response similarity value below 100.0% indicates that the sources returned different results. Before investigating these differences in detail, first verify that both sources contain identical data - ensure your ingest process is properly dual-writing and that neither source contains old records. Once data consistency is confirmed, examine the specific differences in the "Details" tab. While minor cosmetic variations may not impact Kibana's visualization, if you discover what appears to be a meaningful difference between sources, please report it by either opening an issue on [Quesma's GitHub repository](https://github.com/QuesmaOrg/quesma/issues/new) or contacting us directly at [email protected].

docs/public/docs/config-primer.md

@@ -192,6 +192,7 @@ The configuration for an index consists of the following configuration options:
    will dual write ingest requests to `my_index` to both ElasticSearch and ClickHouse.
    Note that ElasticSearch/OpenSearch is the only supported backend for the `*` entry.
    If no targets are provided (example: `target: []`) in the configuration of an index in the ingest processor, ingest for that index will be disabled and incoming data will be dropped.
+   For the query processor by specifing multiple targets, [Compatibility report](/compatibility-report.md) will be enabled. See [Compatibility report documentation](/compatibility-report.md) for more details.
 
    Some backend connectors have additional attributes which may be used. For example the following configuration sets `useCommonTable` for `backend-clickhouse` target:
    ```yaml

docs/public/docs/eap-docs.md

@@ -1,8 +1,8 @@
 # What is Quesma?
 
-Quesma is an actively developed database gateway currently in Early Access Preview. Route your queries through a translation layer that seamlessly connects your apps with the modern database platforms of your choice. No more query refactoring during migrations.
+Quesma is an actively developed database gateway. Route your queries through a translation layer that seamlessly connects your apps with the modern database platforms of your choice. No more query refactoring during migrations.
 
-In its current (Early Access Preview) form, Quesma is a lightweight database proxy that enables database interoperability between Elasticsearch/OpenSearch and ClickHouse/Hydrolix.
+In its current form, Quesma is a lightweight database proxy that enables database interoperability between Elasticsearch/OpenSearch and ClickHouse/Hydrolix.
 
 
 The high-level architecture is as follows:

docs/public/docs/index.md

@@ -7,7 +7,6 @@ hero:
     light: /logo/quesma-logo-black-full-svg.svg
     dark: /logo/quesma-logo-white-full-svg.svg
   name: "Quesma Documentation"
-#text: "Quesma EAP docs"
   tagline: Welcome to the Quesma Documentation! This website will help you get started.
   actions:
     - theme: brand

docs/public/docs/limitations.md

@@ -18,6 +18,10 @@ Quesma has been tested with the following software versions:
 | OpenSearch/OpenSearch Dashboards | `2.12.0`        |
 | Hydrolix                         | `v4.8.12`       |
 
+::: danger
+Quesma supports only ElasticSearch/Kibana 8.0 or above.
+:::
+
 ### ClickHouse limitations
 * When using a cluster deployment of ClickHouse, the tables automatically created by Quesma (during [Ingest](/ingest.md)) will use the `MergeTree` engine. If you wish to use the `ReplicatedMergeTree` engine instead, you will have to create the tables manually with  `ReplicatedMergeTree` engine before ingesting data to Quesma.
   * *Note: On ClickHouse Cloud, the tables automatically created by Quesma will use the `ReplicatedMergeTree` engine (ClickHouse Cloud default engine).*