Add Compatibility Report documentation

docs/public/docs/.vitepress/config.mts

@@ -47,6 +47,7 @@ export default defineConfig({
             items: [
               { text: 'Configuration primer', link: '/config-primer'},
               { text: 'Ingest', link: '/ingest' },
+              { text: 'A/B testing', link: '/ab-testing' },
             ],
           },
           { text: 'Known limitations or unsupported functionalities', link: '/limitations' },

docs/public/docs/ab-testing.md

@@ -0,0 +1,49 @@
+# A/B testing
+
+Quesma can help validate your migration by sending queries to both Elasticsearch and ClickHouse simultaneously and comparing the results. This allows you to verify that queries return equivalent results from both systems before fully switching over.
+
+You can enable A/B testing on an individual index level or for all (unconfigured) indexes via `*`. The configuration supports specifying which source (Elasticsearch or ClickHouse) should be the primary source to return results from and compare to.
+
+Apart from validating correctness, A/B testing also measures the execution time difference (speedups or slowdowns) and generates a report per each dashboard and index.
+
+## Configuration
+
+Enabling A/B testing is as simple as adding two targets to an index:
+
+```yaml
+processors:
+  - name: my-query-processor
+    type: quesma-v1-processor-query
+    config:
+      indexes:
+        kibana_sample_data_ecommerce:
+          target: [ backend-elastic, backend-clickhouse ]
+```
+
+The order of targets matters in the configuration - the first target will be the primary target that Kibana (or other applications) receives results from. In the example above, Quesma will:
+
+1. Receive an incoming query from Kibana/OpenSearch Dashboards
+2. Send the same query to both Elasticsearch and ClickHouse 
+3. Compare the responses to ensure they match
+4. Return the response from the Elastic to the client
+5. Log any discrepancies for analysis
+
+## A/B testing report
+
+In the Quesma managment UI (default port `9999`) the "A/B" tab shows a compatibility report based on the collected data:
+
+![Kibana dashboards compatibility report](./public/quesma-ab/ab-1.png)
+
+Upon clicking on the "Details" link, you can see a more detailed information about the discovered mismatch between sources.
+
+### Analyzing A/B testing report
+
+The compatibility report helps identify potential issues with performance and correctness between data sources.
+
+The "performance gain" column shows the relative difference in query execution time between the two data sources. A positive percentage indicates that the second source (e.g. ClickHouse) is faster than the primary source (e.g. Elasticsearch), while a negative percentage means it's slower. We recommend starting your analysis by focusing on the dashboard panels most important for you. 
+
+If you notice slower performance for certain panels, you can analyze the specific queries by clicking "Details". Based on that information, consider manually optimizing the ClickHouse schema accordingly - for example by adjusting the table's `ORDER BY` clause or converting string columns to `LowCardinality(String)` type to improve query efficiency.
+
+The "response similarity" column indicates whether there are any discrepancies between query results from different sources. Similar to performance analysis, we recommend prioritizing the validation of your most critical dashboard panels.
+
+A response similarity value below 100.0% indicates that the sources returned different results. Before investigating these differences in detail, first verify that both sources contain identical data - ensure your ingest process is properly dual-writing and that neither source contains old records. Once data consistency is confirmed, examine the specific differences in the "Details" tab. While minor cosmetic variations may not impact Kibana's visualization, if you discover what appears to be a meaningful difference between sources, please report it by either opening an issue on [Quesma's GitHub repository](https://github.com/QuesmaOrg/quesma/issues/new) or contacting us directly at [email protected].

docs/public/docs/config-primer.md

@@ -192,6 +192,7 @@ The configuration for an index consists of the following configuration options:
    will dual write ingest requests to `my_index` to both ElasticSearch and ClickHouse.
    Note that ElasticSearch/OpenSearch is the only supported backend for the `*` entry.
    If no targets are provided (example: `target: []`) in the configuration of an index in the ingest processor, ingest for that index will be disabled and incoming data will be dropped.
+   For the query processor by specifing multiple targets, [A/B testing](/ab-testing.md) will be enabled. See [A/B testing documentation](/ab-testing.md) for more details.
 
    Some backend connectors have additional attributes which may be used. For example the following configuration sets `useCommonTable` for `backend-clickhouse` target:
    ```yaml

docs/public/docs/limitations.md

@@ -18,6 +18,10 @@ Quesma has been tested with the following software versions:
 | OpenSearch/OpenSearch Dashboards | `2.12.0`        |
 | Hydrolix                         | `v4.8.12`       |
 
+::: danger
+Quesma does not support ElasticSearch/Kibana 7.x versions.
+:::
+
 ### ClickHouse limitations
 * When using a cluster deployment of ClickHouse, the tables automatically created by Quesma (during [Ingest](/ingest.md)) will use the `MergeTree` engine. If you wish to use the `ReplicatedMergeTree` engine instead, you will have to create the tables manually with  `ReplicatedMergeTree` engine before ingesting data to Quesma.
   * *Note: On ClickHouse Cloud, the tables automatically created by Quesma will use the `ReplicatedMergeTree` engine (ClickHouse Cloud default engine).*