StreamFusion/mkdocs.yml at main · datafusion-contrib/StreamFusion · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
site_name: StreamFusion
site_description: Run Apache Flink SQL faster with a native Rust + Apache Arrow/DataFusion accelerator.
site_url: https://streamfusion.tech/
repo_url: https://github.com/datafusion-contrib/StreamFusion
repo_name: datafusion-contrib/StreamFusion
edit_uri: edit/main/docs/
docs_dir: docs

theme:
  name: material
  palette:
    - media: "(prefers-color-scheme: light)"
      scheme: default
      primary: indigo
      toggle:
        icon: material/brightness-7
        name: Switch to dark mode
    - media: "(prefers-color-scheme: dark)"
      scheme: slate
      primary: indigo
      toggle:
        icon: material/brightness-4
        name: Switch to light mode
  features:
    - navigation.tabs
    - navigation.sections
    - navigation.top
    - navigation.indexes
    - content.code.copy
    - content.action.edit
    - toc.follow
    - search.suggest

markdown_extensions:
  - tables
  - admonition
  - pymdownx.details
  - pymdownx.superfences
  - pymdownx.highlight
  - pymdownx.inlinehilite
  - toc:
      permalink: true

extra:
  social:
    - icon: fontawesome/brands/github
      link: https://github.com/datafusion-contrib/StreamFusion
      name: GitHub
    - icon: fontawesome/brands/discord
      link: https://discord.gg/gCKHfb96Q
      name: Discord

nav:
  - Introduction: index.md
  - Blogs:
      - blogs/index.md
      - Streaming for the AI age: blogs/streaming-for-the-ai-age.md
  - Connectors:
      - connectors/index.md
      - Kafka: connectors/kafka.md
      - Parquet: connectors/parquet.md
      - Delta Lake: connectors/delta.md
      - Fluss: connectors/fluss.md
      - Formats:
          - JSON: connectors/formats/json.md
          - CSV: connectors/formats/csv.md
          - Avro: connectors/formats/avro.md
          - Avro Confluent: connectors/formats/avro-confluent.md
          - Protobuf: connectors/formats/protobuf.md
          - Raw: connectors/formats/raw.md
          - CDC JSON: connectors/formats/cdc-json.md
  - Operators:
      - operators/index.md
      - Calc / filter: operators/calc-filter.md
      - Union: operators/union.md
      - Expand: operators/expand.md
      - Changelog normalize: operators/changelog-normalize.md
      - Unnest / Correlate: operators/unnest-correlate.md
      - GROUP BY: operators/group-by.md
      - Window aggregate: operators/window-aggregate.md
      - OVER: operators/over.md
      - Deduplication: operators/deduplication.md
      - Joins:
          - operators/joins/index.md
          - Regular join: operators/joins/regular-join.md
          - Interval join: operators/joins/interval-join.md
          - Window join: operators/joins/window-join.md
          - Temporal table join: operators/joins/temporal-join.md
          - Lookup join: operators/joins/lookup-join.md
      - Top-N: operators/top-n.md
      - LIMIT: operators/limit.md
      - Event-time sort: operators/event-time-sort.md
      - Watermark assigner: operators/watermark-assigner.md
      - Unsupported operators: operators/unsupported.md
  - Backends:
      - backends/index.md
      - Memory: backends/memory.md
      - RocksDB: backends/rocksdb.md
      - Canonical state format: backends/canonical-state.md
  - Deployment: deployment.md
  - Releasing: releasing.md
  - Configuration: configuration.md
  - Alpha API stability: api-stability.md
  - Memory management: memory-management.md
  - Native metric parity: metrics-parity.md
  - Upstream Flink suite: upstream-flink-suite.md
  - Benchmarks: benchmarks.md
  - Optimizations:
      - optimizations/index.md
      - Foundational bets: optimizations/foundational-bets.md
      - Zero-copy JNI handover: optimizations/zero-copy-jni-handover.md
      - Columnar flow, transposes at edges: optimizations/columnar-flow-transposes-at-edges.md
      - Native columnar exchange: optimizations/native-columnar-exchange.md
      - Zero-copy local shuffle: optimizations/zero-copy-local-shuffle.md
      - Columnar changelog operators: optimizations/columnar-changelog-operators.md
      - Expression compile-once: optimizations/expression-compile-once.md
      - Calc projection before filtering: optimizations/calc-projection-before-filter.md
      - Export Calc schemas once: optimizations/calc-schema-once-c-data.md
      - Shared FFI allocator: optimizations/shared-ffi-allocator.md
      - DataFusion hash joins: optimizations/datafusion-hash-joins.md
      - Row-major transpose: optimizations/row-major-transpose.md
      - Arrow unsafe memory checks: optimizations/arrow-unsafe-memory-checks.md
      - Zero-copy exit transpose: optimizations/zero-copy-exit-transpose.md
      - String copy reduction: optimizations/string-copy-reduction.md
      - Projection pruning into the transpose: optimizations/projection-pruning-transpose.md
      - Batched native decode: optimizations/batched-native-decode.md
      - Decoder projection pushdown: optimizations/decoder-projection-pushdown.md
      - SIMD JSON decode: optimizations/simd-json-decode.md
      - Arrow-row byte state: optimizations/arrow-row-state.md
      - Borrowed key probes: optimizations/borrowed-key-probes.md
      - Mini-batch logical boundary: optimizations/mini-batch-logical-boundary.md
      - Top-N emit decodes distinct rows: optimizations/topn-emit-decodes-distinct-rows.md
      - Aggregate specialization fast paths: optimizations/aggregate-specialization-fast-paths.md
      - Allocation discipline: optimizations/allocation-discipline.md
      - Memory accounting off the hot path: optimizations/memory-accounting-off-hot-path.md
      - Checkpoint partitioning in one pass: optimizations/checkpoint-partition-in-one-pass.md
      - Asynchronous Top-N memory checkpoints: optimizations/async-topn-memory-checkpoints.md
      - Sub-plan reuse digest barriers: optimizations/subplan-reuse-digest-barriers.md
      - UDFs via columnar upcall: optimizations/udf-columnar-upcall.md
      - Host-exact builtins via upcall: optimizations/host-exact-builtins-upcall.md
      - Regex pattern cache: optimizations/regexp-pattern-cache.md
      - Lookup joins kept in the island: optimizations/lookup-join-in-island.md
      - Identity expression passthroughs: optimizations/identity-expression-passthroughs.md
      - Parquet sink file coalescing: optimizations/parquet-sink-file-coalescing.md
      - Parquet column-chunk streaming: optimizations/parquet-column-chunk-streaming.md
      - Batch-native Kafka JSON serialization: optimizations/batch-native-kafka-json-serialization.md
      - DATE_FORMAT pattern cache: optimizations/date-format-pattern-cache.md
      - Kafka timestamps as fixed JSON syntax: optimizations/kafka-timestamp-fixed-json.md
      - Post-exchange batch coalescing: optimizations/post-exchange-coalescing.md
      - Shared native sources: optimizations/shared-native-sources.md