Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
ae311e0
[DOCS]: Add Switch transpiler documentation
hiroyukinakazato-db Sep 24, 2025
47b8b5c
Merge branch 'main' into switch-documentation
hiroyukinakazato-db Oct 18, 2025
53f56d2
docs: update Switch documentation for new llm-transpile command
hiroyukinakazato-db Oct 18, 2025
ef4d162
docs: update Switch installation description in pluggable transpilers…
hiroyukinakazato-db Oct 18, 2025
eee5f3a
docs: refine requirements and installation instructions for Switch
hiroyukinakazato-db Oct 27, 2025
756d68f
docs: update requirements section for Switch installation
hiroyukinakazato-db Oct 27, 2025
7319971
docs: update permissions and configuration details in Switch document…
hiroyukinakazato-db Oct 27, 2025
cc521d9
docs: refine token limits and threshold configuration details in Swit…
hiroyukinakazato-db Oct 27, 2025
69ab45b
docs: update model options section for clarity and formatting
hiroyukinakazato-db Oct 27, 2025
cd2a903
docs: clarify model flexibility in transpilation overview
hiroyukinakazato-db Oct 27, 2025
2b53a00
docs: update llm-transpile command parameters for consistency and cla…
hiroyukinakazato-db Oct 28, 2025
3ebf6c3
docs: update description to clarify LLM capabilities in handling SQL …
hiroyukinakazato-db Oct 28, 2025
46ac4fb
docs: update run_url format in Switch command response example for ac…
hiroyukinakazato-db Oct 29, 2025
2d17455
Merge branch 'main' into switch-documentation
hiroyukinakazato-db Nov 5, 2025
488db44
docs: update installation and command parameters for Switch transpiler
hiroyukinakazato-db Nov 6, 2025
597c02f
docs: update installation instructions and command parameters for Swi…
hiroyukinakazato-db Nov 6, 2025
25b7f17
fix: correct parameter name from --foundational-model to --foundation…
hiroyukinakazato-db Nov 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/lakebridge/docs/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,11 @@ For migrating your SQL workloads we provide transpilers that can:
- Translate SQL code from a variety of source platforms to Databricks SQL.
- Translate some orchestration and ETL code to Databricks SQL.

Internally, Lakebridge can use two different transpilers:
Internally, Lakebridge can use three different transpilers:

- *BladeBridge*, a mature transpiler that can handle a wide range of source dialects as well as some ETL/orchestration.
- *Morpheus*, a next-generation transpiler that currently handles a smaller set of dialects, but includes experimental support for dbt.
- *Switch*, an LLM-powered transpiler that uses Large Language Models to convert SQL and other source formats to Databricks notebooks.

The table below summarizes the source platforms that we currently support:

Expand Down
14 changes: 14 additions & 0 deletions docs/lakebridge/docs/transpile/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,20 @@ Whether you're migrating a single job or thousands, BladeBridge delivers predict

More details about the BladeBridge converter [here](/docs/transpile/pluggable_transpilers/bladebridge_overview)

### Switch

Switch is an LLM-powered Lakebridge transpiler that extends beyond SQL to convert various source formats into Databricks-compatible outputs. Using [Mosaic AI Model Serving](https://docs.databricks.com/aws/en/machine-learning/model-serving/), it understands code intent and semantics to handle complex transformations.

Key characteristics:
- **LLM-powered extensibility** - Built-in prompts for multiple SQL dialects (T-SQL, Snowflake, Teradata, Oracle, etc.) and generic formats (Python, Scala, Airflow)
- **Custom prompt support** - Add YAML prompts to handle any source format not covered by built-in options
- **Flexible outputs** - Python notebooks with Spark SQL (default), SQL notebooks (experimental), or any text-based format
- **Databricks-native processing** - Executes as scalable Lakeflow Jobs with serverless compute
- **Model flexibility** - Uses Databricks Foundation Model APIs with configurable endpoint selection
- **Complex logic handling** - Excels at stored procedures and business logic beyond ANSI SQL/PSM standards

Switch's LLM-based approach enables support for any source format through custom prompts. For built-in prompt details and custom prompt creation, see the [Switch documentation](/docs/transpile/pluggable_transpilers/switch).

### Supported dialects

| Source Platform | Converter: BladeBridge | Converter: Morpheus | Output: SQL | Output: SparkSql | Output: PySpark | Other features available |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,11 @@ custom: # this section is optional, it is passed to the transpiler at startup
<key 1>: <value 1> # can be pretty much anything
```

Databricks provides 2 transpilers: _Morpheus_, its AST-based transpiler, and _BladeBridge_, a pattern-based transpiler.
Databricks provides 3 transpilers: _Morpheus_, its AST-based transpiler, _BladeBridge_, a pattern-based transpiler, and _Switch_, an LLM-powered transpiler.
These transpilers are installed by `Lakebridge` itself as part of running the `install-transpile` command, as follows:
- the latest _Morpheus_ is fetched from [Maven Central](https://central.sonatype.com/), and installed at `.databricks/labs/remorph-transpilers/databricks-morph-plugin/`.
- the latest _BladeBridge_ is fetched from [PyPi](https://pypi.org/), and installed at `.databricks/labs/remorph-transpilers/bladebridge/`.
- the latest _Switch_ is fetched from [PyPi](https://pypi.org/), and installed as an optional direct dependency (enabled with `--include-llm-transpiler true`).

Installing 3rd party transpilers is the responsibility of their provider.

Expand Down
Loading
Loading