Skip to content

Commit 30d5380

Browse files
SouravRoy-ETLSouravRoy-ETL
committed
feat: 25 more languages (60 total) + DuckDB Quack remote protocol
i18n pass 3 brings the supported locale set to 60: Norwegian, Danish, Finnish, Catalan, Bulgarian, Slovak, Croatian, Serbian, Slovenian, Lithuanian, Latvian, Estonian, Khmer, Burmese, Sinhala, Nepali, Swahili, Afrikaans, Welsh, Irish, Icelandic, Albanian, Azerbaijani, Mongolian, Kazakh. Native names in the topbar selector. Translated via the deep-translator / Google backend in ~12 minutes for 2 500+ strings. Quack remote protocol (DuckDB May 2026, HTTP on port 9494): - src.quack: ATTACHes a remote DuckDB via the quack: URL scheme with a SECRET-based token, then reads tables qualified by duckle_src.schema.table. Goes through the existing build_relational_source path (mode=table / mode=sql). - snk.quack: writes through build_relational_sink so all four modes (append / overwrite / truncate / upsert) work without protocol-specific wiring. - quack_attach() emits "CREATE OR REPLACE SECRET duckle_quack_secret (TYPE QUACK, TOKEN '...'); ATTACH 'quack:host:port' AS duckle_src (READ_ONLY);" - omits SECRET when no token is provided so test servers without auth work too. - Default port is 9494. If the host string already carries a port it's used verbatim (host:port form, but skipped if it looks like an IPv6 literal in brackets). - Two unit tests: SECRET + ATTACH are emitted with the right shape when a token is set; SECRET is omitted when token is empty. Frontend wiring: - Palette: src.quack + snk.quack added to the Cloud Warehouses group right after MotherDuck. Same group routes through synthWarehouseSource / Sink, where I added compact Quack manifests (host, port=9494, token, schemaName, tableName, and the standard write-mode picker on the sink). - README capability tables: 73 sources -> 74 (Quack listed under Cloud warehouses), 57 sinks -> 58 (same), i18n line bumped from 35 to 60 languages. Requires DuckDB built with quack support on both sides. Duckle's bundled DuckDB CLI is 1.5.3; users wanting Quack today need to swap in a v2.0+ build via Engine setup. Error surfaces clearly from DuckDB ("Unknown SECRET type 'QUACK'" or similar) on older builds without any Duckle-side breakage.
1 parent 1d07448 commit 30d5380

32 files changed

Lines changed: 5728 additions & 12 deletions

‎README.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,9 @@
5656
**Reference**
5757

5858
- [Capabilities matrix](#capabilities)
59-
- [Sources](#sources-73-available)
59+
- [Sources](#sources-74-available)
6060
- [Transforms](#transforms-123-available)
61-
- [Sinks](#sinks-57-available)
61+
- [Sinks](#sinks-58-available)
6262
- [Data quality](#data-quality-12-available)
6363
- [Custom code](#custom-code-7-available)
6464
- [Control flow](#control-flow-14-available)
@@ -138,7 +138,7 @@ The sidebar on the right is **Duckie AI Assistant** - powered by **Qwen 2.5 Code
138138
| **Git-friendly by design** | Pipelines, connections, contexts, and routines persist as plain files in a folder you pick. Diff them, branch them, review them. |
139139
| **290+ connectors that work** | Files, databases, warehouses, lakehouses, object stores, SaaS APIs, NoSQL, streaming brokers, vector DBs, FTP, IMAP, SMTP. Each is covered by tests. |
140140
| **Honest about scope** | Single-machine and embedded by design. Built to make local and small-team data work fast, not to replace a distributed warehouse. |
141-
| **35 UI languages** | Topbar, palette, chat assistant, and common dialogs ship localized. English, Spanish, Chinese (Simplified + Traditional), Hindi, Arabic, Portuguese (Brazil), Bengali, Russian, Japanese, Punjabi, German, Korean, French, Vietnamese, Telugu, Marathi, Turkish, Tamil, Urdu, Persian, Polish, Italian, Ukrainian, Indonesian, Thai, Dutch, Hebrew, Swedish, Greek, Czech, Hungarian, Romanian, Filipino, Malay. RTL (Arabic, Hebrew, Persian, Urdu) supported. Switch languages from the topbar globe. |
141+
| **60 UI languages** | Topbar, palette, chat assistant, properties panel, and common dialogs ship localized. English, Spanish, Chinese (Simplified + Traditional), Hindi, Arabic, Portuguese (Brazil), Bengali, Russian, Japanese, Punjabi, German, Korean, French, Vietnamese, Telugu, Marathi, Turkish, Tamil, Urdu, Persian, Polish, Italian, Ukrainian, Indonesian, Thai, Dutch, Hebrew, Swedish, Greek, Czech, Hungarian, Romanian, Filipino, Malay, Norwegian, Danish, Finnish, Catalan, Bulgarian, Slovak, Croatian, Serbian, Slovenian, Lithuanian, Latvian, Estonian, Khmer, Burmese, Sinhala, Nepali, Swahili, Afrikaans, Welsh, Irish, Icelandic, Albanian, Azerbaijani, Mongolian, Kazakh. RTL (Arabic, Hebrew, Persian, Urdu) supported. Switch languages from the topbar globe. |
142142
| **Open source** | Dual-licensed MIT OR Apache-2.0. Yours to use, fork, and extend. |
143143

144144
---
@@ -185,7 +185,7 @@ The component palette ships **313 nodes** so the roadmap is visible in the produ
185185

186186
Duckle is not a CSV tool with extras. It reads a broad set of formats and sources, ships a deep transform library, and writes to files, databases, object storage, vector DBs, message buses, and email.
187187

188-
### Sources (73 available)
188+
### Sources (74 available)
189189

190190
| Group | Connectors | Status |
191191
|---|---|---|
@@ -197,7 +197,7 @@ Duckle is not a CSV tool with extras. It reads a broad set of formats and source
197197
| **Network relational DBs** | SQL Server (TDS), Oracle (Instant Client at runtime), ClickHouse (HTTP API) | Available |
198198
| **Network relational DBs** | IBM DB2, generic JDBC | Planned |
199199
| **Object storage** | Amazon S3, Google Cloud Storage, Azure Blob, HTTP(S), MinIO, Cloudflare R2, Backblaze B2 | Available (live CI for MinIO) |
200-
| **Cloud warehouses** | MotherDuck, Snowflake (SQL API + PAT/JWT), BigQuery, Redshift (postgres ATTACH), Databricks SQL (Statement Execution + chunk follow), Azure Synapse (TDS) | Available |
200+
| **Cloud warehouses** | MotherDuck, Snowflake (SQL API + PAT/JWT), BigQuery, Redshift (postgres ATTACH), Databricks SQL (Statement Execution + chunk follow), Azure Synapse (TDS), **DuckDB Quack** (May 2026 remote protocol - HTTP on :9494, SECRET-based token auth) | Available |
201201
| **Streaming** | Apache Kafka / Redpanda (pure-Rust `rskafka`), NATS JetStream, GCP Pub/Sub (REST + auto-ack), RabbitMQ (`lapin` AMQP), AWS Kinesis (HTTP + SigV4 - no AWS SDK) | Available |
202202
| **Streaming** | Pulsar, Event Hubs, multi-shard Kinesis | Planned |
203203
| **APIs and SaaS (REST)** | Salesforce, HubSpot, Pipedrive, Zendesk, Intercom, Stripe, QuickBooks, Xero, Shopify, Notion, Airtable, Asana, Trello, ClickUp, Monday.com, GitHub, GitLab, Linear, Jira, Slack, Discord, Telegram, Twilio, Mailchimp, SendGrid, Segment - thin pre-configured wrappers over `src.rest` / `src.graphql` | Available |
@@ -265,7 +265,7 @@ Validators split their input: passing rows continue on the main port, failures r
265265
| **JavaScript UDF** | Per-row JS transform via pure-Rust `boa` interpreter. Sandboxed. Define a `transform(row)` function. |
266266
| **Python / Rust UDFs** | Embedded-language stages | Planned |
267267

268-
### Sinks (57 available)
268+
### Sinks (58 available)
269269

270270
| Group | Connectors | Status |
271271
|---|---|---|
@@ -277,7 +277,7 @@ Validators split their input: passing rows continue on the main port, failures r
277277
| **Network relational DBs** | SQL Server / Azure Synapse (TDS, multi-row VALUES batched), Oracle (Instant Client; INSERT ALL), ClickHouse (HTTP JSONEachRow) | Available |
278278
| **Network relational DBs** | IBM DB2, generic JDBC | Planned |
279279
| **Object storage** | S3, GCS, Azure Blob via DuckDB `httpfs` (MinIO / R2 / B2 via endpoint) | Available |
280-
| **Cloud warehouses** | MotherDuck, Snowflake (PAT or JWT RS256), BigQuery, Redshift, Databricks SQL, Azure Synapse | Available |
280+
| **Cloud warehouses** | MotherDuck, Snowflake (PAT or JWT RS256), BigQuery, Redshift, Databricks SQL, Azure Synapse, **DuckDB Quack** (concurrent writers to remote DuckDB via the May 2026 protocol) | Available |
281281
| **HTTP APIs** | REST (POST/PUT/PATCH batched JSON-array), Webhook (one POST per row), GraphQL mutations | Available |
282282
| **Email (SMTP)** | Per-row SMTP send via pure-Rust `lettre` + rustls. Plain text v1; HTML + attachments follow. | Available |
283283
| **NoSQL** | MongoDB (insert_many batched), Cassandra / ScyllaDB (CQL), Elasticsearch / OpenSearch (`_bulk` NDJSON), Redis (pipelined SET) | Available |
@@ -454,10 +454,10 @@ A wider tour of the workflow.
454454

455455
| Step | What you do | Where to look |
456456
|---|---|---|
457-
| **1. Sources** | Drag a source, point it at a file / DB / cloud URL / SaaS endpoint. Click **Autodetect schema** to read columns + a sample. | [Sources reference](#sources-73-available) |
457+
| **1. Sources** | Drag a source, point it at a file / DB / cloud URL / SaaS endpoint. Click **Autodetect schema** to read columns + a sample. | [Sources reference](#sources-74-available) |
458458
| **2. Transforms** | Wire transforms to source output ports. Configure in the Properties panel. **Preview** tab shows live rows; **Plan** tab shows generated SQL. | [Transforms reference](#transforms-123-available) |
459459
| **3. Data quality** | Drop in a validator (Not-Null, Range, Regex, Uniqueness). Passing rows continue on the main port; failures route to the **reject** port. | [Data quality reference](#data-quality-12-available) |
460-
| **4. Sinks** | Finish with a sink (file, DB, cloud, vector DB, message bus, email). Set write mode (overwrite, append, truncate, upsert). | [Sinks reference](#sinks-57-available) |
460+
| **4. Sinks** | Finish with a sink (file, DB, cloud, vector DB, message bus, email). Set write mode (overwrite, append, truncate, upsert). | [Sinks reference](#sinks-58-available) |
461461
| **5. Run** | Press **Run** to execute on DuckDB. Nodes light up stage by stage; **Output** + **Console** show row counts, timing, errors. Stop button kills mid-run. | [Run feedback](#orchestration-and-workspace) |
462462
| **6. Ask Duckie** | For anything you can describe in English, the AI assistant can sketch a pipeline. Iterate by editing the graph or asking follow-ups. | [Meet Duckie](#meet-duckie---the-local-ai-pipeline-assistant) |
463463
| **7. Reuse** | Save Connections, Context variables, and SQL Routines in the workspace; reference `${context.var}` in any field. Everything persists as plain files. | [Workspace and Git flow](#workspace-and-git-flow) |

‎crates/duckdb-engine/src/plan.rs‎

Lines changed: 127 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3731,7 +3731,7 @@ fn build_view_sql(
37313731
}
37323732
"src.postgres" | "src.cockroach" | "src.mysql" | "src.mariadb"
37333733
| "src.motherduck" | "src.ducklake" | "src.pgvector"
3734-
| "src.redshift" | "src.bigquery" => build_relational_source(component_id, props),
3734+
| "src.redshift" | "src.bigquery" | "src.quack" => build_relational_source(component_id, props),
37353735
"src.avro" => Ok(build_avro_source(props)),
37363736
"src.excel" => Ok(build_excel_source(props)),
37373737
"src.iceberg" => Ok(build_iceberg_source(props)),
@@ -5793,6 +5793,8 @@ fn attach_prelude(component_id: &str, props: &JsonValue) -> String {
57935793
"snk.mysql" | "snk.mariadb" => return db_attach(props, "mysql", 3306, false),
57945794
"src.motherduck" => return md_attach(props, true),
57955795
"snk.motherduck" => return md_attach(props, false),
5796+
"src.quack" => return quack_attach(props, true),
5797+
"snk.quack" => return quack_attach(props, false),
57965798
"src.ducklake" => return ducklake_attach(props, true),
57975799
"snk.ducklake" => return ducklake_attach(props, false),
57985800
// BigQuery via the duckdb-bigquery community extension. The
@@ -6068,6 +6070,52 @@ fn md_attach(props: &JsonValue, read_only: bool) -> String {
60686070
format!("ATTACH '{}' AS {}{}; ", sql_escape(&url), alias, mode)
60696071
}
60706072

6073+
/// Quack remote protocol (DuckDB 2.0+, May 2026). The remote DuckDB
6074+
/// instance runs `quack_serve(...)` on port 9494 by default and exposes
6075+
/// its database to multiple concurrent clients over HTTP using a
6076+
/// custom `application/duckdb` MIME type. Client side: a SECRET
6077+
/// carries the auth token, then ATTACH names the URL.
6078+
///
6079+
/// Requires DuckDB built with quack support; older builds will surface
6080+
/// a clear error at runtime ("Unknown ATTACH option 'TYPE'" or
6081+
/// similar) without any Duckle-side breakage.
6082+
fn quack_attach(props: &JsonValue, read_only: bool) -> String {
6083+
let host = match string_prop(props, "host").filter(|s| !s.is_empty()) {
6084+
Some(h) => h,
6085+
None => return String::new(),
6086+
};
6087+
let port = props
6088+
.get("port")
6089+
.and_then(|v| v.as_u64())
6090+
.filter(|p| *p > 0)
6091+
.unwrap_or(9494);
6092+
let token = string_prop(props, "token").filter(|s| !s.is_empty());
6093+
6094+
// If the host already carries an explicit :port, respect it; otherwise
6095+
// append the default 9494.
6096+
let url = if host.contains(':') && !host.starts_with('[') {
6097+
format!("quack:{}", host)
6098+
} else {
6099+
format!("quack:{}:{}", host, port)
6100+
};
6101+
6102+
let (alias, mode) = if read_only {
6103+
("duckle_src", " (READ_ONLY)")
6104+
} else {
6105+
("duckle_dst", "")
6106+
};
6107+
6108+
let secret = match token {
6109+
Some(t) => format!(
6110+
"CREATE OR REPLACE SECRET duckle_quack_secret (TYPE QUACK, TOKEN '{}'); ",
6111+
sql_escape(&t)
6112+
),
6113+
None => String::new(),
6114+
};
6115+
6116+
format!("{}ATTACH '{}' AS {}{}; ", secret, sql_escape(&url), alias, mode)
6117+
}
6118+
60716119
/// Excel sink: COPY ... TO '<path>' (FORMAT 'xlsx'). The form's
60726120
/// `hasHeader` toggle becomes HEADER true/false. v1.2+ ships native
60736121
/// xlsx writer in the excel extension.
@@ -7395,7 +7443,7 @@ fn build_sink_sql(
73957443
"snk.sqlite" | "snk.duckdb" => Ok(build_db_sink(props, from_view)),
73967444
"snk.postgres" | "snk.cockroach" | "snk.mysql" | "snk.mariadb"
73977445
| "snk.motherduck" | "snk.ducklake" | "snk.pgvector"
7398-
| "snk.redshift" | "snk.bigquery" => build_relational_sink(component_id, props, from_view),
7446+
| "snk.redshift" | "snk.bigquery" | "snk.quack" => build_relational_sink(component_id, props, from_view),
73997447
"snk.excel" => Ok(build_excel_sink(props, from_view)),
74007448
"snk.spatial" => Ok(build_spatial_sink(props, from_view)),
74017449
"snk.iceberg" => Ok(build_iceberg_sink(props, from_view)),
@@ -7629,6 +7677,83 @@ mod tests {
76297677
.contains("TO '/tmp/out.parquet' (FORMAT PARQUET"));
76307678
}
76317679

7680+
#[test]
7681+
fn quack_source_emits_attach_with_secret() {
7682+
let p = pipeline_from_json(
7683+
r#"{
7684+
"nodes": [
7685+
{"id":"s1","position":{"x":0,"y":0},"data":{
7686+
"label":"Quack","componentId":"src.quack",
7687+
"properties":{"host":"duck.example.com","port":9494,
7688+
"token":"super_secret","tableName":"orders"}}},
7689+
{"id":"k1","position":{"x":0,"y":0},"data":{
7690+
"label":"CSV","componentId":"snk.csv",
7691+
"properties":{"path":"/tmp/out.csv","hasHeader":true}}}
7692+
],
7693+
"edges": [
7694+
{"id":"e1","source":"s1","target":"k1",
7695+
"data":{"connectionType":"main"}}
7696+
]
7697+
}"#,
7698+
);
7699+
let compiled = compile(&p).unwrap();
7700+
let src_sql = &compiled.stages[0].sql;
7701+
assert!(
7702+
src_sql.contains("CREATE OR REPLACE SECRET duckle_quack_secret"),
7703+
"missing SECRET creation: {}",
7704+
src_sql
7705+
);
7706+
assert!(src_sql.contains("TYPE QUACK"), "wrong SECRET type: {}", src_sql);
7707+
assert!(src_sql.contains("'super_secret'"), "token not in SECRET: {}", src_sql);
7708+
assert!(
7709+
src_sql.contains("ATTACH 'quack:duck.example.com:9494'"),
7710+
"wrong ATTACH URL: {}",
7711+
src_sql
7712+
);
7713+
assert!(src_sql.contains("AS duckle_src"), "wrong alias: {}", src_sql);
7714+
assert!(src_sql.contains("READ_ONLY"), "missing READ_ONLY: {}", src_sql);
7715+
assert!(
7716+
src_sql.contains("SELECT * FROM duckle_src"),
7717+
"missing SELECT from alias: {}",
7718+
src_sql
7719+
);
7720+
}
7721+
7722+
#[test]
7723+
fn quack_source_omits_secret_when_no_token() {
7724+
// Unauthenticated test servers: leave the SECRET off entirely
7725+
// rather than emitting an empty TOKEN clause.
7726+
let p = pipeline_from_json(
7727+
r#"{
7728+
"nodes": [
7729+
{"id":"s1","position":{"x":0,"y":0},"data":{
7730+
"label":"Quack","componentId":"src.quack",
7731+
"properties":{"host":"localhost","tableName":"t"}}},
7732+
{"id":"k1","position":{"x":0,"y":0},"data":{
7733+
"label":"CSV","componentId":"snk.csv",
7734+
"properties":{"path":"/tmp/o.csv","hasHeader":true}}}
7735+
],
7736+
"edges": [
7737+
{"id":"e1","source":"s1","target":"k1",
7738+
"data":{"connectionType":"main"}}
7739+
]
7740+
}"#,
7741+
);
7742+
let compiled = compile(&p).unwrap();
7743+
let src_sql = &compiled.stages[0].sql;
7744+
assert!(
7745+
!src_sql.contains("CREATE OR REPLACE SECRET"),
7746+
"should not emit empty SECRET: {}",
7747+
src_sql
7748+
);
7749+
// Default port 9494 is appended when host has no explicit port.
7750+
assert!(
7751+
src_sql.contains("'quack:localhost:9494'"),
7752+
"missing default port: {}",
7753+
src_sql
7754+
);
7755+
}
7756+
76327757
#[test]
76337758
fn rejects_cycles() {
76347759
let p = pipeline_from_json(

‎frontend/src/i18n/index.ts‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,31 @@ import hu from './locales/hu.json';
4848
import ro from './locales/ro.json';
4949
import tl from './locales/tl.json';
5050
import ms from './locales/ms.json';
51+
import no from './locales/no.json';
52+
import da from './locales/da.json';
53+
import fi from './locales/fi.json';
54+
import ca from './locales/ca.json';
55+
import bg from './locales/bg.json';
56+
import sk from './locales/sk.json';
57+
import hr from './locales/hr.json';
58+
import sr from './locales/sr.json';
59+
import sl from './locales/sl.json';
60+
import lt from './locales/lt.json';
61+
import lv from './locales/lv.json';
62+
import et from './locales/et.json';
63+
import km from './locales/km.json';
64+
import my from './locales/my.json';
65+
import si from './locales/si.json';
66+
import ne from './locales/ne.json';
67+
import sw from './locales/sw.json';
68+
import af from './locales/af.json';
69+
import cy from './locales/cy.json';
70+
import ga from './locales/ga.json';
71+
import is_ from './locales/is.json';
72+
import sq from './locales/sq.json';
73+
import az from './locales/az.json';
74+
import mn from './locales/mn.json';
75+
import kk from './locales/kk.json';
5176

5277
// Locales may be missing newer keys (filled in by scripts/i18n-translate.py).
5378
// i18next's fallbackLng resolves missing keys to en, so loose typing is safe.
@@ -88,6 +113,32 @@ const RESOURCES: Record<LangCode, LocaleBundle> = {
88113
'ro': { common: ro },
89114
'tl': { common: tl },
90115
'ms': { common: ms },
116+
// Pass 3 additions (25 languages, 35 -> 60 total)
117+
'no': { common: no },
118+
'da': { common: da },
119+
'fi': { common: fi },
120+
'ca': { common: ca },
121+
'bg': { common: bg },
122+
'sk': { common: sk },
123+
'hr': { common: hr },
124+
'sr': { common: sr },
125+
'sl': { common: sl },
126+
'lt': { common: lt },
127+
'lv': { common: lv },
128+
'et': { common: et },
129+
'km': { common: km },
130+
'my': { common: my },
131+
'si': { common: si },
132+
'ne': { common: ne },
133+
'sw': { common: sw },
134+
'af': { common: af },
135+
'cy': { common: cy },
136+
'ga': { common: ga },
137+
'is': { common: is_ },
138+
'sq': { common: sq },
139+
'az': { common: az },
140+
'mn': { common: mn },
141+
'kk': { common: kk },
91142
};
92143

93144
void i18n

0 commit comments

Comments
 (0)