|
| 1 | +# Embedding YDB in Go applications |
| 2 | + |
| 3 | +## Stack |
| 4 | + |
| 5 | +The only YDB Go SDK is **`github.com/ydb-platform/ydb-go-sdk/v3`**. The same package exposes two surfaces: |
| 6 | + |
| 7 | +- a **native** API with the modern Query Service (`db.Query().Do/DoTx(...)`) and the legacy Table Service (`db.Table().Do/DoTx(...)`), |
| 8 | +- a **`database/sql`** driver registered as `"ydb"` (blank-import `_ "github.com/ydb-platform/ydb-go-sdk/v3"`) for code that needs the stdlib interface. |
| 9 | + |
| 10 | +Default new code to the native Query Service. The Table Service is in legacy mode and has a 1000-row default result cap (surfaced as an error on `s.Execute` in v3 by default, restored to v2-style silent truncation if `ydb.WithIgnoreTruncated` is set on the driver — see <https://github.com/ydb-platform/ydb-go-sdk/blob/master/MIGRATION_v2_v3.md>). Connection-string format and authentication environment variables: see [`../../../ydb-core/SKILL.md#connecting`](../../../ydb-core/SKILL.md#connecting). Worked examples for both surfaces: <https://github.com/ydb-platform/ydb-go-sdk/tree/master/examples>. `database/sql` specifics: <https://github.com/ydb-platform/ydb-go-sdk/blob/master/SQL.md>. |
| 11 | + |
| 12 | +## Query execution |
| 13 | + |
| 14 | +Canonical native-API pattern — open the driver once with `ydb.Open(...)`, then run all work inside a `db.Query().Do(...)` closure. The closure is the retry unit: the SDK invokes it again on every retryable error. |
| 15 | + |
| 16 | +```go |
| 17 | +db, err := ydb.Open(ctx, "grpc://localhost:2136/local") |
| 18 | +if err != nil { return err } |
| 19 | +defer db.Close(ctx) |
| 20 | + |
| 21 | +err = db.Query().Do(ctx, func(ctx context.Context, s query.Session) error { |
| 22 | + res, err := s.Query(ctx, |
| 23 | + `SELECT name FROM users WHERE id = $id;`, |
| 24 | + query.WithParameters(ydb.ParamsBuilder(). |
| 25 | + Param("$id").Uint64(42).Build()), |
| 26 | + ) |
| 27 | + if err != nil { return err } |
| 28 | + defer func() { _ = res.Close(ctx) }() |
| 29 | + // iterate result sets / rows, build the value inside the closure |
| 30 | + return nil |
| 31 | +}, query.WithIdempotent()) |
| 32 | +``` |
| 33 | + |
| 34 | +Three load-bearing pieces: |
| 35 | + |
| 36 | +- **`query.WithIdempotent()`** declares the closure safe to replay on *conditionally* retryable failures (connection drop, gRPC reset, session loss). Set on reads and on writes keyed by a client-generated id; do not set on a non-idempotent write such as a counter increment. |
| 37 | +- **`query.WithParameters(ydb.ParamsBuilder()...)`** binds values rather than concatenating them. Closes SQL injection and per-distinct-text plan-cache churn. The server infers types from the builder, so a leading `DECLARE` block is optional for scalar parameters — write one only when you want an explicit contract (typically for `List<Struct<...>>` and other compound types) or to fail-fast on a parameter-type mismatch from the caller. |
| 38 | +- **All data processing happens inside the closure.** Assign to outer variables only on the success path — the line that returns `nil`. Anything assigned earlier survives across retry attempts and produces wrong values. |
| 39 | + |
| 40 | +Source: <https://github.com/ydb-platform/ydb-go-sdk> README "Example Usage". |
| 41 | + |
| 42 | +## Transactions |
| 43 | + |
| 44 | +YDB has two transaction styles, and `ydb-go-sdk/v3` supports both: |
| 45 | + |
| 46 | +- **Non-interactive** (default for new code) — the SDK manages the transaction inside `db.Query().DoTx(ctx, func(ctx, tx query.TxActor) error { ... })`. Per the upstream `query/client.go` godoc: *"If op TxOperation returns nil — transaction will be committed"*. Open the driver with `ydb.WithLazyTx(true)` so the begin is deferred onto the first query, and pass `query.WithCommit()` to the last write so the commit rides on its RPC — zero standalone begin/commit round-trips. |
| 47 | +- **Interactive** — the developer writes the begin and commit explicitly. Same fusing, two shapes depending on statement count: |
| 48 | + - *Multi-statement*: first call is `tx, result, err := s.Execute(ctx, table.TxControl(table.BeginTx(table.WithSerializableReadWrite())), firstQuery, params)` — `txControl` is `s.Execute`'s positional second argument and the begin rides on this RPC. Subsequent calls use the returned `tx` handle. The last call uses `tx.Execute(ctx, lastQuery, params, options.WithCommit())` (where `options` is `github.com/ydb-platform/ydb-go-sdk/v3/table/options`) — the commit rides on the final write's RPC. Canonical form: `table/example_test.go` `Example_lazyTransaction` in upstream. |
| 49 | + - *Single-statement*: combine begin and commit on the only `s.Execute`: `s.Execute(ctx, table.TxControl(table.BeginTx(table.WithSerializableReadWrite()), table.CommitTx()), sql, params)` — one RPC carries begin + write + commit. Canonical form: `examples/ttl/series.go`. |
| 50 | + - *Query Service* equivalent: `s.Query(ctx, sql, query.WithTxControl(...), query.WithCommit())` — txControl and commit are `ExecuteOption`s on `s.Query(...)`. |
| 51 | + |
| 52 | +Worked non-interactive example: <https://github.com/ydb-platform/ydb-go-sdk/blob/master/examples/transaction/query/main.go>. |
| 53 | + |
| 54 | +For the transaction-mode list (`SerializableRW`, `SnapshotRO`, `StaleRO`, `OnlineRO`) and the consequence for application-level optimistic locking, see [`../working-with-data.md`](../working-with-data.md). |
| 55 | + |
| 56 | +## Retries |
| 57 | + |
| 58 | +`Do` / `DoTx` retry the closure internally; there is no need for an outer `for` loop or a `time.Sleep`-based retrier in caller code. The SDK classifies the error through `retry/mode.go` `MustRetry(isOperationIdempotent bool)`: |
| 59 | + |
| 60 | +- **Non-retryable** — propagated to the caller (`PRECONDITION_FAILED`, schema mismatch, bad parameters). |
| 61 | +- **Unconditionally retryable** — always retried regardless of idempotency (`ABORTED`, `OVERLOADED`). |
| 62 | +- **Conditionally retryable** — retried only when `WithIdempotent` was passed (transport drops, session loss, timeouts). |
| 63 | + |
| 64 | +Backoff and jitter are built in; configure via retry options on the `Do` / `DoTx` call, not by wrapping. |
| 65 | + |
| 66 | +Source: <https://github.com/ydb-platform/ydb-go-sdk/blob/master/retry/mode.go>. |
| 67 | + |
| 68 | +## Bulk upsert |
| 69 | + |
| 70 | +Use `db.Table().BulkUpsert(ctx, tablePath, rows)` for non-transactional ingest: |
| 71 | + |
| 72 | +```go |
| 73 | +err := db.Table().BulkUpsert(ctx, |
| 74 | + path.Join(db.Name(), "events"), |
| 75 | + table.BulkUpsertDataRows(types.ListValue(values...)), |
| 76 | +) |
| 77 | +``` |
| 78 | + |
| 79 | +For when the bulk API is the right call versus `AS_TABLE` inside a transaction — and when it is forbidden (synchronous secondary indexes, attached changefeeds) — see [`../working-with-data.md`](../working-with-data.md). |
| 80 | + |
| 81 | +Source: <https://github.com/ydb-platform/ydb-go-sdk/blob/master/examples/opensource_night2024/main.go>. |
| 82 | + |
| 83 | +## Connection |
| 84 | + |
| 85 | +See [`../../../ydb-core/SKILL.md#connecting`](../../../ydb-core/SKILL.md#connecting). |
0 commit comments