Skip to content

Commit 92c373c

Browse files
alex-clickhousealex-clickhouse
authored
Merge pull request #6075 from ClickHouse/entityframework-migrations
add latest EF info
2 parents ef29168 + f7332d4 commit 92c373c

1 file changed

Lines changed: 112 additions & 18 deletions

File tree

‎docs/integrations/language-clients/csharp.md‎

Lines changed: 112 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1900,7 +1900,7 @@ The official Entity Framework Core provider for ClickHouse. Map C# classes to Cl
19001900
- **Source**: [GitHub](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore)
19011901

19021902
:::note
1903-
This provider is in early development. It supports **read-only queries** and **inserts**. UPDATE, DELETE, migrations, JOINs, and subqueries are not yet implemented.
1903+
This provider is in active development. Current release supports LINQ queries (including JOINs, subqueries, and set operations), `INSERT` via `SaveChanges` / `BulkInsertAsync`, migrations with full DDL (CREATE / ALTER / DROP), and ClickHouse-specific table engine configuration. `UPDATE` / `DELETE` are not supported.
19041904
:::
19051905

19061906
#### Installation {#ef-core-installation}
@@ -1961,7 +1961,7 @@ var topPages = await ctx.PageViews
19611961
| **Time** | `Time`, `Time64(N)` | `TimeSpan` |
19621962
| **UUID** | `UUID` | `Guid` |
19631963
| **Network** | `IPv4`, `IPv6` | `IPAddress` |
1964-
| **Arrays** | `Array(T)` | `T[]` or `List<T>` |
1964+
| **Arrays** | `Array(T)` | `T[]`, `List<T>`, `IList<T>`, `ICollection<T>`, `IReadOnlyList<T>`, `IReadOnlyCollection<T>`, `IEnumerable<T>` |
19651965
| **Maps** | `Map(K, V)` | `Dictionary<K,V>` |
19661966
| **Tuples** | `Tuple(T1, ...)` | `Tuple<...>` or `ValueTuple<...>` |
19671967
| **Variant** | `Variant(T1, T2, ...)` | `object` |
@@ -1974,13 +1974,33 @@ Use `ClickHouseDecimal` (from `ClickHouse.Driver.Numerics`) instead of `decimal`
19741974

19751975
#### Supported LINQ operations {#ef-core-linq}
19761976

1977-
**Queries:** `Where`, `OrderBy`, `Take`, `Skip`, `Select`, `First`, `Single`, `Any`, `Count`, `Distinct`, `AsNoTracking`
1977+
**Queries:** `Where`, `OrderBy`, `Take`, `Skip`, `Select`, `First`, `Single`, `Any`, `All`, `Count`, `Distinct`, `AsNoTracking`
19781978

1979-
**GROUP BY & Aggregates:** `GroupBy` with `Count`, `LongCount`, `Sum`, `Average`, `Min`, `Max` — including `HAVING` (`.Where()` after `.GroupBy()`), multiple aggregates in a single projection, and `OrderBy` on aggregate results.
1979+
**GROUP BY & aggregates:** `GroupBy` with `Count`, `LongCount`, `Sum`, `Average`, `Min`, `Max` — including `HAVING` (`.Where()` after `.GroupBy()`), multiple aggregates in a single projection, and `OrderBy` on aggregate results.
19801980

1981-
**String methods:** `Contains`, `StartsWith`, `EndsWith`, `IndexOf`, `Replace`, `Substring`, `Trim`/`TrimStart`/`TrimEnd`, `ToLower`, `ToUpper`, `Length`, `IsNullOrEmpty`, `Concat` (and `+` operator)
1981+
**JOINs:** `Join` (INNER), `GroupJoin`/`SelectMany` patterns (LEFT and CROSS). LEFT JOIN returns real `null` for non-matching rows (see [LEFT JOIN null semantics](#ef-core-join-nulls) below).
19821982

1983-
**Math functions:** Standard `Math` and `MathF` methods are translated to their ClickHouse equivalents, including arithmetic, logarithmic, trigonometric, and utility functions.
1983+
**Subqueries:** correlated `Contains` / `IN`, `Any` / `EXISTS`, `All`, and scalar subqueries in projections.
1984+
1985+
**Set operations:** `Concat` (→ `UNION ALL`), `Union` (→ `UNION DISTINCT`), `Intersect`, `Except`.
1986+
1987+
**Inline local collections:** joins and `Contains` against in-memory collections (`int[]`, `List<T>`, etc.) translate into a series of UNIONs.
1988+
1989+
**String methods:** `Contains`, `StartsWith`, `EndsWith`, `IndexOf`, `Replace`, `Substring`, `Trim`/`TrimStart`/`TrimEnd`, `ToLower`, `ToUpper`, `Length`, `IsNullOrEmpty`, `Concat` (and `+` operator).
1990+
1991+
**Math functions:** standard `Math` and `MathF` methods translated to their ClickHouse equivalents — arithmetic, logarithmic, trigonometric, and utility functions.
1992+
1993+
##### LEFT JOIN null semantics {#ef-core-join-nulls}
1994+
1995+
The provider injects `set_join_use_nulls=1` into every connection path automatically to match Entity Framework expectations on JOIN behavior.
1996+
1997+
If your ClickHouse server or profile forbids changing this setting (e.g. a `readonly=1` profile), opt out with:
1998+
1999+
```csharp
2000+
optionsBuilder.UseClickHouse(connectionString, o => o.DisableJoinNullSemantics());
2001+
```
2002+
2003+
With the opt-out enabled, LEFT JOIN returns ClickHouse column defaults and EF's null-based navigation detection no longer works as expected. Use explicit comparisons against `0` / `""` instead of `== null`.
19842004

19852005
#### Inserting data {#ef-core-insert}
19862006

@@ -2219,20 +2239,94 @@ entity.Property(e => e.Data).HasColumnType("Json");
22192239
- **Integer precision** — ClickHouse JSON stores all integers as `Int64`. When reading via `JsonNode`, use `GetValue<long>()` rather than `GetValue<int>()`.
22202240
:::
22212241

2222-
#### Limitations {#ef-core-limitations}
2242+
#### Table engines {#ef-core-engines}
2243+
2244+
Configure ClickHouse table engines and engine-specific clauses via the `ToTable(name, t => ...)` fluent API. When no engine is configured, the provider defaults to `MergeTree` with `ORDER BY` derived from the entity's primary key.
22232245

2224-
| Feature | Status |
2246+
```csharp
2247+
modelBuilder.Entity<Event>(e =>
2248+
{
2249+
e.ToTable("events", t => t
2250+
.HasMergeTreeEngine()
2251+
.WithOrderBy("UserId", "Timestamp")
2252+
.WithPartitionBy("toYYYYMM(Timestamp)")
2253+
.WithPrimaryKey("UserId")
2254+
.WithSettings("index_granularity = 8192"));
2255+
});
2256+
```
2257+
2258+
Supported engine families:
2259+
2260+
| Engine | Fluent method | Notes |
2261+
|---|---|---|
2262+
| `MergeTree` | `HasMergeTreeEngine()` | Default when none configured |
2263+
| `ReplacingMergeTree` | `HasReplacingMergeTreeEngine("Version", "IsDeleted")` or `HasReplacingMergeTreeEngine<T>(e => e.Version)` | Version / IsDeleted columns optional |
2264+
| `SummingMergeTree` | `HasSummingMergeTreeEngine(…)` or `HasSummingMergeTreeEngine<T>(e => new { … })` | Optional columns-to-sum |
2265+
| `AggregatingMergeTree` | `HasAggregatingMergeTreeEngine()` | — |
2266+
| `CollapsingMergeTree` | `HasCollapsingMergeTreeEngine("Sign")` or `HasCollapsingMergeTreeEngine<T>(e => e.Sign)` | `Sign` column must be `Int8` |
2267+
| `VersionedCollapsingMergeTree` | `HasVersionedCollapsingMergeTreeEngine("Sign", "Version")` or `<T>(e => e.Sign, e => e.Version)` | — |
2268+
| `GraphiteMergeTree` | `HasGraphiteMergeTreeEngine("config_section")` | — |
2269+
| `Log`, `TinyLog`, `StripeLog`, `Memory` | `HasLogEngine()`, `HasTinyLogEngine()`, `HasStripeLogEngine()`, `HasMemoryEngine()` | No ORDER BY / PARTITION BY |
2270+
2271+
**Engine clauses:** `WithOrderBy`, `WithPartitionBy`, `WithPrimaryKey`, `WithSampleBy`, `WithTtl`, `WithSettings`. All attach to the engine builder returned from `HasXxxEngine()`.
2272+
2273+
**Column-level features:** `HasCodec`, `HasTtl`, `HasComment`, `HasDefault` — all participate in migrations.
2274+
2275+
**Data-skipping indexes** — via `HasIndex(...).HasSkippingIndexType(...)`:
2276+
2277+
```csharp
2278+
modelBuilder.Entity<Event>()
2279+
.HasIndex(e => e.UserId)
2280+
.HasSkippingIndexType("minmax")
2281+
.HasGranularity(4);
2282+
2283+
// Index with parameters (e.g. bloom_filter, tokenbf_v1):
2284+
modelBuilder.Entity<Event>()
2285+
.HasIndex(e => e.Tag)
2286+
.HasSkippingIndexType("bloom_filter")
2287+
.HasSkippingIndexParams("0.01")
2288+
.HasGranularity(1);
2289+
```
2290+
2291+
Standard (non-skipping) indexes are silently ignored since ClickHouse has no equivalent. Unique indexes throw, as ClickHouse does not enforce uniqueness.
2292+
2293+
#### Migrations {#ef-core-migrations}
2294+
2295+
Standard EF Core migrations workflow:
2296+
2297+
```bash
2298+
dotnet ef migrations add InitialCreate
2299+
dotnet ef database update
2300+
```
2301+
2302+
Supported operations:
2303+
2304+
| Operation | Emits |
2305+
|---|---|
2306+
| `CREATE TABLE` | Includes engine clause, ORDER BY, PARTITION BY, SETTINGS, column codecs/TTL/comments/defaults |
2307+
| `ALTER TABLE ADD COLUMN` | — |
2308+
| `ALTER TABLE DROP COLUMN` | — |
2309+
| `ALTER TABLE MODIFY COLUMN` | Handles type change plus annotation add/remove (CODEC, TTL, COMMENT, DEFAULT) |
2310+
| `ALTER TABLE RENAME COLUMN` | — |
2311+
| `RENAME TABLE` | — |
2312+
| `ALTER TABLE ADD INDEX` / `DROP INDEX` | Data-skipping indexes only |
2313+
| `CREATE DATABASE` / `DROP DATABASE` | Via `EnsureCreated` / `EnsureDeleted` and migrations |
2314+
2315+
#### Migration limitations {#ef-core-limitations}
2316+
2317+
| Feature | Reason |
22252318
|---|---|
2226-
| SELECT / WHERE / ORDER BY / GROUP BY | Supported |
2227-
| INSERT via `SaveChanges` / `BulkInsertAsync` | Supported |
2228-
| UPDATE / DELETE | Not supported (ClickHouse mutations are async, not OLTP-compatible) |
2229-
| Migrations | Not supported |
2230-
| JOINs, subqueries, set operations | Not supported |
2231-
| Transactions | No-op (ClickHouse does not support ACID transactions) |
2232-
| Server-generated values (auto-increment) | Not supported |
2233-
| Nested types | Not supported |
2234-
| JSON path query translation (`.Data["key"]` in LINQ) | Not supported |
2235-
| Owned entities as JSON (`.ToJson()`) | Not supported |
2319+
| Foreign keys | ClickHouse does not enforce foreign keys. Migrations reject `AddForeignKey`; the model validator emits a warning at model build time. |
2320+
| Unique constraints / unique indexes | ClickHouse does not enforce uniqueness. Unique indexes throw at migration time. |
2321+
| Server-generated values (auto-increment / `IDENTITY`) | ClickHouse has no equivalent. |
2322+
| `Nested(…)` columns | Not yet supported as a mapped CLR type. |
2323+
| Owned entities as JSON (`.ToJson()`) | Structural JSON mapping for owned entities is not yet implemented. Use `JsonNode` / `string` on a `Json` column instead (see [JSON columns](#ef-core-json)). |
2324+
2325+
Beyond migrations, the provider also does not yet support:
2326+
2327+
- **`UPDATE` / `DELETE`**
2328+
- **Transactions**: `BeginTransaction` is a no-op. No support for ACID transactions in ClickHouse.
2329+
- **JSON path query translation**: `entity.Data["key"]` in LINQ does not translate to ClickHouse's `data.key` SQL syntax. Filter on non-JSON columns and inspect JSON in memory.
22362330

22372331
## Limitations {#limitations}
22382332

0 commit comments

Comments
 (0)