Merge pull request #259 from ducflair/dev

ci: remove legacy logic for ducpy ci

Author: jorgedanisc

.github/workflows/release-ducpy.yml

@@ -98,7 +98,6 @@ jobs:
           TAG=$(git describe --tags --abbrev=0 --match 'ducpy@*')
           VERSION="${TAG#ducpy@}"
           echo "Building Pyodide wheel for version ${VERSION}"
-          uv run python scripts/sync_schema.py
           RUSTUP_TOOLCHAIN=nightly SETUPTOOLS_SCM_PRETEND_VERSION="${VERSION}" pyodide build --outdir dist
 
       - name: Verify wheel platform tag

packages/ducjs/README.md

@@ -20,22 +20,13 @@ For detailed documentation, including all available types and utility functions,
 
 ## Tools
 
-- [Playground](https://ducflair.com/core): Experiment with the `duc` format in a live environment.
+- [Playground](https://scopture.com/w/hollow): Experiment with the `duc` format in a live environment.
 - [Documentation](https://duc.ducflair.com): Comprehensive guides and API references.
 
 ## Contributing
 
-At the moment we are not accepting contributions to this package. However, we welcome feedback and suggestions for future improvements. Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
+Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
 
 ## License
 
-This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.
-
-
-## Commit Message Guidelines
-
-To ensure smooth releases with semantic-release, please follow [these guidelines](https://semantic-release.gitbook.io/semantic-release#how-does-it-work).
-
----
-
-*The duc format and libraries are constantly evolving, aiming to set new standards in the 2D CAD industry. Be a part of this transformation and help shape the future of design technology!*
+This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.

packages/ducpdf/README.md

@@ -23,22 +23,13 @@ For detailed documentation, including all available types and utility functions,
 
 ## Tools
 
-- [Playground](https://ducflair.com/core): Experiment with the `duc` format in a live environment.
+- [Playground](https://scopture.com/w/hollow): Experiment with the `duc` format in a live environment.
 - [Documentation](https://duc.ducflair.com): Comprehensive guides and API references.
 
 ## Contributing
 
-At the moment we are not accepting contributions to this package. However, we welcome feedback and suggestions for future improvements. Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
+Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
 
 ## License
 
-This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.
-
-
-## Commit Message Guidelines
-
-To ensure smooth releases with semantic-release, please follow [these guidelines](https://semantic-release.gitbook.io/semantic-release#how-does-it-work).
-
----
-
-*The duc format and libraries are constantly evolving, aiming to set new standards in the 2D CAD industry. Be a part of this transformation and help shape the future of design technology!*
+This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.

packages/ducpdf/src/duc2pdf/README.md

@@ -65,18 +65,10 @@ For detailed documentation, including all available types and utility functions,
 
 ## Tools
 
-- [Playground](https://ducflair.com/core): Experiment with the `duc` format in a live environment.
+- [Playground](https://scopture.com/w/hollow): Experiment with the `duc` format in a live environment.
 - [Documentation](https://duc.ducflair.com): Comprehensive guides and API references.
 
 
 ## License
 
-This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.
-
-## Commit Message Guidelines
-
-To ensure smooth releases with semantic-release, please follow [these guidelines](https://semantic-release.gitbook.io/semantic-release#how-does-it-work).
-
----
-
-*The duc format and libraries are constantly evolving, aiming to set new standards in the 2D CAD industry. Be a part of this transformation and help shape the future of design technology!*
+This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.

packages/ducpy/README.md

@@ -12,54 +12,76 @@
   </p>
 </p>
 
-
-The `ducpy` package provides a robust Python implementation for the `duc` 2D CAD file format, a cornerstone of our advanced design system. Designed for professionals seeking precision and efficiency, this package ensures seamless integration and manipulation of `duc` files within your Python projects.
-
-## Introduction
-
-The `ducpy` package offers comprehensive Python types and helper functions to work effortlessly with the `duc` CAD file format. Built with efficiency and performance in mind, this package enables you to parse, validate, and manipulate `duc` files with ease.
-
-## Features
-
-- **Python Support:** Strongly-typed interfaces for all aspects of the `duc` file format, ensuring type safety and IDE support.
-- **Utility Functions:** Easy-to-use functions for parsing, validating, and manipulating `duc` files.
-- **Extensible Architecture:** Designed to integrate seamlessly with other `ducflair` packages and your custom tools.
-- **Performance Optimized:** Efficient processing to handle complex CAD data with ease.
+The `ducpy` package is the official Python implementation of the `.duc` 2D CAD file format. It is built on top of the [`duc`](../ducrs) Rust crate, exposed as the `ducpy_native` extension. A `.duc` file is a zlib-compressed SQLite database — `ducpy` gives you both a high-level builder DSL and direct low-level access to that schema, plus parsing, serialization, and search helpers.
 
 ## Installation
 
-Install the package using pip:
-
 ```bash
 pip install ducpy
 ```
 
-## Usage
+or with [`uv`](https://docs.astral.sh/uv/):
+
+```bash
+uv add ducpy
+```
+
+## Quick start
 
 ```python
 import ducpy as duc
 ```
 
+## API overview
+
+`ducpy` is organised around four conceptual entry points, matching the structure of the full [API reference](https://duc.ducflair.com):
+
+### Builders API (High-level)
+
+The easy way to build and manage `.duc` files. Construct elements, apply styles, manage layers, build blocks, and handle document state through the `duc.builders` module.
+
+See the worked examples:
+
+- [Element creation](src/examples/element_creation_demo.py) — building rectangles, ellipses, polygons, lines, arrows, text, frames and plots with the fluent builder DSL.
+- [Mutating elements](src/examples/mutation_demo.py) — updating element properties in place and observing version changes.
+- [Document elements](src/examples/document_element_demo.py) and [model elements](src/examples/model_element_demo.py) — for the higher-level container / model element types.
+
+### SQL Builder (Low-level)
+
+A `.duc` file is a zlib-compressed SQLite database. Use `duc.builders.sql_builder` (`DucSQL`) for direct schema access, bulk queries, and low-level manipulation.
+
+See the worked example: [SQL Builder](src/examples/sql_builder_demo.py) — `DucSQL.new()` to create a `.duc` from scratch, `DucSQL(path)` to query an existing one.
+
+### Search
+
+Query and search elements and files programmatically through the `duc.search` API.
+
+### File I/O
+
+Read and write `.duc` files using the `duc.parse` and `duc.serialize` modules.
+
+See the worked examples:
+
+- [Parsing](src/examples/parsing_demo.py) — `parse_duc` / `parse_duc_lazy` / `get_external_file` / `list_external_files`.
+- [Serialization](src/examples/serialization_demo.py) — `serialize_duc` from builder-created elements.
+- [External files](src/examples/external_files_demo.py) — attaching binary blobs (images, PDFs) to a `.duc` document.
+
 ## Documentation
 
-For detailed documentation, including all available types and utility functions, visit our [Documentation](https://duc.ducflair.com).
+For detailed documentation, including the full API reference, schema specification, and end-to-end examples, see:
+
+- API reference & guides: [duc.ducflair.com](https://duc.ducflair.com)
+- Source-level reference: [docs/](docs/index.rst)
+- Full example index: [docs/examples.rst](docs/examples.rst)
 
 ## Tools
 
 - [Playground](https://scopture.com): Experiment with the `duc` format in a live environment.
 
 ## Contributing
 
-At the moment we are not accepting contributions to this package. However, we welcome feedback and suggestions for future improvements. Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
+Feel free to open an issue or contact us at [Ducflair Support](https://www.ducflair.com/support).
 
 ## License
 
-This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.
-
-## Commit Message Guidelines
-
-To ensure smooth releases with semantic-release, please follow [these guidelines](https://semantic-release.gitbook.io/semantic-release#how-does-it-work).
-
----
-
-*The duc format and libraries are constantly evolving, aiming to set new standards in the 2D CAD industry. Be a part of this transformation and help shape the future of design technology!*
+This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for more details.

packages/ducpy/src/ducpy/builders/sql_builder.py

@@ -31,12 +31,71 @@
 import os
 import sqlite3
 import tempfile
+import zlib
 from pathlib import Path
-from typing import Any, List, Optional, Union
+from typing import Any, List, Optional, Sequence, Union
 
 import ducpy_native
 
-__all__ = ["DucSQL"]
+__all__ = ["DucSQL", "quote_sql_identifier"]
+SQLITE_HEADER_MAGIC = b"SQLite format 3\x00"
+
+
+def quote_sql_identifier(value: str) -> str:
+    """Quote a SQLite identifier such as an attached database alias."""
+    if not value:
+        raise ValueError("SQLite identifier cannot be empty.")
+    return '"' + value.replace('"', '""') + '"'
+
+
+def _is_sqlite_bytes(data: bytes) -> bool:
+    return data.startswith(SQLITE_HEADER_MAGIC)
+
+
+def _normalize_sqlite_image(data: bytes) -> bytes:
+    image = bytearray(data)
+    if len(image) > 19:
+        if image[18] == 2:
+            image[18] = 1
+        if image[19] == 2:
+            image[19] = 1
+    return bytes(image)
+
+
+def _inflate_duc_bytes(data: bytes) -> bytes:
+    if _is_sqlite_bytes(data):
+        return _normalize_sqlite_image(data)
+    inflated = zlib.decompress(data, -zlib.MAX_WBITS)
+    if not _is_sqlite_bytes(inflated):
+        raise ValueError("decompressed .duc payload does not start with a SQLite header")
+    return _normalize_sqlite_image(inflated)
+
+
+def _deflate_duc_bytes(data: bytes) -> bytes:
+    compressor = zlib.compressobj(level=-1, wbits=-zlib.MAX_WBITS)
+    return compressor.compress(data) + compressor.flush()
+
+
+def _write_temp_sqlite(data: bytes) -> str:
+    tmp = tempfile.NamedTemporaryFile(suffix=".duc", delete=False)
+    try:
+        tmp.write(_inflate_duc_bytes(data))
+        tmp.close()
+        return tmp.name
+    except Exception:
+        tmp.close()
+        os.unlink(tmp.name)
+        raise
+
+
+def _sqlite_path_for_duc(path: Union[str, Path]) -> tuple[str, Optional[str]]:
+    path = str(path)
+    with open(path, "rb") as f:
+        data = f.read()
+    if _is_sqlite_bytes(data):
+        return path, None
+    temp_path = _write_temp_sqlite(data)
+    return temp_path, temp_path
 
 
 def _get_current_schema_version() -> int:
@@ -105,12 +164,14 @@ def __init__(self, path: Union[str, Path]):
         path = str(path)
         if not os.path.exists(path):
             raise FileNotFoundError(f"File not found: {path}")
-        self.conn: sqlite3.Connection = sqlite3.connect(path)
+        sqlite_path, temp_path = _sqlite_path_for_duc(path)
+        self.conn: sqlite3.Connection = sqlite3.connect(sqlite_path)
         self.conn.row_factory = sqlite3.Row
         _apply_pragmas(self.conn)
         _apply_migrations(self.conn)
-        self._path: Optional[str] = path
-        self._temp: Optional[str] = None
+        self._path: Optional[str] = sqlite_path
+        self._temp: Optional[str] = temp_path
+        self._attached_temps: list[str] = []
         self._closed = False
 
     @classmethod
@@ -126,29 +187,68 @@ def new(cls, path: Union[str, Path, None] = None) -> DucSQL:
         inst.conn.executescript(_read_schema_sql())
         inst._path = target if path else None
         inst._temp = None
+        inst._attached_temps = []
         inst._closed = False
         return inst
 
     @classmethod
     def from_bytes(cls, data: bytes) -> DucSQL:
         """Open a ``.duc`` from raw bytes (temp file, cleaned up on close)."""
-        tmp = tempfile.NamedTemporaryFile(suffix=".duc", delete=False)
+        temp_path = _write_temp_sqlite(data)
         try:
-            tmp.write(data)
-            tmp.close()
             inst = object.__new__(cls)
-            inst.conn = sqlite3.connect(tmp.name)
+            inst.conn = sqlite3.connect(temp_path)
             inst.conn.row_factory = sqlite3.Row
             _apply_pragmas(inst.conn)
             _apply_migrations(inst.conn)
-            inst._path = tmp.name
-            inst._temp = tmp.name
+            inst._path = temp_path
+            inst._temp = temp_path
+            inst._attached_temps = []
             inst._closed = False
             return inst
         except Exception:
-            os.unlink(tmp.name)
+            os.unlink(temp_path)
             raise
 
+    @classmethod
+    def attach_many(
+        cls,
+        paths: Sequence[Union[str, Path]],
+        aliases: Optional[Sequence[str]] = None,
+        read_only: bool = True,
+    ) -> DucSQL:
+        """Open an in-memory SQLite connection with multiple ``.duc`` files attached.
+
+        This is useful for querying multiple drawings in one SQL statement. The
+        returned ``DucSQL`` owns only the in-memory connection; attached files are
+        not deleted on close.
+        """
+        if aliases is not None and len(aliases) != len(paths):
+            raise ValueError("aliases must have the same length as paths.")
+
+        inst = object.__new__(cls)
+        inst.conn = sqlite3.connect(":memory:")
+        inst.conn.row_factory = sqlite3.Row
+        inst._path = None
+        inst._temp = None
+        inst._attached_temps = []
+        inst._closed = False
+
+        for index, path in enumerate(paths):
+            alias = aliases[index] if aliases is not None else f"d{index}"
+            sqlite_path, temp_path = _sqlite_path_for_duc(path)
+            if temp_path:
+                inst._attached_temps.append(temp_path)
+            inst.conn.execute(
+                f"ATTACH DATABASE ? AS {quote_sql_identifier(alias)}",
+                (sqlite_path,),
+            )
+
+        if read_only:
+            inst.conn.execute("PRAGMA query_only = ON")
+
+        return inst
+
     # ------------------------------------------------------------------
     # SQL execution
     # ------------------------------------------------------------------
@@ -186,7 +286,7 @@ def save(self, path: Union[str, Path, None] = None) -> None:
             finally:
                 dst.close()
 
-    def to_bytes(self) -> bytes:
+    def to_bytes(self, compressed: bool = False) -> bytes:
         """Export the database as raw bytes."""
         self.commit()
         tmp = tempfile.NamedTemporaryFile(suffix=".duc", delete=False)
@@ -196,7 +296,8 @@ def to_bytes(self) -> bytes:
             self.conn.backup(dst)
             dst.close()
             with open(tmp.name, "rb") as f:
-                return f.read()
+                data = f.read()
+            return _deflate_duc_bytes(data) if compressed else data
         finally:
             os.unlink(tmp.name)
 
@@ -210,6 +311,9 @@ def close(self) -> None:
             self._closed = True
             if self._temp and os.path.exists(self._temp):
                 os.unlink(self._temp)
+            for temp_path in getattr(self, "_attached_temps", []):
+                if os.path.exists(temp_path):
+                    os.unlink(temp_path)
 
     def __enter__(self) -> DucSQL:
         return self