v0.0.8

What's Changed explained

Updated arrow to the latest version and some minor tweaks to the readme, now you can see a benchmark result!!

What's Changed

• Improve benchmark and add results to README.md by @surister in #64
• Bump postgres from 0.19.11 to 0.19.12 in /conecta-core by @dependabot[bot] in #60
• Update arrow to 57.0.0 in 8bfee40
• Update pyo3 and pyo3-arrow to latest (0.27.1, 0.15.0) in 2120da1

Full Changelog: v0.0.7...v0.0.8

v0.0.7

What's changed explained

• PostGis is now supported:

from conecta import read_sql
table = read_sql(
    "postgres://postgres:postgres@localhost:32789/test" ,
    query="""
    SELECT 
        ST_GeomFromText( 'POLYGON((0 0,0 1,1 1,1 0,0 0))', 4326 ) as geo1,
        ST_AsEWKB( ST_GeomFromText('POLYGON((0 0,0 1,1 1,1 0,0 0))', 4326) ) as geo2,
        ST_AsBinary( ST_GeomFromText('POLYGON((0 0,0 1,1 1,1 0,0 0))', 4326) ) as geo3,
        ST_AsText( ST_GeomFromText( 'POLYGON((0 0,0 1,1 1,1 0,0 0))', 4326 ) ) as geo4
    """,
)

print(table)
# pyarrow.Table
# geo1: binary
# geo2: binary
# geo3: binary
# st_astext: string
# ----
# geo1: [[0103000020E61000000100000005000000000000000000000000000000000000000000000000000000000000000000F03F00 (... 94 chars omitted)]]
# geo2: [[0103000020E61000000100000005000000000000000000000000000000000000000000000000000000000000000000F03F00 (... 94 chars omitted)]]
# geo3: [[01030000000100000005000000000000000000000000000000000000000000000000000000000000000000F03F0000000000 (... 86 chars omitted)]]
# geo4: [["POLYGON((0 0,0 1,1 1,1 0,0 0))"]]

# pip install geoarrow-pyarrow
from geoarrow.pyarrow import as_geoarrow, as_wkt, to_geopandas

for column in table:
    print(as_wkt(as_geoarrow(column)))
# [
#   [
#     "POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))"
#   ]
# ]
# [
#   [
#     "POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))"
#   ]
# ]
# [
#   [
#     "POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))"
#   ]
# ]
# [
#   [
#     "POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))"
#   ]
# ]

for column in table:
    print(to_geopandas(as_geoarrow(column)))

# 0    POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))
# dtype: geometry
# 0    POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))
# dtype: geometry
# 0    POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))
# dtype: geometry
# 0    POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))
# dtype: geometry

• Postgres native datatypes are supported as lists of floats.
• Postgres binary datatypes are supported now.

What's Changed

• Feat/pg geo types by @surister in #56
• Implement Polygon, Path, Lseg, Box, and Line by @surister in #58
• Feat/pg binary types by @surister in #61
• Feat/postgres postgis by @surister in #63

Full Changelog: v0.0.6...v0.0.7

v0.0.6

Changes explained:

• Stabilize API:
  • now: read_sql queries parameter was renamed to query for better interop with other libraries
  • disable_preallocation was renamed to preallocation
  • preallocation was set to default False
• Fix bug where return_backend was not testing the right imports
• Add integration tests in the CI
• Restructure documentation
• Add method to use conecta with polars easily patch_polars and unpatch_polars
• Add array_text and array_boolean datataypes to Postgres
• Update all dependencies to latest

What's Changed

• Add postgres's datatype array_text by @surister in #42
• Add postgres' datatype array_boolean by @surister in #43
• Bump chrono from 0.4.41 to 0.4.42 in /conecta-core by @dependabot[bot] in #40
• Bump serde from 1.0.219 to 1.0.223 in /conecta-core by @dependabot[bot] in #41
• Bump serde_json from 1.0.143 to 1.0.145 in /conecta-python by @dependabot[bot] in #39
• Re-structure docs to follow diátaxis.fr by @surister in #44
• Add patch_polars and unpatch_polars. by @surister in #45
• Feat/re implement partition plan by @surister in #47
• Feat/stabilize read sql api by @surister in #53
• Bump sqlparser from 0.58.0 to 0.59.0 in /conecta-core by @dependabot[bot] in #51
• Bump postgres from 0.19.10 to 0.19.11 in /conecta-core by @dependabot[bot] in #50
• Bump actions/setup-python from 5 to 6 by @dependabot[bot] in #54
• Bump serde from 1.0.223 to 1.0.228 in /conecta-core by @dependabot[bot] in #49

Full Changelog: v0.0.5...v0.0.6

v0.0.5

Changes explained

• Add support for the following postgres datataypes:
  UUID, UUID_ARRAY, INT2_ARRAY, INT8_ARRAY, FLOAT4_ARRAY and FLOAT8_ARRAY
• Add metadata to python package, to have more information in pypi's page

What's Changed

• Add support for more postgres datatypes by @surister in #38

Full Changelog: v0.0.4...v0.0.5

v0.0.4

Changes explained

• Dependencies updates, most important one (arrow to latest 56.1.0)

What's Changed

• Bump rayon from 1.10.0 to 1.11.0 in /conecta-core by @dependabot[bot] in #26
• Bump serde_json from 1.0.142 to 1.0.143 in /conecta-python by @dependabot[bot] in #31
• Add support for new postgres datatypes. by @surister in #32
• Bump arrow from 56.0.0 to 56.1.0 in /conecta-core by @dependabot[bot] in #35
• Bump actions/checkout from 4 to 5 by @dependabot[bot] in #33
• Bump log from 0.4.27 to 0.4.28 in /conecta-core by @dependabot[bot] in #36
• Bump log from 0.4.27 to 0.4.28 in /conecta-python by @dependabot[bot] in #37
• Bump actions/attest-build-provenance from 2 to 3 by @dependabot[bot] in #34
• Bump flamegraph from 0.6.8 to 0.6.9 in /conecta-core by @dependabot[bot] in #30

Full Changelog: v0.0.3...v0.0.4

v0.0.3

Changes explained:

• Added an utility function to bind parameters to a query:

from conecta import sql_bind
>>> sql_bind('select IDENT(:col) FROM tbl1 t WHERE t.value = :var1 and t.name = :var2', {'col': 'col1', 'var1': 1, 'var2':'somename'})
select "col1" FROM tbl1 t WHERE t.value = 1 and t.name = 'somecol'

• Add more documentation
• Queries in postgres source now have alias in subqueries
• Conecta now has interfaces to implement other databases
• Postgres source now supports: INT2, BPCHAR, VARCHAR, BOOL, TIMESTAMP, TIME and INT4_ARRAY types

What's Changed

• Bump actions/download-artifact from 4 to 5 by @dependabot[bot] in #20
• Add bind_sql utility to the python package. by @surister in #22
• Feat/more docs 2 by @surister in #25
• Refactor/multiple sources by @surister in #27
• Add aliases to all queries in postgres.rs. by @surister in #28
• Add support for many new postgres datatypes. by @surister in #29

Full Changelog: v0.0.2...v0.0.3

v0.0.2

Changes explained

• There is now documentation at https://conecta.surister.dev
• conecta.read_sql now pre-allocate memory before start loading data
• Implement debug logging, it can be set with conecta.set_debug_log(mode)
• Update arrow to latest (56.0)
• Add the configuration parameter disable_preallocation to disable pre-allocation of memory, it also disables the count(*) query.

What's Changed

• Get the total count of every partition and pre-allocate arrays before loading data by @surister in #11
• Write more documentation pages by @surister in #12
• Implement set_debug_log to easily set different debug loggers inside conecta. by @surister in #13
• Implement max_pool_size. by @surister in #14
• Add conecta-benchmark. by @surister in #16
• Bump pyo3-arrow from 0.10.1 to 0.11.0 in /conecta-python by @dependabot[bot] in #17
• Bump arrow from 55.2.0 to 56.0.0 in /conecta-core by @dependabot[bot] in #19
• Bump serde_json from 1.0.141 to 1.0.142 in /conecta-python by @dependabot[bot] in #18
• Pre-allocation improvements by @surister in #15

Full Changelog: v0.0.1...v0.0.2

v0.0.1

First release of conecta, the package can be installed from https://pypi.org/project/conecta/

conecta.read_sql and conecta.create_partition_plan are now available.

What's Changed

• Bump tokio from 1.45.1 to 1.46.1 in /conecta-core by @dependabot[bot] in #5
• Bump peak_alloc from 0.2.1 to 0.3.0 in /conecta-core by @dependabot[bot] in #4
• Bump pyo3 from 0.25.0 to 0.25.1 in /conecta-python by @dependabot[bot] in #2
• Bump arrow from 55.1.0 to 55.2.0 in /conecta-core by @dependabot[bot] in #3
• Implement metadata fetching and partitioning by @surister in #1

New Contributors

• @dependabot[bot] made their first contribution in #5
• @surister made their first contribution in #1

Full Changelog: https://github.com/surister/conecta/commits/v0.0.1