0.0.14

0.0.14

Enhancements

• Support async batch uploads for pinecone connector
• Migrate embedders Move embedder implementations from the open source unstructured repo into this one.

Fixes

• Misc. Onedrive connector fixes

0.0.13

What's Changed

• Improve pinecone payload size handling by @vangheem in #91

Full Changelog: 0.0.12...0.0.13

0.0.12

0.0.12

Fixes

• Fix invalid replace() calls in uncompress - replace() calls meant to be on str versions of the path were instead called on Path causing errors with parameters.

0.0.11

0.0.11

Fixes

• Fix OpenSearch connector OpenSearch connector did not work when http_auth was not provided

0.0.10

0.0.10

Fixes

• Fix tar extraction Tar extraction function assumed archive was gzip compressed which isn't true for supported .tar archives. Updated to work for both compressed and uncompressed tar archives.

0.0.9

0.0.9

Enhancements

• Chroma dict settings should allow string inputs
• Move opensearch non-secret fields out of access config
• Support string inputs for dict type model fields Use the BeforeValidator support from pydantic to map a string value to a dict if that's provided.
• **Move opensearch non-secret fields out of access config

Fixes

• Fix uncompress logic Use of the uncompress process wasn't being leveraged in the pipeline correctly. Updated to use the new loca download path for where the partitioned looks for the new file.

0.0.8

Enhancements

• Add fields_to_include option for Milvus Stager Adds support for filtering which fields will remain in the document so user can align document structure to collection schema.
• Add flatten_metadata option for Milvus Stager Flattening metadata is now optional (enabled by default) step in processing the document.

0.0.7

0.0.7

Enhancements

• support sharing parent multiprocessing for uploaders If an uploader needs to fan out it's process using multiprocessing, support that using the parent pipeline approach rather than handling it explicitly by the connector logic.
• OTEL support If endpoint supplied, publish all traces to an otel collector.

Fixes

• Weaviate access configs access Weaviate access config uses pydantic Secret and it needs to be resolved to the secret value when being used. This was fixed.
• unstructured-client compatibility fix Fix an error when accessing the fields on PartitionParameters in the new 0.26.0 Python client.

0.0.6

Enhancements

Fixes

• unstructured-client compatibility fix Update the calls to unstructured_client.general.partion to avoid a breaking change in the newest version.

0.0.5

Enhancements

• Add Couchbase Source Connector Adds support for reading artifacts from Couchbase DB for processing in unstructured
• Drop environment from pinecone as part of v2 migration environment is no longer required by the pinecone SDK, so that field has been removed from the ingest CLI/SDK/
• Add KDBAI Destination Connector Adds support for writing elements and their embeddings to KDBAI DB.

Fixes

• AstraDB connector configs Configs had dataclass annotation removed since they're now pydantic data models.
• Local indexer recursive behavior Local indexer was indexing directories as well as files. This was filtered out.