Commit 744f6a8
authored
fix: Fix another compatibility issue for unstructured-client==0.26.0 (#79)
* fix: Fix another compatibility issue for unstructured-client==0.26.0
The internals of the client have changed, and the `PartitionParameters` class is now a Pydantic
model. We were using `dataclass.fields` to get a list of available params, which will break against
the new client. For compatibility, let's try to access `PartitionParameters.__fields`, and fall back
to the dataclass approach for clients < 0.26.0.
Testing:
Try a simple ingest command with the old and new clients:
```
pip install unstructured-client==0.25.5
unstructured-ingest local --input-path /path/to/some/files --output-dir local-ingest-output --num-processes 2 --recursive --partition-by-api --partition-endpoint https://api.unstructuredapp.io/general/v0/general --api-key $MY_SERVERLESS_KEY
pip install git+https://github.com/Unstructured-IO/unstructured-python-client
Traceback (most recent call last):
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/pipeline/interfaces.py", line 158, in run_async
return await self._run_async(fn=fn, **kwargs)
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/pipeline/steps/partition.py", line 66, in _run_async
partitioned_content = await fn(**fn_kwargs)
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 215, in run_async
return await self.partition_via_api(filename, metadata=metadata, **kwargs)
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/utils/dep_check.py", line 50, in wrapper_async
return await func(*args, **kwargs)
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 200, in partition_via_api
partition_params = self.create_partition_parameters(filename=filename)
File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 167, in create_partition_parameters
possible_fields = [f.name for f in fields(PartitionParameters)]
File "/Users/austin/.pyenv/versions/3.9.16/lib/python3.9/dataclasses.py", line 1035, in fields
raise TypeError('must be called with a dataclass type or instance')
pip install -e .
```
* Cut the 0.0.7 release1 parent 6dcdae8 commit 744f6a8
File tree
3 files changed
+11
-3
lines changed- unstructured_ingest
- v2/processes
3 files changed
+11
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
167 | | - | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
168 | 175 | | |
169 | 176 | | |
170 | 177 | | |
| |||
0 commit comments