Open
Description
I tried running this target (via meltano) with postgres_schema
set to an existing schema which already contained tables (with data). Some of the tables had comments on them, as created by:
COMMENT ON TABLE public.mytable IS ''my custom table with important data';
When running the target, it crashed with the following stacktrace:
target-postgres | ERROR Exception writing records
target-postgres | Traceback (most recent call last):
target-postgres | File "~/myproject/.meltano/loaders/target-postgres/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 237, in write_batch
target-postgres | self.setup_table_mapping_cache(cur)
target-postgres | File "~/myproject/.meltano/loaders/target-postgres/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 224, in setup_table_mapping_cache
target-postgres | table_path = json.loads(raw_json).get('path', None)
target-postgres | File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/__init__.py", line 357, in loads
target-postgres | return _default_decoder.decode(s)
target-postgres | File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/decoder.py", line 337, in decode
target-postgres | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
target-postgres | File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/decoder.py", line 355, in raw_decode
target-postgres | raise JSONDecodeError("Expecting value", s, err.value) from None
target-postgres | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
target-postgres | CRITICAL ('Exception writing records', JSONDecodeError('Expecting value: line 1 column 1 (char 0)'))
With some extra logging, raw_json
actually contains the value of the comment mentioned above, which clearly isn't JSON.
This is the line of code that breaks
target-postgres/target_postgres/postgres.py
Line 223 in 9c095d9
Is this a bug, or does target-postgres
expect the schema to be empty when it starts?
Metadata
Metadata
Assignees
Labels
No labels