CHANGELOG.md

Changelog

0.2.0

• NOTE: The minor version bump is not expected to have much effect on folks. This was done to signal the output change from the below bug fix. It is our impression not many are using this feature yet anyways. Since this was not a patch change, we decided to make this a minor instead of major change to raise less concern. Thank you for your patience!
• FEATURES:
  • Performance improvement for creating tmp tables necessary for uploading data
    • PostgreSQL dialects allow for creating a table identical to a parent table in a single command
    • CREATE TABLE <name> (LIKE <parent-name>);
    • Previously we leveraged using our upsert helpers to create new tables. This resulted in many calls to remote, of varying complexity.
• BUG FIX: No STATE Message Wrapper necessary
  • FIX LINK
  • STATE messages are formatted as {"value": ...}
  • target-potgres emitted the full message
  • The official singer-target-template, doesn't write out that value "wrapper", and just writes the JSON blob contained in it
  • This fix makes target-postgres do the same

0.1.11

• BUG FIX: canonicalize_identifier Not called on all identifiers persisted to remote
  • FIX LINK
  • Presently, on column splits/name collisions, we add a suffix to an identifier
  • Previously, we did not canonicalize these suffixes
  • While this was not an issue for any targets currently in production, it was an issue for some up and coming targets.
  • This fix simply makes sure to call canonicalize_identifier before persisting an identifier to remote

0.1.10

• FEATURES:
  • Root Table Name Canonicalization
    • The stream name is used for the value of the root table name in Postgres
    • stream names are controlled exclusively by the tap and do not have to meet many standards
    • Previously, only stream names which were lowercase, alphanumeric, etc.
    • Now, the target can canonicalize the root table name, allowing for the input stream name to be whatever the tap provides.

0.1.9

• Singer-Python: bumped to latest 5.6.1
• Psycopg2: bumped to latest 2.8.2
• FEATURES:
  • STATE Message support
    • Emits message only when all records buffered before the STATE message have been persisted to remote.
  • SSL Support for Postgres
    • Added config options for enabling/supporting SSL support.
• BUG FIX: ACTIVATE_VERSION Messages did not flush buffer
  • FIX LINK
  • When we issue an activate version record, we presently do not flush the buffer after writing the batch. This results in more records being written to remote than need to be.
  • This results in no functionality change, and should not alleviate any known bugs.
  • This should be purely performance related.

0.1.8

• Singer-Python: bumped to latest
• Minor housekeeping:
  • Updated container versions to latest
  • Updated README to reflect new versions of PostgreSQL Server

0.1.7

• BUG FIX: A bug was identified for de-nesting.
  • ISSUE LINK
  • FAILING TESTS LINK
  • FIX LINK
  • Subtables with subtables did not serialize column names correctly
    • The column names ended up having the table names (paths) prepended on them
    • Due to the denested table schema and denested records being different no information showed up in remote.
    • This bug was ultimately tracked down to the core denesting logic.
  • This will fix failing uploads which had nullable columns in subtables but no data was seen populating those columns.
    • The broken schema columns will still remain
  • Failing schemas which had non-null columns in subtables will still be broken
    • To fix will require dropping the associated tables, potentially resetting the entire db/schema

0.1.6

• BUG FIX: A bug was identified for path to column serialization.
  • LINK
  • A nullable properties which had multiple JSONSchema types
    • ie, something like [null, string, integer ...]
    • Failed to find an appropriate column in remote to persist None values to.
  • Found by usage of the Hubspot Tap

0.1.5

• FEATURES:
  • Added the persist_empty_tables config option which allows the Target to create empty tables in Remote.

0.1.4

• BUG FIX: A bug was identified in 0.1.3 with stream key_properties and canonicalization.
  • LINK
  • Discovered and fixed by @mirelagrigoras
  • If the key_properties for a stream changed due to canonicalization, the stream would fail to persist due to:
    • the persist_csv_rows key_properties values would remain un-canonicalized (sp?) and therefore cause issues once serialized into a SQL statement
    • the pre-checks for tables would break because no values could be pulled from the schema with un-canonicalized fields pulled out of the key_properties
  • NOTE: the key_properties metadata is saved with raw field names.

0.1.3

• SCHEMA_VERSION: 1
  • LINK
  • Initialized a new field in remote table schemas schema_version
  • A migration in PostgresTarget handles updating this
• BUG FIX: A bug was identified in 0.1.2 with column type splitting.
  • LINK
  • A schema with a field of type string is persisted to remote
    • Later, the same field is of type date-time
      • The values for this field will not be placed under a new column, but rather under the original string column
  • A schema with a field of type date-time is persisted to remote
    • Later, the same field is of type string
      • The original date-time column will be made nullable
      • The values for this field will fail to persist
• FEATURES:
  • Added the logging_level config option which uses standard Python Logger Levels to configure more details about what Target-Postgres is doing
    • Query level logging and timing
    • Table schema changes logging and timing