-
Notifications
You must be signed in to change notification settings - Fork 3
Description
This came to mind today as I was considering the effect of currently_syncing, this is a standard top-level state key (via singer-python) that allows a tap to resume from the last stream it was syncing upon interruption. Just setting it and skipping streams on resume until you get to the correct stream is fine, but there are some best practices here. E.g., sort streams into an expected order always, reshuffle streams instead of skipping so that all streams will get attempted per successful run, etc.
In what I'm currently thinking of as the Singer "Standard" (a level above the spec where discovery, metadata, catalogs, etc. live), I'm sure there are a lot more semantic usages of standard metadata and/or other state keys that should be understood and documented in a discoverable location.