SAC-30337: Add dynamic schema workflow with discovery support#1
Merged
satyendra101 merged 8 commits intogl-masterfrom Apr 1, 2026
Merged
SAC-30337: Add dynamic schema workflow with discovery support#1satyendra101 merged 8 commits intogl-masterfrom
satyendra101 merged 8 commits intogl-masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a metadata-driven (OData $metadata) discovery workflow to dynamically generate Singer catalogs/schemas and introduces a runtime stream implementation to sync discovered entity sets from SAP SuccessFactors, including an OAuth2 (SAML bearer / refresh token / static token) authentication client.
Changes:
- Implement dynamic discovery by parsing OData
$metadatainto Singer schemas + metadata and emitting a generated catalog. - Add a dynamic stream/sync framework (including parent/child relationship inference) with schema emission and bookmark handling.
- Introduce an HTTP client with OAuth token refresh and mapped error handling for SAP SuccessFactors responses.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
tap_sap_success_factors/__init__.py |
Tap entrypoint wiring for discover/sync execution and required config keys. |
tap_sap_success_factors/auth.py |
Builds OAuth token request payloads (SAML bearer / refresh token). |
tap_sap_success_factors/client.py |
Adds HTTP client, token management, retries/backoff, and error mapping. |
tap_sap_success_factors/discover.py |
Runs dynamic discovery and constructs a Singer Catalog. |
tap_sap_success_factors/exceptions.py |
Defines exception hierarchy and HTTP status → exception mapping. |
tap_sap_success_factors/metadata_discovery.py |
Parses OData $metadata and infers schemas/relationships/replication metadata. |
tap_sap_success_factors/schema.py |
Emits Singer schema messages and wires child streams for syncing. |
tap_sap_success_factors/sync.py |
Orchestrates syncing selected streams with parent/child handling. |
tap_sap_success_factors/streams/__init__.py |
Declares streams package. |
tap_sap_success_factors/streams/abstracts.py |
Implements the base stream sync logic (pagination, bookmarks, child sync). |
tap_sap_success_factors/streams/dynamic.py |
Implements runtime stream instances backed by discovered catalog entries. |
sample_config.json |
Updates sample config keys to match the new client configuration model. |
README.md |
Updates example configuration to reflect new config keys. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix grammar: 'due permissions' -> 'due to permissions' in sync.py - Fix update_currently_syncing to use state.pop() avoiding null state write - Update type hint to Optional[str] on update_currently_syncing - Use tap_stream_id consistently and add parent-stream fallback in child_map - Pass pre-built child_map to write_schema to avoid O(n) catalog re-traversal - Handle 204 No Content and empty bodies in _make_request to avoid JSON decode errors - Guard int() cast in _get_retry_after with ValueError fallback for HTTP-date headers
* SAC-30338: Add sync logic for the tap along with unittests and spikes * Update tests to use unittests * Resolve Copilot comments * Add discovery time stream probe and prune strategy * Remove redundant files and code lines * Integration tests (#3) * Add Integration tests for the tap * Update integration testsand yml file * Resolve Copilot Comments
RushiT0122
approved these changes
Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of change
Manual QA steps
Risks
Rollback steps