docs: update overview and add ITX wrappers component diagram#68
Open
docs: update overview and add ITX wrappers component diagram#68
Conversation
Replace the outdated Overview section (which described a now-abandoned 'v1 meetings vs. native v2 meetings' split) with an accurate two-point summary of what this repository actually does: 1. Real-time streaming replication of PostgreSQL and DynamoDB data into the v1-objects NATS KV bucket, consumed by v2 wrapper services for indexing and access-control pipelines. 2. Bidirectional sync for core (non-ITX) resources — projects and committees — giving LFX One a self-contained stack that removes the dependency on the LFX/Salesforce/ITX stack in developer environments. Add a cross-reference to the new component diagram in the Overview for readers who want the big-picture view. Add the ITX wrappers component diagram (flowchart) as the first subsection of Architecture Diagrams, showing how the LFX One platform, the v1-sync-helper replication pipeline, and ITX-hosted services fit together. Update the Architecture Diagrams preamble bullet to replace the stale mention of v1 Meetings pushing directly to OpenSearch/OpenFGA with an accurate note pointing to the new component diagram. 🤖 Generated with [GitHub Copilot](https://github.com/features/copilot) (via Zed) Signed-off-by: Eric Searcy <eric@linuxfoundation.org>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates repository documentation to reflect the current ITX “wrapper services + NATS KV replication” architecture and replaces outdated meetings-v1/v2 split content.
Changes:
- Rewrites the README Overview section to describe (1) real-time replication into the
v1-objectsNATS KV bucket and (2) bidirectional sync for core resources. - Adds an “ITX wrappers component diagram” Mermaid flowchart to the Architecture Diagrams section.
- Updates the Architecture Diagrams preamble to point readers to the new component diagram rather than the stale meetings note.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| FGA | ||
| JoinFlow | ||
| end | ||
| 2. **Bidirectional sync for "core" resources.** Projects and committees are fully synced in both directions between LFX v1 and LFX One. This gives LFX One a self-contained stack for these entity types, which simplifies developer environment stand-up by removing the dependency on the highly-interconnected LFX/Salesforce/ITX stack. |
README.md
Outdated
Comment on lines
+85
to
+86
| - The planned bidirectional sync (LFX One changes back to v1) is included in the diagrams. | ||
| - "Projects API" is representative of most data entities. However, v1 Meetings push straight to OpenSearch and OpenFGA (via platform services)—this is not shown. | ||
| - "Projects API" is representative of the core resources that have bidirectional sync (projects, committees). ITX-hosted resources such as Meetings are handled by wrapper services that subscribe to the NATS KV bucket instead—see the component diagram below. |
Add DynamoDB and dynamodb-stream-consumer as participants in the Live data sync section of both the Data extraction/replication and Combined sequence diagrams, mirroring the existing wal-listener pattern: - LFX v1 API writes to DynamoDB via ITX API (create/update/delete). - DynamoDB fires a Streams event to dynamodb-stream-consumer. - dynamodb-stream-consumer notifies v1-sync-helper on the "dynamodb_streams" subject. Each source (wal-listener and dynamodb-stream-consumer) has its own "store record (or soft-deletion)" step on v1-sync-helper. Add DynamoDB Scan tables / results to the Meltano backfill section of both diagrams, alongside the existing Postgres query, reflecting that Meltano also backfills from DynamoDB tables (>= LAST_MONTH, with full re-scan also supported). Remove the now-stale preamble caveats: - "The DynamoDB source is not currently included in the diagrams" — it is now included. - "The planned bidirectional sync is included in the diagrams" — bidirectional sync has shipped; no caveat needed. 🤖 Generated with [GitHub Copilot](https://github.com/features/copilot) (via Zed) Signed-off-by: Eric Searcy <eric@linuxfoundation.org>
andrest50
approved these changes
Mar 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the README Overview section and Architecture Diagrams section to reflect the current "wrapper" strategy for ITX-hosted resources (e.g. Meetings), and brings the sequence diagrams up to date with the current implementation.
Changes
Overview section — replaces the stale text + Mermaid flowchart (which described an abandoned "v1 meetings vs. native v2 meetings" split) with a concise two-point description of what this repo actually does:
v1-objectsNATS KV bucket, consumed by v2 wrapper services for indexing and access-control pipelines.Architecture Diagrams — new first subsection — adds the ITX wrappers component diagram (flowchart), showing how Traefik, Heimdall, wrapper services, v1-sync-helper, the NATS KV bucket, and the ITX AWS infrastructure all relate. Source:
2026-01-LFX-v2-ITX-wrappers/architecture.mmd.Architecture Diagrams preamble — removes two now-stale caveats ("DynamoDB not included" and "bidirectional sync is planned") and updates the remaining bullet to replace the stale mention of v1 Meetings pushing directly to OpenSearch/OpenFGA.
Data extraction/replication and Combined sequence diagrams — both updated with:
dynamodb-stream-consumer, which notifies v1-sync-helper on thedynamodb_streamssubject. Each source (wal-listener and dynamodb-stream-consumer) now has its own "store record (or soft-deletion)" step.>= LAST_MONTH(alongside the existing Postgres query), with full re-scan also supported.