[cli] multi identifier preference record uploads#11
Open
bencmbrook wants to merge 16 commits intomainfrom
Open
[cli] multi identifier preference record uploads#11bencmbrook wants to merge 16 commits intomainfrom
bencmbrook wants to merge 16 commits intomainfrom
Conversation
Bring the legacy CLI preference-upload refactor into the monorepo, including the new configure flow, schema-backed parallel uploads, receipts state, the upload skill, and the reconcile helper script. Made-with: Cursor
@transcend-io/cli
@transcend-io/privacy-types
@transcend-io/sdk
@transcend-io/utils
commit: |
bencmbrook
changed the title
bencmbrook
changed the title
Member
Author
|
Relative to old Ported Or Adapted
Replaced Or Dropped
Net Effect
|
… re-exports Made-with: Cursor
Resolve conflicts from SDK extraction (PR #34) merging into multi-identifier feature branch: - Take HEAD's feature code for CLI preference-management files - Restore HEAD's enhanced versions (FileFormatState, multi-identifier, split-on-validation in getPreferencesForIdentifiers) - Fix imports: splitCsvToList from @transcend-io/utils, withPreferenceRetry and ConsentPreferenceResponse from @transcend-io/sdk - Delete uploadPreferenceManagementPreferencesInteractive (replaced by pool) - Take main's mise.lock and SDK getPreferencesForIdentifiers Made-with: Cursor
Delete 6 CLI files that were duplicated in SDK after the merge: - codecs.ts, getPreferenceMetadataFromRow.ts (identical) - checkIfPendingPreferenceUpdatesAreNoOp.ts, checkIfPendingPreferenceUpdatesCauseConflict.ts, getPreferenceUpdatesFromRow.ts (only import path diffs) - getPreferencesForIdentifiers.ts (ported enhanced split-on-validation logic into SDK version with Logger DI) Move PreferenceUploadProgress type to SDK types.ts so SDK getPreferencesForIdentifiers can use it. Update all CLI consumers to import from @transcend-io/sdk directly. SDK typecheck pass | CLI typecheck pass | 86 tests pass Made-with: Cursor
UploadProgressSink interface: - New packages/sdk/src/preference-upload/progress.ts with structured callbacks (onFileStart, onFileProgress, onFileComplete, onError, onJobComplete) for CLI receipts/dashboard, container progress.json, or agent events - noopProgressSink for when reporting isn't needed Upload core moved to SDK (packages/sdk/src/preference-upload/): - batchUploader.ts — Logger DI, stripped colors - loadReferenceData.ts — Logger DI, calls SDK fetchAll* - buildPendingUpdates.ts — pure transforms using SDK codecs - getPreferenceIdentifiersFromRow + NONE_PREFERENCE_MAP moved to SDK preference-management (pure functions, no CLI deps) Pooling moved to SDK (packages/sdk/src/pooling/): - runPool.ts — stripped colors, installInteractiveSwitcher now injectable callback (CLI passes it, container skips it) - spawnWorkerProcess.ts — openLogWindows replaced with onLogFilesCreated callback - types.ts, logRotation.ts, ensureLogFile.ts, safeGetLogPathsForSlot.ts CLI rewired to import from @transcend-io/sdk for moved modules. SDK typecheck ✓ | SDK build ✓ | CLI typecheck ✓ | 154 tests pass ✓ No colors in SDK ✓ Made-with: Cursor
Upload pipeline: - Delete CLI batchUploader.ts, loadReferenceData.ts, buildPendingUpdates.ts, types.ts (all moved to SDK preference-upload/) - Clean up upload/index.ts and transform/index.ts barrels - Move loadReferenceData.test.ts to SDK package Pooling: - Delete CLI runPool.ts, types.ts, spawnWorkerProcess.ts, logRotation.ts, ensureLogFile.ts, safeGetLogPathsForSlot.ts (all moved to SDK pooling/) - Update CLI pooling barrel to only export CLI-specific files - Rewire 15+ consumer files to split imports: SDK items from @transcend-io/sdk, CLI items from CLI pooling barrel Note: Some pooling tests need migration to SDK (mock targets reference deleted local modules). This is a known follow-up. SDK typecheck ✓ | CLI typecheck ✓ Made-with: Cursor
…ific Move packages/sdk/src/pooling/ to packages/utils/src/pooling/ since runPool, spawnWorkerProcess, logRotation, and types are generic multi-process infrastructure (like chunkOneCsvFile, RateCounter) not Transcend API-specific code. - Fix RateCounter import to relative (now within same package) - Add @transcend-io/type-utils dependency to utils package.json - Update utils barrel, remove pooling from SDK barrel - Update ~34 CLI consumer files: pooling imports from @transcend-io/utils Utils typecheck ✓ | Utils build ✓ | SDK typecheck ✓ | CLI typecheck ✓ Made-with: Cursor
| import { | ||
| CHILD_FLAG, | ||
| type PoolHooks, | ||
| runPool, |
Member
There was a problem hiding this comment.
if (err instanceof PoolCancelledError) {
Resolve conflicts by taking main's polished pooling versions. Fix duplicate imports, add installInteractiveSwitcher + PoolCancelledError to upload-preferences, add worker.ts as build entry point. Made-with: Cursor
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrated from @JonnavithulaGirish PR at transcend-io/cli#444
Summary
packages/cli, includingconsent configure-preference-upload, schema-backed uploads, worker orchestration, receipts state, and the upload transform pipeline.cursorpreference upload skill, movereconcile-preference-recordsintopackages/cli/scripts, and regenerate the CLI README/schema artifacts in the monorepo layoutTest plan
pnpm -F @transcend-io/cli genfilespnpm -F @transcend-io/cli typecheckpnpm -F @transcend-io/cli buildpnpm -F @transcend-io/cli testpnpm -F @transcend-io/cli check-exportspnpm qualityMade with Cursor