Skip to content

Creating an NA <-> EU migration option #3005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

Chaffexd
Copy link

@Chaffexd Chaffexd commented May 9, 2025

I've spent some time adding functionality to allow bi-directional space transfer(s) between NA and EU data centers as this functionality doesn't currently exist.

How it works:
contentful migrate datacenter --help ✔    at 18:35:19 

migrate datacenter

Migrate a space between Contentful data centers

Options:
-h, --help Show help [boolean]
--source Source data center (na or eu)
[required] [choices: "na", "eu"]
--target Target data center (na or eu)
[required] [choices: "na", "eu"]
--source-space-id Source space ID [string] [required]
--target-space-id Target space ID [string] [required]
--environment-id Environment ID (e.g. master)
[string] [default: "master"]
--source-token Source CMA token [string] [required]
--target-token Target CMA token [string] [required]
--use-verbose-renderer Display progress in new lines instead of spinner
[boolean] [default: false]
--include-taxonomies Migrate taxonomies (Concepts & Concept Schemes)
before importing content [boolean] [default: false]
--source-org-id Organization ID where taxonomies should be exported
from [string]
--target-org-id Organization ID where taxonomies should be imported
to [string]
--skip-content-publishing Skips content publishing. Creates content but does
not publish it [boolean] [default: false]

An example:
contentful migrate datacenter
--source na
--target eu
--source-space-id
--target-space-id
--source-token
--target-token
--source-org-id
--target-org-id
--include-taxonomies
--skip-content-publishing

I have manually tested this and it works both ways, it allows you to include taxonomies if you use them as otherwise the space import will fail, if you don't use them then you can simple use --include-taxonomies false

@Chaffexd Chaffexd requested a review from a team as a code owner May 9, 2025 16:18
@wiz-inc-38d59fb8d7
Copy link

wiz-inc-38d59fb8d7 bot commented May 9, 2025

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities
Data Finding Sensitive Data
Secret Finding Secrets
IaC Misconfiguration IaC Misconfigurations
Total

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

@Chaffexd Chaffexd changed the title Creating an NA -> EU migration option Creating an NA <-> EU migration option May 11, 2025

if (argv.includeTaxonomies) {
if (!argv.sourceOrgId || !argv.targetOrgId) {
throw new Error(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional suggestion, you can use yargs implies instead of manually validating/throwing.

await organizationExport({
context: { managementToken: argv.sourceToken },
organizationId: argv.sourceOrgId,
header: argv.header,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

header needs to be a defined property in the yargs right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after review, its actually deadweight so I'm removing it

`${emojic.inboxTray} Importing taxonomies into target organization ${argv.targetOrgId}...`
)

const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question, is there a benefit to requiring inline here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at the time of writing, I just put them there but moving them to be globally available seems unnecessary

@ethan-ozelius-contentful
Copy link
Contributor

Hello @Chaffexd, thanks a bunch for investing the time to improve this tool. I left a couple non-blocking questions/suggestions but otherwise the code looks very clean and well tested.

After discussing with the Developer Experience PM, we have a couple thoughts around the name of the command migrate-datacenter. And while we fully acknowledge that we're bikeshedding here, we have 2 ideas we'd love your thoughts on:

  1. instead of using datacenter, use region. Contentful doesn't have physical datacenters, and conceptually I defer to the AWS regional model, where you can pick a region like US East (Northern Virginia) Region or Europe (Frankfurt) Region. But if you think our customers would understand datacenter more easily, we're open to it.
  2. instead of using migration, use sync? There already exists the concept of migration in the cli tool, which is a migration in the sense that data is being mutated. This PR is more exporting/importing data from different continents, but not making any changes to the underlying data.

Let us know what you think!

Lastly, this PR would really benefit from having a dedicated doc for the new command, similar to other commands have readme with basic descriptions/options/limitations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file and the tmp_export directory should be reverted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants