Skip to content

Fix 654 Support Alternative URLs with Production/Staging/Development Hosts Options #656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

ronilan
Copy link
Contributor

@ronilan ronilan commented May 19, 2022

Overview

This pull request enables the use of alternative s3 urls (path style) and s3-compatible backends (e.g. min.io) in conjunction with the development_host, staging_host and production_host options. It fixes #654.

The change in this pull request is non-breaking. It is fully backwards compatible with current way of defining hosts (as string) and bucket/region (as properties of binary).

This pull request comes "on top" of (i.e. includes changes from) #655 (which in turn comes "on top" of #652, which is "on top" of #651, which is "on top" of #648, #649, #650. It's pull requests all the ay down 🐢🐢🐢🐢😉).

Change

Moving forward hosts will be defined as an object with one required property endpoint and additional optional properties: bucket region and s3ForcePathStyle.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": {
      "endpoint": "https://a-bucket.s3.us-east-1.amazonaws.com",
    }
  }
}

Example 2: (Path Style alternative host)

{
  "binary": {
    "host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-production",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Example 3: (Utilizing Staging/Development)

{
  "binary": {
    "host": {
      "endpoint": "https://my-production-bucket.s3.us-east-1.amazonaws.com",
    },
    "staging_host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-staging",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    },
  }
}
{
  "binary": {
    "host": {
      "endpoint": "https://dns.over.some.bucket.example.com",
      "bucket": "the.real.bucket.name",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    },
    "staging_host": {
      "endpoint": "https://my-staging-bucket.s3.us-east-1.amazonaws.com",
    },
    "development_host": {
      "endpoint": "https://play.min.io",
      "bucket": "temp-dev-bucket",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Note: the object keys are the same as the ones passed to the AWS sdk.

Backwards compatibility

Existing string definitions and existing keys are still supported.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": "https://a-bucket.s3.us-east-1.amazonaws.com",
  }
}

Example 2: (Alternative S3 URL)

{
  "binary": {
    "host": "https://play.min.io",
    "bucket": "node-pre-gyp-production",
    "region": "us-east-1",
    "s3ForcePathStyle": true
  }
}

Example 3: (Utilizing Staging/Production for S3 Virtual Host URL)

{
  "binary": {
    "staging_host": "https://npg-bucket-staging.s3.us-east-1.amazonaws.com",
    "production_host": "https://npg-bucket-production.s3.us-east-1.amazonaws.com"
  }
}

Implementation

  • Implementation attempts to limit code changes. There was no refactoring done, not even in the vicinity of changed code.
  • Backwards compatibility is archived by adding standarize_config function to versioning. The function is called by validate_config (exported) and mutates the object holding the parsed package.json, so that any hosts (host, staging_host, development_host) are defined as objects with an endpoint key, rather than strings.
  • versioning.evaluate (exported) was adapted to work with object format of the hosts.

Tests

  • Added app1.3 - identical to app1 but using host, staging_host and development_host configuration with an object.
  • Testing of versioning was expanded and refactored.
  • All tests pass

@ronilan ronilan marked this pull request as ready for review May 19, 2022 20:32
@ronilan ronilan changed the title Fix 654 Support Alternative URLs with Production/Staging /Development Hosts Optios Fix 654 Support Alternative URLs with Production/Staging/Development Hosts Options May 19, 2022
@ronilan ronilan force-pushed the Fix-Alternative-URLs-with-Staging-Production-Option-654 branch 3 times, most recently from bbe0e41 to 0bf0db9 Compare June 30, 2024 21:42
@cclauss
Copy link
Collaborator

cclauss commented Apr 24, 2025

Please rebase now that #900 has been merged and our GitHub Actions are passing again.

@ronilan ronilan force-pushed the Fix-Alternative-URLs-with-Staging-Production-Option-654 branch from 0bf0db9 to 828b78d Compare April 24, 2025 05:50
@ronilan ronilan requested a review from a team as a code owner April 24, 2025 05:50
@ronilan ronilan force-pushed the Fix-Alternative-URLs-with-Staging-Production-Option-654 branch from 828b78d to ab8b306 Compare April 24, 2025 06:04
@ronilan ronilan force-pushed the Fix-Alternative-URLs-with-Staging-Production-Option-654 branch from ab8b306 to 32c4464 Compare April 25, 2025 05:34
ronilan and others added 14 commits April 25, 2025 09:24
…is versa. Removed previous one.

Fix CodeQL errors.
- Added a GitHub Actions workflow that runs whenever there is a push to the repo.
- Workflow includes two jobs:
    - A matrix job of node versions (10, 12, 14, 16, 18) and operating systems (Linux (ubuntu), Mac and Windows (2019 Enterprise)) that runs all tests against mock and then runs s3 tests against a bucket (located at us-east-1-bucket) specified as a repo secret.
    - A matrix job of and NW.js versions (0.64.0, 0.50.2) and node versions (10, 12, ,14, 16) that runs the NW.js test script.
- Moved logic regarding host selection to versioning where all user defined values from package.json are transformed into command options.
- Moved testing of feature from `run.test.js` to `versioning.test.js`.
- Added `development_host` option. Becomes default option for `publish` `unpublish` when present.
- Changed behavior when alternate hosts are defined. Now `production_host` acts as alias to host. Defining `staging_host` or `development_host` is enough to default `publish` and `unpublish` away from production.
- When a chain of commands that includes `publish` or `unpublish`, when host not specifically set via command line or environment variable, ALL commands in the chain default away from production.
- An invalid `s3_host` option does not result in error and is instead silently ignored.
- Change is backwards compatible with previously valid configurations.
- added a standarize_config function. function is called by validate_config and mutates the object holding the parsed package.json, so that hosts are defined as objects with endpoint key, rather than strings.
- modified versioning.evaluate to work with object format of hosts.
- added app1.3 - identical to app1 but using host, staging_host and development_host configuration with an object.
- expanded and refactored versioning testing.
- modified bucket switching script to work with new package.json format.
…ost option when package.json has only host key specified.
@ronilan ronilan force-pushed the Fix-Alternative-URLs-with-Staging-Production-Option-654 branch from 32c4464 to dda6d5c Compare April 25, 2025 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can't use Alternative S3 URLs with Staging/Production Option
2 participants