Skip to content

Use perl instead of replace for dependency updates #103216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 8, 2025
Merged

Conversation

aduth
Copy link
Member

@aduth aduth commented May 7, 2025

Proposed Changes

  • Removes usage of the replace dependency, replacing with equivalent sed perl command

Why are these changes being made?

  • Use natively-available tools where feasible and where readability is not severely impacted
  • Reduce number of dependencies: 22 unique dependencies removed
  • Improve performance: We don't need to recursively search the code for yarn.lock files. There's only one.

Testing Instructions

I don't know the specific scenario where a http: URL would be added to yarn.lock, but it's easy enough to replicate by manually editing one or more https URLs in yarn.lock to http and running the perl portion of the command to verify that the replacements are made and written in-line.

diff --git a/yarn.lock b/yarn.lock
index c6ec62b1a6d..05bcfe2c217 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -6878,5 +6878,5 @@ __metadata:
 
-"@php-wasm/fs-journal@https://wpcomplaygroundpackages.wpcomstaging.com/packages/v1.0.31/@php-wasm-fs-journal-1.0.31.tar.gz":
+"@php-wasm/fs-journal@http://wpcomplaygroundpackages.wpcomstaging.com/packages/v1.0.31/@php-wasm-fs-journal-1.0.31.tar.gz":
   version: 1.0.31
-  resolution: "@php-wasm/fs-journal@https://wpcomplaygroundpackages.wpcomstaging.com/packages/v1.0.31/@php-wasm-fs-journal-1.0.31.tar.gz"
+  resolution: "@php-wasm/fs-journal@http://wpcomplaygroundpackages.wpcomstaging.com/packages/v1.0.31/@php-wasm-fs-journal-1.0.31.tar.gz"
   dependencies:
perl -pi -e s,http://,https://,g yarn.lock

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
    • For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@aduth aduth requested a review from a team May 7, 2025 18:30
@matticbot matticbot added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. labels May 7, 2025
@matticbot
Copy link
Contributor

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

@matticbot
Copy link
Contributor

matticbot commented May 7, 2025

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • blaze-dashboard
  • command-palette-wp-admin
  • help-center
  • notifications
  • odyssey-stats
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug remove/replace on your sandbox.

Copy link
Member

@tyxla tyxla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find @aduth! 👍

Just a minor question on cross-platform support

@aduth aduth changed the title Use sed instead of replace for dependency updates Use perl instead of replace for dependency updates May 8, 2025
@@ -138,7 +138,7 @@
"translate:calypso": "wp-babel-makepot './{client,packages,apps}/**/*.{js,jsx,ts,tsx}' --ignore '**/node_modules/**,**/test/**,**/*.d.ts' --base './' --dir './build/strings' --output './public/calypso-strings.pot'",
"tsc": "NODE_OPTIONS='--max-old-space-size=4096' tsc",
"typecheck": "yarn run tsc --project client",
"update-deps": "rm -rf yarn.lock && yarn run distclean && yarn install && replace --silent 'http://' 'https://' . --recursive --include='yarn.lock' && touch -m node_modules",
"update-deps": "rm -rf yarn.lock && yarn run distclean && yarn install && perl -pi -e s,http://,https://,g yarn.lock && touch -m node_modules",
Copy link
Member Author

@aduth aduth May 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a side-point: I'm somewhat confused by some of what this command is doing, and how we use it in our workflows.

On what this command is doing: Is it strictly necessary to include distclean and the touch parts of this command? It's not clear what impact those would have on a script presumably designed to update our transitive dependencies. It's also unclear what circumstances (if any) actually result in http:// links being included, which sounds like the sort of thing that could be a bug or edge-case no longer relevant in recent versions of Yarn.

On how we use it in our workflows: While I can imagine some value in having a script like this to facilitate a task of updating transitive dependencies, I've been unable to find any documentation or internal references that mention it. I don't think it makes sense to include scripts here that aren't part of some documented procedure and require a developer to be aware of its existence through close examination of package.json.

Personally, I'd rather take the opposite stance, and remove any and all commands that aren't directly referenced as part of CI or documented workflows. And if that disrupts someone's own personal workflow, then the onus should be on them to either make it a documented or CI-integrated process, or find another way to use it locally if it's not valuable to be common to the project.

Copy link
Contributor

@ciampo ciampo May 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we look at git history, can we find why this code was introduced? We may have more context and be able to make a better decision about keeping it or removing it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we look at git history, can we find why this comment was introduced? We may have more context and be able to make a better decision about keeping it or removing it.

Good call! It looks like it has a very long history, originating back to an old Makefile task added in #5386 (reviewed by a familiar face! 😂 ) and was originally documented in a docs/shrinkwrap.md file. I'm guessing as we switched to Yarn the documentation was lost or replaced, resulting in the command not being referenced anywhere.

The http: stuff itself was added in #29718, which specifically dealt with a NPM issue. I'd imagine it probably doesn't even apply for Yarn.

In summary, I think we could probably just remove this command altogether.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, let's get rid of it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in #103307

@aduth aduth merged commit 9491e22 into trunk May 8, 2025
15 of 16 checks passed
@aduth aduth deleted the remove/replace branch May 8, 2025 16:53
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label May 8, 2025
@aduth aduth mentioned this pull request May 9, 2025
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants