Description
This issue is intended to track & discuss any and all issues related to old dependencies:
Issues tracking dependency work
Core dependency updates needed (priority)
- feat: update ipfs-http-client #1840
- Replace
ipld-dag-pbwith@ipld/dag-pbipld-explorer-components#329- PR is out for this: feat!: replace
ipld-dag-pbwith@ipld/dag-pbipld-explorer-components#330
- PR is out for this: feat!: replace
- Remove dependency on ipld-git #2045
- Remove dependency on ipld-ethereum #2046
- Remove dependency on ipfs-block-service #2047
- Update to the latest multiformats package #2048
Other work
- chore: update ipld-explorer/i18n deps #1924
- chore(deps): update react-scripts #1969
- chore: install missing dependencies #1981
- chore: Update storybook #1982
- chore: Remove engines-strict and update package.json->engines #1987
- update
ipfsandipfsd-ctl#1988 - chore: remove unused dependencies #2015
- remove unused exports to clean up code #2036
Nice to have but not required
Items blocked by outdated deps in one way or another
- feat: import DAG CAR #1838
- Can't build with nodejs v17 and npm v8 #1940
- npm install error #1975
- Error: No resolver found for codec "dag-json" (and dag-cbor) #1923
- Enabling WebTransport in Kubo v0.16.0-rc1 breaks Status page #2033
- Failing Master Branch (error:0308010C:digital envelope routines::unsupported) ipld-explorer-components#342
Issues blocking dependency updates
Investigating dependencies
npm outdated --all > npm-outdated.raw.log gives us this: npm-outdated.raw.log
Then we want to remove duplicates, because npm outdated doesn't do that for us for some reason (echo "$(head -n 1 npm-outdated.raw.log && tail -n +2 npm-outdated.raw.log | sort | uniq -d)" > npm-outdated.uniq.log): npm-outdated.uniq.log
cat npm-outdated.raw.log | awk '{print $6}' | sort | uniq -c | sort > most-impactful-dependencies.log
To see which dependencies are mentioned the most in the depended by column, we can run cat npm-outdated.uniq.log | awk '{print $6}' | sort | uniq -c | sort > most-impactful-dependencies.log to get: most-impactful-dependencies.log
and then to get the top 10 most impactful dependency updates, tail -n 10 most-impactful-dependencies.log:
8 ipfs-http-client
9 @babel/core
9 @storybook/theming
9 css-loader
10 jest-cli
11 ipfs-core-utils
11 ipfs-utils
11 react-dev-utils
15 @storybook/components
51 @babel/preset-env
So which of those versions do we 'want'?
> tail -n 10 most-impactful-dependencies.log | awk '{print $2}' | xargs -n1 -I% sh -c 'cat npm-outdated.uniq.log | grep "^%"'
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/helper-compilation-targets
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/helper-create-class-features-plugin
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-class-properties
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-dynamic-import
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-export-namespace-from
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-logical-assignment-operators
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-nullish-coalescing-operator
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-numeric-separator
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-optional-chaining
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-proposal-private-methods
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-syntax-flow
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-syntax-typescript
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-transform-flow-strip-types
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-transform-modules-commonjs
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/plugin-transform-typescript
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @babel/preset-typescript
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @jest/transform
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core @svgr/plugin-jsx
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core istanbul-lib-instrument
@babel/core 7.15.5 7.18.6 7.18.6 node_modules/@babel/core jest-snapshot
@storybook/theming 5.3.21 5.3.21 6.5.9 node_modules/@storybook/theming @storybook/components
ipfs-core-utils 0.10.5 0.10.5 0.15.1 node_modules/ipfs-core-utils ipfs-http-client
@babel/preset-env 7.12.11 7.18.6 7.18.6 node_modules/@babel/preset-env @svgr/webpack
And why are they there? tail -n 10 most-impactful-dependencies.log | awk '{print $2}' | xargs -n1 -I% sh -c 'npm explain %' > explain.txt: explain.txt
That's super hard to understand, so let's aggregate again... cat explain.txt | grep 'from the root project' | awk '{$1=$1};1' | sort | uniq -c | sort:
1 dev @babel/core@"^7.15.5" from the root project
1 dev @storybook/addon-links@"^5.3.19" from the root project
1 dev @storybook/addons@"^5.3.19" from the root project
1 ipld-explorer-components@"^2.4.1" from the root project
2 dev @svgr/cli@"^5.4.0" from the root project
4 ipfs-http-client@"49.0.2" from the root project
5 dev @storybook/addon-a11y@"^5.3.19" from the root project
6 dev @storybook/addon-actions@"^5.3.19" from the root project
6 dev @storybook/addon-knobs@"^5.3.19" from the root project
29 dev @playwright/test@"^1.12.1" from the root project
106 dev ipfs@"0.58.3" from the root project
117 dev ipfsd-ctl@"^7.2.0" from the root project
190 dev jest@"^26.6.3" from the root project
213 dev @storybook/react@"^5.3.19" from the root project
301 dev jest-playwright-preset@"^1.7.0" from the root project
754 react-scripts@"^4.0.3" from the root project
Breakdown of the above:
- List all outdated npm modules
- find the outdated modules that are mentioned the most
- find and count the npm modules that depend on the most-mentioned out-of-date modules.
- These are the "most impactful dependencies"
- get npm to tell us why these most impactful dependencies are in our repo.
- filter through step4 above to tell us which module we actually depend on is at fault for all the above
- find the most-mentioned module from step5 that we depend on
- The module with the most entries should be the first target for our upgrade plan.
So what does this mean?
Using react-scripts as an example: there are 754 listings of react-scripts being the ROOT CAUSE. react-scripts is the package we depend on, that is listed in the npm-explain output, when trying to determine why out-of-date modules are listed in our dependency graph.
Why did you do all this?
I attempted to upgrade some (see main...SgtPooki:ipfs-webui:fix/node-14-windows) to resolve the build errors showing at https://github.com/SgtPooki/ipfs-webui/actions/runs/2605014389, (see #1961) and was consistently blocked by npm because of semver conflicts of wanted packages. In order to move forward.. since there is no npm-dependency-graph-resolver upgrade tool, I figured I would need all of this data.