Skip to content

Commit 559d41c

Browse files
markeriksonmarkerikson
authored
Merge pull request #195 from replayio/feature/PRO-600-sourcemap-upload-instructions
Revamp "Upload Sourcemaps" page with options, deletion instructions, and better Next example
2 parents 371432e + 4c7bd69 commit 559d41c

File tree

2 files changed

+199
-47
lines changed
  • src/app/reference
    • replay-cli/source-maps
    • test-runners/playwright/github-actions

2 files changed

+199
-47
lines changed

‎src/app/reference/replay-cli/source-maps/page.md

+194-45
Original file line numberDiff line numberDiff line change
@@ -8,89 +8,238 @@ Source maps exist to try and preserve the original debugging experience as best
88

99
## Upload Process
1010

11-
For production use cases where sourcemaps may not be exposed publicly, we provide tooling for preemptively uploading sourcemaps to our servers so that when replays are made, they can make use of these maps. We provide both a CLI command, and a Webpack plugin to make integrating this into your build process as easy as possible.
11+
For production use cases where sourcemaps may not be exposed publicly, we provide tooling for preemptively uploading sourcemaps to our servers so that when replays are made, they can make use of these maps and you can still debug the recordings with the original source code. We provide both a CLI command and a Webpack plugin to make integrating this into your build process as easy as possible.
1212

1313
### API Keys
1414

1515
We recommend using the `REPLAY_API_KEY` environment variable to set up the API for use in all of our following examples, but each also allows passing the key directly as an argument. API Keys are available in the [Team Settings](/reference/ci-workflows/generate-api-key) modal.
1616

17-
### CLI
17+
**Be sure to add the `REPLAY_API_KEY` for the workspace to the build and deployment settings for your project**, such as in Vercel or Github Actions - it needs to be available during the build step in order for the upload logic to upload successfully.
1818

19-
You can upload source maps using the `upload-sourcemaps` command in the Replay CLI.
19+
## CLI
20+
21+
You can upload source maps using the `replayio upload-source-maps` command in the [Replay CLI](/reference/replay-cli/commands).
2022

2123
The simplest usage will be running the following after your production JS build has finished.
2224

2325
```sh
24-
npx replayio upload-sourcemaps --group <Version or SHA> <buildOutputDir>
26+
npm run build
27+
npx replayio upload-source-maps --group <Version or SHA> <buildOutputDir>
2528
```
2629

2730
You must ensure that your build tool is configured to output sourcemaps into the build directory you provide.
2831

29-
### Webpack
32+
### CLI Options
33+
34+
The `replayio upload-source-maps` command has options available to help locate and filter which sourcemaps are available. Currently, those options are:
35+
36+
```bash
37+
-g, --group <name> The name to group this source map into, e.g. A commit SHA or release version.
38+
-x, --extensions <exts> A comma-separated list of file extensions to process; default ".js,.map"
39+
-i, --ignore <pattern> Ignore files that match this pattern
40+
--root <dirname> The base directory to use when computing relative paths
41+
```
3042

31-
The `sourcemap-upload` Webpack plugin is available on npm at `@replayio/sourcemap-upload-webpack-plugin` and you can explore the available options at [sourcemap-upload-webpack-plugin](https://github.com/replayio/replay-cli/tree/main/packages/sourcemap-upload-webpack-plugin) and [sourcemap-upload](https://github.com/replayio/replay-cli/tree/main/packages/sourcemap-upload).
43+
## Webpack
3244

33-
The simplest usage will be to edit your `webpack.config.js` to include a new entry in the `plugins` array at the top level of the file, e.g.
45+
We have a Webpack plugin that can upload sourcemaps, available on NPM as [`@replayio/sourcemap-upload-webpack-plugin`](https://npmjs.com/package/@replayio/sourcemap-upload-webpack-plugin).
3446

35-
```javascript
36-
const ReplaySourceMapUploadWebpackPlugin = require("@replayio/sourcemap-upload-webpack-plugin");
47+
To use the plugin, edit your `webpack.config.js` to include a new entry in the `plugins` array at the top level of the file, e.g.
48+
49+
```js
50+
const ReplaySourceMapUploadWebpackPlugin = require('@replayio/sourcemap-upload-webpack-plugin')
3751

3852
module.exports = {
3953
// Ensure that Webpack has been configured to output sourcemaps.
40-
devtool: "source-map",
41-
// ... plugins:
42-
[
43-
// Enable our plugin to upload the sourcemaps once the build has completed.
44-
// This assumes NODE_ENV is how you distinguish production builds. If that
45-
// is not the case for you, you will have to tweak this logic.
46-
process.env.NODE_ENV === "production"
47-
? [new ReplaySourceMapUploadWebpackPlugin({
48-
filepaths: ["outputDir/"],
49-
group: "<A unique version string or git SHA>"
50-
})]
51-
: []
54+
devtool: 'source-map',
55+
// ...
56+
plugins: [
57+
// Enable our plugin to upload the sourcemaps once the build has completed.
58+
// This assumes NODE_ENV is how you distinguish production builds. If that
59+
// is not the case for you, you will have to tweak this logic.
60+
process.env.NODE_ENV === 'production'
61+
? [
62+
new ReplaySourceMapUploadWebpackPlugin({
63+
filepaths: ['outputDir/'],
64+
group: '<A unique version string or git SHA>',
65+
}),
66+
]
67+
: [],
5268
],
5369
}
5470
```
5571

56-
Keep in mind that your build output will now contain sourcemaps, so you should take care to ensure that these maps aren't also exposed on your production site.
72+
### Webpack Upload Plugin Options
73+
74+
The Webpack plugin accepts several options to configure searching for sourcemaps and build artifacts on disk, excluding them from upload, and configuring deletion.
75+
76+
```ts
77+
// Exported from `@replayio/sourcemap-upload`
78+
export interface UploadOptions {
79+
/**
80+
* The files/directories to search for sourcemaps. All files that match the
81+
* 'extensions' list and fail to match 'ignore' will be searched for
82+
* sourcemap JSON or `//#sourceMappingURL=` coments in order to find pairs
83+
* of generated-file + sourcemap, and the sourcemap will be uploaded.
84+
*/
85+
filepaths: Array<string> | string
86+
/**
87+
* To allow for tracking and browsing of maps that have been uploaded, we
88+
* require uploaded sourcemaps to have an overall group name associated with
89+
* them. This could for instance be a version number, or commit hash.
90+
*/
91+
group: string
92+
/**
93+
* The API key to use when connecting to Replay's servers.
94+
* Defaults to `process.env.REPLAY_API_KEY`.
95+
*/
96+
key?: string
97+
/**
98+
* Run all of the local processing and searching for maps, but skip uploading them.
99+
*/
100+
dryRun?: boolean
101+
/**
102+
* Delete all found sourcemap files after they have been uploaded.
103+
*/
104+
deleteAfterUpload?: boolean
105+
/**
106+
* If sourcemaps can't be matched to generated files by their sourceMappingURL, try matching by filenames on disk
107+
*/
108+
matchSourcemapsByFilename?: boolean
109+
/**
110+
* The set of file extensions to search for sourcemap-related data.
111+
* Defaults to [".js", ".map"].
112+
*/
113+
extensions?: Array<string>
114+
/**
115+
* The set of pattern for files to ignore when searching for sourcemap-related data.
116+
*/
117+
ignore?: Array<string>
118+
/**
119+
* Set the directory that relative paths should be computed with respect to.
120+
* The relative path of sourcemaps is included in the uploaded entry, and will be
121+
* visible in the UI, so this can be used to strip off unimportant directories in
122+
* the build path. Defaults to `process.cwd()`.
123+
*/
124+
root?: string
125+
/**
126+
* A callback function that will be called with log messages.
127+
*/
128+
log?: LogCallback
129+
/**
130+
* URL of the Replay server to upload to. Defaults to `https://api.replay.io`.
131+
*/
132+
server?: string
133+
/**
134+
* The number of concurrent uploads to perform. Defaults to 25.
135+
*/
136+
concurrency?: number
137+
/**
138+
* A string to append to the User-Agent header when making requests to the Replay API.
139+
*/
140+
userAgentAddition?: string
141+
}
142+
143+
// Exported from `@replayio/sourcemap-upload-webpack-plugin`
144+
export interface PluginOptions extends UploadOptions {
145+
// Choose how verbose the plugin should be when logging.
146+
logLevel?: 'quiet' | 'normal' | 'verbose'
147+
148+
// Normally failure to upload the sourcemaps will result
149+
// in a build error. If you'd like to simply warn instead
150+
// of failing in this case, you can set this to true.
151+
warnOnFailure?: boolean
152+
}
153+
```
154+
155+
### Sourcemap Visibility in Production
156+
157+
Keep in mind that your build output will now contain sourcemaps, and so your app source code will be viewable in the browser devtools for anyone viewing your production site.
158+
159+
If this is a concern for you, you can keep your sourcemaps hidden from end users by deleting them from the build output directory. The Replay Webpack plugin has a `deleteAfterUpload` option to allow this. You can also configure Webpack's sourcemap generation settings to use [`devtool: "hidden-source-map"`](https://webpack.js.org/configuration/devtool/#production), which skips adding a `sourceMappingURL` reference to the generated JS build artifacts to keep the browser from trying to download and view sourcemaps. If you enable that, you should also enable the Webpack plugin's `matchSourcemapsByFilename` option, to ensure the plugin still correctly finds the right sourcemaps:
160+
161+
```js
162+
const ReplaySourceMapUploadWebpackPlugin = require('@replayio/sourcemap-upload-webpack-plugin')
163+
164+
module.exports = {
165+
// Ensure that Webpack has been configured to output sourcemaps,
166+
// but without the `sourceMappingURL` references in buidl artifacts.
167+
devtool: 'hidden-source-map',
168+
// ...
169+
plugins: [
170+
// Enable our plugin to upload the sourcemaps once the build has completed.
171+
// This assumes NODE_ENV is how you distinguish production builds. If that
172+
// is not the case for you, you will have to tweak this logic.
173+
process.env.NODE_ENV === 'production'
174+
? [
175+
new ReplaySourceMapUploadWebpackPlugin({
176+
filepaths: ['outputDir/'],
177+
group: '<A unique version string or git SHA>',
178+
deleteAfterUpload: true,
179+
matchSourcemapsByFilename: true,
180+
}),
181+
]
182+
: [],
183+
],
184+
}
185+
```
186+
187+
The Replay Webpack plugin uploads sourcemaps to the workspace associated with your API key, and automatically matches up sourcemaps with the build artifacts in a recorded replay when you debug it. This way, you can avoid having sourcemaps visible on your production site, but still debug replays with the original source code.
57188

58189
### NextJS
59190

60191
NextJS uses Webpack internally, so the plugin above will also be used for NextJS, but the config that needs editing is `next.config.js` instead. NextJS's [own documentation](https://nextjs.org/docs/api-reference/next.config.js/custom-webpack-config) is a good place to start and shows how to add Webpack plugins to your build process. You'll also need to enable [production sourcemaps](https://nextjs.org/docs/advanced-features/source-maps), so you'll end up with the following additional config in your `next.config.js` file:
61192

62-
```javascript
63-
module.exports = {
193+
```ts
194+
/** @type {import('next').NextConfig} */
195+
const nextConfig = {
64196
productionBrowserSourceMaps: true,
65-
webpack: (config, { buildId, dev, isServer }) => {
197+
198+
webpack: (config, { isServer, dev, buildId }) => {
199+
/** @type {import('webpack').Configuration} */
200+
const webpackConfig = config
201+
202+
// This configures Webpack to generate sourcemaps, but not include
203+
// the `sourceMappingURL` reference in generated JS bundles.
204+
// This keeps the sourcemaps hidden from users, but still available
205+
webpackConfig.devtool = 'hidden-source-map'
206+
66207
if (!dev && !isServer) {
67-
config.plugins.push(
68-
new ReplaySourceMapUploadWebpackPlugin({
69-
// If you've configured a custom build directory, this should
70-
// point to that directory. filepaths: [".next/"],
71-
// Potentially the 'buildId' argument here could be used.
72-
group: '<A unique version string or git SHA>',
73-
}),
74-
)
75-
// 'productionBrowserSourceMaps' will output your sourcemaps
76-
// into the build directory, which can expose them from your
77-
// production server as well, so you will likely want to delete
78-
// the .map files once they have been uploaded.
79-
80-
config.plugins.push((compiler) =>
81-
compiler.hooks.afterEmit.tapPromise('DeleteSourceMaps', () =>
82-
findAndDeleteSourceMaps(),
83-
),
84-
)
208+
/** @type {import('@replayio/sourcemap-upload-webpack-plugin').PluginOptions} */
209+
const uploadOptions = {
210+
// If you've configured a custom build directory, this should point to that directory.
211+
filepaths: ['.next/'],
212+
// You can use `buildId` here to differentiate sourcemaps between different builds.
213+
group: buildId,
214+
215+
// Don't upload or delete server-related sourcemaps
216+
ignore: ['**/server/**'],
217+
// You can configure the plugin to automatically delete sourcemaps from the local build
218+
// output folder after they've been uploaded, to keep them from being visible to users.
219+
deleteAfterUpload: true,
220+
// When `devtool: "hidden-source-map"` is used, the upload plugin can't match up
221+
// sourcemaps with bundles automatically because there's no `sourceMappingURL`.
222+
// Set this to true to match them up by similar filenames instead.
223+
matchSourcemapsByFilename: true,
224+
// The plugin will look for the API key as `process.env.REPLAY_API_KEY` by default.
225+
// You can provide it here if it's stored in a different environment variable.
226+
// key: process.env.REPLAY_API_KEY,
227+
}
228+
config.plugins.push(new ReplaySourceMapUploadWebpackPlugin(uploadOptions))
85229
}
230+
86231
return config
87232
},
88233
}
234+
235+
module.exports = nextConfig
89236
```
90237

91-
As mentioned, NextJS's `buildId` can be useful, but please [see their docs](https://nextjs.org/docs/api-reference/next.config.js/configuring-the-build-id) to ensure verify if it is a good fit for your application.
238+
You can see [a full working example of Next config with sourcemap deletion here](https://github.com/replayio-public/replay-sourcemap-upload-example/blob/main/next.config.mjs), and check out [the demo site](https://replay-sourcemap-upload-example-site.vercel.app/) to confirm that the original app source code is not included in the production site.
239+
240+
Also see the Next.js [docs on "Configuring the Build ID"](https://nextjs.org/docs/api-reference/next.config.js/configuring-the-build-id) for details on generating a consistent build ID, such as with a Git hash.
92241

93-
### Debugging problems
242+
## Debugging problems
94243

95244
Each of these approaches will log the sourcemaps as they are uploaded so that you can validate that all of the files you expect are being detected. They each also have a verbose mode, either `logLevel: "verbose"` for the JS plugins, and `--verbose` for the CLI.
96245

‎src/app/reference/test-runners/playwright/github-actions/page.md

+5-2
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: GitHub actions
44

55
Playwright tests can be configured via `playwright.config.ts` and the [command line](https://playwright.dev/docs/test-cli).
66

7-
In real world scenarios, the `playwright.config.ts` has the defualt setup and is extended by the command line options in either the terminal, `package.json` or github action.
7+
In real world scenarios, the `playwright.config.ts` has the default setup and is extended by the command line options in either the terminal, `package.json` or github action.
88

99
## Simple setup
1010

@@ -46,16 +46,19 @@ jobs:
4646
steps:
4747
- name: Checkout
4848
uses: actions/checkout@v4
49+
# If you're using Yarn or PNPM, use the appropriate install command here
4950
- name: Install dependencies
5051
run: npm ci
5152
- name: Install Replay Chromium
5253
run: npx replayio install
5354
- name: Run Playwright tests
54-
run: npx playwright test
55+
run: npx playwright test --project replay-chromium
5556
env:
5657
REPLAY_API_KEY: ${{ secrets.REPLAY_API_KEY }}
5758
```
5859
60+
Finally, add the workspace API key you were given to the Github repo settings as `REPLAY_API_KEY`, so that the CI job uploads to the right workspace.
61+
5962
## Advanced setup
6063

6164
In most setups, you'll want to to have additional control. In these cases, it's common to want to split up the test run and upload steps.

0 commit comments

Comments
 (0)