-
Notifications
You must be signed in to change notification settings - Fork 42
Produce and Publish the set of vscode built in extensions
The "builtins", "built-ins" or "vscode builtin extensions" are a set of extensions that whose code resides in the public vscode repository. Some contain no code as such and contribute the means to perform static rule-based syntax highlighting (through contributing a TextMate grammar) and sometimes also some code snippets, for a particular programming language. E.g. vscode.bat, vscode.javascript. A few built-in extensions contribute advanced language support for a given language, including a Language Server, a debug adapter, command/tasks contributions and more. e.g. vscode.typescript-language-features, vscode.markdown-language-features, vscode.html-language-features, 'vscode.json-language-features'. Finally, some built-ins contribute misc potentially useful features, like various themes, integration to a 3PP services like vscode.github* and more.
Every sub-folder of vscode/extensions/ is one built-in vscode extension (with a couple of exceptions like node_modules after a build):
https://github.com/microsoft/vscode/tree/main/extensions
They are built along and packaged as a group (not individual .vsix), in the Visual Studio Code product as well as in products based on Code OSS and derivatives such as VSCodium. The built-in extensions are distributed as a part of their IDE product and are not made available separately, e.g. on the Visual Studio Marketplace.
See also: Consuming Builtin and External VS Code Extensions
In Theia-based application, there is often a need for the features provided by various built-ins. To obtain some or all of the builtin extensions, the IDE author can include all built-in extensions by adding to the app's package.json, the URL of one of our builtins extension pack - open-vsx.org, produced and published alongside the builtin extensions. It's possible to exclude unwanted extensions, referenced by the pack, by adding them to the theiaPluginsExcludeIds list. It's also possible to curate the exact list of wanted built-ins and add them individually to the application's package.json and let the user install more later if needed. In theory they could decide to bundle no built-ins at all, if that fits their purpose.
From the example Theia application, in section "theiaPlugins":
theiaPlugins": {
"vscode.markdown-language-features": "https://open-vsx.org/api/vscode/markdown-language-features/1.64.2/file/vscode.markdown-language-features-1.64.2.vsix",
"eclipse-theia.builtin-extension-pack": "https://open-vsx.org/api/eclipse-theia/builtin-extension-pack/1.70.2/file/eclipse-theia.builtin-extension-pack-1.70.2.vsix",
"EditorConfig.EditorConfig": "https://open-vsx.org/api/EditorConfig/EditorConfig/0.14.4/file/EditorConfig.EditorConfig-0.14.4.vsix",
"dbaeumer.vscode-eslint": "https://open-vsx.org/api/dbaeumer/vscode-eslint/2.1.20/file/dbaeumer.vscode-eslint-2.1.20.vsix"
},How to exclude certain pack extensions:
"theiaPluginsExcludeIds": [
"ms-vscode.js-debug-companion",
"vscode.extension-editing",
"vscode.git",
"vscode.git-base",
"vscode.github",
"vscode.github-authentication",
"vscode.ipynb",
"vscode.microsoft-authentication",
"ms-vscode.references-view"
]
Eclipse Theia is an Eclipse Foundation project, we follow the Foundation's IP process regarding 3PPs (runtime dependencies as well as "build and test" dependencies, obtained from the public npm registry). See the Eclipse Foundation Project Handbook IP section for more details. We use dash-licenses to obtain (mostly) automated approval for the 3PP dependencies used to build the built-in extensions.
For any new version of the built-in extensions we want to produce and publish, we:
- open an IP Check ticket on the Eclipse Foundation Gitlab, attach the source code of all the extension we use
- Run the 3PP License check[*] on that version of the built-ins, automatically open IP tickets for new found dependencies of unclear license. See below for more details about these tickets
When all tickets have been approved, we can proceed with publishing the new version.
[*] details here
At the moment this is only informational. This check uncovers known vulnerabilities in 3PP dependencies, in specific versions of dependencies, used by the current set of builtins. There used to be a lot when we were a couple of years late in producing a set of older built-ins. The portrait has recently improved a lot, because we have made rapid progress in supporting newer vscode extensions API versions. This means the vscode built-in extensions we produced recently are based on more recent packages, that have accumulated less vulnerability reports. When we catch-up and stay on top of new APIs as they are released, the built-in extensions we build will have close to zero vulnerabilities, in the moment when we produce and publish them, just like their twins that are part of the most recently released version of Visual Studio Code product.
See here for how to run yarn audit, for the full set of built-in extensions.
Start with the latest master state:
cd vscode-builtin-extensions
git clean -ffdx
git checkout master
git pull origin masterCreate a branch for the vscode builtins you intend to build/publish:
BIVERSION="1.72.2" # for example
git branch $BIVERSION
git checkout $BIVERSIONCheckout vscode git submodule at wanted version
git submodule init
git submodule update
cd vscode
git clean -ffdx
git checkout $BIVERSION
cd ..Build the built-ins, package them as .vsix and generate an extension pack:
yarn && yarn build:extensions
# proceed if everything built correctly
yarn package-vsix:latest && yarn create-extension-pack:latestThe extensions, including the pack, will be under folder dist, each packaged as a .vsix file.
vscode-builtin-extensions$ ls dist/
bat-1.72.1.vsix html-language-features-1.72.1.vsix references-view-1.72.1.vsix
builtin-extension-pack-1.72.1.vsix ini-1.72.1.vsix restructuredtext-1.72.1.vsix
builtin-notebook-renderers-1.72.1.vsix ipynb-1.72.1.vsix ruby-1.72.1.vsix
clojure-1.72.1.vsix jake-1.72.1.vsix rust-1.72.1.vsix
coffeescript-1.72.1.vsix java-1.72.1.vsix scss-1.72.1.vsix
configuration-editing-1.72.1.vsix javascript-1.72.1.vsix search-result-1.72.1.vsix
cpp-1.72.1.vsix json-1.72.1.vsix shaderlab-1.72.1.vsix
csharp-1.72.1.vsix json-language-features-1.72.1.vsix shellscript-1.72.1.vsix
css-1.72.1.vsix julia-1.72.1.vsix simple-browser-1.72.1.vsix
css-language-features-1.72.1.vsix latex-1.72.1.vsix sql-1.72.1.vsix
dart-1.72.1.vsix less-1.72.1.vsix swift-1.72.1.vsix
debug-auto-launch-1.72.1.vsix log-1.72.1.vsix theme-abyss-1.72.1.vsix
debug-server-ready-1.72.1.vsix lua-1.72.1.vsix theme-defaults-1.72.1.vsix
diff-1.72.1.vsix make-1.72.1.vsix theme-kimbie-dark-1.72.1.vsix
docker-1.72.1.vsix markdown-1.72.1.vsix theme-monokai-1.72.1.vsix
emmet-1.72.1.vsix markdown-language-features-1.72.1.vsix theme-monokai-dimmed-1.72.1.vsix
extension-editing-1.72.1.vsix markdown-math-1.72.1.vsix theme-quietlight-1.72.1.vsix
fsharp-1.72.1.vsix media-preview-1.72.1.vsix theme-red-1.72.1.vsix
git-1.72.1.vsix merge-conflict-1.72.1.vsix theme-solarized-dark-1.72.1.vsix
git-base-1.72.1.vsix npm-1.72.1.vsix theme-solarized-light-1.72.1.vsix
github-1.72.1.vsix objective-c-1.72.1.vsix theme-tomorrow-night-blue-1.72.1.vsix
github-authentication-1.72.1.vsix perl-1.72.1.vsix typescript-1.72.1.vsix
go-1.72.1.vsix php-1.72.1.vsix typescript-language-features-1.72.1.vsix
groovy-1.72.1.vsix php-language-features-1.72.1.vsix vb-1.72.1.vsix
grunt-1.72.1.vsix powershell-1.72.1.vsix vscode-theme-seti-1.72.1.vsix
gulp-1.72.1.vsix pug-1.72.1.vsix xml-1.72.1.vsix
handlebars-1.72.1.vsix python-1.72.1.vsix yaml-1.72.1.vsix
hlsl-1.72.1.vsix r-1.72.1.vsix
html-1.72.1.vsix razor-1.72.1.vsixThis section assumes you have a local clone of the main Theia repo.
cd theia # go in the theia main git repository folderIf required, step the supported vscode API in file dev-packages/application-package/src/api.ts. Luckily, it did not need to be done in this case:
/**
* The default supported API version the framework supports.
* The version should be in the format `x.y.z`.
*/
export const DEFAULT_SUPPORTED_API_VERSION = '1.72.2';Clean-build the example application:
git clean -ffdx
yarn && yarn browser build
If already present, delete folder plugins. We will instead use the builtins we previously built
rm -rf plugins
mkdir pluginsCopy the builtin extensions we produced, to Theia's plugins folder.
cd vscode-builtin-extensions # back to vscode-builtin0-extensions repo
cp -a dist/* ../theia/plugins # adjust according to where your theia folder residesGet rid of a few builtins that will interfere with testing (note: we keep these extensions where they were generated, but remove them from our test Theia application):
cd theia # back to theia repo
rm -rf plugins/ipynb-*
rm -rf plugins/extension-editing-*
rm -rf plugins/github-*
To test vscode builtin git, we need to remove the Theia-specific git extension from the example application:
theia$ git diff
diff --git a/examples/browser/package.json b/examples/browser/package.json
index da42a2b6004..f5f52d7ffcd 100644
--- a/examples/browser/package.json
+++ b/examples/browser/package.json
@@ -25,7 +25,6 @@
"@theia/file-search": "1.36.0",
"@theia/filesystem": "1.36.0",
"@theia/getting-started": "1.36.0",
- "@theia/git": "1.36.0",
"@theia/keymaps": "1.36.0",
"@theia/markers": "1.36.0",
"@theia/memory-inspector": "1.36.0",
diff --git a/examples/electron/package.json b/examples/electron/package.json
index 1f81d6076c6..35d1b6d927c 100644
--- a/examples/electron/package.json
+++ b/examples/electron/package.json
@@ -27,7 +27,6 @@
"@theia/file-search": "1.36.0",
"@theia/filesystem": "1.36.0",
"@theia/getting-started": "1.36.0",
- "@theia/git": "1.36.0",
"@theia/keymaps": "1.36.0",
"@theia/markers": "1.36.0",
"@theia/memory-inspector": "1.36.0",Re-build and start the example application (e.g. the browser version):
yarn && yarn browser build
yarn browser start- Connect to
localhost:3000with a browser - Observe backend log for new exceptions, specially during activation of builtin extensions
- quick TypeScript test
- quick JSON test
- quick git test
- Submit PR for current builtins versions for review and merge.
Register issues for problems found. Some problems may require changing how we build or package, in which case a fix would be made on vscode-builtin-extensions as part of the ongoing release PR. If the issue is with the upstream Theia repo, we open the issue there.
While testing buitins 1.72.2, we found the following:
- RangeError: Maximum call stack size exceeded with recent vscode.html builtin extension #12434
- [builtins] [proposed API] [vscode.markdown-language-features]: Theia misses proposed API:
Document Paste - [builtins] [proposed API] [vscode.git@1.72.2]: Theia misses proposed API:
Edit session identifier provider
Once you are confident that the new set of builtins do not have obvious issues, you can proceed with publishing them to open-vsx.org. It's ok if there are issues that will later be fixed in Theia - older version of the builtin can be temporarily used instead in most cases.
Commit your changes:
cd vscode-builtin-extensions # back to vscode-builtin0-extensions repo
git branch # confirm you are in the "version" branch, e.g. "1.72.2"
# add any file you modified and commit
git add vscode/ # the change in checked-out version of the vscode git sub-module
git commit -s -m "changes for builtins v1.72.2"
git push origin 1.72.2 # from there, create a PR.Publishing is done using GitHub Actions. In the vscode-builtin-extensions repo, a publish token for open-vsx.org has been set, that can be used to publish under the identity of the openvsx publish bot. This means that we need to automatically obtain the extensions to publish, without manual intervention on our part.
Before publishing: Make sure that you have cleared the 3PP dependencies from the Licenses aspect: https://github.com/eclipse-theia/vscode-builtin-extensions/wiki/Produce-and-Publish-the-set-of-vscode-built-in-extensions/#3pp-license-check
Our GitHub workflow is aware of two "magic" git branches, that we use to publish:
They are respectively used to trigger the run of a workflow that will attempt to publish a "latest" or "next" version of the built-ins set.
If required, update the relevant GitHub workflow. In this case, vscode-builtin-extensions/.github/workflows/publish-vsx-specific-latest.yml
For example if a new node version is required to build, update that. If the build/packaging procedure changes, reflect the change in the workflow(s). You may want to include these changes in your PR under review.
git checkout 1.72.2
git branch -D ovsx-publish # if the branch already exists
git branch ovsx-publish
git checkout ovsx-publishWhen ready, push the branch to the upstream repo: this will trigger an attempt at building and publishing the builtins: note: this requires write permission in upstream repo, i.e. being a committer
git checkout ovsx-publish
git push origin ovsx-publish
# if will probably fail because we already have an old version on that branch upstream. Use the force:
git push -f origin ovsx-publish Go in the "Actions" tab to follow the publishing: https://github.com/eclipse-theia/vscode-builtin-extensions/actions