Add contribution guideline (#28)

In this PR:

- Adds CONTRIBUTING.md file
- Clean up the README.md file
- Adds the experimental folder for adding new benchmarks by external
contributors

Author: rmahdav

.gitignore

@@ -12,5 +12,9 @@
 /resources/litert-js/dist/
 /resources/litert-js/models/
 
+/resources/experimental/node_modules/
+/resources/experimental/dist/
+/resources/experimental/models/
+
 .vscode
 .angular

docs/code-of-conduct.md CODE_OF_CONDUCT.mddocs/code-of-conduct.md renamed to CODE_OF_CONDUCT.md

@@ -0,0 +1,54 @@
+# How to Contribute
+
+We would love to accept your patches and contributions to this project.
+
+## Before you begin
+
+### Sign our Contributor License Agreement
+
+Contributions to this project must be accompanied by a
+[Contributor License Agreement](https://cla.developers.google.com/about) (CLA).
+You (or your employer) retain the copyright to your contribution; this simply
+gives us permission to use and redistribute your contributions as part of the
+project.
+
+If you or your current employer have already signed the Google CLA (even if it
+was for a different project), you probably don't need to do it again.
+
+Visit <https://cla.developers.google.com/> to see your current agreements or to
+sign a new one.
+
+### Review our Community Guidelines
+
+This project follows [Google's Open Source Community
+Guidelines](https://opensource.google/conduct/).
+
+## Contribution process
+
+If you believe a model should be included in our benchmark, please submit a pull request to add it as a new workload.
+
+As of Q1 2026, the benchmark is under active development, and we are refining aspects such as the scoring calculation. We welcome your input on design considerations or other general feedback. Please open an issue to start a discussion.
+
+To propose a new workload, please add it to the `resources/experimental` directory and submit a pull request. In your pull request, please explain the workload's relevance and provide technical details about its integration.
+
+### Adding Experimental Workloads
+
+- Update the `description` and `dependencies` in `resources/experimental/package.json`.
+- Inside `resources/experimental/src/index.mjs`, add a new async function and `ModelConfig` for your workload similar to the existing example.
+- Adjust the code inside `resources/experimental/src/index.html` if neede.
+- Update `resources/experimental/src/download-models.mjs` to download the model. You can refer to `resources/transformers-js/src/download-models.mjs` and `resources/litert-js/src/download-models.mjs` as examples.
+- Add licensing information of the new model in `resources/experimental/lisense/NOTICEmd` and add any required lince file in `resources/experimental/lisense/`.
+- Add `<your-new-workload-name>.mjs` inside `resources/experimental/src`, similar to the existing ones.
+- Add an entry and a plugin for the new workload in `resources/experimental/webpack.common.js`.
+- Update `.gitignore` if needed.
+- Run `npm run build` in the repository root directory. Alternatively, you can run `npm install` and `npm run build` inside `resources/experimental`.
+- Add the workload to `resources/default-tests.mjs` with the `Experimental-` prefix, analogous to the existing workloads.
+- Serve the overall runner via `npm run dev` in the repository root directory.
+- Browse to `http://localhost:8080` and click "Run" to run the benchmark.
+- To run individual experimental workloads,  navigate to `http://localhost:8080/resources/experimental/dist/<workload>.html` then do `manualRun()` in dev console.
+
+### Code Reviews
+
+All submissions, including submissions by project members, require review. We
+use [GitHub pull requests](https://docs.github.com/articles/about-pull-requests)
+for this purpose.

CONTRIBUTING.md

@@ -14,10 +14,9 @@ See the Speedometer repo for a more detailed explanation, e.g., in which phases
 - Running a workload in the browser: TODO.
 - Inspecting and understanding metrics: TODO.
 - Most important files:
-    - Workloads are in `resources/*/`.
+    - Workloads are in `resources/transformers-js/` and `resources/litert-js`.
     - Shared files are in `resources/shared/`, which is depended-upon as a local package.
     - The default suite / tests to run are in `resources/default-tests.mjs`.
-    - An example config (which can be loaded from externally) is in `resources/config.json`, but it is not used (more an example).
 
 TODO: Add a screenshot of the runner, after updating logo and name and some minor styling of the workloads.
 
@@ -27,43 +26,9 @@ TODO: Add a screenshot of the runner, after updating logo and name and some mino
 - `npm run dev` in the root directory to start the server
 -  Navigate to `http://localhost:8080/resources/<workload-group>/dist/<workload>.html` (e.g. http://localhost:8080/resources/transformers-js/dist/feature-extraction-cpu.html ) then do `manualRun()` in dev console.
 
-## How to Add a New Workload
-
-### Transformers.js-based workloads
-
-- Inside `resources/transformers-js/src/index.mjs`, add a new async function and `ModelConfig` for your workload.
-- Add the name of your model to `MODELS_TO_DOWNLOAD` in `resources/transformers-js/src/download-models.mjs`.
-- Add `<your-new-workload-name>.mjs` inside `resources/transformers-js/src`, similar to the existing ones.
-- Add an entry and a plugin for the new workload in `resources/transformers-js/webpack.commom.js`.
-- Run `npm install` and `npm run build` inside `resources/transformers-js` to produce output in `dist/`.
-- Add the workload to `resources/default-tests.mjs`, analogous to the existing workloads.
-- Serve the overall runner via `npm run dev` in the repository root directory.
-- Browse to http://localhost:8080, click on run to see the new workload.
-
-### LiteRT.js-based workloads
-
-- Inside `resources/litert-js/src/index.mjs`, add a new async function and `ModelConfig` for your workload.
-- Add the name of your model to `MODELS_TO_DOWNLOAD` in `resources/litert-js/src/download-models.mjs`.
-- Add `<your-new-workload-name>.mjs` inside `resources/litert-js/src`, similar to the existing ones.
-- Add an entry and a plugin for the new workload in `resources/litert-js/webpack.commom.js`.
-- Run `npm install` and `npm run build` inside `resources/litert-js` to produce output in `dist/`.
-- Add the workload to `resources/default-tests.mjs`, analogous to the existing workloads.
-- Serve the overall runner via `npm run dev` in the repository root directory.
-- Browse to http://localhost:8080, click on run to see the new workload.
-
-### Other workloads
-
-- Make a copy of `resources/transformers-js` and rename it to `resources/<your-selected-name>`.
-- Update the `description` and `dependencies` in `resources/<your-selected-name>/package.json`.
-- Adjust the code inside `resources/<your-selectedname>/src/index.html`, `resources/<your-selected-name>/src/index.mjs` and `resources/<your-selected-name>/src/download-models.mjs`.
-- Update entries and a plugins in `resources/<your-selected-name>/webpack.commom.js`.
-- Update `.gitignore`.
-- Delete `resources/<your-selected-name>/models` if you have any.
-- Run `npm install` and `npm run build` inside `resources/<your-selected-name>` to produce output in `dist/`.
-- Add the workload to `resources/default-tests.mjs`, analogous to the existing workloads.
-- Serve the overall runner via `npm run dev` in the repository root directory.
-- Browse to http://localhost:8080, click on run to see the new workload.
+## Contributing
 
+See the [CONTRIBUTING](CONTRIBUTING.md) file for how to help out.
 
 ## Source Code Headers
 

README.md

@@ -15,7 +15,7 @@
     },
     "license": "SEE LICENSE IN LICENSE",
     "scripts": {
-        "build": "cd resources/litert-js && npm install && npm run build && cd ../transformers-js && npm install && npm run build",
+        "build": "cd resources/litert-js && npm install && npm run build && cd ../transformers-js && npm install && npm run build && cd ../experimental && npm install && npm run build",
         "dev": "node tests/server.mjs",
         "lint:check": "eslint **/*.{js,mjs,jsx,ts,tsx}",
         "lint:fix": "eslint \"**/*.{js,mjs,jsx,ts,tsx}\" --fix",

docs/contributing.md

@@ -133,4 +133,16 @@ export const defaultSuites = [
         tags: ["default", "hand-detection", "webgpu", "litert-js"],
         type: "remote",
     },
+    {
+        name: "Experimental-Text2Text-Generation-wasm",
+        url: "resources/experimental/dist/text2text-generation-cpu.html",
+        tags: ["default", "text2text-generation", "wasm", "transformers-js"],
+        type: "remote",
+    },
+    {
+        name: "Experimental-Text2Text-Generation-webgpu",
+        url: "resources/experimental/dist/text2text-generation-gpu.html",
+        tags: ["default", "text2text-generation", "webgpu", "transformers-js"],
+        type: "remote",
+    },
 ];

package.json

@@ -0,0 +1,19 @@
+An experimental workload runner.
+
+## Build steps
+
+Webpack bundles all the output into the `dist` folder.
+
+```
+npm install
+npm run build
+```
+
+## Requirements
+
+The only requirement is an installation of Node, to be able to install dependencies and run scripts to serve a local server.
+
+```
+* Node (min version: 18.13.0)
+* NPM (min version: 8.19.3)
+```