Skip to content

Commit 555935b

Browse files
AlexanderKaranAlexanderKaran
authored
Updated Docs (#116)
* Init pass * Grammer and spelling check * Updated Contributing * Updated docs * Format * Update Roadmap.md
1 parent b9f1e76 commit 555935b

3 files changed

Lines changed: 149 additions & 86 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 133 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,53 @@
11
# Contributing to Framework Tracker
22

3-
Framework Tracker is part of the e18e.dev community. Want to get involved head to our Discord at https://chat.e18e.dev.
3+
Framework Tracker is part of the e18e.dev community. Want to get involved? Head to our Discord at https://chat.e18e.dev. If you're keen to contribute to the project and get stuck straight into code, we have an open list of good first issues in our GitHub repo: [Good First Issues](https://github.com/e18e/framework-tracker/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22good%20first%20issue%22). We keep this list updated and add new issues every week. It's one of the easiest ways to get involved and help push the project forward.
44

55
## Where to Start
66

77
Framework Tracker has a few different areas which you can contribute to:
88

9-
- **Improving Metrics**: Suggest new metrics or improve the way we collect and display existing ones. Find out current roadmap and metrics in [initial-comparison-list.md](./initial-comparison-list.md#ComparableMetrics).
9+
- **Improving Metrics**: Suggest new metrics or improve the way we collect and display existing ones.
1010

11-
- **Documentation**: Enhance our documentation site by fixing typos, improving explanations, or adding new sections. UI/UX improvements are also welcome!
11+
- **Documentation**: Enhance our documentation site by fixing typos, improving explanations, or adding new sections.
1212

13-
- **Adding New Frameworks**: Help us expand our list of tracked frameworks by adding new ones. Check out [initial-comparison-list.md](./initial-comparison-list.md#frameworks) for the list of planned tracked frameworks.
13+
- **UI/UX Improvements**: Help us improve the documentation site by suggesting design improvements, fixing bugs, or adding new features.
14+
15+
- **Adding New Frameworks**: Help us expand our list of tracked frameworks by adding new ones. See the [Adding a New Framework](./CONTRIBUTING.md#adding-a-new-framework) section below for details, as each new framework adds significant maintenance burden.
16+
17+
### Project Structure
18+
19+
This is a monorepo managed with pnpm workspaces hosting multiple packages. However, some of the packages are not part of the workspace.
20+
21+
Included in the workspace are the documentation site, built with Astro, and the stats generator, which is a tool for collecting and processing the metrics for each framework.
22+
23+
```
24+
framework-tracker/
25+
├── packages/
26+
│ ├── docs/ # Astro-based documentation site
27+
│ └── stats-generator/ # Tool for collecting framework metrics
28+
```
29+
30+
Outside of the workspace, we have individual packages for each framework's starter project and app project. These are used to run the actual measurements for each framework.
31+
32+
```
33+
framework-tracker/
34+
├── starter-*/ # Metaframeworks configured using default set up
35+
├── app-*/ # Metaframeworks configured for run time tests
36+
```
37+
38+
Any project marked with `starter` in the name should be a direct setup of the meta-framework's recommended default configuration. For most of these, we have followed the official getting started guide and used the CLI to set up the project with the recommended path. We often use these to measure dev time performance metrics like build times, dependency counts, and CI performance. It's hard to compare meta-frameworks, so sometimes it's easier to compare the default setup of each framework, as this is what most users will start with and gives a good baseline for comparison.
39+
40+
Any project marked with `app` in the name is a more complex setup on which we often run runtime performance tests. These are often more customized and have more features added on top of the default starter set up. This is because we want to test the performance of each framework under more real world conditions and with more complex features implemented. For example, we might add a blog page with dynamic routing, or a dashboard page with client side interactivity. This allows us to test the performance of each framework under more realistic conditions and see how they perform as the complexity of the app increases.
41+
42+
Having both a `starter` and `app` project for each framework allows us to get a more comprehensive view of the performance of each framework across different use cases and levels of complexity and create more fair comparisons.
1443

1544
### How Metrics Work
1645

17-
We currently run scripts in the CI to measure and collect metrics for each framework. Right now we measure each frameworks starter project.
46+
Please note that metrics collection is in early stages and the process is likely to change as we iterate on it. The current process is as follows but is subject to change:
47+
48+
We currently run scripts in CI using GitHub Actions or if the test is more complex we run it locally in container and push the results to the repo.
49+
50+
#### CI Metrics Collection
1851

1952
The current flow for collecting metrics is as follows:
2053

@@ -51,12 +84,102 @@ All frameworks are configured in `.github/frameworks.json`. Each entry specifies
5184
},
5285
```
5386

87+
#### Container Metrics Collection
88+
89+
Coming soon but will also pull from `.github/frameworks.json`
90+
91+
### Versioning
92+
93+
We use [Dependabot](https://docs.github.com/en/code-security/dependabot) to keep framework versions up to date. Dependabot is configured to open PRs for minor and major version bumps on a weekly schedule, grouped by framework (e.g., a single PR for both `starter-nextjs` and `app-nextjs`).
94+
95+
When a Dependabot PR is opened, a CI workflow (`sync-version`) runs automatically to check that the `starter` and `app` packages for each framework are using the same version of the core framework package. If there is a mismatch, the CI will fail and provide instructions on which `package.json` to update. This ensures that we are always comparing the same version of a framework across both project types.
96+
97+
If you need to manually bump a framework version, make sure to update both the `starter-*` and `app-*` packages for that framework to the same version, then run `pnpm install` in each package directory to update the lockfile.
98+
5499
### Adding a New Framework
55100

56-
To add a new framework to the tracker:
101+
Adding a new framework increases the maintenance burden, so please open an issue to discuss it before starting work. If approved, follow these steps:
102+
103+
1. **Create the starter package**: Add a new directory in `packages/` (e.g., `packages/starter-my-framework`). Set it up using the framework's official CLI or getting started guide with the recommended defaults. The starter should not be added to the pnpm workspace — it has its own independent `package.json` and lockfile. Pin the core framework dependency to an exact version (e.g., `"my-framework": "2.0.0"` not `"^2.0.0"`), as Dependabot will handle version bumps.
104+
105+
2. **Create the app package** (optional): If runtime performance testing is planned, add an `app-*` package (e.g., `packages/app-my-framework`) with a more complex setup that includes features like dynamic routing or client-side interactivity. Make sure the framework dependency is pinned to the same exact version as the starter package.
106+
107+
3. **Add an entry to `.github/frameworks.json`**: Configure the framework's measurements:
108+
109+
```json
110+
{
111+
"name": "my-framework",
112+
"displayName": "My Framework",
113+
"frameworkPackage": "my-framework",
114+
"focusedFramework": false,
115+
"starter": {
116+
"package": "starter-my-framework",
117+
"buildScript": "build:my-framework",
118+
"buildOutputDir": "dist",
119+
"measurements": [
120+
{ "type": "install", "runFrequency": 5 },
121+
{ "type": "build", "runFrequency": 5 },
122+
{ "type": "dependencies" }
123+
]
124+
}
125+
}
126+
```
127+
128+
Set `focusedFramework` to `false` for new additions unless the framework is a priority for tracking.
129+
130+
4. **Add a Dependabot group in `.github/dependabot.yml`**: Add an entry so the framework's versions are automatically bumped. If the framework has both a starter and app package, group them together so they update in a single PR:
131+
132+
```yaml
133+
- package-ecosystem: 'npm'
134+
directories:
135+
- '/packages/starter-my-framework'
136+
- '/packages/app-my-framework'
137+
schedule:
138+
interval: 'weekly'
139+
day: 'monday'
140+
allow:
141+
- dependency-name: 'my-framework'
142+
ignore:
143+
- dependency-name: 'my-framework'
144+
update-types: ['version-update:semver-patch']
145+
```
146+
147+
5. **Test locally**: Make sure the framework builds successfully by running the build script from inside the package directory.
148+
149+
6. CI for `sync-version` and `validate-stats` will automatically run on the new framework once it's added to `frameworks.json`.
150+
151+
7. **Submit a PR**: Open a pull request with the new packages and configuration. Once merged, the CI will automatically pick up the new framework and raise a PR with new metrics.
152+
153+
### Getting Started
154+
155+
To get the project running locally:
156+
157+
1. **Prerequisites**: Make sure you have [Node.js](https://nodejs.org/) (v24+) and [pnpm](https://pnpm.io/) installed.
158+
159+
2. **Clone the repo**:
160+
161+
```bash
162+
git clone https://github.com/e18e/framework-tracker.git
163+
cd framework-tracker
164+
```
165+
166+
3. **Install workspace dependencies**:
167+
168+
```bash
169+
pnpm install
170+
```
171+
172+
This installs dependencies for the workspace packages (docs and stats-generator). The `starter-*` and `app-*` packages are not part of the workspace and have their own lockfiles.
173+
174+
4. **Run the docs site locally**:
57175

58-
1. Create a package in `packages/` (e.g., `packages/starter-astro`)
59-
2. Add necessary scripts to the root `package.json` (e.g., `"build:astro": "pnpm --filter starter-astro build"`)
60-
3. Add an entry to `.github/frameworks.json`:
176+
```bash
177+
pnpm dev:docs
178+
```
61179

62-
The CI will automatically pick up the new framework and run only the configured measurements.
180+
5. **Linting and formatting**:
181+
```bash
182+
pnpm lint:all # Run linting across workspace and framework packages
183+
pnpm format # Format code with Prettier
184+
pnpm type-check:all # Run type checking across workspace and framework packages
185+
```

‎README.md‎

Lines changed: 1 addition & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -9,84 +9,9 @@ This project aims to provide objective, data-driven comparisons of meta-framewor
99
- **Dev Time Performance**: Dependencies, build times, CI/CD performance, and developer experience
1010
- **Runtime Performance**: Core Web Vitals, bundle sizes, hydration strategies, and end-user experience
1111

12-
## Roadmap
13-
14-
See [initial-comparison-list.md](./initial-comparison-list.md) for the full list of frameworks we're tracking and the metrics we're measuring.
15-
1612
## Getting Involved
1713

18-
We welcome contributions from the community! Whether you're interested in adding new frameworks, improving existing metrics, or enhancing the documentation site, your help is appreciated. Check out [CONTRIBUTING.md](./CONTRIBUTING.md) for more details on how to get started.
19-
20-
## Project Structure
21-
22-
This is a monorepo managed with pnpm workspaces:
23-
24-
```
25-
framework-tracker/
26-
├── packages/
27-
│ ├── docs/ # Astro-based documentation site
28-
│ └── stats-generator/ # Tool for collecting framework metrics
29-
│ └── starter-*/ # Metaframeworks configured using default set up
30-
│ └── app-*/ # Metaframeworks configured for run time tests
31-
├── initial-comparison-list.md # Roadmap and metrics plan
32-
└── package.json # Workspace root configuration
33-
```
34-
35-
## Getting Started
36-
37-
### Prerequisites
38-
39-
- [Node.js](https://nodejs.org/) (version specified in package.json engines, or latest LTS)
40-
- [pnpm](https://pnpm.io/) (recommended package manager)
41-
42-
### Installation
43-
44-
```bash
45-
# Install dependencies
46-
pnpm install
47-
```
48-
49-
### Development
50-
51-
```bash
52-
# Start the documentation site in development mode
53-
pnpm dev:docs
54-
55-
# The site will be available at http://localhost:4321
56-
```
57-
58-
### Build
59-
60-
```bash
61-
# Build the documentation site for production
62-
pnpm build:docs
63-
```
64-
65-
### Preview
66-
67-
```bash
68-
# Preview the production build locally
69-
pnpm preview:docs
70-
```
71-
72-
### Code Quality
73-
74-
```bash
75-
# Check code formatting
76-
pnpm format:check
77-
78-
# Fix code formatting
79-
pnpm format
80-
81-
# Check for linting errors
82-
pnpm lint
83-
84-
# Automatically fix linting issues
85-
pnpm lint:fix
86-
87-
# Run TypeScript type checking
88-
pnpm type-check
89-
```
14+
We welcome contributions from the community! Whether you're interested in adding new frameworks, improving existing metrics, or enhancing the documentation site, your help is appreciated. Check out [CONTRIBUTING.md](./CONTRIBUTING.md) for more details on how to get started, our current roadmap, areas where we need help, and how the metrics collection process works.
9015

9116
## Data and Metrics
9217

‎Roadmap.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Framework Tracker Roadmap
2+
3+
Our current Roadmap stems from our initial comparison list which we created at the start of the project to define the scope and direction of our work. This list is not set in stone but it gives a good overview of the frameworks we're tracking and the metrics we're measuring [initial-comparison-list.md](./initial-comparison-list.md).
4+
5+
To make it clearer what we are actively working on, here is a breakdown of our current focus areas. This list is not exhaustive but it gives a good overview of where our efforts are right now.
6+
7+
## Current Focus Areas
8+
9+
- **SPA**: Adding a SPA test and metrics for each metaframework.
10+
- **In the Wild Data**: Pull in real world data for each framework from the HTTP Archive
11+
- **Bundle Analysis**: Scanning and checking output of each metaframework looking for things like use of modern web APIs and redundant dependencies or polyfills.
12+
13+
## Future Focus Areas
14+
15+
- **Improve HTTP Archive Data**: Enhance the quality and accuracy of data by improving metaframework detection and filtering techniques.

0 commit comments

Comments
 (0)