Skip to content

Refactor: Migrate to Bun, implement stacks and real-time stats #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 357 commits into
base: main
Choose a base branch
from

Conversation

Its4Nik
Copy link
Owner

@Its4Nik Its4Nik commented Mar 19, 2025

Summary by Sourcery

Refactor the project to use Bun.sh and Elysia.js, and introduce a plugin system for custom logic and notifications. The API now provides real-time container metrics via WebSocket, multi-host Docker environment monitoring, Compose stack deployment/management, historical stats storage (SQLite), Swagger API documentation, and a web dashboard.

New Features:

  • Introduce a plugin system for custom logic and notifications.
  • Provide real-time container metrics via WebSocket.
  • Add multi-host Docker environment monitoring.
  • Implement Compose stack deployment/management.
  • Incorporate historical stats storage (SQLite).
  • Provide Swagger API documentation.
  • Include a web dashboard (DockStat).

Summary by Sourcery

Refactor the project to use Bun.sh and Elysia.js, introducing a comprehensive Docker monitoring and management platform with enhanced features and architectural improvements.

New Features:

  • Implement a plugin system for custom logic and notifications
  • Add real-time container metrics via WebSocket
  • Introduce multi-host Docker environment monitoring
  • Develop Compose stack deployment and management functionality
  • Incorporate historical stats storage using SQLite
  • Provide Swagger API documentation
  • Create a web dashboard integration

Enhancements:

  • Migrate from Node.js to Bun runtime
  • Switch to Elysia.js framework
  • Implement type-safe tRPC procedures
  • Improve logging and error handling
  • Add comprehensive database abstraction layer
  • Enhance Docker host connection management

Build:

  • Update Dockerfile to use Bun runtime
  • Configure multi-stage Docker build process
  • Add GitHub Actions for CI/CD pipeline

CI:

  • Implement dependency graph generation workflow
  • Add automated testing and build verification
  • Configure multi-platform Docker image builds

Its4Nik and others added 30 commits January 3, 2025 07:48
* Chore: new testing workflows (dropping playwright)

* Chore: updating github workflow

* Fix: Fixing some minor things

* Chore: Updated to ES2020 syntax and AMD module

* Feat: startServer function to start the server with a different port

* Fix: Adjusting testing files based on workflow restrictions

* Fix: Adjusting testing files based on workflow restrictions

* Chore: Updating swagger (wrong branch bruh)

* Docs: Update swagger documentation (#26)

* Chore: Updated swagger

* Fix: Typo

* Fix: Fixing dockerfiles for prod/dev environment

* Feat: Add `/graph` and `/graph/image` endpoints (#27)

* Feat: Server side HTML generation => Client side rendering

* Fix: This _might_ fix the workflow

* Fix: Remove unused function

* Fix: Please make it stop

* Fix: Setting up python before hand

* Fix: Remove unused dep

* Fix: Using node20 instead of latest

* Fix: Works on my end...

* Feat: Master Nodes

* Feat: Icon for master node (needs testing)

* Fix: Adjusting function (needs testing)

* Fix: Adjusting function (needs testing)

* Fix: Removed some graph rendering features (will be back but better)

* Feat: render html file as png using puppeteer

ToFix: svgs dont render

* Fix: Hell yeah we got image creation!

* Fix: Adjusted routes since they need an absolute path

* Fix: Remove unused dependencies

* Fix: Exclude CWE-200 from CodeQl

* Feat: Respomse examples in swagger

Fix: Fixing some catch blocks

* Fix: Adjusting catches

* Fix: Adjusted catch to typing

* Feat: Stack creation

* Feat: Stack creation + starting and stopping

* Fix: Project root instead of path

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: Logging adustment

* Fix: Allow undescores and dashes in stack name

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: Propagate error

* Fix: Inline variable that is immediately returned

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: move some things around

* Fix: Minor adjustments

* Feat: Get a stack's docker-compose

* Feat: automatic Stack environmental file management

* Fix: sample-varaible.json adjustment

* Fix: Potential fix for code scanning alert no. 102: Log injection

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Fix: fix for code scanning alert no. 94: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Fix: fix for code scanning alert no. 92: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* FiX: fix for code scanning alert no. 106: Log injection

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Fix: Logger vulnerability and CI graph generation

* Feat: change logger verbosity and spelling fix

* Feat: ToDo comments to GH issue

* Fix: Add checkout

* Fix: May fix the ToDo workflow

* Fix: Remove todo

* Fix: Re-Add commit

* Fix: Remove TODO

* Fix: Re-add TODO

* Fix: Where tf did my package lock go 😭

* CI/CD: Remove ToDo

* CI/CD: Add ToDo

* CI/CD: Fix command

* CI/CD: Add checkout

* Fix: CPU value was a percentage the whole time?

* Feat: Websocket endpoints for logs and container metrics

* Fix: Make linter happy

* Fix: Fix import

* Fix: Fix tsc build

* Jest: Fix tests

* Jest: Fix Tests

* Fix: Typo in src/config/swagger.yaml

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: Typo in src/config/swagger.yaml

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: Tyypo in src/config/swagger.yaml

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* (code-quality): Inline variable that is immediately returned

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* (code-quality): Prefer object destructuring when accessing and using properties.

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* (code-quality): Prefer object destructuring when accessing and using properties.

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* (code-quality): Prefer object destructuring when accessing and using properties.

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix: Update extractHostData.ts

* Update TODO.md

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: ItsNik <[email protected]>
@Its4Nik
Copy link
Owner Author

Its4Nik commented Jun 2, 2025

Changelog: Changes in PR #43 (Last 100 Commits)

(Made by copilot, i'm a lazy guy)

Compared:


Major Highlights

  • Docker & CI/CD Refactor

    • Old Dockerfile and Dockerfile-dev deleted; replaced by docker/Dockerfile-base and docker/Dockerfile-dev.
    • Docker Compose file moved to docker/docker-compose.yaml with improved volume and user config.
    • All build, validation, and image workflow files updated to reference new Dockerfiles and context structure.
    • .gitignore updated for new Docker structure.
  • Scripts & Tooling

    • New npm scripts for Docker builds and local development.
    • Dependency graph and utility scripts moved to src/misc/dependencyGraphs and src/misc.
    • minifyDist.sh improved to only minify .js files and handle concurrency better.
  • Project Structure

    • Numerous scripts and config files moved for improved structure and clarity.
    • New folders for typings, handlers, and misc utilities.
    • Dependency graph outputs and scripts reorganized.
  • Dependency Updates

    • Updated TypeScript, ESLint, and many other dependencies in package.json and package-lock.json.
    • Improved type-safety, project typings (src/typings/), and removed redundant inline interface definitions.
    • Added new typings for atomic writes, host data, responses, Docker config, and high-availability.
  • Logging & Configuration

    • Logger refactored; config moved and streamlined.
    • Old loggerConfig file removed; new logger utility enhanced.
    • Atomic file writes implemented for safer concurrency (new utils/atomicWrite.ts).
    • Logging output improved for clarity and structure.
  • Controllers, Utils, Handlers

    • Major refactor to use atomic file writes in controllers, config, and utility modules.
    • High-availability, fetchData, and containerService modules improved and cleaned up.
    • All major modules now rely on centralized typings and atomic utility methods.
  • Workflow Improvements

    • Workflows (validation.yaml, build-image.yaml) now use context and file options for Docker build steps.
    • Node.js setup steps streamlined and some redundant steps removed.
    • test-building and validation jobs improved; dependencies and needs updated.
  • README & Docs

    • README badges and pipeline status improved for clarity and visual appeal.
    • TODO.md updated to reflect project progress and new priorities.

Notable File Changes

  • .github/workflows/build-image.yaml, .github/workflows/validation.yaml
  • .gitignore
  • Dockerfile (deleted), Dockerfile-dev (deleted)
  • docker/Dockerfile-base, docker/Dockerfile-dev (added)
  • docker-compose.yamldocker/docker-compose.yaml (renamed & updated)
  • package.json, package-lock.json
  • Scripts: src/misc/minifyDist.sh, src/misc/dependencyGraphs/createDependencyGraph.sh, etc.
  • Typings: src/typings/atomicWrite.ts, src/typings/hostData.ts, src/typings/dockerConfig.ts, etc.
  • Controllers, utils, and initialization files refactored for better structure and atomic operations.
  • Logger and configuration logic refactored and enhanced.

Breaking Changes

  • Docker Build & Run: All Docker-related scripts and file paths have changed. Update local tooling and CI/CD to use docker/Dockerfile-base, docker/Dockerfile-dev, and docker/docker-compose.yaml.
  • Type Definitions: Types and interfaces are now imported from src/typings rather than inline in modules.
  • Atomic File Writes: All critical file writes now use atomic operations. Adjust custom scripts accordingly.

Migration Notes

  • Review new npm scripts in package.json for updated development and deployment commands.
  • Adjust any custom Docker Compose or Docker build references to use new paths and file names.
  • If you have custom scripts that referenced now-deleted files or old structure, update them for compatibility.

For Full Details

Review all commits in the comparison range here:
👉 Compare 100 commits in PR #43

Its4Nik and others added 4 commits June 7, 2025 07:19
This commit updates `dockerode` to version 4.0.7 and adds `.gitai.json` to `.gitignore` and `.dockerignore` to prevent it from being included in the Docker image or tracked by Git. It also includes updates to other dependencies and devDependencies.
@Its4Nik Its4Nik linked an issue Jun 17, 2025 that may be closed by this pull request
Its4Nik and others added 2 commits June 19, 2025 04:41
feat(config): enhance plugin management and dependency updates

This commit introduces enhancements to the plugin management system and updates various dependencies.

The plugin management is enhanced by:
- Introducing a status indicator for plugins.
- Handling errors during plugin registration by setting the plugin status to "inactive".
- Displaying both active and inactive plugins in the plugin list.

Dependency updates include:
- Upgrading `@its_4_nik/gitai` to version 1.1.14.
- Upgrading `@types/dockerode` to version 3.3.41.
- Upgrading `@types/node` to version 22.15.32.

The changes improve the robustness and manageability of plugins and ensure the project is using the latest versions of its dependencies.
```
@Its4Nik Its4Nik closed this Jun 26, 2025
@github-project-automation github-project-automation bot moved this from In progress to Done in DockStatAPI Jun 26, 2025
@Its4Nik Its4Nik reopened this Jun 26, 2025
@Its4Nik Its4Nik moved this from Done to In progress in DockStatAPI Jun 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Breaking change enhancement New feature or request new feature A new feature will be added by this PR
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

Feat: Stacks auto scaling
2 participants