Skip to content

Tweak Nx caching settings #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Tweak Nx caching settings #98

wants to merge 6 commits into from

Conversation

jumski
Copy link
Contributor

@jumski jumski commented May 6, 2025

Summary by CodeRabbit

  • Chores
    • Added or updated caching settings for build, test, and other project tasks across multiple packages.
    • Standardized output directory paths for build targets.
    • Removed deprecated publish targets from some project configurations.
    • Updated lint ignore paths for improved accuracy.
    • Improved input/output tracking for test targets to enhance build determinism and performance.

jumski added 2 commits May 6, 2025 08:55
…vements

- Enabled caching for several project targets across multiple packages
- Added outputs and cache options to improve build performance
- Adjusted dependencies and build targets for consistency
- Updated test and dev commands to optimize workflows
- Removed redundant or disabled cache settings where appropriate
…fix ignore path in edge-worker

- Changed output paths to use {workspaceRoot} prefix across multiple packages
- Corrected ignore path in edge-worker to point to dist directory instead of src
- Updated build output targets for example-flows and website to match new output structure
Copy link

changeset-bot bot commented May 6, 2025

⚠️ No Changeset found

Latest commit: 7a9054e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

coderabbitai bot commented May 6, 2025

Warning

Rate limit exceeded

@jumski has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 13 minutes and 52 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 1722c04 and 7a9054e.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml (1 hunks)
  • .github/workflows/release.yml (1 hunks)

Walkthrough

This update introduces explicit caching controls and output path specifications across multiple project configuration files. Targets for building, testing, and serving now declare cache settings and, where applicable, output directories. Some legacy publish targets are removed. These changes standardize and clarify the build system's behavior regarding cache usage and artifact locations.

Changes

File(s) Change Summary
examples/playground/project.json Added "cache": false to the "build" target, explicitly disabling caching for this build configuration.
pkgs/cli/project.json Added explicit "cache" properties to build (true), serve (false), test (true), test:e2e:install (true), and test:e2e:compile (true) targets. Declared outputs for the build target. Minor formatting adjustment in test target.
pkgs/core/project.json Removed jsr:publish target. Added "cache": true to build, lint:sqruff, lint, test, test:pgtap, test:vitest. Added "cache": false to fix-sql, and all supabase:* targets. Added trailing commas in dependency arrays.
pkgs/dsl/project.json Removed jsr:publish target. Changed build target's outputs to pkgs/dsl/dist and added "cache": true. Added "cache": true to test target.
pkgs/edge-worker/project.json Added "cache": false to build, all supabase:*, and db:ensure targets. Enabled caching for test:nx-deno, lint, test:unit, test:integration, and aggregate test targets. Updated lint ignore path. Added input/output tracking for test:unit and test:integration. Added dependencies for test:integration.
pkgs/example-flows/project.json Changed build target's outputs to pkgs/example-flows/dist and enabled caching with "cache": true.
pkgs/website/project.json Changed build target's outputs to pkgs/website/dist and enabled caching. Set "cache": false for dev and preview targets.

Sequence Diagram(s)

sequenceDiagram
    participant Developer
    participant NxExecutor
    participant CacheSystem
    participant FileSystem

    Developer->>NxExecutor: Run target (e.g., build, test)
    NxExecutor->>CacheSystem: Check cache (if cache: true)
    alt Cache enabled and hit
        CacheSystem-->>NxExecutor: Return cached outputs
    else Cache enabled and miss or cache: false
        NxExecutor->>FileSystem: Execute target steps
        FileSystem-->>NxExecutor: Produce outputs
        alt Cache enabled
            NxExecutor->>CacheSystem: Store outputs in cache
        end
    end
    NxExecutor-->>Developer: Return results
Loading

Poem

A hop and a skip, I cache and I build,
With outputs declared, my tasks are fulfilled.
Old publish commands, I bid you adieu,
Now every target knows just what to do!
With caching set true or sometimes set false,
My burrow of configs is neat—what a waltz!
🐇✨


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

netlify bot commented May 6, 2025

Deploy Preview for pgflow-demo canceled.

Name Link
🔨 Latest commit 7a9054e
🔍 Latest deploy log https://app.netlify.com/sites/pgflow-demo/deploys/6819b8959993f700087cedaf

Copy link

nx-cloud bot commented May 6, 2025

View your CI Pipeline Execution ↗ for commit 7a9054e.

Command Status Duration Result
nx affected -t lint typecheck ❌ Failed 28s View ↗

☁️ Nx Cloud last updated this comment at 2025-05-06 07:23:02 UTC

- Add explicit step names for checkout, setup-node, and other actions for clarity
- Specify fetch-depth: 0 for full history in checkout steps
- Update pnpm version to 8.14.1 in setup actions
- Add step names for installing Deno and Atlas tools
- Enhance readability and maintainability of CI and release workflows
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
examples/playground/project.json (1)

9-10: Optional: Define outputs for build target
If the build target will eventually produce artifacts, consider adding an "outputs" array (e.g. "outputs": ["dist/examples/playground"]) so that caching can be re-enabled with a real executor in the future.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c531a93 and 1722c04.

📒 Files selected for processing (7)
  • examples/playground/project.json (1 hunks)
  • pkgs/cli/project.json (3 hunks)
  • pkgs/core/project.json (7 hunks)
  • pkgs/dsl/project.json (1 hunks)
  • pkgs/edge-worker/project.json (6 hunks)
  • pkgs/example-flows/project.json (1 hunks)
  • pkgs/website/project.json (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: Cloudflare Pages
🔇 Additional comments (33)
examples/playground/project.json (1)

9-10: Cache setting explicitly disabled for 'build'
This aligns with the broader pattern of controlling cache behavior per target and ensures that this noop executor won’t be cached.

pkgs/edge-worker/project.json (14)

9-10: Explicitly disable caching on noop build target
Clearing dependsOn and setting "cache": false for the build executor (noop) ensures every run is fresh and not pulled from cache.


30-31: Enable caching for Deno-based tests
Adding "cache": true to the test:nx-deno target with defined outputs allows Nx to reuse results and speed up repeated test runs.


37-39: Update lint ignore path and enable caching
Changing the ignore path to pkgs/edge-worker/dist/ skips generated artifacts, and enabling caching speeds up lint checks on unchanged code.


47-48: Disable caching for supabase:start
Side-effectful commands like supabase start must run every time; disabling cache is correct.


56-57: Disable caching for supabase:stop
Disabling cache ensures the stop command always executes to clean up state.


65-66: Disable caching for supabase:status
The status output may vary over time, so caching it could mask state changes.


74-75: Disable caching for supabase:restart
Combining stop/start commands should always execute fresh; disabling cache is appropriate.


90-91: Disable caching for supabase:reset
Database reset operations must always run; caching would skip critical migration steps.


102-103: Disable caching for supabase:functions-serve
Serving pipeline functions should reflect the latest code on every run.


112-113: Disable caching for db:ensure
Ensuring database schema is an environment-specific task; disabling cache prevents stale states.


118-122: Add explicit inputs/outputs and enable caching for unit tests
Specifying inputs and outputs for the test:unit target ensures proper cache invalidation and reuse for subsequent runs.

Also applies to: 129-131


134-134: Add dependencies, inputs/outputs and enable caching for integration tests
Defining dependsOn, explicit file globs in inputs, and outputs allows Nx to enforce ordering and cache integration test results.

Also applies to: 136-140, 147-149


159-160: Disable caching for E2E tests
E2E tests interact with external state; disabling cache ensures they always run against a fresh environment.


163-164: Enable caching for aggregate test target
Enabling cache on the top-level test target (which aggregates unit and integration) will speed up composite pipelines when inputs haven’t changed.

pkgs/example-flows/project.json (2)

11-11: Explicitly define build outputs
Good addition: specifying "outputs": ["{workspaceRoot}/pkgs/example-flows/dist"] ensures Nx can track the directory of generated artifacts for this library.


18-19: Enable caching for build target
Adding "cache": true here aligns with other packages and will accelerate incremental builds.

pkgs/dsl/project.json (2)

17-19: Standardize build outputs and enable caching
Specifying "outputs": ["{workspaceRoot}/pkgs/dsl/dist"] alongside "cache": true brings consistency with other packages and speeds up rebuilds.


27-28: Cache test results
Enabling "cache": true for the test target will cache test outputs under coverage, reducing redundant work on unchanged tests.

pkgs/website/project.json (3)

10-16: Standardize build outputs and enable caching
Defining explicit "outputs": ["{workspaceRoot}/pkgs/website/dist"], matching the outputPath, and setting "cache": true ensures consistent artifact tracking and faster builds.


23-24: Disable caching for dev server
"cache": false on the dev target is appropriate for a long-running development process, ensuring changes are always picked up.


31-32: Disable caching for preview command
Disabling cache on preview prevents stale builds when previewing local artifacts.

pkgs/cli/project.json (5)

23-25: Standardize build outputs and enable caching
Adding "outputs": ["{workspaceRoot}/pkgs/cli/dist"] and "cache": true aligns the CLI package with other libraries and improves incremental build performance.


32-33: Disable caching for serve target
Disabling cache ("cache": false) on the serve command is correct for a continuously running process that should always reflect the latest code.


37-38: Cache noop test aggregator
Marking the test noop target as cacheable avoids redundant invocations of the grouped E2E commands.


52-53: Cache E2E install step
Enabling "cache": true on test:e2e:install can significantly speed up CI builds if the Supabase setup is deterministic. Please verify that caching side-effects here won't skip necessary initialization.


67-68: Cache E2E compile step
Similarly, caching test:e2e:compile will improve CI performance. Ensure generated artifacts are invalidated correctly on source changes.

pkgs/core/project.json (6)

76-79: Enable caching and outputs for build
Adding "outputs": ["{workspaceRoot}/pkgs/core/dist"] and setting "cache": true ensures compiled artifacts are tracked and speeds up incremental builds.


89-91: Enable caching for lint targets
Marking both lint:sqruff and the composite lint target as cacheable avoids redundant lint executions on unchanged schemas.

Also applies to: 94-95


106-108: Disable caching for SQL fixes
The fix-sql target intentionally disables caching ("cache": false), forcing fresh lint/fix runs each time.


117-119: Disable caching for Supabase lifecycle commands
Disabling cache on all supabase:* targets prevents stale database state, ensuring each command runs afresh as required.

Also applies to: 126-128, 135-137, 144-146, 153-155, 163-164


168-170: Enable caching for test workflows
Caching the test suite aggregators (test, test:pgtap, test:vitest) speeds up CI by skipping runs when inputs haven’t changed.

Also applies to: 178-180, 188-190


215-216: Cache generated types tasks
Enabling cache for gen-types and verify-gen-types ensures type generation and verification steps are skipped when inputs haven’t changed, improving iteration speed.

Also applies to: 234-235

Copy link

cloudflare-workers-and-pages bot commented May 6, 2025

Deploying pgflow with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7a9054e
Status: ✅  Deploy successful!
Preview URL: https://bf5c2aa2.pgflow.pages.dev
Branch Preview URL: https://update-nx-caching-settings.pgflow.pages.dev

View logs

jumski added 3 commits May 6, 2025 09:11
…urations

- Updated project.json files in core and edge-worker packages to set cache to false
- Ensures that all test, lint, verify, and build commands do not use caching
- Improves consistency and reliability of test and build processes by avoiding cached results
…n, and enhance project

cache settings

- Enabled Nx Cloud task distribution and removed deprecated start-ci-run command
- Refined cache settings for core and edge-worker projects to ensure proper caching
- Updated CI steps for database-dependent tests to depend on cacheable jobs
- Improved overall CI configuration for better performance and cache management
…uild steps

Refactors CI workflow to skip non-cacheable targets, run only necessary affected
targets, and introduce dedicated steps for database-dependent builds and tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant