Skip to content

chore: change build producer in build + fix lint#328

Merged
shimib merged 4 commits into
llm-d:mainfrom
zdtsw-forking:chore_build_producer
Jul 24, 2026
Merged

chore: change build producer in build + fix lint#328
shimib merged 4 commits into
llm-d:mainfrom
zdtsw-forking:chore_build_producer

Conversation

@zdtsw

@zdtsw zdtsw commented Jul 20, 2026

Copy link
Copy Markdown
Member

What does this PR do?

  • root module used a local replace github.com/llm-d/llm-d-async/producer => ./producer can add as import dependency instead of replace by local changes
  • fix a "make lint" error: chose to update config to exclude the file

Why is this change needed?

/enhancement

How was this tested?

  • Unit tests added/updated
  • Integration/e2e tests added/updated
  • Manual testing performed

Checklist

  • Commits are signed off (git commit -s) per DCO
  • Code follows project contributing guidelines
  • Tests pass locally (make test)
  • Linters pass (make lint)
  • Documentation updated (if applicable)

Related Issues

Release note

NONE

cc @vishbhat @yizhaodev

Copilot AI review requested due to automatic review settings July 20, 2026 16:08
@zdtsw zdtsw changed the title Chore build producer chore: change build producer in build + fix lint Jul 20, 2026
@zdtsw
zdtsw force-pushed the chore_build_producer branch from 8b7edca to 4e7dfa8 Compare July 20, 2026 16:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the root Go module dependency setup to stop using a local replace for the producer submodule and adjusts golangci-lint configuration to address a make lint failure by excluding an additional Redis implementation file from gochecknoglobals.

Changes:

  • Remove the root go.mod local replace directive for github.com/llm-d/llm-d-async/producer => ./producer.
  • Add go.sum entries for github.com/llm-d/llm-d-async/producer v0.7.4 now that it’s resolved as a normal module dependency.
  • Update .golangci.yml to exclude pkg/redis/sortedset_impl.go from gochecknoglobals (via a combined path regex).

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File Description
go.mod Removes the local replace for the producer submodule so it can be resolved as a normal module dependency.
go.sum Records module checksums for producer v0.7.4 after switching away from local replacement.
.golangci.yml Expands gochecknoglobals exclusions to include sortedset_impl.go to fix lint failures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .golangci.yml
Comment thread go.mod

replace github.com/llm-d/llm-d-async/api => ./api

replace github.com/llm-d/llm-d-async/producer => ./producer

@yizhaodev yizhaodev Jul 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishbhat
should we also remove those?

replace github.com/llm-d/llm-d-async/api => ./api
replace github.com/llm-d/llm-d-async/pipeline => ./pipeline

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think api and pipeline are being used in some impl. We need it during dev, so that that changes in api and pipeline are immediately picked up.
For producer its fine because the only import is in e2e_test.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One item which we might need to handle: test/e2e/e2e_test.go still imports github.com/llm-d/llm-d-async/producer. After this change, it will resolve from the published v0.7.4 module instead of the local ./producer source. This is fine for CI, but if someone is iterating on both producer/ and e2e tests locally, they'll need to temporarily re-add the replace directive to pick up their local changes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should remove replace, and use go.work to use local module for development

@yizhaodev yizhaodev Jul 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go.work will like this, then you don't need hacking go.mod with replace

go 1.25.8

use (
    .
    ./api
    ./pipeline
    ./producer
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, but i think this is how was designed at the beginning to make these modulization.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a different topic, we should bump go to 1.26 sooner than later

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes bit for api and pipeline to use go.work now

zdtsw added 3 commits July 23, 2026 10:31
- root module used a local `replace github.com/llm-d/llm-d-async/producer
   => ./producer` can add as import dependency instead of replace by
   local changes

Signed-off-by: Wen Zhou <wenzhou@redhat.com>
The cleanupRequestStateScript Lua script global follows the same
package-level redis.NewScript pattern already grandfathered for
pkg/redis/redisimpl.go. Fold sortedset_impl.go into that existing
gochecknoglobals exclusion so make lint passes.

Signed-off-by: Wen Zhou <wenzhou@redhat.com>
in go.mod

- go.wok indicates use local api and pipeline
- need explicitly set GOWORK to off for "producer" when run "go" cli
  since we do not want it be in go.wok (use local) but still can verify
  the code locally if needed
- image build gets go.work and local api pipeline + download producer from pinned
  version

Signed-off-by: Wen Zhou <wenzhou@redhat.com>
@zdtsw zdtsw closed this Jul 23, 2026
@zdtsw zdtsw reopened this Jul 23, 2026
@zdtsw
zdtsw force-pushed the chore_build_producer branch 2 times, most recently from 8f2303a to f38da95 Compare July 23, 2026 08:57

@shimib shimib left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go.work approach is a nice improvement over the replace directives — api/pipeline resolve to local source for dev without version-skew, and it's idiomatic. Dockerfile (drops COPY producer/, which cmd/main.go never needs) and the GOWORK=off producer targets both look right.

One suggestion, picking up the thread above: add ./producer to the workspace too —

use (
    .
    ./api
    ./pipeline
    ./producer
)

test/e2e/e2e_test.go imports producer, so as written the e2e exercises the published v0.7.4 rather than local ./producer (as @vishbhat noted). Adding it to go.work restores the local co-dev/e2e loop at no cost — cd producer && GOWORK=off go test still validates producer standalone against its published deps, so you keep the "works as a released module" check either way. There isn't a strong reason to single producer out.

Minor (also flagged by Copilot): a targeted //nolint:gochecknoglobals on the specific cleanupRequestStateScript var in sortedset_impl.go would preserve lint signal for any future globals vs. excluding the whole file — though the file-level exclusion is consistent with the existing redisimpl.go rule, so either is fine.

- keep producer in go.work, drop it in Dockerfile build

Signed-off-by: Wen Zhou <wenzhou@redhat.com>
@zdtsw

zdtsw commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

The go.work approach is a nice improvement over the replace directives — api/pipeline resolve to local source for dev without version-skew, and it's idiomatic. Dockerfile (drops COPY producer/, which cmd/main.go never needs) and the GOWORK=off producer targets both look right.

One suggestion, picking up the thread above: add ./producer to the workspace too —

use (
    .
    ./api
    ./pipeline
    ./producer
)

test/e2e/e2e_test.go imports producer, so as written the e2e exercises the published v0.7.4 rather than local ./producer (as @vishbhat noted). Adding it to go.work restores the local co-dev/e2e loop at no cost — cd producer && GOWORK=off go test still validates producer standalone against its published deps, so you keep the "works as a released module" check either way. There isn't a strong reason to single producer out.

Minor (also flagged by Copilot): a targeted //nolint:gochecknoglobals on the specific cleanupRequestStateScript var in sortedset_impl.go would preserve lint signal for any future globals vs. excluding the whole file — though the file-level exclusion is consistent with the existing redisimpl.go rule, so either is fine.

thanks @shimib for the review
i've done some updates for (1)

  1. add producer into go.work but leave the COPY of producer out of container build.

as for (2):
keep lint fix in .golanci.yml --as file level exclusion for consistency with existing redisimpl.go rule.

@shimib

shimib commented Jul 24, 2026

Copy link
Copy Markdown
Member

/approve

@shimib
shimib merged commit a44115d into llm-d:main Jul 24, 2026
8 checks passed
@shimib

shimib commented Jul 24, 2026

Copy link
Copy Markdown
Member

Heads-up before we cut v0.8.0: dropping the root replace directives in favor of go.work breaks the release flow. Surfacing it here so we can decide the fix together.

Root cause. go.work overrides where a module's source is read from, but Go's module-graph loader still requires every required sibling version to actually exist (tagged or replaced). With only go.work and no root replaces, bumping the root requires to a not-yet-tagged version fails.

Proof (on current main, bumping the root requires to v0.8.0):

  • go.work only → go build ./... fails: github.com/llm-d/llm-d-async/pipeline@v0.8.0: reading .../pipeline/go.mod at revision pipeline/v0.8.0: unknown revision pipeline/v0.8.0
  • Re-add the root replace directives → go build ./... succeeds.

(go list -m pipeline reports the local dir either way — but building still needs the required version to exist.)

Impact on releasing:

  1. make set-version VERSION=v0.8.0 fails at the root go mod tidy step (same reason).
  2. More importantly, the version-bump PR itself won't build — pre-commit CI runs go vet ./... / go test, which need v0.8.0 before that tag exists. Chicken-and-egg. The pre-chore: change build producer in build + fix lint #328 replace directives avoided this because they fully override without needing a tag.

Dockerfile wrinkle. Simply re-adding the root replaces isn't sufficient on its own: this PR's Dockerfile now go work edit -dropuse ./producer and no longer COPY producer/, so a permanent replace producer => ./producer would break go mod download in the image (missing dir). Partial replaces don't help either — go vet ./... compiles test/e2e, which imports producer, so all three siblings must resolve.

Options (your call — you designed the go.work migration):

  1. Restore the three root replace directives and revert the Dockerfile -dropuse ./producer back to COPY producer/; keep go.work as a dev/IDE convenience. Restores the working release flow.
  2. Move to a tag-first release flow: tag the repo, let CI create the submodule tags, then bump the root requires in a follow-up once the tags exist.
  3. Something else you have in mind.

Happy to open whichever PR you prefer.

cc @zdtsw @yizhaodev @vishbhat

@yizhaodev

Copy link
Copy Markdown
Contributor

Option 2 (tag-first) would require a 2-step release flow, which adds complexity.

Option 1 seems simpler: I prefer that option.

shimib added a commit to shimib/llm-d-async that referenced this pull request Jul 24, 2026
llm-d#328 replaced the root replace directives with go.work and made the
Dockerfile drop ./producer. But go.work overrides module *source*, not
the module-graph version requirement, and `go mod tidy`/`docker build`
ignore it — so any bump to a not-yet-tagged version (e.g. v0.8.0) fails
to build (PR container-build, release image build, and `make set-version`).

Restore the three root replace directives and revert the Dockerfile to
COPY producer/ so all siblings resolve locally without needing tags.
go.work is kept for IDE/dev; llm-d#328's lint fix and Makefile tweaks are
retained. See llm-d#328 for the broader discussion.

Signed-off-by: Shimi Bandiel <shimib@google.com>
@shimib

shimib commented Jul 24, 2026

Copy link
Copy Markdown
Member

Following up on the release-flow breakage above: I've had to partially revert this PR in #348 (the v0.8.0 release PR), because without it v0.8.0 can't build or release at all.

What #348 reverts:

  • Re-adds the three root replace directives.
  • Reverts the Dockerfile back to COPY producer/ (drops the go work edit -dropuse ./producer step).

Why a targeted revert instead of a full git revert of this PR: a full revert would also remove the .golangci.yml change here, which reintroduces a gochecknoglobals failure on pkg/redis/sortedset_impl.go (the package-level Redis script vars). So I kept this PR's lint fix, the Makefile GOWORK=off tweaks, and go.work itself — only the two release-breaking pieces are reverted.

Root cause recap for the record: go.work overrides where a module's source is read from, but the module-graph loader still needs every required sibling version to exist (tagged or replaced), and both go mod tidy and docker build ignore go.work entirely. With go.work alone, bumping to an untagged version fails in three places — PR container-build, the tag-time release image build (its Dockerfile runs in parallel with CI - Tag Go Submodules, so the submodule tags don't exist yet), and make set-version.

go.work is still valuable for local/IDE multi-module dev, so this isn't "go.work bad" — it's "go.work is not a substitute for replace in the release/build path." If we want to drop the replaces again, it needs a release-aware approach (e.g. keep replace for build/release resolution alongside go.work, or a tag-first release flow). Happy to pair on that as a follow-up after v0.8.0 ships.

@zdtsw

zdtsw commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

Following up on the release-flow breakage above: I've had to partially revert this PR in #348 (the v0.8.0 release PR), because without it v0.8.0 can't build or release at all.

What #348 reverts:

  • Re-adds the three root replace directives.
  • Reverts the Dockerfile back to COPY producer/ (drops the go work edit -dropuse ./producer step).

Why a targeted revert instead of a full git revert of this PR: a full revert would also remove the .golangci.yml change here, which reintroduces a gochecknoglobals failure on pkg/redis/sortedset_impl.go (the package-level Redis script vars). So I kept this PR's lint fix, the Makefile GOWORK=off tweaks, and go.work itself — only the two release-breaking pieces are reverted.

Root cause recap for the record: go.work overrides where a module's source is read from, but the module-graph loader still needs every required sibling version to exist (tagged or replaced), and both go mod tidy and docker build ignore go.work entirely. With go.work alone, bumping to an untagged version fails in three places — PR container-build, the tag-time release image build (its Dockerfile runs in parallel with CI - Tag Go Submodules, so the submodule tags don't exist yet), and make set-version.

go.work is still valuable for local/IDE multi-module dev, so this isn't "go.work bad" — it's "go.work is not a substitute for replace in the release/build path." If we want to drop the replaces again, it needs a release-aware approach (e.g. keep replace for build/release resolution alongside go.work, or a tag-first release flow). Happy to pair on that as a follow-up after v0.8.0 ships.

sounds reasonable, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants