Skip to content

Move from install to sync cmd #450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Move from install to sync cmd #450

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
!integration/testdata
.bin
bin/
build
.idea/
linux
darwin
windows
windows
11 changes: 10 additions & 1 deletion install_process.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,17 @@ func (p PoetryInstallProcess) Execute(workingDir, targetPath, cachePath string)
if !exists {
installOnly = "main"
}
poetryVersion, exists := os.LookupEnv("BP_POETRY_VERSION")
if !exists {
poetryVersion = "2.*"
}
installCmd := []string{"sync"}
// Can be remove once support for poetry v1 is removed
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought as we do this for other buildpacks.

Maybe the arguments could be configurable? Like BP_POETRY_INSTALL_ARGS. You could default that to the appropriate args for each version. Users could then override that if they want other args.

Seems like it would solve this problem but continue to be a useful feature beyond this as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, but I don't think the default install buildpack works as intended.

I have included the following example as a test project I created for myself.

pyproject.toml

[tool.poetry]
package-mode = false

[tool.poetry.dependencies]
python = "^3.13"
django = "^5.2.4"
fastapi = "^0.116.1"

project.toml

[build]
exclude = [
    ".venv",
]

Procfile

web: python3 main.py

main.py

try:
    import django
    print(django)
except ImportError:
    print("Fail to import django")
try:
    import fastapi
    print(fastapi)
except ImportError:
    print("Fail to import fastapi")

So I built an image with

pack build --builder=paketobuildpacks/builder-jammy-base:latest --run-image=paketobuildpacks/run-jammy-base:latest test-image

Then run it

docker run test-image

Output

<module 'django' from '/layers/paketo-buildpacks_poetry-install/poetry-venv/non-package-mode-xS3fZVNL-py3.13/lib/python3.13/site-packages/django/__init__.py'>
<module 'fastapi' from '/layers/paketo-buildpacks_poetry-install/poetry-venv/non-package-mode-xS3fZVNL-py3.13/lib/python3.13/site-packages/fastapi/__init__.py'>

As expected, okay, let's remove fastapi from pyproject.toml and update lock file. And repeat the build and run command.

Output

<module 'django' from '/layers/paketo-buildpacks_poetry-install/poetry-venv/non-package-mode-xS3fZVNL-py3.13/lib/python3.13/site-packages/django/__init__.py'>
<module 'fastapi' from '/layers/paketo-buildpacks_poetry-install/poetry-venv/non-package-mode-xS3fZVNL-py3.13/lib/python3.13/site-packages/fastapi/__init__.py'>

Hmm, weird, where does FastAPI come from? If I rebuild with no cache

<module 'django' from '/layers/paketo-buildpacks_poetry-install/poetry-venv/non-package-mode-xS3fZVNL-py3.13/lib/python3.13/site-packages/django/__init__.py'>
Fail to import fastapi

if strings.HasPrefix(poetryVersion, "1") {
installCmd = []string{"install", "--sync"}
}

args := []string{"install", "--only", installOnly}
args := append(installCmd, "--only", installOnly)

env := append(
os.Environ(),
Expand Down
45 changes: 43 additions & 2 deletions install_process_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ func testInstallProcess(t *testing.T, context spec.G, it spec.S) {
Expect(os.RemoveAll(packagesLayerPath)).To(Succeed())
Expect(os.RemoveAll(cacheLayerPath)).To(Succeed())
Expect(os.RemoveAll(workingDir)).To(Succeed())
os.Unsetenv("BP_POETRY_VERSION")
})

context("Execute", func() {
Expand All @@ -78,7 +79,7 @@ func testInstallProcess(t *testing.T, context spec.G, it spec.S) {

Expect(executableInvocations[0]).To(MatchFields(IgnoreExtras, Fields{
"Args": Equal([]string{
"install", "--only", "main",
"sync", "--only", "main",
}),
"Dir": Equal(workingDir),
"Env": ContainElements([]string{
Expand All @@ -100,7 +101,47 @@ func testInstallProcess(t *testing.T, context spec.G, it spec.S) {

Expect(venvDir).To(Equal("/some/path/to/some/venv"))
Expect(buffer.String()).To(ContainLines(
fmt.Sprintf(" Running 'POETRY_CACHE_DIR=%s POETRY_VIRTUALENVS_PATH=%s poetry install --only main'", cacheLayerPath, packagesLayerPath),
fmt.Sprintf(" Running 'POETRY_CACHE_DIR=%s POETRY_VIRTUALENVS_PATH=%s poetry sync --only main'", cacheLayerPath, packagesLayerPath),
" //some/path/xyz/../to/some/venv//",
" stderr output",
))
})

it("runs installation v1", func() {
os.Setenv("BP_POETRY_VERSION", "1.8.5")
venvDir, err := poetryInstallProcess.Execute(workingDir, packagesLayerPath, cacheLayerPath)
Expect(err).NotTo(HaveOccurred())

Expect(executable.ExecuteCall.CallCount).To(Equal(2))
Expect(executableInvocations).To(HaveLen(2))

Expect(executableInvocations[0]).To(MatchFields(IgnoreExtras, Fields{
"Args": Equal([]string{
"install", "--sync", "--only", "main",
}),
"Dir": Equal(workingDir),
"Env": ContainElements([]string{
fmt.Sprintf("BP_POETRY_VERSION=%s", "1.8.5"),
fmt.Sprintf("POETRY_VIRTUALENVS_PATH=%s", packagesLayerPath),
fmt.Sprintf("POETRY_CACHE_DIR=%s", cacheLayerPath),
}),
}))

Expect(executableInvocations[1]).To(MatchFields(IgnoreExtras, Fields{
"Args": Equal([]string{
"env", "info", "--path",
}),
"Dir": Equal(workingDir),
"Env": ContainElements([]string{
fmt.Sprintf("BP_POETRY_VERSION=%s", "1.8.5"),
fmt.Sprintf("POETRY_VIRTUALENVS_PATH=%s", packagesLayerPath),
fmt.Sprintf("POETRY_CACHE_DIR=%s", cacheLayerPath),
}),
}))

Expect(venvDir).To(Equal("/some/path/to/some/venv"))
Expect(buffer.String()).To(ContainLines(
fmt.Sprintf(" Running 'POETRY_CACHE_DIR=%s POETRY_VIRTUALENVS_PATH=%s poetry install --sync --only main'", cacheLayerPath, packagesLayerPath),
" //some/path/xyz/../to/some/venv//",
" stderr output",
))
Expand Down
65 changes: 63 additions & 2 deletions integration/default_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ func testDefault(t *testing.T, context spec.G, it spec.S) {
Expect(logs).To(ContainLines(" Executing build process"))
Expect(logs).To(ContainLines(
MatchRegexp(fmt.Sprintf(
" Running 'POETRY_CACHE_DIR=/layers/%s/cache POETRY_VIRTUALENVS_PATH=/layers/%s/poetry-venv poetry install --only main'",
" Running 'POETRY_CACHE_DIR=/layers/%s/cache POETRY_VIRTUALENVS_PATH=/layers/%s/poetry-venv poetry sync --only main'",
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
)),
Expand Down Expand Up @@ -134,7 +134,68 @@ func testDefault(t *testing.T, context spec.G, it spec.S) {
Expect(logs).To(ContainLines(" Executing build process"))
Expect(logs).To(ContainLines(
MatchRegexp(fmt.Sprintf(
" Running 'POETRY_CACHE_DIR=/layers/%s/cache POETRY_VIRTUALENVS_PATH=/layers/%s/poetry-venv poetry install --only main,dev'",
" Running 'POETRY_CACHE_DIR=/layers/%s/cache POETRY_VIRTUALENVS_PATH=/layers/%s/poetry-venv poetry sync --only main,dev'",
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
)),
))
Expect(logs).To(ContainLines(
MatchRegexp(` \- Installing ipython \(\d+\.\d+\.\d+\)`),
))
Expect(logs).ToNot(ContainLines(
MatchRegexp(` \- Installing ruff \(\d+\.\d+\.\d+\)`),
))
Expect(logs).To(ContainLines(MatchRegexp(` Completed in \d+\.\d+`)))
Expect(logs).To(ContainLines(
" Configuring build environment",
MatchRegexp(fmt.Sprintf(` PATH -> "/layers/%s/poetry-venv/default-app-.*-py\d+\.\d+/bin:\$PATH"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
MatchRegexp(fmt.Sprintf(` POETRY_VIRTUALENVS_PATH -> "/layers/%s/poetry-venv"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
MatchRegexp(fmt.Sprintf(` PYTHONPATH -> "/layers/%s/poetry-venv/default-app-.*-py\d+\.\d+/lib/python\d+\.\d+/site-packages:\$PYTHONPATH"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
"",
" Configuring launch environment",
MatchRegexp(fmt.Sprintf(` PATH -> "/layers/%s/poetry-venv/default-app-.*-py\d+\.\d+/bin:\$PATH"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
MatchRegexp(fmt.Sprintf(` POETRY_VIRTUALENVS_PATH -> "/layers/%s/poetry-venv"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
MatchRegexp(fmt.Sprintf(` PYTHONPATH -> "/layers/%s/poetry-venv/default-app-.*-py\d+\.\d+/lib/python\d+\.\d+/site-packages:\$PYTHONPATH"`, strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"))),
))

container, err = docker.Container.Run.
WithCommand("gunicorn server:app").
WithEnv(map[string]string{"PORT": "8080"}).
WithPublish("8080").
Execute(image.ID)
Expect(err).ToNot(HaveOccurred())

Eventually(container).Should(BeAvailable())
Eventually(container).Should(Serve(ContainSubstring("Hello, World!")).OnPort(8080))
})

it("builds and runs successfully with develop dependencies (poetry v1)", func() {
var err error
var logs fmt.Stringer

image, logs, err = pack.WithNoColor().Build.
WithPullPolicy("never").
WithEnv(map[string]string{
"BP_POETRY_VERSION": "1.8.5",
"BP_POETRY_INSTALL_ONLY": "main,dev",
}).
WithBuildpacks(
settings.Buildpacks.CPython.Online,
settings.Buildpacks.Pip.Online,
settings.Buildpacks.Poetry.Online,
settings.Buildpacks.PoetryInstall.Online,
settings.Buildpacks.BuildPlan.Online,
).
Execute(name, source)
Expect(err).ToNot(HaveOccurred(), logs.String)

Expect(logs).To(ContainLines(
MatchRegexp(fmt.Sprintf(`%s \d+\.\d+\.\d+`, buildpackInfo.Buildpack.Name)),
))
Expect(logs).To(ContainLines(" Executing build process"))
Expect(logs).To(ContainLines(
MatchRegexp(fmt.Sprintf(
" Running 'POETRY_CACHE_DIR=/layers/%s/cache POETRY_VIRTUALENVS_PATH=/layers/%s/poetry-venv poetry install --sync --only main,dev'",
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
strings.ReplaceAll(buildpackInfo.Buildpack.ID, "/", "_"),
)),
Expand Down
4 changes: 4 additions & 0 deletions integration/testdata/default_app/project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[build]
exclude = [
".venv",
]
Loading