How to build go code with private repo lib and enable goproxy ?

[morya]

Guys, I am trying packs, trying to understand builder, and buildpack.
but I run into a minor problem, which blocks me from build go source code.

I use packs inside a place where accessing https://go.dev/dl/go1.20.9.linux-amd64.tar.gz and other go lib dependencies quite hard.
But one cant use env variables with GOPRIVATE and GOPROXY to hack a little bit.

And, how could I enable this inside a packs tool?

with GOPRIVATE

thanks a lot

---

edited:

What I need:

• download dist from a mirror, like
  • download go1.20.9.linux-amd64.tar.gz
  • download node-18.xz
  • download python-3.10/ python-3.11.xz
• download common packages from mirror, like
  • install moment pkg from mirror instead of from official npm, ( maybe we are inside a company)
  • install tornado (python ilb) from mirror
  • install github.com/labstack/echo/v4 from mirror
• download pkg from a private git server
  • install a go get git.my-company.com/lib/encrypt from git.my-company.com, and with personal access token

[robdimsdale]

In general the platform (pack, etc) propagates environment variables from outside to inside (via the pack build -e <env> command).

However, I don't think those variables are applicable to the downloading of the go toolchain itself.

How would you download that file outside of the buildpacks ecosystem? E.g. with curl. Are there other environment variables you would set to enable curl, wget, etc to work?

[morya]

In my opion, it should be a common need to install packages/libs from a mirror and install from a

• private npm registry
• private golang lib
• private python package

and, one cannot download a tree of dependencies with curl and it shouldnot be like this.

What I need:

• download dist from a mirror, like
  • download go1.20.9.linux-amd64.tar.gz
  • download node-18.xz
  • download python-3.10/ python-3.11.xz
• download common packages from mirror, like
  • install moment pkg from mirror instead of from official npm, ( maybe we are inside a company)
  • install tornado (python ilb) from mirror
  • install github.com/labstack/echo/v4 from mirror
• download pkg from a private git server
  • install a go get git.my-company.com/lib/encrypt from git.my-company.com, and with personal access token

[robdimsdale]

I agree. What I'm asking for is how you would solve this problem outside of buildpacks.

Specifically, how would you download go1.20.9.linux-amd64.tar.gz in your current environment?

[morya]

Yes, I am using dockerfile build ECO system.

FROM golang:1.20 as builder    # no need to download golang dist package
WORKDIR /code
COPY . /code

ENV GOPROXY=https://goproxy.xxx.io   # set a special proxy, so you can download a lot packages quite fast
ENV GOPRIVATE=git.my-company.com # so you donnot download some lib from official site.
COPY ./netrc /root/.netrc   # there are personal access token inside this file, so Golang could download private libs without being blocked

RUN go build -o app 

FROM debian:buster
COPY --from=builder /code/app /app

CMD ["/app"]

that's it.

There is only one dis-advantadge, without setting GOPATH, you cannot reuse gopath cache. I am using Drone with volume binding, to support this feature currently.

[robdimsdale]

Gotcha. So you're not downloading the go distribution directly, you're pre-downloading a docker image and (presumably) hosting that on a local registry.

[morya]

Yes, is there a better solution for these needs,
and, including simillar requirements with python/node/php and things like that...

It would be the best if I can set a pipy mirror like http://mirrors.aliyun.com/pypi/simple/.

[morya]

I posted a new issue here too.

https://github.com/orgs/paketo-buildpacks/discussions/223