Skip to content

Commit 40b2a4d

Browse files
committed
feat: add support for using remote (Git) templates
Enables fetching templates directly from remote Git URIs. This change vastly improves template management and rendering efficiency. ### New Remote Fetch Behavior The template fetch process is optimized for speed and minimal data transfer: * **Shallow Fetch:** Uses `git fetch --depth 1` for minimal network transfer. * **Sparse Checkout:** Leverages Git sparse-checkout (requires Git 2.25+) to retrieve **only** the directory containing the template, not the whole repository. ### Rendering Pipeline Change The entire apiproxy and sharedflow rendering pipeline is now simplified: * The entire contents of the directory containing the template are copied into a temporary directory. * The template is rendered from within this temporary directory. * The rendered output is transformed into the final Apigee proxy or shared flow from within this same directory. This makes it so that we no longer need to manually copy supporting resources (like specs or Javascript) during the rendering process, cleaning up and simplifying the example templates.
1 parent da590bf commit 40b2a4d

27 files changed

+1099
-92
lines changed

docs/render/commands/render-apiproxy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ The `render apiproxy` command takes the following parameters:
3636

3737

3838
```text
39-
-t, --template string path to main template"
39+
-t, --template string path to main template
4040
-i, --include string path to helper templates (globs allowed)
4141
-o, --output string output directory or file
4242
--debug boolean prints rendered template before transforming into API proxy"

docs/render/commands/render-sharedflow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ The `render sharedflow` command takes the following parameters:
2828

2929

3030
```text
31-
-t, --template string path to main template"
31+
-t, --template string path to main template
3232
-i, --include string path to helper templates (globs allowed)
3333
-o, --output string output directory or file
3434
--debug boolean prints rendered template before transforming into shared flow"

docs/render/commands/render-template.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Under the hood, this command uses the Go [text/template](https://pkg.go.dev/text
2727
The `render template` command takes the following parameters:
2828

2929
```text
30-
-t, --template string path to main template"
30+
-t, --template string path to main template
3131
-i, --include string path to helper templates (globs allowed)
3232
-o, --output string output directory or file
3333
-d, --dry-run boolean prints rendered template to stdout"

docs/render/mcp.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,20 @@
11
# MCP
2+
<!--
3+
Copyright 2025 Google LLC
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
218

319
!!! Note
420

@@ -46,7 +62,6 @@ Generate the API proxy bundle using the `render apiproxy` command. This command
4662
```shell
4763
apigee-go-gen render apiproxy \
4864
--template ./examples/templates/mcp/apiproxy.yaml \
49-
--include ./examples/templates/mcp/*.tmpl \
5065
--set-oas spec=./examples/specs/oas3/weather.yaml \
5166
--set base_path=/mcp/weather \
5267
--output ./out/apiproxies/weather.zip

docs/render/using-custom-helpers.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Using Custom Helpers
22
<!--
3-
Copyright 2024 Google LLC
3+
Copyright 2025 Google LLC
44
55
Licensed under the Apache License, Version 2.0 (the "License");
66
you may not use this file except in compliance with the License.
@@ -18,10 +18,11 @@
1818
Sometimes it's useful to create your own custom helper functions that you can use during the
1919
template rendering process.
2020

21-
Use the `--include` flag to specify one or more files containing helper functions.
21+
You can define helper functions within the main template file itself, or in separate helper files.
2222

23-
!!!Note
24-
You can pass the `--include` flag multiple times, or even use [glob patterns](https://pkg.go.dev/github.com/bmatcuk/doublestar#readme-patterns) to include multiple files.
23+
By default, the tool will look for a file named `_helpers.tpl` in the same directory as the template, and include it.
24+
25+
Alternatively, you can use the `--include` flag to specify one or more files containing helper functions.
2526

2627

2728
## Example
@@ -33,9 +34,9 @@ Below is a sample helper function. Let's see how to use it.
3334
{{- end -}}
3435
```
3536

36-
1. First, place this block inside a helper file (e.g. `helper.tmpl`)
37+
1. First, place this block inside a helper file (e.g. `my_helpers.tpl`)
3738

38-
2. Then, pass `--include ./helper.tmpl` to the `render`) command.
39+
2. Then, pass `--include ./my_helpers.tpl` to the `render`) command.
3940

4041
Finally, in order to invoke the custom helper function, use the `{{ include ... }}` built-in function from your main template.
4142

docs/render/using-git-templates.md

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Using Git Templates
2+
3+
The `--template` argument, used in various `apigee-go-gen render` commands, accepts two types of sources for your template files: a local file path or a remote file path via a Git repository URI.
4+
5+
## 1. Local File Path
6+
7+
This is the simplest usage, pointing directly to a file on your local machine.
8+
9+
```
10+
# Relative path example
11+
apigee-go-gen render ... --template ./templates/my_api.yaml.tpl
12+
13+
# Absolute path example
14+
apigee-go-gen render ... --template /home/user/project/templates/my_api.yaml.tpl
15+
```
16+
17+
## 2. Remote Git Repository URI
18+
19+
!!! Info
20+
To use this remote fetching feature, you need **Git 2.25 or later** installed and available in your `$PATH`.
21+
22+
You can specify a template file located in a public or private Git repository using a structured URI. The tool fetches only the necessary file and its containing directory using a sparse checkout, minimizing download time.
23+
24+
A Git URI is composed of three parts: **Repository URL**, **Reference (Ref)**, and **Resource Path**.
25+
26+
### Git URI Structure
27+
28+
The Git URI must conform to one of two primary formats for parsing: **Web-View** or **Generic**.
29+
30+
| Format | Purpose |
31+
|--------------|---------------------------------------------------------------------|
32+
| **Web-View** | Standard format seen in GitHub, GitLab, and Bitbucket. |
33+
| **Generic** | A flexible format for any Git URL, including SSH and generic HTTPS. |
34+
35+
### A. Web-View Format
36+
37+
These formats match the path structure you see when browsing files on major Git hosting platforms.
38+
39+
| Style | Structure |
40+
|---------------|--------------------------------------|
41+
| **GitHub** | `<REPO_URL>/blob/<REF>/<RESOURCE>` |
42+
| **GitLab** | `<REPO_URL>/-/blob/<REF>/<RESOURCE>` |
43+
| **BitBucket** | `<REPO_URL>/src/<REF>/<RESOURCE>` |
44+
45+
#### Example Usage
46+
47+
```
48+
# Using GitHub web-view format with the 'main' branch
49+
apigee-go-gen render ... --template https://github.com/my-org/templates/blob/main/templates/my/api.yaml
50+
51+
# Using GitLab web-view format with a specific tag
52+
apigee-go-gen render ... --template https://gitlab.com/apigee/starter/-/blob/v1.2.3/templates/my/api.yaml
53+
54+
# Using BitBucket web-view format with a commit hash
55+
apigee-go-gen render ... --template https://bitbucket.org/team/project/src/a1b2c3d4e5f6/templates/my/api.yaml
56+
```
57+
58+
### B. Generic Format
59+
60+
This flexible format uses the `/-/` separator and is useful for non-HTTP Git protocols (like SSH) or when you need explicit control over the **Repository URI**, **Reference** and **Resource** .
61+
62+
| Style | Structure | Purpose |
63+
|-------------|-----------------------------|---------------------------------------------------------------|
64+
| **Simple** | `<REPO>/-/<REF>/<RESOURCE>` | Used when `<REF>` has no slashes (e.g., `main`). |
65+
| **Complex** | `<REPO>/-/<REF>#<RESOURCE>` | Used when `<REF>` contains slashes (e.g., `feature/new-api`). |
66+
67+
#### Example Usage
68+
69+
```
70+
# Using SSH protocol (implicit) with a simple branch name ('main')
71+
apigee-go-gen render ... --template git@github.com:my-org/project.git/-/main/my/api.yaml
72+
73+
# Using SSH protocol (explicit) with a simple branch name ('main')
74+
apigee-go-gen render ... --template ssh://git@github.com:my-org/project.git/-/main/my/api.yaml
75+
76+
# Using HTTPS protocol with a tag ('v1.2.3')
77+
apigee-go-gen render ... --template https://gitlab.com/my-org/project.git/-/v1.2.3/my/api.yaml
78+
79+
# Using a reference with a slash (branch 'feature/jira-1337') requiring '#'
80+
apigee-go-gen render ... --template https://repo.example.com/project.git/-/feature/jira-1337#my/api.yaml
81+
```
82+
83+
### Reference Types
84+
85+
The reference (`<REF>`) component of the URI is flexible and can point to any valid Git object:
86+
87+
* **Branch Name:** The name of a branch (e.g., `main`, `develop`, `feature/new-service`).
88+
89+
* **Tag:** A version tag (e.g., `v1.0.0`, `release-2024-05`).
90+
91+
* **Commit Hash:** A full or short commit hash (e.g., `a1b2c3d4e5f6` or `a1b2c3d`).
92+
93+
## Private Repositories (SSH)
94+
95+
To access private Git repositories, the preferred and most secure mechanism is using **SSH keys**. This requires your SSH client to have the correct key loaded into the `ssh-agent`.
96+
97+
### SSH Key Setup
98+
99+
Before attempting to fetch a private template, ensure you have an SSH key pair generated and added to your Git hosting platform account (GitHub, GitLab, etc.).
100+
101+
**A. Generate an SSH Key Pair**
102+
103+
If you do not have an SSH key, use `ssh-keygen`. We recommend using the default file name (`id_rsa` or `id_ed25519`) and adding a secure passphrase.
104+
105+
```sh
106+
ssh-keygen -t ed25519 -C "your_email@example.com"
107+
```
108+
109+
**B. Add Key to SSH-Agent**
110+
111+
You must start the `ssh-agent` and add your private key so that Git can use it without prompting for your passphrase on every fetch operation.
112+
113+
```sh
114+
# 1. Start the ssh-agent in the background
115+
eval "$(ssh-agent -s)"
116+
117+
# 2. Add your private key (use the path to your generated key)
118+
ssh-add ~/.ssh/id_ed25519
119+
```
120+
121+
### Troubleshooting SSH Connectivity
122+
123+
If you encounter issues when attempting to fetch a template, it is often related to the client's knowledge of the Git host's server key.
124+
125+
**Error: `ssh: handshake failed: knownhosts: key is unknown`**
126+
127+
This indicates that your `~/.ssh/known_hosts` file does not contain the public SSH key for the Git server (e.g., GitHub.com or Bitbucket.org). You need to add it:
128+
129+
```sh
130+
# Replace bitbucket.org with your Git host (e.g., github.com, gitlab.com)
131+
ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts
132+
```
133+
134+
**Error: `ssh: handshake failed: knownhosts: key mismatch`**
135+
136+
This means the existing key for the Git SSH server in your `~/.ssh/known_hosts` file is incorrect (e.g., the host changed their key). You must first remove the old key, and then add the new one.
137+
138+
```sh
139+
# 1. Remove the existing, incorrect key
140+
ssh-keygen -R bitbucket.org
141+
142+
# 2. Add the correct host keys
143+
ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts
144+
```
145+
146+
147+
## Private Repositories (HTTPS)
148+
149+
If you cannot use SSH, you can access private repositories using the HTTPS URI format by embedding a Personal Access Token (PAT) as the password component.
150+
151+
1. **Generate a PAT:** Create a Personal Access Token in your Git hosting platform's security settings. This token must have the scope necessary to read repository contents (`repo` or similar).
152+
153+
2. **Use the Token in the URI:** The token replaces the password in the standard HTTPS URL structure.
154+
155+
**Structure:** `https://<USERNAME>:<TOKEN>@<GIT_HOST>/<REPO_PATH>`
156+
157+
!!! Warning
158+
This method is less secure than using SSH, and should be avoided if possible.
159+
Your Personal Access Token is briefly written to a temporary `.git` directory during the `git checkout` operation.
160+
Once the `git checkout` operation completes, the `.git` directory is deleted.
161+
162+
#### Example Usage (GitHub)
163+
164+
```
165+
# Replace 'USERNAME' and 'TOKEN'
166+
apigee-go-gen render --template https://USERNAME:TOKEN@github.com/my-org/templates/-/main/path/to/template.tpl
167+
```

docs/render/using-graphql-schema.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Using GraphQL Schema
22
<!--
3-
Copyright 2024 Google LLC
3+
Copyright 2025 Google LLC
44
55
Licensed under the Apache License, Version 2.0 (the "License");
66
you may not use this file except in compliance with the License.
@@ -44,7 +44,6 @@ apigee-go-gen render apiproxy \
4444
--set-string "api_name=resorts-api" \
4545
--set-string "base_path=/resorts/graphql" \
4646
--set-string "target_url=https://example.com/graphql" \
47-
--include ./examples/templates/graphql/*.tmpl \
4847
--output ./out/apiproxies/resorts.zip
4948
```
5049

@@ -56,6 +55,5 @@ apigee-go-gen render apiproxy \
5655
--set-string "api_name=resorts-api" \
5756
--set-string "base_path=/resorts/graphql" \
5857
--set-string "target_url=https://example.com/graphql" \
59-
--include ./examples/templates/graphql/*.tmpl \
6058
--output ./out/apiproxies/resorts.zip
6159
```

docs/render/using-grpc-proto.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Using gRPC Proto
22
<!--
3-
Copyright 2024 Google LLC
3+
Copyright 2025 Google LLC
44
55
Licensed under the Apache License, Version 2.0 (the "License");
66
you may not use this file except in compliance with the License.
@@ -46,7 +46,6 @@ apigee-go-gen render apiproxy \
4646
--template ./examples/templates/grpc/apiproxy.yaml \
4747
--set-grpc proto=./examples/protos/greeter.proto \
4848
--set-string "target_server=example-target-server" \
49-
--include ./examples/templates/grpc/*.tmpl \
5049
--output ./out/apiproxies/greeter.zip
5150
```
5251

@@ -56,6 +55,5 @@ apigee-go-gen render apiproxy \
5655
--template ./examples/templates/grpc/apiproxy.yaml \
5756
--set-grpc proto=./examples/protos/greeter.proto \
5857
--set-string "target_server=example-target-server" \
59-
--include ./examples/templates/grpc/*.tmpl \
6058
--output ./out/apiproxies/greeter
6159
```

docs/render/using-openapi-description.md

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Using OpenAPI Description
22
<!--
3-
Copyright 2024 Google LLC
3+
Copyright 2025 Google LLC
44
55
Licensed under the Apache License, Version 2.0 (the "License");
66
you may not use this file except in compliance with the License.
@@ -43,7 +43,6 @@ Here is how you would use the [render apiproxy](./commands/render-apiproxy.md) c
4343
apigee-go-gen render apiproxy \
4444
--template ./examples/templates/oas3/apiproxy.yaml \
4545
--set-oas spec=./examples/specs/oas3/petstore.yaml \
46-
--include ./examples/templates/oas3/*.tmpl \
4746
--output ./out/apiproxies/petstore.zip
4847
```
4948

@@ -52,7 +51,6 @@ apigee-go-gen render apiproxy \
5251
apigee-go-gen render apiproxy \
5352
--template ./examples/templates/oas3/apiproxy.yaml \
5453
--set-oas spec=./examples/specs/oas3/petstore.yaml \
55-
--include ./examples/templates/oas3/*.tmpl \
5654
--output ./out/apiproxies/petstore
5755
```
5856

@@ -64,16 +62,15 @@ Add the `--dry-run xml` or `--dry-run yaml` flag.
6462

6563
Note that dry-run is only useful when the rendered template produces valid YAML.
6664

67-
If your template has issues, and it does not produce valid YAML, you can use the `--debug true` flag.
68-
69-
This will print out the rendered template before even attempting to parse it as YAML.
65+
!!! Note
66+
If your template has issues, and it does not produce valid YAML, you can use the `--debug true` flag.
67+
This will print out the rendered template before even attempting to parse it as YAML.
7068

7169

7270
=== "XML output"
7371
```shell
7472
apigee-go-gen render apiproxy \
7573
--template ./examples/templates/oas3/apiproxy.yaml \
7674
--set-oas spec=./examples/specs/oas3/petstore.yaml \
77-
--include ./examples/templates/oas3/*.tmpl \
7875
--dry-run xml
7976
```

0 commit comments

Comments
 (0)