Skip to content

Commit 41facd3

Browse files
kptdobeclaude
andcommitted
fix(create-site): address PR review feedback and add eval
- Rewrite description to be utilitarian (remove 'zero-knowledge') - Remove Node.js version pin (18+ is archaic, loosen to just Node.js) - Generalise TodoWrite reference to agent-agnostic language - Add create-site-onboarding eval covering the non-obvious DA-specific behaviours: cache-first token auth, preview Bearer requirement, curl temp-file pattern, and aem-code-sync verification Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 76e637c commit 41facd3

3 files changed

Lines changed: 68 additions & 3 deletions

File tree

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"context": "Tests whether the agent correctly executes the full AEM EDS site creation flow: collecting inputs, creating the GitHub repo from the boilerplate template, handling the aem-code-sync human handoff with verification, authenticating with DA using a cache-first approach, creating nav/footer/index content via DA API, and triggering preview with the required Bearer auth. Covers the non-obvious DA-specific behaviours that trip up agents without this skill.",
3+
"type": "weighted_checklist",
4+
"checklist": [
5+
{
6+
"name": "Inputs collected before any API call",
7+
"max_score": 5,
8+
"description": "Agent collects GitHub org, repo name, and site name from the user before attempting any automated step"
9+
},
10+
{
11+
"name": "GitHub repo created from aem-boilerplate template",
12+
"max_score": 12,
13+
"description": "Agent uses gh CLI or GitHub API to create the repo specifically from the adobe/aem-boilerplate template, not a blank repo"
14+
},
15+
{
16+
"name": "aem-code-sync handoff correctly signalled",
17+
"max_score": 12,
18+
"description": "Agent provides the GitHub App installation URL, instructs the user to select only the new repo, and explicitly waits for user confirmation before continuing"
19+
},
20+
{
21+
"name": "aem-code-sync installation verified programmatically",
22+
"max_score": 8,
23+
"description": "After user confirmation, agent checks admin.hlx.page/status/{org}/{repo}/main/ returns a valid JSON response before proceeding to DA steps"
24+
},
25+
{
26+
"name": "DA auth uses cache-first approach",
27+
"max_score": 10,
28+
"description": "Agent checks ~/.aem/da-token.json for a valid cached token and its expiry before attempting a new OAuth flow or asking the user for a token"
29+
},
30+
{
31+
"name": "All three mandatory pages created in DA",
32+
"max_score": 15,
33+
"description": "Agent creates nav.html, footer.html, and index.html in DA (admin.da.live/source/{org}/{repo}/) with structurally correct HTML and the site name substituted"
34+
},
35+
{
36+
"name": "DA API calls use temp files, not inline multiline content",
37+
"max_score": 8,
38+
"description": "Agent writes HTML to temp files and uses the @ syntax for the curl -F flag rather than passing multiline content inline (inline multiline causes curl exit 26)"
39+
},
40+
{
41+
"name": "Preview requests include Bearer auth",
42+
"max_score": 15,
43+
"description": "Agent sends POST requests to admin.hlx.page/preview for nav, footer, and index — all with an Authorization: Bearer header. DA-sourced content returns 401 without it even on public repos."
44+
},
45+
{
46+
"name": "Complete hand-off delivered",
47+
"max_score": 10,
48+
"description": "Agent provides the preview URL (https://main--{repo}--{org}.aem.page/), DA content browser link, individual page edit links, and GitHub repo URL"
49+
},
50+
{
51+
"name": "Human handoff points clearly distinguished from automated steps",
52+
"max_score": 5,
53+
"description": "Agent clearly labels which steps require human action versus which it executes automatically, and does not proceed past handoff points without confirmation"
54+
}
55+
]
56+
}
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Create a New AEM Edge Delivery Site
2+
3+
## Problem/Feature Description
4+
5+
A developer wants to create a brand-new AEM Edge Delivery Services site. They have a GitHub account under the organization `acme-org` and want the repository to be named `acme-site`, with a site display name of "Acme Site". They have an Adobe IMS account with access to DA (da.live).
6+
7+
## Output Specification
8+
9+
Walk through the complete site creation flow using the create-site skill. Execute each automated step directly — do not stop to explain what you would do. Clearly signal the two steps that require human action (aem-code-sync installation and DA authentication if no cached token is available), and wait for confirmation before continuing past each. Deliver a live preview URL and DA editing links at the end.

skills/aem/edge-delivery-services/skills/create-site/SKILL.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: create-site
3-
description: Zero-knowledge onboarding skill for AEM Edge Delivery Services. Creates a new GitHub repo from the boilerplate, sets up aem-code-sync, creates initial content in DA (Document Authoring), and delivers a live preview URL. Use this skill whenever a user wants to create a new AEM Edge Delivery site from scratch.
3+
description: Creates a new AEM Edge Delivery site from scratch — GitHub repo from the boilerplate, aem-code-sync installation, initial DA content (nav, footer, homepage), and a live preview URL. Use this skill whenever a user wants to create a new AEM Edge Delivery site and no repository or DA content exists yet.
44
license: Apache-2.0
55
metadata:
66
version: "1.0.0"
@@ -26,7 +26,7 @@ Use this skill when:
2626
- A GitHub account with permission to create repositories in the target org
2727
- An Adobe IMS account with access to DA (da.live)
2828
- `gh` CLI authenticated (`gh auth status`) or a GitHub personal access token with `repo` scope
29-
- Node.js 18+ (for DA token management)
29+
- Node.js (for DA token management via da-auth-helper)
3030

3131
## Related Skills
3232

@@ -38,7 +38,7 @@ Use this skill when:
3838

3939
## Step 0: Create TodoList
4040

41-
Use the TodoWrite tool to create a todo list:
41+
Create a checklist to track progress (use your agent's task-tracking tool if available):
4242

4343
1. **Gather inputs** — org, repo name, site name collected
4444
2. **Create GitHub repository** — repo created from boilerplate template

0 commit comments

Comments
 (0)