|
| 1 | +# Stack Updates Workflow Instructions for Oryx |
| 2 | + |
| 3 | +This document provides comprehensive instructions for AI assistants working with the Oryx repository to handle stack version updates. This applies to all supported stacks: dotnet, node (nodejs), python, php |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The Oryx stack update workflow involves two main components: |
| 8 | +1. **SDK Publishing**: Building and publishing SDK artifacts to Azure storage accounts |
| 9 | +2. **Runtime Image Building**: Creating runtime images that reference the published SDKs |
| 10 | + |
| 11 | +## Key Files and Structure |
| 12 | + |
| 13 | +### 1. Central Configuration - `images/constants.yml` |
| 14 | +- **Purpose**: Central version configuration for runtime image building |
| 15 | +- **Content**: Version numbers, SHA256 hashes, GPG keys, and OS flavor mappings |
| 16 | +- **Format**: |
| 17 | + ```yaml |
| 18 | + node18Version: 18.20.8 |
| 19 | + node20Version: 20.19.3 |
| 20 | + node22Version: 22.17.0 |
| 21 | + php81Version: 8.1.33 |
| 22 | + php81Version_SHA: 9db83bf4590375562bc1a10b353cccbcf9fcfc56c58b7c8fb814e6865bb928d1 |
| 23 | + php81_GPG_keys: "528995BFEDFBA7191D46839EF9BA0ADA31CBD89E 39B641343D8C104B2B146DC3F9C39DC0B9698544" |
| 24 | + php82_GPG_keys: "1198C0117593497A5EC5C199286AF1F9897469DC 39B641343D8C104B2B146DC3F9C39DC0B9698544" |
| 25 | + php83_GPG_keys: "1198C0117593497A5EC5C199286AF1F9897469DC AFD8691FDAEDF03BDF6E460563F15A9B715376CA" |
| 26 | + php84_GPG_keys: "AFD8691FDAEDF03BDF6E460563F15A9B715376CA 9D7F99A0CB8F05C8A6958D6256A97AF7600A39A6" |
| 27 | + python310_GPG_keys: A035C8C19219BA821ECEA86B64E628F8D684696D |
| 28 | + python311_GPG_keys: A035C8C19219BA821ECEA86B64E628F8D684696D |
| 29 | + python39Version: 3.9.23 |
| 30 | + python310Version: 3.10.18 |
| 31 | + ASPNET_CORE_APP_80: 8.0.18 |
| 32 | + ASPNET_CORE_APP_80_SHA: 896e9cab7c3ea5384c174e7e2cffae3c7f8f9ed5d6d2b7434b5a2b0dc3f02b611ff8668f5d70c0b356a6a5d85a28fe40756cf356b168d0306370da11646b4b23 |
| 33 | + NET_CORE_APP_80: 8.0.18 |
| 34 | + NET_CORE_APP_80_SHA: 15d754a01c93183ea98bd608f2691193c86f284ec7feddfc810fad919e2f7ba20d41e1de45789fc1d9ac9fcd8be82d49cb8fe4c471dec892f91272fea2e53f08 |
| 35 | + ``` |
| 36 | +
|
| 37 | +### 2. Platform-Specific Version Files - `platforms/{stack}/versions/{os-flavor}/versionsToBuild.txt` |
| 38 | +- **Purpose**: Define which versions should be built and published for SDK generation |
| 39 | +- **Target OS Flavors**: The specific OS flavors to update are determined by the corresponding `{stackVersion}osFlavors` property in `constants.yml` |
| 40 | +- **Workflow**: When updating a stack version (e.g., Node.js 18), reference `node18osFlavors` in `constants.yml` to identify which OS flavor directories require `versionsToBuild.txt` updates |
| 41 | +- **File Structure**: |
| 42 | + ``` |
| 43 | + platforms/{stack}/versions/{os-flavor}/versionsToBuild.txt |
| 44 | + ``` |
| 45 | +- **Examples**: |
| 46 | + - For Node.js 20: Update files in OS flavors listed in `node20osFlavors` (typically `bullseye,bookworm`) |
| 47 | + - For Python 3.14: Update files in OS flavors listed in `python314osFlavors` (typically `noble`) |
| 48 | + - For .NET 9.0: Update files in OS flavors listed in `dotnet90osFlavors` (typically `bookworm`) |
| 49 | +
|
| 50 | +## OS Flavor Mapping |
| 51 | +
|
| 52 | +The repository supports multiple Debian/Ubuntu flavors: |
| 53 | +- **bullseye**: Debian 11 (current) |
| 54 | +- **bookworm**: Debian 12 (current) |
| 55 | +- **noble**: Ubuntu 24.04 (latest) |
| 56 | +
|
| 57 | +## Stack Update Workflow |
| 58 | +
|
| 59 | +### Step 1: Update versionsToBuild.txt Files |
| 60 | +
|
| 61 | +1. **Identify Target Versions**: Determine which stack versions need to be added/updated |
| 62 | +2. **Check OS Flavor Support**: Reference the osFlavors mapping in constants.yml to determine which OS flavors support the target version |
| 63 | +3. **Update Each OS Flavor File**: For each supported OS flavor, update the corresponding versionsToBuild.txt file with the new version: |
| 64 | +
|
| 65 | + **Node.js Format** (simple version list): |
| 66 | + ``` |
| 67 | + 18.20.8 |
| 68 | + 20.19.3 |
| 69 | + 22.17.0 |
| 70 | + ``` |
| 71 | +
|
| 72 | + **PHP Format** (version, SHA256, GPG keys - trailing comma required): |
| 73 | + ``` |
| 74 | + 8.1.33, 9db83bf4590375562bc1a10b353cccbcf9fcfc56c58b7c8fb814e6865bb928d1, 528995BFEDFBA7191D46839EF9BA0ADA31CBD89E 39B641343D8C104B2B146DC3F9C39DC0B9698544, |
| 75 | + 8.4.10, 475f991afd2d5b901fb410be407d929bc00c46285d3f439a02c59e8b6fe3589c, 1198C0117593497A5EC5C199286AF1F9897469DC 39B641343D8C104B2B146DC3F9C39DC0B9698544, |
| 76 | + ``` |
| 77 | +
|
| 78 | + **Python Format** (version, SHA256): |
| 79 | + ``` |
| 80 | + 3.11.13, fedcba0987654321abc123def456789012345678901234567890123456789012 |
| 81 | + 3.12.8, 1234567890abcdef123456789012345678901234567890123456789012345678 |
| 82 | + ``` |
| 83 | +
|
| 84 | + **.NET Format** (SDK version, SHA256): |
| 85 | + ``` |
| 86 | + 8.0.403, 15d754a01c93183ea98bd608f2691193c86f284ec7feddfc810fad919e2f7ba20d41e1de45789fc1d9ac9fcd8be82d49cb8fe4c471dec892f91272fea2e53f08 |
| 87 | + 9.0.100, e273b592ae9e1c75e91ce3be6f4f2d23143276900141e29c673d46490118d0115f6d1968ae6cfed598ca300fe889ba20fa060787ebb540308433580a3c6c5cd8 |
| 88 | + ``` |
| 89 | + |
| 90 | + **Important .NET Notes**: |
| 91 | + - **SDK versions** (e.g., 9.0.100) go in versionsToBuild.txt |
| 92 | + - **TWO runtime versions** go in constants.yml: `NET_CORE_APP_90` and `ASPNET_CORE_APP_90` |
| 93 | + - Runtime versions may differ from each other (e.g., both could be 9.0.7) |
| 94 | + - Each runtime component has its own SHA256 hash |
| 95 | +
|
| 96 | + **Important Notes**: |
| 97 | + - Maintain chronological or logical version ordering |
| 98 | + - Verify SHA256 hashes against official sources before adding |
| 99 | + - Ensure GPG keys are current and valid |
| 100 | +
|
| 101 | +### Step 2: Update constants.yml for Runtime Images |
| 102 | +
|
| 103 | +1. **Version Updates**: Update the central version declarations: |
| 104 | + ```yaml |
| 105 | + node22Version: 22.17.0 |
| 106 | + php84Version: 8.4.10 |
| 107 | + python314Version: 3.14.0rc2 |
| 108 | + # .NET has TWO runtime components (versions may differ): |
| 109 | + NET_CORE_APP_90: 9.0.7 # .NET Core runtime version |
| 110 | + ASPNET_CORE_APP_90: 9.0.7 # ASP.NET Core runtime version |
| 111 | + ``` |
| 112 | + |
| 113 | +2. **SHA256 Updates**: Update corresponding SHA256 hashes: |
| 114 | + ```yaml |
| 115 | + php84Version_SHA: 14983a9ef8800e6bc2d920739fd386054402f7976ca9cd7f711509496f0d2632 |
| 116 | + # .NET requires BOTH runtime component SHAs (usually different): |
| 117 | + NET_CORE_APP_90_SHA: e273b592ae9e1c75e91ce3be6f4f2d23143276900141e29c673d46490118d0115f6d1968ae6cfed598ca300fe889ba20fa060787ebb540308433580a3c6c5cd8 |
| 118 | + ASPNET_CORE_APP_90_SHA: b175d4d0578f9f5d735d59def3f44459462ef36b4bd07d9ca0ed9853bf42d90c7bace195ff264a9dcf6dd4d6d452c87059085146268fce3540ed58eaf39629eb |
| 119 | + ``` |
| 120 | +
|
| 121 | +3. **OS Flavor Mapping**: Update the osFlavors declarations if adding support for new OS versions: |
| 122 | + ```yaml |
| 123 | + node22osFlavors: bullseye,bookworm,noble # Added noble support |
| 124 | + ``` |
| 125 | +
|
| 126 | +### Step 3: Update Test Version Constants |
| 127 | +
|
| 128 | +Update version constants in BuildScriptGenerator source files for test compatibility: |
| 129 | +
|
| 130 | +- **Node.js**: `src/BuildScriptGenerator/Node/NodeVersions.cs` |
| 131 | +- **PHP**: `src/BuildScriptGenerator/PhpVersions.cs` |
| 132 | +- **Python**: `src/BuildScriptGenerator/PythonVersions.cs` |
| 133 | +- **.NET**: Update both files with **separate versions**: |
| 134 | + - `src/BuildScriptGenerator/DotNetCore/DotNetCoreRunTimeVersions.cs` - Add **runtime versions** |
| 135 | + - `src/BuildScriptGenerator/DotNetCore/DotNetCoreSdkVersions.cs` - Add **SDK versions** |
| 136 | + |
| 137 | +Add new version strings to the version arrays in these files to ensure tests recognize the new versions. **Important: .NET runtime and SDK versions are different!** |
| 138 | + |
| 139 | +### Step 4: Trigger SDK Build and Publication |
| 140 | + |
| 141 | +1. **Build Process**: The build system reads versionsToBuild.txt files to determine what to build |
| 142 | +2. **Storage Publishing**: SDKs are built and published to Azure storage accounts |
| 143 | +3. **Verification**: Built artifacts are verified using SHA256 hashes and/or GPG signatures |
| 144 | + |
| 145 | +### Step 5: Runtime Image Building |
| 146 | + |
| 147 | +1. **Dockerfile Updates**: Runtime Dockerfiles in `images/runtime/` reference constants.yml variables |
| 148 | +2. **Multi-Platform Builds**: Images are built for supported OS flavors |
| 149 | +3. **Registry Publishing**: Runtime images are published to container registries |
| 150 | + |
| 151 | +## Best Practices |
| 152 | + |
| 153 | +### Version Management |
| 154 | +- **Incremental Updates**: Add new versions while maintaining existing ones for backward compatibility |
| 155 | +- **Security Patches**: Prioritize security patch versions |
| 156 | +- **LTS Versions**: Give priority to Long Term Support versions |
| 157 | + |
| 158 | +### File Maintenance |
| 159 | +- **Consistent Formatting**: Maintain the exact format expected by each stack's build scripts |
| 160 | +- **Hash Verification**: Always include and verify SHA256 hashes for security |
| 161 | +- **GPG Key Management**: For PHP, ensure GPG keys are current and valid |
| 162 | + |
| 163 | +### Testing Strategy |
| 164 | +- **Development Environment**: Test changes in development storage accounts first |
| 165 | +- **OS Flavor Coverage**: Ensure all supported OS flavors are updated consistently |
| 166 | +- **Build Validation**: Verify that builds complete successfully before runtime updates |
| 167 | + |
| 168 | +## Common Tasks |
| 169 | + |
| 170 | +### Adding a New Stack Version |
| 171 | + |
| 172 | +**General Process for Any Stack (Node.js, PHP, Python, .NET):** |
| 173 | + |
| 174 | +1. **Check OS Flavor Support**: Look up `{stack}{version}osFlavors` in `constants.yml` |
| 175 | +2. **Update versionsToBuild.txt files**: Add version to each supported OS flavor |
| 176 | +3. **Update constants.yml**: Add version and SHA256 (if required) |
| 177 | +4. **Update Test Version Constants**: Update version constants in BuildScriptGenerator source files |
| 178 | + |
| 179 | +#### Node.js Example |
| 180 | +```bash |
| 181 | +# 1. Check constants.yml: node20osFlavors: bullseye,bookworm |
| 182 | +# 2. Update versionsToBuild.txt files |
| 183 | +echo "20.20.0" >> platforms/nodejs/versions/bullseye/versionsToBuild.txt |
| 184 | +echo "20.20.0" >> platforms/nodejs/versions/bookworm/versionsToBuild.txt |
| 185 | +
|
| 186 | +# 3. Update constants.yml |
| 187 | +# node20Version: 20.20.0 |
| 188 | +
|
| 189 | +# 4. Update Test Version Constants |
| 190 | +# In src/BuildScriptGenerator/Node/NodeVersions.cs, change Node20Version to "20.20.0" |
| 191 | +``` |
| 192 | + |
| 193 | +#### PHP Example |
| 194 | +```bash |
| 195 | +# 1. Check constants.yml: php84osFlavors: bullseye,bookworm |
| 196 | +# 2. Get SHA256 and GPG keys for PHP 8.4.11 |
| 197 | +# 3. Update versionsToBuild.txt files |
| 198 | +echo "8.4.11, <sha256>, <gpg_keys>," >> platforms/php/versions/bullseye/versionsToBuild.txt |
| 199 | +echo "8.4.11, <sha256>, <gpg_keys>," >> platforms/php/versions/bookworm/versionsToBuild.txt |
| 200 | +
|
| 201 | +# 4. Update constants.yml |
| 202 | +# php84Version: 8.4.11 |
| 203 | +# php84Version_SHA: <sha256> |
| 204 | +
|
| 205 | +# 5. Update Test Version Constants |
| 206 | +# In src/BuildScriptGenerator/PhpVersions.cs, change Php84Version to "8.4.11", update Php84TarSha256 to "<sha256>" and Php84GpgKeys to "<gpg_keys>" |
| 207 | +``` |
| 208 | + |
| 209 | +#### Python Example |
| 210 | +```bash |
| 211 | +# 1. Check constants.yml: python312osFlavors: bullseye,bookworm |
| 212 | +# 2. Get SHA256 for Python 3.12.8 |
| 213 | +# 3. Update versionsToBuild.txt files |
| 214 | +echo "3.12.8, <sha256>" >> platforms/python/versions/bullseye/versionsToBuild.txt |
| 215 | +echo "3.12.8, <sha256>" >> platforms/python/versions/bookworm/versionsToBuild.txt |
| 216 | +
|
| 217 | +# 4. Update constants.yml |
| 218 | +# python312Version: 3.12.8 |
| 219 | +
|
| 220 | +# 5. Update Test Version Constants |
| 221 | +# In src/BuildScriptGenerator/PythonVersions.cs, change Python312Version to "3.12.8" |
| 222 | +``` |
| 223 | + |
| 224 | +#### .NET Example |
| 225 | +```bash |
| 226 | +# 1. Check constants.yml: dotnet90osFlavors: bookworm |
| 227 | +# 2. Get SHA256 for .NET SDK 9.0.100 (NOT runtime version) |
| 228 | +# 3. Update versionsToBuild.txt files with SDK VERSION |
| 229 | +echo "9.0.100, <sdk_sha256>" >> platforms/dotnet/versions/bookworm/versionsToBuild.txt |
| 230 | +
|
| 231 | +# 4. Update constants.yml with BOTH RUNTIME COMPONENTS |
| 232 | +# NET_CORE_APP_90: 9.0.7 # .NET Core runtime version |
| 233 | +# NET_CORE_APP_90_SHA: <netcore_sha256> # .NET Core runtime SHA |
| 234 | +# ASPNET_CORE_APP_90: 9.0.7 # ASP.NET Core runtime version (may differ) |
| 235 | +# ASPNET_CORE_APP_90_SHA: <aspnet_sha256> # ASP.NET Core runtime SHA (different from .NET Core) |
| 236 | +
|
| 237 | +# 5. Update Test Version Constants with SEPARATE VERSIONS |
| 238 | +# DotNetCoreRunTimeVersions.cs: Add BOTH runtime versions "9.0.7" for NET_CORE_APP and ASPNET_CORE_APP |
| 239 | +# Also update their corresponding SHA256 hashes in the same file |
| 240 | +# DotNetCoreSdkVersions.cs: Add SDK version "9.0.100" |
| 241 | +``` |
| 242 | + |
| 243 | +### Adding Support for New OS Flavor |
| 244 | +```bash |
| 245 | +# 1. Create new version directory |
| 246 | +mkdir -p platforms/nodejs/versions/noble |
| 247 | +
|
| 248 | +# 2. Create versionsToBuild.txt with supported versions |
| 249 | +cp platforms/nodejs/versions/bookworm/versionsToBuild.txt platforms/nodejs/versions/noble/ |
| 250 | +
|
| 251 | +# 3. Update constants.yml osFlavors mapping |
| 252 | +# node20osFlavors: bullseye,bookworm,noble |
| 253 | +``` |
| 254 | + |
| 255 | +## Troubleshooting |
| 256 | + |
| 257 | +### Common Issues |
| 258 | +1. **Format Mismatch**: Each stack expects a specific format in versionsToBuild.txt |
| 259 | +2. **Missing Hashes**: PHP, DotNet require SHA256 hashes |
| 260 | +3. **GPG Key Issues**: PHP, Python builds may fail with invalid or missing GPG keys |
| 261 | + |
| 262 | +### Verification Steps |
| 263 | +1. **File Format**: Ensure versionsToBuild.txt follows the exact format for each stack |
| 264 | +2. **Hash Validation**: Verify SHA256 hashes match official sources |
| 265 | +3. **Build Logs**: Check build output for verification failures |
| 266 | +4. **Storage Account**: Confirm artifacts are published to expected storage locations |
| 267 | + |
| 268 | +## Related Documentation |
| 269 | +- Platform-specific build scripts in `platforms/{stack}/` |
| 270 | +- Runtime Dockerfile configurations in `images/runtime/` |
| 271 | + |
| 272 | +This workflow ensures consistent, verified, and reliable stack updates across the entire Oryx build system. |
0 commit comments