Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Triggering Paths #33395

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update Triggering Paths #33395

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
34a1c1d
touch a bunch of ci.yml files
scbedd Mar 14, 2025
fe88edb
add common/config to trigger paths for every core package
scbedd Mar 14, 2025
d3cfb84
Apply suggestions from code review
scbedd Mar 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 16 additions & 66 deletions eng/scripts/Language-Settings.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,8 @@ $MetadataUri = "https://raw.githubusercontent.com/Azure/azure-sdk/main/_data/rel
$GithubUri = "https://github.com/Azure/azure-sdk-for-js"
$PackageRepositoryUri = "https://www.npmjs.com/package"
$ReducedDependencyLookup = @{
'core' = @('@azure-rest/synapse-access-control', '@azure/arm-resources', '@azure/identity', '@azure/service-bus', '@azure/template')
'test-utils' = @('@azure-tests/perf-storage-blob', '@azure/arm-eventgrid', '@azure/ai-text-analytics', '@azure/identity', '@azure/template')
'identity' = @('@azure-tests/perf-storage-blob', '@azure/ai-text-analytics', '@azure/arm-resources', '@azure/identity-cache-persistence', '@azure/identity-vscode', '@azure/storage-blob', '@azure/template')
'test-utils' = @('@azure-tests/perf-storage-blob')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume these aren't in the trigger paths because these are artifacts in the ci.yml file. I wonder how much we care about these triggering those builds. It would be nice to eliminate this map and code to handle it completely.

Also if these aren't in the ci.yml artifact lists then is there even a packageinfo for them and can they even be included in the package set?

'identity' = @('@azure-tests/perf-storage-blob')
}

. "$PSScriptRoot/docs/Docs-ToC.ps1"
Expand Down Expand Up @@ -39,75 +38,26 @@ function Get-javascript-AdditionalValidationPackagesFromPackageSet {
[Parameter(Mandatory = $true)]
$AllPkgProps
)
$additionalValidationPackages = @()

function isOther($fileName) {
$startsWithPrefixes = @(".config", ".devcontainer", ".github", ".scripts", ".vscode", "common", "design", "documentation", "eng", "samples")

foreach ($prefix in $startsWithPrefixes) {
if ($fileName.StartsWith($prefix)) {
return $true
}
}

return $false
}

$targetedFiles = $diffObj.ChangedFiles
if ($diff.DeletedFiles) {
if (-not $targetedFiles) {
$targetedFiles = @()
}
$targetedFiles += $diff.DeletedFiles
}

# The targetedFiles needs to filter out anything in the ExcludePaths
# otherwise it'll end up processing things below that it shouldn't be.
foreach ($excludePath in $diffObj.ExcludePaths) {
$targetedFiles = $targetedFiles | Where-Object { -not $_.StartsWith($excludePath) }
}

$changedServices = @()
foreach ($file in $targetedFiles) {
$pathComponents = $file -split "/"
# handle changes only in sdk/<service>/<file>/<extension>
if ($pathComponents.Length -eq 3 -and $pathComponents[0] -eq "sdk") {
$changedServices += $pathComponents[1]
}

# handle any changes under sdk/<file>.<extension> or in the root of
# the repository
if (($pathComponents.Length -eq 2 -and $pathComponents[0] -eq "sdk") -or
($pathComponents.Length -eq 1)) {
$changedServices += "template"
}
}
$othersChanged = @()

if ($targetedFiles) {
$othersChanged = $targetedFiles | Where-Object { isOther $_ }
}
$changedServices = $changedServices | Get-Unique

if ($othersChanged) {
$additionalPackages = $ReducedDependencyLookup["core"] | ForEach-Object { $me = $_; $AllPkgProps | Where-Object { $_.Name -eq $me } | Select-Object -First 1 }
$additionalValidationPackages += $additionalPackages
}
$existingPackages = $LocatedPackages | ForEach-Object { $_.Name }
Copy link
Member

@benbp benbp Mar 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$existingPackages = $LocatedPackages | ForEach-Object { $_.Name }
$existingPackages = @($LocatedPackages | ForEach-Object { $_.Name })

Not sure if necessary, but if it is a singular result this could screw up the -notcontains check below.

$additionalDetectedPackages = @()
$uniqueResultSet = @()

# we don't currently have a way to trigger a package that doesn't exist in an artifact set
# so we can't trigger @azure-tests/perf-storage-blob based on changes to sdk/test-utils/ using any common tooling
# for now we will handle this in this function.
foreach ($changedService in $changedServices) {
if ($ReducedDependencyLookup.ContainsKey($changedService)) {
$additionalPackages = $ReducedDependencyLookup[$changedService] | ForEach-Object { $me = $_; $AllPkgProps | Where-Object { $_.Name -eq $me } | Select-Object -First 1 }
$additionalValidationPackages += $additionalPackages
}
else {
$additionalPackages = $AllPkgProps | Where-Object { $_.ServiceDirectory -eq $changedService }
$additionalValidationPackages += $additionalPackages
$additionalPackages = $ReducedDependencyLookup[$changedService] `
| ForEach-Object { $me = $_; $AllPkgProps | Where-Object { $_.Name -eq $me } | Select-Object -First 1 }

# we don't need to worry about duplicates here, we'll handle that later
$additionalDetectedPackages += $additionalPackages
}
}

$uniqueResultSet = @()
foreach ($pkg in $additionalValidationPackages) {
if ($uniqueResultSet -notcontains $pkg -and $LocatedPackages -notcontains $pkg) {
foreach ($pkg in $additionalDetectedPackages) {
$alreadyIncluded = $uniqueResultSet | ForEach-Object { $_.Name }
if ($existingPackages -notcontains $pkg.Name -and $alreadyIncluded -notcontains $pkg.Name) {
$pkg.IncludedForValidation = $true
$uniqueResultSet += $pkg
}
Expand Down
30 changes: 30 additions & 0 deletions sdk/core/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,31 +40,61 @@ extends:
Artifacts:
- name: azure-abort-controller
safeName: azureabortcontroller
triggeringPaths:
- /common/config/
- name: azure-core-amqp
safeName: azurecoreamqp
triggeringPaths:
- /common/config/
- name: azure-core-auth
safeName: azurecoreauth
triggeringPaths:
- /common/config/
- name: azure-core-client
safeName: azurecoreclient
triggeringPaths:
- /common/config/
- name: azure-rest-core-client
safeName: azurerestcoreclient
triggeringPaths:
- /common/config/
- name: azure-core-rest-pipeline
safeName: azurecorerestpipeline
triggeringPaths:
- /common/config/
- name: azure-core-lro
safeName: azurecorelro
triggeringPaths:
- /common/config/
- name: azure-core-paging
safeName: azurecorepaging
triggeringPaths:
- /common/config/
- name: azure-core-tracing
safeName: azurecoretracing
triggeringPaths:
- /common/config/
- name: azure-core-xml
safeName: azurecorexml
triggeringPaths:
- /common/config/
- name: azure-core-util
safeName: azurecoreutil
triggeringPaths:
- /common/config/
- name: azure-logger
safeName: azurelogger
triggeringPaths:
- /common/config/
- name: azure-core-http-compat
safeName: azurecorehttpcompat
triggeringPaths:
- /common/config/
- name: azure-core-sse
safeName: azurecoresse
triggeringPaths:
- /common/config/
- name: typespec-ts-http-runtime
safeName: typespectshttpruntime
triggeringPaths:
- /common/config/
4 changes: 3 additions & 1 deletion sdk/eventgrid/ci.mgmt.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.

trigger:
branches:
include:
Expand Down Expand Up @@ -30,3 +30,5 @@ extends:
Artifacts:
- name: azure-arm-eventgrid
safeName: azurearmeventgrid
triggeringPaths:
- /sdk/test-utils/
16 changes: 16 additions & 0 deletions sdk/identity/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,26 @@ extends:
Artifacts:
- name: azure-identity
safeName: azureidentity
triggeringPaths:
- /sdk/test-utils/
- /.config/
- /.devcontainer/
- /.github/
- /.scripts/
- /.vscode/
- /common/
- /design/
- /documentation/
- /eng/
- /samples/
- name: azure-identity-cache-persistence
safeName: azureidentitycachepersistence
triggeringPaths:
- /sdk/identity/
- name: azure-identity-broker
safeName: azureidentitybroker
- name: azure-identity-vscode
safeName: azureidentityvscode
skipPublishDocMs: true
triggeringPaths:
- /sdk/identity/
14 changes: 13 additions & 1 deletion sdk/resources/ci.mgmt.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.

trigger:
branches:
include:
Expand Down Expand Up @@ -32,5 +32,17 @@ extends:
Artifacts:
- name: azure-arm-resources
safeName: azurearmresources
triggeringPaths:
- /sdk/identity/
- /.config/
- /.devcontainer/
- /.github/
- /.scripts/
- /.vscode/
- /common/
- /design/
- /documentation/
- /eng/
- /samples/
- name: azure-arm-resources-profile-2020-09-01-hybrid
safeName: azurearmresourcesprofile20200901hybrid
13 changes: 12 additions & 1 deletion sdk/servicebus/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.

trigger:
branches:
include:
Expand Down Expand Up @@ -36,3 +36,14 @@ extends:
Artifacts:
- name: azure-service-bus
safeName: azureservicebus
triggeringPaths:
- /.config/
- /.devcontainer/
- /.github/
- /.scripts/
- /.vscode/
- /common/
- /design/
- /documentation/
- /eng/
- /samples/
4 changes: 3 additions & 1 deletion sdk/storage/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.

trigger:
branches:
include:
Expand Down Expand Up @@ -38,6 +38,8 @@ extends:
Artifacts:
- name: azure-storage-blob
safeName: azurestorageblob
triggeringPaths:
- /sdk/identity/
- name: azure-storage-file-share
safeName: azurestoragefileshare
- name: azure-storage-file-datalake
Expand Down
13 changes: 13 additions & 0 deletions sdk/template/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,5 +43,18 @@ extends:
Artifacts:
- name: azure-template
safeName: azuretemplate
triggeringPaths:
- /sdk/test-utils/
- /sdk/identity/
- /.config/
- /.devcontainer/
- /.github/
- /.scripts/
- /.vscode/
- /common/
- /design/
- /documentation/
- /eng/
- /samples/
- name: azure-template-dpg
safeName: azuretemplatedpg
3 changes: 3 additions & 0 deletions sdk/textanalytics/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,6 @@ extends:
Artifacts:
- name: azure-ai-text-analytics
safeName: azureaitextanalytics
triggeringPaths:
- /sdk/test-utils/
- /sdk/identity/
Loading