Skip to content

Azure Storage Sample Automation #39944

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

Yionse
Copy link
Member

@Yionse Yionse commented Mar 5, 2025

By changing the current Storage samples, can enable them to run automatically in the pipeline.

For the latest pipeline results: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=4713055&view=results

The supported packages are as follows:

  • azure-storage-blob
  • azure-storage-file-datalake
  • azure-storage-file-share
  • azure-storage-queue

The specific modifications are as follows:

  • Skipped azure-storage-blob/blob_samples_proxy_configuration.py because it uses a proxy and is not suitable for running on pipelines.

  • Change the environment variable name in samples to be consistent with that exported in test-resources.json.

  • Change the path of the local files used in the samples to an absolute address.
    Update

    SOURCE_FOLDER = "./sample-blobs/"
    

    to

    current_dir = os.path.dirname(os.path.abspath(__file__))
    SOURCE_FOLDER = os.path.join(current_dir, "./sample-blobs/")
    
  • Comment out the following code snippet in blob_samples_container_access_policy.py, as setting the allowBlobPublicAccess attribute in test-resources.json is unstable and may not always take effect.

    container_client.set_container_access_policy(signed_identifiers=identifiers, public_access=public_access)
    
  • Create required resources and add output in test-resources.json.

@xiangyan99 for notification.

@github-actions github-actions bot added the Storage Storage Service (Queues, Blobs, Files) label Mar 5, 2025
@Yionse Yionse force-pushed the feat/automation_storage branch 2 times, most recently from 000080f to 6e560df Compare March 5, 2025 03:23
@azure-sdk
Copy link
Collaborator

API change check

API changes are not detected in this pull request.

@Yionse Yionse force-pushed the feat/automation_storage branch 3 times, most recently from ecb648c to e3c413c Compare March 6, 2025 02:53
@Yionse Yionse force-pushed the feat/automation_storage branch 2 times, most recently from d79c18d to 91e72c1 Compare March 7, 2025 06:47
@Yionse Yionse force-pushed the feat/automation_storage branch from 4cae2c5 to 0a3561d Compare March 14, 2025 02:10
@Yionse Yionse marked this pull request as ready for review March 14, 2025 09:25
@Copilot Copilot AI review requested due to automatic review settings March 14, 2025 09:25
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR automates Azure Storage samples for pipeline execution by standardizing environment variable names, adjusting local file paths to absolute paths, and commenting out unstable code sections.

  • Environment variables have been renamed from AZURE_STORAGE_* to STORAGE_* in most samples.
  • File paths have been converted from relative to absolute paths in several samples.
  • Unstable container access policy code has been commented out in async samples.

Reviewed Changes

Copilot reviewed 67 out of 67 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
sdk/storage/azure-storage-blob/samples/blob_samples_client_side_encryption_keyvault.py Updated env var names and added key generation logic for client-side encryption.
sdk/storage/azure-storage-blob/samples/blob_samples_enumerate_blobs_async.py Renamed environment variable and updated container name for async enumeration.
sdk/storage/azure-storage-blob/samples/blob_samples_enumerate_blobs.py Renamed environment variable and updated container name for blob enumeration.
sdk/storage/azure-storage-blob/samples/blob_samples_directory_interface*.py Renamed environment variable and hardcoded container name for directory interface samples.
scripts/devops_tasks/test_run_samples.py Added proxy configuration sample to the list of skipped samples.
sdk/storage/azure-storage-blob/samples/blob_samples_container_access_policy_async.py Renamed environment variable and commented out unstable access policy setting.
sdk/storage/azure-storage-blob/samples/blob_samples_copy_blob*.py Updated env var names and container names; added container creation call.
sdk/storage/azure-storage-blob/samples/blob_samples_containers*.py Renamed environment variable, updated file path for source file, and changed container names.
sdk/storage/azure-storage-blob/samples/blob_samples_authentication* Updated environment variable names for connection string and account keys.
sdk/storage/azure-storage-blob/samples/blob_samples_client_side_encryption.py Updated environment variable name in sample header and connection string retrieval.
sdk/storage/azure-storage-blob/samples/blob_samples_common*.py Renamed env var and updated local file path for sample source file.
Comments suppressed due to low confidence (2)

sdk/storage/azure-storage-blob/samples/blob_samples_common.py:32

  • The environment variable used here is 'STORAGE_CONNECTION_STRING_SOFT', which is inconsistent with other samples that reference 'STORAGE_CONNECTION_STRING'. Please unify the variable naming.
connection_string = os.getenv("STORAGE_CONNECTION_STRING_SOFT")

sdk/storage/azure-storage-blob/samples/blob_samples_common_async.py:31

  • The use of 'STORAGE_CONNECTION_STRING_SOFT' here is inconsistent with other parts of the code that expect 'STORAGE_CONNECTION_STRING'. Consider using the same environment variable name across all samples.
connection_string = os.getenv("STORAGE_CONNECTION_STRING_SOFT")

@ChenxiJiang333 ChenxiJiang333 force-pushed the feat/automation_storage branch from 2767729 to b6292f7 Compare March 18, 2025 04:25
@v-xuto v-xuto requested review from lmazuel and mccoyp March 24, 2025 05:31
@ChenxiJiang333 ChenxiJiang333 force-pushed the feat/automation_storage branch 2 times, most recently from e8d8727 to ff171e3 Compare March 24, 2025 07:25
@ChenxiJiang333 ChenxiJiang333 force-pushed the feat/automation_storage branch from ff171e3 to f5a69f4 Compare March 24, 2025 07:28
Copy link
Member

@jalauzon-msft jalauzon-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Thanks for your work on this.

I will start a run of the test pipeline on this PR and assuming that looks good, this should be good to go.

@jalauzon-msft
Copy link
Member

/azp run python - storage - tests

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants