Merge pull request #1 from Azure/master

Update /

Author: BigCat20196

.github/CODEOWNERS

@@ -128,7 +128,7 @@
 /common/smoketest/                                                   @lmazuel @chlowell @annatisch @rakshith91 @shurd @southpolesteve
 
 # Management Plane
-/**/*mgmt*/     @00Kai0 @msyyc @jsntcy @changlong-liu
+/**/*mgmt*/                                                          @RAY-316 @msyyc 
 
 
 ###########

.gitignore

@@ -92,10 +92,17 @@ venv/
 code_reports
 
 # Azure Storage test credentials
+sdk/storage/azure-storage-blob/tests/_shared/settings_real.py
+sdk/storage/azure-storage-queue/tests/_shared/settings_real.py
+sdk/storage/azure-storage-file-share/tests/_shared/settings_real.py
+sdk/storage/azure-storage-file-datalake/tests/settings_real.py
+
+# The locations below are deprecated - keep to prevent any accidental secrets leakage ==========
 sdk/storage/azure-storage-blob/tests/settings_real.py
 sdk/storage/azure-storage-queue/tests/settings_real.py
 sdk/storage/azure-storage-file-share/tests/settings_real.py
-sdk/storage/azure-storage-file-datalake/tests/settings_real.py
+# ==============================================================================================
+
 *.code-workspace
 sdk/cosmos/azure-cosmos/test/test_config.py
 

.vscode/cspell.json

@@ -6,6 +6,7 @@
     "powershell"
   ],
   "ignorePaths": [
+    "**/tests/recordings/**",
     ".vscode/cspell.json"
   ],
   "words": ["azsdk", "conda", "tenvparallel"],

SUPPORT.md

@@ -0,0 +1,29 @@
+# Support
+
+## How to file issues and get help  
+
+Customers with an [Azure support plan](https://azure.microsoft.com/support/options/) can open an [Azure support ticket](https://azure.microsoft.com/support/create-ticket/).
+**We recommend this option if your problem requires immediate attention.** 
+
+### Github issues
+We use [GitHub Issues](https://github.com/Azure/azure-sdk-for-python/issues/new/choose) to track bugs, questions, and feature requests. 
+GitHub issues are free, but **response time is not guaranteed.** See [GitHub issues support process](https://devblogs.microsoft.com/azure-sdk/github-issue-support-process/) for more details.
+
+### Community resources
+- Search for similar issues in [our GitHub repository](https://github.com/Azure/azure-sdk-for-python/issues)
+- Ask a question on [StackOverflow](https://stackoverflow.com/questions/tagged/azure-sdk-python) and tag it with azure-sdk-python
+- Share or upvote feature requests on [Feedback Page](https://feedback.azure.com/forums/34192--general-feedback).
+- Take a look at the [Azure SDK blog](https://devblogs.microsoft.com/azure-sdk/).
+- Chat with other community members on [gitter](https://gitter.im/Azure/azure-sdk-for-python?source=orgpage)
+- Ask a question on [Twitter](https://twitter.com/AzureSDK)
+- Ask a question at [Microsoft Q&A](https://docs.microsoft.com/answers/products/azure?WT.mc_id=Portal-Microsoft_Azure_Support&product=all)
+- Ask a question at [Microsoft Tech Community](https://techcommunity.microsoft.com/t5/azure/ct-p/Azure)
+
+### Security bugs
+Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center([email protected]). 
+You should receive a response within 24 hours. 
+Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue?rtc=1)
+
+## Microsoft Support Policy
+
+Please refer to [Azure SDK Support and Lifecycle information](https://azure.github.io/azure-sdk/policies_support.html)

doc/dev/conda-builds.md

@@ -4,6 +4,8 @@
 
 Follow the instructions [here](https://docs.conda.io/projects/conda-build/en/latest/install-conda-build.html) to install `conda` and `conda-build`.
 
+**The Azure SDK Conda artifacts support `python3.8` and `python3.9` only.**
+
 ## CI Build Process
 
 There will be a `CondaArtifact` defined in the `ci.yml` of each service directory. (`sdk/<service>`)
@@ -15,9 +17,18 @@ A Conda Artifact defines:
 - Any other necessary details.
 
 ## How to Build an Azure SDK Conda Package Locally
+#### If using powershell, you will need to prep your environment before proceeding to the next step
+
+```
+powershell -ExecutionPolicy ByPass -NoExit -Command "& '<path-to-conda-folder>\shell\condabin\conda-hook.ps1' ; conda activate '<path-to-conda-folder>' "
+```
+
+Afterwards, invoke `conda init powershell` and re-create the pshell session.
 
+By default, your powershell environment will now load `conda`. If you want pure pip, you will need to use explicit invocations of your `python` locations to create virtual envs.
 ### Set up your conda environment
 
+
 You will notice that all the azure-sdk conda distributions have the **same** version number and requirement set. This is due to the fact that the azure-sdk team pushes our conda packages out in waves. To support this, all versions are set via a common environment variable `AZURESDK_CONDA_VERSION`.
 
 We keep this environment variable set properly across all our builds by using a common `conda_env.yml` when creating our build environment. This environment definition ensures that: 

doc/eng_sys_checks.md

@@ -15,13 +15,23 @@ Example PR build:
  - `Analyze` tox envs run during the `Analyze job.
  - `Test <platform>_<pyversion>` runs PR/Nightly tox envs, depending on context.
 
+
+## Targeting a specific package at build time
+
+In both `public` and `internal` projects, all builds allow a filter to be introduced at build time to narrow the set of packages build/tested.
+
+1. Click `Run New` on your target build.
+2. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable. Add variable `BuildTargetingString` with value of a valid glob string.
+   1. For example, setting filter string `azure-mgmt-*` will filter a build to only management packages. A value of `azure-keyvault-secrets` will result in only building THAT specific package.
+3. Once it's set, run the build!
+
 ## Skipping a tox test environment at queue time
 
 All build definitions allow choice at queue time as to which `tox` environments actually run during the test phase.
 
 1. Find your target service `internal` build.
-2. Click `Run New`
-3. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable. The value should be a comma separated list of tox environments that you want to run in the test phase.
+2. Click `Run New`.
+3. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable of name `Run.ToxCustomEnvs`. The value should be a comma separated list of tox environments that you want to run in the test phase.
 4. Once it's set, run the build!
 
 This is an example setting of that narrows the default set from `whl, sdist, depends, latestdependency, minimumdependency`.

eng/.docsettings.yml

@@ -69,6 +69,7 @@ known_content_issues:
   - ['sdk/core/azure-common/README.md', '#4554']
   - ['sdk/core/azure-servicemanagement-legacy/README.md', '#4554']
   - ['sdk/eventgrid/azure-eventgrid/README.md', '#4554']
+  - ['sdk/monitor/azure-monitor-query/README.md', '#4554']
   - ['sdk/graphrbac/azure-graphrbac/README.md', '#4554']
   - ['sdk/loganalytics/azure-loganalytics/README.md', '#4554']
   - ['sdk/servicebus/azure-servicebus/README.md', '#4554']
@@ -84,8 +85,10 @@ known_content_issues:
   - ['sdk/monitor/azure-monitor-opentelemetry-exporter/README.md', '#4554']
   - ['sdk/digitaltwins/azure-digitaltwins-core/swagger/README.md', '#4554']
   - ['sdk/textanalytics/azure-ai-textanalytics/swagger/README.md', '#4554']
+  - ['sdk/translation/azure-ai-translation-document/swagger/README.md', '#4554']
   - ['sdk/purview/azure-purview-catalog/swagger/README.md',  '#4554']
   - ['sdk/purview/azure-purview-scanning/swagger/README.md',  '#4554']
+  - ['sdk/agrifood/azure-agrifood-farming/swagger/README.md',  '#4554']
   - ['sdk/containerregistry/azure-containerregistry/swagger/README.md', '#4554']
   - ['sdk/appconfiguration/azure-appconfiguration/swagger/README.md', '#4554']
   - ['sdk/attestation/azure-security-attestation/swagger/README.md', '#4554']

eng/common/TestResources/AzurePowerShellV4/Utility.ps1

@@ -33,10 +33,12 @@ parameters:
 
 
 steps:
-  - template: /eng/common/TestResources/setup-az-modules.yml
+  - template: /eng/common/pipelines/templates/steps/cache-ps-modules.yml
+
+  - template: /eng/common/TestResources/setup-environments.yml
 
   - pwsh: |
-      eng/common/TestResources/Import-AzModules.ps1
+      eng/common/scripts/Import-AzModules.ps1
 
       $subscriptionConfiguration = @'
         ${{ parameters.SubscriptionConfiguration }}

eng/common/TestResources/Import-AzModules.ps1

@@ -21,7 +21,7 @@ parameters:
 
 steps:
   - pwsh: |
-      eng/common/TestResources/Import-AzModules.ps1
+      eng/common/scripts/Import-AzModules.ps1
 
       $subscriptionConfiguration = @"
         ${{ parameters.SubscriptionConfiguration }}

eng/common/TestResources/deploy-test-resources.yml

@@ -0,0 +1,34 @@
+# Cloud Configuration will be splat into parameters of `Add-AzEnvironment`. It
+# should be JSON in the form (not all fields are required):
+# {
+#   "Name": "<environment name>",
+#   "PublishSettingsFileUrl": "<publish settings file url>",
+#   "ServiceEndpoint": "<service endpoint>",
+#   "ManagementPortalUrl": "<management portal url>",
+#   "ActiveDirectoryEndpoint": "<active directory endpoint>",
+#   "ActiveDirectoryServiceEndpointResourceId": "<active directory service endpoint resource id>",
+#   "ResourceManagerEndpoint": "<resource manager endpoint>",
+#   "GalleryEndpoint": "<gallery endpoint>",
+#   "GraphEndpoint": "<graph endpoint>",
+#   "GraphAudience": "<graph audience>",
+#   "AzureKeyVaultDnsSuffix": "<key vault suffix>",
+#   "AzureKeyVaultServiceEndpointResourceId": "<key vault service endpoint resource id>"
+# }
+
+steps:
+  - bash: sudo chown -R runner ~/.Azure
+    displayName: (MacOS) Grant access to ~/.Azure
+    condition: contains(variables['OSVmImage'], 'mac')
+
+  - task: Powershell@2
+    displayName: Register Dogfood environment
+    inputs:
+      targetType: inline
+      pwsh: true
+      script: |
+        eng/common/scripts/Import-AzModules.ps1
+
+        $environmentSpec = @"
+          $(env-config-dogfood)
+        "@ | ConvertFrom-Json -AsHashtable;
+        Add-AzEnvironment @environmentSpec