Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage] Migrate Storage to ESM/vitest #33329

Open
wants to merge 124 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 94 commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
81a0359
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 3, 2025
0394fc8
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 6, 2025
8e8559e
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 6, 2025
2355803
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 7, 2025
b2e0c97
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 10, 2025
369c35f
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 10, 2025
a385e38
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 10, 2025
0fb747c
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 10, 2025
5da4fec
[storage] Migrate storage projects to use snippets extraction
mpodwysocki Mar 10, 2025
58f3aef
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
18e7aa4
Migration: Update test config
mpodwysocki Mar 10, 2025
a23948f
Migration: Clean up files
mpodwysocki Mar 10, 2025
3f2e98c
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
a94e4b8
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
e79001a
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
427b2de
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
50e68ff
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
ea6db58
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
f7dbf6c
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
943f677
Migration: npm run format
mpodwysocki Mar 10, 2025
a0c2017
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
66d7b82
Migration: Update test config
mpodwysocki Mar 10, 2025
ce34eeb
Migration: Clean up files
mpodwysocki Mar 10, 2025
c04e29e
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
34979a0
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
0a59f04
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
b7e9fa9
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
00b809b
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
91093af
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
b1f2df5
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
63b3502
Migration: npm run format
mpodwysocki Mar 10, 2025
36a2c73
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
167f703
Migration: Update test config
mpodwysocki Mar 10, 2025
ee3a036
Migration: Clean up files
mpodwysocki Mar 10, 2025
14294ca
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
b81f179
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
8cb5302
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
faa47b5
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
05fa242
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
489445b
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
0b43ad6
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
831170f
Migration: npm run format
mpodwysocki Mar 10, 2025
94be438
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
3d6ccbe
Migration: Update test config
mpodwysocki Mar 10, 2025
629586c
Migration: Clean up files
mpodwysocki Mar 10, 2025
6e1e85e
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
665997d
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
60549fe
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
88afb38
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
d2ce8cf
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
60d8094
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
de76bfa
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
c1a7e99
Migration: npm run format
mpodwysocki Mar 10, 2025
c29130a
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
9a1ade0
Migration: Update test config
mpodwysocki Mar 10, 2025
88ae770
Migration: Clean up files
mpodwysocki Mar 10, 2025
524950e
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
aa6dd9a
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
31595e2
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
f5fa691
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
65957d3
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
3b11f12
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
841e4c9
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
5d40ce4
Migration: npm run format
mpodwysocki Mar 10, 2025
aab2289
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
c330406
Migration: Update test config
mpodwysocki Mar 10, 2025
33f673e
Migration: Clean up files
mpodwysocki Mar 10, 2025
11a848a
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
b4fb5be
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
856632d
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
ad1f2b0
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
8c681bd
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
dc87dfd
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
f3ef61c
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
0dc3e68
Migration: npm run format
mpodwysocki Mar 10, 2025
e96004d
Migration: Update package.json, tsconfig.json, and api-extractor.json
mpodwysocki Mar 10, 2025
963a165
Migration: Update test config
mpodwysocki Mar 10, 2025
02dbb39
Migration: Clean up files
mpodwysocki Mar 10, 2025
ceb3e96
Migration: Apply codemod: "fixSourceFile"
mpodwysocki Mar 10, 2025
a605498
Migration: Apply codemod: "fixTestingImports"
mpodwysocki Mar 10, 2025
33f7e83
Migration: Apply codemod: "replaceAssertIsRejected"
mpodwysocki Mar 10, 2025
efebab4
Migration: Apply codemod: "replaceSinonStub"
mpodwysocki Mar 10, 2025
b64078d
Migration: Apply codemod: "addViHelper"
mpodwysocki Mar 10, 2025
44291cd
Migration: Apply codemod: "replaceSupportTracing"
mpodwysocki Mar 10, 2025
555cf70
Migration: Apply codemod: "replaceTestUtils"
mpodwysocki Mar 10, 2025
9e1ce09
Migration: npm run format
mpodwysocki Mar 10, 2025
f065bcf
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 10, 2025
803267f
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 10, 2025
3567885
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 10, 2025
ebca217
Update sdk/storage/storage-common/api-extractor.json
mpodwysocki Mar 11, 2025
605dad2
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 11, 2025
66be26f
Merge branch 'feat/storage-snippets-esm' of https://github.com/Azure/…
mpodwysocki Mar 11, 2025
dc6aeab
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 11, 2025
d9c25a2
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 11, 2025
e94fb16
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 11, 2025
70ad0da
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
3444700
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
665e637
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
8514a2a
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
4850b45
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
136d04d
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
3d9b281
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
ab3c894
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
df71f61
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 12, 2025
77c82c2
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…
mpodwysocki Mar 13, 2025
1efadc6
[] Migrate projects to ESM/vitest
mpodwysocki Mar 13, 2025
f88c3c9
[] Migrate projects to ESM/vitest
mpodwysocki Mar 14, 2025
7d9f03b
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 14, 2025
6488c26
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 14, 2025
1f933ce
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 14, 2025
a2124e4
start cleaning this up
xirzec Mar 14, 2025
17f25c2
more fixes
xirzec Mar 15, 2025
e2843fa
fix stream issue
xirzec Mar 15, 2025
ae0d857
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 15, 2025
77fe7c1
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 15, 2025
770ab05
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 16, 2025
bf5180d
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 16, 2025
ba22379
[storage] Migrate storage projects to ESM/vitest
mpodwysocki Mar 17, 2025
92aa3be
[storage] Move @azure/storage-blob to ESM/vitest
mpodwysocki Mar 17, 2025
6583e9b
fix flakyness of "should abort after aborter timeout" test
HarshaNalluru Mar 18, 2025
759a6e4
Merge branch 'main' of https://github.com/azure/azure-sdk-for-js into…
HarshaNalluru Mar 18, 2025
1c2cc23
lock file
HarshaNalluru Mar 18, 2025
8209632
[storage] Move @azure/storage-blob to ESM/vitest
mpodwysocki Mar 19, 2025
29c760c
[storage] Move @azure/storage-blob to ESM/vitest
mpodwysocki Mar 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1,464 changes: 271 additions & 1,193 deletions common/config/rush/pnpm-lock.yaml

Large diffs are not rendered by default.

17 changes: 9 additions & 8 deletions rush.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
* This is the main configuration file for Rush.
* For full documentation, please see https://rushjs.io
*/{
*/ {
"$schema": "https://developer.microsoft.com/json-schemas/rush/v5/rush.schema.json",
/**
* (Required) This specifies the version of the Rush engine to be used in this repo.
Expand Down Expand Up @@ -560,9 +560,7 @@
"projectFolder": "common/tools/dev-tool",
"versionPolicyName": "utility",
// Add Identity to decoupledLocalDependencies so that dev-tool uses the package from npm, avoiding a cyclic dependency.
"decoupledLocalDependencies": [
"@azure/identity"
]
"decoupledLocalDependencies": ["@azure/identity"]
},
{
"packageName": "@azure/eventgrid",
Expand Down Expand Up @@ -602,9 +600,7 @@
{
"packageName": "@azure/identity",
"projectFolder": "sdk/identity/identity",
"decoupledLocalDependencies": [
"@azure/keyvault-keys"
],
"decoupledLocalDependencies": ["@azure/keyvault-keys"],
"versionPolicyName": "client"
},
{
Expand Down Expand Up @@ -717,6 +713,11 @@
"projectFolder": "sdk/storage/storage-internal-avro",
"versionPolicyName": "utility"
},
{
"packageName": "@azure/storage-common",
"projectFolder": "sdk/storage/storage-common",
"versionPolicyName": "utility"
},
{
"packageName": "@azure/storage-blob",
"projectFolder": "sdk/storage/storage-blob",
Expand Down Expand Up @@ -2333,4 +2334,4 @@
"versionPolicyName": "management"
}
]
}
}
119 changes: 82 additions & 37 deletions sdk/storage/storage-blob-changefeed/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,9 @@ This library offers a client you can use to fetch the change events.

The `BlobChangeFeedClient` requires almost the same parameters as `BlobServiceClient` to initialize. Refer to [storage-blob](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/storage/storage-blob#create-the-blob-service-client) for how to create the blob service client. Here is an example using `StorageSharedKeyCredential`.

```javascript
const { StorageSharedKeyCredential } = require("@azure/storage-blob");
const { BlobChangeFeedClient } = require("@azure/storage-blob-changefeed");
```ts snippet:ReadmeSampleCreateClient
import { StorageSharedKeyCredential } from "@azure/storage-blob";
import { BlobChangeFeedClient } from "@azure/storage-blob-changefeed";

// Enter your storage account name and shared key
const account = "<account>";
Expand All @@ -88,45 +88,71 @@ const changeFeedClient = new BlobChangeFeedClient(

Use `BlobChangeFeedClient.listChanges()` to get iterators to iterate through the change events.

```javascript
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
```ts snippet:ReadmeSampleListChanges
import { StorageSharedKeyCredential } from "@azure/storage-blob";
import { BlobChangeFeedClient } from "@azure/storage-blob-changefeed";

let changeFeedEvents = [];
const account = "<account>";
const accountKey = "<accountkey>";
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const changeFeedClient = new BlobChangeFeedClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential,
);

// Use for await to iterate through the change feed
for await (const event of changeFeedClient.listChanges()) {
changeFeedEvents.push(event);
console.log(`Event: ${event.eventType}`);
console.log(`Event time: ${event.eventTime}`);
console.log(`Event data: ${JSON.stringify(event.data)}`);
}
```

By page.

```javascript
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");

let changeFeedEvents = [];
for await (const eventPage of changeFeedClient.listChanges().byPage()) {
for (const event of eventPage.events) {
changeFeedEvents.push(event);
// Use `byPage` to iterate through the change feed
for await (const page of changeFeedClient.listChanges().byPage()) {
console.log(`Page: ${JSON.stringify(page)}`);
for (const event of page.events) {
console.log(`Event: ${event.eventType}`);
console.log(`Event time: ${event.eventTime}`);
console.log(`Event data: ${JSON.stringify(event.data)}`);
}
}
```

### Resuming reading events with a continuationToken

```javascript
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
```ts snippet:ReadmeSampleListChanges_Continuation
import { StorageSharedKeyCredential } from "@azure/storage-blob";
import { BlobChangeFeedClient } from "@azure/storage-blob-changefeed";

let changeFeedEvents = [];
const firstPage = await changeFeedClient.listChanges().byPage({ maxPageSize: 10 }).next();
for (const event of firstPage.value.events) {
changeFeedEvents.push(event);
}
const account = "<account>";
const accountKey = "<accountkey>";
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const changeFeedClient = new BlobChangeFeedClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential,
);

// Resume iterating from the previous position with the continuationToken.
for await (const eventPage of changeFeedClient
.listChanges()
.byPage({ continuationToken: firstPage.value.continuationToken })) {
for (const event of eventPage.events) {
changeFeedEvents.push(event);
let iterator = changeFeedClient.listChanges().byPage({ maxPageSize: 2 });
let response = (await iterator.next()).value;
// Prints 2 page ranges
if (response.pageRange) {
for (const pageRange of response.pageRange) {
console.log(`Event: ${pageRange.eventType}`);
console.log(`Event time: ${pageRange.eventTime}`);
console.log(`Event data: ${JSON.stringify(pageRange.data)}`);
}
}
// Gets next marker
let marker = response.continuationToken;
// Passing next marker as continuationToken
iterator = changeFeedClient.listChanges().byPage({ continuationToken: marker, maxPageSize: 10 });
response = (await iterator.next()).value;
// Prints 10 page ranges
if (response.pageRange) {
for (const pageRange of response.pageRange) {
console.log(`Event: ${pageRange.eventType}`);
console.log(`Event time: ${pageRange.eventTime}`);
console.log(`Event data: ${JSON.stringify(pageRange.data)}`);
}
}
```
Expand All @@ -137,24 +163,43 @@ Pass start time and end time to `BlobChangeFeedClient.listChanges()` to fetch ev

Note that for now, the change feed client will round start time down to the nearest hour, and round end time up to the next hour.

```javascript
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
```ts snippet:ReadmeSampleListChangesTimeRange
import { StorageSharedKeyCredential } from "@azure/storage-blob";
import { BlobChangeFeedClient } from "@azure/storage-blob-changefeed";

const account = "<account>";
const accountKey = "<accountkey>";
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const changeFeedClient = new BlobChangeFeedClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential,
);

const start = new Date(Date.UTC(2020, 1, 21, 22, 30, 0)); // will be rounded down to 22:00
const end = new Date(Date.UTC(2020, 4, 8, 21, 10, 0)); // will be rounded up to 22:00

let changeFeedEvents = [];
// You can also provide just a start or end time.
// Use for await to iterate through the change feed
for await (const event of changeFeedClient.listChanges({ start, end })) {
changeFeedEvents.push(event);
console.log(`Event: ${event.eventType}`);
console.log(`Event time: ${event.eventTime}`);
console.log(`Event data: ${JSON.stringify(event.data)}`);
}

// Use `byPage` to iterate through the change feed
for await (const page of changeFeedClient.listChanges({ start, end }).byPage()) {
console.log(`Page: ${JSON.stringify(page)}`);
for (const event of page.events) {
console.log(`Event: ${event.eventType}`);
console.log(`Event time: ${event.eventTime}`);
console.log(`Event data: ${JSON.stringify(event.data)}`);
}
}
```

## Troubleshooting

Enabling logging may help uncover useful information about failures. In order to see a log of HTTP requests and responses, set the `AZURE_LOG_LEVEL` environment variable to `info`. Alternatively, logging can be enabled at runtime by calling `setLogLevel` in the `@azure/logger`:

```javascript
```ts snippet:SetLogLevel
import { setLogLevel } from "@azure/logger";

setLogLevel("info");
Expand Down
4 changes: 2 additions & 2 deletions sdk/storage/storage-blob-changefeed/api-extractor.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",
"mainEntryPointFilePath": "types/latest/storage-blob-changefeed/src/index.d.ts",
"mainEntryPointFilePath": "dist/esm/index.d.ts",
"docModel": {
"enabled": true
},
Expand All @@ -11,7 +11,7 @@
"dtsRollup": {
"enabled": true,
"untrimmedFilePath": "",
"publicTrimmedFilePath": "./types/latest/storage-blob-changefeed.d.ts"
"publicTrimmedFilePath": "dist/storage-blob-changefeed.d.ts"
},
"messages": {
"tsdocMessageReporting": {
Expand Down
Loading
Loading