Merge pull request #62 from atc-net/feature/delete-partition

rickykaare · web-flow · commit e9bde2d616af · 2024-10-21T14:57:56.000+02:00
Add Preview feature for deleting resources by partition key
diff --git a/README.md b/README.md
@@ -38,14 +38,6 @@ Once the library is added to your project, you will have access to the following
 * [`ICosmosBulkReader<T>`](src/Atc.Cosmos/ICosmosBulkReader.cs)
 * [`ICosmosBulkWriter<T>`](src/Atc.Cosmos/ICosmosBulkWriter.cs)
 
-When using the preview version, you will have access to the following interfaces, used for reading and writing Cosmos document resources:
-* [`ILowPriorityCosmosReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosReader.cs)
-* [`ILowPriorityCosmosWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosWriter.cs)
-* [`ILowPriorityCosmosBulkReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkReader.cs)
-* [`ILowPriorityCosmosBulkWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkWriter.cs)
-
-The interfaces that are prefixed with `ILowPriority` require priority-based execution to be enabled on the CosmosDB account. Priority-based execution is currently is not enabled by default and to get started using it you need to fill out this [nomination form](https://forms.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR_kUn4g8ufhFjXbbwUF1gXFUMUQzUzFZSVkzODRSRkxXM0RKVDNUSDBGNi4u). After submitting, a member of the CosmosDb team will reach out and enable the feature on the accounts you listed and contact you to let you know it’s ready for use.
-
 A document resource is represented by a class deriving from the [`CosmosResource`](src/Atc.Cosmos/CosmosResource.cs) base-class, or by implementing the underlying [`ICosmosResource`](src/Atc.Cosmos/ICosmosResource.cs) interface directly.
 
 To configure where each resource will be stored in Cosmos, the `ConfigureCosmos(builder)` extension method is used on the `IServiceCollection` when setting up dependency injection (usually in a `Startup.cs` file).
@@ -185,15 +177,6 @@ The registered interfaces are:
 |[`ICosmosBulkReader<T>`](src/Atc.Cosmos/ICosmosBulkReader.cs)| Represents a reader that can perform bulk reads on Cosmos resources. |
 |[`ICosmosBulkWriter<T>`](src/Atc.Cosmos/ICosmosBulkWriter.cs)| Represents a writer that can perform bulk operations on Cosmos resources. |
 
-For the preview version, the registered interfaces also include:
-
-|Name|Description|
-|-|-|
-|[`ILowPriorityCosmosReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosReader.cs)| Represents a reader that can read Cosmos resources with low priority. |
-|[`ILowPriorityCosmosWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosWriter.cs)| Represents a writer that can write Cosmos resources with low priority. |
-|[`ILowPriorityCosmosBulkReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkReader.cs)| Represents a reader that can perform bulk reads on Cosmos resources with low priority. |
-|[`ILowPriorityCosmosBulkWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkWriter.cs)| Represents a writer that can perform bulk operations on Cosmos resources with low priority. |
-
 The bulk reader and writer are for optimizing performance when executing many operations towards Cosmos. It works by creating all the tasks and then use the `Task.WhenAll()` to await them. This will group operations by partition key and send them in batches of 100.
 
 When not operating with bulks, the normal readers are faster as there is no delay waiting for more work.
@@ -235,7 +218,72 @@ To do this you will need to:
 
 *Note: The change feed processor relies on a HostedService, which means that this feature is only available in AspNet Core services.*
 
-### Unit Testing
+## Preview Features
+
+The library also has a preview version that exposes some of CosmosDB preview features.
+
+### Priority Based Execution
+
+When using the preview version, you will have access to the following interfaces, used for reading and writing Cosmos document resources:
+
+|Name|Description|
+|-|-|
+|[`ILowPriorityCosmosReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosReader.cs)| Represents a reader that can read Cosmos resources with low priority. |
+|[`ILowPriorityCosmosWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosWriter.cs)| Represents a writer that can write Cosmos resources with low priority. |
+|[`ILowPriorityCosmosBulkReader<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkReader.cs)| Represents a reader that can perform bulk reads on Cosmos resources with low priority. |
+|[`ILowPriorityCosmosBulkWriter<T>`](src/Atc.Cosmos/ILowPriorityCosmosBulkWriter.cs)| Represents a writer that can perform bulk operations on Cosmos resources with low priority. |
+
+In order to use these interfaces the "Priority Based Execution" feature needs to be enabled on the CosmosDB account.
+
+This can be done by either enabling it directly in Azure Portal under Settings -> Features tab on the CosmosDB resource.
+
+Alternatively through Azure CLI:
+
+```bash
+# install cosmosdb-preview Azure CLI extension
+az extension add --name cosmosdb-preview
+
+# Enable priority-based execution
+az cosmosdb update  --resource-group $ResourceGroup --name $AccountName --enable-priority-based-execution true
+```
+
+See [MS Learn](https://learn.microsoft.com/en-us/azure/cosmos-db/priority-based-execution) for more details.
+
+### Delete resources by partition key
+
+The preview version of the library extends the `ICosmosWriter` and `ILowPriorityCosmosWriter` with and additional method `DeletePartitionAsync` to delete all resources in a container based on a partition key. The deletion will be executed in a CosmosDB background service using a percentage of the RU's available. The effect are available immediatly as all resources in the partition will not be available through reads or queries.
+
+In order to use this new method the "Delete All Items By Partition Key" feature needs to be enabled on the CosmosDB account. 
+
+This can be done through Azure CLI:
+
+```bash
+# Delete All Items By Partition Key
+az cosmosdb update  --resource-group $ResourceGroup --name $AccountName --capabilities DeleteAllItemsByPartitionKey
+```
+
+or wih bicep:
+
+```bicep
+resource cosmosAccount 'Microsoft.DocumentDB/databaseAccounts@2024-05-15' = {
+  name: cosmosName
+  properties: {
+    databaseAccountOfferType: 'Standard'
+    locations: location
+    capabilities: [
+      {
+        name: 'DeleteAllItemsByPartitionKey'
+      }
+    ]
+  }
+}
+```
+
+If the feature is not enabled when calling this method then a `CosmosException` will be thrown.
+
+See [MS Learn](https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/how-to-delete-by-partition-key) for more details.
+
+## Unit Testing
 The reader and writer interfaces can easily be mocked, but in some cases it is nice to have a fake version of a reader or writer to mimic the behavior of the read and write operations. For this purpose the `Atc.Cosmos.Testing` namespace contains the following fakes:
 
 |Name|Description|
diff --git a/src/Atc.Cosmos/Atc.Cosmos.csproj b/src/Atc.Cosmos/Atc.Cosmos.csproj
@@ -13,8 +13,8 @@
     <PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" Version="8.0.0" />
     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.2" />
     <PackageReference Include="System.Text.Json" Version="8.0.5" />
-    <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.43.0-preview.0" Condition="$(IsPreview)" />
-    <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.43.1" Condition="!$(IsPreview)" />
+    <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.45.0-preview.0" Condition="$(IsPreview)" />
+    <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.44.1" Condition="!$(IsPreview)" />
   </ItemGroup>
 
   <ItemGroup Condition="$(TargetFramework) == 'netstandard2.0'">
diff --git a/src/Atc.Cosmos/ICosmosWriter.cs b/src/Atc.Cosmos/ICosmosWriter.cs
@@ -149,6 +149,26 @@ public Task<bool> TryDeleteAsync(
             string documentId,
             string partitionKey,
             CancellationToken cancellationToken = default);
+#if PREVIEW
+
+        /// <summary>
+        /// Preview Feature DeleteAllItemsByPartitionKey.<br/>
+        /// Deletes all resources in the Container with the specified <see cref="PartitionKey"/>.
+        /// Starts an asynchronous Cosmos DB background operation which deletes all resources in the Container with the specified value.
+        /// The asynchronous Cosmos DB background operation runs using a percentage of user RUs.
+        /// </summary>
+        /// <remarks>
+        /// A <see cref="CosmosException"/>
+        /// with StatusCode <see cref="HttpStatusCode.BadRequest"/>
+        /// will be thrown if the DeleteAllItemsByPartitionKey feature is not enabled.
+        /// </remarks>
+        /// <param name="partitionKey">Partition key of the resource.</param>
+        /// <param name="cancellationToken">The <see cref="CancellationToken"/> used.</param>
+        /// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
+        public Task DeletePartitionAsync(
+            string partitionKey,
+            CancellationToken cancellationToken = default);
+#endif
 
         /// <summary>
         /// Updates a <typeparamref name="T"/> resource that is read from the configured
diff --git a/src/Atc.Cosmos/Internal/CosmosWriter.cs b/src/Atc.Cosmos/Internal/CosmosWriter.cs
@@ -175,6 +175,21 @@ await container
 
             return true;
         }
+#if PREVIEW
+
+        public Task DeletePartitionAsync(
+            string partitionKey,
+            CancellationToken cancellationToken = default)
+            => container
+                .DeleteAllItemsByPartitionKeyStreamAsync(
+                    new PartitionKey(partitionKey),
+                    new ItemRequestOptions
+                    {
+                        PriorityLevel = PriorityLevel,
+                    },
+                    cancellationToken: cancellationToken)
+                .ProcessResponseMessage();
+#endif
 
         public Task<T> UpdateAsync(
             string documentId,
diff --git a/src/Atc.Cosmos/Internal/ResponseMessageExtensions.cs b/src/Atc.Cosmos/Internal/ResponseMessageExtensions.cs
@@ -0,0 +1,14 @@
+using System.Threading.Tasks;
+using Microsoft.Azure.Cosmos;
+
+namespace Atc.Cosmos.Internal
+{
+    public static class ResponseMessageExtensions
+    {
+        public static async Task ProcessResponseMessage(this Task<ResponseMessage> responseMessage)
+        {
+            using ResponseMessage message = await responseMessage.ConfigureAwait(false);
+            message.EnsureSuccessStatusCode();
+        }
+    }
+}
diff --git a/src/Atc.Cosmos/Testing/FakeCosmos.cs b/src/Atc.Cosmos/Testing/FakeCosmos.cs
@@ -481,6 +481,16 @@ Task<bool> ICosmosWriter<T>.TryDeleteAsync(
                     documentId,
                     partitionKey,
                     cancellationToken);
+#if PREVIEW
+
+        Task ICosmosWriter<T>.DeletePartitionAsync(
+            string partitionKey,
+            CancellationToken cancellationToken)
+            => ((ICosmosWriter<T>)Writer)
+                .DeletePartitionAsync(
+                    partitionKey,
+                    cancellationToken);
+#endif
 
         Task<T> ICosmosWriter<T>.UpdateAsync(
             string documentId,
diff --git a/src/Atc.Cosmos/Testing/FakeCosmosWriter.cs b/src/Atc.Cosmos/Testing/FakeCosmosWriter.cs
@@ -153,6 +153,18 @@ await DeleteAsync(
 
             return true;
         }
+#if PREVIEW
+
+        public virtual Task DeletePartitionAsync(
+            string partitionKey,
+            CancellationToken cancellationToken = default)
+        {
+            Documents.RemoveAll(d
+                => d.PartitionKey == partitionKey);
+
+            return Task.CompletedTask;
+        }
+#endif
 
         public virtual Task<T> UpdateAsync(
             string documentId,
diff --git a/test/Atc.Cosmos.Tests/CosmosWriterTests.cs b/test/Atc.Cosmos.Tests/CosmosWriterTests.cs
@@ -49,6 +49,14 @@ public CosmosWriterTests()
             container
                 .PatchItemAsync<object>(default, default, default, default)
                 .ReturnsForAnyArgs(response);
+#if PREVIEW
+
+            var responseMessage = Substitute.For<ResponseMessage>();
+            responseMessage.StatusCode.Returns(HttpStatusCode.Accepted);
+            container
+                .DeleteAllItemsByPartitionKeyStreamAsync(default, default, default)
+                .ReturnsForAnyArgs(responseMessage);
+#endif
 
             reader = Substitute.For<ICosmosReader<Record>>();
             reader
@@ -288,6 +296,34 @@ public async Task Should_Return_False_When_Trying_To_Delete_NonExisting_Resource
 #endif
                     cancellationToken: cancellationToken);
         }
+#if PREVIEW
+
+        [Theory, AutoNSubstituteData]
+        public async Task DeletePartitionAsync_Calls_DeleteAllItemsByPartitionKeyStreamAsync_On_Container(
+            CancellationToken cancellationToken)
+        {
+            await sut.DeletePartitionAsync(record.Pk, cancellationToken);
+            _ = container
+                .Received(1)
+                .DeleteAllItemsByPartitionKeyStreamAsync(
+                    new PartitionKey(record.Pk),
+                    Arg.Is<ItemRequestOptions>(o => o.PriorityLevel == PriorityLevel.High),
+                    cancellationToken: cancellationToken);
+        }
+
+        [Theory, AutoNSubstituteData]
+        public Task DeletePartitionAsync_Throws_CosmosException_If_ResponseMessage_Is_Not_Sucessful(
+            CancellationToken cancellationToken)
+        {
+            using var responseMessage = new ResponseMessage(HttpStatusCode.BadRequest);
+            container
+                .DeleteAllItemsByPartitionKeyStreamAsync(default, default, default)
+                .ReturnsForAnyArgs(responseMessage);
+
+            Func<Task> act = () => sut.DeletePartitionAsync(record.Pk, cancellationToken);
+            return act.Should().ThrowAsync<CosmosException>();
+        }
+#endif
 
         [Theory, AutoNSubstituteData]
         public async Task UpdateAsync_Reads_The_Resource(
diff --git a/test/Atc.Cosmos.Tests/LowPriorityCosmosWriterTests.cs b/test/Atc.Cosmos.Tests/LowPriorityCosmosWriterTests.cs
@@ -51,6 +51,12 @@ public LowPriorityCosmosWriterTests()
                 .PatchItemAsync<object>(default, default, default, default)
                 .ReturnsForAnyArgs(response);
 
+            var responseMessage = Substitute.For<ResponseMessage>();
+            responseMessage.StatusCode.Returns(HttpStatusCode.Accepted);
+            container
+                .DeleteAllItemsByPartitionKeyStreamAsync(default, default, default)
+                .ReturnsForAnyArgs(responseMessage);
+
             reader = Substitute.For<ILowPriorityCosmosReader<Record>>();
             reader
                 .ReadAsync(default, default, default)
@@ -257,6 +263,32 @@ public async Task Should_Return_False_When_Trying_To_Delete_NonExisting_Resource
                     cancellationToken: cancellationToken);
         }
 
+        [Theory, AutoNSubstituteData]
+        public async Task DeletePartitionAsync_Calls_DeleteAllItemsByPartitionKeyStreamAsync_On_Container(
+            CancellationToken cancellationToken)
+        {
+            await sut.DeletePartitionAsync(record.Pk, cancellationToken);
+            _ = container
+                .Received(1)
+                .DeleteAllItemsByPartitionKeyStreamAsync(
+                    new PartitionKey(record.Pk),
+                    Arg.Is<ItemRequestOptions>(o => o.PriorityLevel == PriorityLevel.Low),
+                    cancellationToken: cancellationToken);
+        }
+
+        [Theory, AutoNSubstituteData]
+        public Task DeletePartitionAsync_Throws_CosmosException_If_ResponseMessage_Is_Not_Sucessful(
+            CancellationToken cancellationToken)
+        {
+            using var responseMessage = new ResponseMessage(HttpStatusCode.BadRequest);
+            container
+                .DeleteAllItemsByPartitionKeyStreamAsync(default, default, default)
+                .ReturnsForAnyArgs(responseMessage);
+
+            Func<Task> act = () => sut.DeletePartitionAsync(record.Pk, cancellationToken);
+            return act.Should().ThrowAsync<CosmosException>();
+        }
+
         [Theory, AutoNSubstituteData]
         public async Task UpdateAsync_Reads_The_Resource(
             string documentId,
diff --git a/test/Atc.Cosmos.Tests/Testing/FakeCosmosWriterTests.cs b/test/Atc.Cosmos.Tests/Testing/FakeCosmosWriterTests.cs
@@ -211,6 +211,45 @@ public async Task DeleteAsync_Should_Replace_Existing_Document(
                 .Should()
                 .NotContain(existingDocument);
         }
+#if PREVIEW
+
+        [Theory, AutoNSubstituteData]
+        public async Task DeletePartitionAsyncAsync_Should_Delete_Existing_Documents(
+            FakeCosmosWriter<Record> sut,
+            Record record1,
+            Record record2,
+            Record record3)
+        {
+            var existingDocument1 = new Record
+            {
+                Id = record1.Id,
+                Pk = record1.Pk,
+            };
+            sut.Documents.Add(existingDocument1);
+            var existingDocument2 = new Record
+            {
+                Id = record2.Id,
+                Pk = record1.Pk,
+            };
+            sut.Documents.Add(existingDocument2);
+            var existingDocument3 = new Record
+            {
+                Id = record3.Id,
+                Pk = record3.Pk,
+            };
+            sut.Documents.Add(existingDocument3);
+
+            await sut.DeletePartitionAsync(record1.Pk);
+
+            sut.Documents
+                .Should()
+                .NotContain(existingDocument1)
+                .And
+                .NotContain(existingDocument2)
+                .And
+                .Contain(existingDocument3);
+        }
+#endif
 
         [Theory, AutoNSubstituteData]
         public void UpdateAsync_Should_Throw_If_Document_Does_Not_Exists(
