Skip to content

Commit 1deba72

Browse files
adthomadthom
authored
Release v1.3.0: Add comprehensive test suite and migrate to isolated worker model (#83)
* test: add test projects for Flattener and Functions - Create CallRecordInsights.Flattener.Tests project - Create CallRecordInsights.Functions.Tests project - Add project references to solution - Configure for Central Package Management * test: add test dependencies to Central Package Management - Add Microsoft.NET.Test.Sdk 17.9.0 - Add xunit 2.6.5 - Add xunit.runner.visualstudio 2.5.6 - Add FluentAssertions 6.12.0 - Add Moq 4.20.70 * fix: replace C# 12 collection expressions with compatible syntax C# 12 collection expression syntax (e.g., []) is not available in .NET 6/C# 10. Replace with traditional new Dictionary<>() and new HashSet<>() syntax for compatibility. This change is required to establish baseline tests on .NET 6 before migrating to .NET 10. * test: add JsonFlattener algorithm tests with test data Add comprehensive tests for the JSON flattening algorithm: - Array expansion with varying cardinalities - Parent value inheritance - Cartesian product validation (multiple sessions/streams) - Null/empty input handling - Missing array handling Includes minimal-callrecord.json test data file with representative Microsoft Teams call record structure (1 session, 1 segment, 1 stream). All 10 tests passing on .NET 6. * test: add comprehensive JSONPath parser tests Add 39 tests covering: - Selector parsing (property, array wildcard, array index, nested arrays) - Token selection (SelectToken/SelectTokens) - Path relationships (IsParentOf, IsSiblingOf, IsRelativeOf) - Path utilities (GetClosestExpansion, GetParentPath, LevelsOfExpansion) - Common ancestor detection - Expandability checks Documents unsupported features (array slices, quoted properties) that have implementation issues in the current parser. All 39 tests passing on .NET 6. * test: add KustoCallRecord type conversion tests Add 13 tests covering all data types in the 266-property model: - Guid parsing (CallId, SessionId, TenantIds) - DateTimeOffset parsing (timestamps) - TimeSpan parsing from ISO 8601 duration format (PT0.02S) - Long/Double numeric types - Boolean values - String values - Null handling for all property types - Configuration validation (266 columns, key property mappings) All 13 tests passing on .NET 6. Total test suite: 62 tests, all passing. * refactor: migrate to Azure Functions isolated worker model packages Update Functions project and package versions: Removed in-process packages: - Microsoft.NET.Sdk.Functions - Microsoft.Azure.Functions.Extensions - Microsoft.Azure.WebJobs.Extensions.* Added isolated worker packages: - Microsoft.Azure.Functions.Worker 1.21.0 - Microsoft.Azure.Functions.Worker.Sdk 1.17.0 - Microsoft.Azure.Functions.Worker.Extensions.EventHubs 6.3.5 - Microsoft.Azure.Functions.Worker.Extensions.Storage.Queues 5.4.0 - Microsoft.Azure.Functions.Worker.Extensions.Http 3.1.0 - Microsoft.Azure.Functions.Worker.Extensions.Timer 4.3.0 - Newtonsoft.Json 13.0.3 (required by Cosmos SDK) Set OutputType to Exe for isolated worker model. Part of .NET 6 to .NET 10 LTS migration plan - Phase 1. * refactor: replace Startup.cs with Program.cs and update DI Replace in-process FunctionsStartup with isolated worker HostBuilder: - Delete Startup.cs (in-process model entry point) - Create Program.cs with HostBuilder and ConfigureFunctionsWorkerDefaults() - Preserve configuration loading (appsettings.json, environment variables) - Preserve all service registrations Add QueueServiceClient registration: - New AddCallRecordsQueueContext() extension method - Required for isolated worker model (no direct queue output bindings) - Registers QueueServiceClient for DI All existing tests still pass (62/62). * refactor: migrate trigger functions to isolated worker model Update EventHub, Queue, and Timer functions for isolated worker: ProcessEventHubEventFunction: - [FunctionName] → [Function] - Inject QueueServiceClient and IConfiguration - Get QueueClient from service client at runtime - Remove queue output binding parameter ProcessCallRecordFunction: - [FunctionName] → [Function] - QueueTrigger remains compatible - Update using statements AddOrRenewCallRecordsSubscriptionFunction: - [FunctionName] → [Function] - TimerTrigger remains compatible - Update using statements All tests still pass (62/62). * refactor: migrate HTTP functions to isolated worker model Update all HTTP functions for isolated worker model: GetCallRecordInsightsHealthFunction: - HttpRequest → HttpRequestData, IActionResult → HttpResponseData - Inject QueueServiceClient and get QueueClient at runtime - Replace GetWebJobsConnectionSection (in-process only) with direct config access - Use request.CreateResponse() and WriteAsJsonAsync() GetCallRecordAdminFunction: - HttpRequest → HttpRequestData, IActionResult → HttpResponseData - Update all response creation to use HttpResponseData - Replace ASP.NET Core status results with HttpStatusCode CallRecordsFunctionsAdmin (3 endpoints): - Inject QueueServiceClient and IConfiguration - Get QueueClient at runtime for manual processing endpoint - Update GetSubscriptionIdFunction response handling - Update AddSubscriptionOrRenewIfExpiredFunction response handling - Update ManuallyProcessCallIdsFunction with queue injection - Convert helper methods to async for HttpResponseData Build succeeds with zero errors/warnings. All tests still pass (62/62). Phase 1 complete: Isolated worker migration on .NET 6. * test: add global using directives for test projects Add Usings.cs files with global using statements for xUnit. These are generated by the dotnet new xunit template and eliminate the need for using Xunit; in each test file. * test: remove template placeholder test file Remove UnitTest1.cs template file - replaced with actual test files: - JsonFlattenerTests.cs - JsonPathExtensionsTests.cs - KustoCallRecordConversionTests.cs * Add DI container integration tests for Functions project - Create DependencyInjectionTests with 12 tests to verify service resolution - Test all critical services: JsonProcessor, GraphServiceClient, TokenCredential, QueueServiceClient, CallRecordsGraphContext, CallRecordsDataContext - Verify configuration loading and Graph client authentication chain - Tests catch NuGet package breaking changes in DI registration - All tests pass (12/12) with minimal test configuration * chore: update app version to 1.3.0 * fix: update Bicep for isolated worker and fix scope resolution - Change FUNCTIONS_WORKER_RUNTIME from 'dotnet' to 'dotnet-isolated' - Fix BCP420 error by splitting conditional scope into two resources - graphChangeTrackingAppKeyVaultRoleAssignmentSeparate (useSeparateKeyVaultForGraph=true) - graphChangeTrackingAppKeyVaultRoleAssignmentShared (useSeparateKeyVaultForGraph=false) * build: update ARM template from Bicep changes Rebuild deploy/resourcemanager/template.json to reflect: - FUNCTIONS_WORKER_RUNTIME: dotnet-isolated - Fixed scope resolution with conditional resources
1 parent 1103c67 commit 1deba72

25 files changed

Lines changed: 1925 additions & 225 deletions

‎CallRecordInsights.sln‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
1212
src\.editorconfig = src\.editorconfig
1313
EndProjectSection
1414
EndProject
15+
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "tests", "tests", "{53A9AC31-32FD-4CD7-BA0A-985DAA3D562E}"
16+
EndProject
17+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CallRecordInsights.Flattener.Tests", "tests\CallRecordInsights.Flattener.Tests\CallRecordInsights.Flattener.Tests.csproj", "{0424019F-D30F-4E62-BA6E-0B30D6B61643}"
18+
EndProject
19+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CallRecordInsights.Functions.Tests", "tests\CallRecordInsights.Functions.Tests\CallRecordInsights.Functions.Tests.csproj", "{05C642EC-0F90-4A8E-874F-155853FAF026}"
20+
EndProject
1521
Global
1622
GlobalSection(SolutionConfigurationPlatforms) = preSolution
1723
Debug|Any CPU = Debug|Any CPU
@@ -26,11 +32,23 @@ Global
2632
{EA094B49-D23B-457B-A7EC-B4CDD4409F8E}.Debug|Any CPU.Build.0 = Debug|Any CPU
2733
{EA094B49-D23B-457B-A7EC-B4CDD4409F8E}.Release|Any CPU.ActiveCfg = Release|Any CPU
2834
{EA094B49-D23B-457B-A7EC-B4CDD4409F8E}.Release|Any CPU.Build.0 = Release|Any CPU
35+
{0424019F-D30F-4E62-BA6E-0B30D6B61643}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
36+
{0424019F-D30F-4E62-BA6E-0B30D6B61643}.Debug|Any CPU.Build.0 = Debug|Any CPU
37+
{0424019F-D30F-4E62-BA6E-0B30D6B61643}.Release|Any CPU.ActiveCfg = Release|Any CPU
38+
{0424019F-D30F-4E62-BA6E-0B30D6B61643}.Release|Any CPU.Build.0 = Release|Any CPU
39+
{05C642EC-0F90-4A8E-874F-155853FAF026}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
40+
{05C642EC-0F90-4A8E-874F-155853FAF026}.Debug|Any CPU.Build.0 = Debug|Any CPU
41+
{05C642EC-0F90-4A8E-874F-155853FAF026}.Release|Any CPU.ActiveCfg = Release|Any CPU
42+
{05C642EC-0F90-4A8E-874F-155853FAF026}.Release|Any CPU.Build.0 = Release|Any CPU
2943
EndGlobalSection
3044
GlobalSection(SolutionProperties) = preSolution
3145
HideSolutionNode = FALSE
3246
EndGlobalSection
3347
GlobalSection(ExtensibilityGlobals) = postSolution
3448
SolutionGuid = {EFF0E08A-B789-4B7C-AF75-CF9D0FCFC01A}
3549
EndGlobalSection
50+
GlobalSection(NestedProjects) = preSolution
51+
{0424019F-D30F-4E62-BA6E-0B30D6B61643} = {53A9AC31-32FD-4CD7-BA0A-985DAA3D562E}
52+
{05C642EC-0F90-4A8E-874F-155853FAF026} = {53A9AC31-32FD-4CD7-BA0A-985DAA3D562E}
53+
EndGlobalSection
3654
EndGlobal

‎Directory.Packages.props‎

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,28 @@
33
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
44
</PropertyGroup>
55
<ItemGroup>
6+
<!-- Azure Functions Isolated Worker Packages -->
7+
<PackageVersion Include="Microsoft.Azure.Functions.Worker" Version="1.21.0" />
8+
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.17.0" />
9+
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.EventHubs" Version="6.3.5" />
10+
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.Storage.Queues" Version="5.4.0" />
11+
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.1.0" />
12+
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.3.0" />
13+
14+
<!-- Azure Service Packages -->
615
<PackageVersion Include="Microsoft.Azure.Cosmos" Version="3.46.0" />
7-
<PackageVersion Include="Microsoft.Azure.WebJobs.Extensions.EventHubs" Version="6.3.5" />
8-
<PackageVersion Include="Microsoft.Azure.WebJobs.Extensions.Storage.Blobs" Version="5.3.3" />
9-
<PackageVersion Include="Microsoft.Azure.WebJobs.Extensions.Storage.Queues" Version="5.3.3" />
16+
<PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
17+
18+
<!-- Microsoft Graph & Identity Packages -->
1019
<PackageVersion Include="Microsoft.Graph" Version="5.65.0" />
1120
<PackageVersion Include="Microsoft.Identity.Web" Version="3.5.0" />
1221
<PackageVersion Include="Microsoft.Identity.Web.GraphServiceClient" Version="3.5.0" />
13-
<PackageVersion Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
14-
<PackageVersion Include="Microsoft.NET.Sdk.Functions" Version="4.6.0" />
15-
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.6.0" />
22+
23+
<!-- Test Packages -->
24+
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
25+
<PackageVersion Include="xunit" Version="2.6.5" />
26+
<PackageVersion Include="xunit.runner.visualstudio" Version="2.5.6" />
27+
<PackageVersion Include="FluentAssertions" Version="6.12.0" />
28+
<PackageVersion Include="Moq" Version="4.20.70" />
1629
</ItemGroup>
1730
</Project>

‎deploy/bicep/deployFunction.bicep‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -386,7 +386,7 @@ resource functionApp 'Microsoft.Web/sites@2022-09-01' = {
386386
{ key: 'AzureWebJobsSecretStorageType', value: 'keyvault' }
387387
{ key: 'AzureWebJobsSecretStorageKeyVaultUri', value: keyvault.properties.vaultUri }
388388
{ key: 'FUNCTIONS_EXTENSION_VERSION', value: '~4' }
389-
{ key: 'FUNCTIONS_WORKER_RUNTIME', value: 'dotnet' }
389+
{ key: 'FUNCTIONS_WORKER_RUNTIME', value: 'dotnet-isolated' }
390390
{ key: 'WEBSITE_CONTENTAZUREFILECONNECTIONSTRING', value: '@Microsoft.KeyVault(VaultName=${keyvault.name};SecretName=${keyvault::storageAccountConnectionString.name})' }
391391
{ key: 'WEBSITE_CONTENTSHARE', value: toLower(functionApp.name) }
392392
{ key: 'SCM_COMMAND_IDLE_TIMEOUT', value: '1800' }
@@ -562,9 +562,19 @@ resource graphChangeTrackingAppRoleAssignment 'Microsoft.Authorization/roleAssig
562562
}
563563
}]
564564

565-
resource graphChangeTrackingAppKeyVaultRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in neededRoles.graphChangeTrackingApp.KeyVault: {
566-
scope: useSeparateKeyVaultForGraph ? graphKeyVault::graphEventHubConnectionString : keyvault::graphEventHubConnectionString
567-
name: guid(graphChangeTrackingAppObjectId, role, resourceGroup().id)
565+
resource graphChangeTrackingAppKeyVaultRoleAssignmentSeparate 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in neededRoles.graphChangeTrackingApp.KeyVault: if (useSeparateKeyVaultForGraph) {
566+
scope: graphKeyVault::graphEventHubConnectionString
567+
name: guid(graphChangeTrackingAppObjectId, role, 'separate', resourceGroup().id)
568+
properties: {
569+
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role)
570+
principalId: graphChangeTrackingAppObjectId
571+
principalType: 'ServicePrincipal'
572+
}
573+
}]
574+
575+
resource graphChangeTrackingAppKeyVaultRoleAssignmentShared 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in neededRoles.graphChangeTrackingApp.KeyVault: if (!useSeparateKeyVaultForGraph) {
576+
scope: keyvault::graphEventHubConnectionString
577+
name: guid(graphChangeTrackingAppObjectId, role, 'shared', resourceGroup().id)
568578
properties: {
569579
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role)
570580
principalId: graphChangeTrackingAppObjectId

0 commit comments

Comments
 (0)