[Bug]: Field names not url-encoded during Process migration

[nrundle]

Version

• I confirm that I am using the latest version

Source Version

Azure DevOps Service

Target Version

Azure DevOps Server 2022

Relevant configuration

{
        "ProcessorType": "ProcessDefinitionProcessor",
        "Enabled": true,
        "Processes": {
            "ACA Agile": [
                "BE Task",
                "Bug",
                "Design Task",
                "DevOps Task",
                "TQ Operational Requirement"
            ]
        },
        "ProcessMaps": {
            "ACA Agile": "ACA Agile"
        },
        "UpdateProcessDetails": true,
        "MaxDegreeOfParallelism": 0,
        "SourceName": "cloud",
        "TargetName": "hq-tfs"
      }

Relevant log output

{"Timestamp":"2025-01-31T14:42:37.7264915-06:00","Level":"Error","MessageTemplate":"Failed on call to get single [{definitionName}] with Id [{definitionId}].\r\nUrl: GET {requestUri}\r\nResponse Code:{statusCode}","TraceId":"20758a68f735da6ccfc03845193527de","SpanId":"fa16e03550b7a0b6","Properties":{"definitionName":"WorkItemTypeField","definitionId":"Custom.TR_TOR#","requestUri":"https://dev.azure.com/acagroundtools/_apis/work/processes/61882cb2-06a8-442e-80f7-87e1e4a4a667/workitemtypes/ACAAgile.TQOperationalRequirement/fields/Custom.TR_TOR#?$expand=All","statusCode":"OK","SourceContext":"MigrationTools.Endpoints.AzureDevOpsEndpoint","versionString":"16.0.9","ProcessId":6764}}
{"Timestamp":"2025-01-31T14:42:37.7342348-06:00","Level":"Error","MessageTemplate":"Failed to synchronize processes.","TraceId":"20758a68f735da6ccfc03845193527de","SpanId":"fa16e03550b7a0b6","Exception":"System.Exception: {\"$id\":\"1\",\"innerException\":null,\"message\":\"VS402645: The field Custom.TR_TOR does not exists in work item type ACAAgile.TQOperationalRequirement\",\"typeName\":\"Microsoft.TeamFoundation.WorkItemTracking.Server.Metadata.ProcessWorkItemTypeFieldDoesNotExistException, Microsoft.TeamFoundation.WorkItemTracking.Server\",\"typeKey\":\"ProcessWorkItemTypeFieldDoesNotExistException\",\"errorCode\":0,\"eventId\":3200}\r\n   at MigrationTools.Endpoints.AzureDevOpsEndpoint.<GetApiDefinitionsAsync>d__8`1.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Endpoints\\AzureDevOpsEndpoint.cs:line 191\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<LoadWorkItemFields>d__26.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 579\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<>c__DisplayClass25_4.<<BuildModel>b__7>d.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 548\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<BuildModel>d__25.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 546\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<Synchronize>d__21.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 124","Properties":{"SourceContext":"MigrationTools.Processors.Infrastructure.Processor","versionString":"16.0.9","ProcessId":6764}}
{"Timestamp":"2025-01-31T14:42:37.7459145-06:00","Level":"Fatal","MessageTemplate":"Error while running {MigrationContextname}","TraceId":"20758a68f735da6ccfc03845193527de","SpanId":"fa16e03550b7a0b6","Exception":"System.Exception: {\"$id\":\"1\",\"innerException\":null,\"message\":\"VS402645: The field Custom.TR_TOR does not exists in work item type ACAAgile.TQOperationalRequirement\",\"typeName\":\"Microsoft.TeamFoundation.WorkItemTracking.Server.Metadata.ProcessWorkItemTypeFieldDoesNotExistException, Microsoft.TeamFoundation.WorkItemTracking.Server\",\"typeKey\":\"ProcessWorkItemTypeFieldDoesNotExistException\",\"errorCode\":0,\"eventId\":3200}\r\n   at MigrationTools.Endpoints.AzureDevOpsEndpoint.<GetApiDefinitionsAsync>d__8`1.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Endpoints\\AzureDevOpsEndpoint.cs:line 191\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<LoadWorkItemFields>d__26.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 579\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<>c__DisplayClass25_4.<<BuildModel>b__7>d.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 548\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<BuildModel>d__25.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 546\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.<Synchronize>d__21.MoveNext() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 154\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n   at MigrationTools.Processors.ProcessDefinitionProcessor.InternalExecute() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools.Clients.AzureDevops.Rest\\Processors\\ProcessDefinitionProcessor.cs:line 91\r\n   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\\a\\azure-devops-migration-tools\\azure-devops-migration-tools\\src\\MigrationTools\\Processors\\Infrastructure\\Processor.cs:line 99","Properties":{"MigrationContextname":"ProcessDefinitionProcessor","SourceContext":"MigrationTools.Processors.Infrastructure.Processor","versionString":"16.0.9","ProcessId":6764}}
{"Timestamp":"2025-01-31T14:42:38.3535559-06:00","Level":"Error","MessageTemplate":"{Context} The Processor {ProcessorName} entered the failed state...stopping run","TraceId":"20758a68f735da6ccfc03845193527de","SpanId":"cdac92b67b77b666","Properties":{"Context":"ProcessDefinitionProcessor","ProcessorName":"MigrationEngine","SourceContext":"MigrationTools.MigrationEngine","versionString":"16.0.9","ProcessId":6764}}

What happened?

I'm trying to migrate a cloud process to an on-prem server and ran into an issue where a field name requires URL-encoding but was not done and thus the tool failed to request the correct field name.

The actual field name is Custom.TR_TOR#
The tool attempted to request Custom.TR_TOR as the # wasn't url-encoded

Debug in Visual Studio

• Visual Studio Debug

[MrHinsh]

Was definitely not aware # was a valid character for a field name.

[gheintz]

Ran into this issue as well. Indeed calling the ADO API using %23 instead of # in the field name will return the proper result. I am not a C Sharp developer, but this appears to be happening at line 181 in AzureDevOpsEndpoint.cs. Would it be possible to use a string replace on definition.Id to encode the # (I am curious if there would be other chars that would need this treatment). Maybe definition.Id.Replace("#", "%23")?