ValidationException: Empty identifier is not allowed. The error is caused by «[]».

[getswift-focusedlabs]

Describe your issue:

I'm attempting to migrate data—just work items at the moment but later pipelines and everything else—from a project in one ADO org to one in a different ADO org. I have set up ReflectedWorkItemId as a field on all issue types for both the target and source project.

When running, WorkItemQuery seems to be trying to SELECT []. This seems similar to part of #1278 but unlike in that issue, I have set up ReflectedWorkItemId as a field on all issue types for both the target and source project.

Describe any Exceptions:

[18:27:23 DBG] WorkItemQuery: ===========GetWorkItems=============
[18:27:23 DBG] WorkItemQuery: TeamProjectCollection: https://dev.azure.com/OrgRedacted/
[18:27:23 DBG] WorkItemQuery: Query: SELECT [System.Id], [] FROM WorkItems WHERE [System.TeamProject] = 'Test' AND  [Microsoft.VSTS.Common.ClosedDate] = '' AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan','Shared Steps','Shared Parameter','Feedback Request') ORDER BY [System.ChangedDate] desc
[18:27:23 DBG] WorkItemQuery: Parameters: {"TeamProject": "Test"}
[18:27:23 DBG] WorkItemQuery: TeamProject: Test
[18:27:23 DBG] Query sent
[18:27:23 ERR]  Error running query
Microsoft.TeamFoundation.WorkItemTracking.Client.ValidationException: Empty identifier is not allowed. The error is caused by «[]».
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.Initialize(WorkItemStore store, String wiql, IDictionary context, Int32[] ids, Int32[] revs, Boolean dayPrecision)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.Query(String wiql, IDictionary context)
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItemsFromQuery(TfsWorkItemMigrationClient wiClient) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 51
[18:27:23 INF] DONE in 00:00:28.3031543
[18:27:23 FTL] Error while running WorkItemMigration
Microsoft.TeamFoundation.WorkItemTracking.Client.ValidationException: Empty identifier is not allowed. The error is caused by «[]».
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.Initialize(WorkItemStore store, String wiql, IDictionary context, Int32[] ids, Int32[] revs, Boolean dayPrecision)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.Query(String wiql, IDictionary context)
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItemsFromQuery(TfsWorkItemMigrationClient wiClient) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 84
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetInternalWorkItems() in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 41
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItems() in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 26
   at MigrationTools._EngineV1.Clients.TfsWorkItemMigrationClient.FilterExistingWorkItems(List`1 sourceWorkItems, TfsWiqlDefinition wiqlDefinition, TfsWorkItemMigrationClient sourceWorkItemMigrationClient) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemMigrationClient.cs:line 48
   at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 184
   at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools\_EngineV1\Processors\MigrationProcessorBase.cs:line 47
[18:27:23 ERR] WorkItemMigration The Processor MigrationEngine entered the failed state...stopping run

Source Details

• Source Version: Azure DevOps Services
• Source Language: English

Target Details

• Target Version: Azure DevOps Services
• Target Language: English

I have completed the following:

• Enable Debug in the Logfile - You can set "LogLevel": "Debug" in the configuration file which will output more information to the log.
• Please specify the Source and Target environments above - Which versions are you migrating from/to Azure DevOps Server 2019, Azure DevOps Services, TFS 20xx.
• Attach the full Logfile from your last run - You can obtuse any of the data that you want, but it's really helpful to have the full log to see what lead up to the issue. Do not copy and paste the entire log in here! Attach the file to the issue!
• Attach the full Configuration file - You can obtuse any of the data that you want, but it's really helpful to have the full config. Do not copy and paste the entire log in here! Attach the file to the issue!

Many issues are due to edge cases and will need debugging in Visual Studio; we cant do that for you on your environment.

[MrHinsh]

Please provide the config file used.

[rowanjacobs]

Config here, thanks.

(edited to add: I'm the same person who opened the ticket; I just used my team's account to open it, in error)

[rowanjacobs]

seems like this is getting an empty string for ReflectedWorkItemIDFieldName even though it's populated in the config.

            var targetQuery =
                string.Format(
                    @"SELECT [System.Id], [{0}] FROM WorkItems WHERE [System.TeamProject] = @TeamProject {1} ORDER BY {2}",
                     _config.AsTeamProjectConfig().ReflectedWorkItemIDFieldName,
                    wiqlDefinition.QueryBit,
                    wiqlDefinition.OrderBit);
                    

[rowanjacobs]

Happily, by adjusting the config a bit (nesting "ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId" correctly under the endpoints), I have progressed to a new and different error:

[17:32:24 ERR]  Error running query
Microsoft.TeamFoundation.WorkItemTracking.Client.VerbatimMessageException: VS402335: The timeout period (30 seconds) elapsed prior to completion of the query or the server is not responding. ---> System.Web.Services.Protocols.SoapException: VS402335: The timeout period (30 seconds) elapsed prior to completion of the query or the server is not responding.
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.RetryHandler.HandleSoapException(SoapException se)
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.QueryWorkitems(String requestId, XmlElement psQuery, QuerySortOrderEntry[] sort, Boolean useMaster, Int32[]& ids, DateTime& asOfDate, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.QueryWorkItems(String requestid, XmlElement queryXml, QuerySortOrderEntry[] sort, Int32[]& ids, DateTime& asof)
   --- End of inner exception stack trace ---
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.QueryWorkItems(String requestid, XmlElement queryXml, QuerySortOrderEntry[] sort, Int32[]& ids, DateTime& asof)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.RunRegularQuery(String requestid)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.RunQuery(String requestId)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.Query(String wiql, IDictionary context)
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItemsFromQuery(TfsWorkItemMigrationClient wiClient) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 51
[17:32:24 INF] DONE in 00:00:30.2521505
[17:32:24 FTL] Error while running WorkItemMigration
Microsoft.TeamFoundation.WorkItemTracking.Client.VerbatimMessageException: VS402335: The timeout period (30 seconds) elapsed prior to completion of the query or the server is not responding. ---> System.Web.Services.Protocols.SoapException: VS402335: The timeout period (30 seconds) elapsed prior to completion of the query or the server is not responding.
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.RetryHandler.HandleSoapException(SoapException se)
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.QueryWorkitems(String requestId, XmlElement psQuery, QuerySortOrderEntry[] sort, Boolean useMaster, Int32[]& ids, DateTime& asOfDate, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.QueryWorkItems(String requestid, XmlElement queryXml, QuerySortOrderEntry[] sort, Int32[]& ids, DateTime& asof)
   --- End of inner exception stack trace ---
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.QueryWorkItems(String requestid, XmlElement queryXml, QuerySortOrderEntry[] sort, Int32[]& ids, DateTime& asof)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.RunRegularQuery(String requestid)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Query.RunQuery(String requestId)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.Query(String wiql, IDictionary context)
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItemsFromQuery(TfsWorkItemMigrationClient wiClient) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 84
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetInternalWorkItems() in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 41
   at MigrationTools._EngineV1.Clients.TfsWorkItemQuery.GetWorkItems() in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\_EngineV1\Clients\TfsWorkItemQuery.cs:line 26
   at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 158
   at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools\_EngineV1\Processors\MigrationProcessorBase.cs:line 47
[17:32:24 ERR] WorkItemMigration The Processor MigrationEngine entered the failed state...stopping run

Is this timeout configurable?

[MrHinsh]

Not that I know of, its coming from the SOAP API. You could try using the modern URL of https://dev.azure.com/organisation

[rowanjacobs]

Thanks, no dice using the new URL though. Do they both use a SOAP API or is a REST API available?

Edit: as suggested in #190 I tried running in a VDI instance in East-US, to no avail. Is there a way to split up the query into multiple shorter queries to get around the timeout?

[MrHinsh]

You like "ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId", is incorrectly nested and should be on the Target item.