TfsSharedQueryProcessor #2663

[09:16:02 INF] [16.1.0] Beginning run of 1 processors
[09:16:02 INF] [16.1.0] Processor: TfsSharedQueryProcessor
[09:16:02 INF] [16.1.0] Migration Context Start: TfsSharedQueryProcessor
[09:16:02 DBG] [16.1.0] TfsLanguageMapOptions::Validate::Success
[09:16:02 DBG] [16.1.0] TfsAuthenticationOptions::Validate::Success
[09:16:02 DBG] [16.1.0] TfsEndpointOptionsValidator::Validate::Success
[09:16:02 FTL] [16.1.0] Error while running TfsSharedQueryProcessor
System.InvalidCastException: Unable to cast object of type 'MigrationTools.Clients.TfsTeamProjectEndpoint' to type 'MigrationTools.Endpoints.TfsEndpoint'.
   at MigrationTools.Processors.TfsSharedQueryProcessor.get_Source() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsSharedQueryProcessor.cs:line 35
   at MigrationTools.Processors.TfsSharedQueryProcessor.InternalExecute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsSharedQueryProcessor.cs:line 46
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 99
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[09:16:04 INF] [16.1.0] TfsSharedQueryProcessor completed in 00:00:00
[09:16:04 ERR] [16.1.0] TfsSharedQueryProcessor The Processor MigrationEngine entered the failed state...stopping run
[09:16:04 INF] [16.1.0] Command ExecuteMigrationCommand completed in 00:00:02.3883806
[09:16:04 INF] [16.1.0] Check the logs for errors: C:\Tools\MigrationTools\logs\20250401091601

I try to migrate all Shared Queries.

Config:

{
  "Serilog": {
    "MinimumLevel": "Debug"
  },
  "MigrationTools": {
    "Version": "16.1",
    "Endpoints": {
      "Source2": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/shsds",
        "Project": "CS-Cinematic-Anatomy",
        "AllowCrossProjectLinking": false,
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "XXX",
          "NetworkCredentials": {
            "UserName": "",
            "Password": "",
            "Domain": ""
          }
        },
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      },
      "Target2": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/SHS-CS-DDO",
        "Project": "CS-Cinematic-Anatomy",
        "TfsVersion": "AzureDevOps",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "XXX",
          "NetworkCredentials": {
            "UserName": "",
            "Password": "",
            "Domain": ""
          }
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "AllowCrossProjectLinking": false,
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      }
    },
    "CommonTools": {
      "WorkItemTypeMappingTool": {
        "Enabled": true,
        "Mappings": {
	"ABB": "ABB",
	"Assumption": "Assumption",
    "Bug": "Bug",
	"Claim": "Claim",
	"Disclaimer": "Disclaimer",
	"Epic": "Epic",
	"Feature": "Feature",
	"Headline": "Headline",
	"Issue": "Issue",
	"Market Requirement": "Market Requirement",
	"OTS": "OTS",
	"Product Backlog Item": "Product Backlog Item",
	"Risk": "Risk",
	"Risk Mitigation": "Risk Mitigation",
	"Risk Safety": "Risk Safety",
	"Task": "Task",
	"Software Requirement": "Software Requirement",
	"SOP": "SOP",
	"Test Case": "Test Case",
	"User Story": "Work Package Item"
        }
      },
      "StringManipulatorTool": {
        "Enabled": true,
        "MaxStringLength": 1000000,
		//Fix SingleString "MaxStringLength": 255,
        "Manipulators": [
          {
            "$type": "RegexStringManipulator",
            "Enabled": true,
            "Pattern": "[^( -~)\n\r\t]+",
            "Replacement": "",
            "Description": "Remove invalid characters from the end of the string"
          }
        ]
      },
      "TfsAttachmentTool": {
        "RefName": "TfsAttachmentTool",
        "Enabled": true,
        "ExportBasePath": "c:\\temp\\WorkItemAttachmentExport",
        "MaxRevisions": 480000000
      },
      "TfsChangeSetMappingTool": {
        "Enabled": false,
        "File": "C:\\temp\\ChangeSetMappingFile.json"
      },
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": [
          {
            "FieldMapType": "FieldtoFieldMap",
            "ApplyTo": ["Defect"],
            "sourceField": "Siemens.Defect.Justification",
            "targetField": "Microsoft.VSTS.CMMI.Justification"
          },
			{
            "FieldMapType": "FieldValueMap",
            "ApplyTo": ["*"],
            "defaultValue": "New",
            "sourceField": "System.State",
            "targetField": "System.State",
            "valueMapping": 
			{
        "New": "New",
		"In Work": "In Work",
		"In Play": "In Play",
		"Active": "Active",
		"Closed": "Closed",
		"Resolved": "Resolved",
        "Removed": "Removed",
		"For Analysis": "For Analysis",
		"Qualified": "Qualified",
		"Done": "Done",
		"Blocked": "Blocked",
		"In Planning": "In Planning",
		"Design": "Design",
		"Inactive": "Inactive",
		"Next Up": "Next Up",
		"Ready": "Ready",
		"To Do": "To Do",
		"Completed": "Completed",
		"Proposed": "Proposed",
		"In Progress": "In Progress"
      }
		  }
        ]
      },
      "TfsGitRepositoryTool": {
        "Enabled": true,
        "Mappings": null
		/*{
          "CS-Cinematic-Anatomy": "CS-Cinematic-Anatomy"
        }*/
      },
      "TfsNodeStructureTool": {
        "Enabled": true,
        "Areas": {
		  "Filters": null,
          //"Filters": [ " *\\Team 1,*\\Team 2\\**" ],
          "Mappings": null
		  /*{
            "CS-Cinematic-Anatomy([\\\\]?.*)$": "CS-Cinematic-Anatomy",
			"^1c78a84c-cf99-4d5b-8ba9-e0cb7d6d1f8e([\\\\]?.*)$": "Ultrasound"
          }*/
        },
        "Iterations": {
          "Filters": null,
          "Mappings": null
		  /*{
            "CS-Cinematic-Anatomy([\\\\]?.*)$": "CS-Cinematic-Anatomy",
			"^1c78a84c-cf99-4d5b-8ba9-e0cb7d6d1f8e([\\\\]?.*)$": "Ultrasound"
          } */
        }, 
        "ShouldCreateMissingRevisionPaths": true,
        "ReplicateAllExistingNodes": false
      },
      "TfsRevisionManagerTool": {
        "Enabled": true,
        "ReplayRevisions": true,
        "MaxRevisions": 0
		// Req Fix        "MaxRevisions": 7
		//EPIC/Feature Fix "MaxRevisions": 34
      },
      "TfsTeamSettingsTool": {
        "Enabled": true,
        "MigrateTeamSettings": true,
        "UpdateTeamSettings": true,
        "MigrateTeamCapacities": true,
        "Teams": null
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": false,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject  AND [System.ID] >= '1' AND [System.WorkItemType] NOT IN ('Test Plan','Test Suite') ORDER BY [System.ID] asc",
        // AND [System.ID] IN ('389','415','416','1032','4519','9574','9575','10520','10521','10813','61701','67214','67326','67325','67324','40618','40870','40906','40907','40921','40922','40871','40908','40913','40923','40924','40872','40909','40911','40914','40915','40925','40926','40928','40929','40877','40930','40911','40931','40878','40919','40933','40934','40879','40920')
		// AND [System.ID] NOT IN ('389','61700')
		// AND [System.ChangedDate] >= '2023-12-17T00:00:00.00Z'
		// AND [System.ID] NOT IN ('563419','674128')
	  // AND [Siemens.Defect.Product] = 'VSIM'
	  // AND [System.ID] >= '440407' 
	  // AND [System.AreaPath] UNDER 'XRay\\Hepton'		
	  // AND [System.Tags] Contains 'migrated'
	  // AND [System.IterationPath] UNDER 'XRay\\Hepton' OR [System.IterationPath] IN ('XRay\\Hepton\\AT2306','XRay\\Hepton\\AT2309','XRay\\Hepton\\AT2312','XRay\\Hepton\\AT2403','XRay\\Hepton\\AT2406')
	  // AND [System.WorkItemType] IN ('Epic','Task','User Story','Feature')
	  // AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan','Shared Steps','Shared Parameter','Feedback Request')
	  // AND [System.State] NOT IN ('Closed') 
	  // AND [System.ID]  IN ('63','64','65','68')
		"FixHtmlAttachmentLinks": false,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": true,
        "PauseAfterEachWorkItem": false,
        "AttachRevisionHistory": false,
        "GenerateMigrationComment": false,
        "SourceName": "Source2",
        "TargetName": "Target2",
        //"WorkItemIDs": [ 12 ],
		"WorkItemIDs": null,
        "MaxGracefulFailures": 0,
        "SkipRevisionWithInvalidIterationPath": false,
        "SkipRevisionWithInvalidAreaPath": false
      },
	  {
      "ProcessorType": "TfsTestPlansAndSuitesMigrationProcessorOptions",
	  "Enabled": false,
      "ReplayRevisions": true,
      "PrefixProjectToNodes": false,
	  //This Part is for Test Plans and Suites (run 2nd)!!!
      //"TestPlanQueryBit": "Select '*' From TestPlan Where [System.ID] >= '58348' AND [System.AreaPath] UNDER 'SENSE\\Magellan Team'",
	  "TestPlanQueryBit": "AreaPath UNDER 'CS-Cinematic-Anatomy' AND [System.ID] >= '303000' AND [System.ID] NOT IN ('290801','303131')",
	  // AND [System.ID] NOT IN ('77828')
	  //'Test Case','Test Plan','Test Suite' AND [System.State] NOT IN ('Closed') AND [System.WorkItemType] IN ('Test Plan','Test Suite') AND [System.AreaPath] UNDER 'SENSE\\Magellan Team',
	  "WIQLOrderBit": "[System.ID] asc",
	  "OnlyElementsWithTag": "Migrate",
	  "SourceName": "Source2",
      "TargetName": "Target2",
	  "RemoveInvalidTestSuiteLinks": false,
	  "UseCommonNodeStructureEnricherConfig": false,
	  "NodeBasePaths": ["123456"],
	  "AreaMaps": {},
      "IterationMaps": {}
    },
	 {
        "ProcessorType": "TfsSharedQueryProcessor",
        "Enabled": true,
        "PrefixProjectToNodes": false,
        "SharedFolderName": "Shared Queries",
        "SourceToTargetFieldMappings": null,
        "SourceName": "Source2",
        "TargetName": "Target2"
     },
	{
		"ProcessorType": "TfsWorkItemDeleteProcessorOptions",
		"Enabled": false,
		"WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.WorkItemType] IN ('Defect','Test Case','Test Plan','Test Suite') AND [System.AreaPath] UNDER 'syngo Application Software\\Migration_DefectsXRay'  ORDER BY [System.ChangedDate] desc",
		// DANGER!, but good for deleting Test Cases!
		// AND [System.WorkItemType] IN ('Test Case') AND [System.AreaPath] = 'BranchDa\\BranchDA'
		"WorkItemIDs": null,
		"SourceName": "Source2",
        "TargetName": "Target2",
		"FilterWorkItemsThatAlreadyExistInTarget": false,
		"PauseAfterEachWorkItem": false,
		"WorkItemCreateRetryLimit": 0
	}
    ]
  }
}

Thanks for your support.

5 replies

MicE44 Apr 2, 2025 — with giscus

If you want share your working v16.1 config with me/us.
Maybe I can compare it, thanks.

MrHinsh Apr 2, 2025
Maintainer

Your config is a mess of unessesary mappings and

Here is mine:

{
  "Serilog": {
    "MinimumLevel": "Debug"
  },
  "MigrationTools": {
    "Version": "16.0",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/nkdagility-preview/",
        "Project": "migrationSource1",
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "Authentication": {
            "AuthenticationMode": "Prompt"
        }
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/nkdagility-preview/",
        "Project": "migrationTest55",
        "TfsVersion": "AzureDevOps",
        "Authentication": {
          "AuthenticationMode": "Prompt"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId"
      }
    },
    "Processors": [
        {
            "ProcessorType": "TfsSharedQueryProcessor",
            "Enabled": true,
            "SourceName": "Source",
            "TargetName": "Target"
        }
    ]
  }
}

vifor2 Apr 2, 2025 — with giscus

Hello @MicE44, I had a similar issue, try changing your EndpointType to TfsEndpoint for both the source and the target like so:

    "Endpoints": {
      "Source2": {
        "EndpointType": "TfsEndpoint",
        [...]
      },
      "Target2": {
        "EndpointType": "TfsEndpoint",
        [...]
      },

MrHinsh Apr 2, 2025
Maintainer

Its interesting that I did not need to do that! But hey! Whatever works right!

potatojaisiladki Apr 4, 2025 — with giscus

@vifor2 - Thank you. Your comment saved my time :)

MicE44 · 2025-04-03T11:02:53Z

MicE44
Apr 3, 2025 — with giscus

Thanks, Ok the EndpointType is the problem.

I need "EndpointType": "TfsEndpoint", for "ProcessorType": "TfsSharedQueryProcessor",

and

"EndpointType": "TfsTeamProjectEndpoint", for "ProcessorType": "TfsWorkItemMigrationProcessor",

[12:58:27 FTL] [16.1.0] Validation of your configuration failed:
The configuration entry at 'MigrationTools:Processors' did not pass validation!
Please check the following failures:
-The Endpoint 'Target2' specified for TfsWorkItemMigrationProcessor is of the wrong type! TfsTeamProjectEndpoint was expected.

0 replies

satano · 2025-05-02T10:45:52Z

satano
May 2, 2025 — with giscus

@MrHinsh Guys here are right. It really throws that exception and I do not understand how could yout configuration work. The problem is this:

azure-devops-migration-tools/src/MigrationTools.Clients.TfsObjectModel/Processors/TfsSharedQueryProcessor.cs

Line 35 in 9c3eb05

public new TfsEndpoint Source => (TfsEndpoint)base.Source;

The Source and Target properties are casted to TfsEndpoint, which is not possible. I wanted to fix it, but it is not simple, because TfsSharedQueryProcessor is using things from TfsEndpoint which are not in TfsTeamProjectEndpoint.

3 replies

MrHinsh May 2, 2025
Maintainer

Ive not run Shared Queries in a while but TfsTeamProjectEndpoint inherts from TfsEndpoint

satano May 2, 2025 — with giscus

No, that is not true. The inheritance is this:

TfsTeamProjectEndpoint → Endpoint<TfsTeamProjectEndpointOptions>
TfsEndpoint → GenericTfsEndpoint<TfsEndpointOptions> → Endpoint<TfsEndpointOptions>

So they have common ancestor Endpoint. But the can not be casted from one to another. And the TfsSharedQueryProcessor uses methods and properties from GenericTfsEndpoint so even cast to base Endpoint will not help.

satano May 2, 2025 — with giscus

Not to mention, that even the base is different, because they have different options. So now to use TfsSharedQueryProcessor, we must use TfsEndpoint. If someone uses work item migration and also shared queries, she must define two sources and two targets. One set as TfsEndpoint for shared queries and the other set as TfsTeamProjectEndpoint for work items.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TfsSharedQueryProcessor #2663

{{title}}

Replies: 5 comments 11 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

TfsSharedQueryProcessor #2663

giscus[bot] bot Mar 26, 2025

TfsSharedQueryProcessor

Replies: 5 comments · 11 replies

MicE44 Mar 26, 2025 — with giscus

MrHinsh Mar 31, 2025 Maintainer

MicE44 Mar 31, 2025 — with giscus

MrHinsh Mar 31, 2025 Maintainer

MrHinsh Mar 31, 2025 Maintainer

MicE44 Apr 1, 2025 — with giscus

MicE44 Apr 2, 2025 — with giscus

MrHinsh Apr 2, 2025 Maintainer

vifor2 Apr 2, 2025 — with giscus

MrHinsh Apr 2, 2025 Maintainer

potatojaisiladki Apr 4, 2025 — with giscus

MicE44 Apr 3, 2025 — with giscus

satano May 2, 2025 — with giscus

MrHinsh May 2, 2025 Maintainer

satano May 2, 2025 — with giscus

satano May 2, 2025 — with giscus

giscus[bot]
bot Mar 26, 2025

Replies: 5 comments 11 replies

MicE44
Mar 26, 2025 — with giscus

MrHinsh Mar 31, 2025
Maintainer

MicE44
Mar 31, 2025 — with giscus

MrHinsh Mar 31, 2025
Maintainer

MrHinsh Mar 31, 2025
Maintainer

MicE44
Apr 1, 2025 — with giscus

MrHinsh Apr 2, 2025
Maintainer

MrHinsh Apr 2, 2025
Maintainer

MicE44
Apr 3, 2025 — with giscus

satano
May 2, 2025 — with giscus

MrHinsh May 2, 2025
Maintainer