TFS WorkItem Migration Processor

[giscus[bot]]

TfsWorkItemMigrationProcessor

The WorkItemMigrationContext processor is used for migrating work items from one Azure DevOps instance to another. This encompasses a variety of activities:

1. Transferring Work Items Between Instances: The primary purpose of the processor is to transfer work items, including bugs, tasks, user stories, features, and more, from one Azure DevOps instance to another.
2. Migrating Work Item History: The processor can also replicate the entire revision history of work items, providing continuity and maintaining a record of changes.
3. Migrating Attachments and Links: The processor can transfer any attachments or links associated with work items. This includes both external links and internal links to other work items.
4. Updating Metadata: If configured, the processor can update the "Created Date" and "Created By" fields on migrated work items to match the original items in the source instance.
5. Filtering Work Items: The processor can be configured to only migrate certain work items based on their area or iteration paths.

Overall, the WorkItemMigrationContext processor is a comprehensive tool for transferring work items and their associated data and metadata between Azure DevOps instances. It should be used whenever there is a need to move work items between instances while preserving as much information as possible.

https://nkdagility.com/learn/azure-devops-migration-tools/Reference/Processors/TfsWorkItemMigrationProcessor/

[salmodxc]

Is there a why to write recursive queries. For example, I need to migrate specific Epics and all children Work Items (direct and indirect)

[MrHinsh]

Not directly. You could tag all the items you want and move all items with the tag.

[hansdoerr]

This is a beautiful tool. Is there a way to do 20000 work items at a time? Like do the query by date or maybe by the year? We've got 64000 work items and it's failing because it says it can only do 20000 at a time.

[hansdoerr]

Never mind, we got that: we ended up adding "AND [System.CreateDate] >= '1/1/2020' AND [System.CreatedDate] <= '12/31/2021'" to go year by year and then just running several instances at the same time. We did two years at a time for most of the years, but we found one two-year period where they had more than 20K records, so it kicked it back. Anyway, we got it.

[MrHinsh]

Thats what I do.

[Lucarian77]

I'm new at the whole migration/setup and tool(great one) but i keep coming across this error using the setup below...,... Any thoughts?

Error....

[22:10:22 INF] [16.1.0] Creating Migration Engine b69ef8f8-1f07-4d8e-b86a-84f2d333c941
[22:10:22 INF] [16.1.0] ProcessorContainer: Of 1 configured Processors only 1 are enabled
[22:10:22 INF] [16.1.0] ProcessorContainer: Adding Processor TfsWorkItemMigrationProcessor
[22:10:22 INF] [16.1.0] Logging has been configured and is set to: unknown.
[22:10:22 INF] [16.1.0]                               Max Logfile: Verbose.
[22:10:22 INF] [16.1.0]                               Max Console: Debug.
[22:10:22 INF] [16.1.0]                  Max Application Insights: Error.
[22:10:22 INF] [16.1.0] The Max log levels above show where to go look for extra info. e.g. Even if you set the log level to Verbose you will only see that info in the Log File, however everything up to Debug will be in the Console.
[22:10:22 INF] [16.1.0] Beginning run of 1 processors
[22:10:22 INF] [16.1.0] Processor: TfsWorkItemMigrationProcessor
[22:10:22 INF] [16.1.0] Migration Context Start: TfsWorkItemMigrationProcessor
[22:10:22 FTL] [16.1.0] Validation of your configuration failed:
The configuration entry at 'MigrationTools:Processors' did not pass validation!
 Please check the following failures:
 -The Endpoint 'Target' specified for `TfsWorkItemMigrationProcessor` was not found.
[22:10:22 INF] [16.1.0] TfsWorkItemMigrationProcessor completed in 00:00:00
[22:10:22 ERR] [16.1.0] TfsWorkItemMigrationProcessor The Processor MigrationEngine entered the failed state...stopping run

---

Setup..... (to migrate only workItems/attachments/screenshots..etc)

{
  "MigrationTools": {
    "Version": "16.1",
    "Endpoints": {
      "Source": {
        "$type": "MigrationTools.Endpoints.Tfs.TfsEndpointOptions, MigrationTools",
        "Name": "Source",
        "Organization": "https://dev.azure.com/xxxxxxxxxxxx(outside Vendor Tenant)/",
        "Project": "xxxxxxxxxxxxxxxxxxxx",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId"
      },
      "Target": {
        "$type": "MigrationTools.Endpoints.Tfs.TfsEndpointOptions, MigrationTools",
        "Name": "Target",
        "Organization": "https://dev.azure.com/xxxxxxxxxxxxxx(OurInternalProject)/",
        "Project": "DevOpsMigration%20Test%20Project",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId"
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
        "SourceName": "Source",
        "TargetName": "Target",
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject",
        "WorkItemCreateRetryLimit": 5,
        "SkipRevisionMigration": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "UpdateSourceReflectedId": false,
        "LinkMigration": true,
        "AttachmentMigration": true,
        "AttachmentWorkingPath": "c:\\temp\\attachments",
        "WorkItemIDs": [],
        "MaxRevisions": 100
      }
    ]
  }
}

[MrHinsh]

Your config is not valid. Where did you get it from?

[Lucarian77]

was doing some research online....but now i updated it to this version: Version (Assembly): 16.1.1-Preview.5 and an updated setup......

Error......command used: devopsmigration.exe execute -c config_v16_schema_fixed_final.json

----------
[13:57:50 INF] [16.1.1-Preview.5] Running with settings: {"ConfigFile": "config_v16_schema_fixed_final.json", "DisableTelemetry": false, "DebugTrace": false, "skipVersionCheck": false, "$type": "ExecuteMigrationCommandSettings"}
[13:57:50 INF] [16.1.1-Preview.5] OSVersion: Microsoft Windows NT 6.2.9200.0
[13:57:50 INF] [16.1.1-Preview.5] Version (Assembly): 16.1.1-Preview.5
[13:57:50 INF] [16.1.1-Preview.5] Logpath: C:\tools\MigrationTools\MigrationTools\logs\20250522135749
[13:57:50 DBG] [16.1.1-Preview.5] DetectOnlineService::IsOnline
[13:57:51 WRN] [16.1.1-Preview.5] You are either not online or have chosen `skipVersionCheck`. We will not check for a newer version of the tools.
                                      _
   ___  __  __   ___    ___   _   _  | |_    ___
  / _ \ \ \/ /  / _ \  / __| | | | | | __|  / _ \
 |  __/  >  <  |  __/ | (__  | |_| | | |_  |  __/
  \___| /_/\_\  \___|  \___|  \__,_|  \__|  \___|

[13:57:51 DBG] [16.1.1-Preview.5] ===================================
[13:57:51 DBG] [16.1.1-Preview.5] Configuring 'MigrationTools:Processors:0:Enrichers'
[13:57:51 DBG] [16.1.1-Preview.5] ===================================
[13:57:51 INF] [16.1.1-Preview.5] No validator found for processor type WorkItemTrackingProcessorOptions
[13:57:51 INF] [16.1.1-Preview.5] Creating Migration Engine b919c413-517c-4d75-9c30-cf11f42535ee
[13:57:51 INF] [16.1.1-Preview.5] ProcessorContainer: Of 1 configured Processors only 1 are enabled
[13:57:51 INF] [16.1.1-Preview.5] ProcessorContainer: Adding Processor WorkItemTrackingProcessor
[13:57:51 DBG] [16.1.1-Preview.5] Constructor found: IOptions`1, CommonTools, ProcessorEnricherContainer, IServiceProvider, ITelemetryLogger, ILogger`1
[13:57:51 DBG] [16.1.1-Preview.5] Attempting to pass parameters: OptionsWrapper`1
[13:57:51 DBG] [16.1.1-Preview.5] ===================================
[13:57:51 DBG] [16.1.1-Preview.5] Configuring 'MigrationTools:CommonTools:FieldMappingTool:FieldMaps'
[13:57:51 DBG] [16.1.1-Preview.5] ===================================
[13:57:51 INF] [16.1.1-Preview.5] Logging has been configured and is set to: unknown.
[13:57:51 INF] [16.1.1-Preview.5]                               Max Logfile: Verbose.
[13:57:51 INF] [16.1.1-Preview.5]                               Max Console: Debug.
[13:57:51 INF] [16.1.1-Preview.5]                  Max Application Insights: Error.
[13:57:51 INF] [16.1.1-Preview.5] The Max log levels above show where to go look for extra info. e.g. Even if you set the log level to Verbose you will only see that info in the Log File, however everything up to Debug will be in the Console.
[13:57:51 INF] [16.1.1-Preview.5] Beginning run of 1 processors
[13:57:51 INF] [16.1.1-Preview.5] Processor: WorkItemTrackingProcessor
[13:57:51 INF] [16.1.1-Preview.5] Migration Context Start: WorkItemTrackingProcessor
[13:57:51 INF] [16.1.1-Preview.5] Processor::InternalExecute::Start
[13:57:51 INF] [16.1.1-Preview.5] Processor::EnsureConfigured
[13:57:51 FTL] [16.1.1-Preview.5] Validation of your configuration failed:
The configuration entry at 'MigrationTools:Processors' did not pass validation!
 Please check the following failures:
 -The Endpoint 'Source' specified for `WorkItemTrackingProcessor` was not found.
[13:57:51 INF] [16.1.1-Preview.5] WorkItemTrackingProcessor completed in 00:00:00
[13:57:51 ERR] [16.1.1-Preview.5] WorkItemTrackingProcessor The Processor MigrationEngine entered the failed state...stopping run
[13:57:51 INF] [16.1.1-Preview.5] Command ExecuteMigrationCommand completed in 00:00:01.1966024
[13:57:51 INF] [16.1.1-Preview.5] Check the logs for errors: C:\tools\MigrationTools\MigrationTools\logs\20250522135749

=========
New Setup.....

{
  "Serilog": {
    "MinimumLevel": "Debug"
  },
  "MigrationTools": {
    "Version": "16.1.1",    
    "Endpoints": {
  "Source": {
    "$type": "MigrationTools.Endpoints.AzureDevOps.AzureDevOpsEndpointOptions, MigrationTools",
    "Name": "Source",
    "Organization": "https://dev.azure.com/xxxxVendorSitexxxxx",
    "Project": "ExternalVendorProject",
    "Authentication": {
      "AuthenticationMode": "AccessToken",
      "AccessToken": "xxxxxxxxxxxxxxxxxxxxxxxx"
    },
    "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId"
  },
  "Target": {
    "$type": "MigrationTools.Endpoints.AzureDevOps.AzureDevOpsEndpointOptions, MigrationTools",
    "Name": "Target",
    "Organization": "https://dev.azure.com/xxxxxxxxxOurInternalSitexxxxxx",
    "Project": "DevOpsMigration Test Project",
    "Authentication": {
      "AuthenticationMode": "AccessToken",
      "AccessToken": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    },
    "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId"
  }
},
    "FieldMaps": [
    {
      "$type": "FieldSkipMapConfig",
      "WorkItemTypeName": "*",
      "targetField": "TfsMigrationTool.ReflectedWorkItemId"
    },
    {
      "$type": "MigrationTools.Endpoints.AzureDevOps.AzureDevOpsEndpointOptions, MigrationTools",
      "WorkItemTypeName": "*",
      "sourceField": "System.State",
      "targetField": "System.State",
      "defaultValue": "New",
      "valueMapping": {
        "New": "New",
        "Validate": "Validate",
        "Approved": "Approved",
        "Committed": "Committed",
        "Development Completed": "Development Completed",
        "QA Verification Completed": "QA Verification Completed",
        "Retest": "Retest",        
        "In Review": "In Review",
        "Resolved": "Resolved",
        "Closed": "Closed",
        "Removed": "Removed",
        "Design": "Design",
        "Ready": "Ready",
        "Inactive": "Inactive",
        "In Planning": "In Planning",
        "In Progress": "In Progress",
        "Completed": "Completed"
      }
    }
  ],
    "Processors": [
      {
        "ProcessorType": "WorkItemTrackingProcessor",
        "Enabled": true,
        "SourceName": "Source",
        "TargetName": "Target",
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject",
        "WorkItemCreateRetryLimit": 5,
        "SkipRevisionMigration": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "UpdateSourceReflectedId": false,
        "LinkMigration": true,
        "AttachmentMigration": true,
        "AttachmentWorkingPath": "c:\\temp\\attachments",
        "AttachmentMaxSize": "480000000",
        "WorkItemIDs": [],
        "MaxRevisions": 100
      }
    ]
  }
}

[MrHinsh]

Stop trying to use AI to make the config and go read the docs!

This config is not valid either.

[Lucarian77]

Appreciate the response, though I must be frank …..after investing significant time iterating through configurations, referencing the official documentation line-by-line, and troubleshooting across multiple tool versions, it's discouraging to receive this kind of reply.
The configuration in question was not solely generated by AI (but I was curios how it performs.

If something specific is invalid, pointing that out would be far more helpful than a blanket statement.
I'm fully dedicated to using the tool correctly being a newbie at it. A constructive explanation of what’s missing or misaligned would be far more valuable than being told to “go read the docs” after already doing so repeatedly.

[MrHinsh]

@Lucarian77 your config includes elements that either no longer exist or were deprecated years ago. Almost all of the issues you listed can be resolved by starting from the Getting Started guide and using the sample provided there.

While some of the inline examples vary in age, every sample in the Examples section of each documented feature is generated directly from the current code and is accurate. If you have an old config, we provide devopsmigration upgrade to clean out obsolete elements and modernise where possible.

Let’s walk through the issues:

1. Version has an invalid value. Compare with any sample in the examples sections or Getting Started.
2. Source and Target are in the wrong place. Again, see Getting Started.
3. $type is not used, and the values you provided are invalid. Check the examples.
4. AzureDevOpsEndpointOptions is not a valid format. Refer to the [AzureDevOpsEndpoint documentation(https://nkdagility.com/learn/azure-devops-migration-tools/Reference/Endpoints/AzureDevOpsEndpoint/).
5. Authentication is not a valid property for [AzureDevOpsEndpoint](https://nkdagility.com/learn/azure-devops-migration-tools/Reference/Endpoints/AzureDevOpsEndpoint/). See the correct usage in both AzureDevOpsEndpoint and Getting Started.
6. AzureDevOpsEndpoint is not the correct endpoint for work item migration. See Getting Started for proper endpoint usage.
7. Organization is not valid for the correct endpoint, which is TfsTeamProjectEndpoint.
8. FieldMaps is in the wrong location. Refer to the FieldMappingTool.
9. Do not use WorkItemTrackingProcessor. Use TfsWorkItemMigrationProcessor instead. Again, Getting Started covers this.

I’m not suggesting our docs are flawless, but everything you needed was right there on the Getting Started page. If you’d actually read through and followed the documented example, you wouldn’t be dealing with these errors.

These tools are provided as-is by Azure DevOps consultants for other Azure DevOps consultants. They're not commercial and they’re not designed to be beginner-friendly. If you're unfamiliar with JSON, debugging in Visual Studio, or how the old TFS Object Model works, you might want to find someone to pair with.

Also, you commented on the correct documentation but clearly didn’t follow it.

One last note: you can format code in GitHub issues using single backticks for inline code and triple backticks for code blocks. See GitHub’s Basic writing and formatting syntax. I’ve cleaned up your post to improve readability.

[deepakgupta82]

We are trying to do the migration of workitems from one azure devops org to another. The work items migrate correctly except for the embedded images. The image placeholders shows up in target workitem Description field but no images in target. When I look at html extract of target work item description field, the image url is pointing to source devops organization.

Following is our processor config:

{
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan','Shared Steps','Shared Parameter','Feedback Request') ORDER BY [System.ChangedDate] desc",
        "FixHtmlAttachmentLinks": true,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": false,
        "GenerateMigrationComment": false,
        "SourceName": "Source",
        "TargetName": "Target"
      },

Can anyone please advise how to enable embedded image migration ?
The source and destination PAT tokens are correctly configured also.

  "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/TargetDevOpsOrg",
        "Project": "VISTA",
        "TfsVersion": "AzureDevOps",
        "AllowCrossProjectLinking": false,
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "YYY"
        },
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      },
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/SourceDevOps/",
        "Project": "VISTA",
        "TfsVersion": "AzureDevOps",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "XXX"
        },
        "AllowCrossProjectLinking": false,
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      }
    },

[MrHinsh]

Your config looks good. Do you get any messages in the logs?

Id run your config through the visual studio debugger and see whats happening in the TfsEmbededImagesTool... why its failing.

[deepakgupta82]

Thanks a lot. I'll give it a shot.

[deepakgupta82]

Thanks a lot @MrHinsh for pointing me to TfsEmbededImagesTool
I got following error after increasing the log level.

ERR] [16.1.0] EmbededImagesRepairEnricher: Unable to fix HTML field attachments for work item 7126 from https://dev.azure.com/Org1/ to https://dev.azure.com/Org2
Microsoft.TeamFoundation.WorkItemTracking.WebApi.RuleValidationException: TF401320: Rule Error for field Prio. Error code: Required, HasValues, LimitedToValues, AllowsOldValue, InvalidEmpty.
at MigrationTools.Tools.TfsEmbededImagesTool.UploadedAndRetrieveAttachmentLinkUrl(String matchedSourceUri, String sourceFieldName, WorkItemData targetWorkItem, String sourcePersonalAccessToken) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsEmbededImagesTool.cs:line 163
at MigrationTools.Tools.TfsEmbededImagesTool.FixEmbededImages(WorkItemData wi, String oldTfsurl, String newTfsurl, String sourcePersonalAccessToken) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsEmbededImagesTool.cs:line 94

Any suggestions would be appreciated.
Best Regards,

[deepakgupta82]

We managed to get it to work by removing the Required option from Custom.Prio field in both Source and Destination.