Rollback commit does not remove created code system version and release branch

[jonmdtn]

Hi,

We're experiencing some strange behavior after updating from version 8.3.0 to version 10.4.2.

The content has all been loaded into the v10 instance using RF2 files and should be as clean as possible. When rolling back a code system branch across our two instances (one on 8.3.0, other on 10.4.2), the one running version 10.4.2 does not delete the created code system version or release branch.

Doing a test on a clean instance only using snapshot imports and a few code systems for testing tells me the rollback functionality works as expected, so we definitely have some state that is not expected by Snowstorm.

I've confirmed the head and base timestamps match, and that manually hard-deleting the release branch and the version entry in codesystem-versions index does remove the code system version and release branch without any noticeable side-effects.

I suspect the whole condition on line 159 of SBranchService on master isn't run since we do not see the warning of child branch using the commit being rolled back in the system logs.

This seems to not affect code systems that only contain reference set members as content, which are children of our national extension. Otherwise it affects both MAIN and our national extension branch(es).

Any ideas on what could be causing this, and is our workaround unsafe to use? We've been thinking we might have some weird integrity issues or something like that, but find it strange this wouldn't work on MAIN, and especially strange it doesn't give any rebase warnings doing this on MAIN.

Thanks in advance,
jonmdtn

[kaicode]

On the v10 instance it may be possible that the import is creating two commits. Take a note of the timestamp before the import and after the first rollback to see if they match. If not try running one more rollback. There is an automation around missing Module Dependency Refset members that could be making a second commit.

[jonmdtn]

I will have a close look at the timestamps while testing this tomorrow, but it may be worth mentioning that we rolled back MAIN past two releases and both branches MAIN/2025-05-01 and MAIN/2025-04-01 were marked as "UP_TO_DATE", so we definitely tried doing multiple rollbacks.

[jonmdtn]

Hi again,
We've been able to replicate this issue using only the international edition on MAIN branch, noting it here as it may help in troubleshooting. We tested both version 10.4.2 and 10.8.2 and saw the same behaviour in both. We have not found any real pattern in which branches this does and does not work for otherwise unfortunately, but the only code systems we don't see this for are ones that only contain refset members, as mentioned.

Steps to reproduce:

1. Start a fresh snowstorm v10.8.2 instance with no data loaded in elasticsearch (we use the official docker hub image and elasticsearch:8.11.1)
2. Do a full import of June 2025 international edition
3. Make a note of MAIN/2025-05-01 base timestamp (May 2025)
4. Perform rollback commit on MAIN until the head timestamp matches the noted version for may.
5. Observe that MAIN/2025-06-01 still exists, and SNOMEDCT code system still has a 20250601 version.

Entities on MAIN/2025-06-01 are of course gone since they come from MAIN. Reimport of June 2025 fails the code system creation since it already exists, but we can now hard-delete MAIN/2025-06-01 and recreate it to "fix" the release branch.

Im assuming this is not expected behaviour even though there might some special exceptions for MAIN?

[kaboyles]

Hi, any thoughts or suggestions considering the ability to reproduce this with an International version, as mentioned above?

Thanks,
Kim

[kaboyles]

Hi! Hoping to hear from someone on this.

Thanks,
Kim

[kaicode]

This is not expected behaviour at all. I will try to reproduce this.

[kaicode]

I was not able to reproduce the issue you saw.

My procedure:

• Start empty Docker with Snowstorm 10.8.2, Elasticsearch 8.11.1
• Import May 2025 Snapshot
• Import June 2025 Snapshot
• Check Codesystem Versions and release branches are correct
• Rollback one commit on MAIN
  • This removed the June 2025 Codesystem version and deleted the MAIN/2025-06-01 release branch
  • The June content is not reverted yet.
  • The June release branch is BEHIND
• Rollback one more commit on MAIN
  • Now June release branch is UP_TO_DATE
  • All May content is removed

Did you have any other child branches? Perhaps a child codesystem or an import onto one of the version branches?

---

Screenshots from my test:

Snowstorm version

Elasticsearch version

Codesystem Versions after imports

Branches after imports

MAIN branch before first rollback

First rollback "200" Success response

Logs after first rollback
Snowstorm seems to be writing log entries twice, but it's doing each thing once, can tell from the millisecond timestamps.

Codesystem Versions after first rollback (June gone)

Release branches after first rollback (June gone)

Second rollback "200" Success response

After second rollback May branch is UP_TO_DATE
This means it has the same content as MAIN. All the June content is gone.

[jonmdtn]

I was only able to reproduce the issue consistently when doing a FULL import. We are currently testing to see if my local machine running ARM chipset might be the issue. Building the data using SNAPSHOT imports always worked as expected.

Thanks for testing though!

[kaicode]

My testing used MacBook Pro M4 with ARM based Apple Silicon.

[jonmdtn]

Alright we have now both tested the procedure on a MacBook Pro M1, and a Dell Windows machine, and we both see this rollback issue on MAIN after performing a FULL import. This was done using the REST API and not using the startup parameters for Snowstorm.

This is the only method we have found so far to illustrate the issue, but it acts the same way with our national extension branches on our live servers. I guess we should consider rebuilding our content using SNAPSHOT imports, but we will then have to check we don't rely on full history in our dependant services.

[kaicode]

Do you need to rollback a release after importing the FULL version?

[jonmdtn]

We do not, but we are experiencing the problem described in the issue on national and local extension branches in our production environment (read only), which are built upon a FULL import of international edition, and on those we do need to be able to roll back. This is currently blocking us from updating from version 8.3.0 in our daily build environment, where the local extension is produced.

We have not been able to reproduce this issue to figure out how to prevent it from happening, other than with the described method on the MAIN branch, which has identical symptoms to what we are seeing in our extension branches.

This is of course open-source and we'd love to properly debug this ourselves, but are limited by resources and will likely bypass this using disk-backups or ES snapshots.

We could provide an ES snapshot or copy of the docker volume where we are seeing this in the descendant branches of MAIN if you think it is worth investigating.

[jonmdtn]

This is currently stopping us from integrating our authoring environment with the new mapping tool from WCI, so if you'd be able to review our workaround, that would be very helpful.

The workaround is simply to do a rollback commit until the desired point, then manually hard-deleting any release branches that should be removed, and then removing the version entry in the codesystem-versions index.

Would you advise against doing this? If so, we will simply use ES snapshots or disk backups to simulate roll backs to specific times.

Thanks in advance,
Jonathan

[kaicode]

Do you really need to import using the FULL? We don't test this much because it's a bit of an edge case. We import the FULL for the SI public browser but we expect everyone else to use a SNAPSHOT import type.

If you need the May release you could just import that snapshot?