OCI 1.1 referrers (Cosign v3 signatures/SBOMs) are not replicated from Artifactory to Harbor

[belmin-devops-eng]

Expected behavior

When replicating images from a JFrog Artifactory registry to Harbor, all OCI 1.1 artifacts associated with the image should be replicated. This includes referrers such as Cosign v3 signatures, attestations, and SBOMs. After replication, the image in Harbor should be functionally equivalent to the source, including its linked supply-chain metadata.

Actual behavior

Only the container image itself and legacy Cosign v2.x artifacts (which use the .sig tag convention) are replicated. OCI 1.1 referrer artifacts (tagless signatures/SBOMs linked via the subject field) are ignored by the replication job and are missing in Harbor after completion.

Note: Harbor 2.15.0 correctly supports and displays these OCI 1.1 artifacts if they are pushed directly or copied via ORAS; the issue is strictly isolated to the Replication Service.

For related context on OCI 1.1 recognition, see issue: Cosign signatures using OCI 1.1 are not recognized #23013

---

Steps to reproduce the problem

• Push an image to JFrog Artifactory and sign it using Cosign v3.x, generating signatures and attaching SBOMs/attestations stored as OCI 1.1 referrers.
• Verify that referrers exist in Artifactory (e.g. using oras discover or via UI).
• Create a replication endpoint in Harbor pointing to Artifactory (tested with both Docker Registry and JFrog Artifactory providers).
• Create and run a Pull-based replication rule from Artifactory to Harbor.
• Observe the replicated repository in Harbor.

---

Versions

• ORAS: 1.3.1
• Cosign: 3.0.5
• Harbor: 2.15.0 (also tested with 2.14.1)
• JFrog Artifactory (Self-Managed): 7.146.10

---

Additional context

• Harbor correctly displays and stores Cosign v3 OCI artifacts when directly pushed.
• The issue appears specific to Harbor replication.
• Legacy Cosign v2.x signatures (sha256-<digest>.sig) replicate correctly. Only OCI 1.1 referrers are affected.
• Using oras copy --recursive between the same Artifactory and Harbor instances successfully copies the image and all referrers (signatures, attestations, SBOMS, etc.). This confirms that both registries support the OCI 1.1 spec and that the artifacts are accessible via standard OCI calls.
• While PR feat(cosign): Support Cosign v3 Bundle signature format #22628 improved Harbor's internal handling of Cosign v3 bundles, the Replication Service appears to lack the logic to traverse the Referrers API during a sync task.

I also tested different replication endpoint providers:

• Provider (Docker Registry)
  Replication succeeds for the image, but misses OCI 1.1 referrers. Legacy Cosign v2.x signatures (<image-digest>.sig) are replicated correctly.
• Provider (JFrog Artifactory)
  Replication fails discovery entirely. The job either finishes with total=0 or hangs indefinitely, despite the endpoint test being successful.

---

I am relatively new to Harbor, so if I have misconfigured something or missed a required setting, please let me know. If additional logs or information are needed, I am happy to provide them. Thank you.

[MinerYang]

Hi @belmin-devops-eng ,

Could you please provide the log for replication task(from UI portal) for both replicated from the DockerHub and the JFrog?

[belmin-devops-eng]

Hi @MinerYang ,

First of all, thank you for the quick response.

To assist with the investigation, I enabled logLevel: debug and configured jobLoggers to output to stdout in Harbor.

---

Configuration Details

To provide more context on how the endpoints and replication rules are configured, I am attaching several screenshots.

Endpoints

The connection test succeeds for both endpoints.
The endpoint URL is identical in both cases and looks like: https://my-docker-remote-repository.my-artifactory-server.com

Replication Rules

---

Logs

After analyzing the logs, I found a significant behavioral difference between the two replication rules.

Docker Registry Replication

The replication rule using the standard docker-registry endpoint works correctly. The replication job is triggered successfully and transfers the container artifacts (except OCI 1.1 referrers, as described in the original issue).

Below are sanitized excerpts from the jobservice logs for a successful execution:

[DEBUG] [/jobservice/api/handler.go:281]: Serve http request 'POST /api/v1/jobs': 202 {"job":{"id":"5ce576aeccd64e94138d265e","status":"Pending","name":"REPLICATION","kind":"Generic","unique":false,"ref_link":"/api/v1/jobs/5ce576aeccd64e94138d265e","enqueue_time":1778247682,"update_time":1778247682,"web_hook_url":"http://harbor-core:80/service/notifications/tasks/661","parameters":{"copy_by_chunk":false,"dst_resource":"{\"type\":\"image\",\"metadata\":{\"repository\":{\"name\":\"docker-registry-namespace/XXX\"},\"artifacts\":[{\"tags\":[\"XXX\"]}],\"v_tags\":null},\"registry\":{\"id\":0,\"name\":\"Local\",\"type\":\"harbor\",\"url\":\"http://harbor-core:80\"},\"override\":true}","speed":0,"src_resource":"{\"type\":\"image\",\"metadata\":{\"repository\":{\"name\":\"repo/XXX\"},\"artifacts\":[{\"tags\":[\"XXX\"]}],\"v_tags\":null},\"registry\":{\"id\":5,\"name\":\"remote-registry\",\"type\":\"docker-registry\",\"url\":\"https://remote-registry-url.com\"},\"override\":false}"}}}
[INFO] [/jobservice/worker/cworker/c_worker.go:77]: Job incoming: {"name":"REPLICATION","id":"5ce576aeccd64e94138d265e","t":1778247682,"args":null}
[INFO] [/controller/replication/transfer/image/transfer.go:140]: client for source registry [type: docker-registry, URL: https://remote-registry-url.com, insecure: false] created
[INFO] [/controller/replication/transfer/image/transfer.go:150]: client for destination registry [type: harbor, URL: http://harbor-core:80, insecure: true] created
[INFO] [/controller/replication/transfer/image/transfer.go:183]: copying repo/XXX:[XXX](source registry) to docker-registry-namespace/XXX:[XXX](destination registry)...
[INFO] [/controller/replication/transfer/image/transfer.go:465]: pulling the manifest of artifact repo/XXX:XXX ...
[INFO] [/controller/replication/transfer/image/transfer.go:471]: the manifest of artifact repo/XXX:XXX pulled
[INFO] [/controller/replication/transfer/image/transfer.go:227]: the artifact docker-registry-namespace/XXX:XXX already exists on the destination registry, skip
[INFO] [/controller/replication/transfer/image/transfer.go:205]: copy repo/XXX:[XXX](source registry) to docker-registry-namespace/XXX:[XXX](destination registry) completed
[INFO] [/jobservice/runner/redis.go:152]: Job 'REPLICATION:5ce576aeccd64e94138d265e' exit with success
[DEBUG] [/jobservice/hook/hook_agent.go:118]: Hook event is successfully sent: status change: job=5ce576aeccd64e94138d265e, status=Success, revision=1778247682->http://harbor-core:80/service/notifications/tasks/661

[DEBUG] [/jobservice/api/handler.go:281]: Serve http request 'POST /api/v1/jobs': 202 {"job":{"id":"da700dee77df85a4eef8afa5","status":"Pending","name":"REPLICATION","kind":"Generic","unique":false,"ref_link":"/api/v1/jobs/da700dee77df85a4eef8afa5","enqueue_time":1778247682,"update_time":1778247682,"web_hook_url":"http://harbor-core:80/service/notifications/tasks/662","parameters":{"copy_by_chunk":false,"dst_resource":"{\"type\":\"image\",\"metadata\":{\"repository\":{\"name\":\"docker-registry-namespace/XXX\"},\"artifacts\":[{\"tags\":[\"XXX\"]}],\"v_tags\":null},\"registry\":{\"id\":0,\"name\":\"Local\",\"type\":\"harbor\",\"url\":\"http://harbor-core:80\"},\"override\":true}","speed":0,"src_resource":"{\"type\":\"image\",\"metadata\":{\"repository\":{\"name\":\"repo/XXX\"},\"artifacts\":[{\"tags\":[\"XXX\"]}],\"v_tags\":null},\"registry\":{\"id\":5,\"name\":\"remote-registry\",\"type\":\"docker-registry\",\"url\":\"https://remote-registry-url.com\"},\"override\":false}"}}}
[INFO] [/jobservice/worker/cworker/c_worker.go:77]: Job incoming: {"name":"REPLICATION","id":"da700dee77df85a4eef8afa5","t":1778247682,"args":null}
[INFO] [/controller/replication/transfer/image/transfer.go:140]: client for source registry [type: docker-registry, URL: https://remote-registry-url.com, insecure: false] created
[INFO] [/controller/replication/transfer/image/transfer.go:150]: client for destination registry [type: harbor, URL: http://harbor-core:80, insecure: true] created
[INFO] [/controller/replication/transfer/image/transfer.go:183]: copying repo/XXX:[XXX](source registry) to docker-registry-namespace/XXX:[XXX](destination registry)...
[INFO] [/controller/replication/transfer/image/transfer.go:465]: pulling the manifest of artifact repo/XXX:XXX ...
[INFO] [/controller/replication/transfer/image/transfer.go:471]: the manifest of artifact repo/XXX:XXX pulled
[INFO] [/controller/replication/transfer/image/transfer.go:227]: the artifact docker-registry-namespace/XXX:XXX already exists on the destination registry, skip
[INFO] [/controller/replication/transfer/image/transfer.go:205]: copy repo/XXX:[XXX](source registry) to docker-registry-namespace/XXX:[XXX](destination registry) completed
[INFO] [/jobservice/runner/redis.go:152]: Job 'REPLICATION:da700dee77df85a4eef8afa5' exit with success
[DEBUG] [/jobservice/hook/hook_agent.go:118]: Hook event is successfully sent: status change: job=da700dee77df85a4eef8afa5, status=Success, revision=1778247682->http://harbor-core:80/service/notifications/tasks/662

---

JFrog Artifactory Replication

In contrast, the replication rule using the JFrog Artifactory provider does not appear to reach the Jobservice execution stage.

When checking the jobservice logs during the scheduled execution time, I could not observe any POST /api/v1/jobs requests related to the JFrog replication task, and there is no indication that a replication job was created.

The jobservice logs only show routine heartbeat checks (GET /api/v1/stats) and scheduler policy loading messages.

Similarly, the core logs do not show evidence that Harbor attempted to start this specific replication task.

---

UI Behavior

Below is the behavior observed in the Harbor UI when manually triggering the replication rules.

Docker Registry Replication

After triggering the docker-registry replication rule, the result looks like this:

When clicking on the execution logs on the right side (for any of the successful tasks), the UI returns:

{"errors":[{"code":"UNKNOWN","message":"internal server error"}]}

JFrog Artifactory Replication

Below is the behavior when triggering the JFrog Artifactory replication rule from the UI:

The execution appears to hang indefinitely. No additional logs are shown in the UI, and there are no corresponding entries in the jobservice logs.

Please let me know if you need any additional information, logs, or tests. I will do my best to assist further.

Thank you.

[dyrnq]

stare

[belmin-devops-eng]

Just to clarify, the main goal of this issue is to ensure that Harbor replication preserves the complete OCI artifact graph during replication (images, OCI 1.1 referrers such as Cosign v3 signatures, attestations, SBOMs, etc.).

Whether the source endpoint is configured as docker-registry or JFrog Artifactory is secondary from my perspective. The docker-registry provider already demonstrates that Harbor can successfully transfer the image content itself, so perhaps it would make sense to focus on fixing the OCI 1.1 referrer replication problem first rather than mixing it with another potentially separate issue.

The JFrog Artifactory provider behavior may be an independent problem, while the core issue is that the Replication Service does not appear to traverse and replicate OCI 1.1 referrers via the Referrers API.