Custom Sharding iOS Firebase Error

[micai23]

Hi I'm trying to implement custom sharding on iOS. Without a custom sharding json file, i'm able to run tests using github actions. However with custom sharding the name of the xctestrun file changes and adds shard_0 to the original xctestrun file uploaded.

Error

2024-04-18_13-11-42.585010_OvOW_shard_0_TIDAL-UITests_UITestsPlan_iphoneos17.2-arm64_shard_0.xctestrun

┌─────────┬──────────────────────┬───────────────┬────────────────────────────────────────────────────────────────┬─────────────────┬───────────────────────┐
│ OUTCOME │      MATRIX ID       │   APP NAME    │                         TEST FILE NAME                         │ TEST AXIS VALUE │     TEST DETAILS      │
├─────────┼──────────────────────┼───────────────┼────────────────────────────────────────────────────────────────┼─────────────────┼───────────────────────┤
│ INVALID │ matrix-24jr1bniisa4d │ Tidal-app.zip │ TIDAL-UITests_UITestsPlan_iphoneos17.2-arm64_shard_0.xctestrun │                 │ MALFORMED_XC_TEST_ZIP │

Here are my config files

On github actions

    - name: Run Firebase UI Tests using Flank
        id: flank_run
        uses: Flank/flank@master
        with:
          version: v23.10.1
          service_account: ${{ secrets.GOOGLE_CREDENTIALS_DEBUG }}
          platform: ios
          flank_configuration_file: ./scripts/flank-firebase-tests.yml

The flank-firebase-tests.yml config file

gcloud:
  record-video: true
  timeout: 45m
  async: false
  test: ./Tidal-app.zip
  xctestrun-file: ./Tidal-UITests_UITestsPlan_iphoneos17.2-arm64.xctestrun
  device:
    - model: iphone14pro
      version: 16.6
      locale: en
      orientation: portrait
  type: xctest
flank:
  max-test-shards: 1
  shard-time: -1
  disable-sharding: false
  custom-sharding-json: ./scripts/firebase-custom-sharding.json

Sharding json file, firebase-custom-sharding.json

{
  "Configuration 1": [
    {
      "EmptyUser1Tests": [
        "TIDALUITestsEmptyUser",
        "TIDALUITestsNowPlayingSuggestions"
      ]
    }
  ]
}

[ypenn21]

The sharding feature is lacking currently. According to this blog.. https://sahil-insights.medium.com/real-device-testing-xcuitest-firebase-test-lab-github-actions-1cb5f07f4ba6 . However for android instrumentation it is available.