The FIO test reported an "Out of TCS" error.

[zzzqwqw]

I conducted FIO testing (test code: https://github.com/axboe/fio) inside a container running with the enclave-cc runtime. The pod configuration is as follows:

apiVersion: v1
kind: Pod
metadata:
  name: enclave-pod
spec:
  containers:
  - name: sysbench-test
    image: sgx-fio222:1.0
    workingDir: "/run/rune/occlum_instance/"
    resources:
      limits:
        sgx.intel.com/epc: 105Mi
    env:
    - name: OCCLUM_RELEASE_ENCLAVE
      value: "1"
    command:
    - /run/rune/occlum_instance/build/bin/occlum-run
    - /bin/fio
    - /bin/fio-seq-read.fio
  runtimeClassName: enclave-cc

For the parameters in the fio-seq-read.fio file, I reduced them as much as possible as follows:

[global]
name=fio-seq-read
filename=/root/fio-seq-read
rw=read
bs=32K
direct=1
numjobs=1
time_based
runtime=30

[file1]
size=32M
ioengine=sync
iodepth=1

The container starts successfully, but it throws the following error during execution:

[mythi]

the default config is

enclave-cc/tools/packaging/build/unified-bundle/agent-instance.filter

Line 4 in 66fed75

.resource_limits.max_num_of_threads = 8 |

but I don't know why numjobs=1is not enough

[mythi]

https://occlum.readthedocs.io/en/latest/edmm_config_guide.html#tcs-number

[zzzqwqw]

Thank you for your reply. If I want to appropriately increase .resource_limits.max_num_of_threads, how should I do it? Is this related to EPC resources?

[mythi]

we don't have good docs for it but it's about changing that file and rebuilding the enclave.

In general, I'd suggest to check with the Occlum team why non-EDMM config with .resource_limits.max_num_of_threads = 8 does not work with your app.

[zzzqwqw]

Thank you for your reply. I have one more small question. When pulling the image, the pod status shows ImagePullBackOff, and the logs also display an out of TCS error. Could you please let me know what might be causing this?

sudo cat /run/containerd/agent-enclave/5608a2a01c5c7382b2f855c2bf79e9968242f1f345f57dc8e285f0d03a614fbc/stderr
[2025-06-06T07:19:30Z INFO  enclave_agent] ttRPC server started: "tcp://127.0.0.1:7788"
[2025-06-06T07:19:30Z INFO  enclave_agent::services::images] Pulling "crpi-7pimyyua3acpc1qy.cn-hangzhou.personal.cr.aliyuncs.com/zzzqw/sgx-fio666:1.0"
[ERROR] occlum-pal: Failed to enter the enclave to execute a LibOS thread (host tid = 14) with error code 0x1003: Out of TCS. (line 21, file src/ocalls/spawn.c)

[mythi]

Could you please let me know what might be causing this?

I don't know for sure but I suspect it's just faulty error handling in the containerd fork this uses (i.e., enclave start failure just shows up as ImagePullBackOff.)