Submission Failed: SIGKILL (Signal 9) during SPICE evaluation

[umme17]

The nocaps on EvalAI. My submission consistently fails during the evaluation phase.
The evaluation script crashes during the SPICE score calculation. Looking at the logs, it appears the Java process is attempting to allocate 8GB of RAM (-Xmx8G), which is triggering an OOM (Out of Memory) event on the worker instance, resulting in a SIGKILL: 9.

Traceback (most recent call last):
  File "/code/scripts/workers/submission_worker.py", line 569, in run_submission
    submission_metadata=submission_serializer.data,
  File "/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/main.py", line 60, in evaluate
    cocoEval.evaluate()
  File "/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/pycocoevalcap/eval.py", line 67, in evaluate
    score, scores = scorer.compute_score(self.gts, self.res)
  File "/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/pycocoevalcap/spice/spice.py", line 79, in compute_score
    cwd=os.path.dirname(os.path.abspath(__file__)))
  File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['java', '-jar', '-Xmx8G', 'spice-1.0.jar', '/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/pycocoevalcap/spice/tmp/tmpgcvy2wid', '-cache', '/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/pycocoevalcap/spice/cache', '-out', '/tmp/tmp80rw7tky/compute/challenge_data/challenge_355/pycocoevalcap/spice/tmp/tmpgeszo2c3', '-subset', '-silent']' died with <Signals.SIGKILL: 9>.

[Danchi-1]

.take Could I work on this?
My goal would be memory optimization while prevent OOM error from happening. Allocating 8gb; this step would cause a number of issues like slower container and startup operations which would best be avoided. We'd want to monitor memory usage in workers and queue evaluation during low memory instances. Enforcing -Xmx4g instead of -Xmx8g. Any better suggestions would be appreciated.