Quick question about the number of demonstrations used in ACT benchmark runs

[jstmn]

Howdy,

I wanted to ask about the number of demonstrations used to train some of the ACT models. It says there were 990 demos for the StackCube task for this run: https://wandb.ai/stonet2000/ManiSkill/runs/7jjujyd6/overview?nw=nwuserstonet2000 this is around 10x what I would expect for a simple task. Was this intentional? Did you see better results be increasing the number of demos?

Thanks

[StoneT2000]

Performance plateaus in my experience, the classic ACT used here is not that good. StackCube also has a pretty high amount of randomization.

[jstmn]

I've seen the same. Why so many then? Is this a multitask dataset?

[StoneT2000]

It's not "multi-task" per say in my opinion (multitask is a very loose term here I guess). It's just difficult, requires a decent amount of precision. The two cubes are also the same size, I think some other benchmarks like robosuite make it easier by making the bottom cube bigger than the cube that needs to be stacked on top.

[jstmn]

Ah to clarify, "multi task" I was wondering if the dataset contains demonstrations from other tasks?

[StoneT2000]

The dataset only contains data generated from that particular environment ID.