Dagger Training Stableness

[HuskyKingdom]

Hi Jacob,

Thanks for your interesting work!
I was recently trying my model on VLNCE, specifically, I have been following the IL+Dagger training schedule like the original paper. It is found that using pure dagger training on the original data (without finetuning and PM) to train the model from iteration 0-9, the loss curve initially decreases during the first few iterations when P is large (hence more chance of selecting GT actions), but it shows an increasing trend in later iterations (iteration 4-5 as shown below, further iterations are still under training).

It is somehow making sense given the fundamental idea behind the Dagger training, but I just wanna ask if you had anything similar in your experiments, or could it be the problem on my side?

Thanks for your time and looking forward to your reply. Please also let me know if you need further information.

[BaiLing09]

Hello, can you download the RxR_VLNCE_v0.zip? I can't open this link. If you can download, could you send this RxR episodes dataset to me by the e-mail? My e-mail is 3309321488@qq.com. Thanks!

[bryce-wan]

Hello, can you download the RxR_VLNCE_v0.zip? I can't open this link. If you can download, could you send this RxR episodes dataset to me by the e-mail? My e-mail is 3309321488@qq.com. Thanks!

The link is expired and I have sent you a copy I downloaded before. Please check your mailbox : )

[jacobkrantz]

Hey @HuskyKingdom,
My intuition from observing your loss curves is that as the GT action probability decays, the agent encounters increased novel states in the state space. If the policy gathers more data before learning useful behavior from the previous dagger iteration, the resulting dataset will contain nonsensical training samples (e.g. "stuck" against walls). These samples may lead your policy to converge on undesirable behaviors. Ideas to avoid this include changing the decay function or increasing training epochs per DAgger iteration. Evaluate these models on a validation split though, because DAgger loss does not necessarily correlate with actual navigation performance. Good luck!

[jacobkrantz]

The RxR data link has been fixed (729d141). Thanks for drawing attention to this!