How to use MINARI datsets in CORL

[ObyBayat]

Question

Hi, my research project requires me to use MINARI datasets in CORL(clean offline reinforcement learning) especially focused on the Decision Transformer. I see that CORL uses d4rl instead of MINARI datasets so there is a lot I had to change in the script and it runs however results are awful even using the same hyperparameters as CORL. Can anyone help me on this matter?

[younik]

Hi @ObyBayat, apologies for the lag in the reply.

@alexdavey used CORL's IQL script for Antmaze datasets in Minari https://gist.github.com/alexdavey/62237c9affefbfb5e4867cc740a60deb

This was adapted from this PR, which also contains the behavioral cloning script: corl-team/CORL#3

[ObyBayat]

Hi @younik , thanks for the reply. I did find that but I specifically need Decision Transformer. I have managed to adapt CORL's decision transformer for MINARI datasets however the results are no where close to CORL so there is definitely issues with the logic. Do you think it is worth it for me to continue persuing adapting the DT for MINARI or is it more than just editing the DT script?

[ObyBayat]

Is it as simple as changing load_d4rl_trajectories function to load_minari_trajectories ofcourse loading MINARI trajectories there, and other small tweaks suited for gymnasium and MINARI?

dt_minari.py

[alexdavey]

Hi @ObyBayat, apologies for the delayed response on this. There is a WIP CORL integration which you can find here: https://github.com/alexdavey/CORL/tree/minari. Unfortunately the script for the CORL decision transformer is not fully implemented yet as the way it loads the dataset is somewhat different.

It might be useful to try running e.g. IQL (either from CORL or the other integrations) on your dataset to confirm that the issues you are seeing are due to the decision transformer implementation and not due to a dataset difference, as several of the Minari datasets are somewhat different to the original D4RL ones.

[ObyBayat]

Yes I definitely have noticed that most MINARI datasets are different from d4rl. I will give a shot testing the adpated IQL. Thanks for the help and I look forward to the complete MINARI/CORL intergration.