In the hepar2 example GRaSP clearly outperforms all other methods (Fig. 8 (c) and (d) in the paper) . After some inspection, it might be because in the instance the variables are topologically ordered and with flag --default (which is set in the script.sh file) GRaSP seems to start its permutation search with the identity permutation (which is already the "correct" one here).
The relevant option which is set when using the default flag is called graspUseDataOrder in Tetrad 1.10 (in later versions it is just called useDataOrder and also available and set when using --default in BOSS). When starting GRaSP without --default and setting all default options except graspUseDataOrder manually, the performance is worse.
More generally, it could be a good idea to (i) randomly permute the variables in the graph instance or (ii) the columns in the generated data to avoid such biases.
In the hepar2 example GRaSP clearly outperforms all other methods (Fig. 8 (c) and (d) in the paper) . After some inspection, it might be because in the instance the variables are topologically ordered and with flag --default (which is set in the
script.shfile) GRaSP seems to start its permutation search with the identity permutation (which is already the "correct" one here).The relevant option which is set when using the default flag is called graspUseDataOrder in Tetrad 1.10 (in later versions it is just called useDataOrder and also available and set when using --default in BOSS). When starting GRaSP without --default and setting all default options except graspUseDataOrder manually, the performance is worse.
More generally, it could be a good idea to (i) randomly permute the variables in the graph instance or (ii) the columns in the generated data to avoid such biases.