Fair team recommendation experiments

[mahdis-saeedi]

Hi @hosseinfani,

I create this issue page to keep track of fairness related experimental issues.

[mahdis-saeedi]

Hi @hosseinfani ,

The following error leads to stop running the model with the following command run:

  python -u main.py cmd=fair data.domain=cmn.publication.Publication data.source=/home/OpeNTF/data/dblp/dblp.v12.json data.output=/home/msaeedi/OpeNTF/output/dblp/dblp.v12.json "models.instances=[mdl.fnn.Fnn]" train.nfolds=5 +models.batch_size=250 +models.fnn.tpw=100 +models.fnn.tnw=1 +models.nsd=unigram ~data.embedding eval.metrics.other=[]  fair.fgender=/home/msaeedi/OpeNTF/output/dblp/dblp.v12.json.mt10.ts2/females.csv    fair.k_max=1000  fair.is_popular_coef=1  hydra.run.dir=/home/msaeedi/OpeNTF/output/dblp/logs/dblp.mt10.ts2.fnn.b250.e100.ns5.lr0.001.es5.h128.spe10.lbce.tpw100.tnw1.nsdunigram.fair &

it seems the issue is related to installing fairsearchcore .

Would you please help me to fix the issue?
should i try to manually install a compatible version of fairsearchcore in the environment?

[hosseinfani]

Hi @mahdis-saeedi
thank you. I'm looking into it.
no, the pipeline should install itself. Specifically, for this package, the pipeline should install it the one from a fork in fani's lab.

[hosseinfani]

Hi @mahdis-saeedi
1- Please update your opentf code with submodule option to get the latest of Adila.
2- In this colab script, if you search for fairsearchcore, you can see that it should be installed from fani's lab fork. Make sure that the machine running the code has access to internet.
3- Also, check this line and see you have it in your local file.

[mahdis-saeedi]

Hi @hosseinfani,
Thank you for the points. The issue was in the requirements.tex in my local. it was old because I usually update only the src folder, not the whole OpeNTF.

[mahdis-saeedi]

Hi @hosseinfani,

We face the following error.

It seems y_pred is saved as a sparse tensor in Adila and we face the error when we try to sort them in _get_labeled_sorted_preds , because pytorch cant sort sparce cpu tensors.

[hosseinfani]

Hi @mahdis-saeedi
I fixed the issue. please update your opentf with recursive submodule option. Now, the adila works with both dense, and sparse pred files up to topK==k_max first-stage cutoff. Let me know if there is an issue.

fani-lab/Adila@2c32bd6
fani-lab/Adila@62700a9

[mahdis-saeedi]

Hi @mahdis-saeedi I fixed the issue. please update your opentf with recursive submodule option. Now, the adila works with both dense, and sparse pred files up to topK==k_max first-stage cutoff. Let me know if there is an issue.

fani-lab/Adila@2c32bd6

Thank you @hosseinfani . I will do and update you here.

[mahdis-saeedi]

Hi @hosseinfani,
We are facing this memory error. Can we decrease the 1000 cutoff? Do you want me to study this hyperparameter to find the limit preventing the memory issue? Thank you.

[hosseinfani]

Hi @mahdis-saeedi
Please reduce the k_max to 100 and let me know the result. Tnx.

[mahdis-saeedi]

Hi @hosseinfani ,
k_max=100 is a good fit for memory. Now we have a naming issue. The name of our previous folders doesn't include the fair options and we face the file not found error.

[hosseinfani]

Hi @mahdis-saeedi
I believe the issue is not about the naming. the exception is raised for assigning to a sparse tensor. I'll look into it and let you know.