[Docs/Website] Decide on requirements for `extra_gated_fields` field and implications for what it means

[deanwampler]

Is there an existing issue for this?

• I have searched the existing issues

Description

This is a follow up to #173.

The extra_gated_fields field is poorly documented and it has implications for whether or not we can include the dataset in our catalog.

My understanding is that this field is used to prevent access to a dataset and its metadata unless and until a terms of use form has been signed by the user. On the face of it, this means the dataset is not truly "open" in our usual sense and therefore should be excluded, by default.

However, there is a grey area where a dataset is mostly free to use, but the creator of it wants to prevent some uses (e.g., commercial purposes) or ensure the user is aware of limitations.

Therefore, we should decide how to handle these cases, whether or not we think they should be excluded (the easiest thing to do) or included in some way.

Affected Areas

• Documentation files (docs/)
• Website content (website/)

Proposed Changes

1. Decide if any restrictions are acceptable.
2. If they are, decide which possible restrictions are acceptable.
3. Determine how to represent those datasets in our catalog.

Additional Context

No response

[deanwampler]

See also #70

[AF-IBM]

Wouldn't these restrictions impact the licensing as well? Is it permissible for a dataset licensed under Apache 2.0 to include extra_gated_fields that prevent access before a separate agreement is signed? If this is the case, could we include it in the catalog but flag it as ambiguous, perhaps with a color-coded indicator?
I think it would be a good addition to flag that this field creates a conflict, making some licenses effectively ambiguous or even invalid.

[jolson-ibm]

Here is how this field is populated in our OTDI scoring data set:

<style> </style>

Count	Extra Gated Fields
435585	None
10	{'Name': 'text', 'Company/Organization': 'text', 'Country': 'text', 'E-Mail': 'text'}
9
9	'task_ids'],[],['Data set info missing'
8	{'Name': 'text', 'Email': 'text'}
4	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Country': 'text'}
4	{'Company/Organization': 'text', 'Country': 'country'}
4	{'Affiliation': 'text', 'Country': 'country', 'I want to use this model for': {'type': 'select', 'options': ['Research', 'Commercial', {'label': 'Other', 'value': 'other'}]}, 'Job title': {'type': 'select', 'options': ['Student', 'Research graduate', 'AI researcher', 'AI developer/engineer', 'Cybersecurity researcher', 'Reporter', 'Other']}, 'geo': 'ip_location'}
4	{'First Name': 'text', 'Last Name': 'text', 'Date of birth': 'date_picker', 'Country': 'country', 'Affiliation': 'text', 'Job title': {'type': 'select', 'options': ['Student', 'Research Graduate', 'AI researcher', 'AI developer/engineer', 'Reporter', 'Other']}, 'geo': 'ip_location', 'By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the Meta Privacy Policy': 'checkbox'}
4	{'I hereby confirm that I have registered on the original Common Voice page and agree to not attempt to determine the identity of speakers in the Common Voice dataset': 'checkbox', 'I hereby confirm that I have accepted the terms of usages on GigaSpeech page': 'checkbox', 'I hereby confirm that I have accepted the terms of usages on SPGISpeech page': 'checkbox'}
3	{'First Name': 'text', 'Last Name': 'text', 'Country': 'country', 'Affiliation': 'text'}
3	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Position': 'text', 'Your Supervisor/manager/director': 'text', 'I agree to the Terms of Access': 'checkbox', 'I agree to use this dataset for non-commercial use ONLY': 'checkbox'}
3	{'Name': 'text', 'Email': 'text', 'Organization': 'text', 'Address': 'text', 'I hereby confirm that I have requested access via the Google Form provided above': 'checkbox', 'I accept the terms of access': 'checkbox'}
2	{'Consent': 'checkbox', 'Purpose of use': {'type': 'select', 'options': ['Research', 'Educational', {'label': 'Other', 'value': 'other'}]}}
2	{'I accept these terms': 'checkbox'}
2	{'Company': 'text', 'Country': 'country', 'Specific date': 'date_picker', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this dataset for non-commercial use ONLY': 'checkbox'}
2	{'I agree to not reshare the GAIA submissions set according to the above conditions': 'checkbox'}
2	{'Full Real Name': 'text', 'Institutional email': 'text', 'Affiliation': 'text', 'Country': 'country', 'My Background': {'type': 'select', 'options': ['Significantly More Knowledge in AI than Radiotherapy', 'Significantly More Knowledge in Radiotherapy than AI', 'Little Knowledge in either AI or Radiotherapy', 'Knowledge in both AI and Radiotherapy']}, 'I agree to use this dataset for non-commercial use ONLY': 'checkbox', 'I agree to register the GDP-HMM challenge if I use the data before the end of the challenge': 'checkbox', 'I understand that my access to the data may be declined/removed if I have not correctly finished above steps': 'checkbox', 'I agree to read the README file and follow citation terms before downloading data': 'checkbox'}
2	{'Full name': 'text', 'Email': 'text', 'Institution': 'text', 'I accept the Terms of Usage': 'checkbox'}
2	{'Name': 'text', 'Affiliation': 'text', 'Country': 'text', 'I agree to use this model for non-commercial use ONLY': 'checkbox'}
2	{'Email': 'text', 'I have read the License and agree with its terms': 'checkbox'}
2	{'First Name': 'text', 'Last Name': 'text', 'Email': 'text', 'Country': 'country', 'Affiliation': 'text', 'Phone': 'text', 'Job title': {'type': 'select', 'options': ['Student', 'Research Graduate', 'AI researcher', 'AI developer/engineer', 'Reporter', 'Other']}, 'Research interest': 'text', 'geo': 'ip_location', 'By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the AgiBot Privacy Policy': 'checkbox'}
2	{'I accept the above license agreement, and will use the Model complying with the set of use restrictions and sharing requirements': 'checkbox'}
2	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Position': 'text', 'Your Supervisor/manager/director': 'text', 'I agree to the Terms of Access': 'checkbox'}
2	{'Company': 'text', 'Country': 'text', 'I agree to use this dataset for non-commercial use ONLY': 'checkbox'}
1	'tags'],[{'name': 'date'
1	{'Company': 'text', 'Country': 'country', 'Specific date': 'date_picker', 'I want to use this model for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this model for non-commercial use ONLY': 'checkbox'}
1	{'First Name': 'text', 'Last Name': 'text', 'Date of birth': 'date_picker', 'Country': 'country', 'Affiliation': 'text', 'I accept the terms and conditions': 'checkbox', 'geo': 'ip_location'}
1	{'Name': 'text', 'Contact mail': 'text', 'Country': 'country', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Personal purpose', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this model for non-commercial use ONLY': 'checkbox'}
1	{'Company/Institution': 'text', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}}
1	{'Competition Agreement': 'checkbox'}
1	'Invalid license'
1	{'Company': 'text', 'Country': 'country'}
1	{'By clicking Submit below I accept the terms of the license and Acceptable Use policy': 'checkbox'}
1	{'Name': 'text', 'Affilation': 'text', 'Company': 'text', 'Country': 'country', 'Specific date': 'date_picker', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this dataset for non-commercial use ONLY': 'checkbox'}
1	{'Full name (first and last)': 'text', 'Current affiliation (no abbreviations)': 'text', 'Type of Affiliation': {'type': 'select', 'options': ['Academia', 'Industry', {'label': 'Other', 'value': 'other'}]}, 'Current and official institutional email (this must match your primary email in your Hugging Face account, @gmail/@hotmail/@qq email domains will be denied)': 'text', 'Please explain your intended research use': 'text', 'I agree to all terms outlined above': 'checkbox', 'I agree to use this model for non-commercial, academic purposes only': 'checkbox', 'I agree not to distribute the model, if another user within your organization wishes to use Patho-Bench data, they must register as an individual user': 'checkbox'}
1	{'Name': 'text', 'Institution': 'text', 'Email': 'text', 'I have read and agree with Terms and Conditions for using the RadGenome Chest CT and CT-RATE dataset': 'checkbox'}
1	{'First Name': 'text', 'Last Name': 'text', 'Country': 'country', 'Affiliation': 'text', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this dataset for non-commercial use ONLY': 'checkbox'}
1	{'tell me an interesting fact': 'text', 'How much do you love money': {'type': 'select', 'options': ['High', 'Medium', {'label': 'Other', 'value': 'other'}]}, 'I will unleash value with this dataset': 'checkbox'}
1	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text'}
1	{'Name': 'text', 'Country': 'country', 'Affiliation': 'text', 'Job title': {'type': 'select', 'options': ['Undergraduate Student', 'Graduate Student', 'AI researcher/developer/engineer', 'Other']}, 'By clicking Submit below I accept the terms and conditions and acknowledge that the information I provide is only for legal purposes and will not be shared with any other entity': 'checkbox'}
1	{'I have read and agree to the terms': 'checkbox'}
1	{'First Name': 'text', 'Last Name': 'text', 'Institution': 'text', 'Intended Use': 'text', 'I agree to use this dataset for non-commercial research ONLY': 'checkbox'}
1	{'First and Last Name': 'text', 'Country': 'country', 'Accept?': 'checkbox'}
1	{'I agree to attribute the creator of this repository': 'checkbox'}
1	{'Tell us what you will use the dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'Other'}]}, "By accessing this dataset, you agree you are 18 years old, and understand the risks of using the dataset, including the sensitive nature of it's content, and are solely responsible for determining the appropriateness of its usage based on your jurisdiction's legal requirements, and you agree NOT to use this dataset for any illegal purposes": 'checkbox'}
1	'license'
1	{'Name': 'text', 'Institution': 'text', 'Email': 'text', 'I have read and agree with Terms and Conditions for using the dataset': 'checkbox'}
1	{'By clicking Submit below, I accept the terms of the conditions and license': 'checkbox'}
1	{'Name': 'text', 'Company': 'text', 'Country': 'text', 'Email': 'text', 'I agree to the terms and conditions of the FUNSD+ license': 'checkbox'}
1	{'Did you register on Wiro AI web site?': 'checkbox', 'Your registered e-mail on Wiro AI': 'text', 'Which tool/model of ours did you try on Wiro AI?': 'text', 'What is your feedback on the tool/model or Wiro AI': 'text', 'Institution/Individiual': 'text'}
1	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Country': 'text', 'Usecase': 'text', 'I have explicitly checked that downloading occiglot-fineweb is legal in my jurisdiction, in the country/region where I am located right now, and for the use case that I have described above, I have also read and accepted the relevant Terms of Use': 'checkbox'}
1	'tags'
1	{'Full Name': 'text', 'Affiliation (Organization/University)': 'text', 'Designation/Status in Your Organization': 'text', 'Country': 'country', 'I want to use this dataset for (please provide the reason(s))': 'text', 'ReLU dataset is free for research use but NOT for commercial use; do you agree if you are provided with the ReLU dataset, you will NOT use for any commercial purposes? Also do you agree that you will not be sharing this dataset further or uploading it anywhere else on the internet': 'checkbox', 'DISCLAIMER The dataset is released for research purposes only and authors do not take any responsibility for any damage or loss arising due to usage of data or any system/model developed using the dataset': 'checkbox'}
1	{'Your full name': 'text', 'Organization or entity you are affiliated with': 'text', 'State or country you are located in': 'text', 'Contact email': 'text', 'Please describe your intended use of the low risk artifact(s)': 'text', 'I understand that this dataset is a research artifact that may contain or produce unfiltered, toxic, or harmful material': 'checkbox', 'I agree to use this dataset for research purposes in accordance with the AI2 Responsible Use Guidelines': 'checkbox', 'I agree that AI2 may use my information as described in the Privacy Policy': 'checkbox', 'I certify that the information I have provided is true and accurate': 'checkbox'}
1	{'You will use the Datasets only for non-commercial research and educational purposes.': {'type': 'select', 'options': [True, False]}, 'You will NOT distribute the Datasets or any parts thereof.': {'type': 'select', 'options': [True, False]}, 'Carnegie Mellon University makes no representations or warranties regarding the datasets, including but not limited to warranties of non-infringement or fitness for a particular purpose.': {'type': 'select', 'options': [True, False]}, 'You accept full responsibility for your use of the datasets and shall defend and indemnify Carnegie Mellon University, including its employees, officers and agents, against any and all claims arising from your use of the datasets, including but not limited to your use of any copyrighted videos or images that you may create from the datasets.': {'type': 'select', 'options': [True, False]}, 'You will treat people appearing in this data with respect and dignity.': {'type': 'select', 'options': [True, False]}, 'This data comes with no warranty or guarantee of any kind, and you accept full liability.': {'type': 'select', 'options': [True, False]}}
1	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Country': 'text', 'Usecase': 'text', 'I have explicitly check with my jurisdiction and I confirm that downloading CulturaY is legal in the country/region where I am located right now, and for the use case that I have described above': 'checkbox', 'You agree to not attempt to determine the identity of individuals in this dataset': 'checkbox'}
1	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Country': 'text', 'Usecase': 'text', 'I have explicitly check with my jurisdiction and I confirm that downloading OSCAR 2301 is legal in the country/region where I am located right now, and for the use case that I have described above': 'checkbox'}
1	{'Name': 'text', 'Email': 'text', 'Affiliation': 'text', 'Country': 'text', 'Usecase': 'text', 'I have explicitly check with my jurisdiction and I confirm that downloading CulturaX is legal in the country/region where I am located right now, and for the use case that I have described above': 'checkbox', 'You agree to not attempt to determine the identity of individuals in this dataset': 'checkbox'}
1	{'Institution': 'text', 'Name': 'text', 'Country': 'country', 'I want to use this dataset for': {'type': 'select', 'options': ['Research']}, 'I agree to use this dataset solely for research purposes': 'checkbox', 'I will not use this dataset in any way that infringes upon the rights of the copyright holders of the original videos, and strictly prohibit its use for any commercial purposes': 'checkbox'}
1	{'Reason for access': 'text', 'Organization (Optional)': 'text', 'Country': 'text', 'I agree to use this dataset for non-commercial use ONLY': 'checkbox', 'I agree to use not share or redistribute this dataset': 'checkbox'}
1	{'Full name': 'text', 'Affiliation / Company / Institution': 'text', 'Email address': 'text', 'Country of residence': 'country', 'Intended use': {'type': 'select', 'options': [{'label': 'Research', 'value': 'research'}, {'label': 'Education', 'value': 'education'}, {'label': 'Other (please specify in the request message)', 'value': 'other'}]}, 'Expected start date': 'date_picker', 'I agree to use this dataset for non-commercial purposes ONLY': 'checkbox', 'I agree to delete local copies if my access is later revoked': 'checkbox'}
1	{'I have agreed to the original ImageNet dataset': 'checkbox'}
1	{'Company': 'text', 'Country': 'text', 'Email': 'text', 'I agree to use this model for non-commercial use ONLY': 'checkbox'}
1	{'Company': 'text', 'Country': 'text', 'I agree to use this model for non-commercial use ONLY': 'checkbox'}
1	{'Name': 'text', 'Affiliation': 'text', 'Country': 'text', 'I agree to accept the license': 'checkbox'}
1	{'Institute': 'text', 'I want to use this dataset for': {'type': 'select', 'options': ['Research', 'Education', {'label': 'Other', 'value': 'other'}]}, 'I agree to use this dataset for non-commerical use ONLY': 'checkbox'}
1	'language'
1	{'I confirm that I am over the age of 18': 'checkbox'}
1	{'I will use this dataset in a way that does not hinder the ability of artists to make a living from their work': 'checkbox', 'I acknowledge that the content contained within this dataset is the intellectual property of the artists who created it': 'checkbox', 'If I should wish to use this dataset for any commercial purposes, it is my responsibility to obtain the appropriate permissions from the copyright holders': 'checkbox'}