polar-semeval.github.io/faq.html at main · Polar-SemEval/polar-semeval.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
<!DOCTYPE HTML>
<html>
	<head>
		<title>FAQs - POLAR @ SemEval-2026</title>
		<meta charset="utf-8" />
		<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
		<link rel="stylesheet" href="assets/css/main.css" />
		<style>
			.centered-image {
				display: block; /* Makes the image a block-level element */
				margin: 0 auto; /* Centers the block-level image horizontally */
				}
		</style>
	</head>
	<body class="is-preload">

		<!-- Wrapper -->
			<div id="wrapper">

				<!-- Main -->
					<div id="main">
						<div class="inner">

							<!-- Header -->
								<header id="header">
									<a href="/" class="logo"><strong>POLAR</strong> @ SemEval-2026</a>
									<ul class="icons">
										<li><a href="https://discord.gg/K5uhEF2jPN" class="icon brands fa-discord"><span class="label">Discord</span></a></li>
										<li><a href="https://groups.google.com/g/polarization-semeval-2026-participants" class="icon solid fa-envelope"><span class="label">Google Group</span></a></li>
										<li><a href="https://github.com/Polar-SemEval/trial-data" class="icon brands fa-github"><span class="label">GitHub</span></a></li>
									</ul>
								</header>

							<!-- Content -->
							<section>
								<header class="main">
								  <h1>Frequently Asked Questions</h1>
								</header>

								<style>
									/* ===== Accordion (Details) Styling for POLAR Theme ===== */
									details {
									  background: #fafafa;
									  border-radius: 8px;
									  margin-bottom: 10px;
									  padding: 0.8em 1em;
									  border: 1px solid #e0e0e0;
									  transition: all 0.3s ease;
									}

									details:hover {
									  background: #fff;
									  border-color: #e57373;
									  box-shadow: 0 1px 4px rgba(229, 115, 115, 0.1);
									}

									summary {
									  font-weight: 600;
									  color: #3d4042;
									  cursor: pointer;
									  list-style: none;
									  position: relative;
									  padding-right: 1.4em;
									}

									summary::marker {
									  display: none;
									}

									/* Custom arrow */
									summary::after {
									  content: "›";
									  position: absolute;
									  right: 0;
									  transform: rotate(90deg);
									  transition: transform 0.3s ease;
									  color: #e57373;
									  font-weight: bold;
									}

									details[open] summary::after {
									  transform: rotate(270deg);
									}

									/* Smooth accordion animation */
									details > *:not(summary) {
									  overflow: hidden;
									  opacity: 0;
									  max-height: 0;
									  transition: all 0.35s ease;
									}

									details[open] > *:not(summary) {
									  opacity: 1;
									  max-height: 1000px;
									  margin-top: 0.6em;
									}

									h2 {
									  color: #3d4042;
									  margin-top: 2em;
									  font-size: 1.4rem;
									  border-bottom: 2px solid #e57373;
									  display: inline-block;
									  padding-bottom: 0.2em;
									}

									hr.major {
									  margin: 2em 0;
									  border: none;
									  border-top: 2px solid #e57373;
									  width: 60%;
									}

									a {
									  color: #e57373;
									  text-decoration: underline;
									}

									a:hover {
									  color: #c94f4f;
									}
								  </style>

								<!-- ======================= PARTICIPATION QUESTIONS ======================= -->
								<h2>Participation Questions</h2>

								<details>
								  <summary>Do I have to participate in all three subtasks?</summary>
								  <p>No. You may choose to participate in one or more subtasks.</p>
								</details>

								<details>
								  <summary>Do I have to work on all languages for a given subtask?</summary>
								  <p>No. You can choose one or more languages.</p>
								</details>

								<details>
								  <summary>Am I allowed to participate in more than one language simultaneously?</summary>
								  <p>Yes. You may work on multiple languages at the same time.</p>
								</details>

								<details>
								  <summary>When will all datasets for the 22 languages be released?</summary>
								  <p>We will announce the release of each dataset as it becomes available. Currently, datasets for <strong>9 languages</strong> have been released, and the remaining ones will follow soon.</p>
								</details>

								<details>
								  <summary>How will you verify my submitted model?</summary>
								  <p>To be included in the final team rankings, participants must submit a <strong>system description paper</strong> detailing their approach and methodology. This ensures transparency and scientific integrity.</p>
								</details>

								<details>
								  <summary>Will I be included in the final ranking if I do not submit a system description paper?</summary>
								  <p>No. A system description paper is mandatory for inclusion in the final ranking.</p>
								</details>

								<details>
								  <summary>I have never written a system description paper. How can I write one?</summary>
								  <p>We will provide an online tutorial and resources to guide you through the process.</p>
								</details>

								<details>
								  <summary>Can we work as a team, or must everyone participate individually?</summary>
								  <p>Each participant can be part of <strong>only one team</strong>. If you believe there are special circumstances that require you to join more than one team, please email us before the evaluation period begins.</p>
								  <p>You may work either as part of a team or independently — but not both.</p>
								</details>

								<details>
								  <summary>I want to add someone to my team but can’t find how to do that on Codabench. How can I add members?</summary>
								  <p>To add members to your team on Codabench:</p>
								  <ul>
									<li>Click your account name in the upper-right corner and select your organization.</li>
									<li>On the organization page, click <strong>“Edit Organization”</strong> in the upper-left corner.</li>
									<li>Scroll to the bottom and click the green <strong>“Invite Members”</strong> button to add teammates.</li>
								  </ul>
								</details>

								<hr class="major" />

								<!-- ======================= DATA AND METHODOLOGY ======================= -->
								<h2>Data and Methodology</h2>

								<details>
								  <summary>When will you release the gold labels?</summary>
								  <ul>
									<li><strong>Dev set:</strong> Released at the start of the evaluation phase.</li>
									<li><strong>Test set(s):</strong> Released after the competition ends.</li>
								  </ul>
								</details>

								<details>
								  <summary>Can I use large language models (LLMs) in the subtasks?</summary>
								  <p>Yes.</p>
								</details>

								<details>
								  <summary>Are there restrictions on the types or sizes of LLMs or on how we use them for data augmentation?</summary>
								  <p>No. You may use any type or size of model — open-source or closed-source. You may also generate or use synthetic data for augmentation.</p>
								</details>

								<details>
								  <summary>Can I use additional datasets (e.g., publicly available ones)?</summary>
								  <p>Yes, but you must properly cite all external datasets in your system description paper.</p>
								</details>

								<details>
								  <summary>How was the data collected?</summary>
								  <ul>
									<li>Data sources include news websites, Reddit, blogs, Bluesky, and regional forums, covering topics such as elections, conflicts, gender rights, and migration.</li>
									<li>Each language dataset contains approximately 3,000–5,000 annotated instances. Annotation platforms include Label Studio, Prolific, Potato, and Mechanical Turk.</li>
								  </ul>
								</details>

								<details>
								  <summary>How was the data annotated? Were LLMs used?</summary>
								  <p>No. All data was annotated by at least three native speakers per instance — no LLMs were used.</p>
								</details>

								<details>
								  <summary>I cannot find the link to download the files under the “Files” tab on Codabench. How can I access the datasets?</summary>
								  <p>You can access all dataset download links through this document:
									<a href="https://docs.google.com/document/d/1pffhec5teDvXrnfAg-OBJvXZ0B45--GQRfhoxeGfOyc/edit?tab=t.0" target="_blank">
									  Google Drive Dataset Access Guide
									</a>.
								  </p>
								</details>

								<details>
								  <summary>What is the “starter pack” uploaded in the Files tab? Should we use it?</summary>
								  <p>The starter pack is an example to help you get started with the data. You may use it as a reference or build your own approach from scratch.</p>
								</details>

								<details>
								  <summary>I’ve downloaded the starter and public_data files. The public_data file includes multiple subtasks — what’s the difference, and do I need all of them?</summary>
								  <p>The public data contain all subtasks, so you only need to download them once even if you plan to participate in multiple subtasks.</p>
								  <p>The starter pack includes a sample notebook that trains a small BERT model for demonstration. You are free to use your own code and models instead.</p>
								</details>

								<hr class="major" />

								<!-- ======================= PUBLICATION AND CONFERENCE ======================= -->
								<h2>Publication and Conference</h2>

								<details>
								  <summary>Do I need to pay conference registration fees or attend SemEval for my paper to be published?</summary>
								  <p>No. Publication does not require attendance. If you do not attend the workshop, you do not need to pay any fees. If you wish to attend, standard registration fees apply.</p>
								</details>

								<details>
								  <summary>Our system did not perform well. Should we still write a system description paper?</summary>
								  <p>Yes. We strongly encourage all teams to submit a paper. Negative results are equally valuable and contribute to advancing research on polarization.</p>
								</details>

								<hr class="major" />

								<!-- ======================= TECHNICAL ISSUES ON CODABENCH ======================= -->
								<h2>Technical Issues on Codabench</h2>

								<details>
								  <summary>It’s been several hours since I submitted my file, but the status still shows “running.”</summary>
								  <p>Codabench may occasionally experience delays in processing submissions. Please refresh the page after a few minutes. If the issue persists, try resubmitting your file.</p>
								</details>

								<details>
								  <summary>My submission failed, but I’m not sure why. How can I check the error?</summary>
								  <p>Click on your submission in the leaderboard or the “My Submissions” tab. You can view the error log under the <strong>“Output”</strong> section to see detailed feedback about the failure.</p>
								</details>

								<details>
								  <summary>The platform shows “Internal Error.” What should I do?</summary>
								  <p>This usually happens if your submission format doesn’t match the expected structure. Verify that your file follows the required format described on the evaluation page, then re-submit.</p>
								</details>

								<details>
								  <summary>I uploaded the wrong file. Can I delete or replace my submission?</summary>
								  <p>You cannot delete a submission, but you can simply re-submit a new file. Only the latest valid submission will be considered for the leaderboard during evaluation.</p>
								</details>

								<details>
								  <summary>I cannot see the “Submit” button on Codabench. How can I upload my file?</summary>
								  <p>Ensure you are logged in and registered for the competition. The “Submit” button appears under the corresponding subtask page once registration is complete.</p>
								</details>

								<details>
								  <summary>My team members can’t see our submissions. Why?</summary>
								  <p>Submissions are only visible within your team. Make sure all members are added under the same organization on Codabench using the <strong>“Invite Members”</strong> option.</p>
								</details>
							  </section>


                            <!-- Footer -->
                                <footer>
                                    <div class="inner">
                                        <p>&copy; 2026 POLAR Shared Task. All rights reserved. Adapted from Editorial Template by <a href="https://html5up.net">HTML5 UP</a></p>
                                    </div>
                                </footer>

						</div>
					</div>

				<!-- Sidebar -->
					<div id="sidebar">
						<div class="inner">

							<!-- Menu -->
								<nav id="menu">
									<img class="centered-image" src="assets/POLAR.png" alt="POLAR Task Banner" height="200" width="200"/>

									<header class="major">
										<h2>Menu</h2>
									</header>
									<ul>
										<li><a href="/">Homepage</a></li>
										<li><a href="tasks.html">Subtasks</a></li>
										<li><a href="participation.html">How to Participate</a></li>
										<li><a href="faq.html">FAQs</a></li>
										<li><a href="dates.html">Important Dates</a></li>
										<li><a href="resources.html">Resources</a></li>
										<li><a href="communication.html">Communication</a></li>
										<li><a href="organizers.html">Organizers</a></li>
										<li><a href="announcements/">Announcements</a></li>
									</ul>
								</nav>

						</div>
					</div>

			</div>

		<!-- Scripts -->
			<script src="assets/js/jquery.min.js"></script>
			<script src="assets/js/browser.min.js"></script>
			<script src="assets/js/breakpoints.min.js"></script>
			<script src="assets/js/util.js"></script>
			<script src="assets/js/main.js"></script>

	</body>
</html>