-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathSPRFormat.html
More file actions
643 lines (630 loc) · 32.8 KB
/
SPRFormat.html
File metadata and controls
643 lines (630 loc) · 32.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>BCS 152 Tutorial</title>
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="tutorial.css">
</head>
<body class="bckgrnd">
<nav id="nav" class="navbar navbar-default navbar-fixed-top"></nav>
<div class="jumbotron spr top">
<h1>Self Paced Reading Example: Preparing your Experiment for CrowdExp</h1>
<p>
[<a href="#format" class="bldspr">Format of stimulus file</a>]
[<a href="#identify" class="bldspr">Identifying sentence regions in your stimuli</a>]
[<a href="#validate" class="bldspr">Validating your stimulus file</a>]
</p>
<p>
CrowdExp will create a Latin Square Design for
you, create the Latin Square Designed lists, and randomize the stimuli
order. How's that for good news?!
</p>
<p id="format">CrowdExp also helps you to create the stimulus file. Use the
<a href="https://www.hlp.rochester.edu/CrowdExp" target = "_blank" class="bldxtrnl">CrowdExp</a>
interface to specify your design, number
of critical items, and numbers of fillers. After your design has been
approved, CrowdExp will then let you download a template stimulus
file (an Excel file) that contains many of the necessary variables.
</p>
</div>
<div class="jumbotron spr">
<h2>Format of stimulus file</h2>
<p>Below are some representative rows from the example file: </p>
<table class="format1">
<tbody>
<tr>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>Experiment</strong></p>
</td>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>ItemName</strong></p>
</td>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>Condition</strong></p>
</td>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>Sentence</strong></p>
</td>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>Question</strong></p>
</td>
<td class="format2">
<p class="line891" style="margin: 0px; padding: 0px;"><strong>Answer</strong></p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-140"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">1</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">LOW_PRED.HIGH_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">I saw the {apple}@TARGET {on the}@SPILLOVER table</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they see the apple?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Y</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-141"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">1</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">LOW_PRED.LOW_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">I saw the {kiwi}@TARGET {on the}@SPILLOVER table</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they see the kiwi?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Y</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-142"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">1</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">HIGH_PRED.HIGH_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">I ate the {apple}@TARGET {on the}@SPILLOVER table</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they eat the apple?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Y</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-143"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">1</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">HIGH_PRED.LOW_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">I ate the {kiwi}@TARGET {on the}@SPILLOVER table.</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they eat the kiwi?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Y</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-144"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">2</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">HIGH_PRED.LOW_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">She typed the {column}@TARGET {for the}@SPILLOVER September issue.</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they read the column?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">N</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-145"></span>
<p class="line862" style="margin: 0px; padding: 0px;">critical</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">2</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">LOW_PRED.HIGH_FREQ</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">She did the {paper}@TARGET {for the}@SPILLOVER September issue.</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did they read the column?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">N</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-146"></span>
<p class="line862" style="margin: 0px; padding: 0px;">...</p>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-147"></span>
<p class="line862" style="margin: 0px; padding: 0px;">filler</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">41</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">-</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">The parcel
arrived too late to be of any use.</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Did the
package arrive on time?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">N</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-148"></span>
<p class="line862" style="margin: 0px; padding: 0px;">filler</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">42</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">-</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">The
espresso machine was broken for months before finally being
fixed.</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Was the
espresso machine fixed?</p>
</td>
<td class="format2">
<p class="line862" style="margin: 0px; padding: 0px;">Y</p>
</td>
</tr>
<tr>
<td class="format2">
<span class="anchor" id="line-149"></span>
<p class="line862" style="margin: 0px; padding: 0px;">...</p>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
<td class="format2"><br>
</td>
</tr>
</tbody>
</table>
<br class="Apple-interchange-newline">
<p>There are six (6) required columns:
<b>Experiment</b>,
<b>ItemName</b>,
<b>Condition</b>,
<b>Sentence</b>,
<b>Question</b>, and
<b>Answer</b>.
</p>
<p>
<ul>
<li>
<b>Experiment</b> distinguishes between critical items and fillers.
You can also distinguish several mini-experiments that are embedded
within the same experiment. For your first experiment, however, we
highly recommend you keep it simple. Just use "critical" for all your
critical items and "filler" for all your filler items.
</li>
<li>
<b>ItemName</b> is to identify an individual item within an
experiment. This should be a number. For critical items this will be the same
across all conditions. In the example above there are four rows
for item 1, one for each of the 2x2 = 4 conditions in the experiment.
You <em>must have as many rows per critical item as conditions</em>.
ItemNames <em>must be unique</em>. The easiest solution to this is
to number your items from 1 to N (where N is the number of items in
your experiment). This will allow CrowdExp to automatically create Latin Square Designed
lists for you. Filler items, on the other hand, are identical across
lists. So you only need one row per filler.
</li>
<li>
<b>Condition</b> is to identify the condition of a critical item.
Only critical items have conditions.
</li>
</ul>
<ul>
<li>
<em>For fillers</em>, Conditionnames must be be "-".
</li>
<li>
<em>For critical items</em>, Condition names should not contain any SPACEs
or special symbols, except for "." like in the example above.
The condition column needs to follow a specific format. Each
manipulation needs to be separated by a ".", and each condition
within a manipulation needs a unique label. In our 2x2 example
design, there are two manipulations (contextual predictability and
word frequency). So, the Condition names have one ".", separating
the value for contextual predictability manipulation from the value
for the word frequency manipulation. This gives us the four possible
Condition names, shown in the table.
</li>
<li>
You can name your conditions what you like but, it is useful to use
descriptive names. In our 2x2 example design, we have chosen
LOW_PRED vs. HIGH_PRED for the contextual predictability
manipulation and LOW_FREQ vs. HIGH_FREQ for the word frequency
manipulation.
</li>
<li>
CrowdExp will be creating the Latin Square design and randomize the
item and filler order for you so the order you put items
or fillers in does not matter. But remember that we are using a Latin
Square Design so participants will only see each item in one of its
possible conditions (e.g. the first sentence of item 1, the second
sentence of item 2, etc.). All participants will see all the fillers.
</li>
</ul>
<ul>
<li>
<b>Sentence</b> is the stimulus sentence that participants will read.
Avoid special characters other than "," and "." This is also the
field in which you will have to mark up the
<a href="Glossary.html#S" target="_blank" class="red" data-toggle="tooltip" title="">sentence regions</a>
of interest. We will talk more about <em>how</em> to do that below.
CrowdExp will collect
<a href="Glossary.html#W" target="_blank" class="red" data-toggle="tooltip" title="A specific type of self-paced reading task. In a word-by-word reading task, each word in a sentence is presented one at a time. Participants read each word and then make a response/answer to see the next word in the sentence.">
word-by-word reading times</a>.
These reading times are typically the primary outcome variable of interest.
(Note: If you're interested in running experiments on language with other
writing systems, let us know. CrowdExp is unicode enabled, providing
support for other writing systems.
</li>
<li>
<b>Question</b> is the comprehension question to ensure that
participants are reading and understanding the sentence.
</li>
<li id="identify">
<b>Answer</b> is the correct answer for the comprehension question.
In the example above all the answers for Item 1 are N.
CrowdExp will record both the reaction time and theanswer provided
by the participant, providing two additional outcome variables for
each sentence.
</li>
</ul>
</div>
<div class="jumbotron spr">
<h2>Identifying sentence regions in your stimuli</h2>
<p>
To analyze the reading data, the word-by-word reading times are
often aggregated into
<a href="Glossary.html#S" target="_blank" class="red" data-toggle="tooltip" title="">sentence regions</a>,
based on the predictions
for the experiment. It is thus <em>very important</em> to mark up
the sentence regions of interest in the <b>Sentence</b> field of
your stimulus file.
</p>
<p>
This is done with the {...}@REGIONNAME syntax, as shown in the table
above. Simply put the word or words that you want to later analyze
as one sentence region into one set of curly parentheses {...}. Don't
worry: the words will still be displayed word-by-word to the participant
and participants also won't see the {...}@REGIONAME mark-up. CrowdExp
will remove this information before displaying the sentences.
</p>
<h3>An example</h3>
<p>
In the example table above, we only marked the TARGET region and
a SPILLOVER region (the two following words). The TARGET region is
the word that we hypothesize to be read more or less slowly
depending on its frequency and contextual predictability. The SPILLOVER
region are the two words following the TARGET word. We marked them
since processing difficulty often '<a href="Glossary.html#S" target="_blank" class="red" data-toggle="tooltip" title="Often in self-paced reading studies differences in reading times can happen both on the sentence regions that are experimentally manipulated as well as several words later.">spills over</a>'
on to subsequent words.
</p>
<p>
This shows that we don't have to mark all regions in a sentence. It
is, however, often a good idea to also mark some other regions, even
if there are not of primary interest. That's because we want to make
sure that those regions for which we do <em>not</em> predict any
differences by condition indeed do not exhibit any differences. The
following is an example of an alternative region coding that we
could use for all items in our example experiment.</p>
<ul>
<li>{I}@SUBJECT {saw}@VERB the {apple}@TARGET {on the}@SPILLOVER {table.}@REST</li>
<li>{I}@SUBJECT {saw}@VERB the {kiwi}@TARGET {on the}@SPILLOVER {table.}@REST</li>
<li>{I}@SUBJECT {ate}@VERB the {apple}@TARGET {on the}@SPILLOVER {table.}@REST</li>
<li>{I}@SUBJECT {ate}@VERB the {kiwi}@TARGET {on the}@SPILLOVER {table.}@REST</li>
</ul>
<h3>Format requirements</h3>
<p>
In most designs, <em>all critical items should have the same regions</em>
(filler items do not require sentence regions).
Region names also typically (though not necessarily) occur in the
same order across all critical items, regardless of the condition.
It is, however, possible that your design
requires you to have some sentence regions that only occur in a
subset of the conditions of your experiment.
</p>
<p>
Regions should be indicated using the following syntax {bli bla
blub}@Name, e.g.:
</p>
<ul>
<li>{word}@regionname</li>
<li>{several words here}@anotherregionname</li>
<li>{This sentence}@subject {has}@verb {multiple regions.}@object</li>
</ul>
<p>
Make sure to always include punctuation (e.g., "!", "?", ",", ".")
right after the words. <em>Never have an open or closing parenthesis
intervene between a word and punctuation</em>. That is, use
</p>
<blockquote>{bli bla blub,}@NAME</blockquote>
<p>
rather than
</p>
<blockquote>{bli bla blub}@NAME,</blockquote>
<p>
which would result in the region being named "NAME,". Also avoid
</p>
<blockquote>{bli bla blub}@NAME ,</blockquote>
<p id="validate">
which would result in the intended region name, but also would
result in the comma being displayed only after an <em>additional
</em> SPACE-bar press after "blub".
</p>
</div>
<div class="jumbotron spr">
<h2>Validating your stimulus file</h2>
<p>
CrowdExp will automatically parse your stimulus file for errors. CrowdExp
won't find all types of errors though, so be sure to try your best to avoid
them. We recap some of the more common error here and recommend that
you check these points before uploading your stimulus file to CrowdExp.
</p>
<ul>
<li>
All <b>ItemName</b>s have to be unique across items. It is best
not to change the ItemNames provided by CrowdExp.
</li>
<li>
<b>Condition</b>s:
<ul>
<li>
Should only be upper and lowercase letters and
periods (see example above). <b> Do not include SPACEs, numbers or
other characters.</b> Generally, it's best not to change the
Condition names that were auto-generated by CrowdExp.
</li>
<li>
Make sure that the condition names are sensible and sufficiently
transparent so that you can later understand the output of the
analysis. Take a note in your lab notebook about what the different
condition names mean.
</li>
</ul>
</li>
<li>
Remove blank/empty rows between stimuli. If you use the auto-
generated template there shouldn't be any such blank/empty rows.
But you might have accidently created some.
</li>
<li>
<b>Sentence</b> fields:
<ul>
<li>
Check that all critical items have <b>sentence regions</b> defined and
that these regions are consistent within and across items.
Fillers should not have sentence regions marked up.
</li>
<li>
Except for the mark-up for the sentence regions, sentences and questions will
be displayed to participants as-is. So do not include extra
characters unless you want those characters to be displayed.
</li>
<li>
<b>Be particularly careful with stray SPACEs</b> (e.g. "this, that" and
"this , that" will be displayed differently). CrowdExp will assume
that each space defines a word boundary. Since CrowdExp displays
word by word, extra SPACEs will thus require additional key presses
to go through the sentences. That will make your reading time measures
very difficult or impossible to interpret.</li>
</ul>
</li>
<li>
All <b>Sentence</b> and <b>Question</b> fields need to be non-empty.
</li>
<li><b>Question</b>s:
<ul>
<li>
Make sure that within an item questions are similar and the
answers are the same.
</li>
<li>
Make sure you have an even number of Y and N answers.
Ideally <!-- What is this supposed to link to?? -->
<a href="" class="bldspr">Y and N answers should also be balanced</a>
a) within condition, and b) across all critical items.
</li>
</ul>
</li>
<li>
Upload the stimulus file to CrowdExp.
</li>
</ul>
</div>
<div class="jumbotron">
<div class="container">
<div class="row">
<table>
<tbody>
<tr>
<td class="col-md-12">
<div><a class="btn btn-primary" href="SPRLists.html">←SPR Example: Lists</a></div>
</td>
<td class="col-md-12">
<div><a class="btn btn-primary" href="Analysis.html">Analysis →</a></div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<!--
<div class="jumbotron">
<button type="button" class="btn btn-block btn-lg btn-danger" data-toggle="collapse" data-target="#demo">Test Your Understanding</button>
<div id="demo" class="collapse">
<p class="question">1. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q1" value="a" id="q1a"><label for="q1a">Answer 1</label></li>
<li><input type="radio" name="q1" value="b" id="q1b"><label for="q1b">Answer 2</label></li>
<li><input type="radio" name="q1" value="c" id="q1c"><label for="q1c">Answer 3</label></li>
<li><input type="radio" name="q1" value="d" id="q1d"><label for="q1d">Answer 4</label></li>
</ul>
<p class="question">2. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q2" value="a" id="q2a"><label for="q2a">Answer 1</label></li>
<li><input type="radio" name="q2" value="b" id="q2b"><label for="q2b">Answer 2</label></li>
<li><input type="radio" name="q2" value="c" id="q2c"><label for="q2c">Answer 3</label></li>
<li><input type="radio" name="q2" value="d" id="q2d"><label for="q2d">Answer 4</label></li>
</ul>
<p class="question">3. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q3" value="a" id="q3a"><label for="q3a">Answer 1</label></li>
<li><input type="radio" name="q3" value="b" id="q3b"><label for="q3b">Answer 2</label></li>
<li><input type="radio" name="q3" value="c" id="q3c"><label for="q3c">Answer 3</label></li>
<li><input type="radio" name="q3" value="d" id="q3d"><label for="q3d">Answer 4</label></li>
</ul>
<p class="question">4. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q4" value="a" id="q4a"><label for="q4a">Answer 1</label></li>
<li><input type="radio" name="q4" value="b" id="q4b"><label for="q4b">Answer 2</label></li>
<li><input type="radio" name="q4" value="c" id="q4c"><label for="q4c">Answer 3</label></li>
<li><input type="radio" name="q4" value="d" id="q4d"><label for="q4d">Answer 4</label></li>
</ul>
<p class="question">5. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q5" value="a" id="q5a"><label for="q5a">Answer 1</label></li>
<li><input type="radio" name="q5" value="b" id="q5b"><label for="q5b">Answer 2</label></li>
<li><input type="radio" name="q5" value="c" id="q5c"><label for="q5c">Answer 3</label></li>
<li><input type="radio" name="q5" value="d" id="q5d"><label for="q5d">Answer 4</label></li>
</ul>
<p class="question">6. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q6" value="a" id="q6a"><label for="q6a">Answer 1</label></li>
<li><input type="radio" name="q6" value="b" id="q6b"><label for="q6b">Answer 2</label></li>
<li><input type="radio" name="q6" value="c" id="q6c"><label for="q6c">Answer 3</label></li>
<li><input type="radio" name="q6" value="d" id="q6d"><label for="q6d">Answer 4</label></li>
</ul>
<p class="question">7. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q7" value="a" id="q7a"><label for="q7a">Answer 1</label></li>
<li><input type="radio" name="q7" value="b" id="q7b"><label for="q7b">Answer 2</label></li>
<li><input type="radio" name="q7" value="c" id="q7c"><label for="q7c">Answer 3</label></li>
<li><input type="radio" name="q7" value="d" id="q7d"><label for="q7d">Answer 4</label></li>
</ul>
<p class="question">8. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q8" value="a" id="q8a"><label for="q8a">Answer 1</label></li>
<li><input type="radio" name="q8" value="b" id="q8b"><label for="q8b">Answer 2</label></li>
<li><input type="radio" name="q8" value="c" id="q8c"><label for="q8c">Answer 3</label></li>
<li><input type="radio" name="q8" value="d" id="q8d"><label for="q8d">Answer 4</label></li>
</ul>
<p class="question">9. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q9" value="a" id="q9a"><label for="q9a">Answer 1</label></li>
<li><input type="radio" name="q9" value="b" id="q9b"><label for="q9b">Answer 2</label></li>
<li><input type="radio" name="q9" value="c" id="q9c"><label for="q9c">Answer 3</label></li>
<li><input type="radio" name="q9" value="d" id="q9d"><label for="q9d">Answer 4</label></li>
</ul>
<p class="question">10. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q10" value="a" id="q10a"><label for="q10a">Answer 1</label></li>
<li><input type="radio" name="q10" value="b" id="q10b"><label for="q10b">Answer 2</label></li>
<li><input type="radio" name="q10" value="c" id="q10c"><label for="q10c">Answer 3</label></li>
<li><input type="radio" name="q10" value="d" id="q10d"><label for="q10d">Answer 4</label></li>
</ul>
<div id="results">
Check
</div>
<div id="alerts"></div>
</div>
</div> -->
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
<script type="text/javascript" src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
<script type="text/javascript" src="testnav.js"></script>
<script type="text/javascript" src="quiz-1.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('li#createstim').addClass('active');
$('li#sprform').addClass('active');
});
</script>
</body>
</html>