-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathSPRProcedure.html
More file actions
263 lines (230 loc) · 14.1 KB
/
SPRProcedure.html
File metadata and controls
263 lines (230 loc) · 14.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>BCS 152 Tutorial</title>
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="tutorial.css">
</head>
<body class="bckgrnd">
<nav id="nav" class="navbar navbar-default navbar-fixed-top"></nav>
<div class="jumbotron spr top">
<h1>Self Paced Reading: Procedure</h1>
<p>
[<a href="#see" class="bldspr">What participants will see</a>]
[<a href="#svsa" class="bldspr">Speed vs Accuracy</a>]
[<a href="#record" class="bldspr">What behavior will CrowdExp-SPR record?</a>]
</p>
<p>
In <a href="Glossary.html#S" target="_blank" class="red" data-toggle="tooltip" title="Experimental paradigm in which words of a sentence are displayed one by one in response to the participant's actions (e.g. pressing a space bar on the computer). ">self-raced reading</a> experiments, participants read stimuli (e.g., words,
sentences, or paragraphs). It is called <em>self-paced</em> because
participants control the timing of how each part of the sentence stimulus is displayed.
</p>
<p id="see">
Don't worry, you don't need to program the experimental procedure yourself.
We (or specifically, Hal Tily, now at Nuance Technology) have programmed a
web-based self-paced reading app. This app, CrowdExp-SPR will display your
stimuli, randomize their presentation, create a
<a href="Glossary.html#L" target="_blank" class="red" data-toggle="tooltip" title="">Latin Square Design</a>
(we'll learn more about what this later), record reading times and more.
More about that ... now.
</p>
</div>
<div class="jumbotron spr">
<h2>What participants will see</h2>
<p>
Here is an example of what participants will see when they take your
experiment (this particular display is specific to CrowdExp-SPR). For
the participants, the stimulus sentence will advance word-by-word with each
press of the SPACE button.
</p>
<p>
<img title="SPR Window" src="spranim_small.gif" width="479" height="300" alt="" />
</p>
<p>
CrowdExp-SPR will record the times between button presses. Those are
the reading times that we will analyze to test our predictions.
</p>
<p>
This particular type of self-paced reading is called <em>moving-window</em>
word-by-word self-paced reading. The <em>moving-window</em>
part refers to the fact that only one word is shown
at each time and that pressing SPACE both advances to the next word
and also hides the previous word. This particular type of self-paced
reading also employs an <em>uninformative</em> mask: alternative
versions of self-paced reading might show the word boundaries in the
mask (e.g., "- --- --- ----- -- --- -----.").
</p>
<p id="svsa">
Notice also the progress bar at the top of the page, telling participants
how many more sentences they still have to read before the experiment
is over.
</p>
</div>
<div class="jumbotron spr">
<h2>Speed vs. Accuracy</h2>
<p>
One challenge whenever experimenters want to measure how <em>fast</em>
we can do something (like how fast we can read) is that participants
might get carried away. For example, someone might just keep pressing
the SPACE bar as fast as they can. That will lead to maximally fast
reading times and make it impossible to detect any effects on those
reading times.
</p>
<p>
So, to prevent this problem, we want to make sure that participants
in our experiment actually <em>read and understand</em> the sentences
we show them. This is achieved by
<a href="Glossary.html#C" target="_blank" class="red" data-toggle="tooltip" title="Comprehension questions provide a measure of whether participants are paying attention during an experiment (i.e. in self-paced reading, ensures that they are reading the sentence, as opposed to just clicking to get through it as quickly as possible). Comprehension questions can be asked after every sentence, block of sentences, or even at the end of the experiment (depending on the type of experiment).">comprehension questions</a>.
By default, CrowdExp-SPR will ask one yes/no comprehension questions after
each stimulus. For the example stimulus given above, we might ask
the following question:
</p>
<p>
<img title="SPR Question" src="spr_question.png" width="479" height="302" alt="" />
</p>
<p id="record">
CrowdExp-SPR will record both the answer (yes/no)—and thus its accuracy
(correct/incorrect)—and the reaction time it took to answer the
question.
</p>
</div>
<div class="jumbotron spr">
<h2>What behavior (outcome variables) will CrowdExp-SPR record?</h2>
<p>
CrowdExp-SPR will return word-by-word reading times, as well as the
reaction time and accuracy of the comprehension question for each
sentence. The word-by-word reading times will be your primary
<a href="Glossary.html#O" target="_blank" class="red" data-toggle="tooltip" title=" In (behavioral) experimental design an outcome variable is a response that is measured and (hopefully) changes in response to experimentally controlled differences in a predictor or independent variable.">outcome variable</a>.
</p>
<p>
That is, your experiment will use reading times to test the predictions
of theories about language processing.
Any effect that leads to changes in processing speed, and thus
word-by-word reading times, can in theory be detected in the self-paced
reading paradigm. For example,
it typically takes between 150-450msecs to read a word (and press the
SPACE key). Words will be read more or less quickly, depending on their
length, frequency, etc., or because they are surprising in context.
</p>
<p>
Beyond the reading times, some hypotheses also make predictions about
the answer reaction time or accuracy. But even if we don't directly
test predictions about these alternative outcome variables, answer
accuracy is standardly used in order to exclude participants who did
not read the sentences carefully. For example, a common exclusion
criterion is to exclude from the analysis all data from any participant
who did not answer at least 85% of the questions correctly. We will discuss
more about exclusions in the Analysis section.
</p>
<p>
Now that you have a bit of an idea of the procedure we will be employing,
it's time to learn more about the structure of experiments and
important constraints on the stimuli materials.
</p>
</div>
<div class="jumbotron">
<div class="container">
<div class="row">
<table>
<tr>
<td class="col-md-12"><div><a class="btn btn-warning" href="SPRDesign.html">←SPR Example: Design</a></div></td>
<td class="col-md-12"><div><a class="btn btn-warning" href="SPRCriticalItems.html">SPR Example: Critical Items →</a></div></td>
</tr>
</table>
</div>
</div>
</div>
<!--<div class="jumbotron">
<button type="button" class="btn btn-block btn-lg btn-danger" data-toggle="collapse" data-target="#demo">Test Your Understanding</button>
<div id="demo" class="collapse">
<p class="question">1. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q1" value="a" id="q1a"><label for="q1a">Answer 1</label></li>
<li><input type="radio" name="q1" value="b" id="q1b"><label for="q1b">Answer 2</label></li>
<li><input type="radio" name="q1" value="c" id="q1c"><label for="q1c">Answer 3</label></li>
<li><input type="radio" name="q1" value="d" id="q1d"><label for="q1d">Answer 4</label></li>
</ul>
<p class="question">2. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q2" value="a" id="q2a"><label for="q2a">Answer 1</label></li>
<li><input type="radio" name="q2" value="b" id="q2b"><label for="q2b">Answer 2</label></li>
<li><input type="radio" name="q2" value="c" id="q2c"><label for="q2c">Answer 3</label></li>
<li><input type="radio" name="q2" value="d" id="q2d"><label for="q2d">Answer 4</label></li>
</ul>
<p class="question">3. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q3" value="a" id="q3a"><label for="q3a">Answer 1</label></li>
<li><input type="radio" name="q3" value="b" id="q3b"><label for="q3b">Answer 2</label></li>
<li><input type="radio" name="q3" value="c" id="q3c"><label for="q3c">Answer 3</label></li>
<li><input type="radio" name="q3" value="d" id="q3d"><label for="q3d">Answer 4</label></li>
</ul>
<p class="question">4. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q4" value="a" id="q4a"><label for="q4a">Answer 1</label></li>
<li><input type="radio" name="q4" value="b" id="q4b"><label for="q4b">Answer 2</label></li>
<li><input type="radio" name="q4" value="c" id="q4c"><label for="q4c">Answer 3</label></li>
<li><input type="radio" name="q4" value="d" id="q4d"><label for="q4d">Answer 4</label></li>
</ul>
<p class="question">5. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q5" value="a" id="q5a"><label for="q5a">Answer 1</label></li>
<li><input type="radio" name="q5" value="b" id="q5b"><label for="q5b">Answer 2</label></li>
<li><input type="radio" name="q5" value="c" id="q5c"><label for="q5c">Answer 3</label></li>
<li><input type="radio" name="q5" value="d" id="q5d"><label for="q5d">Answer 4</label></li>
</ul>
<p class="question">6. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q6" value="a" id="q6a"><label for="q6a">Answer 1</label></li>
<li><input type="radio" name="q6" value="b" id="q6b"><label for="q6b">Answer 2</label></li>
<li><input type="radio" name="q6" value="c" id="q6c"><label for="q6c">Answer 3</label></li>
<li><input type="radio" name="q6" value="d" id="q6d"><label for="q6d">Answer 4</label></li>
</ul>
<p class="question">7. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q7" value="a" id="q7a"><label for="q7a">Answer 1</label></li>
<li><input type="radio" name="q7" value="b" id="q7b"><label for="q7b">Answer 2</label></li>
<li><input type="radio" name="q7" value="c" id="q7c"><label for="q7c">Answer 3</label></li>
<li><input type="radio" name="q7" value="d" id="q7d"><label for="q7d">Answer 4</label></li>
</ul>
<p class="question">8. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q8" value="a" id="q8a"><label for="q8a">Answer 1</label></li>
<li><input type="radio" name="q8" value="b" id="q8b"><label for="q8b">Answer 2</label></li>
<li><input type="radio" name="q8" value="c" id="q8c"><label for="q8c">Answer 3</label></li>
<li><input type="radio" name="q8" value="d" id="q8d"><label for="q8d">Answer 4</label></li>
</ul>
<p class="question">9. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q9" value="a" id="q9a"><label for="q9a">Answer 1</label></li>
<li><input type="radio" name="q9" value="b" id="q9b"><label for="q9b">Answer 2</label></li>
<li><input type="radio" name="q9" value="c" id="q9c"><label for="q9c">Answer 3</label></li>
<li><input type="radio" name="q9" value="d" id="q9d"><label for="q9d">Answer 4</label></li>
</ul>
<p class="question">10. What is the answer to this question?</p>
<ul class="answers">
<li><input type="radio" name="q10" value="a" id="q10a"><label for="q10a">Answer 1</label></li>
<li><input type="radio" name="q10" value="b" id="q10b"><label for="q10b">Answer 2</label></li>
<li><input type="radio" name="q10" value="c" id="q10c"><label for="q10c">Answer 3</label></li>
<li><input type="radio" name="q10" value="d" id="q10d"><label for="q10d">Answer 4</label></li>
</ul>
<div id="results">
Check
</div>
<div id="alerts"></div>
</div>
</div> -->
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
<script type="text/javascript" src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
<script type="text/javascript" src="testnav.js"></script>
<script type="text/javascript" src="quiz-1.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('li#procedure').addClass('active');
$('li#sprpro').addClass('active');
});
</script>
</body>
</html>