-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathParticipants.html
More file actions
204 lines (186 loc) · 12.3 KB
/
Participants.html
File metadata and controls
204 lines (186 loc) · 12.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>BCS 152 Tutorial - Design</title>
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="tutorial.css">
</head>
<body class="bckgrnd">
<nav id="nav" class="navbar navbar-default navbar-fixed-top"></nav>
<div class="jumbotron ex top">
<h1>Participants</h1>
<p>
[<a href="#type" class="bldgn">Type of participants</a>]
[<a href="#numb" class="bldgn">Number of participants</a>]
</p>
<p id="type">
Another important design decision is <em>what type</em> of participant
and <em>how many</em>
<a href="Glossary.html#P" target="_blank" class="red" data-toggle="tooltip" title="The people who participate in a study (sometimes called subjects).">participants</a>
your experiment should have.
</p>
<h2>Type of participants</h2>
<p>
The type of participant you should administer depends on the
<a href="Glossary.html#P" target="_blank" class="red" data-toggle="tooltip" title=" The complete group of items we are interested in.">population</a>
your hypothesis is about. When we run participants, we assume that
these participants are independently sampled from the population.
This is a critical assumption, because we want the participants to
be <em>representative</em> for the population. It is because of this
assumption that we feel we can take the results of our experiments
as evidence of the likelihood that our hypothesis holds
for the population of interest. To continue the example from above,
we might hypothesize that any healthy native speaker of English
without reading problems reads high-frequency words more quickly than
low-frequency words. In that case "healthy native speaker of English
without reading problems" would be our population.
</p>
<p id="numb">
Note that we often make simplifying assumptions in selecting our
participants. For example, college students are most likely <em>not</em>
a representative example of the average "healthy native speaker of English
without reading problems". Yet, it is commonly accepted to generalize
from college student participants to the larger population of English
speakers.
</p>
</div>
<div class="jumbotron ex">
<h2>Number of participants</h2>
<p>Think about it: Why aren't we just administering a single participant for
our experiment? You probably have the intuition that this would be problematic.
For one thing, there's only so much data we can get from one participant
before that person gets tired or bored (and starts providing less and less
reliable data). But you probably also have the intuition that there's another
reason beyond this. Even if we had thousands of data points from a single
participants, we might wonder whether it is just something idiosyncratic
about that participant that gives us those results. For example, what if
that one participant--despite our best efforts to screen for it--has some
form of dyslexia or other deficits that will affect how that participant
reads? In that case, the participant would not be representative of our
target population. This problem of representativeness is arguably particularly
severe for experiments on aspects of higher-level cognition (including
language), because there's more individual variation with regard to those
aspects of cognition.
</p>
<p>
So, we typically want to administer quite a few participants, in order
to feel confident that our results will generalize to the population of
interest. But how many participants is enough?
On the one hand, more participants cost more
money and it can also take longer to recruit them, delaying the completion
of your experiment. On the other hand, more participants make it more likely
that we obtain
<a href="Glossary.html#R" target="_blank" class="red" data-toggle="tooltip" title="">reliable results</a>.
</p>
<p>
One important distinction is between the overall number of participants
you plan to administer and the number of
<a href="Glossary.html#S" target="_blank" class="red" data-toggle="tooltip" title=" A participants who is included in an analysis of an experiment. I.e. completed the experimental task and was not excluded based on a priori exclusion criteria.">successful participants</a>
you plan to administer. As we will discuss later, we sometimes have to exclude
participants from the analysis. This could happen, for example, because there
was a technical problem and no responses were recorded from them. Or the participant
fails to meet <i>a priori</i> defined
<a href="Glossary.html#I" target="_blank" class="red" data-toggle="tooltip" title="">inclusion-criteria</a>.
It is the number of <em>successful</em> participants that determines how reliable
your results will be.
</p>
<p>
How many participants we should enroll in an experiment is often a
complicated question. But there are rules of thumb. First, if you are
conducting an experiment that is very similar to previous experiments,
have a look at how many participants these experiments employed. Also
ask yourself whether those experiments obtained reliable results: Were
there results clearly significant (or just barely significant)? Were
there several studies and did they all replicate
the significant result with approximately the same number of participants?
If previous work suggests that the effect is somewhat unreliable (e.g.,
if previous work always reports significance values <em>just</em> below
p < .05), you might want to plan to administer more participants than
previous work.
</p>
<p>
Depending how comfortable you are with statistics, you can also run
<a href="Glossary.html#P" target="_blank" class="red" data-toggle="tooltip" title="Statistical analysis that lets us determine how big of a sample size we need to detect an effect with a certain confidence.">power simulations</a>,
in which you assess how likely you would be to
find an effect under certain assumptions about the size of the hypothesized
effect (based on previous experiments).
</p>
<p>
Regardless of how you decide how many participants to administer for
your experiment, the most important thing is that you make this decision
<em>before you start collecting data</em>. If you start to run you experiment
and then decide later to add further participants, then you risk biasing
your results.
<a href="http://pss.sagepub.com/content/early/2011/10/17/0956797611417632.abstract" class="sup">SimmonsEtAl11</a>
</p>
</div>
<div class="jumbotron">
<div class="container">
<div class="row">
<table>
<tr>
<td class="col-md-12"><div><a class="btn btn-primary" href="HDIStart.html">←How Do I Start?</a></div></td>
<td class="col-md-12"><div><a class="btn btn-primary" href="CriticalItems.html">Critical Items→</a></div></td>
</tr>
</table>
</div>
</div>
</div>
<div class="jumbotron">
<button type="button" class="btn btn-block btn-lg btn-danger" data-toggle="collapse" data-target="#demo">Test Your Understanding</button>
<div id="demo" class="collapse">
<p class="question">1. What is a condition?</p>
<ul class="answers">
<li><input type="radio" name="q1" value="a" id="q1a"><label for="q1a">A set of controlled specifications </label></li>
<li><input type="radio" name="q1" value="b" id="q1b"><label for="q1b">A measurable variable that is collected from the experiment</label></li>
<li><input type="radio" name="q1" value="c" id="q1c"><label for="q1c">A type of error that leads to inconclusive results</label></li>
</ul>
<p class="question">2. True or False: Many variables should be simultaneously
changed to differentiate between conditions.</p>
<ul class="answers">
<li><input type="radio" name="q2" value="a" id="q2a"><label for="q2a">True</label></li>
<li><input type="radio" name="q2" value="b" id="q2b"><label for="q2b">False - conditions should be clearly distinguished by one or two manipulated variables to eliminate any confounding factors</label></li>
</ul>
<p class="question">3. Researchers want to test whether a time delay between
presentation of the first stimuli and the second stimuli will affect how fast
and how accurately participants match them. Which type of design should they use?</p>
<ul class="answers">
<li><input type="radio" name="q3" value="a" id="q3a"><label for="q3a"> Between Subjects;
because people's reaction times will be roughly the same, it makes more
sense to compare how much each individual's performance is affected by different conditions </label></li>
<li><input type="radio" name="q3" value="b" id="q3b"><label for="q3b">Between Subjects; because
people's reaction times will be roughly the same, it makes more sense to separate them into groups and compare them</label></li>
<li><input type="radio" name="q3" value="c" id="q3c"><label for="q3c">Within Subjects; because people's reaction
times will be roughly the same, it makes more sense to separate them into groups and compare them</label></li>
<li><input type="radio" name="q3" value="d" id="q3d"><label for="q3d">Within Subjects; because people's
reaction times will be roughly the same, it makes more sense to compare how much each individual's performance is affected by different conditions</label></li>
</ul>
<p class="question">4. Researchers want to test whether students who participated
in sports performed better on different types of reading tasks than students who did not. Which type of design should they use?</p>
<ul class="answers">
<li><input type="radio" name="q4" value="a" id="q4a"><label for="q4a">Within Subjects; because they are all students, there is minimal variation between the two groups but variation individually</label></li>
<li><input type="radio" name="q4" value="b" id="q4b"><label for="q4b">Within Subjects; because some students will be exposed to sports in one condition while others will not be</label></li>
<li><input type="radio" name="q4" value="c" id="q4c"><label for="q4c">Between Subjects; because some students will be exposed to sports in one condition while others will not be</label></li>
<li><input type="radio" name="q4" value="d" id="q4d"><label for="q4d">Between Subjects; because they are all students, there is minimal variation between the two groups but variation individually</label></li>
</ul>
<div id="results">
Check
</div>
<div id="alerts"></div>
</div>
</div>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
<script type="text/javascript" src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
<script type="text/javascript" src="testnav.js"></script>
<script type="text/javascript" src="quiz-design.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('li#design').addClass('active');
$('li#des2').addClass('active');
});
</script>
</body>
</html>