You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: aulas/aula-6.qmd
+48-1Lines changed: 48 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -8,9 +8,56 @@ endnote: no
8
8
9
9
# Desenho de questionários
10
10
11
+
## Krosnick, J. A. (2018). Questionnaire Design. Em D. L. Vannette & J. A. Krosnick (Orgs.), _The Palgrave Handbook of Survey Research_ (p. 439–455). Springer International Publishing.
12
+
13
+
> I have come to the view that question wording is where the crises lies at the moment, that's one area where we severely need more work. (p. 440)
14
+
15
+
> First of all, it is true that in experimental comparisons, open-ended questions take, on average, about twice as long to answer as closed questions and respondents prefer closed questions. On practical grounds, it might appear that closed questions are preferable. In studies of reliability, open questions prove to be more reliable than closed questions and in lots of different studies of validity, open questions prove to be superior to close questions across the board using these various different methods of assessing validity. (p. 443)
16
+
17
+
> The conclusion that I take this literature to suggest is that we should ask open-ended questions when you can’t be sure of the universe of possible answer to a categorical question and the other specify option does not work. The only way to be sure that we know the universe is to pre-test, ask open questions from the population we care about, build the big list, offer it to people and we could do that, but it’s so much work we might as well as the open-ended question in the real survey. And lastly, if we’re looking for a number we should just ask for the number. That is what I take that literature to suggest and I don’t know if that’s really widely recognized and followed. (p. 445)
18
+
19
+
> The evidence in terms of response speed as rating scales get longer, from 2 to 11 points, for example, indicates that the speed goes up. Although, it’s sort of interesting that five-point scales take significantly less time than their neighbors do, that’s a hint that it is preferable to have mid-points. This is even more direct evidence, when people are asked how difficult it is to use the scale, you can see that three points, seven points, and nine points, these are significantly less difficult than the scales of other lengths. **So these are reinforcing that notion that in this case seven points is the optimal way to measure bipolar scales and that not only is it less difficult for people but that it produces more valid and reliable results**. (p. 447)
20
+
21
+
> The last thing in this arena that we have found is that it’s helpful to branch bipolar dimensions. This is one of the first branching questions that I paid attention to from the ANES: “generally speaking do you consider yourself to be a Republican, a Democrat, an independent or what?” and Republicans and Democrats are asked if they’re strong or not very strong. The independents are asked if they lean one way or another. You can produce seven-point scale like this. If you do it in those two steps up here it goes more quickly and it produces more valid and reliable results rather than presenting all seven of these, which people have to slog through and place themselves on. (p. 447)
22
+
23
+
> If the mid-point is precise, it turns out that asking people, do you lean one way or another actually adds noise. In order to produce the seven-point scale, the people who place themselves at a precise mid-point like this belong there. We shouldn’t branch the mid-point, instead we should branch the end points into three categories. So the people who say increase should be asked, do you want it increased a little, a moderate amount, or a great deal? (p. 447)
24
+
25
+
> Now, with regard to dimensions that have a natural metric, so for example I could ask you how often do you go to the movies, very often, often, sometimes, rarely or never? I’m not going to take the time to go through all of the evidence on this so forgive me for just skipping this. What I can tell you is what this literature says is using those kinds of what you might think of as vague quantifiers actually cause many more problems than they solve. **If what you want is a number just ask for the number. Other than saying, how often do you go to the movies, you ask people in the last month how many times did you go to the movies; you can get around that problem.** (p. 450)
26
+
27
+
> First of all, use simple, direct, and comprehensible words. Don’t use jargon, be specific in your question, avoid ambiguous words, avoid double barreled questions that ask two things at the same time, and avoid negations if you can avoid the word not. Avoid leading questions, and include filter questions. The common-sense version of using filter questions is don’t ask people what brand of car they have if they might not have a car. Be sure that questions read smoothly aloud, avoid emotionally charged words, avoid prestige names. When you look at textbooks and research design, it’s almost like the later ones copied the earlier ones because they are remarkably consistent in this kind of advice. (p. 450)
28
+
29
+
> First, lots and lots of studies are done that help us to inform the issues of questionnaire design and yet there is much more work to be done, especially with regard to language. I think we understand a lot about structure, but we understand much less about language. There is also an issue of dissemination. NSF of course has commitment not only to making discoveries but also to disseminating those discoveries through educational efforts and outreach. I think there is a real opportunity here because there are lots of people who don’t know it’s a bad idea to offer a ‘don’t know’ response option. There are lots of people who don’t know the order of answer choices and close questions matter and how to handle that. There are lots and lots of people who think it’s fine to ask agree/disagree questions, so a major educational outreach effort to disseminate the findings of this literature would help. (p. 455)
30
+
31
+
## Wolf, C., Joye, D., Smith, T. W., & Fu, Y. (2016). _The SAGE handbook of survey methodology_. Sage. Cap. 16: Designing Questions and Questionnaires, by Jolene D. Smyth
32
+
33
+
> This is why questionnaire design texts suggest avoiding starting with boring, embarrassing, or sensitive questions and instead make the first question simple, interesting, and applicable to all sample members with a professional design and visual layout that makes the questionnaire look simple to complete. (p. 219)
34
+
35
+
> An analysis plan can also help one refine their initial research question(s) and list of constructs to measure. For example, while age, race, and gender are not explicitly identified in the aforementioned research question, the researcher would likely want to measure them because they may be related to both engagement in risky behaviors and health outcomes,making them necessary for subgroup analyses or control variables in a regression, both of which should be expressed in the final research questions but are sometimes initially overlooked. These and other constructs may also need to be measured for weighting and adjustment purposes, which should also be planned carefully upfront in the context of the research question(s). (p. 220)
36
+
37
+
> Questions should be written to accommodate the retrieval process, which means accounting for how people recall information. People are better able to recall events that happened recently, are distinctive (e.g., buying a new car versus buying groceries), or that are important (e.g., a wedding celebration versus a birthday celebration) (Tourangeau et al., 2000). People can also more accurately recall events when the recall period is shorter (e.g., the last week versus last year). (p. 222)
38
+
39
+
> Thus, in writing questions, it is important to determine whether exact enumeration of events is needed or whether an estimate is good enough. With very frequent and mundane events, estimation might just have to be good enough. However, for events respondents are expected to be able to recall and enumerate, a shorter recall period will help. Slowing down the survey to give more time to think can also improve retrieval, and can be done by asking longer (not more complex) questions (Bradburn and Sudman, 1979; Tourangeau et al., 2000). (p. 222)
40
+
41
+
> In this case, respondent motivation to provide quality open-ended responses can be increased by emphasizing the importance of the question. Statements like, ‘This question is very important to this research’ can increase the length and quality of open-ended responses (Oudejans and Christian, 2011; Smyth et al., 2009). (p. 223)
42
+
43
+
> Using construct-specific rather than vague quantifier labels can also make reporting more straightforward and reduce measurement error (e.g., asking ‘How would you rate the quality of your new car?’ instead of ‘How much do you agree or disagree that your new car is high quality?’) (Saris et al., 2010). (p. 223)
44
+
45
+
> Reliability and validity seem to be maximized at 5 to 7 scale points for bipolar scales (i.e., scales that measure both direction and magnitude like very satisfied/ very dissatisfied) and 4 to 5 scale points for unipolar scales (scales that measure only magnitude, like never to always) (Krosnick and Fabrigar, 1997). In addition, ensuring that the scale points are conceptually equidistant can also help. (p. 223)
46
+
47
+
> Questionnaire designers also have to make sure respondents are willing to provide accurate answers at the reporting stage. Some respondents may be hesitant because of social or normative concerns. Social desirability is a tendency to answer questions in a way that makes one look good (or not look bad) rather than providing the most accurate answer (Tourangeau and Yan, 2007). It can take the form of underreporting negative behaviors like illicit drug use, abortion, and poor college performance or overreporting positive behaviors like voting and church attendance (Bernstein et al., 2001; Hadaway et al., 1993; Kreuter et al., 2008; Tourangeau and Smith, 1996). Respondents are more susceptible to social desirability in interviewer-administered surveys (Aquilino, 1994; de Leeuw, 1992; Dillman et al., 1996), but it is a concern in all modes. (p. 224)
48
+
49
+
> Aside from increasing response privacy by asking sensitive questions in a private, self- administered mode, question design strategies to combat social desirability sometimes focus on making it either more acceptable or safe to answer honestly. Question wording is commonly changed to make the question less threatening such as asking ‘Have you happened to …’, ‘Some people believe <desirableopinion> and others believe <undesirableopinion>’, ‘Do you believe <desirableopinion> or <undesirableopinion>?’, and ‘There are many reasons people might not do <desirableactivity> such as not having time, not having transportation, or being sick. How about you, did you do <desirableactivity>?’ (Bradburn et al., 2004). While these strategies make intuitive sense, the few empirical evaluations that have been done suggest they are not more effective than direct inquiries (Bradburn et al., 2004; Schuman and Presser, 1981; Yeager and Krosnick, 2012). (p. 224)
50
+
51
+
> For example, an important word may be made more visible by increasing its font size or bolding or italicizing it. Likewise, nonessential information (e.g., office use only, data entry codes, etc.) can be deemphasized by making it smaller and lighter in color and placing it in areas respondents are less likely to look. Applying properties allows designers to impact how respondents understand the relationship between elements on a page. (p. 226)
52
+
53
+
> There are several ways question ordering can help with retrieval. For example, grouping topically similar questions gains efficiencies because respondents can use retrieved information to answer all questions on a topic before moving to a different topic (Dillman et al., 2014). However, one also has to be watchful for unintentional priming effects, which occur when information retrieved for an early question is used to answer a later question simply because it is more easily accessible. For example, Todorov (2000) found that asking respondents about specific chronic conditions early in the National Health Interview Survey on Disability increased the likelihood that they would identify one of the chronic conditions asked about as a cause of their disability in a later question. (p. 229)
54
+
55
+
> Orienting oneself to how respondents will experience the questionnaire and their response process can provide a useful framework for making these design decisions. This includes keeping in mind the goals of encouraging response, promoting optimizing, providing a clear navigational path, and, of course, collecting high quality measurements. It also requires one to think about how their design will impact each stage of the response process. In addition, understanding the response process and where it can break down can help one determine what pretesting method(s) are most appropriate for a given questionnaire and troubleshoot problems. (p. 231)
56
+
57
+
11
58
## Anotações de aula
12
59
13
-
Teoria do responde guarda muita semelhança com o que vimos do Zaller, mas não exatamente. Zaller foca na recepção e no fato de que as pessoas exercem um filtro na resposta, exercendo algum grau de amostragem mental para responder.
60
+
Teoria do respondente guarda muita semelhança com o que vimos do Zaller, mas não exatamente. Zaller foca na recepção e no fato de que as pessoas exercem um filtro na resposta, exercendo algum grau de amostragem mental para responder.
14
61
15
62
A teoria do respondente assume que as pessoas possuem valores mais ou menos lantentes, mas não valores inatos, mas memórias de longo prazo positivas ou negativas relacionadas a certos aspectos. Tenho uma série de memórias que não me cruzam diariamente, mas que surgem de maneira mais ou menos clara da ideia. Respondentes passam por etapas cognitivas para acessar atitudes da memória de longo-prazo (Touranngeau e Rasinski, 1988).
## Wang, W., Rothschild, D., Goel, S., & Gelman, A. (2015). Forecasting elections with non-representative polls. _International Journal of Forecasting_, 31(3), 980–991
12
+
13
+
14
+
## Hanretty, C. (2020). An Introduction to Multilevel Regression and Post-Stratification for Estimating Constituency Opinion: _Political Studies Review_, 18(4), 630–645
15
+
16
+
17
+
## Lax, J. R., & Phillips, J. H. (2009). How Should We Estimate Public Opinion in the States. _American Journal of Political Science_, 53(1), 107–121
18
+
19
+
20
+
## Buttice, M. K., & Highton, B. (2013). How does multilevel regression and poststratification perform with conventional national surveys? _Political Analysis_, 21(04), 449–467.
21
+
22
+
23
+
## Other materials
24
+
25
+
-[Multilevel regression and post-stratification case studies](https://bookdown.org/jl5522/MRP-case-studies/), by Lopez-Martin, J., Phillips, J. H., & Gelman, A. (2022).
0 commit comments