Panel 1
Traugott, M. Jozkowski, K.N., Crawford, B.L., Turner, R.C., & Lo, W.J. (2021, May 11-14). Developing new measures to assess abortion attitudes in the United States. American Association for Public Opinion Research (AAPOR) Annual Meeting.
Abortion is a nuanced and complex issue, which makes measuring abortion attitudes and opinions conceptually and methodologically challenging. Previous research suggests that people may feel conflicted or ambivalent toward abortion or may hold seemingly competing opinions when responding to abortion polling questions. For example, some people who identify as pro-life and indicate abortion is morally wrong may also indicate abortion should be legal. Given these conflicting perspectives, we theorize multiple dimensions may comprise people’s attitudes toward abortion. Standard polling questions, which can be limited in scope and rely on dichotomous response options, may not be equipped to capture this multi-dimensionality. This may, in part, explain potential dissonance across polling data, public discourse, and legislation. To account for this multi-dimensionality, we assembled an interdisciplinary team to conduct a multi-year study aimed at developing sets of abortion attitudes measures (e.g., individual items and scales) in English and Spanish. This panel will (1) include an overview of the study, (2) describe our methodological approaches, and (3) present findings from formative data collections.
The first presentation will provide an overview of the project goals and specific aims as well as a description of the methodological design for the entire multi-year, multi-method project, including a discussion of our complexity framework. The second presentation will include results from three formative surveys and cognitive interviews demonstrating potential inconsistencies in how people interpret certain wording in abortion attitude items (e.g., “for any reason”, “any time”). The final presentation will include findings from a formative methodological study aimed at assessing item response format issues using replications of the General Social Survey abortion items.
Jozkowski, K.N., Crawford, B.L., Turner, R.C., Lo, W.J., & Traugott, M. (2021, May 11-14). An interdisciplinary, multi-dimensional approach to developing new abortion attitude measures in the United States: A project overview. American Association of Public Opinion Research (AAPOR) Annual Meeting.
We theorize people’s attitudes toward abortion are complex and multi-dimensional, requiring innovative approaches to assess such multi-dimensionality. We are conducting a multi-phase, multi-year project to concurrently develop a comprehensive array of instruments (e.g., individual polling items; item scale sets) in English and Spanish to assess abortion attitudes in the US. The purpose of this presentation is to provide a detailed description of the methodological approaches we are using to develop these bilingual measures.
After conducting comprehensive literature reviews of both prior content and methodology, we began formative data collections in English and Spanish. After cognitive pre-testing, we fielded an open-ended survey to a sample drawn from NORC’s AmeriSpeak panel (N=608). These data provided insights into people’s salient behavioral, normative, and contextual beliefs regarding abortion. We conducted two pilot surveys (N1 = 512; N2 = 1586) comprising existing and experimental items aimed at distinguishing people who may be consistently polarized in their abortion perspective (anti- or pro-abortion) from those with more nuanced perspectives, with follow-up pilot interviews (N = 24). After refinement, we will administer a finalized version of this survey to NORC’s AmeriSpeak panel (N = 600) and conduct follow-up in-depth interviews (N = 200). We also conducted four additional formative studies designed to test item content and instrument design hypotheses related to abortion attitude measurement. We will integrate these findings with other results to identify common themes and underlying constructs that will drive item development for our new abortion attitude scales and polling items. Multi-stage pilot assessments and cognitive interviews will be used to assess readability, item-level content validity, psychometric functioning, and consistency across survey modes, with revisions made as needed. Finally, we will administer a comprehensive set of new items to a large, nationally representative sample of US English- and Spanish-speaking adults (N = ~26,000).
Turner, R.C., Jozkowski, K.N., Crawford, B.L., Lo, W.J., & Traugott, M. (2021, May 11-14). All for one or one for all: The impact of using terms with multiple interpretations on item response distributions. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Determining wording for survey items assessing complex issues like abortion can be difficult. Item response trends can provide important information regarding item interpretation and content validity. We investigated the terms “for any” in reference to circumstances and “at any” in reference to timeframes for abortion items to examine the potential for multiple interpretations of “any” as defined in Black’s Law Dictionary (1990, p. 94) -- “’Any’…may indicate ‘all’ or ‘every’ as well as ‘some’ or ‘one’ depend[ing] upon the context…”
Using data from three studies, we investigated how participants interpreted “any” within item contexts. First, using pilot data collected via Qualtrics (N = 245), we assessed eleven abortion attitude items modeled after the seven General Social Survey (GSS) abortion legality items. We compared patterns of responses across a range of abortion circumstances (e.g., rape, life endangerment) with the “…if she wants [an abortion] for any reason” GSS item. Significantly fewer respondents support abortion legality under two circumstances than the “for any reason” item. Interview data indicated that some respondents interpret “for any reason” to mean for all reasons or in all circumstances, whereas others interpreted “any reason” to mean in at least one circumstance. We replicated this study using Ipsos’ nationally representative Knowledge Panel (N=919), finding similar trends. Finally, we collected data from a second Qualtrics sample (N=2,442) to investigate responses to the GSS abortion legality items when gestational timeframes were added to item stems. Timeframes included 6, 12, and 20 weeks, ending with “any time” during the pregnancy. More participants indicated abortion should be legal “any time” during a pregnancy than at 20 weeks, suggesting potential diversity in interpretation of “any” in reference to timeframes. Understanding different interpretations of “any” may help researchers contextualize potential inconsistencies in people’s responses to abortion legality items.
Lo, W.J., Turner, R.C., Crawford, B.L., Jozkowski, K.N., & Traugott, M. (2021, May 11-14). A psychometric evaluation of 2-point, 4-point, and 6-point Likert scales to assess complexity in abortion attitudes. American Association of Public Opinion Research (AAPOR) Annual Meeting.
There does not appear to be consensus regarding the optimal number of points to use on rating scales when assessing people’s attitudes toward social issues like abortion (Krosnick & Presser, 2010). Although dichotomous scales (e.g., Yes, No) are widely used in many surveys, adding more points to a rating scale provides participants with greater diversity in response options, which could result in more valid and reliable findings when assessing attitudes toward complex issues like abortion. The purpose of this study is to evaluate the Likert scale with three even number response formats in terms of their latent constructs and to compare the intensity of attitudes toward abortion using General Social Survey (GSS) abortion items.
A total of 1,006 participants were randomly assigned to complete one of three sets of questions, which included different modifications to the six GSS items assessing abortion under different circumstances (e.g., rape, life endangerment). The first set of items modified the original dichotomous options from Yes/No to Agree/Disagree. The second set provided a 4-point option by adding “strongly agree” and “strongly disagree” to the 2-point agree/disagree format. The third set provided a 6-point scale, adding “strongly,” “moderately,” and “slightly” to both agree and disagree categories.
We first conducted a chi-square test to evaluate whether our 2-point scale had similar response patterns to the nationally representative GSS data from 2018. Second, we conducted confirmatory factor analysis (CFA) to evaluate whether each scale still held the two-factor model (Muthen, 1981). Third, to investigate the influences between the different number of response options, we aggregated a 4- and 6-point scale to a dichotomous scale in comparison with the standard GSS format. Our findings suggest that adding points to the rating scale demonstrate greater complexity in response patterns. Implications for abortion measurement will be discussed.
Crawford, B.L., Jozkowski, K.N., Turner, R.C., & Lo, W.J. (2021, May 11-14). Legal or illegal: Examining differential response patterns in abortion attitude items when asked about legality versus illegality. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Because abortion is a complex social issue, responses to abortion attitude items on surveys may be particularly sensitive to differences in question and answer formats. Specifically, differences in the number of response options, question framing, and the dimensionality of answer choices may yield significantly different assessments of abortion attitudes. For example, questions assessing abortion attitudes in specific circumstances, such as the core abortion items in the General Social Survey, focus on the dimension of legality. These questions typically ask if abortion should be legal in a particular context followed by answer choices of “Yes” and “No”. However, circumstantial abortion questions are seldomly worded such that the respondent is asked if abortion in that circumstance should be illegal. Although responding with a “No” to these types of circumstantial abortion legality questions is technically the practical equivalent of saying abortion should be “illegal”, we hypothesize that participants may be more uncomfortable saying that abortion should be illegal, and therefore less likely to select “Illegal” than “No” on similar questions.
To test this hypothesis, respondents from IPSOS’s Knowledge Panel (N=919) were provided both the web equivalent of the core GSS abortion items with response options of “Yes” and “No” and versions of the questions with response options ranging from “Definitely Illegal” to “Definitely Legal”. Results show there is higher agreement between answering “Yes” to the GSS questions and “Legal” to the modified version (average agreement of 90%) than answering “No” and “Illegal” (average agreement of 72%). In addition to discussing the differences in agreement across the seven items, we will also discuss which demographic sub-groups were most likely to have higher levels of disagreement between the “No” and “Illegal” response options. Our presentation will conclude with a discussion of the relevance of our findings when assessing abortion attitudes and other complex social issues.
*Kaplan, A.M., Solon, M., Crawford, B.L., Turner, R.C., Lo, W.J., Jozkowski, K.N. (2021, May 11-14). Acculturation and Latinx abortion attitudes: An assessment of multiple acculturation measures. American Association for Public Opinion Research (AAPOR) Annual Meeting.
Given the growing influence and heterogenous nature of the US Latinx community, measuring Latinx attitudes toward socially contentious topics as well as factors contributing to variance in such attitudes is important for understanding trends in the US social and political climate. This study examines the relationship between acculturation—or cultural adaptation—and US Latinx attitudes toward the legality and morality of abortion. Research suggests that heightened acculturation is associated with more supportive attitudes toward abortion. However, differences in approaches to measuring acculturation across studies make conclusions about this relationship tentative. Thus, in addition to assessing the relationship between acculturation and abortion attitudes for a US Latinx sample, this study compares the predictive properties of several acculturation measures. A web-based survey was administered using Qualtrics’ panels to English- and Spanish-speaking Latinx adults (N=109). Acculturation was measured via the Hispanic Acculturation Index (HAI) the Short Acculturation Scale for Hispanics (SASH), and common acculturation proxies of language use and immigrant generation. Abortion attitudes were assessed with two scales related to respondents’ opinions on the (1) morality and (2) legality of abortion under various circumstances. Multiple regression models were used to compare the predictive properties of the HAI, SASH, and acculturation proxies alongside common demographics (e.g. gender, education level, bible literalism). Comparison of the regression models showed a significant effect of demographic factors on abortion morality (F(9,94)=6.74, p<0.001,R2=0.39) and legality (F(9,94)= 8.20, p<0.001, R2=0.44), but no significant differences were found between models when the acculturation measures were added. Additionally, no differences were found between different acculturation measures. Given that our sample was relatively varied in acculturation, our results suggest little relationship between acculturation and Latinx abortion attitudes. Nevertheless, we urge further exploration of acculturation with different methods or additional measures to assess and better understand the role of Latinx identity in abortion attitudes.
Panel 2
Solon, M., Crawford, B.L., Turner, R.C., Lo, W.J. & Jozkowski, K.N.(2021, May 11-14). Using propensity score matching to examine differences in abortion attitudes by language of survey. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Previous research suggests that the language in which a survey is administered can affect survey responses. Among a Latino sample, for instance, Lee and Pérez (2014) found that survey language was significantly related to various political opinions even after controlling for differences in demographic and background variables. When considering multilanguage surveys and multilingual participants, it is important to know if and how participants’ responses may have differed had they taken the survey in a different language. Although randomization is the ideal way to assess these potential differences, randomization of survey language is not always possible. Some disciplines have used propensity score analyses (PSA) as a way to assess “treatment effects” in the absence of randomization. We propose the novel approach of utilizing PSA to assess the potential impact of survey language. Specifically, we use PSA to investigate the influence of survey language on US Latinx responses to web-based survey questions related to abortion attitudes. By viewing survey language as a treatment or intervention, PSA allows researchers to estimate how a participant’s answers to a survey may have differed had they completed the survey in a different language (the counterfactual). Additionally, because PSA uses both a selection and an outcome model, PSA allows researchers to use separate sets of covariates related to treatment selection (in this case, survey language) and the outcome of interest (abortion attitudes). Our findings are based on results from two surveys (n=3,000; n=1,500) where approximately 1/3 of the sample completed the survey in Spanish. We will discuss multiple PSA approaches and the benefits of each approach. To conclude, we will highlight how PSA can help estimate the impact of survey language on response as well as help identify problems with translations.
*Kaplan, A.M., Montenegro, M., Weese, J.D., Turner, R.C., Lo, W.J., Crawford, B.L., Jozkowski, K.M. (2021, May 11-14). Latinx and non-Latinx attitudes toward abortion morality and legality and its influence in their support toward women seeking an abortion. American Association for Public Opinion Research (AAPOR) Annual Meeting.
Abortion continues to permeate US socio-political discourse. However, there is little consensus among what is known about abortion attitudes among the Latinx community—the fastest-growing US population, and how their beliefs compare to non-Latinx community. This presentation compares Latinx and non-Latinx people’s support for their family and friends’ decisions to have an abortion or continue an unwanted pregnancy -- as related to demographic and abortion attitude-related variables. We conducted two sets of hierarchical regression analyses to assess whether support for a friend or family member seeking an abortion or continuing a pregnancy is predicted by participants’ attitudes toward abortion morality and legality while controlling for demographic variables (e.g., Latinx identity, religious affiliation, abortion identity). Abortion identity was significant when predicting support for abortion. Those identifying as pro-life were significantly less likely to support women seeking an abortion than participants with other abortion identities. When adding morality and legality attitudes into the model, abortion identity-maintained significance and held the same trend. Participants with higher abortion morality and legality scores (indicating abortion is moral and should be legal) had higher support for women seeking an abortion. Conversely, identifying as pro-life and Christian was associated with higher support for continuing a pregnancy. There were no significant differences between Latin and non-Latinx regarding support for abortion or continuing a pregnancy. Studies suggest that the US Latinx community has more pro-life leaning tendencies than non-Latinx members. Our findings suggest that support for a family member or friend considering abortion or continuing an unwanted pregnancy are similar for Latinx and non-Latinx adults, after controlling for key demographic and attitudinal variables. Thus, although abortion attitude differences may occur at the aggregate level for these groups, differences may be accounted for by factors similar to those used in this study.
Jozkowski, K.N., LaRoche, K.J., Turner, R.C., Crawford, B.L., & Lo W.J. (2021, May 11-14). Abortion attitudes in the context of the Coronavirus pandemic. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Background: At the onset of the coronavirus pandemic, many states placed temporary bans on medical services deemed “nonessential;” several states initially deemed abortion non-essential. This, coupled with the fact that seventeen states ban use of telemedicine for abortion, re-ignited debates about abortion accessibility in the U.S. Although, federal judges eventually blocked orders categorizing abortion as nonessential, such events demonstrate the political saliency of abortion. We examined whether and why people thought abortion should be (1) considered an essential medical service and (2) permitted via telemedicine during the coronavirus pandemic.
Method: Two waves of data were collected from Ipsos’ Knowledge Panel (N = 919, Wave 1; N = 711 Wave 2), a probability sample of U.S. adults. Participants were asked whether they thought abortion should be considered an essential medical service and whether telemedicine should be permitted for medication abortion during the pandemic across 11 different circumstances (e.g., life endangerment, financial reasons). Participants were also asked open-ended questions to explain their close-ended responses.
Results: When asked generally, 44.9% indicated abortion should be essential and 46.7% indicated abortion should be available via telemedicine. However, there was greater support (60% to 81%) for abortion as essential for medical-related circumstances (e.g., life endangerment, baby not expected to survive) and less support (54% to 67%) for social circumstances (e.g., financial difficulties, not wanting more children). Participants’ reasons for thinking abortion is/is not essential were primarily circumstantial, although some participants mentioned the importance of timing.
Discussion: Relying on general items might be misleading regarding abortion attitudes in complex situations—as essential services or in the context of telemedicine--as support varied substantially. Open-ended findings suggest people’s perception of abortion as essential may depend on circumstances people consider “essential.” People also recognized “time is of the essence” regarding abortion. Implications for abortion attitude measurement will be discussed.
Crawford, B.L., Jozkowski, K.N., Turner, R.C., Lo, W.J., & Traugott, M. (2021, May 11-14). Triangulating quantitative and qualitative data to unpack complexity and dissonance in abortion attitudes. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Abortion is a complex social issue. Thus, assessing abortion attitudes via survey questions can be challenging, particularly given that people’s attitudes may be multi-dimensional. How people respond to survey questions may vary based on underlying factors such as if the question frames abortion as a legal or moral issue, the circumstances surrounding the abortion decision (e.g. pregnancy as a result of rape, parents are not ready), and the gestational age of the pregnancy (e.g., 6 weeks). Further, the complexity of respondents’ abortion attitudes may manifest in seemingly dissonant responses when asking questions that span different contexts (e.g., abortion due to rape; abortion at 6 weeks gestation) and dimensions (e.g., legality, morality). To unpack the complexity and dissonance of survey responses, we triangulated findings from a quantitative pilot survey with findings from qualitative pilot follow-up interviews. Specifically, we conducted an online survey using GfK’s opt-in panel (English N = 1,093; Spanish N = 493) with follow-up in-depth interviews from survey respondents (English N = 16; Spanish N = 8). The survey comprised items assessing multiple dimensions of abortion attitudes; interviews focused on assessing people’s salient beliefs regarding abortion as well as points of potential dissonance, allowing participants to explain their complex conceptualizations of abortion.
After outlining our process for the mixed-method pilot study, we will discuss how themes from participants’ in-depth interviews map onto their survey responses, providing context for potentially complex and dissonant survey responses. We will then share lessons learned and changes made to the protocol for the final data collection that we will administer via NORC’s AmeriSpeak panel. We will conclude with a discussion of how the triangulation of these data can be used in the creation of new abortion attitude measures that may reduce dissonance or provide insight into the seemingly dissonant responses.
Turner, R.C., Crawford, B.L., Lo, W.J., Jozkowski, K.N. (2021, May 11-14). Should it be legal and is it moral? A multi-dimensional assessment of abortion attitudes in the U.S. American Association of Public Opinion Research (AAPOR) Annual Meeting.
People’s attitudes toward abortion are complex, potentially due to the multi-dimensional conceptualizations some people have regarding abortion, creating challenges for measurement. To examine the multi-dimensionality of abortion attitudes, we examined people’s attitudes through frameworks of two dimensions--morality and legality. Using a nationally representative sample (N = 919) collected via Ipsos’ Knowledge Panel, we asked whether abortion should be legal and is moral in seventeen circumstances, based on items from the General Social Survey. At the aggregate, abortion was rated lower on the Morality scale (M = 3.158, SD = 1.241; 1 to 5 range) than Legality scale (M = 3.477, SD = 1.276) indicating people are more likely to believe abortion should be legal than is moral.
When disaggregated by type of circumstance, the difference between morality and legality scores was smallest for circumstances related to woman’s physical and mental health and rape--for these circumstances agreement that abortion is moral and should be legal was highest. Endorsement for both morality and legality was lower for abortion in circumstances such as financial reasons, contraception failure, and gender selection. The largest discrepancy in endorsement of abortion being moral and legal occurred for social reasons (e.g., financial, marital status of the couple). Across political affiliation, the smallest discrepancy between endorsement of morality and legality occurred for Republicans (.191 to .262) and those with no political affiliation (.091 to .292)—groups that most often oppose abortion. Independents and Democrats supported abortion legality at a higher rate than abortion being moral, with Democrats having the largest discrepancy (.109 to .517).
People’s attitudes toward abortion vary depending on the framework used to measure attitudes and this variability may be exasperated across certain subgroups. Researchers should consider other dimensions that may underlie abortion attitudes. Additional implications for abortion attitude measurement development will be discussed.
Individual
Lo, W.J., Turner, R.C., Crawford, B.L., Jozkowski, K.N. (2021, May 11-14). An even or odd number of response category: Investigating the effect of middle alternative in GSS abortion attitude measures. American Association of Public Opinion Research (AAPOR) Annual Meeting.
The Likert scale has been applied to many questionnaires and surveys after Likert introduced his 5-point summative rating scale to measure attitude in 1932. However, the question remains regarding the middle response category endorsement in attitude measures. Krosnick and Presser (2010) summarized the potential satisficing effect when a survey included the middle point response category (i.e., an odd number of response format). They also pointed out offering a midpoint may be useful in increasing reliability and validity if respondents’ attitude toward the question is neutral or unclear. Therefore, the purpose of this study is to evaluate and compare four different number response formats (i.e., 4, 5, 6, and 7 categories) in terms of latent classes and constructs using General Social Survey (GSS) abortion items.
A total of 1,338 participants were randomly assigned to one of four sets of questions. These sets only differed in the number of response categories and this study only analyzed 6 GSS items related to abortion circumstances. The factor mixture model (FMM) that combined latent class and factor analysis (Muthen, 2008) was used. In addition, we applied both Bayesian model parameter estimations and the maximum likelihood estimation with standard errors and mean-adjusted chi-square test which we can compare results from different estimators. The findings regarding the differences in terms of the number of latent classes revealed and similarities of latent constructs are discussed.
LaRoche, K.J., Haus, K.R., Maier, J.M., Kaplan, A.M., & Jozkowski, K.N.(2021, May 11-14). "I wasn’t really thinking about myself”: Exploring who participants think about when answering survey questions about abortion. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Abortion attitudes may differ depending on who participants think about when responding to survey questions. In a pilot survey assessing abortion attitudes in English (n=31) and Spanish (n=25), we randomly assigned participants to answer questions that asked them to think about themselves/themselves and their partner, or a hypothetical woman. We conducted follow-up cognitive interviews (CIs) and used content and thematic analysis to assess how participants arrived at their answers. Then, using NORC’s AmeriSpeak panel (N=608), we administered a revised survey, assigning participants to the same conditions, and asked them who they thought about when answering questions. Based on who participants reported thinking about, we categorized participants as fully, somewhat, or not at all compliant. Descriptive statistics and chi-squared tests were used to assess compliance across survey conditions. Regression will be used to examine participants’ demographic characteristics and compliance. In the CIs, participants described using a variety of strategies when responding to the survey questions. Many drew on their own experiences even if they were asked to think about someone else, and vice versa. Findings from the survey data indicate that 21.5 to 28.6% of participants did not consider wording of the survey at all when responding to items (non-compliance) and 20.8% to 39.9% reported mixed compliance. There was a significant difference in survey condition (χ2 = 101.24, df = 2, p < .001), such that people assigned to the “you” condition were more likely to think of themselves than those assigned to other conditions. Our findings suggest that people draw on various personal experiences and cognitive strategies when answering questions about abortion, and many people are unable to separate thoughts about themselves from other people in this context. Low rates of compliance with survey instructions have implications for the development of polling items related to abortion.
Montenegro, M., Maier, J., Kaplan, A., Jozkowski, K.N., Crawford, B.L., Turner, R.C., Lo, W.J. (2021, May 11-14). The influence of religion on Latinx Spanish and English speakers’ abortion attitudes. American Association of Public Opinion Research (AAPOR) Annual Meeting.
Background: A vast majority of Latinx people in the US identify with a religious group and report some religiosity level; both religious denomination and religiosity are linked to people’s attitudes toward abortion. For instance, scholars have found that higher degrees of religiosity and identifying as Catholic and evangelical are associated with less support for abortion. While there is research on the relationship between religiosity and abortion attitudes in the general population, there is a dearth of research focusing specifically on the US Latinx population. This study examines the nuance of whether and how religion influences Latinx’s views on abortion.
Method: This analysis is part of a larger cognitive interview study examining Spanish (n = 25) and English-speaking (n = 73) participants’ understanding of a salient belief elicitation survey about abortion. Participants answered open-ended survey questions related to abortion, followed by an interview assessing how they arrived at their answers. We conducted a thematic analysis on Latinx participants’ interviews conducted in English and Spanish (n = 32), examining the role of religion in shaping Latinx participants’ abortion attitudes.
Results: Participants described varying degrees of religious beliefs, which they indicated, directly and indirectly, affected their abortion views. Many participants identified as religious and therefore believed abortion is killing a life given by God. Alternatively, a significant minority indicated that although they do not identify as religious, religion permeates the Latinx culture and indirectly influences their thoughts on abortion. For example, although they support abortion, they see it as something shameful.
Conclusion: While religion informs participants’ attitudes toward abortion, some participants’ views on abortion are nuanced, complex, and sometimes contradictory. More research is needed to examine how communities largely influenced by religious beliefs and norms (e.g., Latinx community) conceptualize abortion and reconcile their religious beliefs with their abortion attitudes.