Sophie Ma Zhu
Doctor of Philosophy in Measurement, Evaluation and Research Methodology (PhD)
Validity evidence for self-report psychological measures based on participants' response processes
The field of measurement is a graduate area of study that does not exist at the undergraduate level. This is not statistics but rather focuses on test/measure/instrument development, reliability, and validation. Many students don't discover this field until they take a (usually required) course at the graduate level and are often surprised by how fascinating this area can be and how they can actually effect change in basic or applied research or clinical practice for children, adolescents, adults, or vulnerable groups. I am interested in working with any student who wants to learn more about psychological or health measurement but am particularly interested in students who would like to explore the relatively new area of 'response processes' as a source of validity evidence. This is a topic that could be of interest to students in measurement or counselling psychology. Response processes examine what is going on for a person when they complete a test, measure, or task and whether that test, measure or task adequately captures the person's experience of the phenomenon (e.g., depression, quality of life, visuo-spatial ability, mindfulness, self-esteem) being measured and if the person's understanding of the test/measure/task matches that of the test developer and user.
G+PS regularly provides virtual sessions that focus on admission requirements and procedures and tips how to improve your application.
No abstract available.
Evidence based on content is one of the key sources of validity evidence described in the Standards for Educational and Psychological Testing. This dissertation used three groups of content experts to gather validity evidence based on content for two sections of the Quality of Life for Homeless and Hard-to-House Individuals (QoLHHI) Survey. One goal of this dissertation was to examine the implications of this evidence for the validity of inferences made from the QoLHHI, while a second goal was to determine if judgmental studies using content experts, a popular method for content validation, can be improved. Quantitative and descriptive validity evidence was collected from 11 subject matter experts (researchers working with individuals who are homeless and vulnerably housed (HVH)), 16 experiential experts (individuals who were HVH), and 8 practical experts (individuals who had administered the QoLHHI). These experts independently rated the relevance and technical quality of the items, response scales, administration instructions and other content elements of the QoLHHI. Content Validity Indices were computed and used to identify elements that were endorsed by the experts. Experts also provided descriptive feedback in the form of comments and suggestions for improving the QoLHHI content. This feedback was used to develop recommendations for revising the content.In terms of evidence based on content for the QoLHHI, over 85% of the content elements were endorsed, indicating good evidence for validity. The recommended revisions are relatively straightforward to implement. Overall, the content of these two sections of the QoLHHI appears likely to produce scores that lead to supportable and appropriate inferences. With regards to methodology, a new approach to content validation is proposed. Traditionally, content validation studies have focused on quantitative assessments, with descriptive data being given a secondary role. It is proposed here that content experts should be viewed as an advisory board rather than a representative sample of a larger population, that members’ feedback should be given individual attention, and that judgmental studies should focus primarily on the experts’ descriptive feedback, which is where their knowledge, experience, and insight are best expressed.
Until recently, concern with body shape and image has primarily been considered a female domain and body image concerns among men were all but ignored. Researchers are now beginning to address this gap in the literature, as evidenced by the rapid rise in the number of studies pertaining to male body image. However, there are currently only a limited number of body image measures available that attempt to tap male body image concern, with these measures predominantly focusing only on the drive for muscularity. As a result, these measures are likely excluding other aspects important to men’s body image. The purpose of the current dissertation was to develop a multidimensional measure of male body image, named the Multidimensional Male Body Concerns Questionnaire (MMBCQ), and determine whether it yields reliable scores and valid interpretations. This purpose was achieved through a series of three studies. The first study consisted of a qualitative investigation that identified nine major aspects of men’s bodies and appearance that are most important to their body image. The second study developed items to assess these nine dimensions. A pilot test then reduced the original 55 items to a total of 39 items, which assessed the dimensions of muscularity, body fat, youthfulness, body hair, and penis. The third study refined the MMBCQ to a total of 35 items and provided additional reliability and validity evidence for the MMBCQ subscales. This new measure will allow researchers to extend their understanding of the male body image construct beyond muscularity alone. Through the use of a mixed methods approach and a combination of CTT and IRT, this dissertation uses modern validity theory to provide a comprehensive model of test development and validation.
Substance Use Disorders (SUDs) are a significant and growing problem for Canadians; low rates of SUD treatment entry and retention suggest that our SUD treatment system requires improvement. Fear of being stigmatized by healthcare professionals is a barrier for treatment-seeking and, despite recent changes in societal attitudes towards substance use, individuals with SUDs continue to be stigmatized. Most counsellors are likely to see clients with SUDs during their careers and their attitudes towards these clients will strongly influence the therapeutic value of these encounters, which can range from helpful to harmful. Little is known about counsellors’ attitudes towards working with clients with SUDs and even less is known about the factors that contribute to them, thus providing a rationale for the current study. Counsellors were recruited from an online Canadian directory (n = 263) and data were collected using an online survey. To measure attitudes, the Medical Condition Regard Scale, modified for use with counsellors, was completed twice: once for clients with SUDs and once for clients with Major Depressive Disorder (MDD). Attitudes were mostly positive towards clients with SUDs, although 11 participants reported negative attitudes. A one-way repeated measures ANOVA found that overall scores were significantly more positive for the MCRS-MDD. However, the items of the MCRS-MDD that yielded more positive responses were all related to the difficulty of treating the disorder. We used one-way between-groups ANOVAs as well as bivariate correlation coefficients to determine which factors had significant bivariate relationships with MCRS-SUD scores. A standard ordinary least squares multiple regression was run to determine which of these variables still had an effect in the presence of the other statistically significant variables. Frequency of contact with clients with SUDs, workplace situational constraints, feelings of anger towards people with SUDs, and age remained as significant variables. The implications of these findings for research and counselling training and practice are discussed.
Although the benefits of antiretroviral therapy (ART) on the quality of life of people living with HIV/AIDS (PLWHA) who use illicit drugs have been well-described, sub-optimal adherence to ART remains an ongoing health concern. Positive health outcomes associated with adherence self-efficacy have been well-documented but there is a paucity of research exploring the antecedents of this construct. This study sought to identify possible determinants of adherence self-efficacy among a cohort of PLWHA who use illicit drugs. From December 2005 to June 2014, data were collected from the AIDS Care Cohort to evaluate Exposure to Survival Services (ACCESS), a prospective cohort of adult PLWHA who use illicit drugs in Vancouver, Canada. We used multivariate linear regression to identify factors independently associated with higher ART adherence self-efficacy. Among 520 participants, older age and Caucasian ethnicity significantly and independently predicted higher levels of ART adherence efficacy expectations and self-regulatory efficacy, being employed predicted only higher levels of ART adherence efficacy expectations, whereas daily cocaine injection and being homeless significantly and independently predicted lower levels of ART adherence self-regulatory efficacy. Although this study highlights the potential importance that some demographic and contextual factors play in shaping adherence self-efficacy within the context of HIV and illicit drug use, efforts should be made to address methodological inconsistencies when investigating self-efficacy across different populations, including PLWHA. Future findings in this area introduce the notion of implementing screening measures for those at-risk for low ART adherence self-efficacy, as well as tailoring psychotherapy among PLWHA to increase aspects of adherence self-efficacy.
Background: Quality of Life (QoL) and functional outcomes appear to be markers of mental health but little is known about their trajectory in the early stages of Bipolar Disorder Type I (BD I). Early phase depression appears to have a significant influence on such outcomes. This study observed the trajectory of QoL and functional outcomes and examined if (a) number of prodromal depressive episodes (prior to the first episode of mania) (b) the total number of episodes of depression experienced by 3 years, and (c) severity of ongoing depressive symptomatology, impacted QoL and functioning, across the first 3 years following the first episode of mania in a cohort of individuals with BD I. Methods: 40 individuals diagnosed with BD I and recovering from their first episode of mania were recruited from an overarching clinical program based in a hospital setting. Participants completed assessments for QoL (the Quality of Life Enjoyment and Satisfaction Questionnaire), functionality (the Multidimensional Scale of Independent Functioning), and depressive symptomology (Hamilton Depression Rating Scale) annually from baseline to year 3. Any history of prodromal and recurrent depressive episodes was also recorded. Results: QoL and functionality were observed to improve between baseline and year 1, and then stabilize up to year 3. OLS regression was utilized to assess the predictors of QoL and functional outcomes. Higher severity of depressive symptomology at baseline and year 3 were found to be predictive of lower QoL at 3 years following the onset of illness. No depression variables were significantly related to QoL or functionality difference scores between year 3 and baseline.Conclusion: This is the first longitudinal study to (a) examine the trajectory of QoL and functionality in the first 3 years of illness in a First Episode Mania (FEM) sample of individuals with BD I and (b) examine the relationship of multiple depression variables to QoL at year 3 as well as change in QoL and functionality from baseline to year 3. The findings support the need for treatment to focus on the management of depressive symptomology in the early stages of BD I, which in turn may enhance QoL.
The Rosenberg Self-Esteem scale (RSES) is a commonly used measure, cited over 3,000 times in the past five years. The aim of this study was to produce a synthesis of the available sources of reliability and validity evidence for the RSES as classified by the Standards for Educational and Psychological Testing (AERA, APA & NCME, 2014). Despite the popularity of the RSES, only 27 articles have examined reliability and validity evidence for the scale. This study showed that the most prevalent source of reliability is based on internal consistency and the most prevalent validity evidence is based on internal structure, followed by relations to other variables. The latter source of evidence primarily consisted of convergent validity evidence. Evidence based on response processes is seldom examined and no studies examined validity evidence based on content or consequences of testing. When examining reliability, internal structure, and relations to other variables, studies tended to overlook the implications of the order in which these factors are studied. There is also a need for researchers to clearly state assumptions and criteria to interpret findings as well as more clarity in the reporting of results. The implications of these findings for researchers interested in the use of the RSES and for measurement experts will be discussed.
This thesis had two primary goals. The first was to examine methods and procedures that researchers use in the process of validating the Satisfaction With Life Scale (SWLS) (Diener, Emmons, Larsen & Griffen, 1985). The second was to suggest a framework to organize and examine validation procedures presented in support of measures found across disciplines and journals. A literature search using the PsycINFO database from 1985 through July 2012 was conducted to capture all published validation studies of the SWLS. Each article was coded for reliability and validity evidence (i.e., test content, response processes, internal structure, relations to other variables, and consequences of testing) as described in the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999). Each area was further broken down into elements specific to each area to account for the rationale for theoretical decisions made, procedures used, and the completeness of the reporting of validation procedures presented. Results indicate that validation studies for the SWLS focused on internal structure and relations to other variables. Relations to other variables evidence consisted mainly of convergent evidence. Where statistical analyses were conducted, criterion values for interpretation of results were rarely provided. A greater understanding is needed of what constitutes evidence of relations to other variables; how to conceptualize this evidence and provide a rationale for constructs, measures and variables used; and how to describe expected relationships and subsequently evaluate the evidence.