2017-18 Evaluation of After-School All-Stars, Los Angeles
This evaluation will examine program quality, attendance, academic outcomes, and positive developmental outcomes of a large, multi-site after-school program in Los Angeles, CA. Specifically, we are focusing on the reasons why youth join the program and how their motivations for staying in the program change as a result of their experience in the program. For this evaluation, I am the project manager under Dr. Tiffany Berry at the Claremont Evaluation Center with two evaluation associates.
Quasi-Experimental Evaluation of Bright Prospect
This evaluation follows entering freshmen, prior to participation in Bright Prospect, and exiting seniors in high school to examine (a) predictors of who participates in Bright Prospect and (b) how psychosocial (e.g., non-cognitive, socioemotional) skills and resources affect high school outcomes and college matriculation, persistence, and graduation. For this evaluation, I am the project manager under Dr. Nazanin Zargarpour at the Claremont Evaluation Center with two evaluation associates.
Changing Expectations – Coding Makerspace
Changing Expectations provides community-based STEM education services to youth in the Austin, TX area. Coding Makerspace was created in response to President Obama’s My Brother’s Keeper, STEM for ALL, and Computer Science for ALL (CSforALL) initiatives. The Coding Makerspace provides hands-on digital and coding projects and creates the space and opportunity for young men of color to learn computer science and cybersecurity. The evaluation focuses on the experiences and short-term outcomes of the dozen young men of color that participate.
Research and Conference Papers
Improving Evidence Use: A Mixed Methods Study Examining Relationship Quality in Research-Practice Partnerships (Dissertation)
At the heart of research-practice partnerships (RPPs) is the importance of relationships among researchers and practitioners. However, while research on RPPs often mention the importance of long-term, mutualistic, and trusting relationships, little research has empirically examined these relationships directly and how partnership relationships lead to subsequent evidence use. The fields of evaluation (e.g., utilization-focused evaluation) and social psychology (e.g., interpersonal relationship theories such as 3+1Cs) have much to add to the current research on evidence use. Using these frameworks, this study will test how and under what conditions relationships among partners in youth-related RPPs across the nation lead to greater evidence use. This study will survey multiple members—including both the researchers and the intended practitioner users—of partnerships on perceptions of relationship quality, tasks used to promote evidence use, and intended or actual conceptual, process, and instrumental use of evidence findings. Follow-up interviews will be conducted with partnerships characterized by high and low quality relationships to further determine the factors that determine relationship quality. Through this research, the investigators seek to provide existing and future partnerships with the information needed on how to foster high quality relationships among partners so that evidence may better be used among practitioners.
Data Visualization in Evaluation
In this working group, we are conducting a series of studies to see research data visualization in evaluation. Our first study is on logic models and how narratives and data visualization principles can enhance–or inhibit–the visual efficiency, aesthetics, and credibility of logic models. The second model is in the initial stages, but we are examining the visualization of effect sizes. We are presenting the first study at Eval17.
Relationships among Non-Cognitive Factors and Academic Performance (AERA 2018)
We use structural equation modeling to test the Consortium on Chicago School Research (Farrington et al., 2012) model of how non-cognitive factors affect academic performance. We found support for the model; however, academic perseverance was not significantly related to academic performance in the context of other non-cognitive factors. Furthermore, we examine differences between freshmen and seniors to determine preliminary developmental changes across high school. The paper was written for presentation at the AERA 2018 conference.
Leveraging Attendance Data in After-School Programs
Attendance data are often not leveraged to their maximum potential. This paper proposes several analytical strategies for leveraging attendance data for program improvement efforts. Specifically, we show how attendance data can be used to answer several meaningful evaluation questions (i.e., Are enough students consistently attending? Who is attending? Does attendance improve as quality of programming improves? Does more consistent attendance improve youth outcomes?). When attendance data is analyzed and combined with other data sources, it can be used to improve the quality and impact of after-school programs.
Politics in Evaluation
Evaluation is a political act. Surveying AEA members, we seek to highlight some of the common political situations that emerge during different stages of the evaluation. We hope to help better understand how politics affects practice and help us recognize the different ways that these situations can be addressed in practice.
Predictors of Grit using Multilevel Modeling
Much of the research on the popular construct of grit consists of what grit predicts (e.g., academic success, persistence), but little is known about what predicts grit. Using data from YouthTruth‘s School Experience Survey, three sets of predictors (i.e., school demographics, student demographics, student experiences) are examined through multilevel modeling to see what has the strongest predictive relationship of grit.
Aesthetic Experiences as Flow: Relationships with Well-Being
This study examines how the aesthetic experience (i.e., the attitudes, perceptions, experiences, or acts of attention involved in viewing art) are related to Csikszentmihalyi’s (1990) conceptualization of flow. Csikszentmiahlyi & Robinson (1990) defined aesthetic experiences as having the same content of flow but differ from other flow experiences by four artistic related dimensions. Thus, we created the Aesthetic Experience Questionnaire, tested its convergent validity, explored how aesthetic experiences differ across persons, and tested whether aesthetic experiences, like flow, relate to well-being. Overall, results offer preliminary evidence that aesthetic experiences relate to well-being and there’s support that aesthetic experiences are a flow experience.
From College Access to Success: Importance of Psychosocial Competencies for Minority Students in College (AERA 2017)
This paper was presented at AERA 2017. We are currently writing it up for publication. The paper highlights the effective principles of practice that led to college success for one particular college access program.