From Learning to Action: Reflections from Eval17

I am home from Eval17 and wanted to reflect on my experiences at the conference. There were many interesting sessions I attended, but three things have really stuck with me.

Evaluators as Knowledge Brokers

I intentionally attended multiple sessions exploring knowledge brokering—and especially evaluators as knowledge brokering. This is something my advisor has considered for some time and those of you who follow me closely on Twitter know that I am taking up her call for research on the topic by choosing to study it for my dissertation.

Volunteering at Eval17 with fellow students!

Our field has focused on evaluation use for decades, but it doesn’t seem until recently that researchers began focusing on research use after realizing much of their research remains in the ivory tower of academia. They have just “discovered” Weiss’s taxonomy of types of use and are exploring methods to bridge the research-practice gap. However, as I will argue in my dissertation, evaluators may be better suited to act as knowledge brokers between researchers and practitioners than researchers themselves. Many evaluators have one foot in the world of research and another in practice (with their mind, body, and spirit situated firmly in evaluation!) and have the interpersonal competencies, know participatory styles of evaluation, and have the ability to understand and adapt to the local context that can best enable them to live in both communities of practitioners and researchers. Researchers could learn a lot from evaluators! This is the argument that I earned me the AEA Student Travel Award.

Evaluator Competencies, Guiding Principles, and Professionalization

In the past couple years, the AEA board created two task forces, one that focused on revising the AEA Guiding Principles (which are supposed to be revisited every 5 years) and another to develop a set of evaluator competencies. These guidelines—in conjunction with the evaluation standards—serve as guides to inform our practice.

Hanging with recovering AEA presidents: Stewart Donaldson and John Gargani

There were many sessions from the two task forces to update AEA membership. The competencies task force is currently finalizing the set of core competencies, which include the domains of professional practice, methodology, context, planning and management, and interpersonal. Results from the membership survey indicate strongest endorsement of competencies that deal with ethical practice within each domain. Not surprisingly, “conducts informal and forma meta-evaluations of studies, identifying their strengths and limitations” and “obtains internal or external evaluation work” were lowest rated items. Our field has not demonstrated much interest in or use of meta-evaluation.

The guiding principles task force is in the early stages of revising the guiding principles. Interestingly, they are proposing to add a fifth guiding principle of equity. As I mentioned in my feedback during the session, I am not sure whether or how it differs from the “respect for people” domain or “respect for public and general welfare” domain (which they recommend renaming to public interest and social benefit). Others mentioned not being sure whether equity should always be a guiding principle (i.e., warranting a separate domain) or if it is sometimes a guiding principle (i.e., it is a sub-component of another domain). Furthermore, equity or equality? There is much work to be done on this and I am not sure how the field will receive these changes.

My advisors: Tiffany Berry and Tarek Azzam

Some evaluators are wary of these changes, particularly citing access and equity as necessary things to consider as we develop our field. However, most seem wary about using the competencies as a move towards professionalization. The competencies task force is not charged with doing anything more than developing a set of competencies, gaining feedback, and providing feedback to the AEA board (and AEA membership) for approval. That has not stopped some evaluators from being very vocal against the competencies. I appreciate their critical feedback, but at the same time I think the competencies will be beneficial in and of themselves even if there is no further work in professionalizing the field.

I very much enjoyed the invigorating conversations on these topics at Eval17 and can’t wait to hear about further work from the task forces between now and Eval18.

Networking

Some members of Twitter lunch! Chelsea Hetherington, Laura Sundstrom, Elizabeth Grim, and Lyn Swackhamer

Through Twitter, I have made many friends in the evaluation community. Since I have finished coursework, Twitter has been a source of professional development and helping me stay abreast of the latest development and news in the field worldwide. I had the pleasure of meeting many Twitter evaluators at Eval17—including Tom Archibald, Nicole Clark, Kylie Hutchinson, Sheila Robinson, Deven Wisner, and many more that I can recall—whom I am not sure I would have met without our connections on Twitter.

I also continued the tradition of having an evaluation Twitter lunch this year. Eight other evaluators joined me at lunch to discuss the benefits of Twitter, how we use Twitter (especially to inform our practice), and how we can help the field get onto Twitter to invigorate the #eval conversations. Through this lunch, we have decided to put out numerous blog posts, host a Twitter chat, and even propose a panel or demonstration session for Eval18 to educate evaluators on the benefits of Twitter and how to use it for professional development.

5 thoughts on “From Learning to Action: Reflections from Eval17

  1. Great blog, Dana — and I really appreciated your tweets throughout the conference. You covered sessions I couldn’t get to. Thanks much. MQP

  2. I love the idea of developing an Eval18 session on the benefits of Twitter for professional development! Both Twitter (and blogging) have been immensely useful in the building of my consulting practice.

    Regarding the guiding principles: I can see the “respect for people” and “respect for public and general welfare” domains being sub-domains housed under an “Equity” domain. Also, this post from Everyday Feminism can be useful for those wanting to know the difference between equity and equality. Equality is treating everyone the same whereas equity is giving everyone what they need to be successful: https://everydayfeminism.com/2014/09/equality-is-not-enough/

    1. Interesting. I think the guiding principles task force would be interested in that perspective! I mentioned to them that there is an interesting cultural difference in prioritizing equity versus equality between the US versus Britain.

  3. As someone who came in to program evaluation by way of social work, I am energized by the evaluation profession increasing their attention on equity! I think it is vital to include underrepresented voice from day one of any work with a group. I felt more connected to the AEA as a group of my peers after Melvin Hall’s incredible talk last week at the conference. It is energizing to see the profession recognizing our role to advance social equity,

Leave a Reply

Your email address will not be published. Required fields are marked *