In December I was lucky enough to be able to go to the Principles Focused Evaluation Workshop facilitated by TerraLuna’s Nora Murphy. (If you haven’t heard of her, check her out. She’s amazing. TerraLuna Bio is here: http://www.terralunacollaborative.com/nora-murphy/)
Coming from New Orleans, traveling to Minneapolis during December was a treat...a very frozen treat.
One of my personal principles is “to be a proponent of daily, continuous learning, both to myself and others…” so, in spite of the cold, I needed to get on that plane and engage in what would turn out to be the most interesting professional development experiences I have had thus far. I scrounged together layers of warm clothes the day before my flight, calling in my friends from the Northeast to add to my stores of layering pieces, hats, and coats. "Can I
But the workshop was worth the cold.
Okay, all jokes about the cold aside (even if I did biff it on ice walking there the first day), this conference really helped me to reorient my approach as an evaluator, and repositioned my beliefs on what was possible for the scope of evaluations themselves. While it has taken me long enough to do, I am excited to finally write my personal reflection on the workshop, principles focused evaluation, and its potential use in New Orleans.
(Mural on the outside of IntermediaArts, the gallery where TLC used to have co-working space)
The Workshop:
The workshop was two days at the beginning of December in Minneapolis. We worked out of a delightful space called Anahata Studio, that was a little difficult to find at first. But after winding through a series of business and yoga studios, I stumbled upon our home for the next two days. And luckily walked right in for breakfast! There were roughly 12 people in attendance, from a mix of backgrounds -- some were studying evaluation, others were professionals who were moving to the field, and other were longtime evaluators. And we were all there to learn the new method of principles-focused evaluation.
The content we went over was informed by the work of Nora and Michael Quinn Patton. Our materials for the workshop consisted of a Principles workbook created by Nora, and Principles-Focused Evaluation: The GUIDE by Michael.
I’m a pretty reserved person. Without a doubt, the emotional and intellectual depth expected in this workshop was challenging. There was no escaping the close scrutiny of our biases, nothing left un-problematized. I had to stretch myself and focus in a way I’d never felt before in a professional setting.
Trust me.. I'm much more excited than I look. This is my "focused" face!
Now a little more about Principles Focused Evaluation:
First and Foremost: the field of Evaluation is ever-evolving. And methods like Principles are evidence. Actually, principles focused evaluation was created because of the need for differentiation of evaluation methods. Among other things, principles focused evaluation is an intriguing way to address complex dynamic interventions (when the thing you’re measuring won't hold still; for example, when a program will inevitably change through its duration) in evaluation. This makes it an increasingly human way to approach the field of evaluation -- both in theory and practice. Some of these other nontraditional and new directions for evaluation is taking are in: mission fulfillment, strategy, advocacy campaigns, policy change, and systems change. And principles can be applied to all of these areas. An important factor about creating principles focused evaluations is that these evaluations be conducted through the lens of knowing things are interrelated, and that hurdles will emerge that are not anticipated (no matter how well you plan).
As you can guess, principles are at the root of this method of evaluation.
Michael Quinn Patton and Charmagne E. Campbell-Patton define effectiveness principles as,
“...a statement that provides guidance about how to think or behave toward some desired result (either explicit or implicit), based on norms, values, beliefs, experience, and knowledge. The statement is a hypothesis until evaluated within some context to determine its relative meaningfulness, truth, feasibility, and utility for those attempting to follow it” (http://www.cehd.umn.edu/OLPD/MESI/spring/2017/Patton-Principles.pdf).
These principles are created from looking in depth at the organization and taking into account what fundamentally matters. This is important because an evaluation is more likely to be used if intended users find the evaluation meaningful, the questions relevant, and care about the findings. Principles can be created using the GUIDE framework and rubric, which is available in Patton’s new book.
"But...aren’t they just like rules?"
NO.
No, they're not.
“A principle is prescriptive. It provides advice and guidance on what to do, how to think, what to value, and how to act to be effective. It offers direction. The wording is imperative: Do this. The guidance is sufficiently distinct that it can be distinguished from contrary or alternative guidance” (http://www.cehd.umn.edu/OLPD/MESI/spring/2017/Patton-Principles.pdf).
A great example of the difference between the two (from the Patton source listed above) is this:
Rule: “30 minutes of aerobic exercise each day.”
Principle: “Exercise regularly at a level that supports health and is sustainable given your health, life style, age, and capacity.”
An example of this method in practice was conducted by Nora Murphy creating “9 evidence-based, guiding principles to help youth overcome homelessness."
The Principles-focused evaluation...
- Identified principles in draft form
- Collaboratively identified fourteen youth
- Interviewed youth, reviewed their case file, interviewed a nominated staff person
- Synthesized information and wrote case stories
- Reviewed stories with the youth
- Analyzed stories, looking for principles and emergent themes
The resulting principles were:
- Journey oriented: Interact with youth to help them understand the interconnectedness of past, present, and future as they decide where they want to go and how to get there.
- Trauma-informed: Recognize that most homeless youth have experienced trauma; build relationships, responses, and services on that knowledge.
- Non-judgemental: Interact with youth without labeling or judging them on the basis of background, experiences, choices, or behaviors.
- Harm reduction: Contain the effects of risky behavior in the short-term and seek to reduce its effects in the long-term.
- Trusting youth-adult relationships: Build relationships by interacting with youth in an honest, dependable, authentic, caring, and supportive way.
- Strengths-based: Start with and build upon the skills, strengths, and positive characteristics of each youth.
- Positive youth development: Provide opportunities for youth to build a sense of competency, usefulness, belonging, and power.
- Holistic: Support youth in a manner that recognizes the interconnectedness of their mental, physical, spiritual, and social health.
- Collaboration: Establish a principles-based, youth-focused system of support that integrates practices, procedures, and services within and across agencies, systems, and policies.
The reason a method such as principles-focused evaluation was so necessary here was because of the uniqueness of each young person the evaluation would come into contact with-- each homeless youth had their own story, with their own adversity and trauma. Each homeless youth is on a personal journey and “has unique needs, experiences, abilities, and aspirations.” Principles focused evaluation allowed for the creation of principles to “provide guidance and direction to those working with homeless youth...” as well as a framework for how the youth are approached, interacted with, and supported.
The full report can be found here: https://www.terralunacollaborative.com/wp-content/uploads/2014/03/9-Evidence-Based-Principles-to-Help-Youth-Overcome-Homelessness-Webpublish.pdf
If you are as inspired as I was after hearing how influential principles can be... here are some reflective words from Nora Murphy to help you begin to formulate your own principles:
- Who do I need to be?
- How do I live it?
- What is the experience?
- How do I recognize?
- What is the outcome?
How does this affect Flux and evaluation here in NOLA?
This experience was a perfect example of how evaluation is evolving to the needs of the world.
With the field of evaluation still being relatively new to New Orleans, these new techniques are one way to meet some of our local organizations’ needs. The Principles method is especially effective because it looks at the heart of the organization -- the culture, the passions, the goals-- in order to guide the evaluation. New Orleans is a city overflowing with heart, culture, passions, and goals... so what could be a more perfect pair than that!
We here at Flux look forward to the opportunity to apply Principles-Based Evaluation with our present and future partners. We are grateful to the TerraLuna Collaborative for giving us the tools to incorporate this method into our work. Interested? Contact me (katherine@fluxrme.com) for further materials and a consult on whether this approach might fit your organization!
Good Luck and Happy Reflection!