Evaluation

action guide

Home
>
Action Guide
/

Evaluation

Evaluation: Begin with the End in Mind

Evaluation, broadly defined as the act of assessing or appraising, ensures accountability for benchmarks on the kind of change your collective action brings about. Accompanying the section on each of the four Action Guide prongs is guidance to help you assess and reflect on decisions reached, actions taken, and progress made toward long-term sustainability of your chosen interventions. Such assessments will enable you to change your strategy or tactics when you encounter barriers.

When you start using this four-pronged strategy for sustainability, among the associated tasks you and your team will face are: 1) devising a method for integrating the diverse program efforts across topic areas; 2) devising some measures to ascertain whether the programs or interventions you seek to sustain are worth sustaining and 3) determining the aspects of sustainability you plan to target.

You should be able to answer these questions:
- What criteria will you use to crosswalk similar programs and interventions?
- What evidence exists that the program or intervention you are promoting is likely to lead to positive impacts in your school and community?
- What results will define your progress toward sustainability of the interventions in your initiative?
- Are some of greater priority than others?
It will be important to identify small victories on the way to achieving your goal; sustaining commitment to the “big picture” will necessitate some recognition of more modest achievements that move things in the right direction.

Determining how to evaluate progress toward your long-term goal of sustaining child development and prevention interventions should be part of your early collective conversations as this creates a roadmap for all subsequent decisions. Measuring progress toward your goal allows you to reflect on whether your objectives are being met, how the process led there, what was successful and what was not, and to identify improvements in your actions. Engaging in an ongoing assessment of progress becomes an organizing frame from which to connect the multiple activities proposed by this Guide, fosters engagement with diverse stakeholders, and supports a shared understanding of the purpose behind the collective movement.

If you begin with the end in mind you improve the chances of achieving your desired outcomes.

Starting from the assumption that your goal is sustainability, there are two shared premises:

That your evidence-based programs (see graphic on the “Definitions & Terms” page) are or can be part of a public health approach using a framework of universal and comprehensive supports and social-emotional development interventions.
That these public health programs will deliver maximum benefits if they are able to sustain their activities over time.

The Action Guide promotes sustainability through policy and systems change, and through community engagement. With complex systems and policy change of the kind this Action Guide promotes, “…evaluation, conducted from a utilization-focused perspective, facilitates ongoing innovation by helping those engaged in (the) innovation to examine the effects of their actions…”^[1]

From our definition of sustainability, i.e., “the continued use of program components and activities for the continued achievement of desirable program and population outcomes”^[2] – you and your Team will customize the four-pronged strategy and identify what your own measures of success will be, and over what general time frame. Together you will develop a process or concept for identifying some “small victories,” recognizing that the specifics may change as opportunities emerge or disappear.

For additional tips and tools on the basic principles of designing and implementing an evaluation approach, see tools and resources on sidebar.

[1] [Adapted from] Patton, MQ. (1997). Utilization-focused evaluation: The new century text. 3^rd Ed. Thousand Oaks, CA: Sage; Patton M.Q. (2011). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and use.
[2] Scheirer, M.A., & Dearing, J.W. (2011). An Agenda for Research on the Sustainability of Public Health Programs. American Journal of Public Health, 101(11), p. 2060.

Guiding Questions

As you pursue sustainability of child development and mental health promotion activities, can you describe the change you are advocating? What are the short-term and long-term results or outcomes that will define the success of your initiative?
How will you know that population, organizational, or system changes are occurring?
What factors might impact the development, implementation, challenges, and successes of the initiative?
What data exists that will help you measure progress over time? What new information will be required to help track movement toward your goal and your objectives?
What lessons are being learned as you implement the initiative? How can these lessons be helpful in refining your strategy or shifting your actions?
How might the impact of the initiative be captured and communicated? To whom should this communication be targeted?

Key Actions

Identify the key stakeholders who should participate in the identification of necessary information, the collection of data, and monitoring of progress in your community-based initiative.
- Some of these individuals may be part of your Action Team – or they may be from outside organizations that can contribute their expertise to this part of your initiative.
- Consider potential allies from other settings (i.e., universities or local colleges, private businesses, faith communities) who may be willing and able partners.
Articulate how your small leadership team will be nested in a larger partnership or coalition. What organizational ties will you create? What coalitions and coordinating teams will be essential to your work plan?
Articulate the overarching goal that represents “sustainability” for your initiative, the measurable objectives, and the numerous activities that need to be conducted to achieve success.
- That goal should be a broad statement about the partnership’s purpose – what brings you all together as one group.
- Objectives are the specific, measurable steps that will be necessary in order to achieve the group’s collective vision. All members of your Team should contribute to these ideas and strategies.
- Activities are the tasks that are necessary to accomplish your objectives. Having an assigned member of your Team responsible for leading an activity and a target date for completing that task allow for accountability and transparency in this process.
Develop feedback loops for your Team that summarize progress toward your stated objectives in ‘real time.’ Regularly soliciting input from your Team on the successes or challenges associated with specific activities allows for adjustments to be made in response to unanticipated changes.
Communicate with various stakeholders.
- After conducting activities, compile information and summarize your lessons learned in a compelling story. It is important to share both your process and your findings to internal and external audiences to strengthen their commitment to your initiative.
- Refer to the Using your Findings and the Communications section for ways in which information could be communicated to key decision-makers.