Learning from Experience
Toolkit Home | Phase 1 | Phase 2 | Phase 3 | Phase 4
Once assessment takes place, there also needs to be a process throughout which teams can communicate, discuss, and act upon what they have learned from the process evaluation.
​
In this step, the team needs to establish a process for:
-
Communicating, discussing, and acting upon key findings from the implementation process
-
Sharing implementation process data with stakeholders.
Tools to use during the Learning from Experience phase:
Summative Evaluation (RE-AIM)
Training that focuses on how to use the RE-AIM framework to evaluate an intervention. Specifically, RE-AIM can be used to assess interventions' translatability and public health impact. Re-AIM specifically evaluates:
​
-
Reach into the target population
-
Effectiveness or efficacy
-
Adoption by target staff, settings, systems and communities
-
Implementation – consistency and cost of delivery of intervention
-
Maintenance/sustainment of intervention effects in individuals and settings over time.
Who should use RE-AIM?
All implementation teams would benefit from working through the RE-AIM training. In particular teams working with complicated or untested interventions would benefit from working through the full RE-AIM framework to build a comprehensive evaluation process. ​
​
How does the tool help with Implementation?
As teams develop their interventions, they should use RE-AIM to develop a summative evaluation plan to assess the validity of the interventions throughout the implementation process. Using a thorough evaluation method like RE-AIM will ensure that the intervention meets the needs of all care centers in real time.
​
Resources recommended:
Time: Using the RE-AIM framework to design and implement a full evaluation process for the team's intervention can be time intensive depending on the needs of the team. Time heavy resource.
​
People: Most of the implementation planning team should be involved in developing the evaluation process with input from those directly working with the interventions. Personnel medium resource.
​
Funding: The only cost for the tool itself is the personnel time to create and implement the evaluation process. Funding heavy resource.
​
-
-
Slide deck with introduction to the RE-AIM framework along with training slides that describes each component of the framework along with examples.
-
​
Normalization Process Theory
NoMad, using the Normalization Process Theory, assesses the extent that an intervention has been routinely embedded into an organization’s practice. The measure captures how the work involved in an intervention must be understood, acted upon, and monitored by various stakeholders involved in the intervention. ​NoMad includes four sets of questions that line up with the four components of the Normalization Process Theory.​
​
-
Coherence (C1) – What is the intervention?​
-
Participation (C2) – Who does the work of the intervention​
-
Action (C3) – How does the intervention get done? ​
-
Monitoring (C4) – reflecting on why the intervention was carried out in a particular way. ​
Who should use NoMad?
The NoMAD helps teams measure how integrated an intervention is into organizational practice, helping the team understand how that the intervention might be sustained beyond the initial implementation.​
​
How does the tool help with Implementation?
For the QIF, the NoMAD should be administered during phase 4 as teams are assessing the implementation process, though the information informs the sustainability of the intervention beyond the scope of the QIF. ​The NoMAD can be paired with the PSAT to provide more details on how the intervention is functioning within an organization to gain a full view on the extent to which an intervention may be sustained and the factors that impact sustainability.
​
Resources recommended:
Time: Filling out NoMad questionnaire itself should not take the implementation teams long, but they will need to set time aside tor review and analyzing their findings. Time light resource.
​
People: The core implementation team will be primarily in charge of working on the NoMad. Personnel light resource.
​
Funding: The only cost for the tool itself is the personnel time to complete the review. Funding light resource.
​
-
-
Slide Deck with tool introduction, instructions on how to fill out the survey, and details on each section of questions.
-
​