SOCIAL INNOVATION IN HEALTH MONITORING & EVALUATION FRAMEWORK
Creative Commons by © Copyright World Health Organization (WHO), 2021. All Rights Reserved.
SOCIAL INNOVATION IN HEALTH MONITORING & EVALUATION FRAMEWORK
Creative Commons by © Copyright World Health Organization (WHO), 2021. All Rights Reserved.
Key Issues | Specific techniques/research methods | Open access resources |
|---|---|---|
Create marketing/dissemination strategy Document and share the process to create the social innovation in health Document and broadly share the impact of the social innovation in health Develop and strengthen partnerships with supporting market actors who can sustain the model Create systems and/or integrate into existing systems and structures to institutionalize the social innovation Identify unmet needs Adaptation and replication in other settings | Documenting best practices: methods proven reliable to lead desirable results Impact analysis: assess the results of the project Delphi techniques: a structured way to aggregate knowledge from a group of people, often used in health guideline development Mathematical modelling: using mathematical formulations to guide problems |
WHO Best Practices Guide: documents criteria and format to analyse best practices Impact Assessment: guide to conduct an impact analysis Nesta’s tools and insights for governments: a guide to improve governance for innovations Anticipatory regulation: an emerging approach that is proactive, iterative and responds to evolving markets Strengthening evaluation capacity: a guide on strengthening the M&E capacity of individuals, organizations, communities and networks |
Key Issues | Specific techniques/research methods | Open access resources |
|---|---|---|
Define longer-term metrics of impact, including fidelity and related implementation metrics Assess potential to scale down to a specific local context or be adapted for another context Assess potential for dissemination of innovation to different contexts and cultures Develop impact and sustainability metrics with stakeholders and end users | Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation guideline development – dissemination of ideas and best practices Implementation research: implementation across subunits/network identi-fication/confirmatory inferences designs/effectiveness-implementation hybrids Qualitative research: ethnographic study, in-depth interviews, other methods Discrete choice experiments: quantitative measure to elicit individual preferences on innovations | Effectiveness-Implementation Hybrid Designs: assess both clinical outcomes and implementation Social Network Analysis: guide to map relationships and networks in communities Adaptive trial designs: guide on designing adaptive clinical trials Discrete Choice Experiment: WHO guide on conducting discrete choice experiments Scaling-up guidance & tools: ExpandNet resources to guide sustainable scale up of innovations |
Key Issues | Specific techniques/research methods | Open access resources |
|---|---|---|
Continue to test different combinations of interventions while collecting metrics on performance and impact Identify aspects of innovation that can be expanded and possible contexts for application Develop communications strategy and risk management plans Integrate community voices and stakeholder engagement (including public officials, local leaders) Communicate findings to all participants and stakeholders Include cost-benefit analysis and an investment plan | Process evaluation: process to determine the development and implementation strategy as planned Economic evaluation: more rigorous economic analyses to examine costs and outcomes (for example, Qualitative research: ethnographic study, in-depth interviews, other methods Participatory action research: qualitative research methods to learn about and engage with participants Time series analysis: serial and comparative interrupted time series analysis Cluster randomized trials: groups are randomized into control and intervention arms Randomized controlled trials: participants are randomly allocated to receive the intervention Observational studies: observing the effect of an intervention without interfering with the exposure or outcome Implementation research: scientific inquiry into questions concerning implementation | Community Based Participatory Action Research: research framework to be conducted in communities Interrupted time series analysis tutorial: stepwise study design to evaluate population-level study at a given point in time Serial and comparative interrupted time series analysis: quantitative measure guide where outcome is evaluated at multiple time points Cluster-randomized trials: Cochrane information guide Observational studies: brief guide to different types of observational studies Nesta – Running RCTs: introductory guide on RCTs in innovation Public and social innovation labs: teams, units and funds dedicated to structuring and embedding innovation methods and practice in government to tackle social and public problems Nesta – People Powered Results: a 100-day challenge methodology that is structured and combined with coaching support |
Key Issues | Specific techniques/research methods | Open access resources |
|---|---|---|
Assess performance of social innovation based on pre-established criteria, especially:
| Observational studies: non-randomized studies of an innovation (for example, quasi-experimental studies, ecological studies) Implementation research: scientific inquiry into questions concerning implementation Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods, participant observations Preliminary economic evaluation: costing studies in the local context Randomized controlled trials: participants are randomly allocated to receive the intervention | Implementation Research – A Practical Guide: a guide on implementation research for multiple stakeholders B Impact Assessment: guide to measure impact of a social business Evaluating a Social Innovation: a developmental evaluation approach with an adaptive process Social Innovation Evaluation Toolbox: a tool to assess impact of investment and management of social innovations Cochrane Risk of Bias Tool: information of bias in study designs TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes RCT Consort: consolidated standards for reporting randomized controlled trials Nesta – Running RCTs: introductory guide on RCTs in innovation |
Key Issues | Specific techniques/research methods | Open access resources |
|---|---|---|
Test different ideas by building tangible products or creating experiences to gather feedback from end users, project delivery staff and key stakeholders Create clear and actionable metrics to test the social innovation Satisfy regulatory and ethical requirements, consider risks and socio-cultural/environmental/ economic/political impact Create a database to capture data about feasibility (given the local context), effectiveness and other key variables | Process evaluation: process to determine the development and implementation strategy as planned Prototype research: studying a sample/design to test the concept or process User-design research: user-focused design studies to meet end users’ needs Intercept interviews: a qualitative market research to collect feedback from consumers | Qualitative Evaluation Checklist: guidance to use qualitative techniques for evaluation purposes Nesta Prototype Framework: structured prototyping toolkit The Most Significant Change Technique: participatory M&E method using systematic participatory interpretation of stories Ethical issues: describes ethical issues associated with open calls TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research |
Key Issues |
Specific techniques/research methods |
Open access resources |
|---|---|---|
|
Build your team with end users, leaders and public and private sector partners Specify the end user population as much as possible and strengthen collaborations with end users Identify overall objectives and aims in partnership with community members Develop your research objective, research design, project plan, roadmap and change theory Build on existing structures, available resources and stakeholder analysis (if applicable) Identify social innovation metrics (key progress indicators) that are specific, measurable, achievable, relevant and culturally appropriate Develop an iterative planning, monitoring, evaluation and feedback plan |
Open challenge contests: public open calls to solicit creative solutions Hackathons (also called designathons): a time-bound event to bring together collective ideas to solve a problem User led design-thinking studies: an iterative design-based process/tool that identifies relevant and beneficial ways to meet end users’ needs Participatory learning and action: qualitative research methods to learn about and engage with participants Results chain: develop a visual model that illustrates how from input, through process, outputs and expected outcomes the innovation will contribute to the impact |
Design for Health: a design process to develop effective solutions Crowdsourcing Clinic: practical tools on open calls Crowdsourcing in Health – A Practical Guide: guidance on open calls Development Impact & You Innovative Toolkit: practical tools to trigger & support social innovations Participatory Approaches: guide on using participatory methods for impact evaluation Community Based Participatory Action Research: research approach to enhance community-based research Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders TDR Massive Open Online Course (MOOC) on Implementation Research (IR): an open online training to design and demonstrate robust IR projects for better health outcomes TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research Results Chain: a guide for enterprises on creating a theory of change and results chain |
Key Issues |
Specific techniques/research methods |
Open access resources |
|---|---|---|
|
Identify an unmet need according to end users, community and partners (co-creation items highlighted in bold throughout) Identify a group of innovators, end users, partners and key individuals through a stakeholder analysis Consider the opportunities in terms of settings, key stakeholders, beneficiaries (and potential for co-creation), power structures, technology and sustainability Consider problems in terms of necessary staffing, funding, time, populations and partnerships for sustainability Identify existing struggling innovators who can benefit from seed financing, project structuring and design support or linkage with partners that they have limited access to |
Scoping reviews: structured literature review synthesizing pre-specified eligibility criteria and existing knowledge Systematic reviews: more extensive literature review with pre-specified eligibility criteria and potential for pooled results Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods Landscape analysis: policy analysis about needs and opportunities Observational studies: observing the effect of an intervention without interfering with the exposure or outcome |
Cochrane Library: systematic reviews PRISMA Statement on Systematic Reviews and Meta Analyses: research checklist Conducting Concerns Surveys: structured way to get community feedback Development Impact & You: Nesta’s toolkit on practical tools to support and trigger ideas for better results Problem Analysis: checklist to discuss and solve problems Qualitative Research Guidelines Project: a comprehensive guide for designing, writing, reviewing and reporting qualitative research Community Based Participatory Research Approach: research approach to enhance community-based research Crowdsourcing Clinic: practical tools on crowdsourcing Crowdsourcing in Health – A Practical Guide: guidance on crowdsourcing Design for Health: a design process to develop effective solutions Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders |
1 Methods need to be adapted to the research questions and multiple methods can be applied at the same time.