Social Innovation in Health
Monitoring &
Evaluation Framework

Regional Social Innovation in Health Contacts

SOCIAL INNOVATION IN HEALTH INITIATIVE HUB NAME (URL)	SIHI HUB CONTACT
China (SESH)	Dr Weiming Tang
Colombia (CIDEIM)	Mrs María Isabel Echavarría Mejía
Ghana (University of Ghana / TDR Africa Regional Training Center)	Dr Phyllis Dako-Gyeke
Honduras (UNAH)	Dr Jackeline Alger
Indonesia (Universitas Gadjah Mada)	Dr Yodi Mahendradhata
Malawi (University of Malawi)	Dr Don Mathanga
Philippines (University of Philippines Manila)	Dr Noel Juban
Nigeria (Nnamdi Azikiwe University Awka)	Dr Obioma Nwaorgu
Rwanda (University of Rwanda)	Dr David Tumusiime
South Africa (University of Cape Town Bertha Centre)	Ms Katusha de Villiers
Uganda (Makerere University)	Dr Phyllis Awor

Social
Innovation
in Health
Initiative

SOCIAL INNOVATION IN HEALTH MONITORING & EVALUATION FRAMEWORK

Phase 7
Changing systems: integrating the social innovation into routine practice

Key Issues	Specific techniques/research methods	Open access resources
Create marketing/dissemination strategy Document and share the process to create the social innovation in health Document and broadly share the impact of the social innovation in health Develop and strengthen partnerships with supporting market actors who can sustain the model Create systems and/or integrate into existing systems and structures to institutionalize the social innovation Identify unmet needs Adaptation and replication in other settings	Documenting best practices: methods proven reliable to lead desirable results Impact analysis: assess the results of the project Delphi techniques: a structured way to aggregate knowledge from a group of people, often used in health guideline development Mathematical modelling: using mathematical formulations to guide problems	WHO Best Practices Guide: documents criteria and format to analyse best practices Impact Assessment: guide to conduct an impact analysis Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders Reporting guidelines for modelling studies: evidence-based tools to guide research reporting Nesta’s Funding Innovation – A Practice Guide: a guide to different financial tools to support innovations Nesta’s tools and insights for governments: a guide to improve governance for innovations Anticipatory regulation: an emerging approach that is proactive, iterative and responds to evolving markets Strengthening evaluation capacity: a guide on strengthening the M&E capacity of individuals, organizations, communities and networks

Key Issues

Specific techniques/research methods

Open access resources

Create marketing/dissemination strategy

Document and share the process to create the social innovation in health

Document and broadly share the impact of the social innovation in health

Develop and strengthen partnerships with supporting market actors who can sustain the model

Create systems and/or integrate into existing systems and structures to institutionalize the social innovation

Identify unmet needs

Adaptation and replication in other settings

Documenting best practices: methods proven reliable to lead desirable results

Impact analysis: assess the results of the project

Delphi techniques: a structured way to aggregate knowledge from a group of people, often used in health guideline development

Mathematical modelling: using mathematical formulations to guide problems

WHO Best Practices Guide: documents criteria and format to analyse best practices

Impact Assessment: guide to conduct an impact analysis

Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders

Reporting guidelines for modelling studies: evidence-based tools to guide research reporting

Nesta’s Funding Innovation – A Practice Guide: a guide to different financial tools to support innovations

Nesta’s tools and insights for governments: a guide to improve governance for innovations

Anticipatory regulation: an emerging approach that is proactive, iterative and responds to evolving markets

Strengthening evaluation capacity: a guide on strengthening the M&E capacity of individuals, organizations, communities and networks

Phase 6
Growing and scaling: scaling up impact or scaling down for local relevance

Key Issues	Specific techniques/research methods	Open access resources
Define longer-term metrics of impact, including fidelity and related implementation metrics Assess potential to scale down to a specific local context or be adapted for another context Assess potential for dissemination of innovation to different contexts and cultures Develop impact and sustainability metrics with stakeholders and end users	Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation guideline development – dissemination of ideas and best practices Implementation research: implementation across subunits/network identi-fication/confirmatory inferences designs/effectiveness-implementation hybrids Qualitative research: ethnographic study, in-depth interviews, other methods Discrete choice experiments: quantitative measure to elicit individual preferences on innovations	Effectiveness-Implementation Hybrid Designs: assess both clinical outcomes and implementation Social Network Analysis: guide to map relationships and networks in communities Adaptive trial designs: guide on designing adaptive clinical trials Discrete Choice Experiment: WHO guide on conducting discrete choice experiments Scaling-up guidance & tools: ExpandNet resources to guide sustainable scale up of innovations

Key Issues

Specific techniques/research methods

Open access resources

Define longer-term metrics of impact, including fidelity and related implementation metrics

Assess potential to scale down to a specific local context or be adapted for another context

Assess potential for dissemination of innovation to different contexts and cultures

Develop impact and sustainability metrics with stakeholders and end users

Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation guideline development – dissemination of ideas and best practices

Implementation research: implementation across subunits/network identi-fication/confirmatory inferences designs/effectiveness-implementation hybrids

Qualitative research: ethnographic study, in-depth interviews, other methods

Discrete choice experiments: quantitative measure to elicit individual preferences on innovations

Effectiveness-Implementation Hybrid Designs: assess both clinical outcomes and implementation

Social Network Analysis: guide to map relationships and networks in communities

Adaptive trial designs: guide on designing adaptive clinical trials

Discrete Choice Experiment: WHO guide on conducting discrete choice experiments

Scaling-up guidance & tools: ExpandNet resources to guide sustainable scale up of innovations

Phase 5
Delivering and implementing: expanding a social innovation project and/or research study to show impact

Key Issues	Specific techniques/research methods	Open access resources
Continue to test different combinations of interventions while collecting metrics on performance and impact Identify aspects of innovation that can be expanded and possible contexts for application Develop communications strategy and risk management plans Integrate community voices and stakeholder engagement (including public officials, local leaders) Communicate findings to all participants and stakeholders Include cost-benefit analysis and an investment plan	Process evaluation: process to determine the development and implementation strategy as planned Economic evaluation: more rigorous economic analyses to examine costs and outcomes (for example, cost-effectiveness) Qualitative research: ethnographic study, in-depth interviews, other methods Participatory action research: qualitative research methods to learn about and engage with participants Time series analysis: serial and comparative interrupted time series analysis Cluster randomized trials: groups are randomized into control and intervention arms Randomized controlled trials: participants are randomly allocated to receive the intervention Observational studies: observing the effect of an intervention without interfering with the exposure or outcome Implementation research: scientific inquiry into questions concerning implementation	Community Based Participatory Action Research: research framework to be conducted in communities Interrupted time series analysis tutorial: stepwise study design to evaluate population-level study at a given point in time Serial and comparative interrupted time series analysis: quantitative measure guide where outcome is evaluated at multiple time points Cluster-randomized trials: Cochrane information guide Observational studies: brief guide to different types of observational studies Nesta – Running RCTs: introductory guide on RCTs in innovation Public and social innovation labs: teams, units and funds dedicated to structuring and embedding innovation methods and practice in government to tackle social and public problems Nesta – People Powered Results: a 100-day challenge methodology that is structured and combined with coaching support

Key Issues

Specific techniques/research methods

Open access resources

Continue to test different combinations of interventions while collecting metrics on performance and impact

Identify aspects of innovation that can be expanded and possible contexts for application

Develop communications strategy and risk management plans

Integrate community voices and stakeholder engagement (including public officials, local leaders)

Communicate findings to all participants and stakeholders

Include cost-benefit analysis and an investment plan

Process evaluation: process to determine the development and implementation strategy as planned

Economic evaluation: more rigorous economic analyses to examine costs and outcomes (for example,
cost-effectiveness)

Qualitative research: ethnographic study, in-depth interviews, other methods

Participatory action research: qualitative research methods to learn about and engage with participants

Time series analysis: serial and comparative interrupted time series analysis

Cluster randomized trials: groups are randomized into control and intervention arms

Randomized controlled trials: participants are randomly allocated to receive the intervention

Observational studies: observing the effect of an intervention without interfering with the exposure or outcome

Implementation research: scientific inquiry into questions concerning implementation

Community Based Participatory Action Research: research framework to be conducted in communities

Interrupted time series analysis tutorial: stepwise study design to evaluate population-level study at a given point in time

Serial and comparative interrupted time series analysis: quantitative measure guide where outcome is evaluated at multiple time points

Cluster-randomized trials: Cochrane information guide

Observational studies: brief guide to different types of observational studies

Nesta – Running RCTs: introductory guide on RCTs in innovation

Public and social innovation labs: teams, units and funds dedicated to structuring and embedding innovation methods and practice in government to tackle social and public problems

Nesta – People Powered Results: a 100-day challenge methodology that is structured and combined with coaching support

Phase 4
Making the case: evaluating a social innovation project and/or research study

Key Issues	Specific techniques/research methods	Open access resources
Assess performance of social innovation based on pre-established criteria, especially: Community Acceptability Usability Social and cultural impact Effectiveness Accessibility Feasibility Resources Potential sustainability Potential scalability Adaptability Reliance on other systems	Observational studies: non-randomized studies of an innovation (for example, quasi-experimental studies, ecological studies) Implementation research: scientific inquiry into questions concerning implementation Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods, participant observations Preliminary economic evaluation: costing studies in the local context Randomized controlled trials: participants are randomly allocated to receive the intervention	Implementation Research – A Practical Guide: a guide on implementation research for multiple stakeholders B Impact Assessment: guide to measure impact of a social business Evaluating a Social Innovation: a developmental evaluation approach with an adaptive process Social Innovation Evaluation Toolbox: a tool to assess impact of investment and management of social innovations Cochrane Risk of Bias Tool: information of bias in study designs TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes RCT Consort: consolidated standards for reporting randomized controlled trials Nesta – Running RCTs: introductory guide on RCTs in innovation

Key Issues

Specific techniques/research methods

Open access resources

Assess performance of social innovation based on pre-established criteria, especially:

Community Acceptability
Usability
Social and cultural impact
Effectiveness
Accessibility
Feasibility
Resources
Potential sustainability
Potential scalability
Adaptability
Reliance on other systems

Observational studies: non-randomized studies of an innovation (for example, quasi-experimental studies, ecological studies)

Implementation research: scientific inquiry into questions concerning implementation

Social impact assessment: investigates social effects (that is, non-medical) on end-user population from the innovation

Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods, participant observations

Preliminary economic evaluation: costing studies in the local context

Randomized controlled trials: participants are randomly allocated to receive the intervention

Implementation Research – A Practical Guide: a guide on implementation research for multiple stakeholders

B Impact Assessment: guide to measure impact of a social business

Evaluating a Social Innovation: a developmental evaluation approach with an adaptive process

Social Innovation Evaluation Toolbox: a tool to assess impact of investment and management of social innovations

Cochrane Risk of Bias Tool: information of bias in study designs

TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes

RCT Consort: consolidated standards for reporting randomized controlled trials

Nesta – Running RCTs: introductory guide on RCTs in innovation

Phase 3
Developing and testing: piloting a social innovation project and/or research study

Key Issues	Specific techniques/research methods	Open access resources
Test different ideas by building tangible products or creating experiences to gather feedback from end users, project delivery staff and key stakeholders Create clear and actionable metrics to test the social innovation Satisfy regulatory and ethical requirements, consider risks and socio-cultural/environmental/ economic/political impact Create a database to capture data about feasibility (given the local context), effectiveness and other key variables	Process evaluation: process to determine the development and implementation strategy as planned Usability: tests the fitness and ease of access of the social innovation Prototype research: studying a sample/design to test the concept or process User-design research: user-focused design studies to meet end users’ needs Intercept interviews: a qualitative market research to collect feedback from consumers	Qualitative Evaluation Checklist: guidance to use qualitative techniques for evaluation purposes Nesta Prototype Framework: structured prototyping toolkit The Most Significant Change Technique: participatory M&E method using systematic participatory interpretation of stories Ethical issues: describes ethical issues associated with open calls TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research

Key Issues

Specific techniques/research methods

Open access resources

Test different ideas by building tangible products or creating experiences to gather feedback from end users, project delivery staff and key stakeholders

Create clear and actionable metrics to test the social innovation

Satisfy regulatory and ethical requirements, consider risks and socio-cultural/environmental/ economic/political impact

Create a database to capture data about feasibility (given the local context), effectiveness and other key variables

Process evaluation: process to determine the development and implementation strategy as planned

Usability: tests the fitness and ease of access of the social innovation

Prototype research: studying a sample/design to test the concept or process

User-design research: user-focused design studies to meet end users’ needs

Intercept interviews: a qualitative market research to collect feedback from consumers

Qualitative Evaluation Checklist: guidance to use qualitative techniques for evaluation purposes

Nesta Prototype Framework: structured prototyping toolkit

The Most Significant Change Technique: participatory M&E method using systematic participatory interpretation of stories

Ethical issues: describes ethical issues associated with open calls

TDR MOOC on implementation research: an open online training to design and demonstrate robust IR projects for better health outcomes

TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research

Phase 2
Generating ideas: developing a social innovation project and/or research study

Key Issues	Specific techniques/research methods	Open access resources
Build your team with end users, leaders and public and private sector partners Specify the end user population as much as possible and strengthen collaborations with end users Identify overall objectives and aims in partnership with community members Develop your research objective, research design, project plan, roadmap and change theory Build on existing structures, available resources and stakeholder analysis (if applicable) Identify social innovation metrics (key progress indicators) that are specific, measurable, achievable, relevant and culturally appropriate Develop an iterative planning, monitoring, evaluation and feedback plan	Open challenge contests: public open calls to solicit creative solutions Hackathons (also called designathons): a time-bound event to bring together collective ideas to solve a problem User led design-thinking studies: an iterative design-based process/tool that identifies relevant and beneficial ways to meet end users’ needs Participatory learning and action: qualitative research methods to learn about and engage with participants Results chain: develop a visual model that illustrates how from input, through process, outputs and expected outcomes the innovation will contribute to the impact	Design for Health: a design process to develop effective solutions Crowdsourcing Clinic: practical tools on open calls Crowdsourcing in Health – A Practical Guide: guidance on open calls Development Impact & You Innovative Toolkit: practical tools to trigger & support social innovations Participatory Approaches: guide on using participatory methods for impact evaluation Community Based Participatory Action Research: research approach to enhance community-based research Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders TDR Massive Open Online Course (MOOC) on Implementation Research (IR): an open online training to design and demonstrate robust IR projects for better health outcomes TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research Results Chain: a guide for enterprises on creating a theory of change and results chain

Key Issues

Specific techniques/research methods

Open access resources

Build your team with end users, leaders and public and private sector partners

Specify the end user population as much as possible and strengthen collaborations with end users

Identify overall objectives and aims in partnership with community members

Develop your research objective, research design, project plan, roadmap and change theory

Build on existing structures, available resources and stakeholder analysis (if applicable)

Identify social innovation metrics (key progress indicators) that are specific, measurable, achievable, relevant and culturally appropriate

Develop an iterative planning, monitoring, evaluation and feedback plan

Open challenge contests: public open calls to solicit creative solutions

Hackathons (also called designathons): a time-bound event to bring together collective ideas to solve a problem

User led design-thinking studies: an iterative design-based process/tool that identifies relevant and beneficial ways to meet end users’ needs

Participatory learning and action: qualitative research methods to learn about and engage with participants

Results chain: develop a visual model that illustrates how from input, through process, outputs and expected outcomes the innovation will contribute to the impact

Design for Health: a design process to develop effective solutions

Crowdsourcing Clinic: practical tools on open calls

Crowdsourcing in Health – A Practical Guide: guidance on open calls

Development Impact & You Innovative Toolkit: practical tools to trigger & support social innovations

Participatory Approaches: guide on using participatory methods for impact evaluation

Community Based Participatory Action Research: research approach to enhance community-based research

Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders

TDR Massive Open Online Course (MOOC) on Implementation Research (IR): an open online training to design and demonstrate robust IR projects for better health outcomes

TDR Implementation Research Toolkit: seven non-linear learning modules on implementation research

Results Chain: a guide for enterprises on creating a theory of change and results chain

Phase 1
Opportunities and challenges: identifying a need and opportunity for social innovation in health

Key Issues	Specific techniques/research methods	Open access resources
Identify an unmet need according to end users, community and partners (co-creation items highlighted in bold throughout) Identify a group of innovators, end users, partners and key individuals through a stakeholder analysis Consider the opportunities in terms of settings, key stakeholders, beneficiaries (and potential for co-creation), power structures, technology and sustainability Consider problems in terms of necessary staffing, funding, time, populations and partnerships for sustainability Identify existing struggling innovators who can benefit from seed financing, project structuring and design support or linkage with partners that they have limited access to	Scoping reviews: structured literature review synthesizing pre-specified eligibility criteria and existing knowledge Systematic reviews: more extensive literature review with pre-specified eligibility criteria and potential for pooled results Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods Landscape analysis: policy analysis about needs and opportunities Observational studies: observing the effect of an intervention without interfering with the exposure or outcome	Cochrane Library: systematic reviews PRISMA Statement on Systematic Reviews and Meta Analyses: research checklist Conducting Concerns Surveys: structured way to get community feedback Development Impact & You: Nesta’s toolkit on practical tools to support and trigger ideas for better results Problem Analysis: checklist to discuss and solve problems Qualitative Research Guidelines Project: a comprehensive guide for designing, writing, reviewing and reporting qualitative research Community Based Participatory Research Approach: research approach to enhance community-based research Crowdsourcing Clinic: practical tools on crowdsourcing Crowdsourcing in Health – A Practical Guide: guidance on crowdsourcing Design for Health: a design process to develop effective solutions Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders

Key Issues

Specific techniques/research methods

Open access resources

Identify an unmet need according to end users, community and partners (co-creation items highlighted in bold throughout)

Identify a group of innovators, end users, partners and key individuals through a stakeholder analysis

Consider the opportunities in terms of settings, key stakeholders, beneficiaries (and potential for co-creation), power structures, technology and sustainability

Consider problems in terms of necessary staffing, funding, time, populations and partnerships for sustainability

Identify existing struggling innovators who can benefit from seed financing, project structuring and design support or linkage with partners that they have limited access to

Scoping reviews: structured literature review synthesizing pre-specified eligibility criteria and existing knowledge

Systematic reviews: more extensive literature review with pre-specified eligibility criteria and potential for pooled results

Qualitative research: ethnographic study, case study, phenomenology, discourse analysis, grounded theory, in-depth interviews, mixed methods

Landscape analysis: policy analysis about needs and opportunities

Observational studies: observing the effect of an intervention without interfering with the exposure or outcome

Cochrane Library: systematic reviews

PRISMA Statement on Systematic Reviews and Meta Analyses: research checklist

Conducting Concerns Surveys: structured way to get community feedback

Development Impact & You: Nesta’s toolkit on practical tools to support and trigger ideas for better results

Problem Analysis: checklist to discuss and solve problems

Qualitative Research Guidelines Project: a comprehensive guide for designing, writing, reviewing and reporting qualitative research

Community Based Participatory Research Approach: research approach to enhance community-based research

Crowdsourcing Clinic: practical tools on crowdsourcing

Crowdsourcing in Health – A Practical Guide: guidance on crowdsourcing

Design for Health: a design process to develop effective solutions

Implementation Research in Health – A Practical Guide: a guide on implementation research for multiple stakeholders

¹ Methods need to be adapted to the research questions and multiple methods can be applied at the same time.

Social Innovation in Health Monitoring & Evaluation Framework

Social Innovation in Health Monitoring & Evaluation Framework

Regional Social Innovation in Health Contacts

SOCIAL INNOVATION IN HEALTHINITIATIVE HUB NAME (URL)

SIHI HUB CONTACT

SocialInnovationin HealthInitiative

Social Innovation in Health

Monitoring & Evaluation Framework

Phase 7 Changing systems: integrating the social innovation into routine practice

Key Issues

Specific techniques/research methods

Open access resources

Phase 6 Growing and scaling: scaling up impact or scaling down for local relevance

Key Issues

Specific techniques/research methods

Open access resources

Phase 5 Delivering and implementing: expanding a social innovation project and/or research study to show impact

Key Issues

Specific techniques/research methods

Open access resources

Phase 4 Making the case: evaluating a social innovation project and/or research study

Key Issues

Specific techniques/research methods

Open access resources

Phase 3 Developing and testing: piloting a social innovation project and/or research study

Key Issues

Specific techniques/research methods

Open access resources

Phase 2 Generating ideas: developing a social innovation project and/or research study

Key Issues

Specific techniques/research methods

Open access resources

Phase 1 Opportunities and challenges: identifying a need and opportunity for social innovation in health

Key Issues

Specific techniques/research methods

Open access resources

Social Innovation in Health
Monitoring &
Evaluation Framework

Social Innovation in Health
Monitoring &
Evaluation Framework

SOCIAL INNOVATION IN HEALTH
INITIATIVE HUB NAME (URL)

Social
Innovation
in Health
Initiative

Monitoring &
Evaluation Framework

Phase 7
Changing systems: integrating the social innovation into routine practice

Phase 6
Growing and scaling: scaling up impact or scaling down for local relevance

Phase 5
Delivering and implementing: expanding a social innovation project and/or research study to show impact

Phase 4
Making the case: evaluating a social innovation project and/or research study

Phase 3
Developing and testing: piloting a social innovation project and/or research study

Phase 2
Generating ideas: developing a social innovation project and/or research study

Phase 1
Opportunities and challenges: identifying a need and opportunity for social innovation in health