Evidence checklist to assess funding applications: Using Evidence for Impact

Print document | Download PDF

Evidence checklist to assess funding applications

March 2016

The Social Policy Evaluation and Research Unit’s (Superu’s) purpose is to increase the use of evidence by people across the social sector so that they can make better decisions – about funding, policies or services – to improve the lives of New Zealanders, New Zealand communities, families and whānau.

The Using Evidence for Impact project takes a big-picture approach and aims to inspire all those working in the New Zealand social sector to use evidence in decision-making. The objectives behind the programme are to drive:

• greater accessibility to evidence

• greater transparency of evidence

• capability development and good practice in using evidence. 

 

This checklist supports decision-makers as they review funding applications. It describes the scope of supporting evidence necessary to make an informed decision.

 

Topic Questions Probes
Outcomes What will this initiative achieve? What are the intended outcomes?
Needs assessment What needs does this proposal address? What is the problem it is attempting to solve? What is the nature, size and severity of the problem, (e.g. number of affected people, economic impact)?
Whose needs/who will benefit? Is the target group easily identifiable?
How will they benefit/what will they gain?
What is the evidence that this need exists? Is there data or convincing evidence of a local/ New Zealand need?
What do we understand about the root causes of the need? What does the evidence tell us about factors that contribute to the problem that the proposal addresses, (e.g. if the need relates to obesity, what is known about causes of obesity)?
Intervention logic What will this initiative do? Intervention logic: what are the inputs, activities, outputs and outcomes?
How will it achieve the outcomes sought? What is the rationale for believing that these activities will lead to these outcomes? What are the assumptions underpinning the theory of change?
Are adverse impacts likely? Will any group of people be affected negatively by the initiative?
What possible unintended outcomes have you considered? Could there be negative outcomes?
Evidence for effectiveness Why was this initiative selected in preference to alternatives? Is there a rationale for selecting the preferred choice over others? What other options did you consider?
Are there similar initiatives existing overseas or in New Zealand (e.g. is this an extension/adjustment of an existing programme)? If yes... What does the evidence tell us about their impact on outcomes and cost effectiveness?
How strong is the evidence? (e.g. quality of evidence suggesting that intervention can impact outcomes, such as random control trials, matched comparison group studies, pre-and-post studies, or qualitative findings). What is the quantity of evidence? (e.g. a number of studies or systematic reviews indicating impact)
How relevant is the evidence? (e.g. studies conducted in different socio-economic, cultural, demographic, policy and legislative contexts)
If there are no similar initiatives... Is there evidence for effectiveness of components of the proposed initiative? (if so discuss the findings, the strength of evidence, and the relevance of the evidence)
What are stakeholders’ and experts’ views on the likely effectiveness?
Is there a commitment to collect sufficient evidence of impact in the future (Through evaluation and/or other research/monitoring – is this demonstrated in the ‘evaluation plan’ - see below)?
How will the initiative fit with the local/ New Zealand context? How will it fit with other government or non-government initiatives? What added value will it provide? Will it displace other activities?
How will it fit with our cultural, demographic, socio- economic, and legislative context? Will it reach the right people in an appropriate way?
Implementation When replicating an evidence-based initiative developed elsewhere ... What are the initiative’s core components that must be present for it to work?
What changes will make it more suitable to the local context, without compromising the core components?
Are there resources available to help with implementation (such as guidance materials or access to experts)?
How will you ensure that there is sufficient capability and capacity to deliver it effectively? Are the organisations and staff that will be involved in delivery ready? If not, what support will be provided?
Will resources be sufficient to ensure that quality implementation is supported?
Evaluation plan How will you evaluate it? [Please note: the scale and nature of the evaluation plan should vary depending on the size of initiative and its existing evidence base.] How will you measure outcomes, and the extent to which the initiative contributed to those outcomes?
How will you evaluate the quality of delivery?
How will the evaluation feed into improving the initiative?
How will you assess the value of the initiative and whether it was a good use of resources?

 

The checklist is specifically about evidence and does not replace the need to appraise other aspects such as strategic fit and implementation plans.

 

back-to-top |

back-to-top |
    Last update: 2 May 2016