Assessment
Grant applications are assessed for relevance and quality as described in the call for grant applications. A call for grant applications is part of a programme. Programmes focus on relevant social and scientific topics that have an impact on health and wellbeing.
The assessment of and decision-making process regarding grant applications submitted to ZonMw must not be influenced by prejudice or bias. To this end, ZonMw implements the ‘Code for dealing with personal interests’. This code of conduct applies to all individuals participating in the consultation and decision-making process regarding the distribution of grant funds, such as members of committees, evaluators and ZonMw employees. The Code for dealing with personal interests describes how ZonMw handles personal interests in the grant distribution process. It ensures that individuals with a personal interest in specific grant applications cannot influence the decision-making process.
ZonMw employees
ZonMw employees verify that grant applications are complete and comply with the conditions of the grant round. They check for usage of the prescribed language, the correct format, and whether the application was submitted before the deadline. If an application is incomplete in one or several aspects, the applicant is offered a single chance to resolve the issues (if possible).
Evaluators and experts by experience
After the ZonMw check, external experts (evaluators) determine the extent to which grant applications comply with the quality criteria in the call for grant applications. Exceptions are made, for instance where low grant sums are involved. The number of evaluators assessing an application depends on the grant sum. The evaluators who assess grant applications are competent and not involved in the applications in any way. Evaluators assess no more than one or a few applications.
Right of reply
We anonymise the evaluators’ quality assessments and forward them to the applicants. The latter are then invited to submit a response in writing (right of reply). In their response, applicants can reply to the evaluators’ queries and respond to any criticisms.
Experts by experience
In some calls for grant applications, an assessment by a patient panel is part of the procedure. Such panels evaluate the relevance of an application. Programme committees also generally have at least one member with first-hand experience.
Programme committee
A programme committee is appointed by the Board of Directors and the Board bears final responsibility for their work. Committee members are appointed as individuals; they do not represent an organisation. ZonMw considers the members’ expertise when establishing a committee. In every programme committee, we strive for representation of (the perspective of) research, education, practice and policy. It is also important that at least one member has first-hand experience. The members of a programme committee are generally listed on the programme page associated with the call for grant applications.
Project idea and advice
After the deadline has passed, the members of the programme committee evaluate the relevance and overall quality of the project ideas. In their evaluation, the members comply with the criteria set forth in the call for grant applications. This step aims to select projects that are the best match for the programme and represent the highest chance of success. It also intends to ensure that the number of project ideas elaborated in a grant application is in line with the available budget.
Detailed application
ZonMw sends all application files submitted to a call for grant applications to the members of the programme committee. Every file contains the grant application, evaluators’ opinions, and the applicant’s reply to queries. The committee issues a final opinion on the quality and relevance of each application that is based on the file. The committee then compiles a list of applications in their order of priority. The order of priority determines whether an application is approved or rejected. ZonMw then makes a decision and sends all applicants a grant approval or rejection letter. The letter also presents the opinion concerning the application content prepared by the programme committee.
Quality assessment
The committee establishes its quality assessment on the basis of the grant application, the evaluators’ opinions, and the applicant’s reply to queries. The committee may deviate from the evaluators’ opinion but it must motivate its opinion and comply with the relevant criteria. If the committee finds that the evaluators did not motivate their opinions to an acceptable extent, it may ignore these opinions. If the committee issues new arguments, the applicant is invited to reply once again.
Relevance assessment
The committee assesses a grant application’s relevance on the basis of the application and, if applicable, the opinions of experts by experience and the applicant’s reply to queries. The committee adheres to the criteria set forth in the call for grant applications in this respect. If there was a project idea stage, the committee also looks at the assessment of the project idea. The assessment must be consistent.
Prioritisation of grant applications
If there are more applications that can be approved than the available budget, the committee will compile a list in their order of priority. It does so primarily on the basis of the final assessment concerning relevance and quality. The prioritisation matrix in the call for grant applications is used here. The committee then looks at aspects such as distribution across areas of interest and other priorities defined in the programme.
Code for dealing with personal interests
It is possible that committee members are active in the research or its execution and as such are involved in grant applications. ZonMw implements a stringent policy to prevent conflicts of interests. To this end, it has established the Code for dealing with personal interests.
Assessment criteria
ZonMw works with a number of general relevance and quality criteria. These are taken into account in all ZonMw programmes. When establishing programme committees, ZonMw makes sure that the committee includes adequate expertise with regard to the criteria. In addition to the general assessment criteria, there are programme-specific criteria that are described in the call for grant applications.
Relevance Criteria
Diversity
The criteria establish how attention is devoted to diversity and target group differentiation based on characteristics such as sex and gender, age, socio-economic situation, level of education, migration and cultural backgrounds, and gender diversity, insofar as these are relevant to the project theme.
Participation of patients and/or end users
Another criterion is how stakeholders, the target group or end users who have experiential expertise are involved in projects. In concrete terms, ‘involve’ here means consulting those concerned, seeking their advice and collaboration and/or involving them in decision-making with regard to the drafting of the grant application and the implementation of the project.
Application in terms of impact
Projects funded by ZonMw must have an impact. New knowledge and expertise must be implemented in practice, policy, education and/or further research.
Quality criteria
With its quality criteria, ZonMw sets requirements for the quality of the various aspects of the application:
Objective and research question or remit
An evaluation will be carried out regarding:
- Clarity. The research question and/or objective must be formulated in a concrete and verifiable manner.
- Scope. What is the significance of the subject matter? Is the theoretical substantiation adequate? What knowledge and experience are already available and what will the application add to this?
- Originality. A grant application must not be a repetition of earlier or ongoing projects unless such repetition adds value, for instance if a study occurs in a new setting or context, or in case of a broader implementation of a previous project.
Strategy
The strategy must be clear and adequate for the remit. It describes the chosen methods and analyses, including theoretical and/or empirical substantiation.
Feasibility
It must be credible that the goal of the application can be achieved within the timeframe set out and with the expertise, manpower, facilities and resources that are available. The project description must devote attention to stimulating and obstructive factors that may occur during project implementation. If applicable, the involvement of parties, target groups and intermediary target groups must be ensured.
Project group or individual
With regard to the quality of the project group or individual who will execute the application, their experience and production in recent years and their impact are relevant. Production covers matters such as publications, reports, guidelines, protocols and interventions. Another indicator are grants obtained and (inter)national contacts with peers and target groups.
However, promising new talent also has a good chance of getting grant application approval. In some programmes, only the qualities of the applicant are determining factors in the assessment of the application’s quality. These are the so-called personal grants, which are presently only provided through ZonMw’s open programmes.