At the occasion of the 25th PRACE Council Meeting in Amsterdam, the PRACE Members ratified a Resolution to proceed with the second phase of their Partnership: PRACE 2. The PRACE 2 programme defines the second period of PRACE from 2017 to 2020. With this agreement, PRACE will strengthen Europe’s position as world-class scientific supercomputing provider, a technology considered a key enabler for knowledge development, scientific research, big data analytics, solving global and societal challenges, and European industrial competitiveness.
See the corresponding press release here.
Annex IV of this Resolution explains the Peer Review Process under the PRACE 2 Programme.
The PRACE peer review process is based on the following principles:
- Transparency: The peer review process is transparent and clear to all stakeholders in PRACE including funding agencies of all member countries and users from research institutions and industry.
- Fairness: Proposals are evaluated on merit and potential high impact on European and international science and economy.
- No parallel assessment: The PRACE peer review process builds on the experiences and best practices of national and international institutions and constitutes a centralised peer review exercise recognised by all PRACE partner countries and scientific communities.
- Reviews are done by experts in the scientific field of the proposal, with no declared conflict of interest, based on criteria published in the PRACE Calls for Proposals and with a periodic reshuffling of reviewer’s appointments.
- Confidentiality: Proposals will be treated with the needed confidentiality by PRACE staff and reviewers. The identities of the peer reviewers shall not be disclosed.
- Right to reply to technical and scientific evaluations.
The PRACE Peer review includes two types of assessment: technical and scientific. The two assessments are carried out separately by different groups of experts with the applicant having the right to reply to the evaluation outcome of both the technical and the scientific reviewers (if necessary). The technical review seeks to assure that the proposal is technically feasible for the system(s) requested. At the end of the peer review process, all applicants, whether their proposal is successful or not, will receive documentation of both the technical and scientific evaluation of their proposal.
1. Technical assessment
All proposals will undergo a technical assessment. The technical assessment can result in three outcomes:
- Accepted. The application fulfills all technical requirements to run on the selected system(s).
- Conditionally accepted. The application does not meet the technical requirements to run in the selected system(s), but the technical reviewers can identify the measure and time frame necessary to meet them.
- Rejected. The application does not meet the minimum technical requirements to run in the selected system(s).
Criteria for technical assessment
A proposal must convincingly address the following criteria:
- The need to use PRACE resources.
- Software availability on the requested resource. The codes necessary for the project must already be available on the system requested, or in case of codes developed by the applicants and Multi-year and Project Access proposals, they must have been sufficiently tested for efficiency, high scalability, and suitability. For Multi-year and Project Access Proposals proof of successful tests must be submitted together with the proposal.
- Feasibility of the requested resource. The requested system(s) must be suitable for the proposed project. The technical assessment may redirect projects to a more appropriate system.
These criteria should be fully addressed in the application. Reviewers will assess proposals against these criteria.
2. Scientific assessment
Scientific review is performed by internationally recognized experts in the field of research of the proposal. During the scientific assessment an increase or decrease in the requested resources can be recommended for consideration during resource allocation.
Criteria for scientific assessment
The proposals must address the following scientific criteria:
- Scientific excellence. The proposed research must demonstrate scientific excellence and a potential for high European and international impact.
- Novelty and transformative qualities. Proposals should be novel, develop an important scientific topic of major relevance to European research, describe possible transformative aspects, and expected advances.
- Relevance to the call. Proposal needs to address how the proposed project is addressing the scope of the call if a specific scope is stated in the call.
- Methodology. The methodology used should be described and appropriate to achieve the goals of the project.
- Dissemination. The planned channels and resources for dissemination and knowledge exchange should be described.
- Management. There must be a solid management structure which will ensure that the project will be successfully completed.
These criteria should be fully addressed in the application. Reviewers will assess proposals against these criteria. Find here the guidelines for scientific reviewers.
Role of reviewers
The PRACE staff handling peer review is responsible for obtaining high quality non-conflicted technical and scientific reviews for each proposal.
The technical reviewers are experienced technical experts in PRACE systems and software. Technical reviewers will provide a report on the technical quality (code, methodology, etc.) of proposals and on the suitability of the project for the system/architecture requested, and the extent of the request.
Scientific reviewers will provide a report on the novelty and quality of the science proposed, its potential impact and on the applicant’s ability to carry it out. They will also assess the proposal based on its relevance to any pre-defined criteria for a particular call.
The scientific reviewers are chosen from among researchers with expertise in the scientific field of the proposal. One of the reviewers, but no more than one, may be chosen from the list of reviewers suggested by the applicant. The PRACE staff managing the peer review process will seek to assure that each proposal is evaluated by non-conflicted reviewers that are experts in at least one aspect of the proposal and together can provide an expert view in the whole of the scientific case presented.
The identity of the technical and scientific reviewers will be known only to the PRACE peer review staff and anonymous to the applicant(s) and others involved in the process.
The reviewers will be required to keep personal details of the applicants and the details of their proposals confidential. In exceptional cases it may be necessary for all personnel and reviewers to sign a Non-Disclosure Agreement (NDA).
Please read the PRACE Conflict of Interest and Confidentiality Policy.
Applicant right to reply
If the reviewers have queries and/or raise issues regarding a proposal, the reviewer’s report (technical and scientific) will be sent to the applicants for an opportunity to respond, correcting any factual inaccuracies or providing any necessary further information in response to issues that the reviewers may have raised. The reviewer report will be made anonymous before being sent to applicants. The applicant will be given a deadline to respond to the reviewers’ comments. Applicant’s responses to the reviewer’s reports will be sent for resource allocation together with the reviewers’ reports.