PRACE Preparatory Access – 35th cut-off evaluation in December 2018

Find below the results of the 35th cut-off evaluation of 3 December 2018 for the PRACE Preparatory Access.

Projects from the following research areas:

 

MHC-TCR complex: a computational study

Project Name: MHC-TCR complex: a computational study
Project leader: Dr Marco D’Abramo
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

In our work we plan to perform several MD simulations of MHC-TCR complex to describe the interactions at atomistic level of details, in a lipid environment and at different physiological conditions. Our goal is to understand if any structural or dynamical components play a major role in the MHC-TCR activation, as well if the recognition generates conformational changes on the receptor, peptide or on MHC I. Beside we will analyze if these changes are affected by the peptide or TCR sequence. The objective is to detect a property which can discriminate between the different interactions. The results might point out new advances in the modern immunology, namely by achieving new insights in the molecular mechanism at the basis of T cell recognition and response to a specific antigen. Due to the high interest in the scientific community about the immune response – especially for the still unanswered “whys” of autoimmune diseases and the alloreactivity – this work can help to unveil these fundamental processes occurring in the immune response. The computational method employed in the current project is the all-atom classical molecular dynamics simulations. Such a technique, able to provide a relatively large sampling of the (bio)molecules conformational behavior, will be used. The reference software package to be used in the project is Gromacs (www.gromacs.org). This molecular dynamics code is very efficient in parallel environments and – depending on the configuration as well as on the size of the simulated system – can be efficiently used to provide a dynamical characterization of very complex systems with high accuracy. Furthermore, the coupling of GPU with CPU provides unprecedented computational efficiency and power needed to efficiently sample the conformational space of very large systems and/or in a very extended time scale. Here, we will focus on MHC/TCR system benchmark to pave the way for an unprecedented description of the behavior of such very complex molecular systems which will be the object of the following PRACE HPC grant request.

top

Preparatory Computations for Ab Initio Modelling of Chemical Vapour Deposition for Efficient Computational Design of New Advanced Coatings

Project Name: Preparatory Computations for Ab Initio Modelling of Chemical Vapour Deposition for Efficient Computational Design of New Advanced Coatings
Project leader: Mr Axel Forslund
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on Curie
Description

This project will aim at facilitate the development of chemical vapour deposition (CVD). CVD is a widely used technique to deposit coatings for a variety of different purposes, including thin films in solar cells, protective wear coatings and as part of thermal barrier coatings. In CVD, reactants in vapour phase form a solid on a substrate. Often, few of the reaction steps and mechanisms during growth are known and in situ experiments to investigate this are hard to conduct. Modelling is therefor important to help gain a better understanding of what might govern the growth of a specific film. During the specific conditions of film growth, generally unstable or meta stable phases might form. Investigation of and understanding such material phases is also a key in gaining knowledge about a deposition process. These phases might be complex and require large systems for computation. Further, CVD is often conducted at high temperatures, demanding large-scale ab initio molecular dynamics. Already these calculations can be extremely computation intensive, and adding to this the requirement of having large systems and the exploration of many configurational variations makes it a very challenging task.

top

Molecular mechanisms of action of dietary antioxidants and chromatin modifying compounds

Project Name: Molecular mechanisms of action of dietary antioxidants and chromatin modifying compounds
Project leader: Dr Andrew Hung
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

The medicinal properties of the leaves and fruit of Olea europaea (olive tree) have been known since antiquity, and consumption of olive oil has been associated with a decreased risk of cardiovascular disease and certain cancers. Increasingly, there is interest in the biological properties of the molecular constituents of olives. For example, hydroxytyrosol has been shown to be a potent antioxidant and has anti-atherogenic and anti-cancer properties. However, the specific constituents responsible for various beneficial effects of olives, as well as their molecular targets, are not well known. The main aim of this project is to use molecular computational modeling and simulation methods to identify key molecular targets of specific bioactive components of olives, and to produce molecular-level characterisation of their mechanisms of action. The outcomes of this project will aid development of novel therapies derived from dietary compounds, which may have substantial advantages over synthetic drugs, including lower dosage requirements and reduced risk of adverse side effects. This project will focus on identifying the mechanisms of action of dietary olive compounds on two specific areas: 1) Epigenetic control: Controlled equilibrium between histone acetylation and deacetylation is essential for normal cell growth, and perturbations have been associated with various diseases. Although numerous compounds have been developed to specifically alter the function of chromatin modifying enzymes (eg. histone deacetylase inhibitors have emerged as potential cancer chemo-therapeutics), we are only at the early stages of understanding the epigenetic effects of dietary compounds. This project area will involve use of computational biophysics approaches, combined with known experimental affinities, to identify potential chromatin modifying compounds derived from Olea europaea. This project will also involve experimental validation of targets using in vitro binding and enzyme activity inhibition assays and identify potential epigenetic effects in a clinical context. 2) Inflammation: Non-steroidal anti-inflammatory drugs (NSAIDs) are among the most widely used therapeutic agents around the world, commonly used to reduce pain. However, there are adverse effects with the use of NSAIDs, including gastrointestinal bleeding and cardiovascular effects. Hence, there has been a rise in the development of alternatives to traditional NSAIDs. A previous study found that oleocanthal, a phenolic compound derived from olive, had similar effects to ibuprofen, a commonly used NSAID. But there is a multitude of additional compounds in olive that have yet to be investigated. Hence, this project will study the mechanisms of olive derived compounds in inflammation using molecular computational modeling and simulation approaches. Enzymes involved in inflammation pathways will be explored as potential targets for olive-derived compounds. Elucidation of the mechanism of action in the inhibition of such enzymes will be valuable in developing novel drugs for the treatment of inflammation.

top

MIMOP (Marie Curie Individual Fellowship Project 793450)

Project Name: MIMOP (Marie Curie Individual Fellowship Project 793450)
Project leader: Dr Louis-Alexandre Couston
Research field: Earth System Sciences
Resource awarded: 100 000 core hours on Hazel Hen
Description

An outstanding problem in polar sciences is how quickly land-based ice moves toward the oceans and contribute to sea-level rise, which is one of the most disruptive consequences of climate change. Acceleration of the outflow from polar ice sheets appears linked to enhanced melting at the ice-shelf—ocean interface. The melting process is, however, poorly understood, because state-of-the-art models cannot resolve the effect of the ocean turbulence or basal roughness of the ice shelves. The project MIMOP (for Modelling Ice-shelf Melting and Ocean Processes) aims to fill this critical gap in our knowledge. MIMOP objectives are to 1) Develop an innovative numerical model of the fluid dynamics of melting at the ice-ocean interface 2) Calibrate the model using new observations of ocean properties beneath an Antarctic ice shelf 3) Determine the sensitivity of melting to changes in temperature and current. The objectives will be achieved by combining a highly-efficient Direct Numerical Simulation (DNS) code with a novel formulation of the equations for the solid/liquid phases of water based on the phase-field method. DNS enables turbulent motions to be simulated without approximation, while the phase-field method allows the ice-ocean interface to be rough and evolve in response to melting. The phase-field method has been applied in metallurgical problems and proof-of-concept simulations have demonstrated its suitability for ice melting. Access to supercomputers in Europe will be critical to running the DNS and achieving the project’s goals.

top

HYDRA-Scale KNL

Project Name: HYDRA-Scale KNL
Project leader: Prof Turlough Downes
Research field: Universe Sciences
Resource awarded: 100 000 core hours on MARCONI -KNL
Description

The multifluid MHD HYDRA code is one of the most advanced codes of its type in the world. It has been used to run the first ever simulations of multifluid turbulence in star forming regions which resulted in several publications in top-rank international astrophysics journals. It was also the main code for the PRACE project “Accretion disk dynamics: the multifluid regime” (PRACE code PRA045) which explored, for the first time, the influence of the multifluid MHD effects known to be crucial to the physics of protoplanetary disks on accretion dynamics. It was also the main code for PRA083 “MF-DISK: Protoplanetary disk dynamics: the multifluid magneto-rotational instability, gaps and jets”. The HYDRA code displayed excellent scaling on the Blue Gene/P system JUGENE, exhibiting strong scaling from 8192 to 294912 cores with 73% efficiency. It also has extremely good scaling characteristics on many other architectures. However, with the continual advancement of available architectures it is necessary to port HYDRA to further systems. Porting should be easy as HYDRA uses no non-standard coding and no non-standard external libraries. In preparation for awarded project access we wish to investigate the scaling on the Marconi-KNL system. In spite of the excellent scaling of HYDRA on all systems tested so far it is both wise and necessary to confirm good scaling on this system which is of an architecture not previously investigated.

top

Setting New Records for Integer Factorisation and Discrete Logarithm

Project Name: Setting New Records for Integer Factorisation and Discrete Logarithm
Project leader: Dr Paul Zimmermann
Research field: Mathematics and Computer Sciences
Resource awarded: 50 000 core hours on MareNostrum, 50 000 core hours on Curie – SKL, 20 000 core hours on JUWELS
Description

The project we aim to submit at the 18th PRACE call aims at setting new records about integer factorization and discrete logarithms records. The current records are respectively 768 bits for integer factorisation (factorisation of RSA-768 in 2009, done by our team with other international partners, https://eprint.iacr.org/2010/006.pdf) and also 768 bits for the discrete logarithm in a prime field (https://link.springer.com/chapter/10.1007/978-3-319-56620-7_7). We aim to break those two records using the open-source CADO-NFS software we develop in our team (http://cado-nfs.gforge.inria.fr/).

top

Understanding the climate variability and predictability at decadal time scale

Project Name: Understanding the climate variability and predictability at decadal time scale
Project leader: Mr Nicolas Lebas
Research field: Earth System Sciences
Resource awarded: 50 000 core hours on Curie – SKL
Description

Preliminary climate projections suggest that the 1.5°C warming target of the Paris Agreement may be reached as early as between 2020 and 2050. It is thus urgent to act but also to reduce uncertainty on this estimation in order to better target the effort necessary. This requires to better understand and predict the near-term (decades) expected changes, notably at the regional scale, and their causes. The latter arise both from internal climate variability as well as externally-forced changes from natural or anthropogenic forcings. Distinguishing between these sources of climate variability and extracting forced climate signals require large ensembles of climate simulations which are very costly in term of computing time . This project is a unique opportunity to advance on the understanding of climate variability and predictability at the decadal timescale. Our focus will be on the North Atlantic and European region and main targets will be the role of the sea ice and Greenland ice sheet melting as well as polar amplification of climate change and the role of the oceanic circulation in the modulation of decadal climate variability. The approach comprises a series of sensitivity studies based on the novel IPSL-CM6 climate model, which has recently been developed for the internationally coordinated 6th Climate Model Intercomparison Projet (CMIP6). A well defined group of researchers primarily based at IPSL propose to exploit the recent efforts in terms of model development and configurations setting to go beyond CMIP6 and deliver a targeted message on expected climate change over the coming years and decades. Due to the improvements in IPSL climate model and the need for large ensemble to face the chaotic behavior of the climate system, this requires an unprecedented amount of computing time.

top

Scalability of high-order discontinuous galerkin solver for complex DNS

Project Name: Scalability of high-order discontinuous galerkin solver for complex DNS
Project leader: Prof Eusebio Valero
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

High order Discontinuous Galerkin methods are shown an increasing maturity in their application to fluid problems where accuracy is a must. In combination with multi-domain decompositions are able to tackle with any kind of complex geometry and their low dissipative and high accuracy (for the same mesh) properties make them the perfect candidate to the analysis of complex flow configurations where large unsteadiness and a variety of different scales are found. Additionally, h/p adaptivity can provide an extra advantage in the optimization of degrees of freedom, increasing the mesh or the polynomial order in the region of interest. However, the application of these methods to realistic problems requires an efficient parallelization of the solver and the appropriate use of the large databases generated in the simulations. In this work, the performance and scalability of the newly developed high order discontinuous galekin solver will be evaluated against two representative test cases: standard DNS of a channel compressible flow and the wavy wall channel flow. The tool solves the Tridimensional Laminar Navier Stokes equations using a multidomain approach. An external or “h” mesh defines the general feature of the geometry and an internal o “p” mesh defines the resolution or degree of the polynomial in each domain. The connection between different domain is made following a mortar methodology: continuous in fluxes and discontinuous in the states variables. Hanging nodes or different degrees of freedom in different connected domains are allowed but makes the parallelization more challenge. Boundary continuous are imposed weakly. The solver can manage, and it is recommended, a high order representation of the boundary surfaces. The solver has been validated in UPM cesvima cluster (http://www.cesvima.upm.es) in the Taylor Green Vortex test case, until a maximum of 256 cores and 10 million nodes. A skew-symmetric formulation of the convective fluxes makes the solver robust in the analysis of under-resolved turbulence problems.

top

ASTROcomRPMDtest

Project Name: ASTROcomRPMDtest
Project leader: Dr Octavio Roncero
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

Complex organic molecules(COMs) have been detected in many astrophysical objects and are considered as prebiotic molecules. The question that remains is how and where they are formed. The formation of these molecules requires to overcome reaction barriers. At low temperatures this was considered impossible in gas phase, and the formation of COMs is model by reactions on ices. However, most of the molecules formed in ices, like methanol, do not desorb, and when excited with UV photons only photofragments photodesorb[1], so the explanation of the detection of COMs in gas phase is under debate. Recently, it was experimentally reported an important increase of the reaction between the hydroxyl radical and methanol, at low temperatures[2,3]. These results were explained by tunneling through the reaction barrier in the context of Transition State Theory. Recent theoretical work[4] stated that the TS used to model the tunneling rates was unrealistic, and that the tunneling rates are far too small to explain the experimental rise observed at low temperature. These authors proposed an alternative through the formation of methanol dimer. It is of crucial importance to determine the mechanism for the reaction in gas phase. The density in the interstellar media is so low that the formation of dimers is very unlikely. For this aim, in this project we propose to performed dynamical calculations below 300K to confirm this finding. Recently, the same behavior was measured for the reaction of OH with formaldehyde[5], and the rise of the rate constant observed were well reproduced by Quassiclassical calculations(QCT) performed on a full dimensional Potential energy surface(PES)[6]. Quantum calculations are required to confirm these simulations, since at these low temperatures zero point energy and tunneling are expected to play a fundamental role. Due to the high dimensionality of these reactions conventional quantum dynamical methods, such as wave packet propagations, are not feasible. In this work we plan to use Ring Polymer Molecular Dynamics(RPMD)[7], which allow to describe quantum effects, such as zero point energy and tunneling. This method describe each atom by a number of beads, number which increase when lowering the temperature. In addition, the reactivity at temperature below 100K, is mediated by long lived complexes, as recently demonstrated by QCT calculations[5,6]. For all these reasons extensive massive parallel computations are required. In this project, we intend to study the H2CO+OH and CH3OH+OH reactions below 100K using the RPMD method. In this preparatory project we shall check the scalability of the code dRPMD that we have recently developed[9] to apply for a PRACE tier-0 project in the next call that finish in October 2018. [1]Cruz-Diaz et al., Astron.Astrophys.,(2016),592,68. [2]Shannon et al., Nature Chem.,5(2013)745 [3]Antiñolo et al., AstroPhys.J.,823,25(2016) [4]Siebrand et al., PCCP,18(2016)22712 [5]Ocaña et al., Astrophys.J.,850(2017),28 [6]Zanchet et al., PCCP,20(2018),5415 [7]Craig and Manolopoulos, J.Chem.Phys.,121(2004)3368. [8] Suleimanov et al., Comput.Phys.Comm.,184(2013)833 [9] Suleimanov, Aguado, Gómez-Carrasco, Roncero,JPClett,9(2018)2133.

top

MIMOP (Marie Curie Individual Fellowship Project 793450)

Project Name: MIMOP (Marie Curie Individual Fellowship Project 793450)
Project leader: Dr Louis-Alexandre Couston
Research field: Earth System Sciences
Resource awarded: 50 000 core hours on Curie – SKL
Description

An outstanding problem in polar sciences is how quickly land-based ice moves toward the oceans and contribute to sea-level rise, which is one of the most disruptive consequences of climate change. Acceleration of the outflow from polar ice sheets appears linked to enhanced melting at the ice-shelf—ocean interface. The melting process is, however, poorly understood, because state-of-the-art models cannot resolve the effect of the ocean turbulence or basal roughness of the ice shelves. The project MIMOP (for Modelling Ice-shelf Melting and Ocean Processes) aims to fill this critical gap in our knowledge. MIMOP objectives are to 1) Develop an innovative numerical model of the fluid dynamics of melting at the ice-ocean interface 2) Calibrate the model using new observations of ocean properties beneath an Antarctic ice shelf 3) Determine the sensitivity of melting to changes in temperature and current. The objectives will be achieved by combining a highly-efficient Direct Numerical Simulation (DNS) code with a novel formulation of the equations for the solid/liquid phases of water based on the phase-field method. DNS enables turbulent motions to be simulated without approximation, while the phase-field method allows the ice-ocean interface to be rough and evolve in response to melting. The phase-field method has been applied in metallurgical problems and proof-of-concept simulations have demonstrated its suitability for ice melting. Access to supercomputers in Europe will be critical to running the DNS and achieving the project’s goals.

top

Optimized catalytic properties from advanced ab-initio computations

Project Name: Optimized catalytic properties from advanced ab-initio computations
Project leader: Dr Sivan Refaely-Abramson
Research field: Chemical Sciences and Materials
Resource awarded: 100 000 core hours on Hazel Hen
Description

Designing new materials with optimized catalytic properties is of great importance towards a more sustainable energy production. Fundamental understanding of the processes taking place at the electrode-electrolyte interface is needed to set design rules and catalytic control pathways. Here we propose a combined experimental-computational project to gain a comprehensive picture of optimal catalytic surfaces for oxygen reduction by generating a mutual back-and-forth feedback between experimental results obtained with Scanning Tunneling Microscopy and X-Ray Photoelectron Spectroscopy studies of surfaces in different environments, and advanced many-body perturbation theory calculations within the GW-Bethe Salpeter Equation approach, by connecting dynamical parameters set from bandstructure and excited state methods with key properties optimizing the catalytic process.

top

Tubulin post-translational modifications: The effect on tubulin tail dynamics and interactions.

Project Name: Tubulin post-translational modifications: The effect on tubulin tail dynamics and interactions
Project leader: Prof Yaakov Levy
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100 000 core hours on  Piz Daint
Description

Microtubules (MTs) provide mechanical support to eukaryotic cells and serve as “highways” for intracellular trafficking. The basic building block of MTs is the tubulin dimer, comprising alpha and beta tubulin monomers. Tubulin monomers are composed of a structured body and disordered C-terminal tails (CTTs). The tubulin CTTs are “hot spots” for various post-translational modifications (PTMs) including polyglutamylation, polyglycilation and phosphorylation. Although tubulin PTMs are ubiquitous across many cell types, their precise function is only partially understood. It has been proposed that tubulin tail modifications affect the interactions of MTs with microtubule binding proteins and the intrinsic properties of tubulin CTTs. Detailed understanding of the effect PTMs have on the intrinsic properties of the tubulin tails and their interactions with MT binding proteins can be achieved using all-atom molecular dynamics simulations. Simulations of MTs with multiple types of PTMs, would enable us to contribute to the ongoing challenge of understanding the role of tubulin PTMs on MT function.

top

Scalability test of C++ code developed for solving compressible Navier-Stokes equations

Project Name: Scalability test of C++ code developed for solving compressible Navier-Stokes equations
Project leader: Dr Polydefkis Diamantis
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 20 000 core hours on JUWELS
Description

This preparatory access project is submitted as a request for performing scalability tests of classical and mixed quantum/classical (QM/MM) molecular dynamics simulations of DNA systems, in the context of a project we are willing to apply for within the currently open PRACE project access Call.

top

Realistinc zeolites NMR based on ab-initio simulations

Project Name: Realistinc zeolites NMR based on ab-initio simulations
Project leader: Prof Petr Nachtigall
Research field: Chemical Sciences and Materials
Resource awarded: 100 000 core hours on Hazel Hen
Description

Zeolite are micro-porous materials formed by an extended network of corner-sharing SiO4 tetrahedra. The Si atoms can be substituted by different metals, Al in particular, making possible to tune the framework charge and properties. Zeolites are an industrially important class of materials and they have been employed over a large set of applications: catalysis, oil cracking, gas storage and as molecular sieves.   Different techniques are employed to analyse the chemical and physical properties of zeolites with solid state NMR being one of the most powerful. NMR is highly sensitive to the changes in the local environment of NMR active atom. Most zeolites are aluminosilicates and 23Al NMR can provide a great insight into atomistic details of these materials. Interpretation of 23Al NMR spectra is complicated by the quadrupolar character of the nucleus. Majority of experimental investigation is carried out under at least partially hydrated conditions. However, most of the computational investigation of zeolites (including those of chemical shielding) are carried out for the models corresponding to 0 K and UHV. We believe that it is critically important to simulate the computational NMR spectra based on realistic models that take into account experimentally relevant environment (including hydration level) and temperature. Our project aims to study and rationalise experimental results of such systems pairing NMR measurements with ab-initio molecular dynamics (AIMD) based on density functional theory (DFT). We propose to run AIMD simulations on a specific class of zeolites of interest to obtain an ensemble of statistically independent structures to reproduce the experimental observations. The computational resources will be mainly used to run the AIMD calculations and simulate the NMR spectra. All the calculations are independent from each other and relies on parallel distribution of the computational load. This makes the project ideal for a PRACE application allowing us to perform simulation on more realistic (larger) system and task-farming. The preparatory access is meant to investigate in particular the AIMD part of the project to identify the best computational setup for Broadwell machines. This calculations are performed with VASP which is a well established commercially available software aimed to perform different types of ab-initio simulations.

top

UEABS_GMX_NAMD

Project Name: UEABS_GMX_NAMD
Project leader: Dr Dimitris Dellis
Research field: Chemical Sciences and Materials
Resource awarded: 100 000 core hours on MARCONI – KNL, 50 000 core hours on MareNostrum, 50 000 core hours on Curie – KNL, 50 000 core hours on Curie – SKL, 100 000 core hours on Hazel Hen, 20 000 core hours on JUWELS, 50 000 core hours on SuperMUC, 100 000 core hours on Piz Daint
Description

Gromacs and NAMD Molecular Dynamics packages will be benchmarked on the Tier-0 systems. Benchmark data sets are already published in prace web site. The results of performance, scaling as well as their hybrid parallelization behavior will be published in Prace 5IP deliverable : D7.5 Evaluation of Accelerated and Non-accelerated Benchmarks.

top

Nanostructured Polymer Nanocomposites (NanoPolComp)

Project Name: Nanostructured Polymer Nanocomposites (NanoPolComp)
Project leader: Prof Vagelis Harmandaris
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

The proposed project concerns a detailed study of the collective behavior of polymer nanocomposites, through atomistic molecular dynamics (MD) simulations. In polymer-based nanocomposite systems the understanding of the relationship between the physical and chemical attributes of the nanofillers (e.g., nanoparticles, nanotubes, clays, graphene, etc.) and the host matrix is of great importance for the design of new materials with specific functionalities. To achieve this simulation studies with atomic details are required. In addition, the study of the dynamical behavior of long polymer chains as well as the investigation of the diffusion, and possibly the aggregation, of nanoparticles require the simulation of huge systems with millions of atoms. In the proposed Preparatory Access of type A, we intend to perform benchmark analysis and produce scalability plots for MD simulations of Poly(butadiene)(PB)/Silica nanocomposites. The used simulation tool will be the GROMACS 5.0.1 package. The system will consist of 16416 cis PB chains each having a molecular weight of 1624.77 g/mol and 216 spherical Silica nanoparticles of diameter equal to 4.2 nm and molecular weight of 52883.85 g/mol. The total system is expected to contain 2,632,824 atoms. The production runs will focus on the investigation of aggregation issues in such nanocomposite systems and the effect on the mechanical properties. We will first equilibrate a one nanoparticle polymer nanocomposite sample of 30%wt in silica and then replicate it in all three dimensions. The produced system is expected to be homogeneous allowing a balancing among the used processors. The benchmark analysis will be based on runs of about 1 ns recording the wall time for different number of used cores.

top

Functional dynamics of the mammalian respiratory complex I

Project Name: Functional dynamics of the mammalian respiratory complex I
Project leader: Prof Dr Ville R. I. Kaila
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

The respiratory complex I functions as the initial electron entry point to aerobic respiratory chains. It catalyzes the reduction of quinone in its hydrophilic domain, and employs the released free energy for pumping protons across a biological membrane, up to a remarkable distance of 200 Å away from the active site. The long-range charge transfer process electrochemically charges up a biological membrane that, in turn, powers ATP synthesis and active transport in cells. Despite recent advances in structural understanding of complex I [1-3], and functional insight from computational work in recent years [4-10], the molecular principles of the energy transduction machinery in complex I are still not well understood. Understanding complex I is of both central biochemical and biomedical interest, as this enzyme accounts for a large part of primary energy capture processes in mitochondria, whereas its dysfunction leads to development of mitochondrial diseases. Based on new cryo-electron microscopy data of different conformational states of the mammalian complex I, we will in this PRACE project employ multi-scale molecular simulations to elucidate how the enzyme dynamics regulates its biological activity. To this end, guided by the experimental electron density data, we will computationally derive structural models of the different conformational states employing a combination of atomistic molecular dynamics simulations, coarse-grained molecular simulations, and free energy calculation techniques. Based on the structural ensemble obtained from these simulations, proton transfer energetics will be further probed using hybrid quantum mechanics/classical mechanics (QM/MM) simulations. Our computational multi-scale approach aims to link for the first time structural transitions on the biochemically relevant micro- to millisecond timescales with the biological charge transfer activity. Main aims: 1. Derive molecular mechanism for the conformational transitions observed in the mammalian complex I 2. Probe the energetics and dynamics of the coupling between electron transfer and proton pumping References: [1] A. A. Agip et al. Nat. Struct. Mol. Biol. 25, 548–556 (2018). [2] J. Zhu, K. R. Vinothkumar, J. Hirst. Nature. 536, 354–358 (2016). [3] V. Zickermann et al. Science. 347, 44–49 (2015). [4] V. R. I. Kaila. J R Soc Interface. 15, 20170916 (2018). [5] V. R. I. Kaila, M. Wikström, G. Hummer. Proc. Natl. Acad. Sci. U. S. A 111, 6988-6993 (2014). [6] A. Di Luca, A. P. Gamiz-Hernandez, V. R. I. Kaila. Proc. Natl. Acad. Sci. U. S. A. 114, 201706278 (2017). [7] J. Warnau et al. Proc. Natl. Acad. Sci. U. S. A. 115, E8413–E8420 (2018). [8] J. G. Fedor, A. J. Y. Jones, A. Di Luca, V. R. I. Kaila, J. Hirst. Proc. Natl. Acad. Sci. U. S. A. 114, 12737–12742 (2017). [9] V. Sharma et al. Proc. Natl. Acad. Sci. U. S. A. 8, 6–11 (2015). [10] A. P. Gamiz-Hernandez, A. Jussupow, M. P. Johansson, V. R. I. Kaila VRI. J. Am. Chem. Soc. 139, 16282-16288 (2017).

top

Transport properties of graphene nanoribbon based devices

Project Name: Transport properties of graphene nanoribbon based devices
Project leader: Dr Carlo Antonio Pignedoli
Research field: Chemical Sciences and Materials
Resource awarded: 20 000 core hours on JUWELS
Description

At Empa research is conducted in the field of carbon derived nanomaterials. In particular, after demonstration in 2010 of a bottom up approach for the fabrication of atomically precise graphene nanoribbons (GNRs) with different edge topologies, whose reliability and flexibility have been confirmed by its successful adoption in different laboratories around the world, the focus is now set on the fabrication of prototype field-effect transistors (FETs) based on GNRs. The research approach, based on a synergy between experiments and theory, has succeeded in the characterization of the electronic and optical properties of GNRs in gas phase and on metallic substrates. More recently, the possibility to fabricate a new family of GNRs was demonstrated. These structures are characterized by topologically protected states, whose electronic properties can be designed by varying the spacing between “extensions” added to the edges of a backbone. After the successful realization of the FET device, the next objective consists in characterizing, from a theoretical and an experimental point of view, the transport properties of different GNRs. In this project we join Empa’s capabilities in the fabrication of GNRs and in the theoretical investigation of their (electronic) structure with the expertise of Professor Mathieu Luisier from ETH Zurich in ab initio device simulation to study the transport properties of prototypical FETs based on an “all carbon” design. The outcome of our project will be a better understanding of the intrinsic factors that may limit the “current vs. voltage” characteristics of ultra-scaled GNR FETs, a study of their ultimate performance, and the design of transistors with optimized contact engineering. For prototypical devices we will compute, be means of Density Functional Theory (DFT), the equilibrium geometry of the ribbon/lead system, we will characterize electronically the ribbon/lead interface and extract the Schottky barrier and we will compute the phonons spectrum. In selected cases (i.e. for geometries that according to DFT calculations results more promising in terms of device performance) we will perform a higher level description of the ribbon/lead interface by means of GW calculations. Once stable device structures will be obtained, we will simulate their transport properties with a quantum transport solver, first in the ballistic regime or in the presence of structural/impurity defects, and, for the most promising components, with electron-phonon scattering. The key questions we want to answer within this projects are the following: What is the maximum current that GNR FETs with graphene leads can carry? How does the graphene-GNR contact interface affect the device performance? How does the Schottky barrier height (SBH) depend on the GNR type and on the leads material? Do the Schottky barrier height and injection efficiency calculated with quantum transport correlate? How do structural or impurity defects in leads or in the ribbon influence the transport properties? What is the effect of electron-phonon scattering on the transport properties of these low dimensional devices? Is self-heating important? How sensible are the transport properties to the ribbon/lead matching pattern?

top

Scaling of the Copenhagen version of RAMSES on Joliot-Curie

Project Name: Scaling of the Copenhagen version of RAMSES on Joliot-Curie
Project leader: Prof Troels Haugboelle
Research field: Universe Sciences
Resource awarded: 50 000 core hours on Curie – SKL
Description

The Copenhagen version of RAMSES is a hybrid OpenMP / MPI adaptive mesh refinement code that has been shown to scale to more than 100,000 cores on problems with well populated AMR levels, and to more than 20,000 cores on more challenging problems with a large number of refinement levels and / or a highly clustered cell distribution. We would like to establish the scaling of UCPH-RAMSES on a modern Skylake system for a specific scientific production problem.

top

CEWAF – Curvature effects in wall-bounded flows

Project Name: CEWAF – Curvature effects in wall-bounded flows
Project leader: Dr Geert Brethouwer
Research field: Engineering
Resource awarded: 20 000 core hours on JUWELS
Description

Streamline curvature effects occur in turbulent flows over wings and turbine blades and in many other engineering and aeronautical applications with curved surfaces. The impact of streamline curvature on the mean flow, turbulence and skin friction drag and other flow properties can be large. However, a systematic numerical study of longitudinal and transverse streamline curvature effects on turbulent flows at sufficiently high Reynolds numbers has not been carried out yet. I propose to perform large-scale direct numerical simulations of turbulent flows in weakly and strongly curved channels with longitudinal and transverse curvature at moderately high Reynolds numbers. Those demanding simulations are only possible with HPC resources. Through these fully resolved simulations of turbulent flows in geometries with various curvatures we obtain a better and more complete physical understanding of streamline curvature effects on wall-bounded turbulent flows. The project, moreover, produces reference data for model development and validation to support modelling of turbulent flows with streamline curvature. Turbulence models that can better take into account streamline curvature effects are valuable for aeronautical and industrial engineering.

top

Scalability tests of L-Gadget and Ginnungagap codes on MareNostrum to more than 20,000 cores.

Project Name: Scalability tests of L-Gadget and Ginnungagap codes on MareNostrum to more than 20,000 cores
Project leader: Prof Gustavo Yepes
Research field: Universe Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

We were requested by the PRACE technical support team to provide actual scalability tests of the parallell N-body code L-GADGET that we plan to us in the PRACE proposal submitted to the 19th call to run 9216**3 particle simulations using the Suppression Variance Method (see Chuang et al 2018, http://arxiv.org/abs/1811.02111 for more details). A simulation of this size has never been done on MarNostrum. The maximum simulation performed so far was a 6144**3 particle simulation with 6144 MPI processors. The proposed simulation is a 3.37 times larger. Our estimation of the computational resources were based on extrapolations from a series of simulations done during the previous PRACE project that finished on September 30th with 2048^3 , 4096^3 and the mentioned 6144^3. Although we think that our extrapolations are quite reasonable, nevertheless we request this preparatory access project to test the scalability of the code to more than 18000 processors. In this regard, we plan to run only the first steps of the simulations which will cost more roughly 150,000 core-hours for a typical run of 9216^3 particle simulation.

top

Paris Simulator code HPC scalability test in MareNostrum 4

Project Name: Paris Simulator code HPC scalability test in MareNostrum 4
Project leader: Prof José Desantes
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

Paris (Parallel Robust Interface Simulator) Simulator is a free code that combines the VOF (Volume of Fluid) and Front-Tracking methods in a DNS (Direct Numerical Simulation) framework, in order to create simulations of interfacial fluid flow such as droplets, bubbles or waves. Thus, it is suitable to study the liquid primary atomization. The scalability of the Paris Simulator code has been demonstrated in the Turing supercomputer at IDRIS and the Vesta-Cetus supercomputer at Argonne National Laboratories. The present project aims at studying the scalability of the Paris Simulator code in an additional HPC cluster architecture, namely MareNostrum 4. The results of the project could allow the application team to request HPC resources in the MareNostrum 4 computer in order to study the fuel primary atomization in propulsion systems.

top

Novel Nanocatalysts from Ligand-Stabilized Metal Nanoclusters

Project Name: Novel Nanocatalysts from Ligand-Stabilized Metal Nanoclusters
Project leader: Dr Sami Malola
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

Engineering new catalysts with high selectivity that optimize consumption of precious metals is a major fundamental scientific challenge that can have potentially huge technological, economic and societal impact. It has been known for a long time that ultrafine metal clusters that have dimensions in the nanometer-scale can be extremely active catalysts for several oxidation and hydrogenation reactions, however, they suffer from poor stability under operational conditions since they tend to coagulate to larger, less active metal particles after several reaction cycles. This proposal exploits recent preliminary ideas suggesting that protecting the ultrafine clusters with ligand molecules not only stabilizes the clusters from coagulation but can increase the selectivity of the reaction. Specifically, this proposal explores atomic-scale reaction mechanisms taking place on ligand-protected gold and copper clusters under hydrogenation reactions in aqueous environments. HPC resources are essential to achieve the goals since the cluster structures and reaction intermediates need to be modelled taking into account the electronic structure and molecular dynamics of the catalytic system in the level of the density functional theory (DFT). Massively scalable DFT simulation codes will be used to produce fundamental new information on the reaction paths, reaction barriers, and effect of the ligands as well as solvent for systems that are selected in close collaborations with relevant experiments.

top

NANONP2AU –Nanoindentation of Nanopolycrystalline Nanoporous Au in Molecular Dynamics Simulations

Project Name: NANONP2AU –Nanoindentation of Nanopolycrystalline Nanoporous Au in Molecular Dynamics Simulations
Project leader: Prof Dan Mordehai
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

Nanoporous Au structures are considered as potential candidates in various emerging applications such as electrocatalyst, electrochemical actuators and sensors, owing to their large surface-to-volume ratio. As structural materials, one of the key issues is having light structures, but yet with desirable mechanical properties. Such, properties can be tailored, for instance, through geometry, owing to the fact that the ligaments are of a few nanometers and that strength is size-dependent, but also through the initial microstructure. Recently, a technique to manufacture ultra-fine grained nanoporous structures was proposed. Using nanoindentation, it was found that the hardness at room temperature is as high as that of ultrafine-grained bulk and it remains high even after heating by almost 300 degrees. The origin of these enhanced mechanical properties remains a riddle, which we aim at explaining in this project. We plan to perform multi-million atoms nanoindentation molecular dynamics simulations of nanopolycrystalline nanoporous Au structures, at various temperatures, to reveal the underlying dislocation mechanisms that govern mechanical properties of these structures. The large-scale simulations, which are planned based on this preparatory access, are intended to shed light on the effect of the initial microstructure and temperature on the deformation of nanoporous Au.

top

Towards the accurate thermal modelling of liquid metal flows

Project Name: Towards the accurate thermal modelling of liquid metal flows
Project leader: Dr Lilla Koloszar
Research field: Engineering
Resource awarded: 100 000 core hours on MARCONI – KNL
Description

Our further aim is to analyse the thermal-hydraulics of new liquid metal cooled nuclear reactor systems with high resolution in space and time and provide useful information to the nuclear researchers and engineers still in the design phase. The goal of the preparatory project is to test how Nektar++ solver is performing in case of heavy computations and scale on HPC systems. The code is developed by ANS, Argonne National Laboratory, primarily for high fidelity simulations. We are targeting to perform Direct Numerical Simulations with this code of low Prandtl number fluids in academic configurations that would not only serve as validation, but help to further analyse and understand the thermal behaviour of liquid metals, and low Prandtl fluids, in general. The Nektar++ solver is based on Spectral Element method and is highly scalable on various HPC systems. Our goal is to compile and test its performance in dedicated thermal cases, establish scalability curves in order to later estimate the necessary computational needs and time for a PRACE project. This is the last step before we confidently can apply to investigate liquid metal thermal hydraulics, since we already successfully tested in the same Preparatory A project framework the other software, OpenFOAM, that we intend to use for RANS modelling (2010PA4401).

top

Paris Simulator code HPC scalability test in Marconi

Project Name: Paris Simulator code HPC scalability test in Marconi
Project leader: Prof Raul Payri
Research field: Engineering
Resource awarded: 100 000 core hours on MARCONI – KNL
Description

Paris (Parallel Robust Interface Simulator) Simulator is a free code that combines the VOF (Volume of Fluid) and Front-Tracking methods in a DNS (Direct Numerical Simulation) framework, in order to create simulations of interfacial fluid flow such as droplets, bubbles or waves. Thus, it is suitable to study the liquid primary atomization. The scalability of the Paris Simulator code has been demonstrated in the Turing supercomputer at IDRIS, the Vesta-Cetus supercomputer at Argonne National Laboratories and more recently MareNostrum IV at Barcelona Supercomputing Center. The present project aims at studying the scalability of the Paris Simulator code in an additional HPC cluster architecture, namely Marconi. The project will not only allow to study the code scaling, but also to compare the time-to-solution for this application with Marconi with the one obtained for MareNostrum 4 (currently, the applicant team estimates that the time in Marconi will be scaled by a factor of 1.5 with respect to the time in MareNostrum, since Marconi’s cores clock-frequency is 1.4 GHz and MareNostrum’s cores is 2.1 GHz. The present project will allow to validate or to modify this assumption accordingly). The results of the project could allow the application team to request HPC resources in the Marconi supercomputer in order to study the fuel primary atomization in propulsion systems.

top

Scalability performance of an MPI-CUDA approach for hypersonic flows with detailed state-to-state air kinetics

Project Name: Scalability performance of an MPI-CUDA approach for hypersonic flows with detailed state-to-state air kinetics
Project leader: Prof Giuseppe Pascazio
Research field: Engineering
Resource awarded: 100 000 core hours on Piz Daint
Description

The aim of this project is to test the scalability performance of an innovative code that has been developed for studying high enthalpy flows such those occurring during the atmospheric entry or around hypersonic vehicles devised for future transcontinental flights. At hypersonic speeds, a strong shock wave increases the gas temperature up to more than 10000 K exciting the internal modes of molecules and activating dissociation and atom ionization. Downstream of the shock wave thermal and chemical processes have a characteristic time comparable with the fluid dynamic one making the flow a system in thermochemical non-equilibrium. The innovative aspect of the present code is that it is able to deal with thermochemical non-equilibrium by using a vibrationally resolved State-to-State (StS) approach in addition to the most popular Park model. The StS approach has demonstrated to be much more accurate than the Park one thanks to its ability to determine the distribution of internal states even when it deviates from the Boltzmann one, a condition that strongly affects reaction rates. Unfortunately, the StS approach is much more computationally expensive than the Park model, thus up to now its use has been limited to 1D simulations with very few exceptions. Indeed, for a neutral air mixture, while the Park model needs 5 species and 17 reactions, the StS approach employed in the present code considers 118 species and about 10000 reactions. The second innovative aspect of the code is the parallelization with an MPI-CUDA approach that allows one to scale the application on a multi-node GPU cluster. The present implementation has demonstrated speed-up values of GPU vs CPU larger than 100 allowing to perform for the first time 2D StS simulations of a neutral air mixture by using a small GPU cluster. In order to perform simulations of larger systems and, in perspective, of 3D configurations, the scalability analysis will be a first step for applying to PRACE calls for Project Access.

top

 

Type B: Code development and optimization by the applicant (without PRACE support) (17)

Elimination of the load imbalance in a parallel particle-in-cell code for plasma simulations

Project Name: Elimination of the load imbalance in a parallel particle-in-cell code for plasma simulations
Project leader: Dr Jacek Niemiec
Research field: Universe Sciences
Resource awarded: 50 000 core hours on JUWELS
Description

This project is a resubmission of 2010PA4546. The main goal of this project is the elimination of computational load imbalance that hinders the efficiency of the THISMPI code, which is a Particle-In-Cell simulation code used for modeling collisionless plasma systems for high-energy astrophysics applications. The most commonly used simulation setup is that of two colliding plasma beams, whose interaction leads to the formation of shock waves that isotropise, heat, and compress the beam plasmas. In its current implementation, the THISMPI code uses the MPI parallelization with a numerical grid divided into fixed-size computational domains, each domain having assigned a single CPU-core that operates on particles and grid points contained within the domain. The scales of the problems investigated currently with the colliding beam setups are huge, and thus numerical box needs to be partitioned into domains in directions both transvese to and along the beam flow. This quickly leads to the load imbalance – CPU-domains in the middle of the box, where the beams interact with each other, are heavily loaded with particles in comparison with the outer domains, that stay mostly idle while waiting for the inner domains to finish their computations and synchronize with other domains, performing communications of the boundary conditions. We want to alleviate the imbalance problem by converting the MPI code to a hybrid MPI+OpenMP code, in which computations of several CPU-domains will be handled jointly by a number of CPU-cores. We assume that the hybrid code uses all CPU-cores on a given CPU-node or in a NUMA domain to perform calculations. To achieve this, each node/NUMA assigned domains will be processed sequentially, but all available cores will operate in parallel on each domain. This strategy should ensure that computations would be very efficient, leaving no idle time for any CPU-core. Moreover, the proposed solution would additionally benefit from less communication, since it will use a shared memory access to data stored on a node. Thus an MPI-transfer of data between CPU-domains on a given node will no longer be needed.

top

Cracks in the Sky: Cosmic strings with CUDA-MPI

Project Name: Cracks in the Sky: Cosmic strings with CUDA-MPI
Project leader: Dr Carlos Martins
Research field: Universe Sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

Cosmic strings and other topological defects [Kibble, 1976] are a natural consequence of many symmetry breaking patterns in many proposed physical theories and expected to have formed in the early Universe by means of the Kibble mechanism. In addition, superstrings produced at the end of brane inflation models [Sarangi & Tye, 2000] both have the same topological character as their field theory cosmic string cousins and can also be stretched to cosmological sizes. While our understanding of these objects has greatly advanced since Kibble’s proposal, in reality there are still many unknowns surrounding them, vital to allow for future detection. Both analytical studies centered on attempts to understand the cosmic string phenomenological zoo (examples include loops, kinks, cusps, junctions) along with effects on network evolution, and observational searches (especially those of next-gen facilities such as COrE (Cosmic Origins Explorer) [Finelli et al., 2016] and LISA (Laser Interferometer Space Antennae) [Pau Amaro-Seoanne et al., 2017]) of more realistic (and complex) defects require simulations with sufficiently large resolution and dynamical range (in field theory this currently means cubic lattices with size N^3 where N ≤ 2048). This means that they are susceptible to being bottlenecked by the simulations themselves. A way to circumvent this is to undergo a paradigm shift: to forsake use of traditional processing units and random access memory, in order to exploit the larger memory bandwidth and instruction throughput ceilings of accelerators, such as Graphics Processing Units. As such, the immediate goal of this preparatory project is to take an already existing Abelian-Higgs string simulation written in the Compute Unified Device Architecture and extend it to support multiple GPU’s (via interoperability with the Message-Passing Interface), such that it can provide adequate performance and scalability for future analytical and observational studies.

top

PRACE / UEABS

Project Name: PRACE / UEABS
Project leader: Mr Cedric Jourdain
Research field: Earth System Sciences
Resource awarded: 200 000 core hours on MARCONI – KNL, 100 000 core hours on MareNostrum, 200 000 core hours on Curie – KNL, 200 000 core hours on Curie – SKL, 250 000 core hours on Hazel Hen, 50 000 core hours on JUWELS, 100 000 core hours on SuperMUC, 100 000 core hours on Piz Daint
Description

The Unified European Application Benchmark Suite (UEABS) is a set of 12 application codes taken from the pre-existing PRACE and DEISA application benchmark suites to form a single suite, with the objective of providing a set of scalable, currently relevant and publically available codes and datasets, of a size which can realistically be run on large systems, and maintained into the future. This work has been undertaken by Task 7.4 “Unified European Applications Benchmark Suite for Tier-0 and Tier-1” in the PRACE Second Implementation Phase (PRACE-2IP) project and will be updated and maintained by subsequent PRACE Implementation Phase projects.

top

UEABS: Code_Saturne for PRACE 5iP WP7

Project Name: UEABS: Code_Saturne for PRACE 5iP WP7
Project leader: Dr Charles Moulinec
Research field: Engineering
Resource awarded: 100 000 core hours on MareNostrum, 50 000 core hours on JUWELS, 100 000 core hours on Piz Daint
Description

The purpose of this work is to test Code_Saturne at scale on various Tier0 systems with different architectures. This work is undertaken through PRACE for its UEABS.

top

PRACE 5IP WP7 Benchmarking (CP2K)

Project Name: PRACE 5IP WP7 Benchmarking (CP2K)
Project leader: Dr Arno Proeme
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MareNostrum, 50 000 core hours on JUWELS, 100 000 core hours on Piz Daint
Description

The aim of this project is to use the machines requested to perform benchmarking of CP2K as part of the PRACE 5IP WP7 benchmarking activity.

top

PRACE 5IP WP7 UEABS Benchmark suite runs on PRACE Tier 0 Systems: QCD

Project Name: PRACE 5IP WP7 UEABS Benchmark suite runs on PRACE Tier 0 Systems: QCD
Project leader: Dr Jacob Finkenrath
Research field: Mathematics and Computer Sciences
Resource awarded: 200 000 core hours on Curie – SKL, 50 000 core hours on JUWELS
Description

The project is part of the work package 7 (WP7) task 3 of the ongoing PRACE Fifth Implementation Phase (PRACE-5IP). The computer time will be used for running the QCD (Quantum ChromoDynamics) Unified European Applications Benchmark Suite (UEABS) on PRACE Tier 0 systems in line with the goals of WP7 Task 3. We will run two different test, one to directly compare the performance of the system with PCP prototypes (limited scaling up to 1000 processes) and one where the standard test cases are used (scaling up to a large partition of the machine).

top

Scaling Block Conjugate Gradient Solvers for FET-HPC Project NLAFET

Project Name: Scaling Block Conjugate Gradient Solvers for FET-HPC Project NLAFET
Project leader: Prof Cevdet Aykanat
Research field: Mathematics and Computer Sciences
Resource awarded: 50 000 core hours on JUWELS,
Description

This is PRACE 5IP T7.2 technical work for the mini-project implementation by Bilkent University.] Classical CG methods suffer from high global synchronization overheads. FET-HPC NLAFET (Parallel Numerical Linear Algebra for Future Extreme Scale Systems) proposes block CG variants, Orthodir and Orthomin, that reduce number of global synchronizations via dynamically reducing number of search directions compared to classical CG counterparts. Sparse matrix dense matrix multiply (SpMM), linear dense matrix multiply (LDM) and dense matrix matrix transpose multiply (DMMT) operations constitute three major bottleneck kernel operations repeated at each iteration. In scaling parallel Orthomin and Orthodir codes, the distributions of SpMM, LDM and DMMT among processors are closely tied to each other due to their input-output dependency in terms of the rows of the input and output dense matrices of SpMM. The objective of this project is to increase the scalability of parallel Orthodir and Orthomin codes on Tier-0 systems by investigating different partitioning models that intelligently distribute the data elements and atomic tasks of SpMM, LDM and DMMT among processors. Our parallel codes take one-dimensional (1D) or two-dimensional (2D) distributions of SpMM as input. Our research experience on sparse matrix partitioning over the past decades has revealed the fact that 2D distribution models that deploy Cartesian partitioning are superior to 1D distribution models in terms of parallel scalability for certain types of applications. This is explained by the fact that communication costs in those 2D models are reduced by nice intrinsic upper bounds while 1D models lack such a nice property. While 2D Cartesian models prove to be the best alternative in many other sparse applications including matrix vector multiplication [1], eigensolvers [2], tensor decomposition [3], the literature lacks the investigation of its performance on block CG solvers. Current distribution configuration used by NLAFET is the simple 1D distribution of the sparse matrix. Another aspect we investigate in this project is the possibly-conflicting objectives of reducing load imbalances for SpMM and LDM-DMMT. We also utilize multi-constraint partitioning tools such as METIS for reducing both imbalances. We successfully applied 1D and 2D cartesian sparse matrix partitioning models as well as two-constraint graph partitioning for reducing total communication overhead during SpMM. In current scalability tests on a local cluster, we observed that 2D and two-constraint models scale better than 1D and single-constraint models, respectively. We want to investigate whether these findings extend to much larger number of cores on a Tier-0 system. [1] B. Hendrickson, R. Leland, and S. Plimpton, “An efficient parallel algorithm for matrix-vector multiplication,” International Journal of High Speed Computing, vol. 07, no. 01, pp. 73–88, 1995. [2] A. Yoo, A. H. Baker, R. Pearce, and V. E. Henson, “A scalable eigensolver for large scale-free graphs using 2d graph partitioning,” in Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’11. New York, NY, USA: ACM, 2011, pp. 63:1–63:11. [3] S. Smith and G. Karypis, “A medium-grained algorithm for distributed sparse tensor factorization,” 30th IEEE International Parallel & Distributed Processing Symposium, 2016.

top

PRACE-5IP WP7 UEABS benchmarks for GPAW

Project Name: PRACE-5IP WP7 UEABS benchmarks for GPAW
Project leader: Dr Martti Louhivuori
Research field: Chemical Sciences and Materials
Resource awarded: 100 000 core hours on MareNostrum, 50 000 core hours on JUWELS, 100 000 core hours on Piz Daint
Description

Run UEABS benchmarks for GPAW and compare results from multiple PRACE Tier-0 systems as part of the benchmarking activity in PRACE-5IP WP7.

top

PRACE UEABS WP7 Alya

Project Name: PRACE UEABS WP7 Alya
Project leader: Mr Cristian Morales
Research field: Engineering
Resource awarded: 200 000 core hours on Curie – KNL, 200 000 core hours on Curie – SKL, 250 000 core hours on Hazel Hen, 50 000 core hours on JUWELS, 100 000 core hours on SuperMUC-NG, 100 000 core hours on Piz Daint
Description

The aim of this project is to use the machines requested to perform benchmarking of Alya as part of the PRACE 5IP WP7 benchmarking activity.

top

PFARM benchmarking for UEABS in PRACE WP7.3

Project Name: PFARM benchmarking for UEABS in PRACE WP7.3
Project leader: Dr Andrew Sunderland
Research field: Fundamental Physics
Resource awarded: 200 000 core hours on MARCONI – KNL, 100 000 core hours on MareNostrum, 200 000 core hours on Curie – KNL, 200 000 core hours on Curie – SKL, 250 000 core hours on Hazel Hen
Description

PFARM is part of a suite of programs based on the ‘R-matrix’ ab-initio approach to the vari-tional solution of the many-electron Schrödinger equation for electron-atom and electron-ion scattering. Accelerator-based implementations have been implemented for both EXDIG and EXAS. EXAS uses offloading via MAGMA (or MKL) for sector Hamiltonian diagonalizations on Xeon Phi and GPU accelerators. EXDIG uses combined MPI and OpenMP to distribute the scattering ener-gy calculations on CPUs efficiently both across and within Xeon Phi accelerators. Benchmarking of PFARM will take place across all Tier-0 systems for a range of datasets. We will evaluate the results on PRACE systems from the standard benchmarks to the accelerated benchmarks, compare where both are available, and will strive to identify reasons for, and patterns in, the performance. PFARM performance results and analysis will be reported in Prcae 5iP deliverable Work Package 7 Deliverable D7.5.

top

HPC simulation of particles deposition in human airways

Project Name: HPC simulation of particles deposition in human airways
Project leader: Dr Beatriz Eguzkitza
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MareNostrum,
Description

In the present proposal, high-performance computing (HPC) simulations of particles deposition in human airways will be performed and validated. Different patient-specific cases will be considered. Computational meshes for each case will be constructed from medical images, simulations for different kind of particles and inflow conditions will be performed. The final goal of the project is to optimise the HPC simulation code, to validate its results and to establish a pipeline to enable the use of HPC resources in the company workflow. The code to be used is Alya, which is part of the benchmark suite of PRACE.

top

Quantum Espresso optimisation and parallel scaling for PRACE UEABS (Unified European Applications Benchmark Suite)

Project Name: Quantum Espresso optimisation and parallel scaling for PRACE UEABS (Unified European Applications Benchmark Suite)
Project leader: Dr Andrew Emerson
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on JUWELS, 100 000 core hours on Piz Daint,
Description

As part of the PRACE 5ip task 7.3 (UEABS) we need to run benchmarks and performance profiles for the final deliverable in Februrary 2019. In this project we will analyse the Quantum.

top

UEABS: PRACE’s Unified European Applications Benchmark Suite

Project Name: UEABS: PRACE’s Unified European Applications Benchmark Suite
Project leader: Mr Sagar Dolas
Research field: Mathematics and Computer Sciences
Resource awarded: 200 000 core hours on MARCONI – KNL, 100 000 core hours on MareNostrum, 200 000 core hours on Curie – KNL, 200 000 core hours on Curie – SKL, 250 000 core hours on Hazel Hen, 50 000 core hours on JUWELS, 100 000 core hours on SuperMUC-NG, 100 000 core hours on Piz Daint
Description

This project will support the activities of PRACE-5IP in testing and running scalability tests on the application codes from an upcoming new release of the Unified European Application Benchmark Suite. This comprises of comparison of accelerated and non accelerated applications.

top

Scaling behavior of the SHOC PRACE synthetic benchmark suite on Tier-0 systems

Project Name: Scaling behavior of the SHOC PRACE synthetic benchmark suite on Tier-0 systems
Project leader: Dr Valeriu Codreanu
Research field: Mathematics and Computer Sciences
Resource awarded: 200 000 core hours on MARCONI – KNL, 100 000 core hours on MareNostrum, 200 000 core hours on Curie – KNL, 200 000 core hours on Curie – SKL, 250 000 core hours on Hazel Hen, 50 000 core hours on JUWELS, 10 0000 core hours on SuperMUC-NG, 100 000 core hours on Piz Daint
Description

This project will support some the activities of the UEABS (Unified European Application Benchmark Suite), particularly the synthetic benchmarks.The SHOC benchmark suite was selected for this purpose already during PRACE-4IP, however, at that point SHOC was mostly used for accelerator benchmarks. With this project we want to assess the suitability and performance of SHOC as a general synthetic benchmark for accelerated and non-accelerated systems.

top

Optimization of the “RISE” application in PET image reconstruction

Project Name: Optimization of the “RISE” application in PET image reconstruction
Project leader: Mr Christos Lemesios
Research field: Engineering
Resource awarded: 50 000 core hours on JUWELS
Description

Positron Emission Tomography (PET) has emerged as a leading in-vivo imaging modality in medical physics. The Athens Model Independent Analysis Scheme (AMIAS[1]) and its application in imaging problems, the “Reconstructed from Image Solutions Ensemble” (RISE[2]) have been applied in various fields (Lattice QCD, SPECT, Thermal imaging, Hadronic Physics) in recent years. AMIAS has been proved to perform very well on difficult inverse problems, especially when characterized by low statistics and/or noisy data. In the proposed project we intend to extend the application of RISE framework to PET tomography. Applying successfully RISE in PET tomography is expected to produce superior tomographic images, reduction of the radiation dose to the patient and/or earlier detection of lesions. AMIAS exploits Monte Carlo simulations and statistical physics concepts to extract parameter Probability Distribution Functions (PDFs), given a dataset associated with a physical process which is described by a model (theory); this model which is cast on a parametric form is considered the forward model describing the data. AMIAS forward projects the physical model repeatedly with different parameterizations and builds an ensemble of possible solutions each one tagged with a probability of representing reality. In RISE the data concern the tomographic image (sinograms) while the forward model is the physical process describing the production, transmission, and detection of the relevant detected radiation (the annihilation photons in the case of PET). The varied parameters in the parametric model of emission tomography (e.g. SPECT, PET, Thermal imaging) include the physical properties (location, shape, intensity etc.) of the emission hotspots. RISE, in the case of emission tomography, creates the ensemble of solutions, which is used to derive each parameter’s PDF, and produces tomographic imaged distributions, given the experimental sets of projections (e.g. sinograms or list-mode format data). RISE performance has been extensively studied in Single Photon Emission Computed Tomography (SPECT) and Thermal Imaging and it was found to be remarkably successful. In the proposed work we are to pursue the application of the same methodology to the far more important but also significantly more demanding and complex PET tomography. However, the forward model in our PET simulations is far more complicated and computationally intensive than SPECT and Thermal Imaging problems, restricting us from applying the method in complex realistic cases/clinical data with the available to us computational power. This project thus seeks to adopt and optimize RISE algorithmic framework in PET tomography, allowing us to apply the current model to clinical data and to introduce sophisticated modeling using GATE (Geant4 Application in Emission Tomography) simulations. Introducing GATE within a Monte Carlo scheme will provide the opportunity to simulate all the physical processes taking place such as scatter, and attenuation in a complex medium (such as the human body), creating high fidelity reconstruction software, applicable to clinical data. 1: https://arxiv.org/pdf/nucl-ex/0703031.pdf 2: https://arxiv.org/pdf/1804.03915.pdf.

top

From resolved convection to the Quasi-biennial oscillation (reconQBO)

Project Name: From resolved convection to the Quasi-biennial oscillation (reconQBO)
Project leader: Dr Marco Giorgetta
Research field: Earth System Sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

The tropical quasi-biennial oscillation (QBO) is one of the most prominent dynamical phenomena of the stratosphere. The theory stipulates that wave-meanflow interaction between vertically propagating waves and zonal jets creates the downward propagating easterly and westerly jets of the QBO. Existing simulations of the QBO in general circulation models (GCMs) rely on the parametrized convective heating as a source for resolved tropical waves and gravity wave parameterizations for sub grid scale gravity wave drag. Recent studies showed that the uncertainty originating from the parameterizations and their tuning effectively hinders the understanding of the full QBO cycle in the current climate and consequently obstructs the assessment of climate change effects on the QBO. We therefore propose a first direct simulation of the QBO in a deep-convection resolving GCM that by construction is independent of parameterizations for convection and gravity waves. By comparison of analyses and the direct QBO simulations we expect to understand better the wave meanflow interaction that generates the QBO. On this base we want to address the following questions: • What is the contribution of different types of tropical waves to the progression of the westerly and easterly phase through a life cycle of the QBO? • How do QBO jets influence the tropical deep convection? • How and why does the QBO respond to a warming climate? For the realization of the experiments we want to make global atmospheric simulations at a horizontal grid resolution of ca. 2.5 km and a vertical resolution of ca. 300 m. The length of the experiments are 6 months for the shorter QBO forecast experiments and 2 to 3 years for the full QBO cycle experiment. Such atmospheric experiments have not been tried so far, owing to the tremendous computational size and costs. Concretely we plan to use the ICON general circulation model for this experiment. The ICON model is a joint development of the Max Planck Institute for Meteorology (MPI-M) and the German weather service (DWD), where the ICON model is used for operational numerical weather forecasting at a resolution of 13 km. For research at MPI-M, the ICON model has been employed in a variety of experiments, including a recent global experiment at 2.5 km resolution, albeit at coarser vertical resolution than needed for the QBO study, and only up to 40 days due to the limited available computational resources. In order to overcome this limit we are currently working towards a GPU enabled version of ICON that can be employed on the very large “Piz Daint” GPU-system at CSCS. This GPU port of ICON is ongoing in the PASC-ENIAC project of ETHZ, MPI-M, MetoSwiss and DWD (https://www.pasc-ch.org/projects/2017-2020/eniac-enabling-the-icon-model-on-heterogeneous-architectures/). In the proposed preparatory project we want to make technical tests and improvements of the ICON code in the configuration that later shall be used for QBO experiments on the “Piz Daint” system.

top

Learning the Epoch of Reionization – LEoR

Project Name: Learning the Epoch of Reionization – LEoR
Project leader: Dr Gillet Nicolas
Research field: Universe Sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

Our project aims to optimize the analysis of upcoming 21-cm images of the epoch of reionization and the cosmic dawn. This cosmic signal, which encodes the highly sought-after properties of the first galaxies, is highly non-Gaussian. To extract as much information as possible from the signal, we have developed Convolutional Neural Networks (CNNs) which were trained on a database of mock 21-cm images to recover the underlying astrophysical parameters which regulate galaxies. CNNs are especially useful for this purpose because they can adaptively select the optimal statistic which maximizes their ability to recover astrophysics. However, our original work was severely limited by computational resources. Network tuning was done “by hand” using only a few configurations, since each training of the CNN took days on our local cluster. Moreover, we were only able to use a tiny fraction of the data, again due to computational constraints. We wish to optimize the performance of CNNs at astrophysical parameter recovery from 21-cm images, by using the efficacy of GPU – parallelized machine learning packages. Specifically, we will develop (i) an automatic calibration of the CNN’s hyper-parameters and (ii) allow the training of CNNs on 21-cm images of a realistic size which include noise. We will do so by developing a wrapper for KERAS. First, the code has to be able to build and change the architecture of the CNN according to a list of hyper-parameters. The wrapper allows for an automatic parallel search of the best hyper-parameter set, according to a network score. This step can be done with a MCMC like framework, i.e. exploring the hyper-parameter set stochastically migrating to the hyper-parameter combination which maximizes the score. The automatic optimization might require thousands of CNN trainings; thus the training must be sped-up using GPU optimization, performed on a GPU-cluster. This preparatory project is the first step towards this goal. We will use allotted resources to (i) port, test and do timing estimates of our CNN on GPUs; and (ii) quantify the timing and the scalability of the automatic hyper-parameter optimization on parallel GPUs. At the end of the preparatory project, we will be able to deploy our network optimization on a Prace tier-0 machine for a production run. The final, optimized CNN will allow us to infer the properties of the unseen first galaxies from realistic 21-cm images of reionization and the cosmic dawn. Having such a tool will allow us to understand upcoming results from the SKA, thus providing a scientific return on this huge investment.

top

 

Type C: Code development with support from experts from PRACE (1)

Decision Analytic Framework to explore the water-energy-food Nexus in complex transboundary water resource systems of fast developing countries (DAFNE)

Project Name: Decision Analytic Framework to explore the water-energy-food Nexus in complex transboundary water resource systems of fast developing countries (DAFNE)
Project leader: Dr Jazmin Zatarain Salazar
Research field: Engineering
Resource awarded: 200 000 core hours on Curie – KNL
Description

The Decision Analytic Framework to explore the water-energy-food Nexus in complex transboundary water resource systems of fast developing countries (DAFNE) project is a H2020 EU project targeted to establish a decision-analytic framework for the exploration of the Water-Energy-Food (WEF) nexus in transboundary river basins. DAFNE integrates multiple and diverse international and local academic expertise from natural sciences, water engineering, environmental economics, as well as water governance and law experts in order to facilitate social understanding of the impact and support comparative analysis of alternative development pathways. The framework will be tested in two pilot case studies in rapidly growing African regions, the Zambezi and the Omo Rivers basins. The DAFNE project integrates a high level integrated water resources management approach, which addresses the WEF nexus explicitly and from a novel perspective, and aims at promoting a green economy in regions where infrastructure development and expanding agriculture have to be balanced with the local social, economic, and environmental dimensions. DAFNE quantify and analyse the WEF nexus with respect to the trade-off between conflicting objectives, such as hydropower production vs. irrigation, land exploitation vs. conservation. The nexus will also be translated into use and non-use economic values and impact on terrestrial and aquatic ecosystems, also taking into consideration ecosystem services. For more information about the project please refer to http://dafne-project.eu/.

top

 

Type D: Optimisation work on a PRACE Tier-1 (2)

Deep Learning applications for big medical image archives: detection of diseases and generation of artificial images

Project Name: Deep Learning applications for big medical image archives: detection of diseases and generation of artificial images
Project leader: Mr Dzmitry Paulenka
Research field: Physiology and Medicine
Resource awarded: 150 000 core hours on Tier-1
Description

We are working on development of new methods and software for computerized disease diagnosis based on Biomedical Images and recent technologies of Deep Learning, Convolutional Neural Networks, and Artificial Intelligence. Our team has access to large-scale medical image data which includes up to 2 000 000 X-ray images of lungs taken from screening and up to 8 000 3D Computed Tomography (CT) scans along with the corresponding annotations provided by doctors. Using such big amounts of data combined with state-of-the-art image analysis methods and software will allow to make a breakthrough in the field of detection of various lung and mediastinum diseases. Additionally, it is assumed that applying modern approaches utilizing Generative Adversarial Networks will allow automatic generation of artificial medical images.

top

Accelerating a high order accurate CFD solver using GPUs

Project Name: Accelerating a high order accurate CFD solver using GPUs
Project leader: Dr Marco Kupiainen
Research field: Mathematics and Computer Sciences
Resource awarded: 150 000 core hours on Tier-1
Description

Heterogeneous HPC architectures are increasingly prevalent in the Top500 list (137 systems in the latest released version, up from 110 six months ago), with CPU based nodes enhanced by accelerators or co-processors optimized for floating-point calculations. For GPU accelerators, applications are typically rewritten in a low-level language such as CUDA or OpenCL. On the other hand, OpenACC enables existing HPC application codes to run on accelerators with minimal source-code changes. This is achieved with the compiler directives and API calls for generating optimized codes and the user guiding performance only where necessary. ESSENSE is a computational fluid dynamics (CFD) code for the simulations of compressible flows. The code is widely used in a broad range of applications, including the study of aerodynamics, aeroacoustics, the climate modeling and in applied mathematics research. In the code, the Navier-Stokes equations are discretized in space by using high-order finite difference method, which is based on summation-by-part (SBP) operators. The SBP operators are written as sparse matrix-vector products in the code. In this proposed project, together with the PRACE expert at PDC Center for High Performance Computing, we will follow our previously developed work and take advantage of the optimized results to port ESSENSE to GPU-accelerated systems. The project will focus on porting and optimizing the most time-consuming parts of ESSENSE to the GPU systems, namely the sparse matrix-vector products using the OpenACC directives and OpenACC API with NVIDIA CUDA libraries (e.g. cuSPARSE). We will also improve the MPI point-to-point communications to decrease the latency of data transfer between sub-domains. In summary, we will address two main tasks in the proposed project: 1) To accelerate ESSENCE code for GPU systems using the OpenACC directives and OpenACC API with CUDA libraries. 2) To improve the MPI point-to-point communications using non-blocked communications and investigate the one-side communication (e.g. PGAS programming model) or hybrid MPI/OpenMP.

top