PRACE Preparatory Access – 31th cut-off evaluation in December 2017

Find below the results of the 31th cut-off evaluation of 1 December 2017 for the PRACE Preparatory Access.

Projects from the following research areas:

 

High-Order Methods for LES 2 (HOMLES2)

Project Name: High-Order Methods for LES 2 (HOMLES2)
Project leader: Dr Koen Hillewaert
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

The objective of the project “High-order methods for LES 2” (HOMLES2) is to assess the parallel performance of high-order Computational Fluid Dynamics (CFD) solvers when dealing with the Implicit Large Eddy Simulation (ILES) of compressible flows. The HOMLES2 project is a follow up of the HOMLES project. In ILES no explicit subgrid model is included but it is the numerical discretization itself that acts like a SGS. The Discontinuous Galerkin (DG) and, more recently, the Flux Reconstruction (FR) methods, thanks to their favourable numerical properties, showed to be well suited for the high-fidelity ILES of turbulent flows. The interest of the scientific community in such methods is demonstrated by the ongoing EU Horizon 2020 project TILDA (Towards Industrial LES/DNS in Aeronautics – Paving the Way for Future Accurate CFD) (http://cordis.europa.eu/project/rcn/193362_en.html) grant agreement No.635962. The HOMLES project involves three partners with a strong background in the development of modern, efficient and accurate solvers: the University of Bergamo, Italy (UNIBG), CENAERO, Belgium and NUMECA, Belgium. The partners of HOMLES are also part of the TILDA consortium, coordinated by NUMECA. This project is intended as a further joint effort towards the very accurate simulation of turbulent flows on industrial configurations. With the HOMLES project the partners want to prove the scaling of their solvers on a number of platforms as large as possible. With this in mind, the partners are asking with this preparatory project the access to several architectures among those available within the call. The partners will evaluate their solver scalability on computational problems close enough to the final intended application, i.e. turbomachinery, possibly comparing the parallel performance of explicit versus implicit time integrators.

top

Three-phase capillary pressure, hysteresis and trapping in mixed-wet rock

Project Name: Three-phase capillary pressure, hysteresis and trapping in mixed-wet rock
Project leader: Dr Johan Olav Helland
Research field: Earth System Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

Capillary-dominated pore-scale displacement of three fluid phases in natural porous media is important in several scientific fields and applications, including utilization and long-term storage of CO2 in geological formations, improved oil recovery strategies in mature hydrocarbon reservoirs, and remediation techniques for soil contamination of non-aqueous phase liquids (NAPL). We have developed a multiphase level set (MLS) method for capillary-controlled displacement on the pore scale to investigate three-phase capillary pressure, hysteresis and trapping on 3-D porous rock geometries. The method computes local energy minima and employs recently presented multiphase techniques to account for wetting state, fluid interfaces and volume preservation. In three-phase flow, initially trapped fluid ganglia surrounded by two other continuous fluids and a solid phase can mobilize under action of capillary forces through so-called double displacements. Such displacements will either involve formation of triple lines (where all three fluids meet) or spreading oil layers separating water and gas. We have simulated this behaviour on small pore geometries (Helland and Jettestuen (2016), Water Resour. Res., doi: 10.1002/2016WR018912), but evaluating the implications on macroscale require pore-scale simulations on larger, representative rock domains. Further, accurate modelling of oil layers on real rock geometries requires adaptive mesh refinement (AMR). We have implemented AMR and parallelism by coupling our MLS codes to the software framework SAMRAI, developed at Lawrence Livermore National Laboratory (USA) for massively parallel applications. We have run our code on the supercomputer Abel at University of Oslo (which is part of the national HPC infrastructure of Norway). Two-phase flow simulations on modestly sized rock samples show that pore filling during drainage occurs as cooperative events with spontaneous fluid redistribution and abrupt pressure drops (Haines jumps), while for imbibition such cooperative effects are less significant due to swelling of wetting phase films, consistent with experiments (Helland et al. (2017), Geophys. Res. Lett., doi: 10.1002/2017GL073442). We expect a similar or even more complex behavior of drainage and imbibition in three-phase flow. However, three-phase simulations are more computationally demanding than corresponding two-phase problems and will generally require a larger number of processors than typical simulation runs on Abel will allow. We will use the PRACE preparatory access project to demonstrate scalability of our MLS code for large numbers of processors, and to determine the required computational resources and timescales of typical simulation runs with high (adaptive) grid resolution for full gas and water invasion cycles in sufficiently large rock domains at different wetting states.

top

NATL60 (North Atlantic at 60th of a degree of resolution)

Project Name: NATL60 (North Atlantic at 60th of a degree of resolution)
Project leader: Dr Laurent Brodeau
Research field: Earth System Sciences
Resource awarded: 100 000 core hours on MARCONI – KNL, 50 000 core hours on MareNostrum, 100 000 core hours on Hazel Hen
Description

This project aims to determine the contribution of the kilometric-scale oceanic flow features, known as submesoscale turbulence, in various components of the global oceanic circulation known for their importance in regulating the Earth’s climate. To achieve this, realistic numerical simulations of the full 3D oceanic flow in the North Atlantic basin, at ultra-high horizontal and vertical resolution, will be designed and performed using the NEMO ocean model. This unprecedented type of high-resolution ocean flow simulations, at the vanguard of ocean modeling (~1 km horizontal grid resolution on 300 vertical levels, with the inclusion of tidal motion), has only been made achievable recently thanks to the advance in HPC capabilities of modern supercomputers. Beside helping the community getting a clearer picture of the interactions between submesoscale turbulence features and the larger-scale components of the ocean flow, the goal of the proposed simulations is to prepare and develop a system ready to assimilate, interpret, and further expand (e.g. below the surface) the invaluable source of information that will be obtained from the upcoming SWOT satellite altimetry mission (https://swot.jpl.nasa.gov/); which will provide the community with observations of the surface ocean dynamics at an unprecedented small-scale resolution. This new type of highly-accurate and high spatial resolution altimetric satellite observation from the SWOT mission, combined to the set of proposed ground-breaking ocean simulations, is expected to lead to major breakthroughs in our understanding of the roles played by small-scale ocean turbulence in influencing the climate.

top

Calculation of non-linear absorption and emission spectra of active probes in biological environments: how the atomistic picture of fluorescent molecules enables simulations of advanced fluorescence experiments

Project Name: Calculation of non-linear absorption and emission spectra of active probes in biological environments: how the atomistic picture of fluorescent molecules enables simulations of advanced fluorescence experiments
Project leader: Dr Stefan Knippenberg
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MARCONI – Broadwell, 50 000 core hours on MareNostrum, 50 000 core hours on Curie
Description

The theme of the proposal can be qualified in light of the two recent years’ Nobel prizes in chemistry, awarded to multiscale modeling (2013) and single molecule fluorescence spectroscopy (2014). The first reflects the outstanding possibilities of modern computational modeling to interpret and predict the properties of molecular systems embedded in complex media, the second reflects the use of spectroscopy to record the information carriers – the scattered photons – of such systems. The program by Dr. Stefan Knippenberg combines these notions so that modeling will be used to unravel the important structure-property and structure- function relationships from spectroscopic measurements of biomolecular systems. The applicant will investigate the behavior and the movement of optically active molecules or probes in biological environments, such as bilipid membranes, microbubbles, Langmuir and Langmuir-Blodgett films. He will thus develop in-silico tools to unravel the informational content in spectroscopy of early-stage structural changes at the molecular level that eventually initiate morphological changes at cell and/or organ levels during the disease evolution process. As a result, Stefan’s computational modeling can be used to optimize contemporaneous optical imaging and spectroscopic early stage biomolecular diagnostics of various diseases and provide rational guidance for identifying their early symptoms. In the anticipated theoretical simulations, the environment of the active molecules is accounted for as it may strongly influence their geometric structure and optical properties. To model the biological tissues, the Gromacs molecular dynamics program will be used, paying special attention to counter ions, the inclusion of explicit water molecules, and the motion of the lipids and embedded proteins. Later on, in an advanced theoretical treatment implemented in the Dalton package of programs, quantum mechanical methods along with molecular mechanics/dynamics simulations will be applied to assess the resulting (non)linear optical response of molecules of interest. Profound attention will also be paid to benchmarking the simulated data, which will be mainly generated reverting to time dependent density functional theory, against high level ab initio methods like coupled cluster theory and the various orders of the Algebraic Diagrammatic Construction scheme. The non-linear two-photon absorption spectra as well as the first hyperpolarizability, which gives access to second harmonic generation, will be simulated and the results within the various environments and lipid bilayer phases will be compared. Force field reparametrizations will be performed to enable classical dynamics on the relevant excited state. Via a vast amount of snapshots, insight can be obtained into the fluorescence of the employed probes, into the fluorescence anisotropy, into the probes’ lifetimes and the dependencies of the lifetimes on the neighboring biological environments.

top

Transonic airfoil aerodynamics: preparatory access

Project Name: Transonic airfoil aerodynamics: preparatory access
Project leader: Prof Neil Sandham
Research field: Engineering
Resource awarded: 100 000 core hours on Hazel Hen
Description

We propose to simulate transonic flow over a laminar-flow wing that is designed to maintain a slightly favourable pressure gradient to deliver laminar flow up to the shock interaction on the upper surface. At this point the flow is tripped or naturally transitions to turbulence and a turbulent boundary layer continues, possibly undergoing separation towards the trailing edge. The spanwise direction will be based on periodic boundary conditions, representing a planar or infinitely-swept configuration. The challenge is to run this test case at the highest Reynolds number feasible, of the order of one million. This will provide much-needed databases to study shock-wave/boundary-layer interactions for a complete lifting surface. Successful completion of this project will open up new lines of research into improving the performance of transonic swept wings, which are the basis of air transport today. Other problems that can be addressed include the onset of buffet, receptivity theory and the control of laminar-turbulent transition in swept transonic flow conditions. A code to realise these goals has already been developed and validated and simulations at Reynolds number 500,000 have been run on a Tier 1 machines using up to 5 billion grid points. This initial process is for preparatory access to a Tier 0 machine to confirm code performance on the target machine.

top

ROHITU – Roughness in highly turbulent Taylor-Couette and Rayleigh-Bénard flows

Project Name: ROHITU – Roughness in highly turbulent Taylor-Couette and Rayleigh-Bénard flows
Project leader: Prof Detlef Lohse
Research field: Fundamental Physics
Resource awarded: 50 000 core hours on MareNostrum, 50 000 core hours on Curie, 100 000 core hours on Hazel Hen
Description

Many wall bounded flows in nature, engineering and transport are affected by surface roughness of the wall. Often, this has adverse effects, for example drag increase leading to higher fuel costs. Computationally, it is notoriously difficult to simulate these flows, because of the large scale separation in highly turbulent flows. This becomes especially challenging when irregular boundaries need to be considered. From a physics perspective, many questions are still unanswered, one of the most urgent ones being the effect of roughness topography on the fluid flow characteristics. We plan to investigate the effect of the mean roughness height, the effective slope (ES) and solidity parameters in the transitionally rough regime. In this regime, both the viscous drag and the pressure contribute to the total drag. We use irregular three-dimensional, roughness elements to capture the structure of sand grain roughness. To study the effect of wall roughness, we plan to use two paradigmatic systems in turbulence research, Taylor-Couette (TC) and Rayleigh-Bénard (RB) flow. TC flow is the flow between two concentric rotating cylinders. RB flow is the buoyancy driven flow of a fluid heated from below and cooled from above. Both systems, mathematically similar to each other, are for smooth walls very well studied and supply us with ample data to compare with. In particular we wish to study the effect of wall roughness on fluid flow structures in the vicinity of the wall, higher order statistics, energy budgets and large-scale flow structures (e.g. plume ejection, Taylor rolls and large-scale circulation). We hope to gain detailed insight into the mechanisms underlying a change in drag (TC) or enhanced heat transfer (RB), distinctive to turbulent fluid flow over rough surfaces. We plan to study this using Direct Numerical Simulations (DNS) carried out with the “AFiD” code (https://github.com/PhysicsofFluids/AFiD), which has been successfully used to simulate highly turbulent RB and TC flow. Wall roughness is implemented by means of the Immersed Boundary Method (IBM).

top

Raman spectra of lithium ion battery materials

Project Name: Raman spectra of lithium ion battery materials
Project leader: Prof Ulrich Aschauer
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MARCONI – Broadwell, 50 000 core hours MareNostrum,
Description

In-situ Raman spectra can be used to monitor lithium ion battery operation and to diagnose mechanisms leading to their failure. The interpretation of the Raman spectra is however often difficult as the assignment of modes to particular lattice vibrations is problematic in inherently disordered structures. Raman spectra computed using density functional theory (DFT) can greatly aid in interpreting these experimental results. In this preparatory project, we aim to collect scalability data for calculations required to predict Raman spectra of lithium ion battery materials based on DFT. The goal is to determine the best suitable machine for this kind of calculations and to estimate the amount of resources required for our production runs.

top

ANTIVIRALS AGAINST ZIKA AND DENGUE VIRUSES USING MOLECULAR DYNAMICS AND DOCKING

Project Name: ANTIVIRALS AGAINST ZIKA AND DENGUE VIRUSES USING MOLECULAR DYNAMICS AND DOCKING
Project leader: Dr Vicente Galiano
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

Among the pathogens which cause the higher rates of mortality and morbidity on humans and animals we can name the viruses.Flaviviridae constitute a large family of viruses to which medically highly relevant human pathogens belong. Viruses such as the hepatitis C virus, the Yellow Fever Virus, West Nile virus, Tick-Borne Encephalitis Viruses, Zika and Dengue belong to this family. Dengue (DENV), as well as Zika (ZIKV), cause the most prevalent arthropod-borne viral disease among humans affecting millions of people per year.Both DENV and ZIKV viruses are perhaps the most important emerging viruses which have and will have a profound effect in the near future on human society. cell. This project aims to find effective antiviral peptides and organic molecules against DENV and ZIKV viruses by screening databases containing hundreds of thousands of molecules (SuperNatural_II, Zinc15) and the use of supercomputers. From the experimental knowledge we have on the structural and non-structural proteins of the DENV and ZIKV viruses interacting with biomembranes, ie, structural proteins E, M and C and non-structural proteins NS2A, NS2B, NS4A and NS4B, we will use their known three-dimensional structures and/or three-dimensional structures obtained by homology. Therefore, in this project we propose to study the molecular dynamics of proteins and derived-peptides and their interaction with biological membrane models to later perform in silico screening of organic molecules obtained from the aforementioned databases.

top

The regulatory mechanism of the SHP-2 protein: a Molecular Dynamics investigation

Project Name: The regulatory mechanism of the SHP-2 protein: a Molecular Dynamics investigation
Project leader: Prof Gianfranco Bocchinfuso
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MARCONI – Broadwell
Description

This proposal is preparatory for a PRACE project aimed at studying the regulatory mechanism of the SHP-2 phosphatase protein. SHP-2 plays a critical role in regulation of a number of important signaling pathways in the cell and it is the first identified phosphatase for which amino acid substitutions increasing the activity have been associated with the development of cancers [1]. The SHP-2 structure includes two N-terminal Src homology 2 domains (N-SH2 and C-SH2) followed by a protein tyrosine phosphatase (PTP) domain, containing the catalytic site, and a C-terminal tail. In the basal state, SHP-2 is auto-inhibited, as a loop of the N-SH2 domain is inserted into the PTP active site (“closed” conformation). SHP-2 activation is mediated by interactions between its SH2 domains with partners containing short amino acid motifs comprising a phosphotyrosyl residue [2]. This event causes a rearrangement of the domains, whose final effect is a greater accessibility of the active site on the PTP domain (“open” conformation). Many pathological amino acid substitutions favor the “open” state [2,3]. Notwithstanding the biological relevance of this process, the active state is poorly characterized from a structural point of view. In the coming PRACE project, we plan to investigate the mechanisms leading to the SHP-2 activation. In particular, we will simulate the activation step, starting from the “closed” state structure (available in the Protein Data Bank) to follow the transition towards the “open” state, in which the PTP catalytic site is accessible to the target proteins. Since active SHP-2 is the prevalent state induced by pathogenic mutations, a deeper understanding of its structure and dynamics is expected to better elucidate the role of both the SH2 domains in the mechanism of SHP-2-mediated signal transduction, as also the effects of uncharacterized lesions. Of note, these studies will be carried out in the framework of a bigger project, funded by the Italian private foundation AIRC (Associazione Italiana per la Ricerca sul Cancro), coordinated by Lorenzo Stella (a “Collaborator” in this project) in which experimental studies are carried out. This tight collaboration with experimentalists will give the possibility to compare computational and experimental data. References: 1] Tartaglia M., et al. Nat. Genet. 2003, 34:148-150; 2] Bocchinfuso G., et al. Proteins 2007, 66:963-974; 3] Tartaglia M., et al. Am. J Hum. Genet. 2006, 78:279-290.

top

Direct numerical simulation of the sediment transport in a turbulent oscillatory boundary layer

Project Name: Direct numerical simulation of the sediment transport in a turbulent oscillatory boundary layer
Project leader: Prof Giovanna Vittori
Research field: Fundamental Physics
Resource awarded: 100 000 core hours on MARCONI – KNL
Description

MOTIVATION The reliable prediction of the initiation of sediment transport and an accurate evaluation of the time development of both sediment transport rate and bottom morphology in the coastal region is of fundamental importance in a large number of applications, including offshore and coastal structures, underwater military operations, the preservation of benthic communities and the prediction of transport of toxic chemicals. Almost all actual methods, which are used to determine the conditions leading to sediment motion and to quantify the sediment transport rate, are based on empirical approaches which are tuned using steady flow data and neglect important aspects of the phenomenon when the sediment motion is driven by sea waves. The assumption that the unsteadiness of the driving flow has a negligible effect on the dynamics of the sediment is perhaps the reason of the deficiencies of the actual approaches in predicting the sediment transport rate, which are characterized by relative errors that can easily overcome 100%. Moreover, we do not fully understand how much the turbulent fluctuations, the mean stress (i.e. drag force) and the flow acceleration (added mass effect) contribute to the overall sediment transport in the oscillatory boundary layer. We also lack experimental data describing the statistical distribution of the turbulent bed stress, the streamwise velocity perturbations and the pressure in the oscillatory boundary layer. Such detailed information would be essential to develop probabilistic modeling that accounts for the intermittent nature of sediment transport. At the present, the detailed investigation of the nearbed turbulence in an oscillatory flow and the associated hydrodynamic forces that mobilize sediment particles is only possible by the means of Direct Numerical Simulations (DNS). The DNS approach is ideally suited to generate the amount of statistical data needed for probabilistic description of the considered sediment transport problem. LONG TERM GOALS Our research goals are to clarify the effects of the unsteadiness of the forcing flow and those of the coherent vortex structures, which cause local rapid variations of the velocity and of the shear stress, on the dynamics of sediment particles and relate them to the generation of sediment transport. The goals will be achieved by using Direct Numerical Simulations (DNSs) of Navier-Stokes and continuity equations without the use of any turbulence model and computing the flow around the sediment grains. PREPARATORY ACCESS Presently, we request the preparatory access in order to perform preliminary tests of the code in order obtain the strong and/or weak scaling of the code aiming at submitting a full project proposal responding to the PRACE 16th call.

top

Calculation of non-linear absorption and emission spectra of active probes in biological environments: how the atomistic picture of fluorescent molecules enables simulations of advanced fluorescence experiments

Project Name: Calculation of non-linear absorption and emission spectra of active probes in biological environments: how the atomistic picture of fluorescent molecules enables simulations of advanced fluorescence experiments
Project leader: Dr Stefan Knippenberg
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

The theme of the proposal can be qualified in light of the two recent years’ Nobel prizes in chemistry, awarded to multiscale modeling (2013) and single molecule fluorescence spectroscopy (2014). The first reflects the outstanding possibilities of modern computational modeling to interpret and predict the properties of molecular systems embedded in complex media, the second reflects the use of spectroscopy to record the information carriers – the scattered photons – of such systems. The program by Dr. Stefan Knippenberg combines these notions so that modeling will be used to unravel the important structure-property and structure- function relationships from spectroscopic measurements of biomolecular systems. The applicant will investigate the behavior and the movement of optically active molecules or probes in biological environments, such as bilipid membranes, microbubbles, Langmuir and Langmuir-Blodgett films. He will thus develop in-silico tools to unravel the informational content in spectroscopy of early-stage structural changes at the molecular level that eventually initiate morphological changes at cell and/or organ levels during the disease evolution process. As a result, Stefan’s computational modeling can be used to optimize contemporaneous optical imaging and spectroscopic early stage biomolecular diagnostics of various diseases and provide rational guidance for identifying their early symptoms. In the anticipated theoretical simulations, the environment of the active molecules is accounted for as it may strongly influence their geometric structure and optical properties. To model the biological tissues, the Gromacs molecular dynamics program will be used, paying special attention to counter ions, the inclusion of explicit water molecules, and the motion of the lipids and embedded proteins. Later on, in an advanced theoretical treatment implemented in the Dalton package of programs, quantum mechanical methods along with molecular mechanics/dynamics simulations will be applied to assess the resulting (non)linear optical response of molecules of interest. Profound attention will also be paid to benchmarking the simulated data, which will be mainly generated reverting to time dependent density functional theory, against high level ab initio methods like coupled cluster theory and the various orders of the Algebraic Diagrammatic Construction scheme. The non-linear two-photon absorption spectra as well as the first hyperpolarizability, which gives access to second harmonic generation, will be simulated and the results within the various environments and lipid bilayer phases will be compared. Force field reparametrizations will be performed to enable classical dynamics on the relevant excited state. Via a vast amount of snapshots, insight can be obtained into the fluorescence of the employed probes, into the fluorescence anisotropy, into the probes’ lifetimes and the dependencies of the lifetimes on the neighboring biological environments.

top

LES of Compressor Turbulence

Project Name: LES of Compressor Turbulence
Project leader: Prof Luca di Mare
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

The project is aimed at conducting Large Eddy Simulations (LES) of turbulent flow in gas turbine compressor passages. The geometry to be simulated represents the stator passages in a research compressor rig in the Whittle laboratory at Cambridge University. Computations will be performed on sectors of the annlus comprising between one and five passages. The domain sizes will vary between 150 and 450 million cells and will require between 44 and 220 nodes (10,560 cores) on the MareNostrum IV system. The computations are intended to study the computational performance of the code as well as to generate data on turbulent structures in compressor passages. Data will be collected to form spectra, single- and two-point statistics as well as to compute budgets of Reynolds stress tensors. The project is the next step in a long-term numerical and experimental investigation into turbulent flow in gas turbine compressor passages [1,2]. [1] L di Mare, T O Jelly and I J Day, Angular response of hot wire probes, Measurement Science and Technology, 28(3), 2017 [2] T. O. Jelly, I.J. Day, L. di Mare, Phase-averaged flow statistics in compressors using a rotated hot-wire technique, 2017 Experiments in Fluids 58, 2017.

top

Code scalability tests for atomistic molecular dynamics simulations of mitochondrial complex I

Project Name: Code scalability tests for atomistic molecular dynamics simulations of mitochondrial complex I
Project leader: Dr Vivek Sharma
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100 000 core hours on MARCONI – KNL, 50 000 core hours on MareNostrum, 100 000 core hours on Piz Daint
Description

Energy is central to our lives either in the form of heat, light or fuel. A number of engineering accomplishments have led to significant advancements in energy generation from renewable sources, but growing demands in the world seriously outpace the supply. However, in this respect we have something to learn from micro-organisms and mitochondria, which generate energy in the form adenosine triphosphate (ATP) with very high efficiency and low toxicity. One such enzyme, mitochondrial respiratory complex I, contributes to about 40% energy (ATP) generation in mitochondria. In this project, we will perform scalability tests using GROMACS software on a very large model system of mitochondrial complex I, comprising ca. 1.5 – 2 million atoms. We will accomplish these scalability tests by performing fully-atomistic molecular dynamics simulations using the highly-parallelized and highly efficient GROMACS program.

top

Scalability tests

Project Name: Scalability tests
Project leader: Prof Alfredo Soldati
Research field: Engineering
Resource awarded: 100 000 core hours on MARCONI -KNL,
Description

The current project is intended to run weak and strong scalability analysis of our proprietary code. These analysis will be used for the 16 Prace call closing on the 21 November 2017. This code performs Direct Numerical Simulation of multiphase flows in a rectangular channel with the eventual presence of a surface active agent and has already been tested, benchmarked and run on several different architectures like Intel Xeon E5 (Marconi A1 Broadwell at CINECA and VSC-3 at Vienna Scientific Cluster) and IBM BG/Q (Vesta at Argonne National Laboratory). For a complete scalability analysis we expect that 100000 core hours will be needed.

top

NATL60 (North Atlantic at 60th of a degree of resolution)

Project Name: NATL60 (North Atlantic at 60th of a degree of resolution)
Project leader: Dr Laurent Brodeau
Research field: Earth System Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

This project aims to determine the contribution of the kilometric-scale oceanic flow features, known as submesoscale turbulence, in various components of the global oceanic circulation known for their importance in regulating the Earth’s climate. To achieve this, realistic numerical simulations of the full 3D oceanic flow in the North Atlantic basin, at ultra-high horizontal and vertical resolution, will be designed and performed using the NEMO ocean model. This unprecedented type of high-resolution ocean flow simulations, at the vanguard of ocean modeling (~1 km horizontal grid resolution on 300 vertical levels, with the inclusion of tidal motion), has only been made achievable recently thanks to the advance in HPC capabilities of modern supercomputers. Beside helping the community getting a clearer picture of the interactions between submesoscale turbulence features and the larger-scale components of the ocean flow, the goal of the proposed simulations is to prepare and develop a system ready to assimilate, interpret, and further expand (e.g. below the surface) the invaluable source of information that will be obtained from the upcoming SWOT satellite altimetry mission (https://swot.jpl.nasa.gov/); which will provide the community with observations of the surface ocean dynamics at an unprecedented small-scale resolution. This new type of highly-accurate and high spatial resolution altimetric satellite observation from the SWOT mission, combined to the set of proposed ground-breaking ocean simulations, is expected to lead to major breakthroughs in our understanding of the roles played by small-scale ocean turbulence in influencing the climate.

top

Characterization of maltose translocation in the MalFGK2E transporter

Project Name: Characterization of maltose translocation in the MalFGK2E transporter
Project leader: Dr Cláudio Soares
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

ATP binding cassette (ABC) importers are transporter proteins involved in nutrient and metal uptake and are exclusive of bacteria. Some play a role in virulence, such as the zinc importer ZnuABC present in Brucella abortus (Tanaka, et al., 2017). ABC importers are also relevant for the intake of aminoacids and other nutrients. Therefore, they are a privileged gateway to the entrance of molecules in bacteria, which makes them a potential target for therapeutics against pathogenic bacteria. One such example is the delivery of synthetic antibiotics that resemble the natural substrates (Tanaka et al., 2017). Hence, it becomes quite relevant to understand the mechanisms of substrate translocation and binding to ABC importers. The Escherichia coli MalFGK2E maltose importer is a type I ATP-binding cassette (ABC) importer responsible for the uptake of malto-oligosaccharides. It is one of the most studied ABC transporters and it is a model system for type I importers. The MalFGK2E maltose importer is constituted by two transmembrane domains: MalF and MalG. Additionally, it contains two ATP-binding MalK domains and a substrate binding protein MalE that fetches maltose from the cytoplasm. Structurally it is known that the transporter alternates between outward-facing and inward-facing conformations, as identified by X-ray crystallography ((M L Oldham & Chen, 2011; Michael L Oldham & Chen, 2011)). However, the molecular details underlying the translocation process and the events required to product release remain undisclosed. In this project, we aim to characterize different stages of the transport process, as well as the energetics of maltose translocation in each step, using equilibrium and non-equilibrium molecular dynamics simulations. The stages we will simulate are: the pre-hydrolysis state with ATP, the post-hydrolysis state with ADP and phosphate, and the state after the exit of hydrolysis products. We will use metadynamics methods to understand the energy landscape in each state and obtain a translocation energy profile. Based on similar studies, we think the most suitable collective variables to describe translocation in metadynamics methods are the coordinate in the transmembrane axis of channel and the angle between maltose and the transmembrane axis (Jensen, Yin, Tajkhorshid, & Schulten, 2007)(Bajaj et al., 2016). PRACE resources will allow running non-equilibrium simulations with a high scalability in order to get microsecond sampling in a feasible time. We aim to use the preparatory access to determine the scalability of a system consisting of a MalFGK2E embedded in a 480 POPC membrane applying the metadynamics algorithm. We will use GROMACS 5.0 (Tribello, Bonomi, Branduardi, Camilloni, & Bussi, 2014) and the PLUMED 2.0 plugin to simulate this system (Tanaka et al., 2017).

top

TARONGIUM – The magnetic field Amplification in ROtating proto-Neutron stars with Greatly resolved magnetic Instabilities Using MP9 – Scalings

Project Name: TARONGIUM – The magnetic field Amplification in ROtating proto-Neutron stars with Greatly resolved magnetic Instabilities Using MP9 – Scalings
Project leader: Prof Miguel-Ángel Aloy
Research field: Universe Sciences
Resource awarded: 50 000 core hours on MareNostrum, 50 000 core hours on SuperMUC
Description

The most massive stars in the universe end their lives, when their centre collapses to a proto-neutron star (PNS). The formation of a PNS produces a shock wave which stalls inside the star. Whether or not and how this shock can be reviewed is one of the most challenging questions of astrophysics. Should it be the case, the star will explode as a core-collapse supernova (CCSN), otherwise it will collapse to a black hole. Many mechanisms of the shock revival have been proposed, eg. neutrino heating or energy transfer by magnetic fields. The latter process ,often invoked for the most energetic CCSNe, depends on the possibility to amplify the weak magnetic field of the progenitor to a dynamically important strength. One of the most promising amplification mechanisms is the magnetorotational instability which can grow on shorter time scales than the time between the formation of the PNS and the supernova (SN) explosion. We plan to perform 3D numerical magnetohydrodynamic (MHD) global simulations of post-collapse cores of massive stars including all possible physical elements that may affect the growth of the magnetic fields initially present in the progenitor star, i.e. an approximate relativistic treatment of the gravitational field and an accurate treatment of the neutrino transport. The foremost goal of our study is to assess whether the action of MRI can amplify the magnetic field from (realistic) to dynamically relevant values during the PNS phase following the collapse of a massive stellar core. That strong magnetic field could tap the rotational energy of the core, power MHD turbulence, and become a potentially important ingredient in rapidly-rotating CCSNe of progenitors of gamma-ray bursts (GRBs), and magnetars.

top

Scalability tests of quantum espresso pw.x for metal-reducible oxides interfaces and alloyed systems

Project Name: Scalability tests of quantum espresso pw.x for metal-reducible oxides interfaces and alloyed systems
Project leader: Ms Giulia Righi
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MARCONI – Broadwell,
Description

In the present project we intend to test the scalability of the code PW.x of Quantum Espresso for applications to the study of the reducibility and catalytic properties of systems constituted of ceria (CeOx) and Ag. This project intends to build the set-up for a further exploration of complex new metal-oxide nanostructured systems which are synthesized by experimental groups we are collaborating with. The idea is to study new materials for application as electrodes in PEMFC fuel cells to substitute the costly and rare Pt electrodes. Materials based on CeO2 have been receiving and are receiving much attention due to the cerium oxide high reducibility, i.e. due to the possibility for Ce ions to quickly modify their oxidation state in a reversible way. So far ceria has been studied mainly as a support for catalyst metal particles. However, Ag has never been considered in these studies. It was found that the electronic interactions between the metal nanoparticles (NP) and the oxide supports control the functionality of the materials, for example, the stability, the activity and the selectivity of the catalysts. The scope of our research is to design the best ceria-Ag combinations (Ag as dopant, as a substrate, or as a nanocluster inclusion) that could increase the catalytic activity of the surface for the oxidation of the H2 molecule. In previous studies the combination of ceria with metals (mainly Pt and Cu) was considered only for the case of the (111) orientation but we plan to extend our analysis to more surface-interface orientations for subsequent applications to nanocrystalline systems. In this proposal a number of different prototypical systems of CeOx/Ag will be considered (bulk, interface and surface) with a variable number of atoms. We intend to find the best parameters, number of nodes and cpus and Openmp and MPI procs (Quantum Espresso has an hybrid model of parallel programming) for the optimal resources use for our calculations. Then, we intend to study how these resources can be optimized for each variation of the number of atoms in the prototypical systems. Scalability tests will be performed as a function of the number of MPI procs and the node number for fixed (but large) unit cells and then with the dimension of the cells and the number of atoms.

top

Intermolecular Quantum Dynamics

Project Name: Intermolecular Quantum Dynamics
Project leader: Prof Edit Matyus
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum, 50 000 core hours on Curie, 100 000 core hours on Hazel Hen
Description

Quantum dynamical properties of molecular complexes are studied by directly solving the rovibrational Schrödinger equation using the GENIUSH code [E. Mátyus, G. Czakó, and A. G. Császár, J. Chem. Phys. 130, 134112 (2009); C. Fábri, E. Mátyus, and A. G. Császár, J. Chem. Phys. 134, 074105 (2011)]. The computed rovibrational transition energies and their comparison with high-resolution spectroscopic measurements provide a stringent test of the underlying intermolecular potential energy surface (and allow for possible refinement of the interaction model). The computed rovibrational energies and wave functions provide a detailed and complete dynamical information about the molecular complex, and hence the properties of the molecular interactions (weakly coupled rotors, hydrogen bonds, tunneling, etc.) can be studied in detail. The direct solution of the rovibrational Schrödinger equation is a highly computational intensive task and requires large-memory and multi-core architectures. Thereby, supercomputing resources are inevitable for using the developed codes for practical molecular applications.

top

NextGenGalMod

Project Name: NextGenGalMod
Project leader: Dr Bruno Henriques
Research field: Universe Sciences
Resource awarded: 50 000 core hours on SuperMUC
Description

This project aims at fully exploring the highly-dimensional parameter space of a new version of the Munich semi-analytic model of galaxy formation, L-Galaxies. This will allow us to gain critical insight into how different physical processes interplay to shape the internal distribution of gas and stars in galactic discs. We have recently developed an extension of the L-Galaxies galaxy formation model in order to track the radial distribution of different cold gas phases, star formation, supernova feedback and chemical enrichment in galaxy discs. This is a crucial step in order to interpret next-generation surveys and required the introduction of new HI and H2 partition recipes, star formation laws and detailed chemical enrichment models. We now plan to analyse the impact of these different choices in global and internal galaxy properties while fully sampling the parameter space using a sophisticated yet efficient MCMC algorithm. A similar approach, allowed us to recently develop the first galaxy formation model correctly describing the observed evolution of two of the most fundamental properties of the galaxy population: mass assembly and star formation rate. We plan on evaluating the results of approximately 500 000 realisations for each of 4 different galaxy formation models, corresponding to the combinations of 2 different cold gas partition recipes and 2 different star formation laws. These are the most uncertain aspect of the new physical model and this study will fully analyse their interplay with the SN feedback and metal enrichment prescriptions and their impact on galaxy properties on global scales. Since the models will be run on top of a representative subset of the Millennium Simulations, we can explore the parameter space and achieve full convergence for a given model with 100 000 CPU hours, corresponding to 100 000 model iterations (i. e. MCMC steps). The 500 000 realisations for each physical model will then allow us to use 5 different observational constraints: the evolution of the stellar mass function, the red fraction, galaxy sizes, cold gas metallicities and cold gas masses. We will then be able to join any combination of observational constraints by multiplying the posterior distributions in parameter space of the individual runs in post-processing. While in the past we have been limited to one “physical model+observational constraint” setup (100,000 steps, 100,000 CPU hours), our proposed evaluation of an unprecedented range of physical models (four), observational constraints (five) and parameter combinations (4*5*100,000 steps = 2 million CPU hours), should allow us to develop the first realistic model of galaxy formation with resolved radial distributions of material in discs, cold gas partition and detailed chemical enrichment.

top

Ab initio Molecular Dynamics simulations of iron lubrication

Project Name: Ab initio Molecular Dynamics simulations of iron lubrication
Project leader: Prof Maria Clelia Righi
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MARCONI – Broadwell, 100 000 core hours on MARCONI – KNL
Description

The goal of the project is to optimize the computational setup of our simulations in order to reach a good scalability for large scale Ab initio Molecular Dynamics simulations performed by using Quantum ESPRESSO package. In particular, we want to study the functionality of solid lubricants, such as graphene, in an iron interface during sliding. We will run a simulation of iron interface lubricated by graphene consisting of about 2400 atoms and in the presence of mechanical stresses. These simulations are very computational demanding because of the large number of atoms, electrons and spin-polarization. To this aim, we consider very important to obtain scalability data in order to apply for a future PRACE call.

top

 

 

Type B: Code development and optimization by the applicant (without PRACE support) (7)

HPC for connected objects

Project Name: HPC for connected objects
Project leader: Dr Bertrand Cirou
Research field: Earth System Sciences
Resource awarded: 100 000 core hours on Piz Daint
Description

The project of the SME AxesSim has been awarded by PRACE SHAPE project in August 2017. With its partners (including Thales communications & security), AxesSim is conceiving a test bench for the electromagnetic design of connected objects. AxesSim is in charge of the simulation software tools attached to the test bench. The objective is to propose to vendors of connected objects measures and simulation tools for optimizing the design. This requires the simulation of electromagnetic waves produced by small antennas and their interaction with biological tissues. Such simulations are complex and expensive. They require to take into account complex geometries. In order to answer to the client in a reasonable time, it is necessary to use highly optimized software, parallel computations and GPUs. This SHPE projects requires 500k computing hours.

top

Fast matrix multiplication and related algorithms

Project Name: Fast matrix multiplication and related algorithms
Project leader: Dr Oded Schwartz
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MARCONI – Broadwell, 250 000 core hours on Hazel Hen
Description

Strassen’s matrix multiplication algorithm (1969) was followed by Winograd’s (1971) who improved it’s complexity by a constant factor. Many asymptotic improvements followed. Yet, they have done so at the cost of large, often gigantic, hidden constants. Consequently, for almost all feasible matrix dimensions Strassen-Winograd’s remains the fastest. The leading coefficient of Strassen-Winograd’s algorithm was believed to be optimal for matrix multiplication algorithms with 2 × 2 base case, due to a lower bound of Probert (1976). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size, the same asymptotic complexity, but with a smaller leading coefficient. To this end, we transform the input matrices to a different basis, which can be done fast. We discuss improvements in the communication costs, it’s effects on parallelization, and the extension of this method to other Strassen-like algorithms. We also prove a generalization of Probert’s lower bound that holds under change of basis. This shows that for matrix multiplication algorithms with a 2 × 2 base case, the leading coefficient of our algorithm cannot be further reduced, hence optimal. In this project we intend to implement, test and benchmark these algorithms and related ones.

top

Skylake-AI

Project Name: Skylake-AI
Project leader: Mr Stephen Blair-Chappell
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MareNostrum
Description

Benchmarking of optimised versions of CAFFE and TENSORFLOW, with comparisons to current publically available versions. Exploration of effect of various Hyper-parameters on the performance on Skylake. Of interest is (a) use of AVX-512 and (b) experimentation with Affinity (c) Scalability.

top

Improving Gysela gyroaverage operator and forward prospects on programming model

Project Name: Improving Gysela gyroaverage operator and forward prospects on programming model
Project leader: Mr Nicolas Bouzat
Research field: Fundamental Physics
Resource awarded: 250 000 core hours on Hazel Hen
Description

Predicting the performance of fusion plasmas in terms of amplification factor, namely the ratio of the fusion power over the injected power, is among the key challenges in fusion plasma physics. In this perspective, turbulence and heat transport need being modeled within a most accurate theoretical framework, using first-principle non-linear simulation tools. The gyrokinetic equation for each species, coupled to Maxwell​ s equations are an appropriate self- consistent description of this problem. Such simulations are extremely challenging and require state-of-the-art high performance computing. Using the non-linear global full-f gyrokinetic 5D code GYSELA, a simulation very close to ITER-size plasmas with kinetic ions and adiabatic electrons has been performed on 8192 cores during 31 days. This type of simulation is at the front-edge of current research in fusion plasma modeling. In each part of the code, parallel algorithms have been designed in order to scale up to thousands of cores using a hybrid MPI/OpenMP approach. Some bottlenecks concerning memory scalability and collective communication costs have been recently removed. With such improvements, the relative efficiency in strong scaling reaches 78% on 65k cores. However some restrictions remain in one of the principal operator: the gyroaverage. The gyroaverage operator averages the cyclotronic motion of the particles around the magnetic field lines to avoid simulating it explicitly. This project intends to improve the parallelization of the gyroaverage and to eliminate the bottlenecks in this operator in order to treat even larger 5D domain sizes. This is mandatory in view of achieving our next objective: to simulate both kinetic electrons and kinetic ions. Another part of the project is the development of a 5D prototype called GYSELA++ using a task-based model (to reduce synchronization times) and using a different mesh structure enabling realistic geometries. This new code will be based on a new domain decomposition over 5 dimensions (r,theta,phi,vpar,mu) instead of 3 and on a reduced polar grid. It is currently being developed on another cluster where it will also be calibrated before proof run on a PRACE machine. We will ask the support team for fine parameters needed to deploy application on Hazel Hen machine. Two weeks are reserved to tune the environment variables and configuration issues to learn how to launch GYSELA on 1k cores and then up to 32k cores.

top

SAIO Test and Optimization WP6.2 Service 3

Project Name: SAIO Test and Optimization WP6.2 Service 3
Project leader: Mr Martin Bidner
Research field: Mathematics and Computer Sciences
Resource awarded: 200 000 core hours on Curie
Description

Since the training process of SAIO for applications and data processing processes consumes a lot of computing resources, it is worth to inspect, if the parallel I/O configuration knowledgebase found on the HPC platform A could be applied to CURIE. This approach could save a lot of computing resources, which are used to searching for optimal I/O configurations, and increase the efficiency of HPC platforms. 2.1 Description of the service: what will be done, why this is relevant; what is current status SAIO is a light weighted and intelligent framework that utilizes machine learning concept to optimize parallel MPI-IO operations. By following the MPI standard and building upon MPI-IO library, it is compatible and scalable with MPI based applications as well as portable across multiple HPC platforms. In addition, it frees the users from struggling with different I/O strategies or I/O configurations for their applications through setting the MPI info objects transparently. Using the built-in prototypical MPI-IO training process, SAIO can find out the optimal combination of MPI info objects for specified applications by consuming some extra computing hours. These found MPI info objects build a knowledgebase for the real production process. While SAIO applies the MPI info objects from this knowledgebase, users benefit from a relatively higher I/O performance even without knowing the existence of SAIO. The prototype is approved on Hazel Hen and Laki Cluster at HLRS. 2.2 Operational plan: describe precise outcomes that will be reached and how will you achieve them. • Prototypical deployment of the tool on two additional PRACE sites • Analyzing the parallel I/O specifications of testing partner’s HPC platform with the help of local support • Evaluating SAIO with the help of local support • Analyzing the relationship of the parallel I/O configuration knowledgebase among the different HPC platforms • Trying to find mapping mechanisms for these different parallel I/O configuration knowledgebases o Testing SAIO on diverse HPC platforms with similar architecture and file system o Using IOR benchmark and possible applications to generate small I/O configuration knowledgebase on these HPC platforms o Trying to figure out the relationship of the I/O configuration knowledgebase among these HPC platforms o Mapping, Transferring and applying the I/O configuration knowledgebase among these HPC platforms • Why this is relevant: o Saving a lot of computing resources searching for the optimal parallel I/O configurations on diverse HPC platforms o Saving manpower for consulting the parallel I/O operations of diverse customer projects of different computing centers o Increasing the efficiency of the HPC platforms of these computing centers.

top

Distributed Computation of Matrix Inverse using a Probabilistic Approach

Project Name: Distributed Computation of Matrix Inverse using a Probabilistic Approach
Project leader: Prof Juan Acebron
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MARCONI -Broadwell
Description

Current (and future) applications require increasingly powerful and efficient resources. In recent years, the hardware has experienced extraordinary advances, more than any other scientific discipline. The reality today is that the most advanced computer systems are made up of millions of cores. However, it is now the software, in particular parallel numerical algorithms, which proves to be the weakest element in solving problems. Large-scale computations have seen an intolerable waste of resources, making it impossible in some cases to exploit the full potential of available resources, mostly due to communication overheads. The study of the properties of complex networks has been the topic increasingly intense research in the last few years. The reason for this interest is that complex networks arise in many different groundbreaking areas of science. This is the case for networks arising in technological, social, biological, and others. Important metrics, such as entropy, node centrality, and communicability require the computation of a function of the network adjacency matrix, an operation that is not viable for large networks. In many cases the representation of the adjacency matrix for these networks is only possible because they are naturally sparse. Unfortunately, in general, the function of a matrix is a full matrix that simply cannot be represented due to its large size. This limitation severely hinders the analysis of networks of interest. Moreover, even for smaller matrices that allow the function of the matrix to fit in the available memory, classical methods for the computation of the function are not easily parallelizable. These methods require an amount of communication that reduces significantly the efficiency and scalability of a parallel solution. A new computational paradigm is proposed as solution. We are developing probabilistic numerical methods for the computation of different operations on matrices, in particular the computation of the inverse, and the exponential. These methods are intrinsically parallel. More importantly, they allow the computation of individual positions of the inverse matrix, hence any operation is feasible on matrices of arbitrary size as long as its representation is available..

top

Probing environmental effects for ultrafast spectroscopy of molecules using stochastic Schroedinger equation and quantum chemistry

Project Name: Probing environmental effects for ultrafast spectroscopy of molecules using stochastic Schroedinger equation and quantum chemistry
Project leader: Dr Emanuele Coccia
Research field: Chemical Sciences and Materials
Resource awarded: 200 000 core hours on Curie
Description

Revealing possible long-living quantum coherence in ultrafast processes allows detecting genuine quantum mechanical effects in molecules. To investigate such effects from a quantum chemistry perspective, we have developed a method for the time evolution of molecular systems based on ab initio calculations that include the effect of an environment, in terms of dephasing, and relaxation of the molecular wave function. Time evolution of the multi-electron wave function of a molecular target under the influence of an electromagnetic field is simulated using the homemade suite of codes named WaveT. The molecule can interact with an embedding solvent, a nanostructure and a bath. Interaction with the solvent and/or the nanostructure is described by means of a polarizable continuum model, while effects of the environment are represented in the framework of stochastic Schrödinger equation (SSE). SSE is defined in the general theory of open systems, in which the concept of classical statistical mixture of quantum states is introduced to describe the interaction with the environment. A real-time SSE propagation produces the same density matrix (i.e, the standard theoretical quantity used to describe open systems) obtained by solving the corresponding master equation, in the limit of a large number of independent quantum realizations. Starting from the embarrassing parallel nature of SSE implemented in WaveT, the main goal of the present project is evaluating how to exploit this parallelism on a Tier-0 system, with an efficient balance between the number of quantum trajectories and that of the MPI tasks. A key step will be given by the detailed investigation of the role of the number of independent trajectories and that of quantum states of the systems in affecting the overall performance. In other words, modifications will be particularly oriented to make the wave function propagation optimal to manage the statistical error when the number of states increases, as in the case of vibronic expansion of the molecular wave function. A vibronic expansion of the molecular wave function can easily involve several thousands of states. The question to be answered is therefore the following: how many trajectories, i.e. MPI tasks, are needed to get a desired accuracy, i.e. a fixed statistical error, as a function of the number of states of the molecular wave function? Achieving a (near-)linear scaling up to 10^5 MPI tasks, with a one-to-one correspondence between trajectories and MPI tasks, represents our major computational objective. Additional work will be done to find and eventually remove bottlenecks and to improve matrix/matrix and matrix/vector multiplications, especially occurring in the propagation part of SSE with a solvent and/or a nanostructure. We will able to study ultrafast processes for multiple chromophores with an explicit vibronic structure (based on quantum chemistry calculations) within a protein, or multiple chromophores embedded in a solvent or interacting with nanostructures. Furthermore, optimization work on WaveT represents a key objective of the EU grant ERC-CoG-2015 No. 681285 “TAME-Plasmons”.

top

 

Type C: Code development with support from experts from PRACE (1)

Optimization and scalability of the tmLQCD package for production of gauge configurations at the physical pion point

Project Name: Optimization and scalability of the tmLQCD package for production of gauge configurations at the physical pion point
Project leader: Prof Silvano Simula
Research field: Fundamental Physics
Resource awarded: 200 000 core hours on MARCONI – KNL
Description

The accurate evaluation of many physical quantities relevant for the interpretation of ongoing and planned experiments in hadronic and flavor physics, require the use of QCD simulations on the lattice. The European Twisted Mass Collaboration (ETMC) has produced many gluon field configurations including the complete effects of the first two quark generations in the sea, the light u- and d-quarks and the more massive strange and charm quarks (Nf = 2+1+1). Adopting the maximally Wilson twisted-mass fermionic action, ETMC has already provided a number of remarkable results for both meson and baryon physics. The major systematic error still present in the ETMC simulations at Nf = 2+1+1 comes from the extrapolation of the results, obtained in the pion mass range 220 – 450 MeV, down to the physical point at ~ 140 MeV. The goal is therefore to generate gauge ensembles with Nf = 2 +1 + 1 dynamical flavors having pion, K- and D-meson masses close to their physical values. To this end, the ETMC has developed the tmLQCD software suite, which is an efficient implementation of the Hybrid-Monte-Carlo (HMC) algorithm for various types of Wilson fermions with support for multi-time-scale integrators, rational approximation frequency splitting as well as Hasenbusch mass preconditioning. The tmLQCD software suite provides support for highly efficient algorithms on the one hand and hardware specialisation support on the other hand through interfaces for various libraries developed together with other LQCD collaborations and hardware specialists. In order to efficiently simulate at the physical pion mass, critical slowing down is overcome by using the DDalphaAMG implementation of a multigrid preconditioned FGMRES solver with an optimised null-space updating scheme suitable for using in the HMC the Hasenbusch mass-splitting. The simulation of the 1+1 non-degenerate doublet also requires highly efficient Krylov subspace solvers on modern architectures such as Intel KNL. To this end, we have significantly extended the JLab/Intel-developed QPhiX solver and kernel library and provide a complete interface to its functionality within tmLQCD. The present proposal aims at optimizing strong-scaling performance of this software tripos on the KNL+OmniPath architecture with the support of PRACE experts on HPC.

top

 

Type D: Optimisation work on a PRACE Tier-1 (0)