PRACE Preparatory Access – 33th cut-off evaluation in June 2018

Find below the results of the 33th cut-off evaluation of 1 June 2018 for the PRACE Preparatory Access.

Projects from the following research areas:

 

Direct Numerical Simulations of Transient Turbulent Auto-Igniting Jets

Project Name: Direct Numerical Simulations of Transient Turbulent Auto-Igniting Jets
Project leader: Dr Miriam Rabacal
Research field: Engineering
Resource awarded: 100 000 core hours on MARCONI – KNL
Description

The objective of the project is to test the scalability of our Direct Numerical Simulation (DNS) code, which is based on the open source spectral element solver Nek5000, developed at Argonne National Laboratory (ANL), and the low Mach number reactive flow plugin developed at the Aerothermochemistry and Combustion Systems Laboratory (LAV), ETH Zurich. The reactive flow code can solve the governing equations of the energy and chemical species in complex geometry using detailed chemistry and transport properties. It can account for both gas-phase and surface kinetics (catalytic combustion), and stationary or time-varying geometries. In this work, the reactive solver will be used to study transient, turbulent, auto-igniting gaseous jets at elevated pressures. This subject is of significant importance to Internal Combustion Engine (ICE) technology, which is relevant for distributed heat and power generation and transport sectors that rely on chemical energy carriers as fuel. The reduction of the of the green-house gases footprint of ICEs can be achieved using biofuels, but inefficient combustion can lead to the emission of pollutants. There is a need to extend the current fundamental understanding of the role of the transient flow on the auto-ignition characteristics of turbulent jets. Limitations in both experimental and modelling approaches inhibit the investigation of mixing and combustion in transient reacting jets. DNS can provide the complete description of the system state fully resolved in space and time. The accurate simulation of laboratory-scale setups using sufficiently detailed descriptions of the combustion kinetics is conditioned upon the availability of high performance computing (HPC) architectures and efficient numerical tools that can harness their computational power. We are requesting computational time for a quantitative performance analysis on the CINECA MARCONI KNL architecture. The performance studies will be as extensive as the maximum allocated time of 50.000 core hours allow. A 3D test configuration of the scientific case, with a representative mesh, initial and boundary conditions, considering 20 chemical species, will be used. The results will serve as a benchmark to demonstrate the readiness of the code for a planned application to the PRACE Project Access call.

top

Collective motion of microswimmers in confined geometries

Project Name: Collective motion of microswimmers in confined geometries
Project leader: Dr Mauro Chinappi
Research field: Engineering
Resource awarded: 50 000 core hours on MARCONI – Broadwell, 100 000 core hours on MARCONI -KNL
Description

The swimming of single bacteria and the collective motion of microorganisms have attracted the interest of a varied community. Accumulation at interface (both solid-liquid and air-liquid) was studied with a number of theoretical, computational and experimental approaches, and several puzzling phenomena such as up-stream flowing and oscillatory motion in microchannel emerged when bacteria swim under strong confinement. In addition, artificial self propelled particles (such as Janus microswimmers) can potentially be employed for several applications ranging from targeted drug delivery and theranostics. Nowadays, the computational fluid dynamics research on microswimmer motion are, in essence, divided in two branches: I) the detailed description of the motion of single specific microswimmers and II) the collective motion of microswimmer swarms. While for the single swimmers researchers usually rely on accurate fluid dynamics solver able to correctly catch the hydrodynamics, the analysis of collective motion is commonly tackled with approximated descriptions such as 2D models, far field hydrodynamics and oversimplified descriptions of the swimmer geometry. Our projects aim at performing the first simulations of collective motion of microswimmers in confined geometries using a fully hydrodynamic solution for realistic models of microswimmers. We will employ an in-house code based on “AFiD” (an open sourse Fortran 90 code developed by the group of Roberto Verzicco, http://www.afid.eu). Our version implements a model for E.coli swimmer and for the widely used squirmer model and it allows the simulation of an arbitrary number of swimmers. In addition, thanks to the immersed boundary approach, different confinement geometries (e.g. plane channels, pipe, irregular walls) can be easily introduced. The code was already tested on local workstation, however, we would like to test the scalability and the performance for various typical problems in view of a possible proposal submission to future PRACE calls..

top

AMGCL scaling – part 2

Project Name: AMGCL scaling – part 2
Project leader: Prof Riccardo Rossi
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum, 100 000 core hours on Piz Daint
Description

The proposed project aims at concluding the publication of an article on the scalability of the AMGCL linear solvers. A preliminary work in this sense was done in september and october (on MareNostrum and PizDaint). A draft of the work currently under review can be found at the link: https://arxiv.org/pdf/1710.03940.pdf As of today the article reviewers are requiring more tests, as well as a comparison to other libraries. Access to the hardware would be therefore critically important to be able to conclude the research work.

top

Extending the scalability and parallelization of SEDITRANS code (continuation)

Project Name: Extending the scalability and parallelization of SEDITRANS code (continuation)
Project leader: Dr Guillermo Oyarzun
Research field: Engineering
Resource awarded: 100 000 core hours Piz Daint
Description

This project is a WP5 activity part of the Initial Training Network SEDITRANS (GA number: 607394), implemented within the 7th Framework Programme of the European Commission under call FP7-PEOPLE-2013-ITN. SEDITRANS is a research network that consists of six academic and four industrial partners within Europe. It is focused in to advancing in the comprehension of coastal processes utilizing high performance computing (HPC) for the numerical simulation of the three-dimensional turbulent flows. Coastal zone is characterized by oscillatory flow motions in the vicinity of the seabed, induced by surface waves. These wave-induced coastal flows interact with the sandy seabed and modify the bed shape by generating coherent small-scale bed structures, generally known as ripples. The presence of ripples in oscillatory flows is of great importance due to the impact they have on the seabed roughness. The bed roughness directly affects the near-bed boundary layer hydrodynamics, which in turn controls sediment transport in coastal areas. Moreover, the bottom boundary layer processes are strongly affected by the geometry of the ripples. Consequently, the thorough understanding and detailed prediction of flow kinematics close to the bed is of great interest in morphological studies regarding coastal management. At the beginning, the parallel code was optimized to be used in medium size clusters by means of a MPI parallelization using a geometric domain decomposition. The typical run of such code consisted in engaging up to 100 CPU cores for each execution. During the type D PRACE project 2010PA3748, the parallelization of the MPI code was tested up to 1024 cores and its implementation was extended for using hybrid supercomputers by means of an MPI+OpenACC execution model with the objective to solve more complex physical problems. So far we have tested the GPU parallelization reporting promising results of speedups in the range of 3X and 8X with respect to the CPU-only version of the code. The objective of this type A project, corresponding to the final part of the overall type D project, is to analyze the scalability of the code when engaging a large number of hybrid nodes in a tier-0 PRACE facility.

top

Nuclear matrix element of neutrinoless double-beta decay

Project Name: Nuclear matrix element of neutrinoless double-beta decay
Project leader: Dr Jun Terasaki
Research field: Fundamental Physics
Resource awarded: 50 000 core hours on MareNostrum
Description

The purpose of our project is to calculate the transition matrix elements of nuclei of the neutrinoless double-beta decay. Several nuclei with the mass number of 80-130 are calculated. This calculation consists of four steps. The first is to obtain the nuclear wave functions of the intermediate states of the decay using the quasiparticle random-phase approximation (QRPA) on the basis of the initial and final states. The second is the calculation of the overlap of those intermediate states obtained by the two QRPA calculations. The output of this calculation is a matrix. The third is the calculation of the matrix elements of the transition operator. The fourth is the calculation of the trace of the product of the above two matrixes and transition-density matrixes from the initial and final states to the intermediate states. This trace is the transition matrix element of nucleus, which is the final output. Five codes are used in this process. Two of them are for the wave functions of the intermediate states (the matrix-element calculation and diagonalization), and each of the other steps is made by one code. In the preparatory access, we will investigate the scalability of the five codes.

top

Targeting Alzheimer’s-associated amyloid-beta using small molecules

Project Name: Targeting Alzheimer’s-associated amyloid-beta using small molecules
Project leader: Prof Michele Vendruscolo
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

Alzheimer’s disease is the most common cause of dementia, but no cure is currently available to treat it. One of the hallmarks of this disease is the formation of neurotoxic plaques through the aggregation of the intrinsically disordered amyloid-β (Aβ) peptide. Decades of experimental studies have elucidated some of the mechanisms behind this aggregation process, but it is becoming increasingly clear that computational approaches can uniquely provide the atomic-level information required to define the detailed molecular events underlying Aβ aggregation and its inhibition. We propose to use molecular dynamics simulations using state-of-the-art enhanced sampling algorithms and experimental restraints to study how a series of small molecules modulates the structural properties of Aβ, which, being intrinsically disordered, should be described as a structural ensemble, in which each structure is assigned a distinct probability of occurring. As there are no experimental methods that can achieve the required accuracy, molecular dynamics simulations are currently the most effective way to obtain this atomic-level information of conformational ensembles of intrinsically disordered proteins. The accuracy required to characterise their modulation by small molecules is only currently possible using large computational resources. The computational resources available offer us the unprecedented opportunity to simulate up to 60 molecules interacting with monomeric Aβ. Taken together, these simulations will help us elucidate the complex mechanism by which small molecules can interact with disordered proteins towards drug discovery for Alzheimer’s disease.

top

Constraining the nature of dark matter with the intergalactic medium

Project Name: Constraining the nature of dark matter with the intergalactic medium
Project leader: Dr Antonella Garzilli
Research field: Fundamental Physics
Resource awarded: 50 000 core hours on MareNostrum
Description

With the current advances in numerical simulations it has become feasible to face the challenge posed by running simulations in WDMs. In fact the problem posed by such simulations are multiple: first of all it has been demonstrated that these models require a substantially larger number of particles than in the CDMs, for resolving the same scales (Wang & White, 2007). The stronger constraints on the free-streaming length of the WDM particles come from the Lyman α forest at high redshift, z ∼ 5 (Viel et al., 2013) and (Irsic et al 2017). These constraints are so tight that on Galactic scales the emerging WDM is indistinguishable from the CDM, and the WDM fails to solve the problems of the CDM. However, we believe that such constraints need to be revisited. In fact, several astrophysical effects, like the feedback from AGN and galaxies on the IGM, star formation and metal cooling, have a strong impact on the statistics usually employed in the study of the IGM, and these effects have been neglected in the past in putting WDM constraints, and the galaxy feedback and reionization time is particularly relevant due to the redshift of interest (Viel et al., 2013). In order to overcome the limitations in the past analyses that we have stated above, we plan to run new cosmological simulations and we will consider feedback effects from reionization and a transfer function that is more appropriate for the specific model of SNs. In fact, we are interested to SNs, rather than a simple thermal relic. Hence, the transfer function used in previous works needs to be reconsidered, in fact in many models there are thought to be two distinct generation mechanisms for the DM particles that could be present at the same time; in this case the free-streaming length would not be the unique parameter to determine the properties of the resulting haloes (Maccio’ et al., 2013), but there would be also a parameter describing the interaction strength of the SN that is the candidate for DM with the two other SNs. We will employ GADGET (presented in Springer (2002)), that is code for cosmological simulations that is based on N-body and smoothing particle hydrodynamics. We will study how the constraints on WDM from Lyman α forest are affected by galaxy feedback and reionization. We will consider a transfer function (as in Macci`o et al. (2013)) that is more realistic with respect to the ones previously employed, and that takes into account the complexity of the WDM generation at early times. We will perform a set of dedicated simulation runs, and analyze the simulation output by producing mock spectra. We expect that we will be able to place new robust constraints on WDM, taking advantage of the recent progresses in understanding physical effect of reionization that affect the flux power spectra and new advances in the theoretical comprehension of the SNs generation mechanism. Our work will shed a new light on the nature of DM.

top

Nuclear matrix element of neutrinoless double-beta decay

Project Name: Nuclear matrix element of neutrinoless double-beta decay
Project leader: Dr Jun Terasaki
Research field: Fundamental Physics
Resource awarded: 50 000 core hours on MareNostrum
Description

This proposal is a continuation of Preparatory Access proposal n°2010PA4334 for taking data of the parallel efficiency of one more code. The purpose of our project is to calculate the transition matrix elements of nuclei of the neutrinoless double-beta decay. Several nuclei with the mass number of 80-130 are calculated. This calculation consists of four steps. The first is to obtain the nuclear wave functions of the intermediate states of the decay using the quasiparticle random-phase approximation (QRPA) on the basis of the initial and final states. The second is the calculation of the overlap of those intermediate states obtained by the two QRPA calculations. The output of this calculation is a matrix. The third is the calculation of the matrix elements of the transition operator. The fourth is the calculation of the trace of the product of the above two matrixes and transition-density matrixes from the initial and final states to the intermediate states. This trace is the transition matrix element of nucleus, which is the final output. Five codes are used in this process. Two of them are for the wave functions of the intermediate states (the matrix-element calculation and diagonalization), and each of the other steps is made by one code. In the previous preparatory access, the data of the code to calculate the matrix elements of the QRPA Hamiltonian was taken successfully. This is one of two codes used for the largest jobs in our project. In this second preparatory access, another code used for the largest jobs will be investigated.

top

High Performance Computing assists drug design in the discovery of innovative anticancer drug candidates as ligands of the shelterin protein TPP1

Project Name: High Performance Computing assists drug design in the discovery of innovative anticancer drug candidates as ligands of the shelterin protein TPP1
Project leader: Prof Vittorio Limongelli
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum, 50 000 core hours on Curie
Description

Aberrant telomeres homeostasis is essential for cell immortality, enabling cells to evade telomere-dependent senescence without reaching Hayflick limit. The Hayflick limit is correlated with the length of the telomere region at the end of DNA. During the DNA replication process, small segments of DNA are unable to be copied and are lost every time when the process is completed. The reaction core of telomere homeostasis can be found in the Shelterin complex, an hetero-hexamer protein complex functioning as the anchor point of the telomerase enzyme, which is responsible for the replication of the telomeric DNA. In this production project, we will explore new possibilities for therapies targeting the shelterin complex. Our idea is based on a very recent discovery of a specific region on one component of the shelterin complex that is called TPP1. These findings will pave the road to target the telomerization mechanism with new small compounds. We already have very promising data concerning a group of candidates with good pharmacokinetic properties, that can bind the Telomerase at the level of its anchor point (TPP1). In order to do so, we have analyzed on-line drugs databases choosing the best small molecules based on selection rules taken from literature. The successful candidates were docked on the most represented conformations of TPP1 that was extracted from a very long classical dynamics simulation. This preliminary step was performed with the aim to obtain lead-compounds with a reasonable good affinity with the active region of TPP1. During this production project, we are going to evaluate the affinity of several promising drug candidates using Funnel-Metadynamics, a free-energy calculation method recently developed in Limongelli’s group, in conjunction with the Multiple Walkers protocol, a new implementation in the Plumed plug-in.

top

Galactic magnetic fields: the connection between large and small scales

Project Name: Galactic magnetic fields: the connection between large and small scales
Project leader: Dr Evangelia Ntormousi
Research field: Universe Sciences
Resource awarded: 50 000 core hours on Curie
Description

Magnetic fields are of paramount importance for understanding the dynamics of galaxies. However, even the magnetic field of our own Galaxy is still largely a mystery, because no observation can give us both its strength and its direction. What is more, we can only measure one component at a time, and for different phases of the interstellar medium. From theoretical considerations, we expect galactic magnetic fields at the largest scales to be connected to the intergalactic space, and directly affected by major events in a galaxy’s evolution, such as mergers, starbursts and gas accretion. At the same time though, the small-scale physics of the galaxy, such as star formation, the associated stellar feedback, and turbulent dissipation, can act to diffuse, restructure, or amplify the magnetic field, with a nonlinear impact on its global structure. What is then the connection between large- and small-scale magnetic fields, and how important is it for the evolution of the galaxy? This fundamental question lies at the heart of both galactic evolution and galactic dynamo theories. The answer, however, passes through the studies of multi-scale physics, such as turbulence and star formation, both notoriously hard to study, observationally and theoretically. The only possible remedy for such a complex issue are state-of-the-art numerical simulations that can efficiently connect the large to the small scales, with a particular focus on the physics of magnetic fields. We propose preliminary scalability studies for a series of high-resolution Milky-Way-like galaxy simulations, which will include a dark matter halo and a stellar disk, a multi-phase interstellar medium, star formation, and supernova feedback, and will track the large-scale evolution of the galactic magnetic field at large (kpc) and intermediate (pc) scales. These simulations will be followed by zoom-in simulations of certain galactic regions, including the explicit implementation of magnetic diffusion mechanisms, to track the small-scale evolution of the magnetic field and its local effects on turbulence and star formation. The models will be run with the RAMSES code, which has been extensively tested and proven to scale very well with the number of processes. However, this is one of very few, projects using the N-body, MHD and stellar feedback modules of the code simultaneously. Therefore, the preparatory access to Curie will be used to test the scaling of RAMSES for this problem, and the efficiency of the zoom-in strategy with MHD.

top

Hydra-scale-MN

Project Name: Hydra-scale-MN
Project leader: Prof Turlough Downes
Research field: Universe Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

The multifluid MHD HYDRA code is one of the most advanced codes of its type in the world. It has been used to run the first ever simulations of multifluid turbulence in star forming regions which resulted in several publications in top-rank international astrophysics journals. It was also the main code for the PRACE project “Accretion disk dynamics: the multifluid regime” (PRACE code PRA045) which explored, for the first time, the influence of the multifluid MHD effects known to be crucial to the physics of protoplanetary disks on accretion dynamics. It was also the main code for PRA083 “MF-DISK: Protoplanetary disk dynamics: the multifluid magneto-rotational instability, gaps and jets”. The HYDRA code displayed excellent scaling on the Blue Gene/P system JUGENE, exhibiting strong scaling from 8192 to 294912 cores with 73% efficiency. It also has extremely good scaling characteristics on many other architectures. However, with the continual advancement of available architectures it is necessary to port HYDRA to further systems. Porting should be easy as HYDRA uses no non-standard coding and no non-standard external libraries. In preparation for an application for project access we wish to investigate the scaling on the MareNostrum system. In spite of the excellent scaling of HYDRA on all systems tested so far it is both wise and necessary to confirm good scaling on this system which is of an architecture not previously investigated.

top

Thermo-hydraulic of advanced liquid metal nuclear reactors

Project Name: Thermo-hydraulic of advanced liquid metal nuclear reactors
Project leader: Dr Lilla Koloszar
Research field: Engineering
Resource awarded: 50 000 core hours on MARCONI – Broadwell,
Description

Our further aim is to analyze the thermohydraulics of new liquid metal cooled nuclear reactor systems with high resolution in space and time and provide useful information to the nuclear researchers and engineers still in the design phase. The goal of the preparatory project is to test how myrrhaFoam solver family, a customized solver group aims to simulate the thermohydraulics of liquid metal cooled nuclear reactors scale on HPC systems. These codes were developed n the framework of several years of collaboration between the von Karman Institute for Fluid Dynamics and SCK•CEN, the Belgian Nuclear Research Center. It is targeting to simulate the flow and thermal field of the primary cooling loop of the Generation IV nuclear research reactor MYRRHA currently under design by SCK•CEN and all the background research including pre-testing of the sub-systems, concepts and prototypes. The solver is based on the OpenFOAM simulation platform, it is a finite volume based computational fluid dynamics solver including multiphase modeling, thermal equation with variable density formulation. Conjugate heat transfer is considered, as well. We aim to extend the system size, simulation duration, as well as, the number of transient scenarios with this PRACE Type A Preparatory Access.

top

WENO-Wombat

Project Name: WENO-Wombat
Project leader: Dr Gianfranco Brunetti
Research field: Universe Sciences
Resource awarded: 100 000 core hours on Hazel Hen
Description

Hybrid OpenMP/MPI programming models are gaining importance as we enter the era of exascale computing. Additionally, one-sided communication is increasingly attractive for maximizing performance on modern interconnects. Few hybrid applications obtain high performance due to inefficiencies in multi-threaded communication in MPI.We have recently implemented a new code for astrophysical magneto-hydrodynamics and gravity that allows thread-level communication and computation overlap on systems using OpenMP, MPI-RMA and asynchronous I/O (Mendygral et al ApjS 2017). However, we have not been able to test our code on the largest scales with the Cray Aries interconnect and establish performance of our WENO solver. In this project, we aim to test our new code on the Cray XE40 system ”Hazel Hen” and establish scaling capabilities of the 5th order WENO MHD solver up to hundreds of thousands of cores. The work includes optimization of the Cray MPI library itself and will be performed in direct collaboration with software engineers from Cray Inc. as part of the Marie Curie International Outgoing Fellowship ”Cosmo Plasmas”.

top

Test Scalability for proposal 2018184349 On request for tecnical review

Project Name: Test Scalability for proposal 2018184349 On request for tecnical review
Project leader: Prof Philip Hoggan
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

Production runs on this project would efficiently use any of the supercomputers in this PRACE call. This project can only run in production on Tier-0 machines since it employs Quantum Monte Carlo (QMC) methods for a heterogeneous catalyst. The solid is platinum, with an exposed compact Pt(111) face. The reaction investigated is selective production of hydrogen as a source of sustainable energy (clean fuel). The water-gas shift equilibrium is shifted towards the hydrogen product by co-adsorbing water and carbon-monoxide on the Pt(111) face. This reaction can only be investigated theoretically by QMC, because it involves drastic changes in electron correlation and activation barriers are quite low and required to within 1 kcal/mol (0.043 eV). The QMC methodology for this system is now mature and two mechanisms: one limited by water dissociation and the other limited by concerted formation of adsorbed formate species have been the topic of our preliminary publication on this topic (ACS Symp. 1234 (2016).p77) . Now, with vastly improved methodology the proposal is ready for production on a scale only available through PRACE (see some software improvements, below).

top

Coarse-graining the thylakoid membrane of higher plants

Project Name: Coarse-graining the thylakoid membrane of higher plants
Project leader: Dr Evangelos Daskalakis
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

Here we propose to produce scaling plots for coarse-grained models and all-atom quantum fragments of the thylakoid membrane of higher plants. The methods of molecular dynamics (MD) simulations and quantum mechanics will be employed. In particular, we are interested in the Parallel Tempering MD (PT-metaD) enhanced sampling technique that can sample the thylakoid membrane organization under photo-protection, with particular interest in the interactions between the major-minor Light Harvesting Complexes (LHCII) and the Photosystem II Subunit S (PsbS). On the other hand, the excited state dynamics of the pigments within the LHCII will be described in the time-dependent density functional theory scheme.

top

Characterization of maltose translocation in the MalFGK2E transporter

Project Name: Characterization of maltose translocation in the MalFGK2E transporter
Project leader: Dr Cláudio Soares
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

ATP binding cassette (ABC) importers are transporter proteins involved in nutrient and metal uptake and are exclusive of bacteria. Some play a role in virulence, such as the zinc importer ZnuABC present in Brucella abortus (Tanaka, et al., 2017). ABC importers are also relevant for the intake of aminoacids and other nutrients. Therefore, they are a privileged gateway to the entrance of molecules in bacteria, which makes them a potential target for therapeutics against pathogenic bacteria. One such example is the delivery of synthetic antibiotics that resemble the natural substrates (Tanaka et al., 2017). Hence, it becomes quite relevant to understand the mechanisms of substrate translocation and binding to ABC importers. The Escherichia coli MalFGK2E maltose importer is a type I ATP-binding cassette (ABC) importer responsible for the uptake of malto-oligosaccharides. It is one of the most studied ABC transporters and it is a model system for type I importers. The MalFGK2E maltose importer is constituted by two transmembrane domains: MalF and MalG. Additionally, it contains two ATP-binding MalK domains and a substrate binding protein MalE that fetches maltose from the cytoplasm. Structurally it is known that the transporter alternates between outward-facing and inward-facing conformations, as identified by X-ray crystallography (M L Oldham & Chen, 2011; Michael L Oldham & Chen, 2011). However, the molecular details underlying the translocation process and the events required to product release remain undisclosed. In this project, we aim to characterize different stages of the transport process, as well as the energetics of maltose translocation in each step, using equilibrium and non-equilibrium molecular dynamics simulations. The stages we will simulate are: the pre-hydrolysis state with ATP, the post-hydrolysis state with ADP and phosphate, and the state after the exit of hydrolysis products. We will use non-equilibrium methods to understand the energy landscape in each state and obtain a translocation energy profile. Based on similar studies, we think the most suitable collective variables to describe translocation are the coordinate in the transmembrane axis of channel and the angle between maltose and the transmembrane axis (Jensen, Yin, Tajkhorshid, & Schulten, 2007)(Bajaj et al., 2016). PRACE resources will allow running non-equilibrium simulations with a high scalability in order to get microsecond sampling in a feasible time. We aim to use the preparatory access to determine the scalability of a system consisting of a MalFGK2E embedded in a 480 POPC membrane. We will use GROMACS 5.0 (Páll, Abraham et al 2015).

top

Direct numerical simulation of adverse pressure gradient boundary layer control

Project Name: Direct numerical simulation of adverse pressure gradient boundary layer control
Project leader: Assoc Prof Ayse Gungor
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

This research project will investigate the statistical and dynamical properties wall-bounded turbulence in adverse and favourable (A/F) pressure gradient (PG) turbulent boundary layers (TBLs) using direct numerical simulation (DNS). The adverse pressure gradient (APG) decelerates the boundary layers while favourable pressure gradient (FPG) accelerates it. By accelerating/decelerating the boundary layers, the pressure force modifies the interplay between the mean flow and turbulence. These modifications may lead in turn to important global effects such as energy losses and stall of an airplane. The research program involves a single DNS flow case which is designed to have the greatest impact in terms of improving our knowledge and understanding of APG and FPG TBLs. This single large-scale simulation will enhance considerably the variety of flow situations available. It will feature the first boundary layer that regains momentum near the wall even if it is still decelerating and the only FPG TBL that is at sufficiently high Reynolds number. The results of the DNS data will be analyzed to improve our understanding of the underlying physics of the layer structure, the similarity and scaling laws of APG and FPG TBLs, the effect of development history, as well as turbulence regeneration mechanisms. By improving the fundamental knowledge of PG TBLs, which are among the most difficult flows to predict using turbulence models, this research will greatly assist in the formulation of better turbulence models. These models are the building blocks of practical simulation codes for the aeronautical and aerospace industries and the design of effective methods of flow control in order to improve the aerodynamic performance of machines.

top

Scaling tests for submitted Tier-0 proposal

Project Name: Scaling tests for submitted Tier-0 proposal
Project leader: Prof Ilian Iliev
Research field: Universe Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

This project aims to check our code scaling on Marenostrum 4 as required for full Tier-0 proposals.

top

ASTROcomRPMD

Project Name: ASTROcomRPMD
Project leader: Dr Octavio Roncero
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

Complex organic molecules(COMs) have been detected in many astrophysical objects. These COMs are considered as prebiotic molecules and the question that remains is how and where they are formed: are they formed in the original molecular cloud that gave rise to the protoplanetary disks and from there to comets or planets? The formation of these molecules requires to overcome reaction barriers. At low temperatures this was considered impossible in gas phase, and the formation of COMs is model by reactions on ices. However, most of the molecules formed in ices, like methanol, do not desorb, and when excited with UV photons only photofragments photodesorb[1], so the explanation of the detection of COMs in gas phase is under debate. Recently, it was experimentally reported an important increase of the reaction between the hydroxyl radical and methanol, at low temperatures[2,3]. These results were explained by tunneling through the reaction barrier in the context of Transition State Theory. Recent theoretical work[4] stated that the TS used to model the tunneling rates was unrealistic, and that the tunneling rates are far too small to explain the experimental rise observed at low temperature. These authors proposed an alternative through the formation of methanol dimer. It is of crucial importance to determine the mechanism for the reaction in gas phase. The density in the interstellar media is so low that the formation of dimers is very unlikely. For this aim, in this project we propose to performed dynamical calculations below 300K to confirm this finding. Recently, the same behavior was measured for the reaction of OH with formaldehyde[5], and the rise of the rate constant observed were well reproduced by Quassiclassical calculations(QCT) performed on a full dimensional Potential energy surface(PES)[6]. Quantum calculations are required to confirm these simulations, since at these low temperatures zero point energy and tunneling are expected to play a fundamental role. Due to the high dimensionality of these reactions conventional quantum dynamical methods, such as wave packet propagations, are not feasible. In this work we plan to use Ring Polymer Molecular Dynamics(RPMD)[7], which allow to describe the two quantum effects, zero point energy and tunneling. This method describe each atom by a number of beads, number which increase when lowering the temperature. In addition, the reactivity at temperature below 100K, is mediated by long lived complexes, as recently demonstrated by QCT calculations[5,6]. For all these reasons extensive massive parallel computations are required. In this project, we intend to study the H2CO+OH reaction below 100K using the RPMD method and the PES already developed[6] in a first period. In a second period we will study the CH3OH+OH on a PES currently under development. [1]Cruz-Diaz et al., Astron.Astrophys.,(2016),592,68. [2]Shannon et al., Nature Chem.,5(2013)745 [3]Antiñolo et al., AstroPhys.J.,823,25(2016) [4]Siebrand et al., PCCP,18(2016)22712 [5]Ocaña et al., Astrophys.J.,850(2017),28 [6]Zanchet et al., PCCP,20(2018),5415 [7]Craig and Manolopoulos, J.Chem.Phys.,121(2004)3368. [8]Y. V. Suleimanov et al., Comput.Phys.Comm.,184(2013)833.

top

TP4GNRF – scale – Thermal properties for IVth generation nuclear reactor fuels

Project Name: TP4GNRF – scale – Thermal properties for IVth generation nuclear reactor fuels
Project leader: Dr Dominik Legut
Research field: Fundamental Constituents of Matter
Resource awarded: 50 000 core hours on MareNostrum
Description

See the Regular project No. 2018184419. Here we intend to do scaling test on Marenostrum as requested by technical evaluation committee.

top

Scalability for the EllipSys3D solver for wind energy purpose, V2

Project Name: Scalability for the EllipSys3D solver for wind energy purpose, V2
Project leader: Prof Niels Nørmark Sørensen
Research field: Engineering
Resource awarded: 50 000 core hours on MareNostrum
Description

The project aim at proving the parallel scaling of the EllipSys3D code for a wide range of wind energy applications, covering Large Eddy Simulations for Atmospheric Boundary Layer flows over complex terrain, DES and RANS simulations of resolved rotor flows, with the aim of paving the way for the large scale simulations needed in the future. Activities within national and international projects push the size of the required simulations beyond what is realistic on in-house resources. Specifically the need for LES simulations of off-shore wind parks, and wind parks in complex terrain will require large domain with high spatial resolution and long simulation times for improved statistics. Secondly, aero-elastic simulations with aerodynamics based on CFD, with resolved inflow turbulence is also requiring high spatial and time resolution and extended time periods to obtain sufficient statistics.

top

QWalk

Project Name: QWalk
Project leader: PhD Jan Brndiar
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

This project is specifically used to obtain scaling and efficiency data of QWalk code requested by the technical evaluator of the 17th PRACE Project Access Call.

top

Scattering simulations of Oriented large Dust particles (SODust)

Project Name: Scattering simulations of Oriented large Dust particles (SODust)
Project leader: Dr Vassilis Amiridis
Research field: Earth System Sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

The Scattering simulations of Oriented large Dust particles (SODust) project aims at quantifying the scattering properties of large, irregularly-shaped desert dust particles, oriented due to triboelectrification processes within the dust cloud. The electrification of desert dust plumes and its effect on climate is the subject of the new ERC Consolidator Grant «D-TECT» and the SODust simulations will be central for realising its scopes. The scattering simulations will be performed with the Discrete Dipole Approximation (DDA) method, considering irregular-shaped particles with different aspect ratios. DDA has been proven capable of accurately reproducing the scattering of irregular-shaped dust particles with radius of only up to ~1.6 μm due to the high computational cost for larger sizes. However, desert dust particle sizes are much larger, while their scattering properties for realistic shapes are completely missing for the size range between 1.6 and 16 μm. The required computational cost for these calculations is quite high, of the order of hundreds of millions CPU core-hours. SODust aims at bridging this existing gap by extending the DDA scattering calculations to larger particles (up to ~5 μm) using the extensive computer resources of the PRACE RI. The SODust unique scattering database is expected to be extended even further in the future and it will be made available to the scientific community to be used by all remote sensing applications considering complex particle shapes. It is indisputable that such simulations will provide unprecedented knowledge, affecting many research fields ranging from Earth System science to Astronomy and Astrophysics.

top

Ribosomal initiation and termination factors as new targets for antibiotic screenings

Project Name: Ribosomal initiation and termination factors as new targets for antibiotic screenings
Project leader: Dr Ana Oliveira
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50 000 core hours on MareNostrum
Description

Antibiotics are urgent topics in the medical biotechnology and pharmaceutical fields. Bacterial resistance to them evolves inevitably. Many infections are becoming harder to treat leading to longer hospital stays, with subsequent higher medical costs and increased mortality. I believe that despite the recent developments in cryo-EM, rapid kinetics and single FRET experiments, and extensive drug discovery programs, it has not yet been possible to deliver compounds with efficacy comparable to that of the first-generation of natural antibiotics and their semisynthetic derivatives. Thus, the present stage of the ribosome field now offers unprecedented opportunities to integrate experimental and theoretical approaches for developing novel antibiotics. First, and most importantly, the successful outcome of the proposed project would provide a firm basis for accessing the viability of IF2 and RFs as therapeutical targets. It would provide the background for the design of innovative strategies against antibiotic resistance. Second, we will attempt the design of low-molecular compounds with antibiotic properties. For this purpose, we need to explore not only novel scaffolds but also the structural-function relationship in the bacterial IF2 and RFs. Our preliminary results indicate that they may constitute targets never explored before against bacterial infections. Lastly, the techniques covered in this proposal have become indispensable tools in reaching this goal.

top

AoA – Assessment of ASTR: Advanced flow Solver for Turbulence Research

Project Name: AoA – Assessment of ASTR: Advanced flow Solver for Turbulence Research
Project leader: Dr Jian Fang
Research field: Engineering
Resource awarded: 100 000 core hours on Hazel Hen
Description

The objective of the overall project is to develop a novel way to control boundary layer separation that may occur in external and internal flows for a wide range of vehicles going from cars to supersonic aircrafts. Flow detachment is directly linked to drag and/or vibrations. An optimal control of the flow would allow reduce energy consumption, safety as well as improve comfort issues. This sort of flow control can potentially be applied to the rooftop of a car, to an airfoil of an aircraft to reduce drag. One way of improving flow detachment is to energise the near-wall fluids by enhancing the momentum transport between the inner and outer layer of the boundary layer, to prevent any adverse pressure gradient and to avoid or at least delay flow separation. Recently, it has been found that the rearrangement of small-scale disturbance (such as spanwise alternatively distributed riblets or roughness) at the wall can induce large-scale vortices across the boundary layer. This phenomenon is crucial for both theoretical research and industrial applications, because of its impact on near-wall turbulence dynamics and its potential in suppressing flow separation. The fact it is not visible to the naked eye due to its tiny scale makes it attractive for both automotive and aerospace industries, in which the control elements on the surface are required to be as small as possible. The objective of the project is to develop a small-scale wall actuation model of spanwise alternatively distributed strips (SADS) and to prove its effectiveness to generate active large-scale streamwise vortices and to suppress flow separation in both low-speed and supersonic flow ranges. By using direct numerical simulation and large-eddy simulation, we will develop a SADS model for spatially-developing boundary layer, for the flow over a backward-facing ramp and for supersonic shock-wave/turbulent boundary interaction flows. The key parameters of SADS will be studied. The wall roughness model will also be investigated as roughness implementation is easier to control by car or airplane manufacturers. If this study is successful, one might imagine the use of decorative elements such as stickers or granular paint to control the boundary layer. This would be effective in two ways, e.g. satisfy CO2 issues and attractiveness for consumers.

top

Nanoparticles for Catalysis Computational Modelling – NCCCM

Project Name: Nanoparticles for Catalysis Computational Modelling – NCCCM
Project leader: Prof Joseph Kioseoglou
Research field: Chemical Sciences and Materials
Resource awarded: 50 000 core hours on MareNostrum
Description

Size-selected nanoclusters exhibit remarkable physical and chemical properties, which diverge significantly from the properties of bulk materials. These properties make them ideal for applications such as novel electronic and photonic materials, sensors or binding sites for biomolecules. Recent advances in cluster sources, which increased the nanocluster yield by orders of magnitude, have enabled the use noble-metal nanoclusters for a new generation of precision catalysts. The properties of nanoclusters are greatly dependent on their size, shape and binding to the support, therefore modelling approaches at the atomistic scale are required to properly describe their behaviour. Ab initio methods are quantum mechanical methods that can provide full information over the properties of a system, but have seen limited application in studies of nanocluster systems, mainly owing to the large size of the models, and the consequent computational cost. This project aims to make use of the most novel ab initio approaches, offering high accuracy and full exploitation of modern massively parallel high-performance computers, to attack open questions in nanocluster science, such as the relation of fundamental properties of elemental and alloy nanoclusters to their size, shape and composition, the interactions between nanocluster and support and its effects on the system’s behaviour and finally the catalytic activities of nanocluster-based catalysts for reactions relevant to fine chemicals and clean energy. The project is expected to have an impact in both fundamental nanocluster science and large-scale industrial manufacturing of nanocluster catalysts.

top

Scaling tests for the SPHINX simulations of the first luminous objects and reionization

Project Name: Scaling tests for the SPHINX simulations of the first luminous objects and reionization
Project leader: Dr Joakim Rosdahl
Research field: Universe Sciences
Resource awarded: 100 000 core hours on MARCONI – KNL, 100 000 core hours on Hazel Hen
Description

The Epoch of reionization (EoR) is a fascinating chapter in the history of the Universe. It began when the first stars formed, bringing an end to the so-called Dark Ages. As their hosting dark matter (DM) haloes grew more massive, intergalactic gas rushed in and these first stars became the first galaxies. They emitted phenomenal amounts of ultraviolet radiation into intergalactic space, which ionised and heated the atoms that make up intergalactic gas, enhancing the pressure of the intergalactic medium to the point where it may have resisted the gravitational pull of the smaller DM haloes, stunting their growth. During the EoR, the large-scale properties of the Universe were thus strongly tied to the small-scale physics of star and galaxy formation. From current observations, we can indirectly infer only limited information about this epoch, when ionised regions grew and percolated to fill the Universe about one billion years after the Big Bang. We don’t know when the EoR started, how long it lasted, what types of galaxies were mainly responsible for making it happen (such as high- versus low-mass), and how this major shift affected the subsequent evolution of galaxies in a now much hotter environment. Soon our view of the EoR will change dramatically, as in 2020 the James Webb Space Telescope (JWST) is deployed into orbit around the Sun, and later the Square Kilometre Array (SKA) comes online. Both telescopes will perform unprecedented observations of the young and far-away Universe, SKA revealing the large-scale process of reionization and JWST allowing the first robust measurements of the physical properties (stellar masses, star formation rates, abundances, clustering, …) of a large population of galaxies during the EoR. Yet, while those telescopes will be extremely powerful, most details surrounding the interplaying physics constituting early galaxy evolution and reionization are still far out of reach observationally. To understand the physics, we need to back the limited information from observations with theory, using cosmological simulations, which combine, in three dimensions, the gravitational forces that led to the formation of galaxies, hydrodynamics and thermochemistry of the collapsing gas, star formation, supernova explosions, emission of radiation from stars, radiation-gas interactions, and gas-magnetic field interactions. In a previous PRACE allocation in 2017, we received computing time to start the SPHINX suite of simulations, running cosmological volumes with almost two thousand resolved galaxies and their contributions to reionisation (Rosdahl+2018). We now wish to expand the SPHINX simulations to an eight times larger cosmological volume, resolving up to 15 thousand galaxies and capturing almost an order of magnitude larger galaxy masses. This unprecedented range of resolved galaxies performed with full radiation-hydrodynamics finally enables us to find out whether reionization of the Universe was powered by a plethora of low-mass dwarf galaxies, a few massive galaxies, intermediate ones, or all of the above. The simulations will aid to clear the picture and understand the underlying physics producing the wealth of data from observations in the coming years..

top

 

Type B: Code development and optimization by the applicant (without PRACE support) (7)

Development of a MPI code for large scale dynamic analysis of fibre failure in hybrid polymer composites

Project Name: Development of a MPI code for large scale dynamic analysis of fibre failure in hybrid polymer composites
Project leader: Prof Pedro Camanho
Research field: Engineering
Resource awarded: 200 000 core hours on Curie
Description

The current project aims at developing the required tools to simulate the micromechanical behaviour of fibre reinforced polymers (FRPs) under uniaxial tension in the fibre direction. The failure of FRPs is controlled by fibre failure, whose strength is a stochastic property dominated by a distribution of defects in the fibres. Fibre failure and the interaction between broken leads to the creation of clusters of broken fibres that will eventually propagate unstably and lead to the failure of the composite. Proper understanding of this interaction and the damage evolution that leads to the final failure of FRPs is of extreme importance because it can guide the effective design of composite laminates. It is particularly relevant to design hybrid composites with multiple types of fibres that, if well design can have improved toughness and/or lower cost when compared with the composite laminates currently available. In this project a dynamic finite element code based on the spring element method will be tested and validated in a MPI setting. The model is based on simplified elements that allow the study of the stress redistribution that occurs when a fibre fails. The code not only takes into account the stress redistribution under static equilibrium but also the dynamic transient effects that occur during fibre failure. These dynamic effects, lead to higher stress concentration in the fibres surrounding a broken one, which will change the damage development and macroscale behaviour of the material. The results from the model will be validated against available experimental results and the changes on damage evolution due to fibre hybridization will be studied.

top

BrainFrame

Project Name: BrainFrame
Project leader: Mr Christos Strydis
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 200 000 core hours on MARCONI -Brodwell
Description

BrainFrame is an ongoing effort shared between National Technical University of Athens (NTUA) and Erasmus Medical Centre Rotterdam (EMC) which aims at providing an accessible, online platform for high-performance neuroscientific network simulation. The platform aims at encompassing many well-known and used by the community models of neuronal simulation (not to be confused with neural networks or A.I.) as well as trusted solvers of said networks. Furthermore, it offers alternative high-performance network solvers which aim at utilizing accelerators in order to conduct neuroscientific experiments at massive scales and impressive speed. BrainFrame envisions conducting computation-heavy neuroscientific experiments via a heterogeneous cluster of computing power comprised of GPUs, FPGAs, Xeon Phis and even traditional, high-end CPUs. As such, BrainFrame is a product of collaboration between experts on different HPC fields aiming at the same goal of developing a competitive solution for neuroscientific workloads.

top

High-fidelity simulation of an industrial swirling combustor

Project Name: High-fidelity simulation of an industrial swirling combustor
Project leader: Dr Daniel Mira
Research field: Engineering
Resource awarded: 100 000 core hours on MareNostrum
Description

The objective of the project is to investigate the reacting flow field of an industrial swirling combustor designed and manufactured by E&M Combustion (EMC), in order to characterize its performance (thermal power, combustion efficiency, and global emissions), so that the SME can make decisions to optimize the system in the future.

top

Symmetric spin initialization for crystal structure prediction in magnetic materials

Project Name: Symmetric spin initialization for crystal structure prediction in magnetic materials
Project leader: Mr Michele Galasso
Research field: Chemical Sciences and Materials
Resource awarded: 200 000 core hours on MARCONI – KNL
Description

Thanks to the modern theories able to describe materials at the atomic scale and to the exponential increase of computing power, the field of Computational Materials Science has become today more important than even before. In particular, a big contribution to this field has been given by the software USPEX (Oganov and Glass, 2006). The USPEX code is an evolutionary algorithm able to predict the most stable crystal structure for a given chemical compound, without requiring any experimental input. It performs the search for the ground state by starting from an initial population of candidate structures and by improving it at each step, until the most stable structure is found. One of the keys of USPEX’s success is to start from an initial population of good candidates, which means a heterogeneous group of structures with energies close to the ground state, so that only the most promising regions of the free energy surface of the compound are explored. In order to achieve this goal, the algorithms which generate initial structures in USPEX do not simply place atoms at random positions in the unit cell, instead they use symmetry as a guide, since structures which possess certain symmetries are often the ones which have low free energy. The search for the ground state of magnetic compounds is still a challenge for USPEX. In this case the introduction of a new degree of freedom, namely the possibility of having non-zero magnetization on some atoms in the unit cell, makes it possible to initialize these magnetizations in many different ways which can differ a lot in terms of free energy. For this reason we have developed a new method for initializing magnetic moments in the unit cell which takes symmetry into account. The main goal of our method is to increase the chance of picking, among all possible configurations, a magnetic configuration with low free energy. Preliminary tests of our method performed on different compounds suggest that the mean energy of symmetrically initialized structures is generally lower than that of randomly initialized structures. In addition, our algorithm performs well also in presence of a high number of magnetic elements and in presence of several magnetic species in the unit cell. The investigation of a possible connection between symmetry in magnetic ordering and free energy is very interesting for two main reasons. On one hand, the confirmation of this hypothesis would be a great help for all scientists working in the field of computational materials discovery. On the other hand, this would be an extremely beautiful evidence that also in this case Nature prefers symmetry.

top

Investigating Application Dynamism on Alya for Energy Efficiency at MareNostrum IV supercomputer

Project Name: Investigating Application Dynamism on Alya for Energy Efficiency at MareNostrum IV supercomputer
Project leader: Dr Ricard Borrell Pol
Research field: Engineering
Resource awarded: 100 000 core hours on MareNostrum
Description

This project is developed in the context of the PRACE-5IP WP7 T7.2 Exascale-project: “Investigating Application Dynamism on Alya for Energy Efficiency”. Which aims to exploit the READEX toolsuite, developed by the READEX FET-HPC project, to investigate the dynamic behaviour of Alya code for increasing energy efficiency. Alya is a multi-physics modular code used to simulate complex engineering problems and has been under development at the BSC since 2004. It is one of the advanced programming methods for Exascale associated with EoCoE (Energy oriented Center of Excellence for computing applications). It is written in Fortran 90/95, parallelised with MPI and OpenMP and designed to run efficiently in parallel supercomputers. Alya has been tested and extensively used on many PRACE Tier-0 machines. It has been also used as a benchmarking code in PRACE Unified European Applications Benchmark Suite (UEABS). It was proven to scale up to 100,000 cores on Blue Waters supercomputer at NCSA, demonstrating a great potential for the upcoming European Exascale systems. READEX (Runtime Exploitation of Application Dynamism for Energy-efficient eXascale computing) is a EU Horizon 2020 FET-HPC project whose objective is to exploit the dynamism found in high-performance computing applications at runtime to achieve efficient computation on Exascale systems. The technologies that are developed in READEX facilitate automatic identification of dynamic resource requirements in an application and tuning hardware-, system software- and application-level parameters for optimised objective values such as performance and energy consumption. The READEX pre-alpha prototype has been exploited on applications of molecular dynamics, quantum mechanics and linear algebra as part of PRACE-4IP WP7 T7.2A towards Exascale. The alpha prototype was released in March 2017 and would be an initial interest in this mini-project. We will then continue using the later versions, particularly the pre-beta prototype in August 2017, beta prototype in February 2018 and the final release in August 2018.

top

Development of an algorithm for the parallel numerical simulation of dust and particle transport in tunnel construction.

Project Name: Development of an algorithm for the parallel numerical simulation of dust and particle transport in tunnel construction
Project leader: Prof Joan Baiges
Research field: Engineering
Resource awarded: 200 000 core hours on Curie
Description

The current project is framed in the Spanish national research project ECOVENT. The ECOVENT project is focused on the ventilation phase of the development of a tunnel by means of perforation and blasting, which is nowadays the most sensitive phase with respect to energy consumption and environmental impact. One of the main bottlenecks in tunnel construction is the need for waiting for dust and particle deposition, and toxic gas ventilation after a blasting before the construction process can continue. This is due to the safety and health issues related to dust particles and explosion gas residual inhalation. The ECOVENT project aims at reducing deposition of dust time thanks to new perforation and blasting techniques. The optimization of these techniques requires a numerical simulation tool capable of representing the dust transport, particle settling and toxic gas ventilation. For this, a parallel implementation for large scale simulation of these phenomena needs to be performed. Major code developments need to be tested and validated in an MPI setting: The first development consists in the models for particle and dust transport which will be implemented, which need to account for the interaction with the aerodynamic flow. Two main possibilities are going to be evaluated: On the one hand one can implement the numerical simulation of particles and their interaction. However this can be difficult to treat in an MPI setting because particles need to traverse from one subdomain on the finite element domain decomposition to another. The second possibility is to use a convection diffusion equation in order to take into account the dust particles in a general setting, although this also requires communications between subdomains, and it will require of a callibration phase. The second phenomena that needs to be implemented is the ventilation of toxic gases from the tunnel. In principle these are non reactive toxic gases, and a convection-diffusion-reaction equation should be enough to model them, calibration in a large scale computing setting will also be required.

top

Optimizing zoom-in simulations: from cosmology to molecular clouds

Project Name: Optimizing zoom-in simulations: from cosmology to molecular clouds
Project leader: Dr Andrea Pallottini
Research field: Universe Sciences
Resource awarded: 100 000 core hours on SuperMUC
Description

This project aims at optimizing the AMR code RAMSES-RT, that we have recently coupled with the thermochemistry solver KROME, as a preparation for PRACE application to Tier-0 projects. This code will be used for zoom-in simulations of the high-redshift universe. In particular we will investigate the physics of early galaxy formation and evolution from cosmological scales to molecular cloud formation. With this code it will be possible to efficiently follow the evolution of galaxies: as the gas falls in dark matter halos located in the knots of the cosmic web, molecular hydrogen forms and cools the gas, eventually leading to the generation of first stars. The energy input from these luminous sources regulates the infall/outflow and the subsequent generations of stars in a feedback cycle. The study of the formation and evolution of galaxies in a cosmological framework requires numerical codes that are able to follow different physical processes, such as gravity, hydrodynamics, radiative transfer and chemistry (via the KROME module). Standard cosmological codes are optimized to solve gravity and hydrodynamics; hence domain splitting routines focus on optimizing the load balance between these two processes only. However, when radiative transfer and chemistry are included, as in our case, the different computational costs must be carefully balanced by a proper domain decomposition. Furthermore, other key physical processes, as star formation, require the creation of particles, which depends on inter-process communications to avoid data race problems. If the computational domain is not evenly split, different processes will have to mutually wait before advancing to the next time-step. This makes the simulation very inefficient in exploiting the available computational resources, especially for large facilities. In this project, we aim at improving the efficiency of RAMSES-RT code on computer clusters with a large number of nodes, by – implement load balance routines that account for different physical processes; – using MPI-OpenMP hybrid schemes to optimize the computational cost; – minimize MPI wait times by optimizing inter-process communications.

top

 

Type C: Code development with support from experts from PRACE (1)

Validation and performance assessment of pipelined CG methods

Project Name: Validation and performance assessment of pipelined CG methods
Project leader: Dr Ricard Borrell Pol
Research field: Mathematics and Computer Sciences
Resource awarded: 100 000 core hours on MareNostrum
Description

This project focuses on the application of advanced strategies to mitigate the effect of collective reductions on the scalability of the Deflated CG method. From the algorithmic point of view we will focus on the implementation of a pipelined version of the algorithm and from the implementation point of view we will consider different options to impose asynchronism and overlapping of communications and computations. This project will be developed and implemented in the Alya system. Alya is the high-performance computational mechanics code developed at the Barcelona Supercomputing Center. The physics solvable with the Alya system include incompressible/compressible flow, solid mechanics, chemistry, particle transport, heat transfer, turbulence modeling, electrical propagation, etc. Alya aims at massively parallel supercomputers; its parallelization includes both the MPI and OpenMP frameworks, as well as heterogeneous options including accelerators. Alya is one of the twelve simulation codes of the Unified European Applications Benchmark Suite (UEABS) and Accelerators Benchmark Suite of PRACE and thus complies with the highest standards in HPC. From the algorithmic point of view, the background is the pipelined CG method which is already implemented in Alya. A novel strategy for the correction of the rounding-off error coming from the pipeline methodology needs to be validated and, subsequently, we aim to extend the methodology to the Deflated version of the CG method, which provides better convergence ratios. From the implementation point of view we aim to carry out a detailed performance analysis to attest the asynchronous behavior of the the pipelined algorithm analysing different implementation options (MPI non-blocking collectives, task parallelism, OmpSS,…), including MPI + X approaches. In summary, this project targets extreme scale applications to run on pre-exascale systems. At this scale, new bottlenecks arise, in particular we will focus on the mitigation of the overhead produced by collective reductions within the CG method.

top

 

Type D: Optimisation work on a PRACE Tier-1 (1)

BiqBin solver

Project Name: BiqBin solver
Project leader: Prof Janez Povh
Research field: Mathematics and Computer Sciences
Resource awarded: 150 000 core hours on Tier-1
Description

Mathematical optimization problems, where the objective function is quadratic and we consider only binary variables, encapsulate wide range of problems, from combinatorial optimization to real life logistic problems. Such problems are in general NP hard, so we can solve instances of relevant size only approximately, using combination of heuristic and meta heuristic algorithms. However, to test these approximation algorithms one still needs a ground truth for small or medium size problems, so algorithms to solve such problems to proven optimum are highly needed. In a joint Slovenian-Austrian research project High-Performance Solver for Binary Quadratic Problems we have designed such algorithm called BiqBin and implemented it in a C++ code. A special branch of the code is designed to solve a special case, called the stable set problem. It is developed to run in parallel on local HPC system (approx.1000 CPUs) and we can solve with this code solve instances of size at most 500. The algorithm scales well, but we want to: – Test scalability on larger HPC machines. – Improve the parallelism of the code, especially the part where we explore in parallel the branch and bound tree and the linear algebra operations, related to solving the semidefinite programming problems. We believe we can (i) more efficiently share the current results among the workers and therefore prune this tree more efficiently and (ii) solve semidefinite programming problems in each branch and bound node much faster. – Solve instances of size up to 2000. So far very few mathematical optimization people have decided to test their algorithms on reasonable HPC system so successfully implementation of this project would give us high visibility in our domain and would motivate other researchers from the same domain to follow us.

top