PRACE Preparatory Access – 28th cut-off evaluation in March 2017

Find below the results of the 28th cut-off evaluation of March 2017 for the PRACE Preparatory Access.

Projects from the following research areas:

 

Plasma-actuator controlled turbulent jets

Project Name: Plasma-actuator controlled turbulent jets
Project leader: Dr Sylvain Laizet
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-KNL
Description

In this proposal for Preparatory Access we would like to investigate the scalability of our Computational Fluid Dynamics solver called Incompact3d (www.incompact3d.com) on the KNL nodes of the CINECA supercomputing facility. This high-order flow solver is able to perform turbulence-resolving simulations based on the incompressible Navier-Stokes equations. It is a finite-difference solver which is based on a powerful 2D domain decomposition (http://www.2decomp.org/). It has lead to excellent parallel efficiency with up to one million computational cores on MIRA in the US. It is used routinely for production runs on several thousands of cores in France and in the UK. For this project, we would like to investigate the behavior of Incompact3d on a large-scale Intel Knight Landing platform. So far, we have been able to test it on the UK ARCHER Knights Landing testing & development platform which is a small system consisting of 12 nodes each fitted with an Intel Knights Landing (KNL) processor (http://www.archer.ac.uk/documentation/knl-guide/#hardware). The scalability performance of our flow solver is excellent on KNL but we were not able to check the performance of Incompact3d with more than 512 KNL computational cores.

top

Scaling of DNS code for very high Reynolds number channel flows

Project Name: Scaling of DNS code for very high Reynolds number channel flows
Project leader: Prof. Dominique Thévenin
Research field: Engineering
Resource awarded: 500000 core hours on Curie, 100000 core hours on Juqueen,
Description

The main goal of the current project is to detect and analyze large-scale coherent structures (“superstructures”) in a channel flow at bulk Reynolds number > 20 000 using direct numerical simulations (DNS). The importance of this hot topic comes from the fact that there is no clear or sharp definition for these superstructures; however, they might indeed dominate the global transport of mass, heat, or momentum. Of course, DNS at such large Reynolds numbers is very challenging. Our DNS studies rely on the in-house code DINO, which was already tested on different HPC systems for many benchmarks, in particular on Karman (local Linux cluster at University of Magdeburg), and on SuperMUC (Leibniz Supercomputing Center in Munich). With this preparatory account we aim at checking parallel scaling of the code for turbulent flows in a very long channel (channel length of the order of 20x channel height H) at high Reynolds number, before applying to a PRACE project. Investigating this flow with DNS involves two main challenges: (1) A very efficient DNS code, (2) An efficient tool to detect and analyze the turbulent structures. Considering the past experience of our group, we believe that we will be able to achieve our goals.

top

On the Strong Scaling of Maritime CFD: Open-Usage Community based viscous-flow CFD code ReFRESCO (www.refresco.org) towards the Exascale Era

Project Name: On the Strong Scaling of Maritime CFD: Open-Usage Community based viscous-flow CFD code ReFRESCO (www.refresco.org) towards the Exascale Era
Project leader: Dr. Eng. Guilherme Vaz
Research field: Engineering
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Hazel Hen
Description

ReFRESCO (www.refresco.org) is a community based open-usage viscous-flow (CFD) solver for the Maritime World. It solves multiphase (unsteady) incompressible viscous flows using the Navier-Stokes equations, complemented with turbulence models, cavitation models and volume-fraction transport equations for different phases. ReFRESCO (v2.4.0) is currently being developed, verified and its several applications validated at MARIN (in the Netherlands) in collaboration with IST (in Portugal), USP-TPN (University of Sao Paulo, Brasil), TUDelft (Technical University of Delft, the Netherlands), UoS (University of Southampton, UK), UTwente (University of Twente, the Netherlands), Chalmers (Chalmers University, Sweden), UMaine (University of Maine, USA), Texas A&M (Texas A&M University, USA), UPB (Universidad Pontificia Bolivariana, Colombia) and WUR (Wageningen University and Research, the Netherlands) . Like most of the current CFD solvers it relies on domain-decomposition and MPI parallel paradigm for performance acceleration, mostly tailored for “traditional” multi-core distributed memory HPC machines. During the last years, an effort has been made on assessing the strong scalability of the whole code, and all its individual components. Bottlenecks have been identified; for instance single-process IO, excess of global MPI communication and low scalability of the linear-system-of-equations solver. In order to solve some of these issues new parallelization techniques (MPI+openMP), new solvers (PETSC, TRILINOS, own asynchronous solvers), new compilers (GNU, Intel, PGI/Nvidia) and algorithm choices have been implemented. Preliminary tests on small HPC machines show encouraging improvements. Tests on CPU+co-processor (Intel KnightsLanding KNL) platforms are undergoing. The objective of the current project is therefore to test all these new paradigms (and their combinations), for several Maritime problems, in more world-class Tier-0 PRACE HPC machines. All this envisaging the extension of ReFRESCO for forthcoming hardware and to prepare it to the Exascale era.

top

An automated platform for fragment evolution

Project Name: An automated platform for fragment evolution
Project leader: Prof Xavier Barril
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100000 core hours on piz Daint
Description

Our group’s long-term goal is to expand the druggable genome through the discovery of bioactive molecules capable of acting on novel sites. To achieve this objective, we have developed several computational methods that allow us to identify and quantify the druggability of binding sites, [1-3] characterize the interaction and solvation preferences of such sites, [4-6] or to identify novel ligands by means of virtual ligand screening.[7-8] These capabilities have been developed in the context of several funded projects, both at the national and international level [“FRAGNET Initial Training Network” (H2020-MSCA-ITN-2015; 2016-2019; Project 675899); “A novel hybrid platform to generate chemical probes. Application to the Fbw7 E3 ligase” (Proyectos I+D; 2016-2018; SAF2015-68749-R)]. In this application we aim to combine our own computational methods with other tools to increase the efficacy and efficiency of fragment evolution. This will take place in the context of the EU-funded “FRAGNET” project, a network of academic and industrial labs that are experts in the field of Fragment-Based Drug Discovery (FBDD). And the computational predictions will be validated experimentally within the network. In the last decade, FBDD has proven to be an effective approach towards the discovery of small molecule compounds (ligands) that can bind to biological target molecules such as proteins and nucleic acids.[9] The key feature of this approach is that ligand discovery begins with the screening of low molecular weight compounds that have a higher chance of binding to a target in comparison to whole (drug) candidate molecules. Such hits can then be grown or merged to provide lead compounds. This means a much smaller investment is required in terms of automation and development of compounds in comparison to those required by High Throughput Screening, which has been the mainstay of hit identification in modern drug discovery. Experience shows that finding fragment hits is never the limiting step in FBDD. Instead, the ability to evolve the initial hits, which are very small and have a binding affinity in the mM range, into efficient lead-like molecules can be a daunting task because – due to the vast size of the chemical space – fragment evolution offers almost infinite possibilities. Currently, fragment evolution is performed manually, in a process that is heavily biased by the medicinal chemistry expertise of those involved in the project. This presents three main disadvantages: 1) a large part of the theoretically possible chemical space is ignored; 2) synthetic feasibility often receives greater consideration than the probability of being active; and, 3) the process is costlier and lengthier than it should. In order to remove these limitations, identifying the most promising and diverse molecules quickly and cheaply, we have initiated a project that combines chemoinformatic and computational chemistry tools in a software platform for automatic fragment evolution. Currently, we have a working prototype, which we have developed using our in house computational server. At this stage, we need to scale up the process, optimize its performance, validate it thoroughly on multiple systems and apply it prospectively in order to demonstrate its potential. [1] J. Seco, F. J. Luque, and X. Barril (2009) J. Med. Chem., vol. 52, no. 8, pp. 2363-71 [2] P. Schmidtke and X. Barril (2010) J. Med. Chem., vol. 53, no. 15, pp. 5858-67 [3] X. Barril (2013) Wiley Interdiscip. Rev. Comput. Mol. Sci., vol. 3, no. 4, pp. 327-338 [4] P. Schmidtke, A. Bidon-Chanal, F. J. Luque, and X. Barril (2011) Bioinformatics, vol. 27, no. 23, pp. 3276-85 [5] D. Alvarez-Garcia and X. Barril (2014) J. Med. Chem., vol. 57, no. 20, pp. 8530-9 [6] D. Alvarez-Garcia and X. Barril (2014) J. Chem. Theory Comput., vol. 10, no. 6, pp. 2608-2614 [7] S. Ruiz-Carmona, D. Alvarez-Garcia, N. Foloppe, a B. Garmendia-Doval, S. Juhos, P. Schmidtke, X. Barril, R. E. Hubbard, and S. D. Morley (2014) PLoS Comput. Biol., vol. 10, no. 4, p. e1003571 [8] Ruiz-Carmona S, Schmidtke P, Luque FJ, Baker L, Matassova N, Davis B, Roughley S, Murray J, Hubbard R, Barril X (2016) Nat Chem. doi: 10.1038/nchem.2660S [9] Erlanson DA, Fesik SW, Hubbard RE, et al (2016) Nat Rev Drug Discov 15:605-619

top

Open source exascale solid mechanics multi-scale framework

Project Name: Open source exascale solid mechanics multi-scale framework
Project leader: Dr Anton Shterenlikht
Research field: Engineering
Resource awarded: 100000 core hours on Hazel Hen
Description

We have developed a scalable deformation and fracture modelling framework for the solid mechanics and materials research communities. The framework is multi-scale. It simulates complex interaction of multiple physical processes occurring at different time and length scales, e.g. from grain boundary migration in polycrystals (nanometres) to engineering scale fracture prediction in complete components (metres). The framework implements cellular automata method with Fortran 2008 and 2015 coarrays. The framework is implemented as an open source library, CGPACK, that can be used to build standalone programs, or can be coupled to other popular solid mechanics tools, such as finite elements. In the latter case a cellular automata finite element (CAFE) concurrent multi-scale model with two-way information transfer is established. To date there are very few real engineering codes using Fortran coarrays. This is partly due to slow compiler support for coarrays. To date Cray support is most advanced. Although Intel and GCC/OpenCoarrays continue to improve their support for coarrays, at this stage only Cray systems deliver coarray performance. Some of our engineering miniapps scale nearly linearly up to 32,000 cores on Tier-1 systems. The purpose of this application is to explore scaling further on a Tier-0 Cray system. Scaling is critically important because many demanding solid mechanics problems cannot be currently solved in practical time. Safely critical structural integrity problems such as interaction of multiple propagating cracks in a ruptured pressure vessel or a uncertainty quantification (which requires a large number of high number of runs with high resolutions of the microstructure) are deemed impractical at present because most solid mechanics codes do not scale well. If successful, this project will open the path towards solving these problems, with immediate benefits to academic research, and industry in diverse sector such as power generation, automotive, shipbuilding, steelmaking, aerospace or defense.

top

Cosmic Ray Induced Turbulence

Project Name: Cosmic Ray Induced Turbulence
Project leader: Dr. Rahul Kumar
Research field: Universe Sciences
Resource awarded: 100000 core hours on Hazel Hen, 50000 core hours on SuperMUC
Description

A significant fraction of the total energy in various astrophysical environments, such as the interstellar medium and supernova remnants, possessed by few relativistic charged particles, known as cosmic rays. The turbulent properties of these astrophysical environments are believed to be significantly influenced by the presence of these cosmic rays which depend on total energy budget and flow properties of cosmic rays, e.g. local anisotropy in their momentum distribution. For instance, in certain cases cosmic rays are believed to be able to drive large-scale wind at galactic scales. Moreover, cosmic rays themselves are affected by the self-generated turbulence and plasma waves in the background gas which can significantly alter their transport properties. The complex interaction between cosmic rays and the background gas require numerical simulations to unravel the details of the dynamics. The generation and dissipation of the turbulence take place at the kinetic scales (e.g. Larmor radius of cosmic rays) of the plasma species which are orders of magnitude smaller than the astrophysical spatial scales. The project aims to study the interaction between cosmic rays and the background ionized gas at these microscopic scales which are not resolved in the large-scale numerical simulations, for example numerical simulations of galaxy formation and MHD simulations of turbulence in the interstellar medium. We use self-consistent particle-in-cell simulations of plasma that resolves kinetic scales of all plasma species, that is to say cosmic rays and the background gas. These simulations help us quantify properties of the turbulence and time scale associated with the growth of several waves for a chosen set of parameters. A semi-analytical models is used to extend the results to a wider range of parameters that can applied to draw some useful conclusion about the turbulence in the astrophysical systems, e.g. total amount of energy lost by the comic rays to the background gas within the advection time.

top

Contact loading of hard, brittle materials using atomic modelling and simulation

Project Name: Contact loading of hard, brittle materials using atomic modelling and simulation
Project leader: Dr Saurav Goel
Research field: Engineering
Resource awarded: 100000 core hours on Hazel Hen,
Description

This project makes use of atomic modelling techniques to test scalability and probe phases of important materials like diamond used in cutting tools, jewellary and in biomedical surgical knifes etc. The outcome of the project will offer future technology options to make manufacturing to adopt Industry 4.0 standards in ultra-precision machining. The proposed project offer a penultimate ground-breaking and step-changing approach for a broadened range of technologies for deployment. Aside from this the PI is also leading the UK efforts in the European Cost funded project CA15102 on substitution of critical raw materials. Atomic modelling is an appropriate substitute to discover materials that could be substituted in lieu of the raw materials critical to machining and manufacturing industries in the EU to make them more reliant rather than on exports.

top

Elucidating the interplay between structure and catalytic activity in nanoporous materials

Project Name: Elucidating the interplay between structure and catalytic activity in nanoporous materials
Project leader: Dr. Kurt Lejaeghere
Research field: Chemical Sciences and Materials
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Marconi-KNL, 50000 core hoursn on Curie,
Description

Heterogeneous catalysts are essential to perform chemical reactions at an industrial scale. When reagents are adsorbed at their surface, energy barriers drop and reaction rates increase. A large surface is therefore ideal. Nanoporous materials take this idea to the extreme: they are riddled with nanoscale cavities, also called pores, each of which enables capturing chemical species and improving the overall reactivity. Moreover, the pore size introduces a certain selectivity, as the desired reagents must be able to reach the active site. Several classes of nanoporous materials are available, each with their own benefits and disadvantages. Zeolites, which are composed of aluminosilicates, are generally quite rigid, and the structural changes on adsorption are usually limited. Metal-organic frameworks (MOFs) and covalent-organic frameworks (COFs), on the other hand, can be flexible, demonstrating large structural changes on adsorption or under varying ambient conditions. This is due to the presence of soft modes, in which the building blocks of the materials, the linkers and the nodes, rotate with respect to each other without much effort. It is therefore essential to understand the effect of the geometry and its variability on the stability and catalytic activity of the material. This not only requires a clear view on the local structure of the periodic energy surface, which can be accessed through energy-versus-volume profiles and normal mode analysis, but also asks for insight in the evolution of catalytic activity as a function of these geometric features. Of prime interest are indicators such as the adsorption strength (acid-base catalysis) and the alignment of electronic levels (photocatalysis). These materials properties can be accessed by means of density-functional theory (DFT), a first-principles computational technique based on quantum physics. This project aims to assess the scalability of a Python-based workflow in combination with the VASP DFT code. Three model systems will be evaluated, i.e. 1 zeolite, 1 MOF and 1 COF. In addition, one particularly large structure will be tested separately to identify the practical limits of the calculations. The results of this study should allow us to define a computationally efficient and feasible workflow for a large-scale project.

top

3D MHD Simulation of the Kruskal-Schwarzschild Instability in a Strongly Magnetized Outflow

Project Name: 3D MHD Simulation of the Kruskal-Schwarzschild Instability in a Strongly Magnetized Outflow
Project leader: Dr Ramandeep Gill
Research field: Universe Sciences
Resource awarded: 100000 core hours on Hazel Hen
Description

Relativistic outflows occur in a wide variety of astrophysical systems. These sources probe extreme physical conditions that are beyond our reach on Earth, such as strong gravity, very large densities and magnetic fields, extremely energetic particles, and relativistic bulk motions. They are promising sources of gravitational waves or high-energy neutrinos, and most likely produce the highest energy cosmic rays. However, their origin and inner workings are still poorly understood. A widely accepted hypothesis is that all of these sources share a common mechanism originally developed for pulsar winds. Namely, it is assumed that relativistic jets in AGNs, GRBs, and micro-quasars are driven by rotating, twisted magnetic fields that transfer the rotational energy to large distances in the form of a Poynting flux. In pulsar winds and AGNs there is good evidence that the jet is indeed initially highly magnetized, while for GRBs and micro-quasars the jet composition, and in particular its initial degree of magnetization is unclear, and therefore of great interest. The focus of this preparatory work is on magnetically dominated jets in which energy is dissipated through the process of magnetic reconnection that is facilitated by plasma instabilities and/or magneto-hydrodynamic (MHD) turbulence. An attractive mechanism that may efficiently dissipate magnetic energy and at the same time contribute to the acceleration of the Poynting flux dominated outflow is the Kruskal-Schwarzschild (KS) plasma instability. This instability has only been studied analytically (Lyubarsky 2010) in the context of strongly magnetized relativistic jets, where the analytic treatment only considered the simplest case of a striped wind with exactly anti-aligned magnetic field lines. More complex and realistic scenarios, where the magnetic field lines can have a general misalignment angle and where the plasma modes can develop at different angles with respect to the magnetic field lines, can only be realized numerically with the help of computationally expensive 3D MHD numerical simulations. To this end, we aim to use the preparatory access to study this instability using the publicly available MHD code Athena. We have already conducted 2D simulations of the KS instability that allowed us to simulate only the exactly anti-aligned magnetic field lines case. These simulations were performed on a single node with 8 cores and thus restricted us to low resolution runs. High resolution 3D simulations of strongly magnetized relativistic plasmas are still challenging and therefore during the preparatory access the code will be tested at different resolutions to see if it is able to resolve the instability at physically important scales. A key test for the simulations will be the comparison of the growth rate of the instability in the linear stage to that obtained from analytic linear stability analysis (which we have already calculated). This will guide us to the minimum resolution needed to resolve the instability so that a longer running simulation, which will explore the non-linear development, can be setup during the production runs. This preparatory access will also be used to conduct a parameter space study to prepare for the higher resolution publication runs.

top

Ab-initio calculation of the third virial coefficient of water

Project Name: Ab-initio calculation of the third virial coefficient of water
Project leader: Dr. Giovanni Garberoglio
Research field: Chemical Sciences and Materials
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Hazel Hen, 50000 core hours on SuperMUC
Description

Virial coefficients are routinely used to correct non-ideality of real gases in common laboratory conditions and/or to constrain parameters appearing in the reference equation of state of a given substance. Virial coefficients higher than the second are quite difficult to measure experimentally, hence their values are characterized by very large uncertainties. Recent advances in ab-initio electronic structure calculations are able to provide very accurate intermolecular potentials from which virial coefficients for simple atoms and molecules can be calculated with accuracy matching or exceeding the experimental one. This has been shown to be the case for the third virial coefficients of helium (J. Chem. Phys. 134, 134106 (2011)) and molecular hydrogen (J. Chem. Phys. 137, 154308 (2012), Chem. Phys. Lett. 557, 26 (2013)) In this project we will calculate the third virial coefficient of water using state-of-the-art two- and three-body potentials, namely those of the CCpol family. The target accuracy is so strict that isotopic effects are expected to be non-negligible and hence path-integral methods are required to take into account the quantum nature of the hydrogen atoms present in the molecule. However, the complexity of the CCpol potentials (which include polarization effects) results in very long run times (of the order of 6000 cpu-hours for a given temperature) which required parallelization of the path-integral Monte Carlo code that was developed for He and H2 and the need of large-scale computing facilities to obtain accurate values in a reasonable time frame. The accuracy of the code and its good scalability up to 128 processors has been already tested in a local facility. This project is aimed at testing the scalability of our code to a larger number of processors and investigate the computational resources needed to calculate the third virial coefficient of water in the temperature range from 200K to 2000K with a statistical error better than the experimental uncertainty.

top

Activation mechanism of the beta2-adrenergic receptor

Project Name: Activation mechanism of the beta2-adrenergic receptor
Project leader: Prof. Ilpo Vattulainen
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100000 core hours on Marconi-KNL, 50000 core hours on Curie, 100000 core hours on Hazel Hen, 50000 core hours on SuperMUC, 100000 core hours on Piz Daint
Description

G protein-coupled receptors (GPCRs) are versatile signalling proteins that mediate diverse cellular responses. With over 800 members, GPCRs constitute the largest family of integral membrane proteins in human genome and represent roughly half of all drug targets in modern medicine. The human ß2-adrenergic receptor (ß2AR) is one of the best-characterized GPCRs. It is expressed in pulmonary and cardiac myocyte tissues and is a therapeutic target for asthma and heart failure. The functional diversity of ß2AR is associated with its structural dynamics. Recently found structures of ß2AR (published in Protein Data Bank) in the inactive and active states have provided valuable insights into the structure-function relationship of ß2AR. However, the dynamics and molecular details of the ß2AR activation process are still missing. Here we aim to fill this gap by using an extensive all-atom Molecular Dynamics (MD) simulations using GROMACS software. System under investigation will be composed of the ß2AR embedded in the lipid bilayer of different lipid compositions. In order to provide a surface large enough to accommodate protein conformational changes and lipid types in a statistically significant quantity, large membranes must be used. A system of this size will require sufficient testing for ensure that the use of computational resources is done as efficiently as possible. In this project we will fine-tune and benchmark the performance of GROMACS for the solvated membrane-protein system.

top

Scaling representative and production applications built with a domain-specific language approach.

Project Name: Scaling representative and production applications built with a domain-specific language approach.
Project leader: Dr Istvan Reguly
Research field: Mathematics and Computer Sciences
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hoursn on Marconi-KNL
Description

The aim of the project is to evaluate the scalability of a number of representative applications (TeaLeaf, CloverLeaf 2D/3D) and a large-scale (production) direct numerical simulation research application (OpenSBLI) that are built on the OPS domain-specific language and library. These applications are all multi-block structured mesh applications that use an unstructured collection of structured mesh blocks. The OPS DSL is developed targeting this domain. Based on their high-level description, OPS automatically parallelize these applications with a variety of approaches (OpenMP, OpenCL, OpenACC, CUDA, etc., and their combinations with MPI) and applies a number of algorithms to improve performance. The key algorithm to evaluate in this specific time allocation request will be cache-blocking tiling, that is applied both within a single process and across MPI processes, improving cache locality and reducing MPI communications. For this project, we would like to evaluate and compare the scalability of the traditional MPI communication scheme that exchanges the halos of datasets before loops on-demand, to an execution scheme that is so far unique to OPS: operations over the mesh are queued instead of immediately executed, then analysis is carried out to come up with a loop schedule that improves locality, and a communication schedule that carries out several larger halo exchanges upfront, and then does not do any until a synchronisation point (such as a reduction) is encountered.

top

Cosmological Volumes of Simulated Galaxies with Resolved Structure and Multiphase Interstellar Medium

Project Name: Cosmological Volumes of Simulated Galaxies with Resolved Structure and Multiphase Interstellar Medium
Project leader: Prof. Robert Feldmann
Research field: Universe Sciences
Resource awarded: 50000 core hours on Marconi-Broadwell
Description

Recently, large-volume cosmological simulations that reproduce many properties of observed galaxies have become available. However, these simulations model core physical processes on interstellar medium scales (such as star formation, stellar feedback, and supermassive black hole growth) using finely tuned sub-grid models, limiting their predictive power and compromising their ability to capture physics internal to galaxies. Zoom-in simulations have begun to implement much more detailed ISM models, but are available only for selected halos. We propose a new set of simulations designed to bridge the gap between current large-volume and zoom-in simulations. The resolution is comparable to today’s best zoom-ins but contain hundreds of galaxies. Our simulations implement the comprehensive stellar feedback model developed as part of the FIRE zoom-in simulation project, which has been demonstrated to reproduce a wide range of key galaxy observables without the need to tune parameters. Furthermore, all runs will be based on a new Meshless Finite Mass (MFM) hydrodynamic solver, which has been demonstrated to provide superior accuracy relative to smoothed particle hydrodynamics (SPH) for a wide range of problems. The preparatory project will be used for scaling tests of the proposed target simulations.

top

Code scalability testing for atomistic molecular dynamics simulations of respiratory complex I

Project Name: Code scalability testing for atomistic molecular dynamics simulations of respiratory complex I
Project leader: Dr Vivek Sharma
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100000 core hours on Marconi-KNL, 100000 core hours on Piz Daint
Description

Energy plays a central role in our lives; it is required for heating, lighting, or as a fuel. In a human body, energy is stored in the form of a small molecule called ATP (adenosine triphosphate). It is produced in the mitochondria of the cell by the action of various enzymes. One such enzyme is respiratory complex I, which contributes to about 40 % of ATP generation in mitochondria. Besides its central role in ATP production, complex I is known to be associated with various mitochondrial disorders. How enzyme functions or dysfunctions during a disease, remains unknown. In this project, we will perform scalability tests of a large atomistic model system of complex I, which as been constructed from an X-ray structure. We will use highly-efficient and highly parallelized molecular dynamics simulation software GROMACS for the purpose. The scalability plots will be used for forthcoming large-scale PRACE projects.

top

Strain-specific interplay of alpha-synuclein with membranes

Project Name: Strain-specific interplay of alpha-synuclein with membranes
Project leader: Dr Liang Xu
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 100000 core hours on Hazel Hen, 100000 core hours on Piz Daint
Description

The pathology of many neurodegenerative diseases is closely related to amyloid deposits in patient’s brain. The accumulation of a-synuclein protein results in Parkinson’s disease (PD), dementia with Lewy bodies (DLB) and multiple system atrophy (MSA). The precise mechanisms of a-synulcein that leads to toxicity and cell death are largely elusive. However, it’s clear that a-synuclein interacts with a variety of cellular membranes and these interactions may contribute to its function and/or pathology. Modulating membrane binding of a-synuclein may provide a therapeutic strategy. Recent studies revealed that different a-synuclein strains exhibit distinct properties such as difference in secondary structure, neurotoxicity, ability of seeding/cross-seeding, and propagation. To better understand the molecular signature that determines the strain-specific interactions between a-synuclein and membranes, we aim to perform multi-scale molecular dynamics (MSMD) simulations to examine the mode of action of a-synuclein in the presence of various model membranes. The aggregation propensity of diverse a-synulcein strains (oligomers or fibrils) will be tested and the effect of membrane composition on the binding preference of a-synuclein will be carefully investigated. The combination of large-scale simulation and advanced analysis methods will provide valuable insight into the strain-dependent interactions of a-synuclein with membranes.

top

Testing the scalaibility of Lattice QCD codes on new computer architectures

Project Name: Testing the scalaibility of Lattice QCD codes on new computer architectures
Project leader: Dr Piotr Korcyl
Research field: Fundamental Physics
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Marconi-KNL
Description

The aim of the project is to benchmark the scaling properties of lattice QCD software on new computer architectures. Lattice QCD is a numerical approach to Quantum Chromodynamics which offers calculations of physical properties of hadronic matter from first principles. It is computationally very demanding as it involves systems of the size of 10^10 variables with complex interactions. Hence it necessitates highly optimized software tailored for specific computer architectures. In this project we will bechmark the performance of our software on the Marconi system, in particular its weak and strong scaling properties, with an aim of a subsequent large scale PRACE computer time application.

top

Characterizing the structural basis for the nucleosome recognition by pioneer transcription factors: preparatory phase

Project Name: Characterizing the structural basis for the nucleosome recognition by pioneer transcription factors: preparatory phase
Project leader: Dr Vlad Cojocaru
Research field: Biochemistry, Bioinformatics and Life sciences
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Marconi-KNL, 50000 core hours on SuperMUC
Description

Transcription factors are proteins that directly or indirectly bind to DNA in order to transcribe genetic information into RNA. In most cases accessibility to DNA is a prerequisite for binding of transcription factors. However, in the nucleus the DNA is packed into chromatin, which quite often is inaccessible to transcription factors. The fundamental unit of chromatin is the nucleosome, which is formed by wrapping 147 DNA base pairs around a core of eight histone proteins. Interestingly, a series of transcription factors are able to bind to closed chromatin states, recognizing their binding sites even in the presence of nucleosomes. These factors, known as “pioneer transcription factors”, can help open chromatin, increase DNA accessibility, and support binding of other transcription factors. How transcription factors recognize their binding site on a nucleosome is not known. In particular, structural data on nucleosome – transcription factor complexes are not available. This would be of utmost importance as characterizing the molecular mechanism of how transcription factors bind to nucleosomes is a crucial step towards understanding how chromatin is opened to eventually exert a certain biological function. In recent years, it has been reported that many of the transcription factors involved in transitions between different cellular identities are pioneer factors. In particular, in a Nobel Prize- awarded discovery, it has been shown that three such factors, Oct4, Sox2, and Klf4 are required to convert a somatic skin cell into a pluripotent stem cell. When introduced in such skin cells, Oct4 and Sox2 recognize their binding sites in DNA wrapped in nucleosomes. Based on chromatin immunoprecipitation followed by sequencing and nucleosome footprinting experiments, Oct4 and Sox2 binding sites and their overlap with nucleosome positions have been identified. Based on the available experimental data, we have built atomic resolution structural models of Oct4 – nucleosome interaction on two native DNA sequences. In collaboration with experimenters at our institute, we partially validated these models using mutations in the DNA binding sites and electrophoretic mobility shift assays. To further validate the models and decode the structural dynamics involved in Oct4-nucleosome binding at atomic resolution, we will now perform a series of molecular dynamics simulations of alternative Oct4-nucleosome configurations. Because these molecular systems and the number of simulations required are large, we require access to additional computational resources for the success of this project. Therefore, we will submit a full PRACE proposal for the upcoming call (deadline 30th of May 2017). For this, we are now requesting preparatory access to several PRACE sites to optimize the scaling of the performance of the software we will be using for our specific systems.

top

Distributed Computation of Matrix Inverse using a Probabilistic Approach

Project Name: Distributed Computation of Matrix Inverse using a Probabilistic Approach
Project leader: Prof Juan Acebron
Research field: Mathematics and Computer Sciences
Resource awarded: 50000 core hours on Marconi-Broadwell, 100000 core hours on Marconi-KNL, 50000 core hours on Curie, 50000 core hours on SuperMUC
Description

Current (and future) applications require increasingly powerful and efficient resources. In recent years, the hardware has experienced extraordinary advances, more than any other scientific discipline. The reality today is that the most advanced computer systems are made up of millions of cores. However, it is now the software, in particular parallel numerical algorithms, which proves to be the weakest element in solving problems. Large-scale computations have seen an intolerable waste of resources, making it impossible in some cases to exploit the full potential of available resources, mostly due to communication overheads. The study of the properties of complex networks has been the topic increasingly intense research in the last few years. The reason for this interest is that complex networks arise in many different groundbreaking areas of science. This is the case for networks arising in technological, social, biological, and others. Important metrics, such as entropy, node centrality, and communicability require the computation of a function of the network adjacency matrix, an operation that is not viable for large networks. In many cases the representation of the adjacency matrix for these networks is only possible because they are naturally sparse. Unfortunately, in general, the function of a matrix is a full matrix that simply cannot be represented due to its large size. This limitation severely hinders the analysis of networks of interest. Moreover, even for smaller matrices that allow the function of the matrix to fit in the available memory, classical methods for the computation of the function are not easily parallelizable. These methods require an amount of communication that reduces significantly the efficiency and scalability of a parallel solution. A new computational paradigm is proposed as solution. We are developing probabilistic numerical methods for the computation of different operations on matrices, in particular the computation of the inverse, and the exponential. These methods are intrinsically parallel. More importantly, they allow the computation of individual positions of the inverse matrix, hence any operation is feasible on matrices of arbitrary size as long as its representation is available.

top

Scalability tests for phonon and electron-phonon calculations using Quantum-Espresso and GW calculations using Yambo on 2D materials typical systems.

Project Name: Scalability tests for phonon and electron-phonon calculations using Quantum-Espresso and GW calculations using Yambo on 2D materials typical systems.
Project leader: Prof Nicola Marzari
Research field: Chemical Sciences and Materials
Resource awarded: 100000 core hours on Marconi-KNL
Description

In this project we will perform scalability test and performances analysis for QuantumEspresso ph.x code (phonon and electron-phonon) and Yambo (GW) on systems of small medium and big size representative of the materials contained in the database of 1844 2D materials build in our group by performing a “computational exfoliation” more than 110000 bulk materials. These tests will represent a preparatory phase for the submission of a project within the 15th PRACE call in which we will propose a detailed study on the electronic, dynamical and transport properties of selected materials of this database.

top

Dynamics in Hybrid Perovskite Materials

Project Name: Dynamics in Hybrid Perovskite Materials
Project leader: Prof. Alison Walker
Research field: Chemical Sciences and Materials
Resource awarded: 50000 core hours on Curie, 100000 core hours on Hazel Hen, 50000 core hours on SuperMUC
Description

In recent years, organic inorganic hybrid perovskite solar cells have attracted a huge research interest due to their exceptional photovoltaic properties as well as easy and cheap fabrication processes.[1] The solar cells made of these materials have recently demonstrated a certified power conversion efficiencies of more than 22%.[2] Due to ‘soft’ nature of these hybrid perovskites, dynamical structural changes influence their optoelectronic properties quite noticeably.[3] Recently, various experimental as well as computational studies have further pointed out that dynamics of inorganic framework is coupled with the organic molecular motions, [3] having prominent effect towards band-gap fluctuations, charge transport, exciton binding energy.[4] Dynamical variation of these properties at ambient conditions has been investigated very rarely.[5] Ab initio molecular dynamics (AIMD) based on density functional theory (DFT) has emerged as the perfect method to explore this field of research very recently.[6] AIMD efficiently and cost-effectively captures the dynamical properties of hybrid perovskites with tens of picosecond time-scale. In our work, we propose to perform AIMD simulations on various kinds of perovskite materials using CP2K package.[7] Very recently, stability of the 3D-perovskite at ambient condition is one of the major problems, creating roadblock to commercialize these in large scale.[8] Apart from various external factors, it has been ditected that perovskites are intirnsically unstable due to low crystallization energy. In our project, we intend to look at the dynamical behavior of the perovskite crystals at atomistic level and propose the possible way to enhance the stability as well as modify photovoltaic properties. Particularly, incorporating different kinds of doping, we will investigate how one can increase the internal interactions among organic and inorganic components. That in turn, can results in enhanced stability of these materials in their perovskite phases. Along with exploring stability of the perovskites, we are also interested to look into the ion migration in these materials.[9] It’s known that ion migration causes the hysteresis in the current-voltage diagram as well as resposible for degradation of their solar-cell device. Though, various computational studies have thoroughly investigated this issue with static DFT-based calculations, finite temperature effect which has large impact, has been largely overlooked due to cost of computation. In our project, we are going to explore defect-migration in these perovskites using AIMD. The exact mechanism of defect migration and its effect towards structural and optoelectronic properties under ambient condition can be studied in atomistic level. With this knowledge, we can explain the physical origin of various recent experimental reports of suppressed hysteresis. We can further propose other probable easy and cost-effective ways to reduce ion-migration in operating solar-cell devices. [1] Science 2012, 338, 643 [2] NREL efficiency chart; https://www.nrel.gov/pv/assets/images/efficiency-chart.png [3] Acc. Chem. Res. 2016, 49, 573 [4] Phys. Rev. B: Condens. Matter Mater. Phys. 2016, 94, 04520 [5] J. Am. Chem. Soc., 2017, 139, 4068 [6] J. Phys.: Condens. Matter, 2016, 29, 043001 [7] https://www.cp2k.org/ [8] Angew .Chem .Int. Ed. 2017, 56, 1190 [9] Acc. Chem. Res., 2016, 49, 286

top

Scalability test of C++ code developed for solving compressible Navier-Stokes equations

Project Name: Scalability test of C++ code developed for solving compressible Navier-Stokes equations
Project leader: Professor Vassilis Theofilis
Research field: Engineering
Resource awarded: 50000 core hours on Marconi-Braodwell, 100000 core hours on Marconi-KNL, 100000 core hours on Hazel Hen
Description

An in-house C++ high fidelity DNS/LES parallel code was developed. The code solves density, momentum and total energy in cylindrical coordinate with a hybrid solver employing a sixth order central finite difference scheme for smooth regions and a fifth order weighted essentially non-oscillatory scheme with local Lax-Friedrichs flux splitting into discontinuous regions. Temporal integration is performed using a fourth-order five-step Runge-Kutta scheme. The subgrid scale terms were computed using Germano’s dynamic model. The one-dimensional non-reflecting boundary conditions are used for the adiabatic walls and outflow regions. The code will be used in a collaborative research project between University of Liverpool (UK) and Monash University (Australia). The good scaling was achieved on national HPC facilities in Australia. The main purpose of this application is to test the scalability of the code on Marconi and Hazel Hen facilities. The long-time goal is applying for the computer time on the Tier-0 supercomputer systems to extend our simulations to larger domains and higher Reynolds numbers.

top

SOWFA scalability tests

Project Name: SOWFA scalability tests
Project leader: Dr Paolo Schito
Research field: Engineering
Resource awarded: 50000 core hours on Marconi-Braodwell, 100000 core hours on Marconi-KNL
Description

Wind farm energy harvesting capability is generally calculated through the wind resource assessment. The real energy production of the wind farm is measured during the operation. The calculation of the loads on the wind turbines is not explicitly calculated during the wind farm operations and the control strategy is currently not taking into account the interference of upwind wind turbines. Current state of the art of wind farm controllers is designing control strategies that minimize the interference between wind turbine wakes and reduces possible extreme loading conditions on the rotors, in order to achieve a higher energy extraction and reducing the fatigue loads on the machine. The H2020 European project is currently investigating advanced wind farm control possibilities, by means of full scale, wind tunnel and CFD modelling. The CFD modelling tool is an advanced framework, where the dynamics of each single turbine, the control of each turbine and the control of the entire wind farm, are all taken into account and related to the incoming wind characteristics. This tool is called a high fidelity model, since the detailed modeling of the wind flow in the wind far is available. The study will investigate different control strategies to achieve the goal of lower LCOE (levelized cost of energy), by reducing the turbine loads and the increase of power extraction by the wind farm.

top

Scalability of ComprEssible DUct fLow

Project Name: Scalability of ComprEssible DUct fLow
Project leader: Dr Davide Modesti
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-KNL
Description

The project aim is to develop and test the scalability performances of an MPI implicit solver for the solution of the compressible Navier-Stokes equations in a duct flow. The main goal of the project is to obtain scalability data on MARCONI-KNL supercomputer, to apply for the next Prace call, on the same machine.

top

Scaling of a discontinuous Gakerkin code for simulation of low-pressure turbines.

Project Name: Scaling of a discontinuous Gakerkin code for simulation of low-pressure turbines.
Project leader: Dr Jean-Sebastien Cagnone
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-KNL
Description

The objective is to perform scaling studies with Argo, a high­-order ?discontinuous Galerkin (DGM) solver. We will evaluate the parallel performance of the code, with the final objective of performing high-resolution Large ?Eddy Simulations (LES) of the flow in three­-dimensional low­-pressure turbines passages, including end­-wall effects and corner separation. The parallel performance of Argo was previously demonstrated on Intel and BlueGene machines of the Prace network. In this preparatory call, we will port the code to Intel’s KNL many-core architecture, and measure the achieved speed-up on this new computing infrastructure.

top

HOSTEL DG (High Order Space-Time Eulerian-Lagrangian Discontinuous Galerkin)

Project Name: HOSTEL DG (High Order Space-Time Eulerian-Lagrangian Discontinuous Galerkin)
Project leader: Ing. Walter Boscheri
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-KNL
Description

Lagrangian algorithms have become very popular in the last decades because they allow the discontinuities which affect the flow motion to be precisely located and tracked. Moving mesh schemes typically present less numerical diffusion, hence achieving a better resolution of the flow features. As a drawback, since the mesh is moving in time and continuously changes its configuration, the computational grid might involve bad elements, i.e. highly stretched or distorted control volumes that would yield a significant reduction of the admissible time step. In some extreme cases, the computational cells can even become invalid, meaning that an element has a negative volume which would inevitably blow up the simulation. This is why a new family of Arbitrary-Lagrangian-Eulerian (ALE) finite volume methods has been developed, in which the mesh velocity can be chosen independently from the local fluid velocity. This allows the mesh to be moved more carefully, taking care of the geometrical quality of the elements. Specifically, we employ a rezoning strategy that aims at locally optimizing the shape of the control volumes, therefore avoiding the occurrence of invalid elements. We will focus on the development of the first better than third order Discontinuous Galerkin schemes in the context of ADER ALE methods. In the high order DG framework we have to take into account a curved geometry configuration that leads to curved boundaries for the definition of the control volumes. Furthermore classical DG methods are known to produce strong oscillations in the presence of discontinuous solutions, therefore we plan to implement an posteriori sub-cell limiter to overcome the problem of limiting. The discrete solution in those cells affected by a strong discontinuity is computed relying on a very robust TVD finite volume scheme which operates on the sub-grid level. The sub-grid is composed by a total number of M=2N+1 subcells per space dimension, where N is the degree of the DG scheme. We want to develop the new algorithm in two and three space dimensions on unstructured meshes and we require the new schemes to be high order accurate both in space and time, keeping the ALE approach that has proved to provide excellent resolution properties especially across contact discontinuities. The DG method will certainly produce more accurate and less diffusive results w.r.t. the ones obtained with a finite volume scheme, hence representing a very powerful tool for Lagrangian-like simulations. The goal of this project is to improve the scalability of our code in order to be ready for a submission for a regular PRACE call.

top

Type B: Code development and optimization by the applicant (without PRACE support) (5)

Parallel Partitioning methods for Hybrid Linear System Solvers

Project Name: Parallel Partitioning methods for Hybrid Linear System Solvers
Project leader: Prof. Cevdet Aykanat
Research field: Mathematics and Computer Sciences
Resource awarded: 250000 core hours on Hazel Hen, 250000 core hours on Juqueen
Description

We propose a new parallel partitoning method for hybrid linear system solvers. The objective is to increase the performance and the scalability of the hybrid sparse linear system solver. Our algorithm utilize the structure of the coefficient matrix and divides it into sub-matrices according to the number of processors in the parallel environment. Each processors solve the small linear system individually. An iterative solver is used to improve the partial solution iteratively. After a number of iterations the solution is acquired.

top

Development of plasma model for aeroacoustics simulations for noise reduction devices in airship design

Project Name: Development of plasma model for aeroacoustics simulations for noise reduction devices in airship design
Project leader: Prof. Joan Baiges
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-Broadwell, 200000 core hours on Curie, 100000 core hours on SuperMUC
Description

The current project aims at the development of the required tools for the numerical simulation of plasma actuators in noise reduction mechanisms in aircraft design. The project is framed in the IMAGE H2020 european project, a Europe – China collaboration project. http://www.cimne.com/vpage/2/2189/Objectives. The control approach proposed in IMAGE involves plasma actuation. To date, Dielectric Barrier Discharge plasma actuators represent one of the most promising avenues for controlling boundary layer flow and noise production. Technologically, advantages of the approach include the lack of any mechanical parts, light weight and the virtually unlimited control strategies offered considering they can be distributed in patches of almost any form. Furthermore, it has been shown that their electro-mechanical efficiency can also be altered by varying the amplitude and temporal variation of the driving voltage. In fact, much has still to be discovered about these actuation devices, but the perspectives are very encouraging considering the results reported in the literature, Two main code developments need to be tested in an MPI setting and validated in this setting: The first consists in the plasma body force which acts at the aerodynamic level and which accounts for the effect of the plasma actuators. We need to ensure that the implemented body force does not harm the scalability of the code. For this, several scalability tests are going to be performed. Also, this body force needs to be calibrated with several test cases, which will also be run in the current project. The tests cases of interest are the flow past a tandem cylinder configuration, for which experimental data at high Reynolds numbers is already available. The second test case will be the flow in a Wing-Mock-Up configuration, again comparison of the developed model agains experimental results will be performed. I The second implementation aspect which will be dealt in this problem is an efficient MPI-IO (output) strategy, which needs to be implemented for the visualization of the results. Our current implementation is based on the GiD postprocessor. However, for very large simulations this format is not suitable. We have implemented a VTK postprocess format which needs to be tested when using a large number of processors. Depending on the performance of this VTK format postprocessor, implementation and testing of HDF5 postprocess format will be also developed during the project.

top

A Scalable Discrete Adjoint Method For Large-Scale Aerodynamic Shape Optimization Problems In Incompressible Turbulent Flows

Project Name: A Scalable Discrete Adjoint Method For Large-Scale Aerodynamic Shape Optimization Problems In Incompressible Turbulent Flows
Project leader: Dr. Mariano Vazquez
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-Broadwell, 100000 core hours on SuperMUC
Description

The objective of this project is to develop a parallel framework to handle shape optimization problems using discrete adjoint method. This framework is to be utilized for designing aerodynamic devices including airplane wings as well as wind turbines in the real flow conditions. Due to the fact that the real 3D flow simulations require millions degrees of freedom, scalable schemes are proposed for resolving the flow and the adjoint equations which make this framework suitable for large-scale design problems. Since the turbulence effects are considered not only for the flow simulations but also for the adjoint sensitivities, the derived sensitivities contain all the physical characteristics of the real flows. In contrast to the conventional turbulent adjoint formulation, the one proposed here is based on the hand-coded linearization of the turbulence model without any approximation giving rise to the exact sensitivities. Hence, the aerodynamic characteristics are enhanced more efficiently in the developed framework.

top

NEWA2HPC

Project Name: NEWA2HPC
Project leader: Dr. Gokhan Kirkil
Research field: Engineering
Resource awarded: 100000 core hours on Marconi-Broadwell,
Description

The New European Wind Atlas (NEWA, http://www.neweuropeanwindatlas.eu) Project is funded under the European Commission’s 7th Framework Programme ERA-Net Plus (http://euwindatlas.eu/) that comprises nine funding agencies from eight EU Member States. The project aims at creating a new and detailed European wind resource atlas using meso- and microscale modeling, as well as collecting data from field experiments to generate a high-fidelity database of wind characteristics. The NEWA project will develop a new reference methodology for wind resource assessment and wind turbine site suitability based on a mesoscale to microscale model-chain. This new methodology will produce a more reliable wind characterization than current models, leading to a significant improvement in the quantification of uncertainties on wind energy production and wind conditions that affect the design of wind turbines. The model-chain, e.g. how the models at various spatial scales share information, will be thoroughly validated across Europe with dedicated experiments and historical wind resource assessment campaigns from industry. High fidelity experiments will be executed to address wind energy specific modelling challenges in complex and forested terrain, coastal transitions and offshore. The reference model-chain code will be offered as open-source together with best practice guidelines to help standardizing the methodology for industrial use. As a result, NEWA database will be published open access, based on a publicly available reference model-chain, whose credibility will be built upon strong sense validation benchmarks. We need PRACE computing resources to support the production phase of the wind atlas for which a PRACE Project Access application will be submitted in 2017. Initially this proposal was submitted for MareNostrum, but it was rejected due to the upcoming upgrade. As we must start as soon as possible with the preparation work for the wind atlas, we re-submit it now for Marconi. The wind atlas climatology will be produced with the Weather Research and Forecast (WRF) model (http://www.wrfmodel.org/index.php). The WRF modeling system should first be installed and tested on Marconi. Afterwards, the focus of this preparatory phase will be on the optimization of the model set-up to make the best use of the allocation of the production phase.

top

Adoption of high performance computing in Neural Designer

Project Name: Adoption of high performance computing in Neural Designer
Project leader: Mr. Fernando Gomez Perez
Research field: Mathematics and Computer Sciences
Resource awarded: 200000 core hours on Marconi-KNL, 100000 core hours on SuperMUC
Description

The predictive analytics market is undergoing an impressive growth. Indeed, organizations that incorporate that technique into their daily operations not only better manage the present, but also increase the probability of future success. Artelnics develops the professional predictive analytics solution called Neural Designer. It makes intelligent use of data by discovering complex relationships, recognizing unknown patterns, predicting actual trends or finding associations. Neural Designer out-stands in terms of usability, functionality and performance. Current technology lacks from advanced model selection techniques, and usually requires many computational resources. The main challenge for Neural Designer is to include a framework capable of untangling complex interactions in big data sets. In order to do that, the software must achieve high performance by means of parallel processing. The users of the solution are professional data scientists, which work at analytics departments of innovative companies, consulting firms specialized in analytics or research centres. Neural Designer will be capable of analysing bigger data sets in less time, providing our customers with results in a way previously unachievable.

top

Type C: Code development with support from experts from PRACE (3)

Quasi-particle self-consistent GW approximation: avoiding the I/O bottleneck

Project Name: Quasi-particle self-consistent GW approximation: avoiding the I/O bottleneck
Project leader: Prof. Mark van Schilfgaarde
Research field: Chemical Sciences and Materials
Resource awarded: 100000 core hours on Marconi-Broadwell, 250000 core hours on Hazel Hen, 100000 core hours on SuperMUC
Description

The Questaal Suite offers a range of electronic structure programs that can be used to model different materials and nanoscale structures. The majority of the codes use an all-electron implementation of density-functional theory. This includes several forms (hamiltonian and Green’s function) that serve different purposes. Additionally there is an all- electron implementation of GW theory, including a quasiparticle self-consistent form of it. These codes share a basis set of atom-centred functions. The basis has its genesis in the Linear Muffin Tin Orbitals (LMTO) method of O. K. Andersen, who formulated the theory of linear methods in band theory. The LMTO and LAPW (Linear Augmented Plane Wave) methods are the most common direct forms of the linear methods, though most approaches (including those based on pseudopotentials) depend on a linearization as well. The present code is a descendent of the “tight binding linear method” that formed the mainstay of Andersen’s group in Stuttgart for many years. Applications include modeling electronic structure, magnetic properties of materials, Landauer-Buttiker formulation of electronic transport, impurity effects in solids, and linear response. Packages distributed in the Questaal suite include: Full Potential LMTO: This is an all-electron implementation of density-functional theory using convolutions of Hankel functions and Gaussian orbitals as a basis set. This code also provides an interface to a QSGW package. It is a?fairly accurate basis, and has been benchmarked against other all-electron schemes. QSGW: GW is usually implemented as an extension to the LDA, i.e. G and W are generated from the LDA. The GW package also has the ability to carry out quasiparticle self-consistency (QSGW). QSGW may be thought of as an optimised form of the GW approximation of Hedin. Self-consistent calculations are more expensive than usual formulations of GW based on a perturbation of density functional theory, but it is much more accurate and systematic. Self-consistency also removes dependence on the starting point and also makes it possible to generate ground state properties that are sensitive to self-consistency, such as the magnetic moment. QSGW is perhaps the most universally applicable, true ab initio theory for electronic states in extended systems that exists today. It has a proven ability to consistently and reliably predict such as quasiparticle (QP) levels for a wide range of materials such as graphene, Fe-based superconductors, and NH3CH3PbI3 (a recently popular solar cell material) in a consistent and parameter-free manner that cannot be achieved by other theories. Many other properties, such as Dresselhaus coefficients, electric field gradients, transmission probability, and spin waves, are well described by the theory. QSGW is more expensive than usual forms of GW because the entire self-energy (iGW) must be calculated. A parallel version of this code has been written. It contains a significant bottleneck, which prohibits the realistic application to systems with more than 40 atoms or so. The aim of this project is to redesign the code and eliminate this bottleneck.

top

Parallel curved mesh subdivision for flow simulation on curved topographies

Project Name: Parallel curved mesh subdivision for flow simulation on curved topographies
Project leader: Dr Xevi Roca
Research field: Engineering
Resource awarded: 100000 core hours on MareNostrum
Description

In the near future, we will use the prodigious potential offered by the ever-growing computing infrastructure to foster and accelerate the European transition to a reliable and low carbon energy supply. We are fully committed to the former goal by establishing an Energy Oriented Centre of Excellence for computing applications, (EoCoE), through the on-going contract H2020-EINFRA-2015-1. EoCoE aims to assist the energy transition via targeted support to four renewable energy pillars: Meteo, Materials, Water and Fusion, each with a heavy reliance on numerical modelling. The primary goal of EoCoE is to create a new, long lasting and sustainable community around computational energy science. To this end, we are resolving the current bottlenecks in application codes, leading to new modelling capabilities and scientific advances among the four user communities. Furthermore, we are developing cutting-edge mathematical and numerical methods, and tools to foster the usage of Exascale computing. For the EoCoE project, we are really interested in improving our turbulent flow simulation code for complex topographies. Nowadays, this code is used with success by the renewable energy company IBERDROLA to improve the production of their wind farm designs. However, we have detected that in very large wind farm simulation analysis, the generation of a mesh that preserves the inherent curvature of the topography can be the principal bottleneck of the whole process. To address this issue, we propose to implement a parallel uniform curved mesh subdivision method, in a HPC code developed at Barcelona Supercomputing Center (BSC) named Alya. The resulting meshes will preserve the inherent curvature of the mesh boundaries in the successive refinements. The advantage of the proposed strategy is that the subdivision is performed and stored in parallel and therefore, there are no memory constraints. For instance, if we start with a coarse mesh composed by 30M tetrahedra, after two consecutive subdivisions we obtain a finer distributed mesh composed by 30M x 8 x 8 = 1920M elements, few seconds later and completely stored in the memory. Furthermore, the finer curved mesh preserves the curvature information described by the first curved mesh. Finally, if the original numbering is conserved, then the post-processing can be performed on the coarse curved mesh.

top

Simulation of the Ocean Cleanup Array

Project Name: Simulation of the Ocean Cleanup Array
Project leader: …
Research field: Engineering
Resource awarded: 100000 core hours on MareNostrum
Description

Every year we produce about 300 million tons of plastic, a portion of which enters and accumulates in the oceans. Due to large offshore current systems called gyres, plastic concentrates in certain offshore areas, of which the Great Pacific Garbage Patch between Hawaii and California is the best-known example. The Ocean Cleanup (www.theoceancleanup.com) is a foundation that develops technologies to extract plastic pollution from the oceans and prevent more plastic debris from entering ocean waters. The main technology is the Ocean Cleanup Array which utilizes long floating barriers to capture and concentrate the plastic such that the system is a passive barrier. Computational Fluid Dynamics (CFD) is being used to study the catch efficiency debris of different sizes and densities, the transport of plastic along the containment boom, and the forces acting on it in order to determine the appropriate shape for their passive barrier concept. A study for the wave and boom influence on particle trajectories has to be done with CFD to investigate the effects of wind-and- wave-induced turbulence on the boom capture efficiently as well as to include the interaction between particles and the dynamic structure in the CFD analyses. This simulation will be carried out by Alya code, which has demonstrated its high potential to face complex problems. The project contains many difficulties that can only be solved by a code like Alya.

top

Type D: Optimisation work on a PRACE Tier-1 (4)

Extending the scalability and parallelization of SEDITRANS code

Project Name: Extending the scalability and parallelization of SEDITRANS code
Project leader: Dr. Guillermo Oyarzun
Research field: Engineering
Resource awarded: 150000 core hours on Tier-1
Description

This project is a WP5 activity part of the Initial Training Network SEDITRANS (GA number: 607394), implemented within the 7th Framework Programme of the European Commission under call FP7-PEOPLE-2013-ITN. SEDITRANS is a research network that consists of six academic and four industrial partners within Europe. It is focused in to advancing in the comprehension of coastal processes utilizing high performance computing (HPC) for the numerical simulation of the three-dimensional, turbulent flow, which is induced in the coastal zone, and mainly in the surf zone, by wave propagation (oblique to the shore), refraction, breaking and dissipation. Currently, the parallel code is optimized to be used in medium size clusters by means of a MPI parallelization, using typically between 100 and 1024 CPUs for each execution. Our aim is to extend the parallelization of the code, in order to run it on hybrid architectures such as the ones of the Tier0. So far we have tested the GPU parallelization in some parts of the code, reporting promising results. However we need to analyze this strategy when we use more GPUs simultaneously combining the distributed memory parallelization using MPI with the stream processing within the GPU.

top

Automation of high fidelity CFD analysis for aircraft design and optimization

Project Name: Automation of high fidelity CFD analysis for aircraft design and optimization
Project leader: Dr Mengmeng Zhang
Research field: Engineering
Resource awarded: 150000 core hours on Tier-1
Description

Airinnvoa is a company to develop computational solutions for aerodynamic shape optimization, which is an important task in aircraft design. The high fidelity CFD (computational fluid dynamics) analysis is a major tool for modern aircraft design and optimization, and the computational power is a limiting factor. To carry out the high fidelity CFD requires the engineers have special skills in making mesh and execute the analysis code, which constraints the use of the CFD analysis only to a limited number of people. The goal of the proposed project is to help engineers to design the aircraft in a more efficient and simpler way by making the core processes automatic. Airinnova has been conducting the PRACE SHAPE project with collaboration with PRACE partner SNIC-KTH. The outcome of the research work has been presented in an AIAA conference paper. In this proposed project, we will follow from our previously work and take advantage of the optimization results and the existing scripts. In the proposed project, we will continue to carry out the high fidelity CFD analysis (RANS), with emphasizing running CFD in an automation way by starting from a watertight aircraft geometry. Gradient-based optimization algorithms by solving the adjoint-based equations will be applied to the final step of the automation process, which allows the flexibility integrate the whole automation process into a MDO (Multi-Disciplinary Optimization) design environment. The tasks mainly consist of: 1. Automation process development: Develop the automation process by starting from a watertight Common Research Model (CRM) aircraft geometry with designed pylons and nacelles. 3. Benchmark: performance analysis for the desired model using proposed automation process including auto-meshing and CFD solver auto-run. 3. Port: deploy and run on a PRACE Tier1 system and prepare for Tier-0 system

top

Scalable Delft3D FM for efficient modelling of shallow water and transport processes

Project Name: Scalable Delft3D FM for efficient modelling of shallow water and transport processes
Project leader: Dr Menno Genseberger
Research field: Earth System Sciences
Resource awarded: 150000 core hours on Tier-1
Description

Forecasting of flooding, morphology and water quality in coastal and estuarine areas, rivers, and lakes is of great importance for society. To tackle this, the modelling suite Delft3D, was developed by Deltares (independent non-profit institute for applied research in the field of water and subsurface). Delft3D is used worldwide. Users range from consultants, engineers and contractors to regulators and government officials. Delft3D has been open source since 2011. It consists of modules for modelling hydrodynamics, waves, morphology, water quality, and ecology. In two previous (small) PRACE projects [1, 2] and the FP7 Fortissimo experiment Delft3D as a Service (see for instance example in [3]) steps have been taken (a.o. with SURFsara and CINECA) to make Delft3D modules more efficient and scalable for high performance computing. Currently, for Delft3D there is a transition from the shallow water solver Delft3D-FLOW for structured computational meshes to D-Flow FM (Flexible Mesh) for unstructured computational meshes. D-Flow FM will be the main computational core of the Delft3D Flexible Mesh Suite. For typical real-life applications, for instance for highly detailed modelling and operational forecasting, there is urgency to make D-Flow FM also more efficient and scalable for high performance computing. As the solver in D-Flow FM is quite different from the one in Delft3D-FLOW some major steps have to be taken. Also for the modules for modelling waves, morphology, water quality, and ecology that connect to D-Flow FM. Aim of the current project is to make significant progress towards Tier-0 systems for the shallow water and transport solvers in the Delft3D Flexible Mesh Suite. Starting point are the results and experiences of the previous projects, mainly obtained on the Tier-1 system Cartesius. First steps with D-Flow FM on Cartesius were set in the Fortissimo project. For D-Flow FM the computational work is parallelized by domain decomposition. At the moment work started on optimal boundary conditions at the interfaces of the subdomains to minimize the amount of solver iterations and therefore the amount of required communication between subdomains (strategy originally developed in [4, 5]). This aspect is essential when making the step to exascale computations. The same technique has been applied before to the shallow water solver Simona [6, 7] which is almost identical to Delft3D-FLOW and was recently used by Deltares and SURFsara at Cartesius to optimize work for forecasting of flooding around the Dutch lakes [8]. With this technique, for more academic type of eigenvalue problems weak scaling up to 16000 cores was obtained at the Curie thin nodes [9, 10]. [1] http://www.prace-project.eu/IMG/pdf/wp100.pdf [2] http://www.prace-project.eu/IMG/pdf/wp177.pdf [3] https://ir.cwi.nl/pub/24648 (under revision for publication) [4] http://www.ddm.org/DD07/Tan_Borsboom.pdf [5] Tan, K.H.: Local coupling in domain decomposition. Ph.D. thesis, Utrecht University (1995) [6] http://www.ddm.org/DD21/homepage/dd21.inria.fr/pdf/borsboomgensebergerhofspee_mini_8.pdf [7] Vollebregt, Roest, Lander: Large scale computing at Rijkswaterstaat. Parallel Computing 29, pp. 1-20 (2003) [8] https://ir.cwi.nl/pub/25171 [9] http://www.ddm.org/DD21/homepage/dd21.inria.fr/pdf/gensebergerm_contrib.pdf [10] Genseberger, Improving the parallel performance of a domain decomposition preconditioning technique in the Jacobi-Davidson method for large scale eigenvalue problems, Applied Numerical Mathematics 60 pp. 1083-1099 (preprint at https://ir.cwi.nl/pub/13582)

top

Radiative Transfer Forward Modelling of Solar Observations with ALMA

Project Name: Radiative Transfer Forward Modelling of Solar Observations with ALMA
Project leader: Dr. Sven Wedemeyer
Research field: Universe Sciences
Resource awarded: 150000 core hours on Tier-1
Description

The Atacama Large Millimeter/submillimeter Array (ALMA), which is currently the world’s largest astronomical observatory, opened up a new window on the universe. The interferometric array is located on the Chajnantor plateau in the Chilean Andes at an altitude of 5000 m and consists of 66 antennas, most of them with a diameter of 12 m. By combining the antennae, they act like a giant telescope with baselines of up to 16 km. Already within the first few years of operation, ALMA led to many scientific discoveries. Since December 2016, ALMA is also used for observations of our Sun. It observes the Sun at a spatial resolution, which is unprecedented in this wavelength range, and offers novel means of determining the properties of the plasma in the Sun’s outer atmospheric layers. Due to the properties of the solar radiation at millimeter wavelengths, ALMA serves as a linear thermometer, mapping narrow layers at different heights. It can measure the thermal structure and dynamics of the solar atmosphere and thus sources and sinks of atmospheric heating. Among other expected scientific results, ALMA promises significant steps towards understanding the intricate dynamics and physical processes that, in combination, might yield the solution of the coronal heating problem – a long standing fundamental question in modern astrophysics. However, ALMA’s novel diagnostic capabilities, which will ultimately progress our understanding of the Sun, still need to be developed and understood further in order to fully exploit the instrument’s potential. Detailed numerical simulations of the solar atmosphere and artificial observations of the Sun play a key role in this respect. Such artificial observations of the Sun will be produced as part of the SolarALMA project at the University of Oslo, which is funded with a Consolidator Grant by the European Research Council (ERC), in collaboration with Dr. de la Cruz Rodriguez from the University of Stockholm. The overall aim of the SolarALMA project is to utilize the first observations of the Sun with ALMA and to develop the required observing and analysis techniques. An important step in this endeavour is the development of realistic numerical models of the solar atmosphere, which can be used to test how to optimally set up solar observations, and how to analyse and interpret them. While 3D numerical models are routinely produced on high-performance computers, the codes available for producing the corresponding (artificial) ALMA observations of such models did not perform well enough so far. We have developed a new code that solves the radiative transfer equation for a 3D numerical model and thus reveals how the modeled part of the Sun would look like through the eyes of ALMA at (sub)millimeter wavelengths. The new code is in an advanced stage but, still needs to be optimized in order to provide the basis for essential studies, which will result in optimizing ALMA’s scientific impact for observing and understanding the Sun.

top

Share: Share on LinkedInTweet about this on TwitterShare on FacebookShare on Google+Email this to someone