PRACE Preparatory Access – 24th cut-off evaluation on March 2016

Find below the results of the 24th cut-off evaluation of March 2016 for the PRACE Preparatory Access.

Projects from the following research areas:

 

Deciphering the update mechanism of a cell penetrating peptide using molecular dynamics simulations and free energy calculations

Project Name: Deciphering the update mechanism of a cell penetrating peptide using molecular dynamics simulations and free energy calculations
Project leader: Prof Elif Ozkirimli
Research field: Medicine and Life Sciences
Resource awarded: 100000 core hours on Fermi, 50000 core hours on MareNostrum, 5000 core hours on MareNostrum Hybrid Nodes
Description

Our long term objective is to design effective beta-lactamase inhibitor peptides that can be transported across the cell wall. We have previously identified a novel antimicrobial peptide by conjugating a beta-lactamase inhibitor peptide with a cell penetrating peptide. This novel peptide has five residues (LLIIL) based on the cell penetrating peptide pVEC sequence and these residues contribute to uptake across the bilayer. A detailed understanding of the mechanism by which the peptide interacts with the cell membrane will enable the development of similar peptide inhibitors. Some questions that need to be answered are: Does it penetrate the membrane without disrupting it or does it disrupt the cell membrane? Is membrane entry concentration dependent? How does the mechanism change when the lipid composition of the membrane mimic is altered? The aim of this project is to computationally study the uptake mechanism of the pVEC and similar peptides, designed with a similar strategy, by performing molecular dynamics simulations, steered molecular dynamics simulations and free energy calculations using the replica exchange umbrella sampling method. We aim to optimize the system size, simulation duration as well as the number of replicas with this PRACE Type A Preparatory Access.

top

Scalability assessment of Stochastic Field Methods using detailed Chemistry

Project Name: Scalability assessment of Stochastic Field Methods using detailed Chemistry
Project leader: Dr. Max Staufer
Research field: Engineering and Energy
Resource awarded: 50000 core hours on MareNostrum
Description

Aim of the project is to assess the scalability of stochastic field methods utilising detailed chemistry mechanisms. Turbulence in the flow field will be accounted for using a Large Eddy Simulation approach. Furthermore the project shall allow to gather information in order to issue a full proposal for high fidelity chemistry simulations in the PRACE framework.

top

Insight into Silver Nanocube-protein interactions by computational simulations

Project Name: Insight into Silver Nanocube-protein interactions by computational simulations
Project leader: Prof. Maria Cristina Menziani
Research field: Chemistry and Materials
Resource awarded: 100000 core hours on Fermi, 50000 core hours on MareNostrum
Description

Cytochrome c, a hemoprotein with an intermediate role in cell apoptosis, will be used as a test case for the binding of proteins to silver nanocubes (Ag-NCs) using Molecular Dynamics (MD) simulations. This research is part of a larger project, involving computational and experimental groups, aimed at improving the Surface-enhance Raman scattering (SERS) techniques for future diagnostic tools of aberrant oligomer species related to protein-deposition disease such as Alzheimer’s and Parkinson’s diseases. SERS effect and efficacy relies in the enhancement of Raman signal of low-abundance biomolecules using strong electro-magnetic fields concentrated on the surface of nanostructured materials. This effect is maximized at the edges of the nanocubes. Unfortunately, the interpretation of the SERS spectra is non-trivial, and MD simulations can facilitate this task by providing a detailed understanding of how proteins interact with Ag-NCs. The Ag-NCs synthesized by our collaborators have a size in the order of one hundreds of nanometers. This contains up to 6 million atoms and simulating a system made by a nanocube covered with proteins and explicit water is computationally unfeasible. To over come this problem, smaller NCs will be built maintaining their sizes big enough to accommodate at least two cytochromes on each surface. Several proteins will be inserted in the simulation box in order to monitor both the adsorption on silver and interactions between bound and unbound proteins. Computational simulations will be carried out both by considering proteins in contact with the naked surface of the Ag-NCs, and with coverage made by Polyvinylpyrrolidone (PVP). Results will show how and where cytochrome c binds to a Ag-NC both in presence and absence of PVP obtaining information about its conformational changes and on the molecular mechanisms of binding on the surface, edges and corners. The aim of this project is to demonstrate the high scalability of this systems on different european supercomputers for a future application to Prace projects.

top

Computing high-order coefficients of the virial equation of states

Project Name: Computing high-order coefficients of the virial equation of states
Project leader: Dr. Hainam Do
Research field: Chemistry and Materials
Resource awarded: 50000 core hours on MareNostrum; 5000 core hours on MareNostrum Hybrid Nodes;
Description

The hard sphere (HS) gas of impenetrable particles is one of the oldest and most studied systems in statistical thermodynamics. It has served as the simplest nontrivial model of fluid structure for more than a century dating back to the early works of the Nobel laureate van der Waals, who in 1873, introduced the concept of hard bodies representing atoms and molecules for the modelling of the pressure-volume-temperature behaviour of fluids. Despite the deceptive simplicity of this system, HS has been the basis for the advance of science in the fields of general liquids, amorphous solids, liquid crystals, colloids, granular matter, etc. It exhibits a rich structural and thermodynamic behaviour, including phase transitions, metastable states, demixing, etc. Many of these properties of HS are not completely understood even until now and are objects of current research. Amongst these are the convergence of the virial series of the HS and the question of the occurrence of the negative virial coefficients. These challenges required the knowledge of high-order virial coefficients, which are not accessible by any mean at the moment. In this work, the thirteenth virial coefficient B13 of HS, which is an unthinkable task, will be computed using a recursive method recently developed by us that is much faster than any previously used techniques. The efficiency of the algorithm is enhanced significantly by employing a parallel algorithm that assigns a similar number of biconnected canonical graphs to each parallel task. This balances the storage and processing load between the tasks, and it is found that the computer time is roughly equally divided between the generation of random chains, checking them for biconnectivity, calculating the total probability of generating the chain configurations, performing the canonical re-ordering, and computing the integrands. To investigate the occurrence of negative virial coefficients for HS, virial coefficients for up to the 12th order will be computed for the first time for the finite barrier potential.

top

Reproducibly Finding Performance Deficits in MPI Libraries

Project Name: Reproducibly Finding Performance Deficits in MPI Libraries
Project leader: Dr Sascha Hunold
Research field: Mathematics and Computer Science
Resource awarded: 100000 core hours on Fermi
Description

Our research focuses on benchmarking MPI libraries, in particular MPI collective communication operations. These collective operations are fundamental building blocks of large scale applications on current supercomputers. Thus, the performance of these libraries is of great interest to scientists as well as to system providers. We have developed a set of MPI benchmarks that enable us to verify self-consistent performance guidelines. A violation of a performance guideline directly pinpoints a performance degradation for a specific function and message size. The benchmark has been tested on several parallel machines, but only on one larger machine that is part of the TOP500 list. We therefore would like to know whether our benchmarking approach leads to insightful results on other supercomputers. The experimental results will allow us to compare the efficiency of different MPI libraries installed on current supercomputers. Our results will also enable MPI developers and system administrators to test and tune individual functions. Preparatory Access is important to test whether the code works, in particular to find a precise method for measuring time (e.g., using assembly instructions such as RDTSC, if supported by the architecture).

top

Molecular dynamics simulation of the nicotinic acetylcholine receptor

Project Name: Molecular dynamics simulation of the nicotinic acetylcholine receptor
Project leader: Dr Pak-Lee Chau
Research field: Medicine and Life Sciences
Resource awarded: 50000 core hours on MareNostrum
Description

I shall perform fully atomistic molecular dynamics simulations on the nicotinic acetylcholine receptor (nAChR) to define how ligand binding causes the receptor to open. The nAChR is a heteropentameric protein located across the cell membrane of the muscle cell. It contains five subunits, arranged pseudo-symmetrically around the central ion channel. On binding to acetylcholine released from the adjacent motor neuron, the ion channel opens and lets cations through. This triggers the muscle cell to contract. Thus this protein is key to the control of muscle contraction by nerves. Similar heteropentameric nAChRs are also found widely in the central nervous system, and are probably implicated in various neurodegenerative diseases. A better understanding of the function of these proteins is therefore of clinical interest. A closely related form of muscle nAChR is from the electric organ of the electric eel Torpedo. The closed-channel form of this protein is the only nAChR whose structure has been determined to 4AA resolution (PDB code 2BG9). Recent studies, subsequent to time limited exposure of this protein to the natural agonist, have furnished images of the open channel structure at 6AA resolution (PDB code 4AQ9), providing a structural image at both the ground state and the activated state of this important oligomeric protein. Both these structures have been determined by Nigel Unwin (MRC Laboratory of Molecular Biology, Cambridge). In my current collaboration with Unwin, we aim to obtain new high-resolution structures in late 2016, and use them to perform large-scale molecular dynamics simulations. Our goal is to investigate the conformational changes pertinent to the transition between the closed-channel and open-channel states of the nAChR. I have previously developed the mutual repulsion method to accelerate rare events (P-L Chau (2001) Chem. Phys. Letts., 334, 343-351); we shall implement it on NAMD and perform simulations using this method. We shall also perform steered molecular dynamics on this structure, to take it from the closed-channel state to the open-channel state. The proposed work is a preparatory access project to perform timing runs on Mare Nostrum. My previous application 2010PA3029 was successful. However, I find that SuperMUC is no longer available for project access on Call 13; only Mare Nostrum and Marconi are available. The architecture of Mare Nostrum is closer to that of SuperMUC, so I am applying for preparatory access to do time trials. Simultaneously, I am applying for project access to Mare Nostrum. I hope that the time trials will be complete when the technical checks on project access is performed. In this preparatory work, I shall obtain the experimental coordinates (PDB code 2BG9) from the Protein Databank, place the nAChR in a heterogeneous membrane patch (POPC/POPA/cholesterol in a 3:1:1 ratio), and fill the simulation box with an ionic aqueous solution. Two such systems will be prepared: one without ligand bound, and another with two acetylcholine molecules docked into the binding sites. I shall perform molecular dynamics on both systems to determine the time requirements, using the molecular dynamics package NAMD. I shall also perform mutual repulsion simulations and steered molecular dynamics. These are the simulations I shall perform on the high-resolution structures of the corresponding states in late 2016.

top

Kinetics of electron transfer between iron-sulfur clusters: consequences of the unique iron atom coordination

Project Name: Kinetics of electron transfer between iron-sulfur clusters: consequences of the unique iron atom coordination
Project leader: Dr Adam Kubas
Research field: Chemistry and Materials
Resource awarded: 50000 core hours on MareNostrum
Description

The production of H2 based on the enzymatic catalysis, in contrary to industrial-scale production methods, avoids unwanted by-products like CO2 and high energy consumption. While it is already possible to incorporate entirely synthetic active centre into a protein matrix, the way how nature switches between oxidation and reduction reactions, i.e. changes the electron flow direction, remains unclear. The project aims to improve our understanding of the electron flow regulation in enzymes relevant to hydrogen production (hydrogenases). The impact of unique ligands and their protonation state on the [Fe4S4] clusters that form electron transport chains can be used to design hybrid natural-synthetic systems with fine-tuned redox characteristics. The major goal of the project is to provide theoretical insides into the biological role of a single cysteine ligand replacement in usually all-cysteine ligated [Fe4S4] cubane clusters found in a variety of proteins. Aside known substitution with highly labile ligand (e.g. water) that creates catalytically active Fe centre like in enzyme aconitase, it is postulated that cysteine to histidine/aspartate mutations in the first coordination sphere of the iron-sulfur cubane provide the protein with an additional flexibility in the transfer parameters modulation. Thus, the aim of the project is to determine the effect of such unique ligand and its protonation state on the Markus parameters of the electron transfer (e.g. electronic coupling matrix element and reorganization energy) and consequently the rate of the electron transfer. The development of highly accurate quantum chemical methods allowed us, for the first time at the atomic level, to look at the properties of the [Fe4S4] clusters. In this work we will take the first step into the determination of the influence of the unique ligands on the electron transfer parameters in the electron transfer chain in the [FeFe] hydrogenases. The calculations will be carried out for the cluster dimers in the vacuum and as surrounded by the protein environment. Thus, we will be able to separate pure electronic contributions caused by modified ligands and the electrostatic effects of the protein matrix. Using only first principles we aim to uncover secrets of the natural regulation processes, that will undoubtedly pave the way to new and efficient bio-inspired catalytical systems even beyond hydrogen production.

top

Hybrid liquids from ZIFs: how do MOFs melt ?

Project Name: Hybrid liquids from ZIFs: how do MOFs melt ?
Project leader: Dr François-Xavier Coudert
Research field: Chemistry and Materials
Resource awarded: 50000 core hours on MareNostrum
Description

Metal-Organic Frameworks (MOFs) constitute a fast-growing class of materials aimed at many different of applications. They are mostly studied as crystalline materials for applications such as gas storage via adsorption or heterogeneous catalysis. Nonetheless, amorphous MOFs (aMOFs) are also of great interest. In fact, amorphization could be used to trap in an amorphous non-porous structure a harmful guest detected in a crystalline MOF. Time-controlled drug delivery could also be performed by amorphization of MOF drug delivery vehicles. While traditional silica-glass is hard to functionalize, MOF amorphization is of great interest to produce functional luminescent or optically active glass-like materials. Furthermore, whereas crystalline MOFs are often criticized because of their weak mechanical stability, aMOFs exhibit much better mechanical properties, which is quite important for industrial applications. In collaboration with experimental teams, we study the temperature-induced amorphization for some Zeolitic Imidazolate Frameworks (ZIF), especially ZIF-4. Experimentally, it is possible to obtain a glass-like material from crystalline ZIF-4 by melting and quenching. However, the liquid phase is very difficult to characterize experimentally, as is the melting process itself, because of the thermal conditions and of the disordered nature of this phase. Our project focuses on this melting process at the molecular level, via ab initio molecular dynamics simulations. This level of description is necessary in order to study the melting process. Indeed, force-field based approach developed for MOFs are still pretty bad at describing the coordination chemistry inherent to metal-organic networks and are often totally unable to describe covalent bond breaking and reforming, which are expected to happen here. The simulation of such systems, that contains several hundred of atoms per cell, can only be performed on parallel architectures. As we carry out simulations in the canonical ensemble, at constant density, we will need to perform them on several systems with different densities. We will then analyze the results in terms of effective normal modes to identify the vibration modes responsible for the melting process. Also, computing the pair distribution functions, we will be able to compare the phase obtained by simulation and the liquid phase obtained experimentally. The aim of this study is to get a deeper understanding of the melting process in MOF at a molecular scale, providing information on the local structuration of the liquid phase. In fact, for different crystalline MOFs, the behavior under melting is different as far as local atomic environment is concerned.

top

Ab-initio simulations of interactions between small biomolecules and inorganic materials is aqueous media

Project Name: Ab-initio simulations of interactions between small biomolecules and inorganic materials is aqueous media?
Project leader: Prof Alexander Lyubartsev
Research field: Chemistry and Materials
Resource awarded: 50000 core hours on MareNostrum, 5000 core hours on MareNostrum Hybrid Nodes
Description

The interactions between nanomaterials and biological matter (proteins, lipids, glucanes) is of primary importance for many important applications: development of bio-sensors, biomedical implants, drug delivery nanodevices, food packaging, antifoiling surfaces for coating in industrial and marine environment, etc. Furthermore, understanding the biocompatibility of inorganic nanoparticles (NP) with biological molecules is a key element in the evaluation of possible risks associated with the use of NPs and the necessary prediction of their toxic effects. It is believed that NPs exposed to biological fluids interact with the proteins and other organic molecules dispersed in the serum and form “protein coronas” of different types, depending on the chemical affinity and binding constants of the components of surrounding biomolecules. The physiochemical properties of the protein corona determine the further fate of the NPs in the organisms, their uptake in the cell through the biomembrane, and thus their potential toxicity. Our study represents a mission-critical point in the understanding of properties of the bio-nano interface. Water molecules are always present in the biological environment and mediate the NPs-blood plasma interactions by forming either one or more structured layers adsorbed on the inorganic materials’ surfaces, as well as hydroxylation pattern on the surface. Understanding of the role of these structured water layers represents a necessary step in the comprehension of any adsorption process involving biological matter retained on either inorganic surfaces or small nanoparticles. Hence, we are investigating how the adsorption properties of biological molecules, like amino acids and small peptides, changes in function of their interactions with inorganic materials like metals and metal oxides. Our particular case study is the TiO2 surface exposed to water, and in presence of small biomolecules. We are planning for extensive use of ab-initio molecular dynamics simulations to collect relevant data for comparison to experimental observations; these data will be also used as the necessary foundation to parametrize interaction potentials for classical molecular dynamics simulations. This will allow us to perform large-scale simulations of many thousand atoms, enabling study of the adsorption mechanism in bigger molecules, like proteins, as well as interactions of coated NPs with biomembranes.

top

Three-dimensional global ideal MHD simulations of vertically stratified turbulent protoplanetary disks with net-vertical field: relative importance of viscous accretion and magnetocentrifugally driven winds

Project Name: Three-dimensional global ideal MHD simulations of vertically stratified turbulent protoplanetary disks with net-vertical field: relative importance of viscous accretion and magnetocentrifugally driven winds
Project leader: Dr. Oliver Gressel
Research field: Astrophysics
Resource awarded: 50000 core hours on MareNostrum
Description

The formation and evolution of a wide range of astrophysical objects is governed by gradual accumulation of mass through a surrounding, rotationally-supported cloud of gas. Interpreting observational signatures emanating from such accretion discs entails accurate modelling of the underlying physical processes. Yet models that go beyond a simple parametrisation have remained elusive for more than four decades. The aim of this project is to devise a new accretion disc model which properly subsumes our current knowledge about instabilities in magnetised rotating shear flows and accounts for the turbulent evolution of both the angular momentum and the embedded magnetic flux, as well as the magneto-centrifugal effect of the large-scale field.

top

 

Type B: Code development and optimization by the applicant (without PRACE support) (10)

Quantum Monte Carlo studies of quantum criticality in metals

Project Name: Quantum Monte Carlo studies of quantum criticality in metals
Project leader: Dr Erez Berg
Research field: Fundamental Physics
Resource awarded: …
Description

Understanding the low-temperature behavior of quantum correlated materials has long been one of the central challenges in condensed matter physics. Such materials exhibit a number of interesting phenomena, such as high-temperature superconductivity, complex phase diagrams, and anomalous non-Fermi liquid behavior. It is clear that the key to many of these properties is the presence of a quantum critical point (QCP) between different phases, which occurs within the metallic state. This calls for a theory of quantum critical points in metals, which, despite years of intense research, are not yet fully understood. In this proposal, we plan to employ numerical Monte Carlo methods to attack this long-standing problem. The key observation is that one can formulate models that capture the essential low-energy physics of many of the relevant QCPs, in such a way that the models are free from the so-called “fermion sign problem” (which often prevents efficient numerical simulations of systems which contain fermions). The availability of controlled (numerically exact) solutions at low temperatures and large system sizes will allow us to benchmark existing field-theoretic predictions, and to gain new insights about the nature of QCPs in metals. In this proposal, we focus on antiferromagnetic and nematic metallic QCPs, relevant to materials like the cuprate and iron-based superconductors, organic superconductors, and some heavy-fermion compounds. References: “The onset of antiferromagnetism in metals: quantum Monte Carlo without the sign problem”, Erez Berg, Max A. Metlitski, Subir Sachdev, Science 338, 1606 (2012); “Competing Orders in a Nearly Antiferromagnetic Metal”, Yoni Schattner, Max H. Gerlach, Simon Trebst, Erez Berg, arXiv:1512.07257.

top

Geometry Dependence of the Diffusion Coefficient in Molecular Dynamics Simulations

Project Name: Geometry Dependence of the Diffusion Coefficient in Molecular Dynamics Simulations
Project leader: Mr. Martin Vögele
Research field: Medicine and Life Sciences
Resource awarded: 100000 core hours on MareNostrum
Description

The function of living cells depends crucially on their membranes, which are highly dynamic systems containing a huge variation of lipids and membrane proteins. Recent advances in experimental methods have opened new possibilities for directly observing diffusion of membrane proteins and even single lipids in crowded membranes. On the computational side, comparable scales of time and size can be reached by coarse-grained molecular dynamics (MD) simulations. Usually, MD simulations are performed in cubic boxes with periodic boundary conditions. In MD simulations of a lipid membrane, the periodic box is usually chosen to be very flat in order to avoid calculation of water particles far away from the membrane. We investigate the dependence of several properties on deviations from the cubic box geometry in MD simulations. In the course of our work on lipid membranes, we found that especially the dynamics is subject to strong finite-size effects. Many lipid membrane simulations have already been performed using periodic boundary conditions. They are all subject to those size effects and might have to be corrected significantly. As our first results correspond to recent theoretical findings, we are now confident to find a quantitative method for determining the real dynamic behavior of a lipid membrane from a series of differently sized periodic simulation boxes and apply this to several important types of lipids and trans-membrane proteins. We then want to transfer the methodology to lipid vesicles in order to investigate common properties and differences between the two types of two-dimensional periodic systems. For this purpose, large-scale simulations of lipid bilayers and vesicles have to be performed and analyzed. The project as a whole is supposed to lead to a better understanding of the transition from three-dimensional to two-dimensional diffusion. It can shed a light on general questions of anisotropic diffusion as well as on the special properties of lipid bilayer dynamics and show a way towards a better comparison of simulations with experiments.

top

Porting and Scaling for Applications of E-CAM Community

Project Name: Porting and Scaling for Applications of E-CAM Community
Project leader: Dr Alan O
Research field: Chemistry and Materials
Resource awarded: 100000 core hours on MareNostrum, 20000 core hours on MareNostrum Hybrid Nodes
Description

E-CAM is an e-infrastructure for software, training and consultancy in simulation and modelling. It is one of eight Centres of Excellence (CoEs) for computing applications within Horizon 2020. Part of the services that E-CAM will provide to it’s user community is a continuous integration infrastructure. A component of this infrastructure will be the use of JUBE as the regression test infrastructure to be used both for verification of correctness and performance at scale. The goal of this project is to take a sample set of user applications and add them to this infrastructure, porting them to PRACE environments and performing scaling tests on PRACE infrastructures. The resulting benchmark meta-data can then be used by the respective community as a reference point in any application development they engage in. This consultancy process will allow the users of the sample set to apply for production PRACE resources at scale. The initial application set will include QuantumEspresso, DL_POLY and Wannier90.

top

From the first stars to the first galaxies with dustyGadget on HPC

Project Name: From the first stars to the first galaxies with dustyGadget on HPC
Project leader: Dr Luca Graziani
Research field: Astrophysics
Resource awarded: 100000 core hours on MareNostrum, 200000 core hours on MareNostrum Hybrid Nodes
Description

The scientific goal of this project is to develop a HPC-Tier-0-enabled code called dustyGadget, extending and improving the physics and parallelization schemes of the well known code Gadget-3. DustyGadget will be able to perform high resolution simulations of the first galaxies, assessing their star formation rates, the nature of their stellar populations, their metallicity and dust content, and their role in cosmic reionization. We aim at developing fully parallel, novel modules of : 1) formation, reprocessing and destruction of dust grains in the interstellar medium. Gadget-3 already implements the life-times for stars of different masses and accounts for the time delay between star formation and the release of energy and metals (by SN and AGB stars) consistently computed for any choice of the stellar Initial Mass Function (IMF); in this step we will follow the production of individual dust species by adding new dust yields to the ones already implemented for the atomic metals, and by using them in parallel OMP tasks. To follow the evolution of dust grains, re-processing in the ISM (i.e. destruction by interstellar shocks and grain growth in dense molecular clouds) needs to be taken into account. This physics will be included in an OpenMP/GPU accelerated module implementing subgrid prescriptions (see de Bennassuti et al. 2014; Mancini et al. 2015, MNRAS) and linking the efficiency of grain destruction and mass accretion to the properties of the star forming SPH particles (Springel & Hernquist 2003, MNRAS). 2) a physical prescription for momentum-driven winds powered by supernovae and radiation pressure from massive stars. The luminosity from massive stars (i.e. their Lyman continuum emission) is likely absorbed by dust particles and then it couples to the gas in a momentun-driven scheme. In a new threaded module we will implement the connection between these winds and the local galaxy properties (i.e. the dust optical depth, the metallicity and the IMF-dependent Lyman continuum luminosity). Each star-forming particle will then “distribute” a fraction of its luminosity to neighboring gas particles so that we can compute the momentum rate deposition when the SN radiative feedback is accounted for. When a cosmological scale is handled and we are forced to a poor mass resolution, we may require to maintain a stochastic selection of wind particles currently implemented in the code, and implement a probability scheme that depends on the mass loading, dm_w/dt = v_w dp/dt, (v_w is free, data-calibrated the initial wind velocity), i.e the selected wind particles will be given a momentum m v_w added to their initial momentum and directed radially outward. A combination of OpenMP ( release 4.0 and experimental 4.5 features) and GPU accelerated routines will be implemented on top of the already existing OMP+MPI Gadget scheme in order to fully exploit the computing power of FAT and hybrid nodes available in current generation of European HPC facilities. The project is part of the ERC funded project FIRST: the first stars and galaxies (PI R. Schneider, web-site: http://www.oa-roma.inaf.it/FIRST/).

top

Coupling the RT code CRASH4 with the AMR code RAMSES

Project Name: Coupling the RT code CRASH4 with the AMR code RAMSES
Project leader: Dr Luca Graziani
Research field: Astrophysics
Resource awarded: 100000 core hours on MareNostrum
Description

This project aims at pairing our version of the AMR code RAMSES with the radiative transfer (RT) code CRASH4, to investigate the impact of RT on the physics of both the interstellar and circumgalactic medium (ISM and CGM) of high-redshift galaxies. To this aim, RAMSES will simulate the galaxy formation proces, while CRASH4 will determine the ionisation and temperature of gas and metals (Graziani et al., 2013 MNRAS; see PRACE Project 2010PA1522 for HPC enabled parallel version). This synergy will be obtained by enabling RAMSES to zoom-in multiple galaxies found in cosmological simulations (see Pallottini et al. 2014, P14 hereafter), and by allowing CRASH4 to post-process the provided AMR levels. RAMSES sub-project punchline (P.I. Pallottini). As first step we will maximize the dynamical range and the efficiency of the hydro simulation by finding an optimal strategy between the multi-mass approach of the zoom-in technique and the number of additional AMR levels (5-10) necessary to resolve both the ISM and CGM of multiple galaxies. After a series of benchmarks, necessary to determine the correct scaling, the scheme will be optimised, also taking advantage of available scaling relations for single zoom-in galaxy (Onorbe et al. 2014). The second step will develop a novel prescription for stellar feedback, necessary when the spatial resolution increases from the cosmological scale — where it is generally treated with few parameters tuned on observations (e.g. P14) — to a ISM scale where more details (e.g. extended chemical networks) should be accounted for. To consistently merge both approaches, the module will require a careful balance of the computational domain because, for example, hydro and chemistry solvers typically have different optimal computational domains. To this aim, we will develop a hybrid OpenMP-MPI chemistry solver based on GRACKLE, the standard chemistry library adopted by the AGORA project (Kim et al. 2014), and we will integrate it with the OpenMP-MPI scheme of RAMSES. CRASH4 sub-project punchline (P.I. L. Graziani). To handle all the N_AMR levels of RAMSES, CRASH4 will first require a new memory management strategy optimised on FAT HPC nodes. The requirements will simply scale with N_AMR. As second step the ray tracing scheme will be enabled to transfer the ionising radiation across the AMR grids, from the refined ISM/CGM, to the coarser cosmological scale. Once radiation is propagated, the ionisation module of CRASH4 will be parallelised in a new scheme repeating the computation on N_AMR parallel grids. The required CPU cores will scale as N_AMR x Np, for Np parallel cores required in a single AMR grid. Np is tuned on the specific grid resolution and gas configuration. Optimisation from PRACE Project 2010PA1522, shows that 512^3 cells generally require 4-8 cores. 4 AMR levels will then require around 16-32 cores/node and can rely on the current OpenMP parallelisation. More AMR levels will require to distribute the N_AMR grids across MPI nodes, with the main communication bottleneck expected during the RT, while gas ionisation and temperature, computed in each node, do not require MPI communications.

top

A QSGW+DMFT method to study strongly correlated materials

Project Name: A QSGW+DMFT method to study strongly correlated materials
Project leader: Prof. Mark van Schilfgaarde
Research field: Fundamental Physics
Resource awarded: 100000 core hours on MareNostrum, 20000 core hours on MareNostum Hybrid Nodes
Description

Last year, a PRACE proposal from our group was accepted with the following details: 2010PA2762 to the PRACE Preparatory Access call Type B, evaluated at the 20th cut-off date. The PRACE resources have been fully exploited to develop, test and optimise our simulation package. As a result of this preliminary development using all the CPU-hours at our disposal, we succeeded in developing a preliminary QSGW+DMFT implementation. A full QSGW+DMFT calculation on Ni was conducted using the resources allocated. The results are currently object of a paper in preparation. The aim of this current proposal is to further optimise and test the code, and upgrade its applicability. The code merges self-consistently a quasi-particle self-consistent GW (QSGW) package with a Dynamical Mean Field Theory (DMFT) program. The QSGW code is an all-electron full-potential implementation composed of two connected parts (a DFT and a GW package), developed by Mark van Schilfgaarde and T. Kotani. The DMFT is implemented in a Continuous Time Quantum MonteCarlo (CTQMC) solver by Kristjan Haule (Rutgers University). More details on the programs can be found in our previous proposal. The completion and optimisation of the procedure of merging these two codes has been the primary task of our previously awarded project. Most of the CPU hours were spent on the CTQMC impurity solver (the most expensive part of the computation). Preliminary simulations on simple materials like Ni and La2CuO4 have been successfully carried out. Our task now is (i) to implement more advanced features to treat several impurities in order to move towards the investigation of composite materials and (ii) to achieve full self-consistency. In more in detail: (i) requires that multiple groups of correlated sub-blocks be addressed, e.g. an antiferromagnetic reference point. (ii) The calculations run so far can be considered as 1-shot QSGW+DMFT simulations. Thus the DMFT part is treated as a perturbative correction to the underlying QSGW electronic structure. Our next aim is to run a full self-consistent loop where the results of the DMFT part are fed back to the QSGW part until convergence. In order to accomplish this, analytic continuation methods (like maximum entropy or stochastic optimization) will be required. Depending on the method used, this step can be computationally demanding. Some details on the codes: DMFT and QSGW are computationally intensive for different reasons. DMFT is solved with a Month Carlo algorithm on a local subspace, and uses large amounts of CPU cycles, which scale exponentially with the number of orbitals in a correlated subspace but is highly parallelised . QSGW treats all the N electrons. Its CPU time scales as N^4 and memory as N^3. Thus for small systems the QMC algorithm dominates while for larger ones QSGW does, provided that the correlated spaces are kept independent. (LDA time and memory scale as N^3 and N^2, and a crossover in LDA+DMFT similarly occurs, at a larger N).

top

Creo Dynamics SHAPE Project: Large scale aero-acoustics applications using open source CFD

Project Name: Creo Dynamics SHAPE Project: Large scale aero-acoustics applications using open source CFD
Project leader: Mr Torbjörn Larsson
Research field: Engineering and Energy
Resource awarded: 100000 core hours on MareNostrum
Description

The project aims to demonstrate how fluid flow simulation processes based entirely on open source CFD software can be tailored and deployed in parallel at large scale to produce robust and efficient results for real life applications. The chosen test case should be of high industrial relevance; e.g. the prediction of the unsteady aerodynamics flow field around a road vehicle. The basic idea is to derive a (semi) automated and fully scripted simulation process starting from prepared CAD. All fundamental steps in the process should be executed in batch on distributed cluster nodes (Linux) in parallel. Primary focus will be on improving parallel meshing performance. Results from the project should be made public to showcase the technology and encourage new users to adopt and invest in high performance computing (HPC) and open source software.

top

Mars project from PRACE SHAPE SME ACOBIOM

Project Name: Mars project from PRACE SHAPE SME ACOBIOM
Project leader: Dr Bertrand Cirou
Research field: Medicine and Life Sciences
Resource awarded: 100000 core hours on SuperMUC
Description

RNA-Seq approach is used in a wide variety of applications. These include identifying disease-related genes, analysing the effects of drugs on tissues, and providing insight into disease pathways. The RNA-Seq is widely used to characterise gene expression patterns associated with tumor formation. Since RNA-Seq provides absolute values and does not require any calibration with arbitrary standards, results can be compared at any time with other data, even raised by independent laboratories. Once collected, these data can be digitalised and then easily and reliably compared in silico with the growing library of RNA-Seq databases generated for normal and pathological situations in other laboratories around the world (Human: ~27000 libraries and Mouse: ~42000 libraries. Average size of a library: 1.7GB. Total size: 120TB). ACOBIOM SME intend to compare and search the specificity of our blood gene expression signatures (eg. the masitinib in pancreas) against already existing data in different tissues or pathologies, in Human and Mouse Omic data. ACOBIOM possesses a bioinformatic and biostatistical platform capable of processing and analyzing millions of pieces of data generated by high-throughput sequencing. The company leverages over 16 years acquiring expertise examining and interpreting this type of data.

top

SHAPE Project Airinnova: High level optimization in aerodynamic design

Project Name: SHAPE Project Airinnova: High level optimization in aerodynamic design
Project leader: Dr Mengmeng Zhang
Research field: Engineering and Energy
Resource awarded: 100000 core hours on MareNostrum
Description

The company is developing computational solutions for aerodynamic shape optimization, which is an important task in aircraft design. The goal is to design a lighter, greener, and quieter airplane by reducing drag especially in high speed. Aerodynamic shape optimization for reduced drag requires a large number of CFD solutions, and computational power is a limiting factor. Current ideas for surrogate modelling are being developed to improve computational efficiency. The company has been discussing the project with SNIC-KTH and the work plan was developed by Airinnova and SNIC-KTH together. For the corresponding SHAPE project will focus on carrying out the high-fidelity (RANS) aerodynamic shape optimization for a reference aircraft model, based on the gradient-based optimization algorithms, using different CFD solvers and methods. The tasks mainly consist of: 1. benchmark: performance analysis for Common Research Model (CRM) model using proposed CFD solvers 2. Port: deploy and run on a PRACE Tier-0 machine

top

HPC methodologies for PharmScreen

Project Name: HPC methodologies for PharmScreen
Project leader: Mr. Thomas Ponweiserg
Research field: Medicine and Life Sciences
Resource awarded: 100000 core hours on MareNostrum
Description

Pharmacelera is a Spanish company that develops hardware and software solutions for the early stages of drug discovery (Computer-Aided Drug Design). The company has developed PharmScreen, a revolutionary piece of software for ligand-based drug design that models molecules entirely in 3D and that is able to find molecules with larger probabilities to become a drug than those found by tools from the competition. Recently, the company partnered in a SHAPE project within the PRACE program with RISC Software GmbH, an Austrian company dedicated to High Performance Computing. The two main goals of this SHAPE project are: (i) to speed up the execution of PharmScreen by using GPUs and by refining the current OpenMP implementation, and (ii) to assess the potential of PharmScreen when several parameters and datasets inputs are modified (sensitivity study). This sensitivity analysis implies changing some key parameters that have a direct impact on computational time (such as the grid spacing in the 3D representation of molecules and the time spent in the Montecarlo search pass) and the parties do not have the HPC capabilities for doing so. Hence, both parties have agreed to fill a Preparatory Access Type B request.

top

Type C: Code development with support from experts from PRACE (2)

Parallel mesh partitioning using space filling curves

Project Name: Parallel mesh partitioning using space filling curves
Project leader: Dr Guillaume Houzeaux
Research field: Engineering and Energy
Resource awarded: 100000 core hours on MareNostrum
Description

The project aims at implementing in Alya, a high performance computational mechanics code, a parallel mesh partitioner based on space filling curves. Traditional mesh partitioning (METIS, SCOTCH) is generally a sequential computational kernel. Although semi-parallel versions exist (PARMETIS, PT-SCOTCH), their performances in terms of load balancing and parallel efficiency remain low. In addition, one has to use a very reduced range of MPI tasks to perform the partitioning in order to achieve a reasonable load balance. This poses therefore implementation and memory issues. All these drawbacks make the runtime re-load balancing (e.g required to an mesh adaptation process) a bottleneck in the simulation process. Space filling curves can be used as a fully parallel mesh partitioner, and can thus be executed on the same number of MPI tasks as the one selected at the beginning of the run, giving the same partitions as its sequential counterpart. This makes the load balancing a fully parallel task that can be used at runtime. The proposed project aims at implementing such a strategy in Alya.

top

Parallel curved mesh subdivision for flow simulation on curved topographies

Project Name: Parallel curved mesh subdivision for flow simulation on curved topographies
Project leader: Dr Xevi Roca
Research field: Engineering and Energy
Resource awarded: 100000 core hours on MareNostrum
Description

In the near future, we will use the prodigious potential offered by the ever-growing computing infrastructure to foster and accelerate the European transition to a reliable and low carbon energy supply. We are fully committed to the former goal by establishing an Energy Oriented Centre of Excellence for computing applications, (EoCoE), through the on-going contract H2020-EINFRA-2015-1. EoCoE aims to assist the energy transition via targeted support to four renewable energy pillars: Meteo, Materials, Water and Fusion, each with a heavy reliance on numerical modelling. The primary goal of EoCoE is to create a new, long lasting and sustainable community around computational energy science. To this end, we are resolving the current bottlenecks in application codes, leading to new modelling capabilities and scientific advances among the four user communities. Furthermore, we are developing cutting-edge mathematical and numerical methods, and tools to foster the usage of Exascale computing. For the EoCoE project, we are really interested in improving our turbulent flow simulation code for complex topographies. Nowadays, this code is used with success by the renewable energy company IBERDROLA to improve the production of their wind farm designs. However, we have detected that in very large wind farm simulation analysis, the generation of a mesh that preserves the inherent curvature of the topography can be the principal bottleneck of the whole process. To address this issue, we propose to implement a parallel uniform curved mesh subdivision method, in a HPC code developed at Barcelona Supercomputing Center (BSC) named Alya. The resulting meshes will preserve the inherent curvature of the mesh boundaries in the successive refinements. The advantage of the proposed strategy is that the subdivision is performed and stored in parallel and therefore, there are no memory constraints. For instance, if we start with a coarse mesh composed by 30M tetrahedra, after two consecutive subdivisions we obtain a finer distributed mesh composed by 30M x 8 x 8 = 1920M elements, few seconds later and completely stored in the memory. Furthermore, the finer curved mesh preserves the curvature information described by the first curved mesh. Finally, if the original numbering is conserved, then the post-processing can be performed on the coarse curved mesh.

top

Share: Share on LinkedInTweet about this on TwitterShare on FacebookShare on Google+Email this to someone