2nd Project Acccess Call – Awarded Projects

Ab Initio Modeling of Solar Active Regions

Project leader: Aake Nordlund, University of Copenhagen, Denmark

Collaborators: Mrs Gisela Baumann, University of Copenhagen Copenhagen, Denmark / Dr Remo Collet, Max Planck Institute for Astrophysics Garching (Munich), Germany / Dr Damian Fabbian, Instituto de Astrofísica de Canarias La Laguna (Tenerife), Spain / Dr Klaus Galsgaard, University of Copenhagen Copenhagen, Denmark / Dr Troels Haugboelle, University of Copenhagen Copenhagen, Denmark / Dr David MacTaggart, University of Copenhagen, Denmark / Prof. Fernando Moreno-Insertis, Instituto de Astrofísica de Canarias La Laguna (Tenerife), Spain / Dr. Roald Schnerr, Royal Swedish Academy of Sciences Stockholm, Sweden / Dr Jacob Trier Frederiksen, University of Copenhagen, Denmark

The Sun is a fascinating astrophysical object, given its proximity and relevance to us, and the intricate physics of its readily observable surface layers, which provide a unique and comprehensive test bench for non-thermal astro- and plasma physics. Studies of the Sun’s atmosphere and heliosphere thus help us understand basic physical processes, and are also of direct importance for understanding the environment in which the Earth moves and the perturbations to which its magnetosphere is subjected (space weather).The overarching scientific aim of this research is to understand the dynamics of solar active regions – sunspots and their neighborhoods – and their interaction with the sub-surface solar convection zone and the overlying solar corona, on scales that range from less than 10 km to about 50,000 km, using both 3-D MHD simulations and 3-D relativistic charged particle (particle-in-cell code) simulations. These simulations will be coupled in an unprecedented way, such as to bridge more than a factor of 5.000 in scales of structure and dynamics of Solar active regions.Right now Europe has a unique opportunity for progress in this area, thanks to access to PetaFlop computing and a new generation of computational tools. For the first time ever, this will allow ab initio modeling of non-thermal processes in a complex and very well observed astrophysical setting; one where a unique combination of spatial, temporal, and wavelength resolution is available from observations. The results from the proposed project will be compared directly with observations from the current space borne observatories SDO (Solar Dynamics Observatory) and RHESSI (Reuven Ramaty High Energy Solar Spectroscopic Imager). Predictions can also be made for the future IRIS (Interface Region Imaging Spectrograph) satellite observatory. European scientists have a heavy involvement in all of these three satellite observatories.The project uses mainly two well-proven MPI-codes that parallelize well to tens of thousands of cores; a staggered mesh magneto-hydrodynamics code (the Copenhagen Stagger Code), and the PhotonPlasma Code — a relativistic particle-in-cell (PIC) code with modular provisions for particle-particle interactions (Coulomb collisions, Bremsstrahlung, Compton scattering, etc.), which also has built-in facilities for on-line computation of non-thermal radiation diagnostics.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 60 000 000 core-hours

Discrete logarithm on a 160-bit elliptic curve over F(p^6)

Project leader: Antoine Joux, University of Versailles St-Quentin-en-Yvelines PRISM, France

Collaborators: Mrs Vanessa Vitse, University of Versailles St-Quentin-en-Yvelines Versailles CEDEX, France

The goal of this proposal is to scale a recent algorithm for computing discrete logarithms in certain elliptic curves to groups of 160-bit size. This choice of size comes from the fact that the usual recommendation for cryptographic applications is to use 160-bit elliptic curves.

Computer system: CURIE, GENCI/CEA
Resource awarded: 1 900 000 core-hours

REFIT – Rotation effects on flow instabilities and turbulence

Project leader: Arne Johansson, KTH Department of Mechanics, Sweden

Collaborators: Dr. Geert Brethouwer, KTH Stockholm, Sweden / Prof. Dan Henningson, KTH Stockholm, Sweden/ Prof. Rebecca Lingwood, University of Cambridge, United Kingdom / Prof. Martin Oberlack, Technische Universität Darmstadt, Germany / Dr. Philipp Schlatter, KTH Stockholm, Sweden

Flows in gas turbines, turbo machinery, pumps, compressors, cyclone separators and other industrial apparatus are often rotating or swirling. They are also usually turbulent since flow rates and thus Reynolds numbers are generally large, meaning that the fluid motions fluctuate in a chaotic and irregular manner in space and time. The induced Coriolis force on the fluid or gas, also occurring when there is a flow over wings, turbine blades and other curved surfaces, causes many intriguing and complex physical phenomena. Coriolis forces, for example, can damp as well as enhance the turbulent fluctuations and influence the mean flow rate. Capturing such effects in engineering turbulence models has so far proved to be elusive and in order to improve and validate those models high quality data of rotating turbulent flows are badly needed.Experiments on rotating flows are inherently complicated since it usually requires turning of equipment. A viable and potentially very accurate alternative is direct numerical simulation (DNS) whereby the whole spatial and temporal range of turbulent scales are resolved without invoking models. Rotating channel flow is particularly relevant from a fundamental and engineering perspective. Recent DNSs in our group have revealed interesting phenomena in channel flows at high rotation rates; turbulence is then damped near both channel walls and the flow can become partly or completely laminar leading to huge flow rate changes at a constant pressure drop. Moreover, preliminary DNS uncovered an instability not observed previously in rotating wall-bounded flows. This instability caused large fluctuations in the turbulence intensity and wall shear stresses in a periodic-like manner.Although previous DNSs of rotating channel flow have produced invaluable information, they were restricted to low Reynolds numbers, Re, since the range of scales that needs to be resolved and thus the computational costs of DNS increase dramatically with Re. The results of those previous DNS cannot simply be extrapolated to industrial flows with a commonly much higher Reynolds number. However, with the resources provided by the PRACE project we are able to perform simulations of rotating turbulent flows at a much higher Reynolds number. Proper simulations of the periodic-like instabilities at high rotation rates and high Re will require especially massive computational resources.The aim of the proposed project is therefore to perform DNS of rotating turbulent channel flow at an order of magnitude higher friction Reynolds number than previously performed DNS. In particular, the goal is to carry out the first w
ell-resolved DNS of the periodic-like instabilities occuring at high rotation rates since they can potentially have an important impact in industrial applications. Those new large-scale DNS can help to address unresolved questions about rotation, swirl and streamline curvature effects in industrial flows.The computed high Reynolds number DNS data are also vital in order to develop and validate engineering models for turbulent flows with rotation, swirl or streamline curvature in industrial applications and to study instabilities in rotating flows. The DNS data will therefore be made available to the wider scientific community.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 46 000 000 core-hours

SOULAC (Simulation Of Ultra-intense Laser ACceleration of ions)

Project leader: Erik Lefebvre, CEA, DAM, DIF DPTA, France

Collaborators: Dr Christophe Cornet, CEA, DAM, DIF, France / Mr Vincent Floquet, CEA, IRAMIS, France / Dr Mickael Grech, Max Planck Institute, Germany / Dr Laurent Gremillet, CEA, DAM, DIF, France / Dr Rachel Nuter, CEA, DAM, DIF, France / Mr Jeremie Rolle, CEA, DAM, DIF, France / Dr Gonzalo Sanchez-Arriaga CEA, DAM, DIF, France

Generation of high-energy ion beams by irradiation of a plasma with an intense laser pulse is one of today’s hot topics in laser-plasma interaction. These sources could prove cheaper and more flexible than conventional accelerators. Laser-accelerated ion beams have indeed particular properties which make them very interesting for a wide range of applications such as proton radiography, fast ignition or hadrontherapy. The goal of this proposal is to advance our understanding of laser-ion acceleration though intensive numerical simulations using a state-of-the-art tool.Laser-driven proton-therapy would greatly benefit from this project. Our team is indeed involved, due to its expertise in laser-plasma modeling, in the SAPHIR consortium, an academic-industrial partnership aiming at demonstrating the feasibility of protontherapy with laser-accelerated proton beams. A strong societal objective of the current proposal is therefore to support, with improved numerical simulations, the analysis of the ongoing proton acceleration experiments. In these experiments, acceleration is due to the Target Normal Sheath Acceleration (TNSA) mechanism, in which ions gain energy from the laser-heated electron plasma expansion. Even if TNSA has been well studied, its scaling laws at high laser-intensity are still debated. Therefore, another important output of our project is the possibility to span a large number of plasma and laser parameters with our numerical simulations, in order to exhibit scaling laws that will help optimize proton acceleration.In addition to the potential high-impact application of protontherapy, laser ion acceleration is also associated to a number of other scientific challenges. In this project, we chose to focus on three of them: laser-ion acceleration from sub-critical (low-density) targets, acceleration of highly charged ions in addition to protons, and a new acceleration scheme using ultra-thin targets. In the first topic (sub-critical targets), we will carry out large 3D simulations to better understand the acceleration mechanisms and scaling laws, as recent results showed that 2D simulations fall short of providing quantitative agreement with experiments in this domain. For the second topic (highly charged ions), we will use an original field ionization model recently added to our code to self-consistently describe heavy ion ionization and acceleration. For the third topic (new acceleration mechanisms), we will focus on a variety of radiation-pressure dominated mechanism known as LinPA. This scheme amounts to irradiating a thin foil with a circularly polarized laser pulse in such way that the radiative pressure is strong enough to accelerate the electron population as a whole. Ions are then expected to be accelerated up to several GeV thanks to the electrostatic charge separation, while keeping small energy dispersion if the laser and target parameters are correctly chosen. This project will provide the 3D simulations needed for a better and more realistic understanding of this acceleration process.The four topics covered in this proposal are related to existing collaboration of our group with various French (LOA, IRAMIS, Amplitude Technologie), German (TUDarmstadt, MPIPKS) or British (QUB) teams, ensuring that this project will also benefit to several research groups and companies across Europe.

Computer system: CURIE, GENCI/CEA
Resource awarded: 7 500 000 core-hours

Ion-switched biomolecular recognition as an assembly tool for nanotechnology

Project leader: Fabrizio Cleri, University of Lille Institute of Electronics Microelectronics and Nanotechnology, France

Collaborators: Dr Ralf Blossey, Centre National de la Recherche Scientifique (CNRS), France / Dr Mauro Boero, Centre National de la Recherche Scientifique (CNRS), France

The intrinsic recognition properties of biomolecules are at the focus of many innovative nanotechnologies. Such molecules as ligand–receptor pairs, complementary DNA or RNA strands, glycoconjugate species, display the two basic features of selectivity and adhesivity, which are the necessary prerequisites to drive the automatic self-assembly of building blocks. Indeed, attaching complementary pairs of molecules to engineered nanostructures, can be a powerful tool to selectively assemble large quantities of matter, in the same spirit of a living organism assembling millions of cells starting from much simpler molecular units. This concept may be key to bypassing the scale-up bottleneck common to most nanotechnologies, when moving from the laboratory scale to real-world applications. One important feature of self-assembling molecular species is the need for a switching mechanism. Ionicity or pH changes probably represent one of the best compromises, providing easy activation, quick switching from/to on/off states, and sharp dependence on the control parameter.In our joint Teams we have been recently focusing on two classes of problems, which represent ideal examples of ion-switched biomolecular recognition and self-assembly: (1) carbohydrate-carbohydrate interactions, and (2) non-Watson-Crick base pairing . These problems share many interesting features. Firstly, both are switched by ionicity changes, are highly selective with structurally simple motifs, and provide stable, reversible bonding. Secondly, both classes of problems have a great relevance in cell biology, while still lacking explanations about their microscopic mechanisms. Finally, both are of the greatest interest in nanotechnology: homophilic carbohydrate interactions can be exploited as selective, powerful surface adhesives for specific self-assembly of nanoscale objects; non-W-C base pairs lead to stable, extended nanowire structures, known as “i-motifs”.Within this framework, we explore the microscopic dynamics of carbohydrate-carbohydrate interactions, with fully quantum mechanical ab initio molecular dynamics, by modelling the role in adhesion of the Lewis-X (LeX) epitope, dependence of its adhesivity on Ca2+ concentration, the relative chemical affinity of its various conformations, and so on. We start from X-ray crystallographic data, to gue
ss probable “in vivo” interacting configurations. Then we will study the dynamics of LeX-pairs in ionized water. The sheer size of the problem is daunting, since sugars have a huge number of degrees of freedom, and their dynamics is extremely slow. Complete characterization of the interaction dynamics, and of the switching mechanisms, entails calculations which can only be carried out on the most advanced supercomputers.Concerning non-Watson-Crick interactions, we begin with the study of the intercalation of poly-C DNA fragments, by means of empirical MD. However, the key mechanisms of proton-mediated base-pairing are still unknown. For a deeper understanding of this matter, which could also serve to a better comprehension of the biological functions of non-Watson-Crick DNA and RNA assembly, we will characterize the protonation interactions of poly-C strands in physiological water by ab initio MD. We will initially use standard motifs for intercalation, such as the TGT insulin spacer. In a further step of the Project, we will also use DNA aptamers, to compare to experiments carried out by a partner laboratory in Tokyo.

Computer system: CURIE, GENCI/CEA
Resource awarded: 3 100 000 core-hours

MHD turbulence in the Interstellar Medium: Linking Star Formation and Galaxy Dynamics

Project leader: Frederic Bournaud, CEA Saclay DSM/IRFU, France

Collaborators: Prof Joao Alves, University of Vienna, Austria / Mr Andreas Bleuler, University of Zurich, Switzerland / Dr Frederic Bournaud, CEA, France / Mr Damien Chapon, CEA, France / Dr Francoise Combes, Observatoire de Paris, France / Dr Emanuele Daddi, CEA, France / Prof Avishai Dekel, The Hebrew University of Jerusalem, Israel / Dr Yohan Dubois, Oxford University, United Kingdom / Dr Bruce Elmegreen, IBM Research, United States / Dr Eric Emsellem, ESO European Southern Observatory, Germany / Dr Sebastien Fromang, CEA, France / Dr Patrick Hennebelle, Ecole Normale Superieure Paris, France / Dr Marie Martig Swinburne, University Hawthorn, Australia / Dr Chiara Mastropietro, CEA, France / Prof Ben Moore, University of Zurich, Switzerland / Dr Leila Powell, CEA, France / Dr Florent Renaud, CEA, France / Prof Dr Romain Teyssier, CEA , France / Prof Dr Romain Teyssier, University of Zurich, Switzerland / Dr Axel Weiss, Max Planck Institut fur Radioastronomie Bonn, Germany

Understanding how Milky Way-like spiral galaxies formed their stars is a major challenge in modern cosmology. Star formation proceeds through small-scale instabilities, but can be ignited only if gas accretion onto galaxies, cooling, gravitational dynamics, supersonic turbulence in the interstellar medium and/or the magnetic field act in concert to form dense and cold clouds of molecular gas. The combination of these fundamental processes has never been explored self-consistently, hence how star formation is actually triggered from individual stars on small scales to entire galaxies cannot be understood yet. Conversely, it is known that star formation has a low efficiency because it is regulated by some mechanism, and identifying how this regulation occurs could solve fundamental issues in galaxy formation. However, whether the main factor is turbulent pressure, the magnetic field, or so-called feedback processes such as supernovae explosions and gas heating by new massive stars, remains unknown because of the lack of self-consistent models.Using the Adaptive Mesh Refinement (AMR) code RAMSES, our group has recently performed the highest resolution simulation of gas dynamics and star formation including all important scales from global galactic dynamics to dense substructures in cold gas clouds, which are the only regions where star formation actually proceeds. This model is at the forefront of the field to model and analyze the physics of star formation in entire galaxies, however it was so far achievable only for a dwarf galaxy much smaller than the Milky Way, and without the magnetic field.We propose here to simulate interstellar gas dynamics and star formation over an entire Milky Way-like galaxy, at an unprecedented resolution of 0.6pc (about two light years) and with a “full physics” approach including gravity, hydrodynamics, cooling and heating processes, star formation and feedback, and MHD. The role of various processes in forming dense gas clouds, triggering star formation therein, and regulating the efficiency of star formation, will be analyzed for the first time in a self-consistent model of a typical spiral galaxy. The results will strongly improve our understanding of the formation of galaxies and their star formation history, and provide fundamental constraints for the next generation of cosmological models. They will also indicate what the initial physics conditions from which the actual formation of individual stars proceeds. Beyond the initial analysis led by team members, the data will be made publicly available within a year, so as to further the scientific outcome of the project.A set of three simulations will be performed on Curie, using 4096 to 8192 cores, for a total of 9 million hours. While our “full physics” approach has large memory requirements that makes smaller clusters or Blue Gene-type facilities unsuitable for the project, the new OpenMP+MPI of the RAMSES code will make optimal use of Curie’s fat nodes with quasi-ideal scaling performances. In addition to addressing central issues in galaxy evolution and star formation, the project will demonstrate the optimization of a major code in the astrophysics community on new Tier-0 facilities.

Computer system: CURIE, GENCI/CEA
Resource awarded: 9 000 000 core-hours

Large-Eddy Simulation of high-frequency instabilities under transcritical conditions

Project leader: Gabriel Staffelbach, CERFACS CFD, France

Collaborators: Prof Sebastian Candel, EM2C, France / Dr Benedicte Cuenot, CERFACS, France / Mrs Layal Hakim, EM2C, France / Dr Boileau Matthieu, EM2C, France / Dr Thierry Poinsot, IMFT, France / Mr Anthony Ruiz, CERFACS, France / Dr Thomas Schmitt, EM2C, France

Turbulent combustion research is a critical field with important theoretical and modeling needs and with broad implications for industry and society. In this vast topic, the participants propose to use a PRACE award to improve the current understanding of high-pressure transcritical flames when such flames are submitted to transverse high-frequency acoustic modes. The physics of these flames is extremely complex: the pressure is high and above the critical value, chemical kinetics interact with turbulence, the thermodynamics range is transcritical inducing large density gradients in the flow, acoustics couple with combustion. Moreover, mastering these flames is of considerable technological importance for applications in aerospace propulsion, especially in Europe where hydrogen/oxygen engine technology is crucial for the Ariane program. The study will focus on modeling, calculation and validation of such flames/acoustic interactions: An experiment carried out by the EM2C laboratory will be calculated using the Large-Eddy Simulation (LES) solver AVBP from CERFACS. The base-line version of AVBP is one the most advanced LES solvers worldwide and it has been used successfully on all existing supercomputers. It has been recently extended to ac
count for real gas thermodynamics which determine the working fluid state in the range of conditions required for the present work. Extensive combustion models developed by both parties allow for the computation of flame/acoustic interactions.This demonstration will advance the state of the art in combustion simulation and its application to the major problem of high-frequency instabilities.

Computer system: CURIE, GENCI/CEA
Resource awarded: 8 500 000 core-hours

Large Scale simulations of Ly-alpha and Ly-break galaxies in the high-z universe: Probing the epoch of reionization.

Project leader: Gustavo Yepes, Universidad Autonoma de Madrid, Spain

Collaborators: Dr. Daniel Ceverino, Hebrew University of Jerusalem, Israel / Dr. Jaime Forero-Romero, Astrophysikalisches Institut Potsdam, Germany / Dr. Tobias Goerdt, Universidad Autonoma de Madrid, Spain / Dr. Stefan Gottloeber, Astrophysikalisches Institut Potsdam, Germany / Dr. Alexander Knebe, Universidad Autonoma de Madrid, Spain / Dr Steffen Knollmann, Universidad Autonoma de Madrid, Spain / Dr. Francisco Prada, Consejo Superior de Investigaciones Cientificas Granada, Spain

The formation of galaxies in an expanding universe is a complicated process that can only be studied properly using numerical simulations. In recent years not only the computational capacities have increased tremendously, also the simulation codes have become more and more sophisticated, allowing to follow the physical processes involved in the formation of galaxies. It has become possible to simulate the assembly of single galaxies in great detail, albeit key processes (like star formation) are still not solved from first principle and need to be approximated with well-tested and adjusted models.Simulating a whole ensemble of galaxies to subject them to rigorous testing with the wealth of observational data available for galaxies in the local Universe is still beyond the reach of computational cosmology. But meanwhile observations at higher redshifts (i.e. at earlier times) are becoming better and better and this allows to put constraints on the properties of those first galaxies which have been formed already a billion years after the big bang. Understanding the first stages of galaxy formation is the clue to the understanding of present day galaxies.While still a major computational challenge, focusing on the early Universe allows us to perform simulations encompassing a statistical significant sample of galaxies which can be directly linked with current observations. This not only provides a check for the model assumptions we have to make but also helps understanding and interpreting the present and future observations of the early galaxies, most of them detected from the light of Lyman-alpha photons that are thought to be caused by an ongoing outburst of star-formation.These systems are of key importance for cosmology because they trace the dark matter halos and, therefore, the evolution of the matter distribution in the universe. It is therefore very important to estimate their abundance as a function of cosmic time for the current cosmological model.In this project we plan to follow the formation and evolution of a volume limited sample of dark matter halos which can host Lyman alpha emitting galaxies up to redshift 3. Due to the extreme resolution needed to resolve the proper physics of the gas cooling and star formation processes ( of the order of 10s of parsecs), it is still not feasible to simulate a whole cosmological volume of hundred of Megaparsecs down to this resolution. On the other hand, thanks to the recent developments in generation of initial conditions, we can select certain objects formed in a low-resolution cosmological simulation, and resimulate them with very high resolution, by using a zooming technique that resamples the fluids with particles of variable masses within the area from which the object is formed.We plan to simulate a cubic volume of 160/h Mpc on a side and generate the largest cosmological initial conditions ever attempted: 16384^3 particles. We will find the interested objects by running a low-resolution (1024^3) version of this initial conditions. All the objects more massive than a Milky-Way type galaxy there ( 10^12 Msun) will be selected for resimulation at different resolutions, down to the highest resolution available of 16384^3. We estimate to find a total of 2000 objects at z=3. All these objects will then be split in samples of different mass cuts that will uniformly covering the whole box and will be resimulated simultaneously. The database of initial conditions for these objects at different resolutions will be made publicly available to the astronomical community as one of the most important deliverables of this project .In addition, a full box dark matter only simulation with 4096^3 particles will be done also. This simulation will serve as the tracer for mass accretion onto high-z halos. Analytical estimates of the Mass-to-Light ratio from the results of the resimulated objects will be used to derive the Ly-alpha emission from the dark halos and to make statistics of Ly-alpha galaxies and cold accretion flows from the whole volume.

Computer system: CURIE, GENCI/CEA
Resource awarded: 5 000 000 core-hours

Probing the Limits of the Standard Model with Lattice Simulations

Project leader: Hartmut Wittig, University of Mainz, Germany

Collaborators: Mr Bastian Brandt, University of Mainz, Germany / Dr John Bulava, DESY Zeuthen, Germany / Dr Michele Della Morte, University of Mainz, Germany /Dr Michael Donnellan, DESY Zeuthen, Germany / Dr Patrick Fritzsch, Humboldt University of Berlin, Germany / Dr Jochen Heitger, University of Muenster, Germany / Mr Benjamin Jaeger, University of Mainz, Germany / Prof Francesco Knechtli, University of Wuppertal, Germany / Dr Bjoern Leder University of Wuppertal, Germany /Mrs Marina Marinkovic, Humboldt University of Berlin, Germany / Prof Harvey Meyer, University of Mainz, Germany / Dr Stefan Schaefer, CERN, Switzerland / Dr Hubert Simma, DESY Zeuthen, Germany / Dr Rainer Sommer, DESY Zeuthen, Germany /Mr Francesco Virotta, DESY Zeuthen, Germany/ Prof Ulli Wolff, Humboldt University of Berlin, Germany / Dr Georg von Hippel, University of Mainz, Germany

One of the major activities in particle physics worldwide is the search for physics beyond the well-established Standard Model (SM). The main purpose of the Large Hadron Collider (LHC) is to unravel the mechanism of electroweak symmetry breaking and to perform direct searches for “new physics”, i.e. particles and phenomena that cannot be accommodated by the SM. Another strategy is to firmly establish devieations between experimental measurements and SM predictions. Here, the mostly widely followed approaches include studying the physics of heavy flavours, with the aim of overconstraining the unitarity relations among the elements of the quark mixing matrix matrix, using a combination of precise experimental and theoretical input. Furthermore, high-precision measurements of quantities such as the the anomalous magnetic moment of the muon, must be confronted with equally precise theoretical estimates.The success or failure of indirect searches for new physics depends crucially on whether all theoretical uncertainties can be brought under co
ntrol. In particular, the effects of the strong interactions must be quantified reliably. Lattice simulations of Quantum Chromodynamics (QCD) provide the framwork for a systematic treatment of the strong interactions at typical hadronic scales. In order to have a significant impact, systematic effects in lattice calculations must be controlled at the level of a few percent.Within our project we make a decisive step towards the goal of providing input for heavy flavour phenomenology with unprecedented accuracy, as well as performing a reliable ab initio determination of the leading hadronic hadronic vacuum polarisation to the muon’s anomalous magnetic moment. The latter is currently the main limiting factor in theoretical predictions of this important quantity. Another main goal of our project is a precision determination of the leptonic decay constants of B-mesons and the mass of the b-quark. Also, we will provide accurate lattice data for form factors which describe semi-leptonic decays of B-mesons.Currently, one observes differences at the level of three standard deviations between theoretical and experimental determinations of the muon’s anomalous magentic moment. A tension of this magnitude is also observed in phenomenological analyses of the leptonic and semi-leptonic B-meson decays rates. Our calculations will help in deciding the important question whether or not these deviations are genuine and constitute hints of physics beyond the SM.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 55 000 000 core-hours

Diversity of Type Ia supernovae from initial conditions of the exploding white dwarf star

Project leader: Ivo Seitenzahl, Max Planck Gesellschaft (MPG), Germany

Collaborators: Mr. Franco Ciaraldi-Schoolmann, Max Planck Gesellschaft (MPG), Germany / Dr Markus Kromer, Max Planck Gesellschaft (MPG), Germany / Prof. Friedrich Röpke, Universität Würzburg, Germany

Type Ia supernovae (SNe Ia) are among the brightest explosions in the Universe. Moreover, they are quite uniform in their properties and thus from their apparent brightnesses cosmic distances can be inferred. As this allows to determine the geometry of the Universe, SNe Ia are one of the most important tools in observational cosmology. Distance determinations to SNe Ia showed that the Universe is expanding at an accelerated rate. The reason for this effect is unclear and has been parametrized as “Dark Energy” forming the main constituent of the Universe today. SN Ia distance determination can contribute to a better understanding of this mysterious energy from. The necessary distance measurements, however, require high precision and great observational effort is currently spent in this field. Therefore it is timely to match these efforts with a sound theoretical understanding of the supernova explosions.The leading scenario of SNe Ia is that a white dwarf star consisting of carbon and oxygen undergoes a thermonuclear explosion when it reaches the limit of its stability — the Chandrasekhar mass, about 1.4 solar masses. This explosion turns the material of the white dwarf star into heavier elements, predominantly nickel-56, which by radioactive decay powers the bright optical display we observe. The details of the explosion physics and the formation of the astronomical observables, however, are complex and can only be studied in sophisticated numerical simulations. The thermonuclear burning ignites near the center of the star and propagates outward as a thin reaction front. This front propagates initially with subsonic velocities (then called a deflagration) but may eventually turn into a supersonic detonation. Such delayed detonations have been shown to reproduce SNe Ia well in preliminary two-dimensional simulations. Therefore they are the leading candidate for “normal” SNe Ia. As such, they should account for the variations observed in “normal” SNe Ia, which have to be calibrated out of the supernova sample for precision applications to cosmology. In our PRACE project we will explore this diversity in a suite of three-dimensional hydrodynamical explosion simulations. With subsequent radiative transfer simulations we will predict observables. These will be compared with astronomical data and we will quantify to which degree the delayed detonation scenario reproduces the variability of the observations. This will provide insight into the mechanism of “normal” SNe Ia and aid in their calibration as standard candles for precision cosmology.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 21 600 000 core-hours

Structural and conformational requisites in the folding process of the DNA quadruplex aptamer TBA

Project leader: Michele Parrinello, ETH Zurich Department of Chemistry and Applied Biosciences, Switzerland

Collaborators: Mrs Anna Berteotti, ETH Zurich, Switzerland / Dr. Vittorio Limongelli, ETH Zurich, Switzerland / Prof. Luciana Marinelli, University of Naples “Federico II”, Italy

Alternative higher order DNA structures that deviate from Watson-Crick double-strand can be formed by sequences that are widely distributed throughout the human genome. In fact guanine-rich (G-rich) stretches of DNA have a high propensity to self-associate to give unusual structures called G-quadruplexes (G4). G4 have been found in a number of important DNA regions, such as those present at the ends of telomeres, in the promoter region of oncogenes, in upstream of the insulin gene, and in the structures of some aptamers.Aptamers are nucleic acid macromolecules (from 15 to 40 nucleotides) that bind to molecular targets, including proteins, with high affinity and specificity. Base composition defines their secondary structure, consisting primarily of helical arms and single-stranded loops. Stable tertiary structures, resulting from peculiar foldings of these secondary structures, allows aptamers to bind to their targets. Consequently the ability to determine specific secondary structures and ad hoc change the sequence of an aptamer also allows to fine-tune the binding affinity and specificity of aptamers.Unfortunately, structural studies of DNA G4 have revealed the large conformational polymorphism of these structures such as that defined by the different polarity of the associating strands (parallel or anti-parallel) and/or by the location of the loops that link the guanine-rich motifs. The determination of the G4 structures is further complicated by the fact that different cations, such as K+ or Na+, by coordinating the carbonyl groups of guanines at the center of the G-tetrad core, stabilize in different manners G4 structures. As a result, the preferred conformations adopted by DNA depend on the nature of cations.In this project we will study through advanced computational techniques the structural and thermodynamic requisites for the folding/unfolding of an anti-parallel DNA quadruplex structure, the aptamer drug TBA (Thrombin Binding Aptamer). TBA is a thrombin inhibitor in development for use as an anticoagulant during coronary artery bypass graft procedures.Processes that take from microseconds to hundreds of seconds, such as protein/nucleic acid folding are impossible to simulate with standard computational techniques such as molecular dynamics, whose typical time scale is hundreds of nanoseconds and the use o
f enhanced sampling techniques is necessary. Thus, to reveal at an atomic level what might happen during the folding process and with the aim to overcome the large free energy barriers encountered in an affordable computational time, we will use a new approach, which combines parallel tempering (PT) and metadynamics (MetaD). Both these techniques have been successfully used in studies of protein folding. By combining PT with MetaD, an improvement over both of these methods could be obtained allowing to reconstruct the free energy of the folding process and to identify the various conformations assumed by DNA as well as the energetic cost of converting one into the other. Our results will provide not only precious insight in understanding aptamer folding mechanism but also useful guide lines for the experiments.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 6 000 000 core-hours

Extreme Earthquake Wave Propagation Modelling (E2WPM)

Project leader: Mike Ashworth, STFC Daresbury Laboratory Computational Science & Engineering, United Kingdom

Collaborators: Mr Eduardo Cabrera, Universitat Autonoma Barcelona, Spain / Prof Mario Chavez, Universidad Nacional Autonoma de Mexico, Mexico / Prof David Emerson, STFC Daresbury Laboratory, United Kingdom / Dr Charles Moulinec, STFC Daresbury Laboratory, United Kingdom / Mr Alejandro Salazar, Universidad Nacional Autonoma de México, Mexico

The scarcity of observational instrumental data for extreme, highly destructive earthquakes has triggered considerable seismological, engineering and socioeconomical interest in modelling future scenarios for these types of events worldwide. Recent events include the Sumatran-Andaman, Indonesia, earthquake of the 26/12/2004, magnitude (Mw) 9.3, and the Sichuan, China, event of the 09/05/2008 Mw 8. About 230,000 casualties resulted for the former and 70,000 casualties, plus 120 billion USA dollars losses for the latter.Realistic modelling of earthquake wave propagation through hundreds of kilometres of the earth’s crust, poses both a numerical and a computational challenge, as it requires enormous amounts of memory and storage, as well as intensive use of computing resources. As well as looking at past events, the work is capable of studying ground motions from hypothetical earthquakes in vulnerable regions, and identifying where motions would be at their greatest, should the earthquake occur. It could also help to assess how adequate an area’s emergency infrastructure would be in such an event.As a part of an ongoing research program, a recently optimized 3D seismic wave propagation parallel finite difference code, the 3DWPFD code was successfully applied to obtain low frequency, f<=0.3 Hz, 3D synthetic seismograms for the aforementioned Sichuan earthquake. The code was run on the KanBalam (UNAM, Mexico) and HECToR (UK National Supercomputing Service) supercomputers. Also, in order to test its scalability out to extreme processor counts, initial experiments were carried out using model grids representing a volume of 500 km x 260 km x 124 km. The grids were generated at a series of decreasing resolutions from 1 km down to 31.25 m. These grids were used to model the propagation of the Mw 8.01, Michoacan, Mexico, 19/09/1985 earthquake, and were performed on HECToR and JAGUAR (ORNL, USA) platforms and included the use of up to 20,480 and 65,536 processors, respectively.Taking those results into consideration, the objectives of this research proposal are1. To carry out more realistic simulations of extreme earthquake scenarios in Mexico, China and the USA.2. To explore further the scalability of the 3DWPFD code and its ability to exploit state-of-the-art high-end systems.3. To optimize the 3DWPFD code to enable delivery of more and better quality scientific outputs, which eventually will lead to improved estimates of the socioeconomic impacts of extreme earthquake scenarios.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 20 000 000 core-hours

The molecular bases of the transport cycle of APC antiporters

Project leader: Modesto Orozco, Institute for Research in Biomedicine Structural and Computational Biology, Spain

Collaborators: Prof Manuel Palacin, Institute for Research in Biomedicine, Spain / Dr Guillem Portella, Institute for Research in Biomedicine, Spain

Amino acids cross cell membranes with the mediation of amino acid transporters. At least 8 families of amino acid transporters are present in mammals. One of these families correspond to the Heteromeric Amino acid Transporters (HAT), with light subunits acting as the catalytic moiety of HAT, which belong to the prokaryotic and eukariotic LAT subfamily within the APC (Amino acids, Polyamines and organoCations) superfamily of transporters. Recent structural developments have shown that APC transporters (AdiC and ApcT) share the same protein fold with sequence unrelated transporters from 4 protein families (e.g., LeuT, vSGLT, Mhp1 and BetP).AdiC transporter exchanges extracellular arginine for intracellular agmantine, therefore acting as a virtual proton pump. The molecular basis of the transport cycle of AdiC is a fundamental question in the transport field. Among the “5-5 inverted repeat” fold transporters with atomic structure solved is the only with an obligatory mechanism of exchange (antiporter), whereas the others are Na+ or H+ coupled transporters (11). Experimental evidence suggests that the binding of a single molecule of substrate is necessary and sufficient to trigger the conformational changes that results in substrate translocation through AdiC. We plan to perform extensive molecular dynamics simulations of the Arg transport across AdiC antiporter embedded in prokaryotic membranes. Our final objective will be to derive an atomistic mechanism connecting conformational changes with the movement of the amino acid substrate along a putative translocation pathway in AdiC antiporter.Resolution of the molecular mechanisms of the transport cycle of AdiC will have at least two obvious applications. On the one hand, explanation at the molecular basis of the transport defect associated to mutation affecting the LAT transporters b0,+AT and y+LAT1, responsible of the human inherited aminoacidurias cystinuria and lysinuric protein intolerance. On the other hand, facilitation of the development of specific and high-affinity inhibitors LAT1, a new target for cancer therapy.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 33 700 000 core-hours

Understanding and Predicting the Properties of Clay-Polymer Nanocomposites using Petascale Computing

Project leader: Peter Coveney, University College of London Department of Chemistry, United Kingdom

Collaborators: Dr Derek Groen, University College of London, United Kingdom / Dr James Suter University College of London, United Kingdom / Mr Jacob Swadling University College of London, United Kingdom


Nanocomposites fall within the realm of the emergent area known as nanotechnology, where materials are designed and built at the atomic level. They consist of two-dimensional mineral layers separated by polymeric or organic material and possess novel mechanical, barrier, thermal and biodegradable properties. Nanotechnology is therefore an area which is of great academic, industrial, health and public interest. Various materials using layered nanoparticles have already been proposed for commercial applications in automotive, packaging, coating and pigment, electrical materials, and biomedical fields.Understanding how the microscopic structure of layered nanomaterials determines their (improved) macroscopic properties requires an approach that tackles a wide range of length scales, from nanometers to microns, each of which is vital for the mechanism of enhancement. Therefore, we require nanomaterials simulations which operate on multiple length scales, and encompass this complete structural hierarchy.With the knowledge of how this structural hierarchy functions will we be able to design novel layered mineral systems with properties tailored to their application. To sample the smallest modes of action we require detail on the molecular scale. We are therefore performing very large scale molecular simulations and using efficient sampling techniques to increase the length and timescales accessible with molecular simulation far beyond what is currently possible, to the micrometre and millisecond range, enough to fully sample all the modes of action of the layered nanomaterial.We plan to use our simulations to answer two important challenges in layered nanomaterials:First, we will evaluate the mechanism of polymer-nanocomposite formation and the effect it has on the overall material strength of the composite. We can only optimize the manufacturing process if we have a good understanding of the rheological properties and the mechanism of formation. This study will allow us to predict which products will create a homogenous nanocomposite with defined materials properties.Second, we will study the interaction between biological molecules and clay surfaces. These interactions are of great interest, as clay minerals are used in the development of new drug delivery systems, gene therapy and origins of life studies. The clay protects the drugs / biological molecules to reach the site of action and maintain a certain concentration during pharmaceutical treatment. For origins of life studies, one of the leading theories concerning the origin of life is the RNA world hypothesis, where RNA molecules carried out the tasks that DNA and proteins perform in contemporary cells. It is hypothesised that the occurrence of various steps towards the formation of a very complex molecule, such as RNA, must have required the presence of a protected confined environment, where RNA, or an RNA-like molecule, could originate and express its biological potential to self-replicate and evolve. In this project, we will be evaluating this hypothesis by understanding how primitive RNA adsorbed on clay-mineral surface may have been in the right conditions to undergo specific chemical reactions, triggering molecular evolution. This is also important in understanding how clay minerals provide a protective environment for biomolecules in gene-therapy.Such complex and challenging simulations can only be performed on today’s most powerful supercomputers, such the JUGENE BlueGene/P Tier-0 machine at FZJ.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 40 500 000 core-hours

LHCb Physics and Nucleon Distribution Amplitudes

Project leader: Roger Horsley, University of Edinburgh School of Physics and Astronomy, United Kingdom

Collaborators: Prof Gunnar Bali, University of Regensburg, Germany / Prof Vladimir Braun University of Regensburg, Germany / Dr Sara Collins, University of Regensburg, Germany / Dr Yoshifumi Nakamura, University of Tsukuba, Japan / Dr Dirk Pleiter DESY Zeuthen, Germany / Dr Paul Rakow, University of Liverpool, United Kingdom / Prof Andreas Schaefer, University of Regensburg, Germany / Dr Rainer Schiel, University of Regensburg, Germany / Dr Hinnerk Stueben, Konrad-Zuse-Zentrum fuer Informationstechnik Berlin (ZIB), Germany/ Dr James Zanotti, University of Edinburgh, United Kingdom

The Standard Model of nuclear and particle physics (SM) encompasses three of the four forces of nature (strong, weak and electromagnetic) and has proven to be extremely successful in describing almost all experimental results. However, the Standard Model fails to explain the matter-antimatter asymmetry of the universe and does not include gravity, the fourth force. Also for other reasons the possibility that the Standard Model is incomplete in some aspects has been widely discussed. Hence a huge amount of resources have been focused for physics beyond the Standard Model (“New Physics”). This is one of the prime objectives of the Large Hadron Collider (LHC) at Cern.Within the Standard Model there are six flavours of quarks (u,d,s,c,b,t) which are bound via the strong force to form protons and neutrons (nucleons), the building blocks of all matter around us. This force is mediated by gluons and is described by a theory known as Quantum Chromodynamics (QCD). Due to the unique self-interacting properties of gluons, it is impossible to calculate analytically the low-energy properties of QCD. The only way to perform such a calculation is to discretise space-time into a four-dimensional grid and perform a numerical simulation on a computer. This technique is known as Lattice QCD.Very recently claims have been made by groups at Fermilab (USA) to have found New Physics in the b-s quark sector. These claims will be verified or refuted by the LHCb experiment within the next few years. The aim of this proposal is to use the PRACE Tier-0 system to contribute missing information via distribution amplitudes to help in the analysis of the LHCb experiment.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 61 000 000 core-hours

Accretion disk dynamics: the multifluid regime

Project leader: Turlough Downes, Dublin City University School of Mathematical Sciences, Ireland

Collaborators: Dr Michael Browne, Irish Centre for High End Computing, Ireland / Mr Wayne O’Keeffe, Dublin City University, Ireland / Dr Stephen O’Sullivan, Dublin Institute of Technology, Ireland

Virtually all low and intermediate mass stars form by accreting material from a surrounding molecular cloud through an orbiting accretion disk and onto the surface of the forming star, or young stellar object (YSO). There is an, as yet unsolved, mystery surrounding the issue of how material can move inward through an accretion disk and onto the YSO. As the material moves inward it must lose angular momentum rapidly – if this did not happen the material could not move in toward the forming star at the observed rates and star formation would be extremely difficult. How, then, does the accreting material get rid of its angular momentum?One possibility is that an instability such as the magnetorotational instability (MRI) could produce turbulence in the accretion disk which would, itself, create a high effective viscosity. This viscosity would then act to tr
ansfer angular momentum from material at a particular point in the disk to material further out in the disk, thereby enhancing the rate of accretion. Accretion disks, though, are complex systems comprised of many chemical species and dust grains some of which are charged (and interact with magnetic fields), and some of which are neutral: in short, it is a multifluid system made up of various charged and neutral fluids. To gain a proper understanding of how turbulence in accretion disks behaves and how it can be generated we must move towards full multifluid modeling.We will use the state-of-the-art, massively parallel multifluid magnetohydrodynamic code HYDRA, coupled with power of the Blue Gene/P system JUGENE, to study the MRI in accretion disks around YSOs and the dynamics of the resulting turbulence. HYDRA is well-tested and proven both from the point of view of the published physical results it has generated and in terms of its scalability: it has been shown to perform remarkably well in hard scaling tests up to almost 300,000 cores.

Computer system: JUGENE, GAUSS/FZJ
Resource awarded: 14 000 000 core-hours

MS-COMB: Multi-Scale Analysis and Numerical Strategies for the Simulation of Premixed Turbulent Combustion in Realistic Geometries

Project leader: Vincent Moureau, CORIA – CNRS UMR6614 Combustion, France

Collaborators: Dr Pascale Domingo, CORIA – CNRS UMR6614, France / Prof Luc Vervisch CORIA – CNRS UMR6614 SAINT-ETIENNE-DU-ROUVRAY FRANCE

The MS-COMB project is dedicated to the understanding of small-scale dynamics occurring in premixed turbulent combustion through a multi-scale analysis, and to the evaluation of numerical strategies to improve the efficiency of large-scale computations and their post-processing. Small-scale dynamics are of primary importance for the prediction of pollutants such as soot or unburned hydrocarbons in gas turbines and automotive engines. Since the small-scale dynamics analysis requires modeling the full range of turbulent and reactive scales, billion-cell meshes are mandatory. Handling such large meshes and solving efficiently the low-Mach Navier-Stokes equations on these meshes is highly challenging.The finite-volume solver YALES2, developed at CORIA, has been specifically tailored to deal with very large meshes and to solve efficiently large linear systems on massively parallel computers. This research code is designed to model turbulent reactive flows and atomization of liquid fuels with body-fitted unstructured meshes. Large computations have been carried out and extensively analyzed in 2009 and 2010 for an industrial swirl burner with 2.6 billion cells. Moreover, during the 2010 Blue Gene Extreme Scaling Workshop at Juelich, the code was able to run on a 21 billion cells mesh.The first objective of the current proposal is to investigate a new database of a model burner with a fine description of the small-scale dynamics. This will require running in a sustainable manner with a 10 billion cells mesh and performing several types of post-processing including filtering at several length scales to exhibit the small-scale dynamics. Such a mesh resolution is needed to resolve all the scales of turbulent flows at realistic Reynolds numbers. The second objective is to evaluate new numerical strategies for the solving of Poisson equations on very large meshes. Recently, a residual recycling technique (Fischer 1998) has been implemented and improved and the YALES2 code. While it allows reducing significantly the cost of the Poisson equation in constant density flows, the gain is not as impressive for propagating flames.

Computer system: CURIE, GENCI/CEA
Resource awarded: 5 000 000 core-hours