1st Project Access Call – Awarded Projects

Nine research projects, two from Germany, two from Spain and one each from France, Hungary, the Netherlands, Portugal and the UK, have been awarded access to the PRACE Research Infrastructure. In total 362.8 Million compute core hours were granted for the JUGENE, IBM BlueGene/P, hosted by the Gauss-Centre for Supercomputing member site in Jülich, Germany, which is the first Petascale HPC system available to researchers through PRACE.

Projects supported in alphabetical order of project leaders:

Droplet growth by coalescence in turbulent clouds: kinetics, fluctuations, and universality

Project leader: Jeremie Bec, Observatoire de la Côte d’Azur, Nice, France

Collaborators: Holger Homann, Observatoire de la Côte d’Azur, Nice, France / Samriddhi Sankar Ray, Observatoire de la Côte d’Azur, Nice, France / Rainer Grauer, Ruhr-University Bochum, Bochum, Germany


Warm clouds are constituted of small water droplets that do not follow exactly the turbulent airflow but have inertia. They thus react with some delay to the fluid motion and feel gravity, so that they distribute non-uniformly in space and can have very large velocity differences. Consequently the rate of collision and growth by coalescence of such droplets cannot be predicted by simple arguments and the timescales of precipitation are often under-estimated. Our objective is to investigate this issue by a direct numerical simulation of coalescing particles that are passively transported by a homogeneous isotropic turbulent flow.

While atmospheric scientists attach importance to account simultaneously for all processes to be as much realistic as possible, the proposed approach simplifies the main mechanisms to obtain a better handling of fundamental questions. The novelty of the intended work originates in extending and adapting to the problem of rain formation the statistical physics tools developed for Lagrangian turbulence. The main ingredient of such approaches is the statistical Lagrangian formalism developed during the last decade, which consists in reformulating transport and mixing problems in terms of averages along particle paths. The key task is then to estimate the cumulative weights of fluctuations along trajectories or to determine the probability of the events giving a dominating contribution to the statistics. This method has two advantages: first, it is well adapted for dealing with systems that are far from equilibrium, and second, it allows controlling the history of individual particle paths. The problem of estimating time scale of rain formation in warm clouds requires tools that cope with these two difficulties.

In order to accompany and validate such analytical modelings, we will perform state-of-the-art numerical simulations of a cloud portion. For being ensured that the airflow turbulence is sufficiently developed, we plan to integrate the fluid phase on a $2048^3$ periodic grid with a high-accuracy pseudo-spectral solver. To match particle loading encountered in strato-cumulus, one billion particles will be seeded in the flow with an initial size distribution that mimics observations and which will be centered around a typical radius equal to one twentieth of the dissipative Kolmogorov scale. The system will then be evolved using an efficient collision detection algorithm and performing particle mergers that conserve mass and momentum. The simulation will be evolved for at least fifty large-scale turnover time to obtain a typical size of particle that has increased by at least one order of magnitude. The objective is to measure timescales for the growth of droplets and to highlight universal properties of the size distribution in the large-time asymptotics. Such measurements will confirm or refute the validity of predictions from standard mean-field kinetic models.

Resource awarded: 50 000 000 core-hours

Ab initio molecular dynamics simulations of proton transport in a biological ion channel

Project leader: Paolo Carloni, German Research School for Simulation Sciences GmbH, Jülich, Germany

Collaborators: Emiliano Ippoliti, German Research School for Simulation Sciences GmbH, Jülich, Germany / Chao Zhang, German Research School for Simulation Sciences GmbH, Jülich, Germany / Fabrizio Marinelli, Max Planck Institute für Biophysik, Frankfurt am Main, Germany


Ion channels are pore-forming membrane proteins regulating ionic flow through cell membrane. These proteins are crucial for cell function and their derangement is involved in many diseases, including heart attack.

The prototype ion channel is the gramidicin A peptide dimer (gA), which selectively allows for fast transport of monovalent cations (K+, Na+, H+). Particularly interesting is the transport of the proton ions, which occurs with the highest efficiency known for any biological channel. Empirical quantum-mechanical methods have lead to several insights on the mechanism of proton permeation, yet the conductance calculated with such approaches is much smaller than the experimental value. This is likely to be related to the absence in the calculations of the membrane potential, which drives the H+ ion from outside the cell to the cytosol.

Here we will use ab initio molecular dynamics to provide insights on mechanism and energetics of the overall proton translocation process through gA. The calculations will be carried out in the presence of the membrane potential and including nuclear quantum effects, which are known to affect the structural determinants of the process. We will use the metadynamics method to evaluate the free energy associated with H+ transport, as the latter is a rare event in the time-scale of the simulation. The calculated H+ conductance, obtained by a kinetic model using the free energy profile, will be compared with experimental data. Contact with experiment will be achieved also by comparing calculated and measured NMR chemical shifts. The system ( 1900 atoms) can be run only on extremely massive computational resources such as those of PRACE Tier-0.

The proposed research may add a new dimension in the simulation of biological ion channels. It will show that the PRACE resources allow predicting quantitatively and fully from first principles, energetics and spectroscopic properties of these fundamental biological systems in laboratory-feasible conditions.

Resource awarded: 48 758 784 core-hours

Entrainment effects in rough-wall boundary layers

Project leader: Javier Jimenez, Universidad Politecnica Madrid, Madrid, Spain

Collaborators: Guillem Borrell, Universidad Politecnica Madrid, Madrid, Spain / Ayse Gungor, Universidad Politecnica Madrid, Madrid, Spain


The aim of this proposal is to clarify a long-standing controversy on the effect of wall roughness on turbulent boundary layers. Classical theory asserts that the direct effect should be restricted to a thin layer near the wall, but persistent experimental reports imply that, under some circumstances, it may extend to the whole layer. The best current explanation is that the effect is global in external boundary layers, but not in internal flows, and that this might be connected with the entrainment of irrotational fluid in the boundary layer. Weaker similar effects have recently been found in boundary layers over smooth walls. The present proposal is to run matched simulations of two layers at similar Reynolds numbers, but with different entrainment rates, by forcing one of them with a smooth body force. That would separate the effects of roughness from those of entrainment. The forcing scheme has been tested in a small-scale problem. As with all the simulations from our group, the results will be made freely available to the community, including the raw data to groups able to manage the large data sets. The code is already optimized for BG/P (Intrepid, in 32 Kcores). This proposal is a resubmission of one presented in the Early Access call, ranked there as number 12 in the acceptable list.

Resource awarded: 40 000 000 core-hours

QCD Thermodynanics with Wilson fermions

Project leader: Sandor Katz, Eotvos University, Budapest, Hungary

Collaborators: Daniel Nogradi, Eotvos University, Budapest, Hungary / Balint Toth, Eotvos University, Budapest, Hungary / Norbert Trombitas, Eotvos University, Budapest, Hungary / Endrodi Gergely, Eotvos University, Budapest, Hungary / Kalman Szabo, University of Wuppertal, Wuppertal, Germany / Zoltan Fodor, University of Wuppertal, Wuppertal, Germany


We propose to continue our study of QCD thermodynamics which is directly relevant for particle physics phenomenology, heavy ion collisions and the early universe. Our projects were started four years ago, last year we successfully applied for computer time on Jugene via the EBP01 European NIC project. During this time we have published several important results (transition temperature and the equation of state in the staggered approach; see the details of our attached report). The present proposal is an extension of our closing EBP01 project.

In this project we concentrate on QCD thermodynamics with the Wilson formulation of dynamical quarks. Most previous works used staggered fermions and there were disagreements in the results. Since staggered fermions use a theoretically not fully understood rooting procedure, it is crucial that all thermodynamics studies are also carried out with a different, e.g. Wilson-type discretization which is theoretically sound.

We plan to use the fixed scale approach, where the temperature is varied by changing the temporal lattice extent and not the lattice spacing. This turns out to be favorable for Wilson fermions. We will determine the temperature dependence of various quantities for three lattice spacings. These quantities include the Polyakov-loop, the chiral condensate/susceptibility and the quark number susceptibility. They can all be used to define transition temperatures. We will use pion masses down to 190 MeV and perform a combined extrapolation to the physical point and the continuum.

The proposed project requires a large number of independent computations. We find it very important to activate own resources in proportion to the requested computer time on Jugene. Funded by the principal investigator’s European Research Council grant a GPU-based cluster has been recently installed in Budapest. A great amount of computations will be carried out here. There are, however, several ingredients which require large scale supercomputing resources. All thermalizations and the production runs for larger lattices require parallelization beyond the capabilities of the available GPU cluster.

The core of our dynamical QCD code has been designed to optimally exploit the benefits of the Blue Gene/P machine at FZ-J”ulich. The code has been in production use for years and all further improvements we thoroughly test on our local clusters. Our Wilson lattice code is running at a sustained performance of 37% with a perfect strong scaling.

After subtracting those tasks which can be performed on the GPU cluster, to achieve the goals of this proposal we require 24 rackmonths, 72 million core-hours of Jugene CPU time.

Resource awarded: 72 000 000 core-hours

Investigating the effects of quantum nuclear motion in an enzyme that employs hydrogen tunnelling

Project leader: Dominik Marx, Ruhr-Universität Bochum, Bochum, Germany

Collaborators: Ian Grant, Ruhr-Universität Bochum, Bochum, Germany / Sergei Ivanov, Ruhr-Universität Bochum, Bochum, Germany / Theodros Zelleke, Ruhr-Universität Bochum, Bochum, Germany / Sebastian Braun, Ruhr-Universität Bochum, Bochum, Germany


Hydrogen transfer reactions are fundamental to many biological processes and integral to many enzymatic mechanisms of catalysis. Experimental studies have shown that some of these reactions exhibit a kinetic isotope effect (that is, a rate acceleration when the species involved in the transfer is hydrogen, rather than deuterium—i.e. kH/kD>1), indicating that nuclear quantum effects such as zero-point motion play a role in the mechanistic step.
Some kinetic isotope effects have been observed that are significantly larger than that which can be explained by considering only the zero-point vibrational energy of the isotopes (this is known as the semi-classical limit, and is around 7). A quantum mechanical tunnelling mechanism whereby the particle can penetrate into regions of phase-space that are classically forbidden can be invoked to account for such inflated KIEs. In this sense, the particle ’tunnels’ through the classical energy barrier represented by the potential energy of the transition state.

However, the role of nuclear quantum effects such as zero-point motion and tunnelling in enzymatic hydrogen transfer processes is still not yet fully understood, and is the subject of much controversy amongst researchers—specifically, whether the effect is truly catalytic (that is, whether the enzyme has evolved to enhance the amount of tunnelling compared to a given reference reaction, sans-enzyme, for example) and to what extent dynamics play a role (for instance, whether particular vibrations in the protein promote tunnelling).

Computer simulation of hydrogen transfer mechanisms with tunnelling can offer detailed, atomic insight into such processes, but incorporating nuclear quantum effects such as zero-point motion and nuclear tunnelling into the simulation of biological systems is challenging, due to the complex nature of the environment and the inherently multi-dimensional nature of tunnelling.

In this proposal we outline a new technique which we believe has the potential to produce the most accurate calculations to date of hydrogen tunnelling in enzyme mechanisms. The calculations make use of the QM/MM method to enable the efficient simulation of large, enzymic systems, the path integral formalism to quantize the nuclear degrees of freedom, and ab initio metadynamics quantum free energy landscapes. Using this technique, we propose to study the effects of quantum nuclear motion in hydrogen transfer in methylamine dehydrogenase, a relatively well-known system that is believed to employ tunnelling (it exhibits a kinetic isotope effect of approximately 17 for the rate-determining step, which is a proton transfer mechanism), but about which crucial questions are still unanswered.

Resource awarded: 32 000 000 core-hours

Turbulent entrainment due to a plume impinging on a density interface

Project leader: Maarten van Reeuwijk, Imperial College London, London, United Kingdom

Collaborators: Harm Jonker, Delft University of Technology, Delft- The Netherlands / Gary Hunt, Imperial College London, London – United Kingdom


The entrainment of fluid across density interfaces is a fundamental physical process with applications throughout the natural sciences and engineering. This process is of importance in density stratified environments which are subject to regions of localized turbulence. For example, in oceanic and atmospheric contexts, turbulent entrainment has a bearing on the rate of deepening of oceanic and atmospheric mixed layers, respectively. This then has a direct impact on the concentration of pollutants released in the atmosphere or trapped in a mixed layer in the ocean. As entrainment involves a transport of fluid between layers, it has widespread applications in water-air quality problems. However, despite its significance, existing entrainment laws, which couple an entrainment velocity to a turbulence intensity and density contrast between the layers, are subject to very significant uncertainties and currently there is no consensus in the literature regarding which is correct or the most appropriate. With laboratory measurements of entrainment rates in identical apparatus varying by orders of magnitude, this then presents the scientific community with real concerns and throws into question many models and modeling approaches which rely on a specification of such entrainment rates. This proposal aims to pin down the entrainment law for localized turbulent patches using direct numerical simulation (DNS).

An improved entrainment law for localized turbulent patches will enable enhanced predictive capability. One pertinent example is the low-energy design of ventilation in buildings. Here, the turbulent entrainment across a thermal interface – an interface separating a cool lower region in a room from the warmer upper region – as is typically induced by a thermal plume rising from localized heat sources, is an extremely important mechanism as it plays a significant role in determining the temperature distribution in the room and thus, the comfort of occupants. Optimization of the energy-efficiency in a ventilated building is only possible if the simplified models typically used to provide design guidance are accurate.

Resource awarded: 30 000 000 core-hours

Non diffusive transport in ITG plasma turbulence

Project leader: Edilberto Sánchez, EURATOM-CIEMAT Association, Madrid, Spain

Collaborators: Iván Calvo, EURATOM-CIEMAT Association, Madrid, Spain / Francisco Castejón, EURATOM-CIEMAT Association, Madrid, Spain / Alejandro Soba, Barcelona Supercomputing Center, Barcelona, Spain / Xavier Xáez, Barcelona Supercomputing Center, Barcelona, Spain / Jose María Cela, Barcelona Supercomputing Center, Barcelona, Spain / Ralf Kleiber, Max-Planck-Institut für Plasmaphysik, Greifswald, Germany


The understanding of particle and energy transport in turbulent fusion plasmas is one of the most relevant and intellectually challenging open problems in the field of magnetic confinement fusion. On the one hand, the great impact of transport on the size and, hence, on the cost of the reactor is the reason why progress in its comprehension is of the utmost importance for the objective of harnessing fusion energy. On the other hand, the transport theory for collisional toroidal plasmas (the so-called neoclassical theory) predicts confinement times up to two orders of magnitude higher than observed experimentally. Therefore, there must be an ingredient other than collisions which, in certain regimes, dominates transport. There is little doubt that turbulence is the cause of the discrepancy between neoclassical theory and experimental data. However, as is well-known, turbulence is one of the major unsolved problems of classical physics so it is not surprising that there is no complete theory of transport in turbulent plasmas.

However, much has been learnt in the last years and a number of beautiful phenomena have been discovered. In this project we will try to advance in the study of one of them, namely the appearance of Lévy statistics (in particular non-Gaussian) in the statistical description of transport in a variety of turbulence plasma models (see Refs. [DelCastillo05, Sanchez08, Sanchez09, Mier08]). In other words, it seems that in realistic turbulent fusion plasmas transport can be non-diffusive (as predicted by collisional theories of transport) and, what is more, non-local and non-markovian. We are specially interested in the problem and results of Refs. [Sanchez08, Sanchez09] where the character of particle transport across zonal sheared flows in a gyrokinetic simulation (a first principle approach to the description of the plasma) has been studied. The simulation of a tokamak plasma in that work revealed that the presence of a sheared zonal flow may change the properties of particle transport not merely by modifying the value of the diffusion coefficients but changing the very nature of the transport process. Concretely, in the cases studied in Refs. [Sanchez08, Sanchez09] transport is diffusive in the absence of zonal flow, subdiffusive but still Gaussian if there is an externally imposed zonal flow and subdiffusive and Lévy if the zonal flow is consistently generated by the plasma.

The singular importance of understanding transport across sheared flows arises from the fact that sheared flows are though to be of great importance in the formation of transport barriers in magnetic confinement devices. The transport barrier control would allow reaching improved performance regimes in fusion plasmas.
We propose to address some specific points in this line of research. First of all, the importance of the results of Refs. [Sanchez08, Sanchez09] deserve, on the first place, an effort to be reproduced in its main aspects by other codes and, if possible, with increased statistics. We propose to carry out simulations corresponding to similar plasma conditions in cylinder geometry by using the global, three-dimensional, gyrokinetic code EUTERPE [Jost01]. In order to have low noise in a long enough plasma simulation, the features of a Tier-0-like machine are essential. More specifically, we will try to understand the following points:

  • (i) How the results of Refs. [Sanchez08, Sanchez09] are modified by increasing the amount of statistics and the signal-to-noise ratio.
  • (ii) How the toroidal geometry influences the results of Refs. [Sanchez08, Sanchez09]. In particular, it is suspected that at early times, a peculiar behaviour observed in the evolution of the self-similarity index might be due to the banana orbits. We will do our computations in a periodic cylinder in order to try to give an answer to this question.
  • (iii) Very recent results obtained by some of us and collaborators [Carreras08, Garcia09] suggest that there is a connection between the topological structure of the turbulent flows and the features of particle transport in a resistive MHD model. The applications of the computational topology tools to the results of the proposed simulations will reveal whether the features of transport can also be understood from the perspective of the topology of the underlying turbulence in a gyrokinetic simulation.

Resource awarded: 20 000 000 core-hours

Predictive full-scale fast ignition with PW plasma amplified laser pulses

Project leader: Luis O. Silva, Instituto Superior Técnico, Lisbon, Portugal

Collaborators: Warren Mori, University of California Los Angeles, Los Angeles, United States / Raoul Trines, Rutherford Appleton Laboratory, Didcot, UK / Frederico Fiuza, Instituto Superior Técnico, Lisbon, Portugal / Ricardo Fonseca, Instituto Superior Técnico, Lisbon, Portugal


Fusion energy is regarded as a possible long-term energy solution for humanity, capable of providing the energy resources to drive economic growth and social development. Fast ignition is one of the most promising and exciting inertial confinement fusion schemes to improve the viability of inertial fusion energy as a practical energy source. Even if up to now experiments have been limited to laser energies still far from ideal conditions for ignition and simulations, which are extremely complex due to the different temporal and spatial scales involved, have been limited to reduced scales/simplified models, in the very near future we expect transformative results as novel laser systems are now coming online, with the National Ignition Facility and, in the near future, the ESFRI roadmap project HiPER (High Power laser Energy Research), reaching the conditions required for ignition. In this proposal, and using massively parallel simulations, we aim to perform, for the first time with realistic target properties (e.g. density, temperature, dimensions) and the correct simulation dimensionality, and with realistic ultra-intense laser pulses obtained from Raman amplification, self-consistent fast ignition simulations including all the relevant microphysics/particle dynamics, taking advantage of the novel hybrid model incorporated into the state-of-the-art relativistic particle-in-cell massively parallel code OSIRIS, with the goal of identifying possible paths to demonstrate fast ignition as an efficient scheme for inertial fusion energy.

Resource awarded: 31 000 000 core-hours

Large scale high resolution blood flow simulations in realistic vessel geometries

Project leader: Federico Toschi, Eindhoven University of Technology, Eindhoven, The Netherlands

Collaborators: Jens Harting, Eindhoven University of Technology, Eindhoven, The Netherlands / Florian Janoschek, Eindhoven University of Technology, Eindhoven, The Netherlands / Stefan Frijters, Eindhoven University of Technology, Eindhoven, The Netherlands / Ariel Narvaez, Eindhoven University of Technology, Eindhoven, The Netherlands / Prasad Perlekar, Eindhoven University of Technology, Eindhoven, The Netherlands


Simulation of human blood flow is a true multi-scale problem: in first approximation, blood may be treated as a dense suspension of highly deformable red blood cells (RBCs) in plasma. The particulate nature requires to resolve the complex properties of individual cells. On the other hand, in realistic vessel geometries typical length scales vary over several orders of magnitude. Current models either treat blood as a homogeneous fluid and cannot track particulate effects or describe a relatively small number of RBCs with high resolution, but are not able to reach relevant time and length scales.

We developed a highly efficient yet still particulate model that allows the simulation of millions of cells (see arXiv:1005.2594). It combines a lattice Boltzmann solver to account for hydrodynamic interactions and a molecular dynamics code with anisotropic model potentials to cover the more complex short-range interactions of the cells.

Within the PRACE project we will apply a refined model to reproduce the complex rheology of blood in more detail and then apply our method to simulations of flow in realistic model geometries in order to study phenomena due to clogging of cells and blocking of vessel branches which can lead to diseases such as hypoxic ischemia or thrombosis.

Resource awarded: 39 000 000 core-hours