PRACEdays15 Posters


Numerical study of mixing in swirling coaxial jets. An application of large eddy simulation.


PDF

Teresa Parra received her PhD in Mechanical Engineering in 1999. She is a researcher in the framework of Fluid Mechanics and Turbo-machinery at the University of Valladolid. Her research interests are in numerical simulation of turbulent and reactive flows using Reynolds Averaged Navier Stokes and Large Eddy Simulation approaches. Her basic research is applied to renewable energies (hydraulic and wind power) as well as ultra-low emissions of stabilized lean swirling flames. Since 2012, she is user of high performance computing infrastructures.

She has published among others in Combustion Explosions and Shock Waves, Journal of Engineering Computations, Combustion Science and Technology, Energy, Building and Environment, Applied Thermal Engineering. Of the 13 publications considered for the h index, 7 have been cited at least 7 times (updated on sep-14).

Supervisor of 5 Ph. D. theses on the following issues: Numerical and experimental study of parallel and inclined turbulent wall jets. Numerical model for two-phase solidification problem in a pipe. Turbulent supercritical mixing-Selection of methods and tools. Numerical Simulation of the Performance of a Human Nasal Cavity. Numerical Analysis of Airfoils Used at Vertical Axis Wind Turbine.

 

Assessment of Large Eddy Simulation (LES) models of confined coaxial swirling jets is the aim of this work. The mid-term application is to improve the stabilization of flames of poor mixtures by means of a swirling flow. This provides saving of fuel as well as a reduction of contaminant emissions. Swirling burners have some advantages when compared with bluff bodies and cross flows. These are lower head losses and soot, less maintenance tasks. A possible application of the smallest burners is the food industry since baking with low emissions improves the taste of the product.

Despite the simple geometrical set-up of the benchmark, the flow pattern shows complex aerodynamic behavior. The simple burner considers the use of two coaxial nozzles: one axial with fuel and another annular with air. The expansion of the flow, when entering the chamber will produce the Outer Recirculation Zone. If swirl number is large enough to let the flow turn back into the centre, the vortex breakdown phenomenon appears to form an Inner Recirculation Zone limited by two stagnation points located in the axis of the chamber. The region between both recirculation zones with high shear is where mixture of fuel-air occurs.

This work is devoted to isothermal flow to gain an insight of flow pattern associated to different swirl numbers and diffusers. Axial swirl injector is composed by 8 fixed vanes in the annular nozzle. It is responsible of the azimuthal momentum of annular jet. The Swirl number is associated to the angle of the trailing edge of the vanes. Besides, the influence of conical diffusers in the mixture is analyzed.

Swirl numbers of 0.2 (low) 0.6 (intermediate) and 1.2 (strong) were tested. To sum up, the strong swirl number had the lead stagnation point near the discharge of the nozzles and provided a mixing length lower than half diameter of the chamber. Intermediate swirl number have bigger Outer Recirculation Zones and the mixing length is more than one diameter. Finally the low swirl number does not have any vortex breakdown and the mixing length is several diameters.

Bearing in mind the influence of conical diffusers, it is more important in the case of intermediate swirl numbers since the diffuser reduces the mixing length. A 140º diffuser is able to avoid the Outer Recirculation Zone for Strong Swirls. The same diffuser setup operating with intermediate swirl number is able to prevent the formation of Taylor-Couette instabilities (counterrotating vortex rings) with the associated reduction of head losses.

These models were tested using the LES algorithm Scale Selective Discretization scheme. Temporal resolution is 10-5 s/timestep with spatial resolution 5 times larger than Kolmogorov scale. It was found that for mesh of 7-9 million cells without multigrid, the optimum is 64 processors. If the multigrid set up is modified to consider more cells in the coarsest level, the optimum number of processors can be increased to 128. Also, the increase of tolerance has an impact for efficient use of larger number of processors.

The author thankfully acknowledges the Spanish Ministry of Science and Innovation for the financial resources in the framework of the project reference ENE2011-25468. We acknowledge PRACE for awarding us access to resource Curie-GENCI@CEA based in France and MareNostrum@BSC based in Spain. Ref. 2010PA1766


Numerical simulation of non-premixed swirling flames


PDF

J. Perez is a Ph.D. candidate in Mechanical Engineering at University of Valladolid, Spain, where he received his degree in Mechanical Engineering in 2011 and in Management Engineering in 2013. His research interests are in CFD for turbulent flows. He worked on design optimization of air radial impulse turbines for Oscillating Water Column Marine Power Plant and on HVAC and indoor air quality in operating rooms. Actually, he works on thermal analysis in the Spanish Interim Nuclear Waste Storage Facility and in power plant simulations as transient hydraulic calculations.

 

The present work focuses on the numerical simulation of diffusive flames in a confined swirl burner. The background motivation for the project arises from the greenhouse gas emissions. In methane operated burners, the methane slip due to incomplete combustion is a problem since methane is a harmful greenhouse gas. Lean flames produce less contaminant emissions and reduce fuel consume, however they are unstable. The swirling flow is a stabilizer of the flame so that poor mixtures can be burned.

The governing equations for 3D, transient, reactive flow are solved with a second order scheme. The 3D mesh has 4 million hexahedral cells. No multi-grid was used for reactive cases because the averaging on temperature field is a precursor of the lack of accuracy on the reaction rate. As for the turbulence model, the k-ε was selected.

Numerical model for no swirl and high swirl burners have been carried out using heat and mass transfer for non reactive cases and a simplified mechanism of reaction for the reactive case. Regarding the reactive case: three stoichiometries were mixed and burned, stoichiometric (lambda = 1), lean (lambda = 1.2) and rich (lambda = 0.8) mixtures. The temporal resolution must be around 10-7 s/timestep, because of the stiffness of the reactive case.

Contrasting non reactive and reactive cases, the last one produces higher axial velocities to keep the mass balance. Hence, it is a precursor of smaller Inner Recirculation Zones (IRZ) in the case of strong swirls. The lead stagnation point of the IRZ plays an important role fixing the location of the flame front in swirling burners. Besides, the hot products of reaction of the IRZ help to warm the fresh mixture. Contrasting flames with swirl number null, 0.6 and 1 it is possible to conclude the decrease of the flame front thickness while increasing the swirl number.
Contrasting different stoichiometries, lean mixtures have lower equilibrium temperature and therefore, the thermal emission of nitrogen oxides is lower. However, strong swirls are needed for very poor mixtures in order to be burn in a stable way.

The author thankfully acknowledges the Spanish Ministry of Science and Innovation for the financial resources in the framework of the project reference ENE2011-25468. We acknowledge PRACE for awarding us access to resource Curie-GENCI@CEA based in France and MareNostrum@BSC based in Spain. Ref. 2010PA1766


AMOEBA and HPC: Parallelising Polarisable Force Fields


PDF

I have graduated from the University of Edinburgh in 2011 with Master degree in Physics with honours in Mathematical Physic. My master project was focused on simulating bootstrap percolation in complex networks. I also did MSc in High Performance Computing at the University of Edinburg and my dissertation project was optimising and parallelising PLINK software package, (whole genome analysis toolset). The last 1.5 years I have been working as an applications developer at the Edinburgh Parallel Computing Centre (EPCC), which is a part of School of Physics at the University of Edinburgh. I have been working on a variety of different collaboration projects making use of high performance computing resources such as CRESTA and APES. I have also been involved in some of the ARCHER (UK’s supercomputer) related activities e.g. providing support for Amber package on ARCHER and HPC training. At the moment my main projects are: parallelising TINKER (MD package) as a part of APES project and developing the online distance learning courses in HPC and Data Science offered by the University of Edinburgh

 

For decades now classical and quantum mechanical based computational chemistry codes have been used to evaluate molecular properties and interactions in gas and condensed phases. However, there are a number of systems that cannot be simulated correctly using the currently available software as they are limited by the use of non-polarisable classical models. These models approximate electrostatic interactions by only using fixed charges, which means that the charges in atoms are not allowed to adapt to changes in their local environment. Hence these models are not capable of providing sufficiently accurate results for systems where the polarization and self-induced charges play a significant role in the interaction. Polarisation changes the geometry and energy of the system by distorting the bonds between atoms and it is a necessary intermolecular interaction when modelling chemical reactions in biological molecules and water-based systems. Therefore, to tackle a wider range of problems in chemistry, biochemistry and material science where charges and polarization are important it is necessary to implement more realistic modelling of molecular forces that take into account these effects.

Thus efforts have been put into developing mutually polarisable models that enhance the fixed charge models by including the polarisation of atoms and molecules in the dynamics of these systems. These polarisable models are more accurate and more complex hence more computationally demanding. Typically the size and number of time steps in these simulations are selected so that the calculation can finish within a reasonable time period but they also need to be long enough to be relevant to the timescales of the process being simulated. For example, most proteins and DNA simulations need to span at least nanoseconds of simulated system time, which can take days to years of wall clock time on a single CPU. Moreover, the use of polarisable fields, which are necessary for most biological systems, will increase this time by ten-folds or more. Hence, there is a strong need to develop the software packages implementing polarisable force fields that are capable of exploiting current and emerging HPC architectures to make modelling ever more complex systems viable.

One of the more prominent polarisable models in this field is AMOEBA (Atomic Multipole Optimised Energies for Bimolecular Applications). This has been implemented and is widely used in codes such as TINKER and AMBER. AMOEBA replaces fixed partial charge with atomic multipoles and includes an explicit dipole polarisation, which allows atoms to respond to the polarisation changes in their molecular environment. AMOEBA has been shown to give more accurate structural and thermodynamic properties of various simulated systems. However, the time needed to calculate the effect of the induced dipoles alone is 15-20 times larger than the cost of one time step for a fixed charge model. If the computational time could be reduced then the AMOEBA force field could, for example, model a much broader range of protein-ligand systems than is currently possible or provide a means of refining in X-ray crystallography for large and challenging data sets such as ribosome crystals.

The larger computational costs associated with the calculation of polarisations needs to be mitigated by improved the parallelisation of codes such as TINKER. In this poster we give more details on the AMOEBA model as implemented in the TINKER molecular modelling package and present our efforts at improving its parallelisation through the use of hybrid MPI and OpenMP techniques.


Novel multiphase simulations investigating cavitation by use of in-situ visualization and Euler/Lagrange coupling


PDF

Mathis Bode is a research assistant and Ph.D. student in Prof. Pitsch’s group at the Institute for Combustion Technology at RWTH Aachen University. He received his Master of Science in Mechanical Engineering from RWTH Aachen University in 2012. His research interests include high fidelity simulations of multiphase flows on massively parallel computers.

 

Flow configurations involving both liquid and gaseous fluids often occur in industrial applications. Engine injection systems, which are used to atomize liquid fuels, are one example. The performance of such atomizers depends on a cascade of physical processes, originating from the nozzle internal flow, cavitation, turbulence, and the mixing of a coherent liquid stream with a gaseous ambient environment. The transfer occurring between liquid and gas is governed by an interface topology. An accurate prediction of this mixing stage is crucial for reliable predictions of the overall system as it is typically the first physical process to be modeled in simulations and uncertainties here will influence, for example, the design and performance of engines all the way down to emission and pollutant formation.

Within the last years, engine experiments have shown that the impact of cavitation on the overall engine processes is much larger than current knowledge would predict. Due to the small size of injection systems, which have outlet diameters on the order of 100 micrometers, and the resulting bubbles, droplets and turbulence structures, which are even much smaller, a detailed investigation using experiments is very difficult and simulations can be quite helpful.

Accurate simulations of the whole atomization process have to include a broad spectrum of different length scales and resolve the interface topology in an exact and robust way, which is hard to achieve even with massively parallel code frameworks on TIER-0 HPC systems. However, recent developments with respect to interface tracking methods and a new simulation approach combining Eulerian and Lagrangian spray simulation techniques in order to decrease the computational cost in physically less important flow regions, enables us to study the impact of cavitation on the mixing process. Additionally, new in-situ visualization techniques enable a smart data management, which stores fully resolved data only in important flow regions leading to a higher information/data size ratio, which is crucial for model development.

This work presents the new simulation techniques as well as its application to realistic atomizers. The CIAO code framework was used on MareNostrum III and JUQUEEN for data generation and the data are studied focusing on the effect of cavitation on commonly used spray models in industrial context.


Massively parallel code to search for gravitational waves from rotating neutron stars in advanced detector data


PDF

Gevorg Poghosyan is head of simulation laboratory for Elementary- and Astro- Particle Physics at Karlsruhe Institute of Technology. He is holding Ph.D. in Physics from Yerevan State University. As an expert for HPC in Astro- and Particle Physics he is working at Steinbuch Centre for Computing on joint research and development projects with scientific and industry groups to ease porting existing codes to supercomputing and enable optimal usage of present and future HPC systems for computational simulations. Gevorg have worked in University Rostock as DAAD Fellow and Institute of Physics of Basel University developing simulation codes for theoretical astrophysics, hydrodynamic and particle physics and studied correlation of superdense hybrid matter to evolution of stars, mergers and supernovae

 

Gravitational waves are the last prediction of general relativity still awaiting a direct experimental verification. Observations of gravitational waves will open a new field – gravitational wave astronomy. First science data from the global network of advanced gravitational wave detectors – LIGO, GE0600 and Virgo long arm interferometers, are expected in July 2015. The advanced detector network will be sensitive to signals all over the sky, although source positions can be determined by triangulation. For these reasons, searching for sources in noisy data is algorithmically challenging, since one has to simultaneously look for different types of signals, and computationally formidable, due to the large parameter space over which the searches must be carried out.

To perform a rapid analysis of all data from the advanced LIGO and Virgo gravitational wave detectors’ network, hundreds of millions of CPU hours will be required — the code utilizing the potential of massively parallel supercomputers is therefore mandatory.

Polgraw-Virgo group in cooperation with the Simulation Laboratory for Elementary- and Astro-Particles have developed a highly scalable computation code parallel PollGrawAllSky, which enables the data analysis of all-sky searches for gravitational wave signals at large scales on acceptable time scales. Benchmarking of the code in framework of PRACE Preparatory access on a Cray XE6 system was performed to show efficiency of our parallelization concept and to demonstrate scaling up to 50 thousand cores in parallel. To estimate the computational requirements when current version of code is used for analysis, we have performed representative tests with the Gaussian noise data at different band frequencies. For example, a serial search for GWs in one particular detection day at only 4 frequencies 600, 1000, 1700 and 2000 will require a total of 20 thousand CPU hours of computation, which is more than two years on a single CPU and correspondingly the output generated by this simulation would be ca. 4 GB.

To face the big challenge of the analysis of all the data that will be collected from the advanced detectors expected to be available by the year 2018, we are developing a hybrid parallelized PollGrawAllSky code able to scaling much above current 50000+ cores. To enhance the scalability of execution of many computations in parallel, we combine many instances consisting of different PolGrawAllSky executions that use different numbers of parallel sub-tasks. This feature is implemented using the dynamic process creation and grouping framework of MPI, with different MPI sub-worlds also known as virtual groups that enables collective and dynamic communication operations across a subset of related tasks. The main PolGrawAllSky code with parallel sky loop is encapsulated into another code, named skyfarmer, equipped with internal scheduling and bookkeeping mechanism.

With further implementation of usage of coprocessors (hardware accelerators) like Graphical Processing Units in parallel code presently optimised to use only standard Central Processing Units we hope to reach scalability level allowing to analyse at least four times more resources, i.e., 1000 million CPU hours, to perform analysis of final data in 2018. These means performance of 1petaFLOPS computer working continuously for one year.


Towards a quantitative understanding of the quark–gluon plasma


PDF

Jon-Ivar Skullerud is a lecturer in Mathematical Physics at Maynooth University, Ireland. He received his first degree in physics and philosophy at the University of Trondheim, Norway, and a PhD in theoretical physics from the University of Edinburgh. His main research interest is in non-perturbative studies of the strong interaction, in particular lattice QCD simulations at high temperature and density. He is also a national representative on the International Particle Physics Outreach Group.

 

At extremely high temperatures (100,000 times those in the core of the Sun), the strong interaction, which holds quarks and gluons together to form protons, neutrons and other hadrons, undergoes a dramatic change of character. The quarks and gluons are no longer bound together, but instead form a new phase of matter called the quark-gluon plasma. At the same time, the quarks that make up ordinary matter become effectively massless as the chiral symmetry of the quarks, which is broken in ordinary matter, is restored. This state of matter existed in the first second after the Big Bang and is currently being produced in collisions between heavy ions (gold or lead) at CERN and Brookhaven.

The FASTSUM collaboration has been carrying out large-scale Monte Carlo simulations of strongly interacting matter at temperatures both above and below the transition to the quark-gluon plasma. These simulations have employed anisotropic lattices, where the lattice spacing in the temporal direction is much smaller than in the spatial directions. This allows a good resolution for temporal correlators, which is crucial to obtaining results for transport properties and survival of bound states in the plasma.

We will show results obtained for the electrical conductivity and charge diffusion as well as for states consisting of heavy (charm and beauty) quarks: the melting temperatures of the latter may be used as a “thermometer” of the quark-gluon plasma. We also show results for nucleons demonstrating the effects of chiral symmetry restoration. These results have been obtained from our “second generation” data ensembles generated with the use of PRACE resources. We are currently in the process of generating “third generation” ensembles which will double our temporal resolution and provide the first step towards a continuum extrapolation of our results.


Edge-elements for geophysical electromagnetic problems: A new implementation challenge.


PDF

Octavio Castillo Reyes has his barchelor’s in Computer Systems engineering from Xalapa Institute of Technology, Mexico and M.Sc. In Networks and Telecommunications from Atenas Veracruzana University, Mexico. He has previously worked as lecturer at the University of Veracruz, particularly in the Master in Telematic Engineering and Bachelor in Administrative Computer Systems.

His scientific interests range in the broad fields of computational methods, finite element method, multiprocessor architectures, memory systems, performance and workload characterization.

Octavio Castillo Reyes is currently associate PhD student at Barcelona Supercomputing Center under the supervision of PhD. José María Cela Espín. He is working closely with REPSOL-BSC Research Center and his research focus is the Edge-based Finite Element Method and it’s coupling with geophysical electromagnetic problems in oil industry. Octavio Castillo is supported by a studentship from Mexican National Council for Science and Technology (CONACYT).

 

Electromagnetic Methods (EM) are an established tool in geophysics, finding application in many areas such as hydrocarbon and mineral exploration, reservoir monitoring, CO2 storage characterization, geothermal reservoir imaging and many others. The last decade has been a period of rapid growth of marine electromagnetics, mostly because of its industrial adoption.

The marine controlled-source electromagnetic (CSEM) method has become an important technique for reducing ambiguities in data interpretation in the offshore environment and a commonplace in the industry. In the traditional configuration, the sub-seafloor structure is explored by emitting low-frequency signals from a high-powered electric dipole source towed close to the seafloor. By studying the received signal, the subsurface structures could be detected at scales of a few tens of meters to depths of several kilometers.

On the other hand, in the Finite Element Method for solving electromagnetic field problems, the use of Edge-based elements (Nédélec elements) has become very popular. In fact, Nédélec elements are often said to be a cure to many difficulties that are encountered (particularly eliminating spurious solutions) and are claimed to yield accurate results. However, the state of the art is marked by a relative scarcity in practice of robust codes to simulate geophysical electromagnetic problems. It’s may be attributed to their theoretical and implementational threshold. Indeed, more care and effort are required to implement them: basis functions, Piola mapping, edge directions and numbering strategy. Latter issues poses additional challenges.

Furthermore, the resultant data volumes of large-scale 3D modeling and simulations can easily overwhelm single core and modest multi core architectures. As a result, this kind of problems requires massively parallel computational resources in order to achieve a time frame acceptable to exploration process.

Based on previous ideas and considering the societal value of exploration geophysics, since this process is essential to among others, we present a novelty parallel implementation of Nédélec Elements for geophysical electromagnetic problems on unstructured meshes in 2D and 3D. The usage of unstructured meshes and mesh refinement make it possible to represent complex geological structures precisely and to improve the solution’s accuracy.

In particular, we present a simple, flexible and parallel implementation for Edge Elements in anisotropic mediums. The described software stack relies on a flexible solution which allows a general point of view. The efficiency and accuracy of the code is evaluated through a convergence test, scalability test, assembly time, and solver time, with a strong emphasis on the performance when the number of elements and degrees of freedom grows.

Since our target application is exploration geophysics, the results of this research stage shapes the future line of work to solve more complex problems such as forward modeling simulations and domain and functional decomposition.


GPU accelerated finite element method for radio frequency ablated cancer treatment – Winner of the PRACEdays15 Best Poster Award


PDF

Name: Panchatcharam Mariappan
Qualification: Ph. D in Mathematics, 2013, IIT Madras, Chennai, India
PhD Thesis Title: GPU accelerated finite point set method for fluid flow problems
PhD thesis is a collaborative work of IIT Madras, India and TU Kaiserslautern, Germany
Research Experience: Post doctoral Fellow, 2013-2014, Fraunhofer ITWM, Kaiserslautern, Germany
Professional Appointments:
Software Development Engineer, 2014-present, NUMA Engineering Services Ltd, Dundlk, Co. Louth, Ireland
Lecturer, 2006-2007 Vysya College, Salem, Tamilnadu, India
Online Tutor, 2007, TWWI, Chennai, India
Publications:
1. Panchatcharam M and Sundar S., Finite Pointset method for 2D-dambreak problem with GPU acceleration, International Journal of Applied Mathematics, 25 (2012) 4:545
2. Panchatcharam M., Sundar S., Vetrivel V., Klar A and Tiwari S., GPU computing for meshfree particle method, Accepted, International Journal of Numerical Analysis and Modeling, Series B, 2013.
3. Panchatcharam M., Sundar S and Klar A., GPU metrics for linear solver, Neural, Parallel and Scientific Computations, 21 (2013) 361-374.
Area of Interest: GPU Computing, Mesh Free Methods, CFD, Numerical Analysis, Numerical Linear Algebra
Awards and Honours:
(1) DAAD Fellow, 2010-2013,
(2) GATE Scholarship, 2007-2010,
(3) NBHM Fellow, 2004-2006

 

Graphics Processing Units (GPUs) are nowadays used for numerical computation , beyond their original purpose of graphics accelerators. Mature hardware and GPU software tools and libraries support double precision and memory correction. GPU accelerated computational fluid dynamics has gained attention in both academia and industry. In this article, we investigate the importance of GPUs as accelerators in the field of biomedical engineering. We developed a software tool to predict the lesion development in cancer patients after the radio frequency ablation cancer treatment. We use Penne’s bioheat model with appropriate boundary conditions and the finite element method for numerical discretization. From the finite element discretization of the bioheat equation, we observe that no explicit element integration is required. Since the problem domain is fixed, we find the neighbours of each node at the first time step and generate a compressed sparse row structured (CSR) matrix which can be used for the entire domain. After the CSR matrix is generated, we send the domain information such as nodes, elements and matrix information (e.g. the CSR matrix rows and columns) to the GPU. The Central Processing Unit (CPU) loads the initial data, finds the neighbours list, generates the CSR matrix and stores the results on the disk, whereas the GPU constructs the shape functions, assembles the local stiffness matrix into the global matrix in the CSR form and solves the sparse linear system with the help of the existing CUDA libraries such as CUBLAS and CUSPARSE. In order to solve the linear system, we employed the ILU preconditioned BiCGStab algorithm, one of the fastest solvers among Krylov subspace solvers. At each time step, the GPU generates the heat source term and solves the cell death model, while the CPU saves the results in vtu/vtp files. The heat source term generation is based on our in-house point source model for approximating the Joule heating effect, and the cell death model is an adapted evolution equation, predicting whether cells near the tumour are alive or dead. The tasks assigned to the GPU are the most time consuming parts of the finite element method and the GPU accelerates them with the desired speed-up and accuracy. The major steps involved in this work are receiving the segmented CT scans of the patient from the doctors, generating the mesh, obtaining the needle position from the CT scans (approximately the centre of the tumour) and simulating them using our software tool. Existing software tools working on multi-core CPUs (Intel i5’s) take 6 hours to predict a lesion for 26 minutes of real treatment time, for around 1 million elements. Our current work with the assistance of the GPU acceleration yields the result in approximately 3 minutes for the same number of elements, where the comparison is done with Intel Xeon CPU E5 – 2680 @ 2.8 GHz, and NVIDIA GeForce Titan Black GPU @ 3.5 GHz (2880 CUDA Cores).


Parallel agent-based simulation of South Korean population dynamics


PDF

Cristina Montañola Sales is a research assistant and Ph.D. student at inLab FIB (Barcelona informatics school laboratory), in the Universitat Politècnica de Catalunya (UPC) – BarcelonaTech. She is currently doing her research in collaboration with Barcelona Supercomputing Center (BSC). She holds an MSc in Computer Science from UPC. Her research interests include agent-based modeling, computer simulation, High-Performance Computing and computational social science.

Professor Josep Casanovas is full professor in Operations Research, specializing in Simulation systems, and coordinator of the Severo Ochoa Research Excellence Program in the Barcelona Supercomputing Center (BSC). He is one of the founders of the Barcelona School of Informatics (FIB). He is also the director of inLab FIB (Barcelona informatics school laboratory), an institution that has been very active in technology transfer to business. One of his recent projects has been the cooperation in the creation of simulation environments for people and vehicle flow in the new airport of Barcelona. He has led several EU funded projects in the area of simulation and operations research and is a strong advocate of the knowledge and technology transfer function between the university and society.

Dr. Chang-Won Ahn is a principal researcher at Software Research Lab., Electronics and Telecommunications Research Institute (ETRI). He is educated on Industrial Engineering, especially stochastic processes and queueing theory 1998 at KAIST (Korea Advanced Institute of Science and Technology) in Daejeon, South Korea and has more than 17 years of experiences in system SW technologies. Since 2013, he has been leading the ABC-D (Agent-Based Computational Demography) project by the demand of Ministry of Health and Welfare, which is to develop a full-scale and flexible agent-based simulation model for Korean population dynamics. Now he is trying to build up the social simulation society in Korea as well as an international network and co-work to solve the global challengeable problems for the better society.

 

Changes in our society have created a challenge for policymakers, who confront a need of tools to evaluate the possible effects of their policies. Agent-based simulation is a promising methodology that can be used in the study of population dynamics. However, it has been little used in demographic research to help explaining dynamics. Simulation methodologies provide the opportunity to develop a virtual laboratory for exploring and validating current and new approaches. The purpose is to avoid conducting real social experiments, which may be expensive, unethical or even infeasible.

Agent-based simulation is commonly used for small scenarios because the number of agents and interactions between them can be extremely large in some of case studies, thus forcing the scientist to limit its number in order to execute the simulation in a standard computer. However, in the case of policy models, both the amount of compute power required and detailed micro-level data are significant. To deal with complex social models we can take advantage of parallel computation. Traditionally, parallel simulation has been applied in numerous scientific simulations such as networks or military. Nevertheless, the number of applications in the social sciences is scarce. One of the obstacles hindering the use of agent-based simulation is its scalability, especially if the analysis requires large-scale models. Currently there is no consensus on how to compute agent-based simulations in High Performance systems. Scalability issues cannot be solved just by distributing the computer workload on High Performance architecture. It depends on many factors, notably the execution platform and the complexity of the agent-based model, which in turn depends on the number of agents, their intelligent behavior and the complexity of their communication. It is not only important to address size problems but also to see whether more realistic agent-based models with complex behavior and communication network can be scaled-up to provide empirically and practically useful results.

A possible solution for scalability issues is to run the agent-based models on top of a scalable parallel discrete-event simulation engine. In our work, we use this solution in the design and development of a simulation framework that gives support for modeling and simulating agent-based demographic systems. It provides the placeholders for different demographic processes such as fertility, mortality, change in economic status, change in marital status, and migration. The main advantage of this approach is the ability to run large agent-based scenarios in High Performance Computing environments, when other current tools present limitations.

Moreover, we present a case study on forecasting demographics of South Korea during 100 years. South Korea is a country that shows the most unprecedented speed of aging in history. According to the latest projections, by 2050 South Korea may be the oldest country on earth. This situation could bring difficult challenges to face for the Korean government. Unless the country takes adequate measures to prepare for the demographic aging trend, it is expected that Korea will face a slower economic growth and living standards stagnation. With the application of agent-based simulation to this case we show how the life course of individuals is evolving, allowing deepen on the movements, interactions, and behaviours of South Korean population. Our model is able to capture individual characteristics and to overcome some data-related limitations with assumptions on behavioural rules. With this real case scenario, we show the potential of parallel agent-based simulation methodology for demographics.


Massively-parallel molecular simulation studies of ice and clathrate-hydrate nano-crystal and pre-cursor formation


PDF

Niall English obtained a First Class Honours degree in Chemical Engineering from UCD in 2000. In 2003, Dr English completed a Ph.D. at UCD (Dept. of Chemical Engineering) on molecular simulation of electromagnetic (e/m)-field effects on methane-hydrate crystallisation. During 2004-2005, Niall carried out further simulation and theoretical studies on hydrate dissolution and on the effect of e/m fields on water and metal oxides at the National Energy Technology Laboratory, a U.S. DOE research facility in Pittsburgh, in conjunction with Dept of Chemical Engineering at the University of Pittsburgh. During 2005 to 2007, he worked for the Chemical Computing Group in Cambridge, GB. Here, Niall developed codes, protocols and methods for biomolecular simulation, chiefly for structure-based drug design applications in the pharmaceutical industry. Niall commenced his position as a lecturer in Chemical Engineering at UCD in 2007, being promoted to senior lecturer in 2014. His research interests encompass clathrate hydrates, solar and renewable energies, and simulation of e/m field effects.

 

Ice growth and decomposition was studied upon approximately spherical ice nano-particles of varying size surrounded by liquid water and at a variety of temperatures and pressures. The system box size was also varied for systems containing of the order of one million water molecules to almost ten million molecules, in order to establish system-size effects upon the growth and dissociation kinetics. It was found that there was a dependence upon system size on growth and dissociation, which points out the limitations of previous earlier simulation attempts in smaller simulation boxes.

Further, supercooled liquid water simulations were performed at various system sizes of the order to one to ten million water molecules, and the subtle re-arrangement of the structure and local density was explored as the system began to transition towards local ice-like conditions. Crucially, a system-size dependence was found upon these structural and dynamical rearrangements, which has been neglected in previous simulations.
The heterogeneous nucleation of ice nano-scale particles and the homogeneous nucleation of methane clathrate hydrates at water-methane interfaces were studied, again addressing the key question of the effect of system-size upon the results. It was found that both phenomena did depend on system-size, and that the subtle interplay between the frequency of box fluctuations and dilations with the underlying molecular rearrangements towards free-energy basins was quite important on influencing the outcome.

In the future, I would hope to continue these studies of clathrate and ice nucleation, growth and dissociation, especially with a view towards engineering applications, like the use of inhibitor compounds and temperature-/pressure-pulse strategies to regulate kinetics. Large-scale supercomputing is required to study these complex non-equilibrium processes without being plagued by the tyranny of small systems and periodic boundary conditions affecting results adversely. I expect that benefits to society will emerge from the greater understanding of these phenomena on a microscopic level, and the greater possibilities of devising kinetics-regulation strategies, e.g., to avoid pipeline blockage by hydrate plugs by inexpensive initial screening on supercomputing platforms, using molecular dynamics as an initial ‘predictive’ design tool.


Use of Graphics Cards (GPU) to Simulate Atoms, Molecules and Nucleus


PDF

Born in Hannover, Germany, on October 6, 1972.
Degree in Physics from the University of Seville (Spain) in 1995.
Ph.D degree (Physics) from the University of Seville (Spain) in 2003.
From 1996 to 2001: predoc grant at the Applied Physics Department of the University of Seville.
2001: Technician at the Laboratorio de Investigación en Baja Radiactividad (LIBRA) , University of Valladolid (Spain).
From 2001: Associate Professor at the Physics Department of the University of Córdoba (Spain).
Current research interests include the use of GPU in modelling and simulation of non-linear systems and Monte Carlo methods in Atomic, Molecular and Nuclear Physics.

 

The realistic description of the physical properties of the microscopic systems with a finite number of particles, such as nuclei or atoms and molecules, isolated or confined inside of molecular complexes, is a basic goal in Physics. These studies, even in the most basic aspects, lead to the use of complex methods that require powerful techniques of calculation being the Quantum Monte Carlo (QMC), one of the most developed in the last years. However, it is well known that QMC methods are computationally expensive.

Recently, a programming approach for performing scientific calculations on a graphics processing units (GPUs) has been developed. GPUs have evolved into a highly efficient data-parallel computing device and first market companies have released programming tools to make use of these technologies.

The implementation of QMC codes in GPUs will result in a better knowlegde of the microscopic systems such as nuclei or atoms and molecules.
A comparison between a GPU implementation of some QMC methods, a serial and a parallel code on CPU is presented for some systems.


Developing a scalable and flexible high-resolution code for direct numerical simulation of two-phase flows


PDF

Lennon Ó Náraigh is a lecturer in Applied and Computational Mathematics in the School of Mathematical Sciences in UCD. He received his PhD in Applied Mathematics from Imperial College London, where he also worked as a research associated in the Department of Chemical Engineering, studying multiphase flow for the oil-and-gas industries. His current research interests are in hydrodynamic stability and high-end computing for multiphase incompressible flows.

 

Description: TPLS (Two-Phase Level Set) is an open-source program for simulation of two- phase flows in 3D channel geometries using high resolution DNS. TPLS solves the incompressible Navier—Stokes equations for a two-phase flow. A regular grid finite-volume discretization is employed based on an idealized channel geometry with a range of different inlet conditions that can be prescribed by the user. The interface between phases is tracked with a Level-Set method. The code evolves the physical variables (pressure, fluid velocities, and interface configuration) through discrete time steps. At each timestep, the key computational tasks performed amount to the solution of large systems of sparse linear equations with tens of millions of unknowns, for the key physical variables. In addition, regular I/O is required to save the system state for later analysis and visualizsation, or to restart in the case of hardware failure.

The code is implemented in Fortran90, initially with MPI parallelization using a 2D domain decomposition and bespoke Jacobi/SOR iterative solvers. Over the last two years, we have improved the TPLS code in several respects to give better performance, scalability and usability, moving from an in-house code specialized for use by the original developers, to an open-source flexible program which can easily be used by others, including academic and industrial users. The culmination of this work is TPLS version 2.0 (presented herein), where we have re-implemented the two most computationally-expensive solvers – the pressure and momentum steps – with calls to the PETSc library. Initial tests using the GMRES with Block-Jacobi preconditioner showed a speedup of 80% in the pressure solver on 2048 cores, along with improved strong scaling behavior. The original gather-to-master I/O strategy which wrote text files has been replaced with the use of NetCDF. As a result, we have obtained an order-of-magnitude reduction in I/O time, a compression factor of 6.7 and removed the memory bottleneck of requiring rank 0 to gather the entire domain. In addition to the Level Set method, we have added a PETSc implementation of the Diffuse Interface method (DIM), which is available as an option to users. Finally, with the support of the Software Sustainability Institute, we have added the ability to configure the code through input files or command-line arguments, obviating the need for users to modify and recompile for every application.

Novelty and Originality of Project: TPLS is unique in several aspects. Unlike other solvers (e.g. those mentioned below), TPLS solver has been purpose-built for supercomputing architectures like ARCHER. Most open-source solvers like OpenFOAM, Gerris, Fluidity and commercial solvers like ANSYS-Fluent/CFX offer only one interface-capturing method (the volume-of-fluid method) thereby limiting the applicability of these solvers to either free-surface, stratified, or wavy-stratified flows. The TPLS solver offers the users a choice of two types of interface capturing mechanisms between Diffuse-Interface Method and the Level-Set method. This enables the solver to accurately simulate a wide variety of physical scenarios. A further key feature of the work is the interdisciplinary composition of the development team, including researchers in HPC applications development, applied mathematics, algorithms design, and the physics of fluid flows.

The need for HPC in the work: Due to the high computational cost of a typical three-dimensional simulation, parallelization is essential, and scaling of the methodology described above has been demonstrated to several thousand CPU cores

Looking forward: TPLS has already been used to gain key fundamental insight into interfacial waves in two-phase flows, as well as in the hydrodynamics of evaporating droplets. Imminent future work will see TPLS applied to simulations of a much wider range of physical phenomena, with a view to gaining fundamental understanding of stratified-slug flow transitions, interfacial turbulence, contact-line motion, phase change, and heat transfer. We hope to enlarge the code’s user base, not only among academics, but also with industrial partners, including but not limited to the oil-and-gas industries.


Intelligent Water Drops Algorithm with Perturbation Operators for Atomic Cluster Optimization


PDF

RITCHIE MAE GAMOT is a PhD student at the Centre for Scientific Computing, University of Warwick. She received her Bachelor’s degree in Applied Mathematics at the University of the Philippines Mindanao and Master’s degree in Computer Science at the University of the Philippines Diliman. Currently, her research focuses on utilizing a nature-inspired algorithm, called Intelligent Water Drops algorithm coupled with appropriate local search algorithms, to solve configurational optimization problems with specific focus on atomic clusters.

 

We present a modified version of the Intelligent Water Drops algorithm (MIWD) that has been adapted to allow it to be applied, for the first time, to global optimization of atomic clusters. Cluster perturbation operators were applied to further generate lower energies. The algorithm, dubbed as MIWD+PerturbOp, is an unbiased type of algorithm where no a priori cluster geometry information and construction were used during initialization which is not the case with other common search methods.

Four modifications were implemented: (a) The probability used to determine components for each agent in the population has factored in the pairwise potential energy; (b) The heuristic undesirability factor was based on an objective function used in a multi-start strategy study of LJ clusters by Locatelli, et al [2]; (c) The total worst agent in each iteration has also been identified, aside from total best agent, and paths belonging to it updated. (d) L-BFGS [3] was utilized to further relax clusters to its nearby local minimum.

Due to the iterative nature of the algorithm and the numerous combinations of parameters involved, HPC architecture was valuable in gathering results efficiently. Test runs reveal that a spherical bounding volume for the initial atom positions and grow-etch perturbation operator is a good combination for implementing final runs. Results achieved high success rates for sizes up to N = 104. This study outperformed the seeded runs of Basin Hopping with Occasional Jumping [4] in terms of success rates for more problematic clusters namely, LJ38, LJ75 – 77, LJ98, LJ101, and LJ103-104.

A detailed property analysis of the clusters of up to 104 atoms against the results in Cambridge Cluster Database (CCD) will be discussed. Preliminary experiments also show the method’s promise to binary LJ mixtures and Morse clusters which could be treated as a good indication of the method’s applicability to ionic, nanoalloy clusters or nanoparticles in general. Initial experiments on small Janus clusters using a potential model with a modified orientation term suited for two-patch Janus particles show promising configurations.


TemPortable task-based programming for Seismic Imaging


PDF

Lionel Boillot is now a expert engineer in HPC at Inria Magique3D team project. He received a PhD in Applied Mathematics and Computer Science from “Université de Pau et des Pays de l’Adour” in 2014. The PhD thesis took place in the framework of a collaboration between the Inria research institute and the Total oil company. He has a previous experience of 2 years as HPC engineer and he is graduated from the SupGalilée engineering school and the University Paris 6 master class, both in the fields of applied mathematics and computer science. His main research interest falls in with HPC for CSE.

 

Seismic imaging is of high interest for oil or gas detection. The most accurate techniques used by oil companies are the RTM (Reverse Time Migration) and the FWI (Full Wave Inversion). These methods are based on seismic wave simulations, ideally in anisotropic elastic media, a generally accepted realistic modeling of the subsurface. The parallelization of these methods is a laborious task. The main difficulty comes from the heterogeneity, at several levels. First, the mesh is generally unstructured and the mesh cells are non-homogeneous in terms of number of degrees of freedom. Second, the anisotropy and the boundary conditions leads to unbalance computational zones. These two points lead to heterogeneous zones with different number of computations in each zone. Finally, the hardware heterogeneity prevents to obtain balanced subdomains in a domain decomposition context because even knowing the exact number of computations does not imply to know the exact time they require to execute, due to the vectorization capabilities, the different memory caches,… In addition, the dependencies between the different computational subdomains create complex data movement patterns.

Current algorithms in shared memory use OpenMP to exploit many-cores architectures, with positive outcomes. Integration with CUDA (or OpenCL) allows for access to the computational power of accelerators. Going outside the node boundary, existing algorithms are using a message passing library to exchange data between nodes, enabling distributed memory versions. At this point, mixing these different approaches in order to achieve high performance across heterogeneous platforms remains a complex, error-prone and time-consuming task. The upcoming heterogeneous manycore revolution motivated by the race toward Exascale will only emphasize this problem. Other programming paradigms, especially task-based approaches, seem to be a suitable approach for such levels of hardware complexity. Task-based programming has been successfully applied for to many computational domains leading to robust and efficient solvers: in dense linear algebra (e.g. the DPLASMA library); in sparse direct and iterative methods and fast algorithms (e.g. FMM or H-Matrix problems). However, this programming paradigm has yet to be applied to large industrial parallel codes and demonstrated that these codes may be turned into portable and efficient task-based programs.

Our study focuses on the integration of the 3D elastodynamics Total code DIVA (Depth Imaging Velocity Analysis) with the PaRSEC runtime system, a specialized runtime for task scheduling, developed at University of Tennessee. The code was initially based on the MPI library for the parallelism. The principle of task programming relies on rewriting the code flow as task executions, each task being a computational function. The description of the tasks with the data they need in input and output forms a DAG (Direct Acyclic Graph) of tasks. The arrows of the DAG contain the data movement information that the runtime uses to determine the data flow exhibiting the parallelism.

We first addressed shared memory architectures with a ccNUMA (cache coherent Non Uniform Memory Access) node and an Intel Xeon Phi accelerator, based on the Intel MIC (Many Integrated Cores) architecture. The preliminary results are very promising since we obtain a positive speed up in comparison with the MPI-based code. The most interesting results concerns the parallel efficiency which decreased with the MPI-based code and which is stable and very close to one with the task-based code. In addition, the performance is portable on these two architectures. These results encouraged us to continue our work and move across the node boundaries into the distributed memory architectures and especially clusters of hybrid multicore nodes.


Dynamics of basic hydrolysis of methyl formate – effect of microhydration


PDF

Ivan Cernusak is professor in Theoretical Chemistry at Comenius University in Bratislava, Slovakia. Currently he is working on atmospheric chemistry problems associated with iodo-carbons and their reactivity in troposphere including the aspects of micro-hydration. He is involved also in theoretical calculations of thermo-kinetic parameters not easily amenable to experiment, e.g. in the primary circuit of a nuclear reactor, spectroscopic and electric properties of exotic diatomics and clusters.

 

Base catalyzed hydrolysis of methyl formate including one, two and three water molecules has been investigated using the ab initio molecular dynamics (MD) with the CP2K code. In MD calculations, we applied exchange functional by Perdew, Burke and Ernzerhof (PBE) with additional empirical dispersion correction (D3 method) by Grimme et al. For all atoms, basis sets of double-zeta quality with polarization in combination with the pseudopotentials proposed by Goedecker, Teter and Hutter (GTH) were used. The MD simulations were conducted at constant volume and temperature 298 and 350 K (NVT ensemble) maintained by CSVR thermostat with a time constant of 0.05 ps in the box of size 24 Å (open boundary conditions).

The study revealed that the important part of this micro-hydration assisted mechanism – the OH- attack on carbon in COH-group – strongly depends on the hydrogen-bonded network in the initial cluster. Several trajectories with different initial geometries of the hydrated CH3 O COH…OH- cluster and total length between 20-30 ps were analyzed. Only a fraction of these trajectories lead to traditional mechanism via BAC2 intermediate, while many end-up in stable but non-reactive hydrated ion coordinated to methyl formate.

Reaction profiles, geometry analysis and detailed participation of the solvent molecules (including animation of important trajectories) in selected cases are discussed.


Gender Inequality in HPC


PDF

I am currently in my second year of my PhD research studies in the School of Physics and Astronomy. My research involves the underrepresentation of women in the High Performance Computing (HPC) field and the identification of the reasons and possible solutions. My scientific background and my previous involvement to the Women in Science (WISE) and the Women in HPC (WHPC) communities provide a better understanding of the female scientists’ matters and support my research.

Qualifications:
PhD – EPCC, School of Physics and Astronomy, University of Edinburgh (2013-Present)
MSc – School of Geoscience, Aristotle University of Thessaloniki, GR (2006-2009)
BSc – School of Geoscience, Aristotle University of Thessaloniki, GR (2000-2005)

Key Skills:
Computer literate: knowledge of various operating systems including Windows XP, working knowledge of Microsoft Office, Open Office, Origin, Latex document preparation, and various image editing and crystal structure visualization and analysis software.
Languages: Greek (native speaker), English (native speaker equivalent), Spanish (advanced knowledge), French (basic knowledge), Japanese (elementary knowledge).
Teaching experience: teaching undergraduate students physics and astrobiology, I have also tutored students in Higher-level biology and chemistry, and have taught dancing and English as a foreign language to all ages and abilities.

Other Achievements:
In the spring of 2013 I was awarded with a Principal’s Career Development Scholarship for my PhD studies (2013-2017) among other competent candidates.
I am a member and fascilitator of Women in HPC Network by providing administrative support. Also, I was a member of the organising committee for the University of Edinburgh Women in Science and Engineering Workshop and member of the organising committee for the Women in High Performance Computing Network launch (April 2014).
I was a member of the organising committee of the 10th International Conference of the Geological Society of Greece (Thessaloniki, April 2004) and of the 2nd Conference of the Committee of Economic Geology, Mineralogy, Geochemistry of the Geological Society of Greece (Thessaloniki, October 2005).
I am a member of the Equality and Diversity Committee of the School of Physics and Astronomy of the University of Edinburgh since the spring of 2014 and I am involved to the missions about Athena Swan and Juno Projects.
I am an EPCC’s outreach member and a STEM Ambassador and I get involved in various events in schools and institutes supporting and launching Science.
I am an active athlete, committee member and beginners’ representative (2014-2015) of the University of Edinburgh Kendo Club, participating in competitions and other events.

 

Gender inequality is a key problem across all scientific disciplines, both in academia and industry. HPC is a discipline that spans multiple traditional science subjects and relies on leading-edge scientific research. It would be plausible that the gender inequality issue, which has been identified and quantified across many fields of science and scientific research, is similarly present in HPC. To motivate action, we must measure and publicise the magnitude of the problem we face. To be effective, we must understand why women do not pursue careers in HPC so that our efforts can be appropriately targeted.

In 2014 the Software Sustainability Institute [1] produced a study on software development in Russell Group Universities [2]. This poster presents evidence of gender inequity in software development from further analysis of this study providing an initial insight into the group of people who use and develop HPC software. Our analysis shows that male researchers are more likely to develop software than female researchers (70% to 30%) and that men are more likely to receive training than women (63% to 39%). This has a profound effect on the people that developed their own software: 56% of the male respondents to the study had received training and developed their own software, whereas only 23% of the female respondents were trained and were developing software. Particularly important, as software developers in academia and HPC users are more likely to work with Linux or Mac. 66.2% of female respondents use Windows, with 33.8% using Mac, Linux or other operating system (Mac 25.7% , Linux 6.6 %). For the male respondents 42.2% use Windows, with the proportion of the male respondents using Mac (27.4%) or Linux (29.6%) doubling compared to men at 57.9%. The higher prevalence of Linux use in men and of Windows in women may be either the cause of the result of the lack of uptake in training by women. In this study we have also identified that women who responded to the survey are more likely to have less mature careers than men. Our analysis shows that there is not a considerable difference between the respondents that have less than 10 years of experience (66.9% of female and 57% of male). However, it is of great interest that 21.7% of the male respondents and only 9.7% of female have more than 20 years of experience.
This poster will present further details of this work and the potential impact of the findings on the HPC community.

[1] Software Sustainability Institute, http://www.software.ac.uk/
[2] Software Sustainability Institute (SSI) Russell Group study on software development, http://www.software.ac.uk/blog/2014-12-04-its-impossible-conduct-research-without-software-say-7-out-10-uk-researchers


The numerical Earth Magnetosphere


PDF

Giovanni Lapenta was born in Italy in 1965 and has now become a Belgian citizen. Lapenta has a PhD in plasma physics (1993) and a master in Nuclear Engineering (1990). Lapenta’s career includes:

– Professor of Applied Mathematics, KU Leuven, Belgium, since January 2007. Consultant for University of California Los Angeles (UCLA) and University of Colorado in Boulder.
– Elected Official in the Plasma Astrophysics Group of the American Physical Society.
– Recipient of the RD100 prize in 2005 for his role in the development of the HPC software CartaBlanca and Parsek (a predecessor of the application software iPic3D to be used here).
– Scientist at Los Alamos National Laboratory (LANL) from 1992 to 2007. Covering over time in sequence the roles of Director’s Postdoc, Scientist and Director of Research in the theory division and in the computer science division.
– Tenured Research Professor of Computational Physics at Politecnico di Torino, 1996-2000. Position held in concurrency with that at LANL.
– Visiting Scientist at the Massachusetts Institute of Technology in 1992.

Lapenta has been PI of many projects at Los Alamos and at KU Leuven, including Coordinator of 3 EC-FP7 projects: Soteria, eHeroes and Swiff. He is co-investigator on the NASA Heliospheric challenge (with UCLA) and the theory support of MMS (with University of Colorado)

 

Coupling microphysics and macrophysics is the grandest of the grand challenges in almost all areas of science and engineering. Large scale system-wide effects can be triggered by the smallest scales in the system. An especially convenient field where the micro-macro coupling can be explored is the Earth space environment and its interaction with the incoming solar wind and cosmic rays. In this case we can access directly and measure both the system-wide scales as well as the smallest scales of interest. There are several space missions covering large regions of space surrounding the Earth (about 100 Earth radii, RE, in radius) capable of measuring the evolution of this environment. But from March 12, 2015 (the launch of the four spacecraft will be in about 12 hours at the moment of writing) we will have the Magnetospheric MultiScale (MMS) mission to probe the smallest scale of interest: the electron response scale that is about 100m in size. Needless to say simulating a sphere of 100 RE (or about 600,000 km) with a resolution of 100 meters is a grand challenge. A grand challenge that we at KU Leuven are attempting to solve using the best computing resources available via PRACE Tier-0 allocations.

In our recent work, we have consumed approximately 30 million CPU hours to achieve substantial advances that cover all major steps in the series of key processes developing in the Earth space environment. Like a bullet in the air, the Earth’s space cocoon (called magnetosphere) cuts through the solar wind flow, creating a shock wave that compresses and heats up the solar plasma. We are performing simulations of this interaction, never done before, successfully capturing the physics of the entire planetary environment in a domain of 160 x 120 x 120 planet radii, and capturing the detailed physics of individual electrons. Each global simulation requires 750 000 cpu hours, and is used to detect the variations of the particle velocity distributions and pressure anisotropies across the shock, and the effects of the local small scale particle physics on the large scale dynamics.

Within this domain we zoom in on the most important regions of energy release where magnetic reconnection develops. This process allows magnetic field to dissipate enormous energies that are suddenly released in bursts, a phenomenon that has defied explanation for decades. We have performed highly resolved simulations of the most important regions of interest in 3D using up to 48,000 cores and showing several new processes not identified before: the presence of switch-off shocks and rotational discontinuities, multiple interconnected sites each forming interacting islands of magnetic field and magnetic flux ropes embedding points of zero magnetic field (called magnetic nulls). Results that are published in the most prestigious journals in the field.

Every new Tier-0 simulation allows us to get closer to the real dynamics and scales of the plasma environment of our planet, which help us to better understand the impact of the Sun on our life and our technology. Our goal is to perform in the near future real scale, fully self-consistent, simulations of the full 3D environment of the Earth: the numerical magnetosphere.

Work supported by the European Commission via the DEEP project (deep-project.eu)