Title Page

Contents

Preface

Course group shot

LHC computing (WLCG): Past, present, and future

Introduction

The past-the LCG project and the WLCG

Grid Services

AAA and Security

Computing

Storage

Information System

The WLCG

Policies

The present-experiences in LHC Run 1

Performance during LHC Run 1

First evolution of the computing models

Lessons learned

The future-the next 10 years

Run 2

Software

Longer-term outlook

Conclusion

Scientific Clouds

Introduction

Cloud computing: Definition and technology recap

The problem of getting predictions right

How to define Cloud Computing

Who does what?

The Cloud Hype

Virtualization or Cloud Computing?

Pros and cons

Pros

Cost reduction

Flexibility and scalability

Democratization of resources

Business opportunities

Cons

Non-exclusive rights

Unavailability

No guarantees

Responsibility

Data privacy

Maturity

The big misunderstanding

Private vs. Public Clouds

More pros or cons then?

Apps in the Cloud

A typical application architecture

Is my application cloud-friendly?

Which adaptation?

Is anything missing?

Federated identities

Local scheduling

Networking and storage strategies

Dynamically extending layer 2 networks

Using VMs in existing data centers

Virtualization penalties

Distributed authorization

PaaS architectures

Portals

Conclusions

Clouds in biosciences: A journey to high-throughput computing in life sciences

Introduction

Short introduction to molecular biology

Short introduction to structural biology

A short introduction to drug discovery

Introduction to medical imaging and neurosciences

Summary

Grid usage in life sciences

Historical introduction

A pioneering application: WISDOM

Grid usage on the plateau of maturity

Structural biology on the grid

Virtual Imaging Platform

Pilot agent platforms

Conclusions

Clouds in life sciences

Deployment of life science applications on public clouds

De novo deployment of scientific applications on academic clouds

Migration to academic clouds using pilot agent platforms

Conclusions

Grids and clouds for e-Health

Introduction

GINSENG

Entering a new world

Introduction

Big data

Conclusions

Monitoring and control of large-scale distributed systems

Introduction

The system design

Registration and discovery

The MonALISA service

Monitoring modules and information gathering

Security infrastructure

Data storage

Wavelets data compression

Monitoring clients and optimization services

Fast data transfer

Network monitoring and management

The Network Monitoring the USLHCnet

Network topology

Monitoring ALICE distributed computing environment

Services monitoring

Jobs and resource monitoring

Analyzing multiple time series data

Automated management

Data transfer services

Abstracted network topology

Network graph model

Path allocation and data transfer scheduling

Time-based path scheduling

Dynamic bandwidth allocation

Priority-based dynamic bandwidth allocation for data transfers

Bandwidth measurements for the ALICE Grid

On demand, end-to-end optical circuits

Optimizing the global connectivity for the EVO videoconferencing system

The monitoring and controlling architecture

Connectivity monitoring

The optimization of the overlay topology

Implementation challenges

Network-related challenges

Scalability and concurrency aspects

Automatic updates publishing mechanism

Summary

Big data: Challenges and perspectives

Introduction

Storage managements and data mangement

Storage media and organisation

Magnetic disk

Storage tuning versus data spreading

Power and cooling

Aggregation and redundancy

Basic approaches: mirroring and striping

RAID and erasure encoding

RAID scalability issues

The EOS storage system at CERN

Flash memory

Hybrid disks

Magnetic tape

An example archive: the CASTOR system

Data organisation

Databases

C++ persistency with ROOT

Data management in grids and clouds

The CAP theorem

Cloud storage

RAIN storage

Multi-site clustering and geo-localisation

Storage market evolution and technology trends

Advanced networking for scientific applications

Introduction

Application-network interactions

Dynamic Circuit Networks

Inter-domain aspects

Networks and exchange points

OGF network services interface

Use cases

eVLBI

ANSE

Software defined networking

Separation of control and data plane

OpenFlow

Programmable networks

Use case example: multipath

Information centric networking

Named data networking

Summary

Networking for high energy physics

Introduction: Networks for HEP and the LHC program

The SCIC and Digital Divide issues

Key Research & Education Network developments in 2014-2015

Networks for the LHC program

LHC-Driven network traffic growth and impact

Network requirements for LHC Run2 and beyond

Evolution of Research and Education Networks

Intercontinental networks

Campus networking evolution

State of the art in high-throughput wide area networking

Global network trends and implications for ICFA and the SCIC

Software Defined Networking

The SCIC Network Monitoring Group: the PingER Project

Conclusions and recommendations. Requests to ICFA

Conclusions

Recommendations and requests

Updates from Research and Education Networks by region

Europe

The Americas

Asia Pacific

Africa

UbuntuNet Member NRENs

UbuntuNet NRENs in Development

Interregional networks and organizations

Network monitoring report for 2014

Glossary

Towards an OpenStack-based Swiss national research infrastructure

Introduction and overview

Cloud definitions

Ecosystem architecture

The Swiss Academic Compute Cloud Project

Infrastructure for SwissACC

Cloud charging models for federated infrastructures

Summary and outlook

List of participants