Loading…
Virtual Event
November 17–November 20, 2020
Learn More and Register to Attend This Event

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon North America 2020 - Virtual to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Eastern Standard Time (UTC–05:00). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change.
Back To Schedule
Thursday, November 19 • 4:50pm - 5:25pm
Speeding Up Analysis Pipelines with Remote Container Images - Ricardo Rocha & Spyridon Trigazis, CERN

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.


Containers have taken a key role in the daily life of physicists at CERN, helping with packaging and sharing code as well as ensuring analysis reproducibility. This session will describe how processes have been adapted to containerize software releases of tens of gigabytes, and how they're used to process hundreds of petabytes of new data every year. In particular, it will focus on how container images are distributed in a large network of connected sites across the world and show how lazy loading of container images using the containerd remote snapshotter has ensured a flat startup time under 6 seconds while dramatically reducing network traffic. A live demo will include a real physics analysis pipeline of hundreds of parallel jobs using the setup described above.
Speakers
avatar for Ricardo Rocha

Ricardo Rocha

Computing Engineer, CERN
Ricardo is a Computing Engineer at CERN IT focusing on containerized deployments, networking and more recently machine learning platforms. He has led for several years the internal effort to transition services and workloads to use cloud native technologies, as well as dissemination... Read More →
ST

Spyros Trigazis

Computing Engineer, CERN
Spyros Trigazis is a computing engineer and a member of the CERN Cloud infrastructure team which provides computing resources to the High Energy Physics community. He has been contributing to open source projects like Fedora, Kubernetes and OpenStack.

Speeding Up Analysis Pipelines with Remote Container Images pdf
Text Transcript YNVQ 4188 txt
Thursday November 19, 2020 4:50pm - 5:25pm EST
Intrado Virtual Event Platform
  Performance